Based on the logs, it looks like you have an older system that doesn’t fully support VT-d IOMMU remapping. You can try updating your BIOS if there is a newer version available. We don’t currently support setting arbitrary module parameters, so we don’t have a way to implement the workaround module param specified in the logs (i.e. disabling interrupt remapping). Maybe there are others on the distribution list that could offer additional suggestions. From: Himanshu Goyal <himanshugoyal500@gmail.com> Date: Thursday, January 24, 2019 at 11:32 AM To: "Peters, Matt" <Matt.Peters@windriver.com> Cc: "Hu, Yong" <yong.hu@intel.com>, "starlingx-discuss@lists.starlingx.io" <starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] Deployment Option (error: compute boot in loop) Hi Peters, Please find output of required command in attached file. Regards, Himanshu Goyal Regards, Himanshu Goyal On Thu, Jan 24, 2019 at 6:56 PM Peters, Matt <Matt.Peters@windriver.com<mailto:Matt.Peters@windriver.com>> wrote: Can you confirm that VT-d is enabled properly by running the following? dmesg | grep -i -e DMAR -e IOMMU From: Himanshu Goyal <himanshugoyal500@gmail.com<mailto:himanshugoyal500@gmail.com>> Date: Thursday, January 24, 2019 at 8:19 AM To: "Peters, Matt" <Matt.Peters@windriver.com<mailto:Matt.Peters@windriver.com>> Cc: "Hu, Yong" <yong.hu@intel.com<mailto:yong.hu@intel.com>>, "starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>" <starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>> Subject: Re: [Starlingx-discuss] Deployment Option (error: compute boot in loop) Hi Peters, Error logs ovs-vswitched.log: ####################################### 2019-01-24T18:42:04.213Z|00830|dpdk|INFO|EAL: PCI device 0000:02:00.0 on NUMA socket 0 2019-01-24T18:42:04.213Z|00831|dpdk|INFO|EAL: probe driver: 8086:10fb net_ixgbe 2019-01-24T18:42:04.215Z|00832|dpdk|ERR|EAL: 0000:02:00.0 failed to select IOMMU type 2019-01-24T18:42:04.215Z|00833|dpdk|ERR|EAL: Driver cannot attach the device (0000:02:00.0) 2019-01-24T18:42:04.215Z|00834|netdev_dpdk|WARN|Error attaching device '0000:02:00.0' to DPDK 2019-01-24T18:42:04.215Z|00835|netdev|WARN|eth0: could not set configuration (Invalid argument) ######################################## Please find attached log file also. Regards, Himanshu Goyal On Thu, Jan 24, 2019 at 6:32 PM Peters, Matt <Matt.Peters@windriver.com<mailto:Matt.Peters@windriver.com>> wrote: The name of eth0 is correct. It is just an assigned name by configuration management. What err logs are present in /var/log/openvswitch/ovs-vswitchd.log? From: Himanshu Goyal <himanshugoyal500@gmail.com<mailto:himanshugoyal500@gmail.com>> Date: Thursday, January 24, 2019 at 6:52 AM To: "Hu, Yong" <yong.hu@intel.com<mailto:yong.hu@intel.com>> Cc: "starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>" <starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>> Subject: Re: [Starlingx-discuss] Deployment Option (error: compute boot in loop) Thanks a lot Yong, Checked OVS/DPDK status, It is giving me the error as output shown below: In br-phy0 is attaching a port name as eth0, but my compute port name is ens513f0, i think that may be the issue. ################################################################################# ovs-vsctl show output: compute-0:/usr/share/openvswitch/scripts$ sudo ovs-vsctl show 543d08f4-ff1e-4a8d-8e48-11f01356750d Manager "ptcp:6640:127.0.0.1" is_connected: true Bridge "br-phy0" Controller "tcp:127.0.0.1:6633<http://127.0.0.1:6633>" is_connected: true fail_mode: secure Port "phy-br-phy0" Interface "phy-br-phy0" type: patch options: {peer="int-br-phy0"} Port "br-phy0" Interface "br-phy0" type: internal Port "lldpabeb30a6-6c" Interface "lldpabeb30a6-6c" type: internal Port "eth0" Interface "eth0" type: dpdk options: {dpdk-devargs="0000:02:00.0", n_rxq="2"} error: "Error attaching device '0000:02:00.0' to DPDK" Bridge br-int Controller "tcp:127.0.0.1:6633<http://127.0.0.1:6633>" is_connected: true fail_mode: secure Port "int-br-phy0" Interface "int-br-phy0" type: patch options: {peer="phy-br-phy0"} Port br-int Interface br-int type: internal ovs_version: "2.9.0" ##################################################################################################### ############################################################################## Output of dpdk-devbind.py: compute-0:/usr/share/openvswitch/scripts$ python /usr/share/openvswitch/scripts/dpdk-devbind.py --status Network devices using DPDK-compatible driver ============================================ 0000:02:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' drv=vfio-pci unused= Network devices using kernel driver =================================== 0000:02:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' if=ens513f1 drv=ixgbe unused=vfio-pci *Active* 0000:04:00.0 'I350 Gigabit Network Connection 1521' if=enp4s0f0 drv=igb unused=vfio-pci 0000:04:00.3 'I350 Gigabit Network Connection 1521' if=enp4s0f3 drv=igb unused=vfio-pci Other Network devices ===================== <none> Crypto devices using DPDK-compatible driver =========================================== <none> Crypto devices using kernel driver ================================== <none> Other Crypto devices ==================== <none> Eventdev devices using DPDK-compatible driver ============================================= <none> Eventdev devices using kernel driver ==================================== <none> Other Eventdev devices ====================== <none> Mempool devices using DPDK-compatible driver ============================================ <none> Mempool devices using kernel driver =================================== <none> Other Mempool devices ===================== <none> compute-0:/usr/share/openvswitch/scripts$ ############################################################# Both management and OAM ports support DPDK. lspci output: ################################################## compute-0:/usr/share/openvswitch/scripts$ lspci -nn | grep Eth 02:00.0 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01) 02:00.1 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01) 04:00.0 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01) 04:00.3 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01) ############################################################### compute-0:/usr/share/openvswitch/scripts$ Many Thanks, Himanshu Goyal On Thu, Jan 24, 2019 at 2:24 PM Hu, Yong <yong.hu@intel.com<mailto:yong.hu@intel.com>> wrote: I saw you enabled VT-D, how about VT-x in BIOS? Another possibility is that your ethernet NIC is not compatible with OVS/DPDK. Please follow these steps, to dig out more info 1. lock your compute-node 2. ssh to your compute-node and check OVS/DPDK status by “sudo ovs-vsctl show” and “python /usr/share/openvswitch/scripts/dpdk-devbind.py --status” From: Himanshu Goyal <himanshugoyal500@gmail.com<mailto:himanshugoyal500@gmail.com>> Date: Thursday, 24 January 2019 at 4:46 PM To: "Hu, Yong" <yong.hu@intel.com<mailto:yong.hu@intel.com>> Cc: "Alonso, Juan Carlos" <juan.carlos.alonso@intel.com<mailto:juan.carlos.alonso@intel.com>>, "starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>" <starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>> Subject: Re: [Starlingx-discuss] Deployment Option (error: compute boot in loop) Hi Yong, I checked server BIOS setting Intel VT-D is enabled in my compute machine compute machine is rebooting only after unlocking, Before reboot it came in unlocked, enabled & online. Please suggest how can i debug this. Many Thanks, Himanshu Goyal On Wed, Jan 23, 2019 at 9:40 PM Hu, Yong <yong.hu@intel.com<mailto:yong.hu@intel.com>> wrote: Intel_iommu=on and immmu are for PCI PT and SR-IOV, as far as I know. There are other 2, like VT-x and VT-d flags, to be enabled. Please have a look in BIOS.