Based on the logs, it looks like you have an older system that doesn’t fully support VT-d IOMMU remapping. You can try updating your BIOS if there is a newer version available. We don’t currently support setting arbitrary module parameters, so we don’t have a way to implement the workaround module param specified in the logs (i.e. disabling interrupt remapping).
Maybe there are others on the distribution list that could offer additional suggestions.
From: Himanshu Goyal <himanshugoyal500@gmail.com>
Date: Thursday, January 24, 2019 at 11:32 AM
To: "Peters, Matt" <Matt.Peters@windriver.com>
Cc: "Hu, Yong" <yong.hu@intel.com>, "starlingx-discuss@lists.starlingx.io" <starlingx-discuss@lists.starlingx.io>
Subject: Re: [Starlingx-discuss] Deployment Option (error: compute boot in loop)
Hi Peters,
Please find output of required command in attached file.
Regards,
Himanshu Goyal
Regards,
Himanshu Goyal
On Thu, Jan 24, 2019 at 6:56 PM Peters, Matt <Matt.Peters@windriver.com> wrote:
Can you confirm that VT-d is enabled properly by running the following?
dmesg | grep -i -e DMAR -e IOMMU
From: Himanshu Goyal <himanshugoyal500@gmail.com>
Date: Thursday, January 24, 2019 at 8:19 AM
To: "Peters, Matt" <Matt.Peters@windriver.com>
Cc: "Hu, Yong" <yong.hu@intel.com>, "starlingx-discuss@lists.starlingx.io" <starlingx-discuss@lists.starlingx.io>
Subject: Re: [Starlingx-discuss] Deployment Option (error: compute boot in loop)
Hi Peters,
Error logs ovs-vswitched.log:
#######################################
2019-01-24T18:42:04.213Z|00830|dpdk|INFO|EAL: PCI device 0000:02:00.0 on NUMA socket 0
2019-01-24T18:42:04.213Z|00831|dpdk|INFO|EAL: probe driver: 8086:10fb net_ixgbe
2019-01-24T18:42:04.215Z|00832|dpdk|ERR|EAL: 0000:02:00.0 failed to select IOMMU type
2019-01-24T18:42:04.215Z|00833|dpdk|ERR|EAL: Driver cannot attach the device (0000:02:00.0)
2019-01-24T18:42:04.215Z|00834|netdev_dpdk|WARN|Error attaching device '0000:02:00.0' to DPDK
2019-01-24T18:42:04.215Z|00835|netdev|WARN|eth0: could not set configuration (Invalid argument)
########################################
Please find attached log file also.
Regards,
Himanshu Goyal
On Thu, Jan 24, 2019 at 6:32 PM Peters, Matt <Matt.Peters@windriver.com> wrote:
The name of eth0 is correct. It is just an assigned name by configuration management.
What err logs are present in /var/log/openvswitch/ovs-vswitchd.log?
From: Himanshu Goyal <himanshugoyal500@gmail.com>
Date: Thursday, January 24, 2019 at 6:52 AM
To: "Hu, Yong" <yong.hu@intel.com>
Cc: "starlingx-discuss@lists.starlingx.io" <starlingx-discuss@lists.starlingx.io>
Subject: Re: [Starlingx-discuss] Deployment Option (error: compute boot in loop)
Thanks a lot Yong,
Checked OVS/DPDK status, It is giving me the error as output shown below:
In br-phy0 is attaching a port name as eth0, but my compute port name is ens513f0, i think that may be the issue.
#################################################################################
ovs-vsctl show output:
compute-0:/usr/share/openvswitch/scripts$ sudo ovs-vsctl show
543d08f4-ff1e-4a8d-8e48-11f01356750d
Manager "ptcp:6640:127.0.0.1"
is_connected: true
Bridge "br-phy0"
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
Port "phy-br-phy0"
Interface "phy-br-phy0"
type: patch
options: {peer="int-br-phy0"}
Port "br-phy0"
Interface "br-phy0"
type: internal
Port "lldpabeb30a6-6c"
Interface "lldpabeb30a6-6c"
type: internal
Port "eth0"
Interface "eth0"
type: dpdk
options: {dpdk-devargs="0000:02:00.0", n_rxq="2"}
error: "Error attaching device '0000:02:00.0' to DPDK"
Bridge br-int
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
Port "int-br-phy0"
Interface "int-br-phy0"
type: patch
options: {peer="phy-br-phy0"}
Port br-int
Interface br-int
type: internal
ovs_version: "2.9.0"
#####################################################################################################
##############################################################################
Output of dpdk-devbind.py:
compute-0:/usr/share/openvswitch/scripts$ python /usr/share/openvswitch/scripts/dpdk-devbind.py --status
Network devices using DPDK-compatible driver
============================================
0000:02:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' drv=vfio-pci unused=
Network devices using kernel driver
===================================
0000:02:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' if=ens513f1 drv=ixgbe unused=vfio-pci *Active*
0000:04:00.0 'I350 Gigabit Network Connection 1521' if=enp4s0f0 drv=igb unused=vfio-pci
0000:04:00.3 'I350 Gigabit Network Connection 1521' if=enp4s0f3 drv=igb unused=vfio-pci
Other Network devices
=====================
<none>
Crypto devices using DPDK-compatible driver
===========================================
<none>
Crypto devices using kernel driver
==================================
<none>
Other Crypto devices
====================
<none>
Eventdev devices using DPDK-compatible driver
=============================================
<none>
Eventdev devices using kernel driver
====================================
<none>
Other Eventdev devices
======================
<none>
Mempool devices using DPDK-compatible driver
============================================
<none>
Mempool devices using kernel driver
===================================
<none>
Other Mempool devices
=====================
<none>
compute-0:/usr/share/openvswitch/scripts$
#############################################################
Both management and OAM ports support DPDK.
lspci output:
##################################################
compute-0:/usr/share/openvswitch/scripts$ lspci -nn | grep Eth
02:00.0 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01)
02:00.1 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01)
04:00.0 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)
04:00.3 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)
###############################################################
compute-0:/usr/share/openvswitch/scripts$
Many Thanks,
Himanshu Goyal
On Thu, Jan 24, 2019 at 2:24 PM Hu, Yong <yong.hu@intel.com> wrote:
I saw you enabled VT-D, how about VT-x in BIOS?
Another possibility is that your ethernet NIC is not compatible with OVS/DPDK.
Please follow these steps, to dig out more info
- lock your compute-node
- ssh to your compute-node and check OVS/DPDK status by “sudo ovs-vsctl show” and “python /usr/share/openvswitch/scripts/dpdk-devbind.py --status”
From: Himanshu Goyal <himanshugoyal500@gmail.com>
Date: Thursday, 24 January 2019 at 4:46 PM
To: "Hu, Yong" <yong.hu@intel.com>
Cc: "Alonso, Juan Carlos" <juan.carlos.alonso@intel.com>, "starlingx-discuss@lists.starlingx.io" <starlingx-discuss@lists.starlingx.io>
Subject: Re: [Starlingx-discuss] Deployment Option (error: compute boot in loop)
I checked server BIOS setting Intel VT-D is enabled in my compute machine
compute machine is rebooting only after unlocking, Before reboot it came in unlocked, enabled & online.
Please suggest how can i debug this.
Many Thanks,
Himanshu Goyal
On Wed, Jan 23, 2019 at 9:40 PM Hu, Yong <yong.hu@intel.com> wrote:
Intel_iommu=on and immmu are for PCI PT and SR-IOV, as far as I know.
There are other 2, like VT-x and VT-d flags, to be enabled. Please have a look in BIOS.