Can you confirm that VT-d is enabled properly by running the following?

 

dmesg | grep -e DMAR -e IOMMU

 

From: Himanshu Goyal <himanshugoyal500@gmail.com>
Date: Thursday, January 24, 2019 at 8:19 AM
To: "Peters, Matt" <Matt.Peters@windriver.com>
Cc: "Hu, Yong" <yong.hu@intel.com>, "starlingx-discuss@lists.starlingx.io" <starlingx-discuss@lists.starlingx.io>
Subject: Re: [Starlingx-discuss] Deployment Option (error: compute boot in loop)

 

Hi Peters,

 

Error logs ovs-vswitched.log:

#######################################

2019-01-24T18:42:04.213Z|00830|dpdk|INFO|EAL: PCI device 0000:02:00.0 on NUMA socket 0

2019-01-24T18:42:04.213Z|00831|dpdk|INFO|EAL:   probe driver: 8086:10fb net_ixgbe

2019-01-24T18:42:04.215Z|00832|dpdk|ERR|EAL:   0000:02:00.0 failed to select IOMMU type

2019-01-24T18:42:04.215Z|00833|dpdk|ERR|EAL: Driver cannot attach the device (0000:02:00.0)

2019-01-24T18:42:04.215Z|00834|netdev_dpdk|WARN|Error attaching device '0000:02:00.0' to DPDK

2019-01-24T18:42:04.215Z|00835|netdev|WARN|eth0: could not set configuration (Invalid argument)

########################################

 

Please find attached log file also.

 

Regards,

Himanshu Goyal

 

On Thu, Jan 24, 2019 at 6:32 PM Peters, Matt <Matt.Peters@windriver.com> wrote:

The name of eth0 is correct.  It is just an assigned name by configuration management.

 

What err logs are present in /var/log/openvswitch/ovs-vswitchd.log?

 

From: Himanshu Goyal <himanshugoyal500@gmail.com>
Date: Thursday, January 24, 2019 at 6:52 AM
To: "Hu, Yong" <yong.hu@intel.com>
Cc: "starlingx-discuss@lists.starlingx.io" <starlingx-discuss@lists.starlingx.io>
Subject: Re: [Starlingx-discuss] Deployment Option (error: compute boot in loop)

 

Thanks a lot Yong,

 

Checked OVS/DPDK status, It is giving me the error as output shown below:

In br-phy0 is attaching a port name as eth0, but my compute port name is ens513f0, i think that may be the issue. 

 

################################################################################# 

ovs-vsctl show output:

compute-0:/usr/share/openvswitch/scripts$ sudo ovs-vsctl show

543d08f4-ff1e-4a8d-8e48-11f01356750d

    Manager "ptcp:6640:127.0.0.1"

        is_connected: true

    Bridge "br-phy0"

        Controller "tcp:127.0.0.1:6633"

            is_connected: true

        fail_mode: secure

        Port "phy-br-phy0"

            Interface "phy-br-phy0"

                type: patch

                options: {peer="int-br-phy0"}

        Port "br-phy0"

            Interface "br-phy0"

                type: internal

        Port "lldpabeb30a6-6c"

            Interface "lldpabeb30a6-6c"

                type: internal

        Port "eth0"

            Interface "eth0"

                type: dpdk

                options: {dpdk-devargs="0000:02:00.0", n_rxq="2"}

                error: "Error attaching device '0000:02:00.0' to DPDK"

    Bridge br-int

        Controller "tcp:127.0.0.1:6633"

            is_connected: true

        fail_mode: secure

        Port "int-br-phy0"

            Interface "int-br-phy0"

                type: patch

                options: {peer="phy-br-phy0"}

        Port br-int

            Interface br-int

                type: internal

    ovs_version: "2.9.0"

#####################################################################################################

 

##############################################################################

Output of dpdk-devbind.py:

 

compute-0:/usr/share/openvswitch/scripts$ python /usr/share/openvswitch/scripts/dpdk-devbind.py --status

 

Network devices using DPDK-compatible driver

============================================

0000:02:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' drv=vfio-pci unused=

 

Network devices using kernel driver

===================================

0000:02:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' if=ens513f1 drv=ixgbe unused=vfio-pci *Active*

0000:04:00.0 'I350 Gigabit Network Connection 1521' if=enp4s0f0 drv=igb unused=vfio-pci

0000:04:00.3 'I350 Gigabit Network Connection 1521' if=enp4s0f3 drv=igb unused=vfio-pci

 

Other Network devices

=====================

<none>

 

Crypto devices using DPDK-compatible driver

===========================================

<none>

 

Crypto devices using kernel driver

==================================

<none>

 

Other Crypto devices

====================

<none>

 

Eventdev devices using DPDK-compatible driver

=============================================

<none>

 

Eventdev devices using kernel driver

====================================

<none>

 

Other Eventdev devices

======================

<none>

 

Mempool devices using DPDK-compatible driver

============================================

<none>

 

Mempool devices using kernel driver

===================================

<none>

 

Other Mempool devices

=====================

<none>

compute-0:/usr/share/openvswitch/scripts$

 #############################################################

 

Both management and OAM ports support DPDK.

 

lspci output:

##################################################

compute-0:/usr/share/openvswitch/scripts$ lspci -nn | grep Eth

02:00.0 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01)

02:00.1 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01)

04:00.0 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)

04:00.3 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)

###############################################################

compute-0:/usr/share/openvswitch/scripts$

 

 

Many Thanks,

Himanshu Goyal

 

 

On Thu, Jan 24, 2019 at 2:24 PM Hu, Yong <yong.hu@intel.com> wrote:

I saw you enabled VT-D, how about VT-x in BIOS?

 

Another possibility is that your ethernet NIC is not compatible with OVS/DPDK.

Please follow these steps, to dig out more info

  1. lock your compute-node
  2. ssh to your compute-node and check OVS/DPDK status by “sudo ovs-vsctl show” and “python /usr/share/openvswitch/scripts/dpdk-devbind.py --status”

 

From: Himanshu Goyal <himanshugoyal500@gmail.com>
Date: Thursday, 24 January 2019 at 4:46 PM
To: "Hu, Yong" <yong.hu@intel.com>
Cc: "Alonso, Juan Carlos" <juan.carlos.alonso@intel.com>, "starlingx-discuss@lists.starlingx.io" <starlingx-discuss@lists.starlingx.io>
Subject: Re: [Starlingx-discuss] Deployment Option (error: compute boot in loop)

Hi Yong,

 

I checked server BIOS setting Intel VT-D is enabled in my compute machine

compute machine is rebooting only after unlocking, Before reboot it came in unlocked, enabled & online.

 

Please suggest how can i debug this.

 

Many Thanks,

Himanshu Goyal

 

On Wed, Jan 23, 2019 at 9:40 PM Hu, Yong <yong.hu@intel.com> wrote:

Intel_iommu=on and immmu are for PCI PT and SR-IOV, as far as I know.

There are other 2, like VT-x and VT-d flags, to be enabled. Please have a look in BIOS.