[Starlingx-discuss] Deployment Option (error: compute boot in loop)

Himanshu Goyal himanshugoyal500 at gmail.com
Fri Jan 25 08:55:16 UTC 2019


Hi Matt,

Please find below output of dpdk-devbind.py --status also:

compute-0:/var/log/puppet/latest# python
/usr/share/openvswitch/scripts/dpdk-devbind.py --status

##########################################################################################################
python /usr/share/openvswitch/scripts/dpdk-devbind.py --status
Network devices using DPDK-compatible driver
============================================
0000:04:00.0 'I350 Gigabit Network Connection 1521' drv=vfio-pci unused=

Network devices using kernel driver
===================================
0000:02:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb'
if=ens513f1 drv=ixgbe unused=vfio-pci
0000:04:00.3 'I350 Gigabit Network Connection 1521' if=enp4s0f3 drv=igb
unused=vfio-pci *Active*

Other Network devices
=====================
0000:02:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb'
unused=vfio-pci

Crypto devices using DPDK-compatible driver
===========================================
<none>

Crypto devices using kernel driver
==================================
<none>

Other Crypto devices
====================
<none>

Eventdev devices using DPDK-compatible driver
=============================================
<none>

Eventdev devices using kernel driver
====================================
<none>

Other Eventdev devices
======================
<none>

Mempool devices using DPDK-compatible driver
============================================
<none>

Mempool devices using kernel driver
===================================
<none>

Other Mempool devices
=====================
<none>
compute-0:/var/log/puppet/latest#
################################################################################

Regards,
Himanshu Goyal

On Fri, Jan 25, 2019 at 2:14 PM Himanshu Goyal <himanshugoyal500 at gmail.com>
wrote:

> Hi Matt,
>
> Sorry, interrupt remapping was disabled in my BIOS, I tried again after
> enable that, Now in dmesg & ovs-vsct there is no errors.* But compute
> still rebooting in loop*.
> Please find attached dmesg log output for reference.
>
> #######################################################################
> compute-0:/home/wrsroot#* sudo ovs-vsctl show*
> 8a1f7179-2014-4b4e-ad23-2065b8024964
>     Manager "ptcp:6640:127.0.0.1"
>         is_connected: true
>     Bridge "br-phy0"
>         Controller "tcp:127.0.0.1:6633"
>             is_connected: true
>         fail_mode: secure
>         Port "lldp64416a6d-7d"
>             Interface "lldp64416a6d-7d"
>                 type: internal
>         Port "phy-br-phy0"
>             Interface "phy-br-phy0"
>                 type: patch
>                 options: {peer="int-br-phy0"}
>         Port "br-phy0"
>             Interface "br-phy0"
>                 type: internal
>         Port "eth0"
>             Interface "eth0"
>                 type: dpdk
>                 options: {dpdk-devargs="0000:04:00.0", n_rxq="2"}
>     Bridge br-int
>         Controller "tcp:127.0.0.1:6633"
>             is_connected: true
>         fail_mode: secure
>         Port br-int
>             Interface br-int
>                 type: internal
>         Port "int-br-phy0"
>             Interface "int-br-phy0"
>                 type: patch
>                 options: {peer="phy-br-phy0"}
>     ovs_version: "2.9.0"
> compute-0:/home/wrsroot#
>
> #############################################################################
>
> Please suggest me any other logs that i can check.
>
> Regards,
> Himanshu Goyal
>
> On Fri, Jan 25, 2019 at 12:21 PM Himanshu Goyal <
> himanshugoyal500 at gmail.com> wrote:
>
>> Thanks Matt,
>>
>> Below are the details of my server BIOS, Currently in my Server  Bios
>> Version: SE5C610.86B.01.01.0014.121820151719,
>> Please suggest which version of Bios we can use & this version is
>> up-gradable or not with required version.
>>
>> #################################################################
>> compute-0:/home/wrsroot# dmidecode -t1
>> # dmidecode 3.0
>> Getting SMBIOS data from sysfs.
>> SMBIOS 2.7 present.
>>
>> Handle 0x0001, DMI type 1, 27 bytes
>> System Information
>>         Manufacturer: Intel Corporation
>>         Product Name: S2600WT2
>>         Version: ....................
>>         Serial Number: ............
>>         UUID: 803BBEFB-BFC1-E511-906E-0012795D96DD
>>         Wake-up Type: Power Switch
>>         SKU Number: SKU Number
>>         Family: Family
>>
>> #############################################################################
>>
>>
>> ################################################################################
>> compute-0:/home/wrsroot# sudo dmidecode --type bios
>> # dmidecode 3.0
>> Getting SMBIOS data from sysfs.
>> SMBIOS 2.7 present.
>>
>> Handle 0x0000, DMI type 0, 24 bytes
>> BIOS Information
>>         Vendor: Intel Corporation
>>         Version: SE5C610.86B.01.01.0014.121820151719
>>         Release Date: 12/18/2015
>>         Address: 0xF0000
>>         Runtime Size: 64 kB
>>         ROM Size: 16384 kB
>>         Characteristics:
>>                 PCI is supported
>>                 PNP is supported
>>                 BIOS is upgradeable
>>                 BIOS shadowing is allowed
>>                 Boot from CD is supported
>>                 Selectable boot is supported
>>                 EDD is supported
>>                 5.25"/1.2 MB floppy services are supported (int 13h)
>>                 3.5"/720 kB floppy services are supported (int 13h)
>>                 3.5"/2.88 MB floppy services are supported (int 13h)
>>                 Print screen service is supported (int 5h)
>>                 8042 keyboard services are supported (int 9h)
>>                 Serial services are supported (int 14h)
>>                 Printer services are supported (int 17h)
>>                 CGA/mono video services are supported (int 10h)
>>                 ACPI is supported
>>                 USB legacy is supported
>>                 LS-120 boot is supported
>>                 ATAPI Zip drive boot is supported
>>                 BIOS boot specification is supported
>>                 Function key-initiated network boot is supported
>>                 Targeted content distribution is supported
>>                 UEFI is supported
>>         BIOS Revision: 0.0
>>         Firmware Revision: 0.0
>>
>> Handle 0x000B, DMI type 13, 22 bytes
>> BIOS Language Information
>>         Language Description Format: Abbreviated
>>         Installable Languages: 1
>>                 enUS
>>         Currently Installed Language: enUS
>>
>> ########################################################################################################
>>
>> Regards,
>> Himanshu Goyal
>>
>> On Thu, Jan 24, 2019 at 10:55 PM Peters, Matt <Matt.Peters at windriver.com>
>> wrote:
>>
>>> Based on the logs, it looks like you have an older system that doesn’t
>>> fully support VT-d IOMMU remapping.  You can try updating your BIOS if
>>> there is a newer version available.  We don’t currently support setting
>>> arbitrary module parameters, so we don’t have a way to implement the
>>> workaround module param specified in the logs (i.e. disabling interrupt
>>> remapping).
>>>
>>>
>>>
>>> Maybe there are others on the distribution list that could offer
>>> additional suggestions.
>>>
>>>
>>>
>>> *From: *Himanshu Goyal <himanshugoyal500 at gmail.com>
>>> *Date: *Thursday, January 24, 2019 at 11:32 AM
>>> *To: *"Peters, Matt" <Matt.Peters at windriver.com>
>>> *Cc: *"Hu, Yong" <yong.hu at intel.com>, "
>>> starlingx-discuss at lists.starlingx.io" <
>>> starlingx-discuss at lists.starlingx.io>
>>> *Subject: *Re: [Starlingx-discuss] Deployment Option (error: compute
>>> boot in loop)
>>>
>>>
>>>
>>> Hi Peters,
>>>
>>>
>>>
>>> Please find output of required command in attached file.
>>>
>>>
>>>
>>> Regards,
>>>
>>> Himanshu Goyal
>>>
>>>
>>>
>>> Regards,
>>>
>>> Himanshu Goyal
>>>
>>> On Thu, Jan 24, 2019 at 6:56 PM Peters, Matt <Matt.Peters at windriver.com>
>>> wrote:
>>>
>>> Can you confirm that VT-d is enabled properly by running the following?
>>>
>>>
>>>
>>> dmesg | grep -i -e DMAR -e IOMMU
>>>
>>>
>>>
>>> *From: *Himanshu Goyal <himanshugoyal500 at gmail.com>
>>> *Date: *Thursday, January 24, 2019 at 8:19 AM
>>> *To: *"Peters, Matt" <Matt.Peters at windriver.com>
>>> *Cc: *"Hu, Yong" <yong.hu at intel.com>, "
>>> starlingx-discuss at lists.starlingx.io" <
>>> starlingx-discuss at lists.starlingx.io>
>>> *Subject: *Re: [Starlingx-discuss] Deployment Option (error: compute
>>> boot in loop)
>>>
>>>
>>>
>>> Hi Peters,
>>>
>>>
>>>
>>> Error logs ovs-vswitched.log:
>>>
>>> #######################################
>>>
>>> 2019-01-24T18:42:04.213Z|00830|dpdk|INFO|EAL: PCI device 0000:02:00.0 on
>>> NUMA socket 0
>>>
>>> 2019-01-24T18:42:04.213Z|00831|dpdk|INFO|EAL:   probe driver: 8086:10fb
>>> net_ixgbe
>>>
>>> 2019-01-24T18:42:04.215Z|00832|dpdk|ERR|EAL:   0000:02:00.0 failed to
>>> select IOMMU type
>>>
>>> 2019-01-24T18:42:04.215Z|00833|dpdk|ERR|EAL: Driver cannot attach the
>>> device (0000:02:00.0)
>>>
>>> 2019-01-24T18:42:04.215Z|00834|netdev_dpdk|WARN|Error attaching device
>>> '0000:02:00.0' to DPDK
>>>
>>> 2019-01-24T18:42:04.215Z|00835|netdev|WARN|eth0: could not set
>>> configuration (Invalid argument)
>>>
>>> ########################################
>>>
>>>
>>>
>>> Please find attached log file also.
>>>
>>>
>>>
>>> Regards,
>>>
>>> Himanshu Goyal
>>>
>>>
>>>
>>> On Thu, Jan 24, 2019 at 6:32 PM Peters, Matt <Matt.Peters at windriver.com>
>>> wrote:
>>>
>>> The name of eth0 is correct.  It is just an assigned name by
>>> configuration management.
>>>
>>>
>>>
>>> What err logs are present in /var/log/openvswitch/ovs-vswitchd.log?
>>>
>>>
>>>
>>> *From: *Himanshu Goyal <himanshugoyal500 at gmail.com>
>>> *Date: *Thursday, January 24, 2019 at 6:52 AM
>>> *To: *"Hu, Yong" <yong.hu at intel.com>
>>> *Cc: *"starlingx-discuss at lists.starlingx.io" <
>>> starlingx-discuss at lists.starlingx.io>
>>> *Subject: *Re: [Starlingx-discuss] Deployment Option (error: compute
>>> boot in loop)
>>>
>>>
>>>
>>> Thanks a lot Yong,
>>>
>>>
>>>
>>> Checked OVS/DPDK status, It is giving me the error as output shown below:
>>>
>>> In br-phy0 is attaching a port name as eth0, but my compute port name
>>> is ens513f0, i think that may be the issue.
>>>
>>>
>>>
>>>
>>> #################################################################################
>>>
>>> *ovs-vsctl show output:*
>>>
>>> compute-0:/usr/share/openvswitch/scripts$ sudo ovs-vsctl show
>>>
>>> 543d08f4-ff1e-4a8d-8e48-11f01356750d
>>>
>>>     Manager "ptcp:6640:127.0.0.1"
>>>
>>>         is_connected: true
>>>
>>>     Bridge "br-phy0"
>>>
>>>         Controller "tcp:127.0.0.1:6633"
>>>
>>>             is_connected: true
>>>
>>>         fail_mode: secure
>>>
>>>         Port "phy-br-phy0"
>>>
>>>             Interface "phy-br-phy0"
>>>
>>>                 type: patch
>>>
>>>                 options: {peer="int-br-phy0"}
>>>
>>>         Port "br-phy0"
>>>
>>>             Interface "br-phy0"
>>>
>>>                 type: internal
>>>
>>>         Port "lldpabeb30a6-6c"
>>>
>>>             Interface "lldpabeb30a6-6c"
>>>
>>>                 type: internal
>>>
>>>         Port "eth0"
>>>
>>>           *  Interface "eth0"*
>>>
>>>                 type: dpdk
>>>
>>>                 options: {dpdk-devargs="0000:02:00.0", n_rxq="2"}
>>>
>>>                 *error: "Error attaching device '0000:02:00.0' to DPDK"*
>>>
>>>     Bridge br-int
>>>
>>>         Controller "tcp:127.0.0.1:6633"
>>>
>>>             is_connected: true
>>>
>>>         fail_mode: secure
>>>
>>>         Port "int-br-phy0"
>>>
>>>             Interface "int-br-phy0"
>>>
>>>                 type: patch
>>>
>>>                 options: {peer="phy-br-phy0"}
>>>
>>>         Port br-int
>>>
>>>             Interface br-int
>>>
>>>                 type: internal
>>>
>>>     ovs_version: "2.9.0"
>>>
>>>
>>> #####################################################################################################
>>>
>>>
>>>
>>>
>>> ##############################################################################
>>>
>>> *Output of dpdk-devbind.py:*
>>>
>>>
>>>
>>> compute-0:/usr/share/openvswitch/scripts$ python
>>> /usr/share/openvswitch/scripts/dpdk-devbind.py --status
>>>
>>>
>>>
>>> Network devices using DPDK-compatible driver
>>>
>>> ============================================
>>>
>>> 0000:02:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb'
>>> drv=vfio-pci unused=
>>>
>>>
>>>
>>> Network devices using kernel driver
>>>
>>> ===================================
>>>
>>> 0000:02:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb'
>>> if=ens513f1 drv=ixgbe unused=vfio-pci *Active*
>>>
>>> 0000:04:00.0 'I350 Gigabit Network Connection 1521' if=*enp4s0f0*
>>> drv=igb unused=vfio-pci
>>>
>>> 0000:04:00.3 'I350 Gigabit Network Connection 1521' if=enp4s0f3 drv=igb
>>> unused=vfio-pci
>>>
>>>
>>>
>>> Other Network devices
>>>
>>> =====================
>>>
>>> <none>
>>>
>>>
>>>
>>> Crypto devices using DPDK-compatible driver
>>>
>>> ===========================================
>>>
>>> <none>
>>>
>>>
>>>
>>> Crypto devices using kernel driver
>>>
>>> ==================================
>>>
>>> <none>
>>>
>>>
>>>
>>> Other Crypto devices
>>>
>>> ====================
>>>
>>> <none>
>>>
>>>
>>>
>>> Eventdev devices using DPDK-compatible driver
>>>
>>> =============================================
>>>
>>> <none>
>>>
>>>
>>>
>>> Eventdev devices using kernel driver
>>>
>>> ====================================
>>>
>>> <none>
>>>
>>>
>>>
>>> Other Eventdev devices
>>>
>>> ======================
>>>
>>> <none>
>>>
>>>
>>>
>>> Mempool devices using DPDK-compatible driver
>>>
>>> ============================================
>>>
>>> <none>
>>>
>>>
>>>
>>> Mempool devices using kernel driver
>>>
>>> ===================================
>>>
>>> <none>
>>>
>>>
>>>
>>> Other Mempool devices
>>>
>>> =====================
>>>
>>> <none>
>>>
>>> compute-0:/usr/share/openvswitch/scripts$
>>>
>>>  #############################################################
>>>
>>>
>>>
>>> Both management and OAM ports support DPDK.
>>>
>>>
>>>
>>> *lspci output:*
>>>
>>> *##################################################*
>>>
>>> compute-0:/usr/share/openvswitch/scripts$ lspci -nn | grep Eth
>>>
>>> 02:00.0 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit
>>> SFI/SFP+ Network Connection [8086:10fb] (rev 01)
>>>
>>> 02:00.1 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit
>>> SFI/SFP+ Network Connection [8086:10fb] (rev 01)
>>>
>>> 04:00.0 Ethernet controller [0200]: Intel Corporation I350 Gigabit
>>> Network Connection [8086:1521] (rev 01)
>>>
>>> 04:00.3 Ethernet controller [0200]: Intel Corporation I350 Gigabit
>>> Network Connection [8086:1521] (rev 01)
>>>
>>> ###############################################################
>>>
>>> compute-0:/usr/share/openvswitch/scripts$
>>>
>>>
>>>
>>>
>>>
>>> Many Thanks,
>>>
>>> Himanshu Goyal
>>>
>>>
>>>
>>>
>>>
>>> On Thu, Jan 24, 2019 at 2:24 PM Hu, Yong <yong.hu at intel.com> wrote:
>>>
>>> I saw you enabled VT-D, how about VT-x in BIOS?
>>>
>>>
>>>
>>> Another possibility is that your ethernet NIC is not compatible with
>>> OVS/DPDK.
>>>
>>> Please follow these steps, to dig out more info
>>>
>>>    1. lock your compute-node
>>>    2. ssh to your compute-node and check OVS/DPDK status by “sudo
>>>    ovs-vsctl show” and “python /usr/share/openvswitch/scripts/dpdk-devbind.py
>>>    --status”
>>>
>>>
>>>
>>> *From: *Himanshu Goyal <himanshugoyal500 at gmail.com>
>>> *Date: *Thursday, 24 January 2019 at 4:46 PM
>>> *To: *"Hu, Yong" <yong.hu at intel.com>
>>> *Cc: *"Alonso, Juan Carlos" <juan.carlos.alonso at intel.com>, "
>>> starlingx-discuss at lists.starlingx.io" <
>>> starlingx-discuss at lists.starlingx.io>
>>> *Subject: *Re: [Starlingx-discuss] Deployment Option (error: compute
>>> boot in loop)
>>>
>>> Hi Yong,
>>>
>>>
>>>
>>> I checked server BIOS setting Intel VT-D is enabled in my compute machine
>>>
>>> compute machine is rebooting only after unlocking, Before reboot it came
>>> in unlocked, enabled & online.
>>>
>>>
>>>
>>> Please suggest how can i debug this.
>>>
>>>
>>>
>>> Many Thanks,
>>>
>>> Himanshu Goyal
>>>
>>>
>>>
>>> On Wed, Jan 23, 2019 at 9:40 PM Hu, Yong <yong.hu at intel.com> wrote:
>>>
>>> Intel_iommu=on and immmu are for PCI PT and SR-IOV, as far as I know.
>>>
>>> There are other 2, like VT-x and VT-d flags, to be enabled. Please have
>>> a look in BIOS.
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20190125/c15ffd3d/attachment-0001.html>


More information about the Starlingx-discuss mailing list