[Starlingx-discuss] stx 6.0 aio mellenox connectX 40GB dual card

M. Vefa Bicakci vefa.bicakci at windriver.com
Tue Apr 12 14:54:56 UTC 2022


Hi,

Unfortunately, the only suggestion I can make is to use the build instructions at [1] to build a customized StarlingX v6.0 ISO image that includes Mellanox OFED v5.1 (or v4.9), so that you have a way forward with Mellanox ConnectX-3 network adapters.

The following patches should be helpful as references: [2] [3] [4] [5]. However, also note that support for mlx4 was removed from StarlingX's master branch with the development story at [6], so there will be further difficulties in supporting Mellanox ConnectX-3 in future versions of StarlingX.

Vefa

[1] https://docs.starlingx.io/developer_resources/build_guide.html

[2] https://review.opendev.org/c/starlingx/tools/+/831338
    (Download script changes for Mellanox OFED v5.5)

[3] https://review.opendev.org/c/starlingx/kernel/+/831461
    (Uprevision to Mellanox OFED v5.5)

[4] https://review.opendev.org/c/starlingx/tools/+/796386
    (Download script changes for uprevision to v5.10 kernel and Mellanox OFED v5.3)

[5] https://review.opendev.org/c/starlingx/kernel/+/799000
    (Uprevision Mellanox OFED from v5.0 to v5.3)

[6] https://storyboard.openstack.org/#!/story/2009878
    (List of changes for Mellanox ConnectX-6, some of which remove support for mlx4.)


On 2022-04-06 00:40, Embedded Devel wrote:
> welp further tho the stx 5.0.1 vs 6.0 with mellanox drivers, 5.0.1 wount even install on the Dell C6620 where 6.0 installs fine
> also stx 5.0.1 doesnt have kernel 5.x, where 6.0 does
> 
> so now i have to ask, who do i have to pay to get a stx 6.0 with  Mellanox OFED v5.1, and not OFED 5.3, this basically rendered 10K in hardware useless for us.
> 
> as we cant seem to get supported OFED v5.1, we installed 6.x and tried top downgrade... and we cant build it for the kernel on the box from sources no dev environment / compiler
> 
> 
> HELP!
> 
> On Wednesday 06 April 2022 10:49:59 AM (+07:00), Embedded Devel wrote:
> 
>> So if im hearing you correctly, i can try to use stx 5.0.1, which i hope has wireguard support, like 6.0 does, and then i guess it also prevents me from ever upgrading to 6.0
>>
>> On Monday 28 March 2022 21:37:47 PM (+07:00), M. Vefa Bicakci wrote:
>>
>> > Hi,
>> >
>> > StarlingX 5.0.1 release has Mellanox OFED v5.1, and I just checked that the RPM file linked below indeed has the mlx4* kernel modules, so I believe that StarlingX 5.0.1 ought to work with Mellanox ConnectX-3 cards:
>> >
>> > http://mirror.starlingx.cengn.ca/mirror/starlingx/release/5.0.1/centos/distro/outputs/RPMS/std/mlnx-ofa_kernel-modules-5.0-OFED.5.0.2.1.8.1.g5f67178.tis.2.x86_64.rpm
> 
>> >
>> > It could be possible to adapt Mellanox OFED v4.9 to StarlingX 6.0, but I am not aware of anyone who has tried this, so it is difficult for me comment with certainty.
>> >
>> > Thank you,
>> >
>> > Vefa
>> >
>> > On 2022-03-26 22:12, Embedded Devel wrote:
>> > > So if updated in 6.0, will stx 5.0 work with them ? or is it possible to build stx 6.0 customized with 4.9
>> > > On Saturday 26 March 2022 01:51:23 AM (+07:00), M. Vefa Bicakci wrote:
>> > >
>> > >> Hi,
>> > >>
>> > >> With StarlingX 6.0, the Mellanox OFED device drivers were updated to version 5.3. According to the following link, nVidia/Mellanox has stopped supporting ConnectX-3 cards such as the one you have (MCX384A) since driver version v5.1:
>> > >>
>> > >> https://docs.nvidia.com/networking/display/MLNXOFEDv512620/Release+Notes
>> > >>
>> > >> As noted at the aforementioned link, Mellanox OFED v4.9 LTS continues to support ConnectX-3 cards. StarlingX supports only the latest version of the Mellanox OFED drivers. Hence, StarlingX no longer ships with a version of the drivers that support ConnectX-3 cards.
>> > >>
>> > >> Also note that ConnectX-3 cards use the mlx4 driver instead of the mlx5 driver, as indicated here:
>> > >>
>> > >> https://docs.nvidia.com/networking/display/MLNXOFEDv494080/Introduction
>> > >>
>> > >> Thank you,
>> > >>
>> > >> Vefa
>> > >>
>> > >> On 2022-03-25 12:59, Khalil, Ghada wrote:
>> > >> > Hi Linda, can someone from the OS team comment on this?
>> > >> >
>> > >> > -----Original Message-----
>> > >> > From: Embedded Devel <lists at optimcloud.com>
>> > >> > Sent: Thursday, March 17, 2022 9:37 AM
>> > >> > To: 'starlingx-discuss at lists.starlingx.io' <starlingx-discuss at lists.starlingx.io>
>> > >> > Subject: [Starlingx-discuss] stx 6.0 aio mellenox connectX 40GB dual card
>> > >> >
>> > >> > [Please note: This e-mail is from an EXTERNAL e-mail address]
>> > >> >
>> > >> > i just deployed a stx 6.0 aio dual on bare metal yet after booting and logging into the system i dont see the mellanox connectX 40Gb interfaces i also dont see a mlx5_en kernel module to load
>> > >> >
>> > >> > server is a Dell C6220 with Dual-Port 40Gb/s QSFP Mezzanine Network Card MCX384A-BCAA 3CYRK
>> > >> >
>> > >> > --
>> > >> > Sent with Vivaldi Mail. Download Vivaldi for free at vivaldi.com
>> > >> >
>> > >> > _______________________________________________
>> > >> > Starlingx-discuss mailing list
>> > >> > Starlingx-discuss at lists.starlingx.io
>> > >> > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
>> > >> >
>> > >> > _______________________________________________
>> > >> > Starlingx-discuss mailing list
>> > >> > Starlingx-discuss at lists.starlingx.io
>> > >> > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
>> > >>
>> > >>
>> > >> _______________________________________________
>> > >> Starlingx-discuss mailing list
>> > >> Starlingx-discuss at lists.starlingx.io
>> > >> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
>> > >>
>> >
>> >
>> > _______________________________________________
>> > Starlingx-discuss mailing list
>> > Starlingx-discuss at lists.starlingx.io
>> > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
>> >




More information about the Starlingx-discuss mailing list