[Starlingx-discuss] stx 6.0 aio mellenox connectX 40GB dual card
i just deployed a stx 6.0 aio dual on bare metal yet after booting and logging into the system i dont see the mellanox connectX 40Gb interfaces i also dont see a mlx5_en kernel module to load
server is a Dell C6220 with Dual-Port 40Gb/s QSFP Mezzanine Network Card MCX384A-BCAA 3CYRK
Hi Linda, can someone from the OS team comment on this?
-----Original Message----- From: Embedded Devel lists@optimcloud.com Sent: Thursday, March 17, 2022 9:37 AM To: 'starlingx-discuss@lists.starlingx.io' starlingx-discuss@lists.starlingx.io Subject: [Starlingx-discuss] stx 6.0 aio mellenox connectX 40GB dual card
[Please note: This e-mail is from an EXTERNAL e-mail address]
i just deployed a stx 6.0 aio dual on bare metal yet after booting and logging into the system i dont see the mellanox connectX 40Gb interfaces i also dont see a mlx5_en kernel module to load
server is a Dell C6220 with Dual-Port 40Gb/s QSFP Mezzanine Network Card MCX384A-BCAA 3CYRK
-- Sent with Vivaldi Mail. Download Vivaldi for free at vivaldi.com
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Hi,
With StarlingX 6.0, the Mellanox OFED device drivers were updated to version 5.3. According to the following link, nVidia/Mellanox has stopped supporting ConnectX-3 cards such as the one you have (MCX384A) since driver version v5.1:
https://docs.nvidia.com/networking/display/MLNXOFEDv512620/Release+Notes
As noted at the aforementioned link, Mellanox OFED v4.9 LTS continues to support ConnectX-3 cards. StarlingX supports only the latest version of the Mellanox OFED drivers. Hence, StarlingX no longer ships with a version of the drivers that support ConnectX-3 cards.
Also note that ConnectX-3 cards use the mlx4 driver instead of the mlx5 driver, as indicated here:
https://docs.nvidia.com/networking/display/MLNXOFEDv494080/Introduction
Thank you,
Vefa
On 2022-03-25 12:59, Khalil, Ghada wrote:
Hi Linda, can someone from the OS team comment on this?
-----Original Message----- From: Embedded Devel lists@optimcloud.com Sent: Thursday, March 17, 2022 9:37 AM To: 'starlingx-discuss@lists.starlingx.io' starlingx-discuss@lists.starlingx.io Subject: [Starlingx-discuss] stx 6.0 aio mellenox connectX 40GB dual card
[Please note: This e-mail is from an EXTERNAL e-mail address]
i just deployed a stx 6.0 aio dual on bare metal yet after booting and logging into the system i dont see the mellanox connectX 40Gb interfaces i also dont see a mlx5_en kernel module to load
server is a Dell C6220 with Dual-Port 40Gb/s QSFP Mezzanine Network Card MCX384A-BCAA 3CYRK
-- Sent with Vivaldi Mail. Download Vivaldi for free at vivaldi.com
Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
So if updated in 6.0, will stx 5.0 work with them ? or is it possible to build stx 6.0 customized with 4.9
On Saturday 26 March 2022 01:51:23 AM (+07:00), M. Vefa Bicakci wrote:
Hi,
With StarlingX 6.0, the Mellanox OFED device drivers were updated to
version 5.3. According to the following link, nVidia/Mellanox has stopped supporting ConnectX-3 cards such as the one you have (MCX384A) since driver version v5.1:
https://docs.nvidia.com/networking/display/MLNXOFEDv512620/Release+Notes
As noted at the aforementioned link, Mellanox OFED v4.9 LTS continues to
support ConnectX-3 cards. StarlingX supports only the latest version of the Mellanox OFED drivers. Hence, StarlingX no longer ships with a version of the drivers that support ConnectX-3 cards.
Also note that ConnectX-3 cards use the mlx4 driver instead of the mlx5
driver, as indicated here:
https://docs.nvidia.com/networking/display/MLNXOFEDv494080/Introduction
Thank you,
Vefa
On 2022-03-25 12:59, Khalil, Ghada wrote:
Hi Linda, can someone from the OS team comment on this?
-----Original Message----- From: Embedded Devel lists@optimcloud.com Sent: Thursday, March 17, 2022 9:37 AM To: 'starlingx-discuss@lists.starlingx.io'
starlingx-discuss@lists.starlingx.io
Subject: [Starlingx-discuss] stx 6.0 aio mellenox connectX 40GB dual
card
[Please note: This e-mail is from an EXTERNAL e-mail address]
i just deployed a stx 6.0 aio dual on bare metal yet after booting and
logging into the system i dont see the mellanox connectX 40Gb interfaces i also dont see a mlx5_en kernel module to load
server is a Dell C6220 with Dual-Port 40Gb/s QSFP Mezzanine Network
Card MCX384A-BCAA 3CYRK
-- Sent with Vivaldi Mail. Download Vivaldi for free at vivaldi.com
Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Hi,
StarlingX 5.0.1 release has Mellanox OFED v5.1, and I just checked that the RPM file linked below indeed has the mlx4* kernel modules, so I believe that StarlingX 5.0.1 ought to work with Mellanox ConnectX-3 cards:
http://mirror.starlingx.cengn.ca/mirror/starlingx/release/5.0.1/centos/distr...
It could be possible to adapt Mellanox OFED v4.9 to StarlingX 6.0, but I am not aware of anyone who has tried this, so it is difficult for me comment with certainty.
Thank you,
Vefa
On 2022-03-26 22:12, Embedded Devel wrote:
So if updated in 6.0, will stx 5.0 work with them ? or is it possible to build stx 6.0 customized with 4.9 On Saturday 26 March 2022 01:51:23 AM (+07:00), M. Vefa Bicakci wrote:
Hi,
With StarlingX 6.0, the Mellanox OFED device drivers were updated to version 5.3. According to the following link, nVidia/Mellanox has stopped supporting ConnectX-3 cards such as the one you have (MCX384A) since driver version v5.1:
https://docs.nvidia.com/networking/display/MLNXOFEDv512620/Release+Notes
As noted at the aforementioned link, Mellanox OFED v4.9 LTS continues to support ConnectX-3 cards. StarlingX supports only the latest version of the Mellanox OFED drivers. Hence, StarlingX no longer ships with a version of the drivers that support ConnectX-3 cards.
Also note that ConnectX-3 cards use the mlx4 driver instead of the mlx5 driver, as indicated here:
https://docs.nvidia.com/networking/display/MLNXOFEDv494080/Introduction
Thank you,
Vefa
On 2022-03-25 12:59, Khalil, Ghada wrote:
Hi Linda, can someone from the OS team comment on this?
-----Original Message----- From: Embedded Devel lists@optimcloud.com Sent: Thursday, March 17, 2022 9:37 AM To: 'starlingx-discuss@lists.starlingx.io' starlingx-discuss@lists.starlingx.io Subject: [Starlingx-discuss] stx 6.0 aio mellenox connectX 40GB dual card
[Please note: This e-mail is from an EXTERNAL e-mail address]
i just deployed a stx 6.0 aio dual on bare metal yet after booting and logging into the system i dont see the mellanox connectX 40Gb interfaces i also dont see a mlx5_en kernel module to load
server is a Dell C6220 with Dual-Port 40Gb/s QSFP Mezzanine Network Card MCX384A-BCAA 3CYRK
-- Sent with Vivaldi Mail. Download Vivaldi for free at vivaldi.com
Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
So if im hearing you correctly, i can try to use stx 5.0.1, which i hope has wireguard support, like 6.0 does, and then i guess it also prevents me from ever upgrading to 6.0
On Monday 28 March 2022 21:37:47 PM (+07:00), M. Vefa Bicakci wrote:
Hi,
StarlingX 5.0.1 release has Mellanox OFED v5.1, and I just checked that
the RPM file linked below indeed has the mlx4* kernel modules, so I believe that StarlingX 5.0.1 ought to work with Mellanox ConnectX-3 cards:
http://mirror.starlingx.cengn.ca/mirror/starlingx/release/5.0.1/centos/distr...
It could be possible to adapt Mellanox OFED v4.9 to StarlingX 6.0, but I
am not aware of anyone who has tried this, so it is difficult for me comment with certainty.
Thank you,
Vefa
On 2022-03-26 22:12, Embedded Devel wrote:
So if updated in 6.0, will stx 5.0 work with them ? or is it possible
to build stx 6.0 customized with 4.9
On Saturday 26 March 2022 01:51:23 AM (+07:00), M. Vefa Bicakci wrote:
Hi,
With StarlingX 6.0, the Mellanox OFED device drivers were updated to
version 5.3. According to the following link, nVidia/Mellanox has stopped supporting ConnectX-3 cards such as the one you have (MCX384A) since driver version v5.1:
https://docs.nvidia.com/networking/display/MLNXOFEDv512620/Release+Notes
As noted at the aforementioned link, Mellanox OFED v4.9 LTS continues
to support ConnectX-3 cards. StarlingX supports only the latest version of the Mellanox OFED drivers. Hence, StarlingX no longer ships with a version of the drivers that support ConnectX-3 cards.
Also note that ConnectX-3 cards use the mlx4 driver instead of the
mlx5 driver, as indicated here:
https://docs.nvidia.com/networking/display/MLNXOFEDv494080/Introduction
Thank you,
Vefa
On 2022-03-25 12:59, Khalil, Ghada wrote:
Hi Linda, can someone from the OS team comment on this?
-----Original Message----- From: Embedded Devel lists@optimcloud.com Sent: Thursday, March 17, 2022 9:37 AM To: 'starlingx-discuss@lists.starlingx.io'
starlingx-discuss@lists.starlingx.io
Subject: [Starlingx-discuss] stx 6.0 aio mellenox connectX 40GB
dual card
[Please note: This e-mail is from an EXTERNAL e-mail address]
i just deployed a stx 6.0 aio dual on bare metal yet after booting
and logging into the system i dont see the mellanox connectX 40Gb interfaces i also dont see a mlx5_en kernel module to load
server is a Dell C6220 with Dual-Port 40Gb/s QSFP Mezzanine Network
Card MCX384A-BCAA 3CYRK
-- Sent with Vivaldi Mail. Download Vivaldi for free at vivaldi.com
Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io
http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io
http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
welp further tho the stx 5.0.1 vs 6.0 with mellanox drivers, 5.0.1 wount even install on the Dell C6620 where 6.0 installs fine also stx 5.0.1 doesnt have kernel 5.x, where 6.0 does
so now i have to ask, who do i have to pay to get a stx 6.0 with Mellanox OFED v5.1, and not OFED 5.3, this basically rendered 10K in hardware useless for us.
as we cant seem to get supported OFED v5.1, we installed 6.x and tried top downgrade... and we cant build it for the kernel on the box from sources no dev environment / compiler
HELP!
On Wednesday 06 April 2022 10:49:59 AM (+07:00), Embedded Devel wrote:
So if im hearing you correctly, i can try to use stx 5.0.1, which i hope
has wireguard support, like 6.0 does, and then i guess it also prevents me from ever upgrading to 6.0
On Monday 28 March 2022 21:37:47 PM (+07:00), M. Vefa Bicakci wrote:
Hi,
StarlingX 5.0.1 release has Mellanox OFED v5.1, and I just checked
that the RPM file linked below indeed has the mlx4* kernel modules, so I believe that StarlingX 5.0.1 ought to work with Mellanox ConnectX-3 cards:
http://mirror.starlingx.cengn.ca/mirror/starlingx/release/5.0.1/centos/distr...
It could be possible to adapt Mellanox OFED v4.9 to StarlingX 6.0, but
I am not aware of anyone who has tried this, so it is difficult for me comment with certainty.
Thank you,
Vefa
On 2022-03-26 22:12, Embedded Devel wrote:
So if updated in 6.0, will stx 5.0 work with them ? or is it
possible to build stx 6.0 customized with 4.9
On Saturday 26 March 2022 01:51:23 AM (+07:00), M. Vefa Bicakci
wrote:
Hi,
With StarlingX 6.0, the Mellanox OFED device drivers were updated
to version 5.3. According to the following link, nVidia/Mellanox has stopped supporting ConnectX-3 cards such as the one you have (MCX384A) since driver version v5.1:
https://docs.nvidia.com/networking/display/MLNXOFEDv512620/Release+Notes
As noted at the aforementioned link, Mellanox OFED v4.9 LTS
continues to support ConnectX-3 cards. StarlingX supports only the latest version of the Mellanox OFED drivers. Hence, StarlingX no longer ships with a version of the drivers that support ConnectX-3 cards.
Also note that ConnectX-3 cards use the mlx4 driver instead of the
mlx5 driver, as indicated here:
https://docs.nvidia.com/networking/display/MLNXOFEDv494080/Introduction
Thank you,
Vefa
On 2022-03-25 12:59, Khalil, Ghada wrote:
Hi Linda, can someone from the OS team comment on this?
-----Original Message----- From: Embedded Devel lists@optimcloud.com Sent: Thursday, March 17, 2022 9:37 AM To: 'starlingx-discuss@lists.starlingx.io'
starlingx-discuss@lists.starlingx.io
Subject: [Starlingx-discuss] stx 6.0 aio mellenox connectX 40GB
dual card
[Please note: This e-mail is from an EXTERNAL e-mail address]
i just deployed a stx 6.0 aio dual on bare metal yet after
booting and logging into the system i dont see the mellanox connectX 40Gb interfaces i also dont see a mlx5_en kernel module to load
server is a Dell C6220 with Dual-Port 40Gb/s QSFP Mezzanine
Network Card MCX384A-BCAA 3CYRK
-- Sent with Vivaldi Mail. Download Vivaldi for free at vivaldi.com
Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io
http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io
http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io
http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Hi,
Unfortunately, the only suggestion I can make is to use the build instructions at [1] to build a customized StarlingX v6.0 ISO image that includes Mellanox OFED v5.1 (or v4.9), so that you have a way forward with Mellanox ConnectX-3 network adapters.
The following patches should be helpful as references: [2] [3] [4] [5]. However, also note that support for mlx4 was removed from StarlingX's master branch with the development story at [6], so there will be further difficulties in supporting Mellanox ConnectX-3 in future versions of StarlingX.
Vefa
[1] https://docs.starlingx.io/developer_resources/build_guide.html
[2] https://review.opendev.org/c/starlingx/tools/+/831338 (Download script changes for Mellanox OFED v5.5)
[3] https://review.opendev.org/c/starlingx/kernel/+/831461 (Uprevision to Mellanox OFED v5.5)
[4] https://review.opendev.org/c/starlingx/tools/+/796386 (Download script changes for uprevision to v5.10 kernel and Mellanox OFED v5.3)
[5] https://review.opendev.org/c/starlingx/kernel/+/799000 (Uprevision Mellanox OFED from v5.0 to v5.3)
[6] https://storyboard.openstack.org/#!/story/2009878 (List of changes for Mellanox ConnectX-6, some of which remove support for mlx4.)
On 2022-04-06 00:40, Embedded Devel wrote:
welp further tho the stx 5.0.1 vs 6.0 with mellanox drivers, 5.0.1 wount even install on the Dell C6620 where 6.0 installs fine also stx 5.0.1 doesnt have kernel 5.x, where 6.0 does
so now i have to ask, who do i have to pay to get a stx 6.0 with Mellanox OFED v5.1, and not OFED 5.3, this basically rendered 10K in hardware useless for us.
as we cant seem to get supported OFED v5.1, we installed 6.x and tried top downgrade... and we cant build it for the kernel on the box from sources no dev environment / compiler
HELP!
On Wednesday 06 April 2022 10:49:59 AM (+07:00), Embedded Devel wrote:
So if im hearing you correctly, i can try to use stx 5.0.1, which i hope has wireguard support, like 6.0 does, and then i guess it also prevents me from ever upgrading to 6.0
On Monday 28 March 2022 21:37:47 PM (+07:00), M. Vefa Bicakci wrote:
Hi,
StarlingX 5.0.1 release has Mellanox OFED v5.1, and I just checked that the RPM file linked below indeed has the mlx4* kernel modules, so I believe that StarlingX 5.0.1 ought to work with Mellanox ConnectX-3 cards:
http://mirror.starlingx.cengn.ca/mirror/starlingx/release/5.0.1/centos/distr...
It could be possible to adapt Mellanox OFED v4.9 to StarlingX 6.0, but I am not aware of anyone who has tried this, so it is difficult for me comment with certainty.
Thank you,
Vefa
On 2022-03-26 22:12, Embedded Devel wrote:
So if updated in 6.0, will stx 5.0 work with them ? or is it possible to build stx 6.0 customized with 4.9 On Saturday 26 March 2022 01:51:23 AM (+07:00), M. Vefa Bicakci wrote:
Hi,
With StarlingX 6.0, the Mellanox OFED device drivers were updated to version 5.3. According to the following link, nVidia/Mellanox has stopped supporting ConnectX-3 cards such as the one you have (MCX384A) since driver version v5.1:
https://docs.nvidia.com/networking/display/MLNXOFEDv512620/Release+Notes
As noted at the aforementioned link, Mellanox OFED v4.9 LTS continues to support ConnectX-3 cards. StarlingX supports only the latest version of the Mellanox OFED drivers. Hence, StarlingX no longer ships with a version of the drivers that support ConnectX-3 cards.
Also note that ConnectX-3 cards use the mlx4 driver instead of the mlx5 driver, as indicated here:
https://docs.nvidia.com/networking/display/MLNXOFEDv494080/Introduction
Thank you,
Vefa
On 2022-03-25 12:59, Khalil, Ghada wrote:
Hi Linda, can someone from the OS team comment on this?
-----Original Message----- From: Embedded Devel lists@optimcloud.com Sent: Thursday, March 17, 2022 9:37 AM To: 'starlingx-discuss@lists.starlingx.io' starlingx-discuss@lists.starlingx.io Subject: [Starlingx-discuss] stx 6.0 aio mellenox connectX 40GB dual card
[Please note: This e-mail is from an EXTERNAL e-mail address]
i just deployed a stx 6.0 aio dual on bare metal yet after booting and logging into the system i dont see the mellanox connectX 40Gb interfaces i also dont see a mlx5_en kernel module to load
server is a Dell C6220 with Dual-Port 40Gb/s QSFP Mezzanine Network Card MCX384A-BCAA 3CYRK
-- Sent with Vivaldi Mail. Download Vivaldi for free at vivaldi.com
Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Hi,
Thanks, Ghada. I wasn't subscribed to the starlingx-discuss@ mailing list before, therefore, I didn't see the original email coming through. but now I am. Yes, Vefa from the OS team has helped replied to the question.
Hope that helps, and thanks, Linda
________________________________ From: Khalil, Ghada Ghada.Khalil@windriver.com Sent: Friday, March 25, 2022 9:59 AM To: Embedded Devel lists@optimcloud.com; 'starlingx-discuss@lists.starlingx.io' starlingx-discuss@lists.starlingx.io; Wang, Linda Linda.Wang@windriver.com Subject: RE: [Starlingx-discuss] stx 6.0 aio mellenox connectX 40GB dual card
Hi Linda, can someone from the OS team comment on this?
-----Original Message----- From: Embedded Devel lists@optimcloud.com Sent: Thursday, March 17, 2022 9:37 AM To: 'starlingx-discuss@lists.starlingx.io' starlingx-discuss@lists.starlingx.io Subject: [Starlingx-discuss] stx 6.0 aio mellenox connectX 40GB dual card
[Please note: This e-mail is from an EXTERNAL e-mail address]
i just deployed a stx 6.0 aio dual on bare metal yet after booting and logging into the system i dont see the mellanox connectX 40Gb interfaces i also dont see a mlx5_en kernel module to load
server is a Dell C6220 with Dual-Port 40Gb/s QSFP Mezzanine Network Card MCX384A-BCAA 3CYRK
-- Sent with Vivaldi Mail. Download Vivaldi for free at vivaldi.com
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
participants (4)
-
Embedded Devel
-
Khalil, Ghada
-
M. Vefa Bicakci
-
Wang, Linda