[Starlingx-discuss] Managing VFs in SRIOV resource list
Hi - We have a requirement to create two SR-IOV resource lists from same physical interface. This requirement is for a use case where we are running Intel Flexran software in a Kuberenetes pod. The Intel Flexran software requires that a u-plane VLAN is assigned to a VF and passed as u-plane VF to the software command line. Similarly, a c-plane VLAN needs to be assigned to a VF and passed as c-plane VF to the software command line. The SR-IOV daemon set supports this configuration: { "resourceName": "intel_sriov_dpdk_cplane", "resourcePrefix": "radisys.com", "selectors": { "rootDevices": [ "0000:18:00.3#4-7" ], "drivers": [ "igb_uio" ] } }, { "resourceName": "intel_sriov_dpdk_uplane", "resourcePrefix": "radisys.com", "selectors": { "rootDevices": [ "0000:18:00.3#0-3" ], "drivers": [ "igb_uio" ] } } In order to achieve this we followed the below steps in StarlingX Create Datanetworks for C and U plane traffic $ system datanetwork-add datanet-u -m 9216 vlan $ system datanetwork-add datanet-c -m 9216 vlan $ system host-if-modify -m 9216 -n sriovfh1 -c pci-sriov -N 16 --vf-driver=vfio controller-0 enp177s0f3 $ system interface-datanetwork-assign controller-0 sriovfh1 datanet-u $ system interface-datanetwork-assign controller-0 sriovfh1 datanet-c And we ended up in getting 2 resource lists (one for the C-PLANE and U-PLANE) but both of them containing the same list of VFs. Is there a way I can configure separate VFs for both the resource list that I shown in the above example?. In a standard Kubernetes deployment I would directly change the configmap & re-run the SRIOV daemon set. How can do this in StarlingX and does any System API support this? $ cat /etc/pcidp/config.json { "resourceName": "pci_sriov_net_datanet_up", "selectors": { "vendors": [ "8086" ], "drivers": [ "vfio-pci" ], "devices": [ "154c" ], "pfNames": [ "enp177s0f3#0,1,2,3,4,5,6,7" ] } }, { "resourceName": "pci_sriov_net_datanet_cp", "selectors": { "vendors": [ "8086" ], "drivers": [ "vfio-pci" ], "devices": [ "154c" ], "pfNames": [ "enp177s0f3#0,1,2,3,4,5,6,7" ] } }, Venkata Veldanda
Resending with a small correction. From: Venkata Ramana Veldanda Sent: Saturday, March 6, 2021 9:02 AM To: starlingx-discuss@lists.starlingx.io Subject: Managing VFs in SRIOV resource list Hi - We have a requirement to create two SR-IOV resource lists from same physical interface. This requirement is for a use case where we are running Intel Flexran software in a Kuberenetes pod. The Intel Flexran software requires that a u-plane VLAN is assigned to a VF and passed as u-plane VF to the software command line. Similarly, a c-plane VLAN needs to be assigned to a VF and passed as c-plane VF to the software command line. The SR-IOV daemon set supports this configuration: { "resourceName": "intel_sriov_dpdk_cplane", "resourcePrefix": "radisys.com", "selectors": { "rootDevices": [ "0000:18:00.3#4-7" ], "drivers": [ "igb_uio" ] } }, { "resourceName": "intel_sriov_dpdk_uplane", "resourcePrefix": "radisys.com", "selectors": { "rootDevices": [ "0000:18:00.3#0-3" ], "drivers": [ "igb_uio" ] } } In order to achieve this we followed the below steps in StarlingX Create Datanetworks for C and U plane traffic $ system datanetwork-add datanet-u -m 9216 vlan $ system datanetwork-add datanet-c -m 9216 vlan $ system host-if-modify -m 9216 -n sriovfh1 -c pci-sriov -N 8 --vf-driver=vfio controller-0 enp177s0f3 $ system interface-datanetwork-assign controller-0 sriovfh1 datanet-u $ system interface-datanetwork-assign controller-0 sriovfh1 datanet-c And we ended up in getting 2 resource lists (one for the C-PLANE and U-PLANE) but both of them containing the same list of VFs. Is there a way I can configure separate VFs for both the resource list that I shown in the above example?. In a standard Kubernetes deployment I would directly change the configmap & re-run the SRIOV daemon set. How can do this in StarlingX and does any System API support this? $ cat /etc/pcidp/config.json { "resourceName": "pci_sriov_net_datanet_up", "selectors": { "vendors": [ "8086" ], "drivers": [ "vfio-pci" ], "devices": [ "154c" ], "pfNames": [ "enp177s0f3#0,1,2,3,4,5,6,7" ] } }, { "resourceName": "pci_sriov_net_datanet_cp", "selectors": { "vendors": [ "8086" ], "drivers": [ "vfio-pci" ], "devices": [ "154c" ], "pfNames": [ "enp177s0f3#0,1,2,3,4,5,6,7" ] } }, Venkata Veldanda
Hi - Any help or suggestions around this? From: Venkata Ramana Veldanda Sent: 06 March 2021 11:05 To: starlingx-discuss@lists.starlingx.io Cc: Srinivas Sadagopan <Srinivas.Sadagopan@radisys.com> Subject: RE: Managing VFs in SRIOV resource list Resending with a small correction. From: Venkata Ramana Veldanda Sent: Saturday, March 6, 2021 9:02 AM To: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: Managing VFs in SRIOV resource list Hi - We have a requirement to create two SR-IOV resource lists from same physical interface. This requirement is for a use case where we are running Intel Flexran software in a Kuberenetes pod. The Intel Flexran software requires that a u-plane VLAN is assigned to a VF and passed as u-plane VF to the software command line. Similarly, a c-plane VLAN needs to be assigned to a VF and passed as c-plane VF to the software command line. The SR-IOV daemon set supports this configuration: { "resourceName": "intel_sriov_dpdk_cplane", "resourcePrefix": "radisys.com", "selectors": { "rootDevices": [ "0000:18:00.3#4-7" ], "drivers": [ "igb_uio" ] } }, { "resourceName": "intel_sriov_dpdk_uplane", "resourcePrefix": "radisys.com", "selectors": { "rootDevices": [ "0000:18:00.3#0-3" ], "drivers": [ "igb_uio" ] } } In order to achieve this we followed the below steps in StarlingX Create Datanetworks for C and U plane traffic $ system datanetwork-add datanet-u -m 9216 vlan $ system datanetwork-add datanet-c -m 9216 vlan $ system host-if-modify -m 9216 -n sriovfh1 -c pci-sriov -N 8 --vf-driver=vfio controller-0 enp177s0f3 $ system interface-datanetwork-assign controller-0 sriovfh1 datanet-u $ system interface-datanetwork-assign controller-0 sriovfh1 datanet-c And we ended up in getting 2 resource lists (one for the C-PLANE and U-PLANE) but both of them containing the same list of VFs. Is there a way I can configure separate VFs for both the resource list that I shown in the above example?. In a standard Kubernetes deployment I would directly change the configmap & re-run the SRIOV daemon set. How can do this in StarlingX and does any System API support this? $ cat /etc/pcidp/config.json { "resourceName": "pci_sriov_net_datanet_up", "selectors": { "vendors": [ "8086" ], "drivers": [ "vfio-pci" ], "devices": [ "154c" ], "pfNames": [ "enp177s0f3#0,1,2,3,4,5,6,7" ] } }, { "resourceName": "pci_sriov_net_datanet_cp", "selectors": { "vendors": [ "8086" ], "drivers": [ "vfio-pci" ], "devices": [ "154c" ], "pfNames": [ "enp177s0f3#0,1,2,3,4,5,6,7" ] } }, Venkata Veldanda
Hi Venkata, You should be able to 'split the nic' by creating a vf type interface as a child of the sriovfh1 with a subset of VFs, then assign each interface to its own data network. For example, if you wanted to give 4 VFs to each u-plane and c-plane: system host-if-modify -m 9216 -n sriovfh1 -c pci-sriov -N 8 --vf-driver=vfio controller-0 enp177s0f3 system host-if-add -c pci-sriov controller-0 sriovfh2 vf sriovfh1 -N 4 --vf-driver=vfio system interface-datanetwork-assign controller-0 sriovfh1 datanet-u system interface-datanetwork-assign controller-0 sriovfh2 datanet-c Then you can create separate network attachment definitions for the c-plane and u-plane with your desired vlan(s). Steve From: Venkata Ramana Veldanda <Venkata.Veldanda@radisys.com> Sent: Saturday, March 6, 2021 12:35 AM To: starlingx-discuss@lists.starlingx.io Cc: Srinivas Sadagopan <Srinivas.Sadagopan@radisys.com> Subject: Re: [Starlingx-discuss] Managing VFs in SRIOV resource list [Please note: This e-mail is from an EXTERNAL e-mail address] Resending with a small correction. From: Venkata Ramana Veldanda Sent: Saturday, March 6, 2021 9:02 AM To: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: Managing VFs in SRIOV resource list Hi - We have a requirement to create two SR-IOV resource lists from same physical interface. This requirement is for a use case where we are running Intel Flexran software in a Kuberenetes pod. The Intel Flexran software requires that a u-plane VLAN is assigned to a VF and passed as u-plane VF to the software command line. Similarly, a c-plane VLAN needs to be assigned to a VF and passed as c-plane VF to the software command line. The SR-IOV daemon set supports this configuration: { "resourceName": "intel_sriov_dpdk_cplane", "resourcePrefix": "radisys.com", "selectors": { "rootDevices": [ "0000:18:00.3#4-7" ], "drivers": [ "igb_uio" ] } }, { "resourceName": "intel_sriov_dpdk_uplane", "resourcePrefix": "radisys.com", "selectors": { "rootDevices": [ "0000:18:00.3#0-3" ], "drivers": [ "igb_uio" ] } } In order to achieve this we followed the below steps in StarlingX Create Datanetworks for C and U plane traffic $ system datanetwork-add datanet-u -m 9216 vlan $ system datanetwork-add datanet-c -m 9216 vlan $ system host-if-modify -m 9216 -n sriovfh1 -c pci-sriov -N 8 --vf-driver=vfio controller-0 enp177s0f3 $ system interface-datanetwork-assign controller-0 sriovfh1 datanet-u $ system interface-datanetwork-assign controller-0 sriovfh1 datanet-c And we ended up in getting 2 resource lists (one for the C-PLANE and U-PLANE) but both of them containing the same list of VFs. Is there a way I can configure separate VFs for both the resource list that I shown in the above example?. In a standard Kubernetes deployment I would directly change the configmap & re-run the SRIOV daemon set. How can do this in StarlingX and does any System API support this? $ cat /etc/pcidp/config.json { "resourceName": "pci_sriov_net_datanet_up", "selectors": { "vendors": [ "8086" ], "drivers": [ "vfio-pci" ], "devices": [ "154c" ], "pfNames": [ "enp177s0f3#0,1,2,3,4,5,6,7" ] } }, { "resourceName": "pci_sriov_net_datanet_cp", "selectors": { "vendors": [ "8086" ], "drivers": [ "vfio-pci" ], "devices": [ "154c" ], "pfNames": [ "enp177s0f3#0,1,2,3,4,5,6,7" ] } }, Venkata Veldanda
participants (2)
-
Venkata Ramana Veldanda
-
Webster, Steven