[Starlingx-discuss] SRIOV in starlingx

Sriram sriram.ec at gmail.com
Thu Oct 1 08:00:26 UTC 2020


Thanks Pratik.  There is no openstack-neutron component.
We have only one k8s cluster in the edge cloud in one subnet.
Some of the docs that I read suggest assigning mac addresses to vf's using
"ip link" command. I'm not sure if that's the only way forward.

Regards,
Sriram



On Thu, Oct 1, 2020 at 7:22 AM Pratik M. <pvmpublic at gmail.com> wrote:

> Hi,
> a.: I believe you should be able to lock, apply these steps, and unlock.
>
> b.: I don't know if StarlingX has an option to specify per node pools.
> Will defer to experts. But if the nodes are in one L2, they would typically
> be in one cluster and thus the VF MAC assignment would be arbitrated by
> cluster wide neutron/CNI, right? If each node is one cluster, maybe the
> user would need to annotate the pods with static MACs.
>
> BR
>
>
> On Tue, 29 Sep, 2020, 23:23 Sriram, <sriram.ec at gmail.com> wrote:
>
>> Hi,
>>
>> How do we ensure the uniqueness of VF mac addresses across all the nodes
>> in the k8s cluster formed on edge nodes.
>> Please let me know if this problem is addressed by starlingX or if it is
>> taken care of by some other means.
>>
>> Regards,
>> Sriram
>>
>> On Fri, Sep 25, 2020 at 1:20 PM Sriram <sriram.ec at gmail.com> wrote:
>>
>>> Hi Pratik,
>>>
>>> Thanks for your reply.
>>>
>>> a. Can these steps be done after the installation is complete, now that
>>> I have already installed.
>>> b. How do we ensure the uniqueness of VF mac addresses across all the
>>> nodes in the k8s cluster formed on edge nodes.
>>>
>>> Regards,
>>> Sriram
>>>
>>> On Fri, Sep 25, 2020 at 1:02 PM Pratik M. <pvmpublic at gmail.com> wrote:
>>>
>>>> Hi,
>>>> You would need to do:
>>>> # system host-label-assign controller-0 sriovdp=enabled
>>>> # system host-if-modify controller-0 <interface> -c pci-sriov -n sriov0
>>>> -N <num vfs>
>>>> # system interface-datanetwork-assign controller-0 <interface>
>>>> <datanetwork>
>>>> # system host-unlock
>>>>
>>>> And that should populate the /etc/pcidp/config.json
>>>>
>>>> Ref:
>>>> https://wiki.openstack.org/wiki/StarlingX/Networking
>>>> Steven Webster's helpful comments in
>>>> https://bugs.launchpad.net/starlingx/+bug/1891889
>>>>
>>>> Thanks
>>>>
>>>> On Thu, Sep 24, 2020 at 3:04 PM Sriram <sriram.ec at gmail.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I have installed distributed starlingx 4.0 in "All in one Duplex"
>>>>> mode. There are two nodes in the central cloud and two in the edge cloud.
>>>>>
>>>>> I have enabled SRIOV in bios settings of edge cloud nodes and set
>>>>> total VFs as 16.
>>>>>
>>>>> After that, while installing starlingX I followed the steps to enable
>>>>> SRIOV.
>>>>>
>>>>> system host-label-assign controller-0 sriovdp=enabled
>>>>>> system host-memory-modify controller-0 0 -1G 100
>>>>>> system host-memory-modify controller-0 1 -1G 100
>>>>>
>>>>> and ran these steps for controller-1 as well.
>>>>>
>>>>> As I understand the first step would label the node "controller-0 and
>>>>> controller-1" as "sriovdp=enabled" and set the number of 1G huge pages to
>>>>> 200.
>>>>> Once the installation was complete, I saw that k8s sriov-device plugin
>>>>> was not coming up. It complained that the resource list was empty.
>>>>>
>>>>> I had to set
>>>>> "/sys/devices/pci0000:3a/0000:3a:00.0/0000:3b:00.0/sriov_numvfs" to 8 (I
>>>>> needed 8 virtual interfaces) and update the resource list in
>>>>> /etc/pcidp/config.json
>>>>>
>>>>> {
>>>>>>   "resourceList": [
>>>>>>             {
>>>>>>                 "resourceName": "bcm_sriov_netdevice",
>>>>>>                 "selectors": {
>>>>>>                     "vendors": ["14e4"],
>>>>>>                     "devices": ["16dc"],
>>>>>>                     "drivers": ["bnxt_en"],
>>>>>>                     "pfNames": ["enp59s0f0#0-7"]
>>>>>>                 }
>>>>>>              }
>>>>>>     ]
>>>>>> }
>>>>>
>>>>>
>>>>> to see that sriov-dp comes up properly.
>>>>>
>>>>> Is there any way to pass on the number of VF's( sriov_numvfs )required
>>>>> per node and resourcelist during the time of installation when we label the
>>>>> nodes as  sriovdp=enabled
>>>>>
>>>>> Regards,
>>>>> Sriram
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Starlingx-discuss mailing list
>>>>> Starlingx-discuss at lists.starlingx.io
>>>>> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
>>>>>
>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20201001/55e5abbe/attachment-0001.html>


More information about the Starlingx-discuss mailing list