[Starlingx-discuss] SRIOV in starlingx

[Pratik M.]

Hi,
a.: I believe you should be able to lock, apply these steps, and unlock.

b.: I don't know if StarlingX has an option to specify per node pools. Will
defer to experts. But if the nodes are in one L2, they would typically be
in one cluster and thus the VF MAC assignment would be arbitrated by
cluster wide neutron/CNI, right? If each node is one cluster, maybe the
user would need to annotate the pods with static MACs.

BR


On Tue, 29 Sep, 2020, 23:23 Sriram, <sriram.ec at gmail.com> wrote:

> Hi,
>
> How do we ensure the uniqueness of VF mac addresses across all the nodes
> in the k8s cluster formed on edge nodes.
> Please let me know if this problem is addressed by starlingX or if it is
> taken care of by some other means.
>
> Regards,
> Sriram
>
> On Fri, Sep 25, 2020 at 1:20 PM Sriram <sriram.ec at gmail.com> wrote:
>
>> Hi Pratik,
>>
>> Thanks for your reply.
>>
>> a. Can these steps be done after the installation is complete, now that I
>> have already installed.
>> b. How do we ensure the uniqueness of VF mac addresses across all the
>> nodes in the k8s cluster formed on edge nodes.
>>
>> Regards,
>> Sriram
>>
>> On Fri, Sep 25, 2020 at 1:02 PM Pratik M. <pvmpublic at gmail.com> wrote:
>>
>>> Hi,
>>> You would need to do:
>>> # system host-label-assign controller-0 sriovdp=enabled
>>> # system host-if-modify controller-0 <interface> -c pci-sriov -n sriov0
>>> -N <num vfs>
>>> # system interface-datanetwork-assign controller-0 <interface>
>>> <datanetwork>
>>> # system host-unlock
>>>
>>> And that should populate the /etc/pcidp/config.json
>>>
>>> Ref:
>>> https://wiki.openstack.org/wiki/StarlingX/Networking
>>> Steven Webster's helpful comments in
>>> https://bugs.launchpad.net/starlingx/+bug/1891889
>>>
>>> Thanks
>>>
>>> On Thu, Sep 24, 2020 at 3:04 PM Sriram <sriram.ec at gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> I have installed distributed starlingx 4.0 in "All in one Duplex" mode.
>>>> There are two nodes in the central cloud and two in the edge cloud.
>>>>
>>>> I have enabled SRIOV in bios settings of edge cloud nodes and set total
>>>> VFs as 16.
>>>>
>>>> After that, while installing starlingX I followed the steps to enable
>>>> SRIOV.
>>>>
>>>> system host-label-assign controller-0 sriovdp=enabled
>>>>> system host-memory-modify controller-0 0 -1G 100
>>>>> system host-memory-modify controller-0 1 -1G 100
>>>>
>>>> and ran these steps for controller-1 as well.
>>>>
>>>> As I understand the first step would label the node "controller-0 and
>>>> controller-1" as "sriovdp=enabled" and set the number of 1G huge pages to
>>>> 200.
>>>> Once the installation was complete, I saw that k8s sriov-device plugin
>>>> was not coming up. It complained that the resource list was empty.
>>>>
>>>> I had to set
>>>> "/sys/devices/pci0000:3a/0000:3a:00.0/0000:3b:00.0/sriov_numvfs" to 8 (I
>>>> needed 8 virtual interfaces) and update the resource list in
>>>> /etc/pcidp/config.json
>>>>
>>>> {
>>>>>   "resourceList": [
>>>>>             {
>>>>>                 "resourceName": "bcm_sriov_netdevice",
>>>>>                 "selectors": {
>>>>>                     "vendors": ["14e4"],
>>>>>                     "devices": ["16dc"],
>>>>>                     "drivers": ["bnxt_en"],
>>>>>                     "pfNames": ["enp59s0f0#0-7"]
>>>>>                 }
>>>>>              }
>>>>>     ]
>>>>> }
>>>>
>>>>
>>>> to see that sriov-dp comes up properly.
>>>>
>>>> Is there any way to pass on the number of VF's( sriov_numvfs )required
>>>> per node and resourcelist during the time of installation when we label the
>>>> nodes as  sriovdp=enabled
>>>>
>>>> Regards,
>>>> Sriram
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Starlingx-discuss mailing list
>>>> Starlingx-discuss at lists.starlingx.io
>>>> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
>>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20201001/d001490a/attachment.html>