[Starlingx-discuss] SRIOV in starlingx

[Sriram]

Hi Pratik,

Thanks for your reply.

a. Can these steps be done after the installation is complete, now that I
have already installed.
b. How do we ensure the uniqueness of VF mac addresses across all the nodes
in the k8s cluster formed on edge nodes.

Regards,
Sriram

On Fri, Sep 25, 2020 at 1:02 PM Pratik M. <pvmpublic at gmail.com> wrote:

> Hi,
> You would need to do:
> # system host-label-assign controller-0 sriovdp=enabled
> # system host-if-modify controller-0 <interface> -c pci-sriov -n sriov0 -N
> <num vfs>
> # system interface-datanetwork-assign controller-0 <interface>
> <datanetwork>
> # system host-unlock
>
> And that should populate the /etc/pcidp/config.json
>
> Ref:
> https://wiki.openstack.org/wiki/StarlingX/Networking
> Steven Webster's helpful comments in
> https://bugs.launchpad.net/starlingx/+bug/1891889
>
> Thanks
>
> On Thu, Sep 24, 2020 at 3:04 PM Sriram <sriram.ec at gmail.com> wrote:
>
>> Hi,
>>
>> I have installed distributed starlingx 4.0 in "All in one Duplex" mode.
>> There are two nodes in the central cloud and two in the edge cloud.
>>
>> I have enabled SRIOV in bios settings of edge cloud nodes and set total
>> VFs as 16.
>>
>> After that, while installing starlingX I followed the steps to enable
>> SRIOV.
>>
>> system host-label-assign controller-0 sriovdp=enabled
>>> system host-memory-modify controller-0 0 -1G 100
>>> system host-memory-modify controller-0 1 -1G 100
>>
>> and ran these steps for controller-1 as well.
>>
>> As I understand the first step would label the node "controller-0 and
>> controller-1" as "sriovdp=enabled" and set the number of 1G huge pages to
>> 200.
>> Once the installation was complete, I saw that k8s sriov-device plugin
>> was not coming up. It complained that the resource list was empty.
>>
>> I had to set
>> "/sys/devices/pci0000:3a/0000:3a:00.0/0000:3b:00.0/sriov_numvfs" to 8 (I
>> needed 8 virtual interfaces) and update the resource list in
>> /etc/pcidp/config.json
>>
>> {
>>>   "resourceList": [
>>>             {
>>>                 "resourceName": "bcm_sriov_netdevice",
>>>                 "selectors": {
>>>                     "vendors": ["14e4"],
>>>                     "devices": ["16dc"],
>>>                     "drivers": ["bnxt_en"],
>>>                     "pfNames": ["enp59s0f0#0-7"]
>>>                 }
>>>              }
>>>     ]
>>> }
>>
>>
>> to see that sriov-dp comes up properly.
>>
>> Is there any way to pass on the number of VF's( sriov_numvfs )required
>> per node and resourcelist during the time of installation when we label the
>> nodes as  sriovdp=enabled
>>
>> Regards,
>> Sriram
>>
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> Starlingx-discuss mailing list
>> Starlingx-discuss at lists.starlingx.io
>> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20200925/c0ddb1db/attachment.html>