<div dir="auto"><div><span style="font-family:sans-serif">Hi,</span><div style="font-family:sans-serif" dir="auto">a.: I believe you should be able to lock, apply these steps, and unlock.</div><div dir="auto" style="font-family:sans-serif"><br></div><div style="font-family:sans-serif" dir="auto">b.: I don't know if StarlingX has an option to specify per node pools. Will defer to experts. But if the nodes are in one L2, they would typically be in one cluster and thus the VF MAC assignment would be arbitrated by cluster wide neutron/CNI, right? If each node is one cluster, maybe the user would need to annotate the pods with static MACs.</div><div dir="auto" style="font-family:sans-serif"><br></div><div dir="auto" style="font-family:sans-serif">BR</div><br><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, 29 Sep, 2020, 23:23 Sriram, <<a href="mailto:sriram.ec@gmail.com">sriram.ec@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi,<div><br></div><div>How do we ensure the uniqueness of VF mac addresses across all the nodes in the k8s cluster formed on edge nodes. </div><div>Please let me know if this problem is addressed by starlingX or if it is taken care of by some other means.</div><div><br></div><div>Regards,<br></div><div>Sriram</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Sep 25, 2020 at 1:20 PM Sriram <<a href="mailto:sriram.ec@gmail.com" target="_blank" rel="noreferrer">sriram.ec@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">Hi Pratik,<div><br></div><div>Thanks for your reply. </div><div><br></div><div>a. Can these steps be done after the installation is complete, now that I have already installed.</div><div>b. How do we ensure the uniqueness of VF mac addresses across all the nodes in the k8s cluster formed on edge nodes.</div><div><br></div><div>Regards,<br>Sriram</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Sep 25, 2020 at 1:02 PM Pratik M. <<a href="mailto:pvmpublic@gmail.com" target="_blank" rel="noreferrer">pvmpublic@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi,<br>You would need to do:<br># system host-label-assign controller-0 sriovdp=enabled <br># system host-if-modify controller-0 <interface> -c pci-sriov -n sriov0 -N <num vfs> <br># system interface-datanetwork-assign controller-0 <interface> <datanetwork> <br># system host-unlock<br><br>And that should populate the /etc/pcidp/config.json<br><br>Ref:<br><a href="https://wiki.openstack.org/wiki/StarlingX/Networking" target="_blank" rel="noreferrer">https://wiki.openstack.org/wiki/StarlingX/Networking</a><br>Steven Webster's helpful comments in <a href="https://bugs.launchpad.net/starlingx/+bug/1891889" target="_blank" rel="noreferrer">https://bugs.launchpad.net/starlingx/+bug/1891889</a><br><br>Thanks<br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Sep 24, 2020 at 3:04 PM Sriram <<a href="mailto:sriram.ec@gmail.com" target="_blank" rel="noreferrer">sriram.ec@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi,<div><br></div><div>I have installed distributed starlingx 4.0 in "All in one Duplex" mode. There are two nodes in the central cloud and two in the edge cloud.</div><div><br></div><div>I have enabled SRIOV in bios settings of edge cloud nodes and set total VFs as 16. </div><div><br></div><div>After that, while installing starlingX I followed the steps to enable SRIOV.</div><div><span style="color:rgb(23,43,77);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,"Noto Sans",Ubuntu,"Droid Sans","Helvetica Neue",sans-serif;font-size:16px;letter-spacing:-0.08px;white-space:pre-wrap"><br></span></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span style="color:rgb(23,43,77);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,"Noto Sans",Ubuntu,"Droid Sans","Helvetica Neue",sans-serif;font-size:16px;letter-spacing:-0.08px;white-space:pre-wrap">system host-label-assign controller-0 sriovdp=enabled<br></span><span style="color:rgb(23,43,77);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,"Noto Sans",Ubuntu,"Droid Sans","Helvetica Neue",sans-serif;font-size:16px;letter-spacing:-0.08px;white-space:pre-wrap">system host-memory-modify controller-0 0 -1G 100<br></span><span style="color:rgb(23,43,77);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,"Noto Sans",Ubuntu,"Droid Sans","Helvetica Neue",sans-serif;font-size:16px;letter-spacing:-0.08px;white-space:pre-wrap">system host-memory-modify controller-0 1 -1G 100</span> </blockquote><div>and ran these steps for controller-1 as well. <br></div><div><br></div><div>As I understand the first step would label the node "controller-0 and controller-1" as "sriovdp=enabled" and set the number of 1G huge pages to 200.</div><div>Once the installation was complete, I saw that k8s sriov-device plugin was not coming up. It complained that the resource list was empty. <br></div><div><br></div><div>I had to set "/sys/devices/pci0000:3a/0000:3a:00.0/0000:3b:00.0/sriov_numvfs" to 8 (I needed 8 virtual interfaces) and update the resource list in </div><div>/etc/pcidp/config.json</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">{<br> "resourceList": [<br> {<br> "resourceName": "bcm_sriov_netdevice",<br> "selectors": {<br> "vendors": ["14e4"],<br> "devices": ["16dc"],<br> "drivers": ["bnxt_en"],<br> "pfNames": ["enp59s0f0#0-7"]<br> }<br> }<br> ]<br>}</blockquote><div><br></div><div>to see that sriov-dp comes up properly.</div><div><br></div><div>Is there any way to pass on the number of VF's( sriov_numvfs )required per node and resourcelist during the time of installation when we label the nodes as
sriovdp=enabled
</div><div><br></div><div>Regards,</div><div>Sriram</div><div><br></div><div> </div><div><br></div><div><br></div><div><br></div><div><br></div><div> </div></div>
_______________________________________________<br>
Starlingx-discuss mailing list<br>
<a href="mailto:Starlingx-discuss@lists.starlingx.io" target="_blank" rel="noreferrer">Starlingx-discuss@lists.starlingx.io</a><br>
<a href="http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss" rel="noreferrer noreferrer" target="_blank">http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss</a><br>
</blockquote></div>
</blockquote></div></div>
</blockquote></div>
</blockquote></div></div></div>