[Starlingx-discuss] (2) How to Set 'isolcpus & HW Accelarator' on Starlingx 5.0
Chris Friesen
chris.friesen at windriver.com
Tue Oct 19 16:36:23 UTC 2021
This is just a hunch, but I suspect that you shouldn't have the
"http://" in front of the isolcpus resource.
If that doesn't fix it, could we get some information from your
installation? For one of the nodes with isolcpus configured please run
the following two commands on the active controller node:
system host-cpu-list <hostname>
kubectl describe node <hostname>
Then ssh to the node in question and run the following commands:
cat /proc/cmdline
cat /sys/devices/system/cpu/isolated
cat /sys/devices/system/cpu/nohz_full
Thanks,
Chris
On 10/18/2021 5:50 PM, 김태준 wrote:
>
> Dear Ildiko Vancsa
>
> Thank you for your reply.
>
> First of all, we have already deployed our pod to the same server
> using WRCP. (with isolcpus & intel_acc100_fec)
>
> The server is trying to query the same pod with Starlingx 5.0 while
> supporting the resources.
>
> After 5.0 installation according to StarlingX Doc, cpu is set to
> Application-isolate in the same way, but the resources are not visible.
>
> 'intel.com/intel_acc100_fec
> <https://urldefense.com/v3/__http://intel.com/intel_acc100_fec__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5AByxEsZ$>:
> ‘16’’
> 'http://windriver.com/isolcpus: 24’
> Is there anything to do separately from starlingsX to make the above
> two resources available?
>
> for Starlingx Installation, I did just below,
>
> https://docs.starlingx.io/deploy_install_guides/r5_release/bare_metal/aio_simplex_install_kubernetes.html
> <https://urldefense.com/v3/__https://docs.starlingx.io/deploy_install_guides/r5_release/bare_metal/aio_simplex_install_kubernetes.html__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5DS4Ev2i$>
>
> for resources, I did belows,
>
> https://protect2.fireeye.com/v1/url?k=67ad2fd7-383616cf-67aca498-0cc47a314e9a-5e0aef83dc32c5cb&q=1&e=5ed23c51-ae04-476f-a95d-f2d4d2a9c1cc&u=https%3A%2F%2Fdocs.starlingx.io%2Fnode_management%2Fkubernetes%2Fhardware_acceleration_devices%2Fset-up-pods-to-use-sriov.html
> <https://urldefense.com/v3/__https://protect2.fireeye.com/v1/url?k=67ad2fd7-383616cf-67aca498-0cc47a314e9a-5e0aef83dc32c5cb&q=1&e=5ed23c51-ae04-476f-a95d-f2d4d2a9c1cc&u=https*3A*2F*2Fdocs.starlingx.io*2Fnode_management*2Fkubernetes*2Fhardware_acceleration_devices*2Fset-up-pods-to-use-sriov.html__;JSUlJSUlJQ!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5INhCM9q$>
> https://protect2.fireeye.com/v1/url?k=33fc70ff-6c6749e7-33fdfbb0-0cc47a314e9a-bc617f111d97e7de&q=1&e=5ed23c51-ae04-476f-a95d-f2d4d2a9c1cc&u=https%3A%2F%2Fdocs.starlingx.io%2Fr%2Fstx.5.0%2Fusertasks%2Fkubernetes%2Fuse-application-isolated-cores.html
> <https://urldefense.com/v3/__https://protect2.fireeye.com/v1/url?k=33fc70ff-6c6749e7-33fdfbb0-0cc47a314e9a-bc617f111d97e7de&q=1&e=5ed23c51-ae04-476f-a95d-f2d4d2a9c1cc&u=https*3A*2F*2Fdocs.starlingx.io*2Fr*2Fstx.5.0*2Fusertasks*2Fkubernetes*2Fuse-application-isolated-cores.html__;JSUlJSUlJSU!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5AH5-nFQ$>
>
> When I check node description, there is no isolcpus and intel_acc100_fec.
>
> Capacity:
> cpu: 64
> ephemeral-storage: 10190100Ki
> hugepages-1Gi: 32Gi
> hugepages-2Mi: 0
> intel.com/pci_sriov_net_ca:
> <https://urldefense.com/v3/__http://intel.com/pci_sriov_net_ca:__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5GbJZtdA$>
> 7
> intel.com/pci_sriov_net_f1c:
> <https://urldefense.com/v3/__http://intel.com/pci_sriov_net_f1c:__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5JvZTDva$>
> 4
> intel.com/pci_sriov_net_f1u:
> <https://urldefense.com/v3/__http://intel.com/pci_sriov_net_f1u:__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5Bi7xLEj$>
> 4
> intel.com/pci_sriov_net_fh0:
> <https://urldefense.com/v3/__http://intel.com/pci_sriov_net_fh0:__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5Bo5Ql8w$>
> 4
> intel.com/pci_sriov_net_fh0m:
> <https://urldefense.com/v3/__http://intel.com/pci_sriov_net_fh0m:__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5PXG9sZK$>
> 4
> intel.com/pci_sriov_net_fh1:
> <https://urldefense.com/v3/__http://intel.com/pci_sriov_net_fh1:__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5JNa4Zdc$>
> 3
> intel.com/pci_sriov_net_fh1m:
> <https://urldefense.com/v3/__http://intel.com/pci_sriov_net_fh1m:__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5JQSZRbg$>
> 4
> memory: 131264360Ki
> pods: 110
>
> Same Server with WRCP case, we can see like this
>
> Capacity:
> cpu: 64
> ephemeral-storage: 10190100Ki
> hugepages-1Gi: 32Gi
> hugepages-2Mi: 0
> intel.com/intel_acc100_fec:
> <https://urldefense.com/v3/__http://intel.com/intel_acc100_fec:__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5Oq_FC4d$>
> 1
> intel.com/pci_sriov_net_ca:
> <https://urldefense.com/v3/__http://intel.com/pci_sriov_net_ca:__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5GbJZtdA$>
> 7
> intel.com/pci_sriov_net_f1c:
> <https://urldefense.com/v3/__http://intel.com/pci_sriov_net_f1c:__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5JvZTDva$>
> 4
> intel.com/pci_sriov_net_f1u:
> <https://urldefense.com/v3/__http://intel.com/pci_sriov_net_f1u:__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5Bi7xLEj$>
> 4
> intel.com/pci_sriov_net_fh0:
> <https://urldefense.com/v3/__http://intel.com/pci_sriov_net_fh0:__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5Bo5Ql8w$>
> 4
> intel.com/pci_sriov_net_fh0m:
> <https://urldefense.com/v3/__http://intel.com/pci_sriov_net_fh0m:__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5PXG9sZK$>
> 4
> intel.com/pci_sriov_net_fh1:
> <https://urldefense.com/v3/__http://intel.com/pci_sriov_net_fh1:__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5JNa4Zdc$>
> 3
> intel.com/pci_sriov_net_fh1m:
> <https://urldefense.com/v3/__http://intel.com/pci_sriov_net_fh1m:__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5JQSZRbg$>
> 4
> memory: 131365668Ki
> pods: 110
> http://windriver.com/isolcpus: 54
>
> Please let me know if there is anything I can do.
>
> Thanks and Best Regards,
> Taejun
>
> --------- *Original Message* ---------
>
> *Sender* : Ildiko Vancsa <ildiko.vancsa at gmail.com>
>
> *Date* : 2021-10-19 03:56 (GMT+9)
>
> *Title* : Re: [Starlingx-discuss] How to Set 'isolcpus & HW
> Accelarator' on Starlingx 5.0
>
> Dear Kim Tae-jun,
>
> Thank you for reaching out with your question.
>
> I think you may need to modify the yaml file to adjust it to the resources available in your system. Which means to modify these entries:
> 'intel.com/intel_acc100_fec <https://urldefense.com/v3/__http://intel.com/intel_acc100_fec__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5AByxEsZ$>: ‘16’’
> 'http://windriver.com/isolcpus: 24’
>
> Do you have the required resources available in your environment to use this feature?
>
> Thanks and Best Regards,
> Ildikó
>
>
> > On Oct 12, 2021, at 18:02, 김태준 <tj85.kim at samsung.com> wrote:
> >
> > Dear starlingx members
> >
> > Hello. This is Kim Tae-jun from Samsung Electronics.
> > We are trying a trial to apply Starlingx to our system.
> >
> > Among them, we are trying to set isolcpus and hw indicator according to the manual in starlingx docs and modify pod.
> > However, even if you create yaml as manual, you can not run properly because you do not have the following resources.
> >
> > Type Reason Age From Message
> > ---- ------ ---- ---- -------
> > Warning FailedScheduling 6s (x5 over 15s) default-scheduler 0/1 nodes are available: 1 Insufficient intel.com/intel_acc100_fec <https://urldefense.com/v3/__http://intel.com/intel_acc100_fec__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5AByxEsZ$>, 1 Insufficienthttp://windriver.com/isolcpus.
> >
> > resources corresponding to the capacity of the node description are not visible.
> >
> > I set it by referring to the links below. I would appreciate it if you would let me know if there is anything missing in the preliminary work or what to refer to.
> >
> > https://protect2.fireeye.com/v1/url?k=67ad2fd7-383616cf-67aca498-0cc47a314e9a-5e0aef83dc32c5cb&q=1&e=5ed23c51-ae04-476f-a95d-f2d4d2a9c1cc&u=https%3A%2F%2Fdocs.starlingx.io%2Fnode_management%2Fkubernetes%2Fhardware_acceleration_devices%2Fset-up-pods-to-use-sriov.html <https://urldefense.com/v3/__https://protect2.fireeye.com/v1/url?k=67ad2fd7-383616cf-67aca498-0cc47a314e9a-5e0aef83dc32c5cb&q=1&e=5ed23c51-ae04-476f-a95d-f2d4d2a9c1cc&u=https*3A*2F*2Fdocs.starlingx.io*2Fnode_management*2Fkubernetes*2Fhardware_acceleration_devices*2Fset-up-pods-to-use-sriov.html__;JSUlJSUlJQ!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5INhCM9q$>
> > https://protect2.fireeye.com/v1/url?k=33fc70ff-6c6749e7-33fdfbb0-0cc47a314e9a-bc617f111d97e7de&q=1&e=5ed23c51-ae04-476f-a95d-f2d4d2a9c1cc&u=https%3A%2F%2Fdocs.starlingx.io%2Fr%2Fstx.5.0%2Fusertasks%2Fkubernetes%2Fuse-application-isolated-cores.html <https://urldefense.com/v3/__https://protect2.fireeye.com/v1/url?k=33fc70ff-6c6749e7-33fdfbb0-0cc47a314e9a-bc617f111d97e7de&q=1&e=5ed23c51-ae04-476f-a95d-f2d4d2a9c1cc&u=https*3A*2F*2Fdocs.starlingx.io*2Fr*2Fstx.5.0*2Fusertasks*2Fkubernetes*2Fuse-application-isolated-cores.html__;JSUlJSUlJSU!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5AH5-nFQ$>
> >
> > Thanks you.
> >
> > <Mail Attachment.gif>
> > _______________________________________________
> > Starlingx-discuss mailing list
> > Starlingx-discuss at lists.starlingx.io
> > https://protect2.fireeye.com/v1/url?k=dd734e99-82e87781-dd72c5d6-0cc47a314e9a-0b594a17d9dfec9d&q=1&e=5ed23c51-ae04-476f-a95d-f2d4d2a9c1cc&u=http%3A%2F%2Flists.starlingx.io%2Fcgi-bin%2Fmailman%2Flistinfo%2Fstarlingx-discuss <https://urldefense.com/v3/__https://protect2.fireeye.com/v1/url?k=dd734e99-82e87781-dd72c5d6-0cc47a314e9a-0b594a17d9dfec9d&q=1&e=5ed23c51-ae04-476f-a95d-f2d4d2a9c1cc&u=http*3A*2F*2Flists.starlingx.io*2Fcgi-bin*2Fmailman*2Flistinfo*2Fstarlingx-discuss__;JSUlJSUlJQ!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5CCsQZo3$>
>
>
> _______________________________________________
> Starlingx-discuss mailing list
> Starlingx-discuss at lists.starlingx.io
> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20211019/22ef9711/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: yvJPCJbq2rCvNa5b.png
Type: image/png
Size: 13402 bytes
Desc: not available
URL: <http://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20211019/22ef9711/attachment-0001.png>
More information about the Starlingx-discuss
mailing list