[Starlingx-discuss] (2) How to Set 'isolcpus & HW Accelarator' on Starlingx 5.0
Webster, Steven
Steven.Webster at windriver.com
Tue Oct 19 20:13:22 UTC 2021
Hi Taejun,
In addition to what Chris mentioned, have you configured the ACC100 device for SR-IOV?
Step 7. In this guide:
https://docs.starlingx.io/node_management/kubernetes/hardware_acceleration_devices/enabling-mount-bryce-hw-accelerator-for-hosted-vram-containerized-workloads.html
Cheers,
Steve
From: Chris Friesen <chris.friesen at windriver.com>
Sent: Tuesday, October 19, 2021 12:36 PM
To: starlingx-discuss at lists.starlingx.io
Subject: Re: [Starlingx-discuss] (2) How to Set 'isolcpus & HW Accelarator' on Starlingx 5.0
This is just a hunch, but I suspect that you shouldn't have the "http://"<https://urldefense.com/v3/__http:/*22__;JQ!!AjveYdw8EvQ!IEwfufuem722L0LUg_q8Rjy-BD9Zrwd7cZR3VY_bK3UmCsn0AzDwi5FSM-5936fQHpof0Q$> in front of the isolcpus resource.
If that doesn't fix it, could we get some information from your installation? For one of the nodes with isolcpus configured please run the following two commands on the active controller node:
system host-cpu-list <hostname>
kubectl describe node <hostname>
Then ssh to the node in question and run the following commands:
cat /proc/cmdline
cat /sys/devices/system/cpu/isolated
cat /sys/devices/system/cpu/nohz_full
Thanks,
Chris
On 10/18/2021 5:50 PM, 김태준 wrote:
Dear Ildiko Vancsa
Thank you for your reply.
First of all, we have already deployed our pod to the same server using WRCP. (with isolcpus & intel_acc100_fec)
The server is trying to query the same pod with Starlingx 5.0 while supporting the resources.
After 5.0 installation according to StarlingX Doc, cpu is set to Application-isolate in the same way, but the resources are not visible.
'intel.com/intel_acc100_fec<https://urldefense.com/v3/__http:/intel.com/intel_acc100_fec__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5AByxEsZ$>: ‘16’’
'http://windriver.com/isolcpus: 24’
Is there anything to do separately from starlingsX to make the above two resources available?
for Starlingx Installation, I did just below,
https://docs.starlingx.io/deploy_install_guides/r5_release/bare_metal/aio_simplex_install_kubernetes.html<https://urldefense.com/v3/__https:/docs.starlingx.io/deploy_install_guides/r5_release/bare_metal/aio_simplex_install_kubernetes.html__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5DS4Ev2i$>
for resources, I did belows,
https://protect2.fireeye.com/v1/url?k=67ad2fd7-383616cf-67aca498-0cc47a314e9a-5e0aef83dc32c5cb&q=1&e=5ed23c51-ae04-476f-a95d-f2d4d2a9c1cc&u=https%3A%2F%2Fdocs.starlingx.io%2Fnode_management%2Fkubernetes%2Fhardware_acceleration_devices%2Fset-up-pods-to-use-sriov.html<https://urldefense.com/v3/__https:/protect2.fireeye.com/v1/url?k=67ad2fd7-383616cf-67aca498-0cc47a314e9a-5e0aef83dc32c5cb&q=1&e=5ed23c51-ae04-476f-a95d-f2d4d2a9c1cc&u=https*3A*2F*2Fdocs.starlingx.io*2Fnode_management*2Fkubernetes*2Fhardware_acceleration_devices*2Fset-up-pods-to-use-sriov.html__;JSUlJSUlJQ!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5INhCM9q$>
https://protect2.fireeye.com/v1/url?k=33fc70ff-6c6749e7-33fdfbb0-0cc47a314e9a-bc617f111d97e7de&q=1&e=5ed23c51-ae04-476f-a95d-f2d4d2a9c1cc&u=https%3A%2F%2Fdocs.starlingx.io%2Fr%2Fstx.5.0%2Fusertasks%2Fkubernetes%2Fuse-application-isolated-cores.html<https://urldefense.com/v3/__https:/protect2.fireeye.com/v1/url?k=33fc70ff-6c6749e7-33fdfbb0-0cc47a314e9a-bc617f111d97e7de&q=1&e=5ed23c51-ae04-476f-a95d-f2d4d2a9c1cc&u=https*3A*2F*2Fdocs.starlingx.io*2Fr*2Fstx.5.0*2Fusertasks*2Fkubernetes*2Fuse-application-isolated-cores.html__;JSUlJSUlJSU!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5AH5-nFQ$>
When I check node description, there is no isolcpus and intel_acc100_fec.
Capacity:
cpu: 64
ephemeral-storage: 10190100Ki
hugepages-1Gi: 32Gi
hugepages-2Mi: 0
intel.com/pci_sriov_net_ca: <https://urldefense.com/v3/__http:/intel.com/pci_sriov_net_ca:__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5GbJZtdA$> 7
intel.com/pci_sriov_net_f1c: <https://urldefense.com/v3/__http:/intel.com/pci_sriov_net_f1c:__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5JvZTDva$> 4
intel.com/pci_sriov_net_f1u: <https://urldefense.com/v3/__http:/intel.com/pci_sriov_net_f1u:__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5Bi7xLEj$> 4
intel.com/pci_sriov_net_fh0: <https://urldefense.com/v3/__http:/intel.com/pci_sriov_net_fh0:__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5Bo5Ql8w$> 4
intel.com/pci_sriov_net_fh0m: <https://urldefense.com/v3/__http:/intel.com/pci_sriov_net_fh0m:__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5PXG9sZK$> 4
intel.com/pci_sriov_net_fh1: <https://urldefense.com/v3/__http:/intel.com/pci_sriov_net_fh1:__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5JNa4Zdc$> 3
intel.com/pci_sriov_net_fh1m: <https://urldefense.com/v3/__http:/intel.com/pci_sriov_net_fh1m:__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5JQSZRbg$> 4
memory: 131264360Ki
pods: 110
Same Server with WRCP case, we can see like this
Capacity:
cpu: 64
ephemeral-storage: 10190100Ki
hugepages-1Gi: 32Gi
hugepages-2Mi: 0
intel.com/intel_acc100_fec: <https://urldefense.com/v3/__http:/intel.com/intel_acc100_fec:__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5Oq_FC4d$> 1
intel.com/pci_sriov_net_ca: <https://urldefense.com/v3/__http:/intel.com/pci_sriov_net_ca:__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5GbJZtdA$> 7
intel.com/pci_sriov_net_f1c: <https://urldefense.com/v3/__http:/intel.com/pci_sriov_net_f1c:__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5JvZTDva$> 4
intel.com/pci_sriov_net_f1u: <https://urldefense.com/v3/__http:/intel.com/pci_sriov_net_f1u:__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5Bi7xLEj$> 4
intel.com/pci_sriov_net_fh0: <https://urldefense.com/v3/__http:/intel.com/pci_sriov_net_fh0:__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5Bo5Ql8w$> 4
intel.com/pci_sriov_net_fh0m: <https://urldefense.com/v3/__http:/intel.com/pci_sriov_net_fh0m:__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5PXG9sZK$> 4
intel.com/pci_sriov_net_fh1: <https://urldefense.com/v3/__http:/intel.com/pci_sriov_net_fh1:__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5JNa4Zdc$> 3
intel.com/pci_sriov_net_fh1m: <https://urldefense.com/v3/__http:/intel.com/pci_sriov_net_fh1m:__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5JQSZRbg$> 4
memory: 131365668Ki
pods: 110
http://windriver.com/isolcpus: 54
Please let me know if there is anything I can do.
Thanks and Best Regards,
Taejun
--------- Original Message ---------
Sender : Ildiko Vancsa <ildiko.vancsa at gmail.com><mailto:ildiko.vancsa at gmail.com>
Date : 2021-10-19 03:56 (GMT+9)
Title : Re: [Starlingx-discuss] How to Set 'isolcpus & HW Accelarator' on Starlingx 5.0
Dear Kim Tae-jun,
Thank you for reaching out with your question.
I think you may need to modify the yaml file to adjust it to the resources available in your system. Which means to modify these entries:
'intel.com/intel_acc100_fec<https://urldefense.com/v3/__http:/intel.com/intel_acc100_fec__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5AByxEsZ$>: ‘16’’
'http://windriver.com/isolcpus: 24’
Do you have the required resources available in your environment to use this feature?
Thanks and Best Regards,
Ildikó
> On Oct 12, 2021, at 18:02, 김태준 <tj85.kim at samsung.com><mailto:tj85.kim at samsung.com> wrote:
>
> Dear starlingx members
>
> Hello. This is Kim Tae-jun from Samsung Electronics.
> We are trying a trial to apply Starlingx to our system.
>
> Among them, we are trying to set isolcpus and hw indicator according to the manual in starlingx docs and modify pod.
> However, even if you create yaml as manual, you can not run properly because you do not have the following resources.
>
> Type Reason Age From Message
> ---- ------ ---- ---- -------
> Warning FailedScheduling 6s (x5 over 15s) default-scheduler 0/1 nodes are available: 1 Insufficient intel.com/intel_acc100_fec<https://urldefense.com/v3/__http:/intel.com/intel_acc100_fec__;!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5AByxEsZ$>, 1 Insufficient http://windriver.com/isolcpus.
>
> resources corresponding to the capacity of the node description are not visible.
>
> I set it by referring to the links below. I would appreciate it if you would let me know if there is anything missing in the preliminary work or what to refer to.
>
> https://protect2.fireeye.com/v1/url?k=67ad2fd7-383616cf-67aca498-0cc47a314e9a-5e0aef83dc32c5cb&q=1&e=5ed23c51-ae04-476f-a95d-f2d4d2a9c1cc&u=https%3A%2F%2Fdocs.starlingx.io%2Fnode_management%2Fkubernetes%2Fhardware_acceleration_devices%2Fset-up-pods-to-use-sriov.html<https://urldefense.com/v3/__https:/protect2.fireeye.com/v1/url?k=67ad2fd7-383616cf-67aca498-0cc47a314e9a-5e0aef83dc32c5cb&q=1&e=5ed23c51-ae04-476f-a95d-f2d4d2a9c1cc&u=https*3A*2F*2Fdocs.starlingx.io*2Fnode_management*2Fkubernetes*2Fhardware_acceleration_devices*2Fset-up-pods-to-use-sriov.html__;JSUlJSUlJQ!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5INhCM9q$>
> https://protect2.fireeye.com/v1/url?k=33fc70ff-6c6749e7-33fdfbb0-0cc47a314e9a-bc617f111d97e7de&q=1&e=5ed23c51-ae04-476f-a95d-f2d4d2a9c1cc&u=https%3A%2F%2Fdocs.starlingx.io%2Fr%2Fstx.5.0%2Fusertasks%2Fkubernetes%2Fuse-application-isolated-cores.html<https://urldefense.com/v3/__https:/protect2.fireeye.com/v1/url?k=33fc70ff-6c6749e7-33fdfbb0-0cc47a314e9a-bc617f111d97e7de&q=1&e=5ed23c51-ae04-476f-a95d-f2d4d2a9c1cc&u=https*3A*2F*2Fdocs.starlingx.io*2Fr*2Fstx.5.0*2Fusertasks*2Fkubernetes*2Fuse-application-isolated-cores.html__;JSUlJSUlJSU!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5AH5-nFQ$>
>
> Thanks you.
>
> <Mail Attachment.gif>
> _______________________________________________
> Starlingx-discuss mailing list
> Starlingx-discuss at lists.starlingx.io<mailto:Starlingx-discuss at lists.starlingx.io>
> https://protect2.fireeye.com/v1/url?k=dd734e99-82e87781-dd72c5d6-0cc47a314e9a-0b594a17d9dfec9d&q=1&e=5ed23c51-ae04-476f-a95d-f2d4d2a9c1cc&u=http%3A%2F%2Flists.starlingx.io%2Fcgi-bin%2Fmailman%2Flistinfo%2Fstarlingx-discuss<https://urldefense.com/v3/__https:/protect2.fireeye.com/v1/url?k=dd734e99-82e87781-dd72c5d6-0cc47a314e9a-0b594a17d9dfec9d&q=1&e=5ed23c51-ae04-476f-a95d-f2d4d2a9c1cc&u=http*3A*2F*2Flists.starlingx.io*2Fcgi-bin*2Fmailman*2Flistinfo*2Fstarlingx-discuss__;JSUlJSUlJQ!!AjveYdw8EvQ!LbGFy4YrzCXcWt9OOkCjHPZSnZwRperUYw6TFcc5oznrlxoJJPnafn9xB9ST5CCsQZo3$>
[cid:image001.gif at 01D7C504.196305E0]
[Image removed by sender.]
_______________________________________________
Starlingx-discuss mailing list
Starlingx-discuss at lists.starlingx.io<mailto:Starlingx-discuss at lists.starlingx.io>
http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss<https://urldefense.com/v3/__http:/lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss__;!!AjveYdw8EvQ!IEwfufuem722L0LUg_q8Rjy-BD9Zrwd7cZR3VY_bK3UmCsn0AzDwi5FSM-5936dFPxmjDw$>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20211019/9255ffc6/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ~WRD0000.jpg
Type: image/jpeg
Size: 823 bytes
Desc: ~WRD0000.jpg
URL: <http://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20211019/9255ffc6/attachment-0001.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.gif
Type: image/gif
Size: 13402 bytes
Desc: image001.gif
URL: <http://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20211019/9255ffc6/attachment-0001.gif>
More information about the Starlingx-discuss
mailing list