[Starlingx-discuss] k8s pods cant use application-isolated cpus
Chris Friesen
chris.friesen at windriver.com
Wed Feb 24 17:47:58 UTC 2021
Upstream Kubernetes doesn't really know how to handle "isolated" CPUs
separately from "normal" CPUs. Because of this sysinv will reserve the
"application-isolated" CPUs from use by Kubernetes when using the
"static" CPU Manager policy.
If you set the kube-cpu-mgr-policyhost label to "none" (via Horizon or
the "sysinv" CLI) Kubernetes will use the default policy and should
include the "application-isolated" CPUs as part of the "Allocatable" cpu
count. You'll have to handle CPU affinity of containerized processes
explicitly though to ensure that they don't interfere with each other.
Commercial products based on StarlingX have support for isolated CPUs
with the "static" CPU Manager policy by modifying upstream Kubernetes.
Hope that helps,
Chris
On 2/18/2021 12:00 PM, Dharwadkar, Sriram wrote:
>
> Hi,
>
> We are using Distributed-StarlingX-4.0.
>
> Our use case requires pods to use isolated cpu’s, so executed the
>
> “system host-cpu-modify -f application-isolated -p0 20 controller-0”
> command to isolate the cpus.
>
> system host-cpu-list controller-1
>
> +--------------------------------------+----------+-----------+----------+--------+-------------------------------------------+----------------------+
>
> | uuid | log_core | processor |
> phy_core | thread | processor_model |
> assigned_function |
>
> +--------------------------------------+----------+-----------+----------+--------+-------------------------------------------+----------------------+
>
> | e64e0004-3096-40c7-804c-d9c79d2ceb17 | 0 | 0 |
> 0 | 0 | Intel(R) Xeon(R) Gold 6212U CPU @ 2.40GHz |
> Platform |
>
> | 28697ba9-c5e5-4d9b-9d97-346ec61c1fa7 | 1 | 0 |
> 1 | 0 | Intel(R) Xeon(R) Gold 6212U CPU @ 2.40GHz |
> Platform |
>
> | 1b111e2e-84e4-4414-870f-73c759ecbac0 | 2 | 0 |
> 2 | 0 | Intel(R) Xeon(R) Gold 6212U CPU @ 2.40GHz |
> Application-isolated |
>
> | 66933896-5c9f-4e0a-9527-72025172e1f6 | 3 | 0 |
> 3 | 0 | Intel(R) Xeon(R) Gold 6212U CPU @ 2.40GHz |
> Application-isolated |
>
> | 027258c8-392e-4497-9e46-6acb007e759f | 4 | 0 |
> 4 | 0 | Intel(R) Xeon(R) Gold 6212U CPU @ 2.40GHz |
> Application-isolated |
>
> | 5d88a23a-68de-4062-9313-173fd91a4bc5 | 5 | 0 |
> 5 | 0 | Intel(R) Xeon(R) Gold 6212U CPU @ 2.40GHz |
> Application-isolated |
>
> | dd804bbb-191c-4499-bda1-272f9bb923af | 6 | 0 |
> 6 | 0 | Intel(R) Xeon(R) Gold 6212U CPU @ 2.40GHz |
> Application-isolated |
>
> | 7ae2a90b-59ca-4af0-b20e-20635af0aceb | 7 | 0 |
> 8 | 0 | Intel(R) Xeon(R) Gold 6212U CPU @ 2.40GHz |
> Application-isolated |
>
> | ada68f62-4d06-43eb-8ed9-cd6ec6d33487 | 8 | 0 |
> 9 | 0 | Intel(R) Xeon(R) Gold 6212U CPU @ 2.40GHz |
> Application-isolated |
>
> | 8c2612fd-8a7f-4732-82ff-fb4717ab9275 | 9 | 0 |
> 10 | 0 | Intel(R) Xeon(R) Gold 6212U CPU @ 2.40GHz |
> Application-isolated |
>
> | 01412e4d-c9ff-41e8-8fd3-c2a0d27ac7a3 | 10 | 0 |
> 11 | 0 | Intel(R) Xeon(R) Gold 6212U CPU @ 2.40GHz |
> Application-isolated |
>
> | c8e8b15b-538b-422e-9d72-2ba96982054a | 11 | 0 |
> 12 | 0 | Intel(R) Xeon(R) Gold 6212U CPU @ 2.40GHz |
> Application-isolated |
>
> | b2efd166-3125-46b9-95ac-4fb00a0de3c7 | 12 | 0 |
> 13 | 0 | Intel(R) Xeon(R) Gold 6212U CPU @ 2.40GHz |
> Application-isolated |
>
> | c33f3dd7-0b06-40c3-93b3-82f065c6db05 | 13 | 0 |
> 16 | 0 | Intel(R) Xeon(R) Gold 6212U CPU @ 2.40GHz |
> Application-isolated |
>
> | e33cda3e-7162-45d4-af79-4dff7f0bb07f | 14 | 0 |
> 17 | 0 | Intel(R) Xeon(R) Gold 6212U CPU @ 2.40GHz |
> Application-isolated |
>
> | 551b7272-371e-4c05-b5c8-7708f95d3a97 | 15 | 0 |
> 18 | 0 | Intel(R) Xeon(R) Gold 6212U CPU @ 2.40GHz |
> Application-isolated |
>
> | d0522e8f-ebd4-49fd-8a0e-26de857f2caf | 16 | 0 |
> 19 | 0 | Intel(R) Xeon(R) Gold 6212U CPU @ 2.40GHz |
> Application-isolated |
>
> | 84dd71f2-ad7a-46bb-8ae2-e62b2a817cc5 | 17 | 0 |
> 20 | 0 | Intel(R) Xeon(R) Gold 6212U CPU @ 2.40GHz |
> Application-isolated |
>
> | f4761605-3208-4758-b31c-d679cb0b631e | 18 | 0 |
> 21 | 0 | Intel(R) Xeon(R) Gold 6212U CPU @ 2.40GHz |
> Application-isolated |
>
> | 66423b6f-134b-4166-bdff-9643d4ec8d4d | 19 | 0 |
> 25 | 0 | Intel(R) Xeon(R) Gold 6212U CPU @ 2.40GHz |
> Application-isolated |
>
> | 45d38571-ca97-4b89-b442-fe5f5282bc5f | 20 | 0 |
> 26 | 0 | Intel(R) Xeon(R) Gold 6212U CPU @ 2.40GHz |
> Application-isolated |
>
> | fc87bbd5-2954-4730-9f67-b8fe8820faa7 | 21 | 0 |
> 27 | 0 | Intel(R) Xeon(R) Gold 6212U CPU @ 2.40GHz |
> Application-isolated |
>
> | 6b3a8a78-d191-47fa-af82-15166a7924fc | 22 | 0 |
> 28 | 0 | Intel(R) Xeon(R) Gold 6212U CPU @ 2.40GHz |
> Application |
>
> | 57739379-ab8f-4d3d-847a-f5aa9663bcfb | 23 | 0 |
> 29 | 0 | Intel(R) Xeon(R) Gold 6212U CPU @ 2.40GHz |
> Application |
>
> +--------------------------------------+----------+-----------+----------+--------+-------------------------------------------+----------------------+
>
> But the k8s is not able to allocate isolated cpus to pods. Allocatable
> cpus are only 2.
>
> controller-0:/home/sysadmin# kubectl get node controller-0 -o json |
> jq '.status.allocatable'
>
> {
>
> * "cpu": "2",*
>
> "ephemeral-storage": "9391196145",
>
> "hugepages-1Gi": "20Gi",
>
> "hugepages-2Mi": "0",
>
> "intel.com/pci_sriov_net_physnet0": "32",
>
> "memory": "70207428Ki",
>
> "pods": "110"
>
> }
>
> I saw this bug report, which seems to be talking about the same issue.
>
> https://bugs.launchpad.net/starlingx/+bug/1894173
> <https://bugs.launchpad.net/starlingx/+bug/1894173>
>
> Kindly let me know when the fix is planned.
>
> If there is any work around to solve this issue, do let me know.
>
> Regards,
>
> Sirram
>
>
> _______________________________________________
> Starlingx-discuss mailing list
> Starlingx-discuss at lists.starlingx.io
> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20210224/45ce13ae/attachment-0001.html>
More information about the Starlingx-discuss
mailing list