[Starlingx-discuss] CPU Isolation discrepancy

[Venkata Ramana Veldanda]

The reason I bring out this concern is even dockerd seem to be running on isolated cores. For some reason the scheduler seem to be completely ignoring isolcpus. Is that correct?

The impact here is that we have certain 100% poll threads that needs to be scheduled on couple of CPUs and we do not have a clue on where to allocate because of this behaviour. Any comments are appreciated.

[cid:image001.jpg at 01D7114F.2BCF1840]

From: Venkata Ramana Veldanda
Sent: Thursday, March 4, 2021 7:21 PM
To: starlingx-discuss at lists.starlingx.io
Cc: Srinivas Sadagopan <Srinivas.Sadagopan at radisys.com>
Subject: CPU Isolation discrepancy


Hi

I am running StarlingX 4.0 in Low Latency Mode in AIO-SX. We have isolated CPUs for application as below using the supported way. However, we can see that kubelet treads are scheduled on application isolated vcpus - Ex: 49, 50, 75.



Could someone explain the behavior here?. Full log is attached.


controller-0:/home/sysadmin# cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-4.18.0-147.3.1.rt24.96.el8_1.tis.8.x86_64 root=UUID=837a0268-a574-45ea-b560-a6e5afc483dd ro security_profile=standard module_blacklist=integrity,ima audit=0 tboot=false crashkernel=auto biosdevname=0 console=tty0 iommu=pt usbcore.autosuspend=-1 selinux=0 enforcing=0 nmi_watchdog=0 softlockup_panic=0 intel_iommu=on user_namespace.enable=1 skew_tick=1 hugepagesz=1G hugepages=47 hugepagesz=2M hugepages=0 default_hugepagesz=1G irqaffinity=0,24,48,72 rcu_nocbs=1-23,25-47,49-71,73-95 nohz_full=1-23,25-47,49-71,73-95 isolcpus=1-22,25-46,49-70,73-94 kthread_cpus=0,24,48,72 nopti nospectre_v2 nospectre_v1


[root at controller-0 sysadmin(keystone_admin)]# pidstat -t -p 134731
Linux 4.18.0-147.3.1.rt24.96.el8_1.tis.8.x86_64 (controller-0) 03/03/2021 _x86_64_(96 CPU)




01:26:28 AM     0         -    145642    0.00    0.00    0.00    0.00    50  |__kubelet
01:26:28 AM     0         -    145643    0.01    0.01    0.00    0.02    72  |__kubelet
01:26:28 AM     0         -    146196    0.05    0.03    0.00    0.08    24  |__kubelet
01:26:28 AM     0         -    147027    0.00    0.00    0.00    0.00    75  |__kubelet
01:26:28 AM     0         -    147028    0.00    0.00    0.00    0.00    49  |__kubelet





system host-cpu-list controller-0


| 456f11f6-231f-4c6a-b9d9-fa35a6f6bff6 | 49    | 0         | 1     | 1      | Intel(R) Xeon(R) Gold 6248R CPU | Application-isolated |
|                                      |       |           |       |        | @ 3.00GHz                       |                      |
|                                      |       |           |       |        |                                 |                      |
| d491dd29-8603-4bf1-8620-9fbdde5d5d38 | 50    | 0         | 2     | 1      | Intel(R) Xeon(R) Gold 6248R CPU | Application-isolated |



| 94bb8566-e726-418a-ae1f-3264807bfbcd | 75    | 1         | 3     | 1      | Intel(R) Xeon(R) Gold 6248R CPU | Application-isolated |





Venkata Veldanda

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20210304/8b12ea21/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 24047 bytes
Desc: image001.jpg
URL: <http://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20210304/8b12ea21/attachment-0001.jpg>