[Starlingx-discuss] CPU Isolation over AIO Controller Nodes
Gaur, Shubham
Shubham.Gaur at commscope.com
Mon Oct 25 08:52:25 UTC 2021
Hi,
After a clean install no isolated cpu are listed under /sys/devices/system/cpu/isolated.
controller-0:~$ cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-4.18.0-147.3.1.rt24.96.el8_1.tis.10.x86_64 root=UUID=da02d506-3eaa-410e-a5d0-6ed82611d77f ro security_profile=standard module_blacklist=integrity,ima audit=0 tboot=false crashkernel=auto biosdevname=0 console=tty0 iommu=pt usbcore.autosuspend=-1 selinux=0 enforcing=0 nmi_watchdog=0 softlockup_panic=0 softdog.soft_panic=1 intel_iommu=on user_namespace.enable=1 skew_tick=1 nopti nospectre_v2 nospectre_v1 hugepagesz=1G hugepages=20 hugepagesz=2M hugepages=0 default_hugepagesz=1G irqaffinity=0,2,4,6 rcu_nocbs=1,3,5,7-23 nohz_full=1,3,5,7-23 isolcpus=1,3,5,7-12,14 kthread_cpus=0,2,4,6
After further debug we found keeping rcu_nocbs, nohz_full equal to isolcpu resolved this problem. Looking at the scripts those parameters are calculated separately.
* /usr/lib64/python2.7/site-packages/sysinv/puppet/platform.py
582,583c582,583
< rcu_nocbs_cpuset = host_cpuset - platform_cpuset
< rcu_nocbs_ranges = utils.format_range_set(rcu_nocbs_cpuset)
---
> # rcu_nocbs_cpuset = host_cpuset - platform_cpuset
> # rcu_nocbs_ranges = utils.format_range_set(rcu_nocbs_cpuset)
589a590,593
> # non-platform logical cpus
> rcu_nocbs_cpuset = host_cpuset - platform_cpuset
> #rcu_nocbs_ranges = utils.format_range_set(rcu_nocbs_cpuset)
> rcu_nocbs_ranges = utils.format_range_set(vswitch_cpuset.union(app_isolated_cpuset))
Is there any specific reason these parameters are calculated separately? Are there any side effects if the following modification has being introduced in platform.py?
Thanks and regards,
Shubham
________________________________
From: Gaur, Shubham <Shubham.Gaur at commscope.com>
Sent: Wednesday, October 20, 2021 11:05 AM
To: starlingx-discuss at lists.starlingx.io <starlingx-discuss at lists.starlingx.io>
Subject: Re: CPU Isolation over AIO Controller Nodes
Hi All,
CPU isolation is not working in starlingx 5.0. Static CPU manager policy has been enabled on all the nodes and the Isol-CPU resource plugin is up &running but there is no isolated CPU resource pool visible. Could not see any isolated CPU annotations (windriver.com/isolcpus CPU annotations). Is there any additional step to get this working?
[sysadmin at controller-0 ~(keystone_admin)]$ system host-cpu-list controller-0 | grep Application-
| bb7e63db-aadd-4ac1-b85e-6d8784ca498b | 2 | 0 | 7 | 0 | Intel(R) Xeon(R) Silver 4216 CPU @ 2.10GHz | Application-isolated
| 94f9f44e-255b-4e48-83bf-bf1480a290f5 | 6 | 0 | 6 | 0 | Intel(R) Xeon(R) Silver 4216 CPU @ 2.10GHz | Application-isolated
| 51eb3110-a097-4b13-a6c3-7b40129c4859 | 10 | 0 | 5 | 0 | Intel(R) Xeon(R) Silver 4216 CPU @ 2.10GHz | Application-isolated
| 7073bc5a-7f43-41d8-bed4-6c0eef448c9e | 14 | 0 | 4 | 0 | Intel(R) Xeon(R) Silver 4216 CPU @ 2.10GHz | Application-isolated
| 609c1870-f182-439a-a0d2-6c56291c107d | 34 | 0 | 7 | 1 | Intel(R) Xeon(R) Silver 4216 CPU @ 2.10GHz | Application-isolated
| 47208322-4a80-416d-8fd8-9427f564345f | 38 | 0 | 6 | 1 | Intel(R) Xeon(R) Silver 4216 CPU @ 2.10GHz | Application-isolated
| 79da3786-aa34-4021-b5cc-be21c60c53ab | 42 | 0 | 5 | 1 | Intel(R) Xeon(R) Silver 4216 CPU @ 2.10GHz | Application-isolated
| 42d5a6d0-fb83-4d98-b884-698ba3e22818 | 46 | 0 | 4 | 1 | Intel(R) Xeon(R) Silver 4216 CPU @ 2.10GHz | Application-isolated
---------------------------------------------------------------------------------------------------------------------------------------
controller-0:~$ kubectl get node controller-0 -o yaml | grep -E " allocatable:" -A 15
allocatable:
cpu: "64"
ephemeral-storage: "9391196145"
hugepages-1Gi: "0"
hugepages-2Mi: "0"
memory: 106113496Ki
pods: "110"
capacity:
cpu: "64"
ephemeral-storage: 10190100Ki
hugepages-1Gi: "0"
hugepages-2Mi: "0"
memory: 131303896Ki
pods: "110"
====================================================================
SYSTEM: edgecloud
====================================================================
controller-0:~$ cat /etc/build.info
###
### StarlingX
### Release 21.05
###
OS="centos"
SW_VERSION="21.05"
BUILD_TARGET="Host Installer"
BUILD_TYPE="Formal"
BUILD_ID="r/stx.5.0"
JOB="STX_5.0_build_layer_flock"
BUILD_BY="starlingx.build at cengn.ca"
BUILD_NUMBER="37"
BUILD_HOST="starlingx_mirror"
BUILD_DATE="2021-05-21 23:03:55 +0000"
FLOCK_OS="centos"
FLOCK_JOB="STX_5.0_build_layer_flock"
FLOCK_BUILD_BY="starlingx.build at cengn.ca"
FLOCK_BUILD_NUMBER="37"
FLOCK_BUILD_HOST="starlingx_mirror"
FLOCK_BUILD_DATE="2021-05-21 23:03:55 +0000"
DISTRO_OS="centos"
DISTRO_JOB="STX_5.0_build_layer_distro"
DISTRO_BUILD_BY="starlingx.build at cengn.ca"
DISTRO_BUILD_NUMBER="35"
DISTRO_BUILD_HOST="starlingx_mirror"
DISTRO_BUILD_DATE="2021-05-18 23:02:22 +0000"
COMPILER_OS="centos"
COMPILER_JOB="STX_5.0_build_layer_compiler"
COMPILER_BUILD_BY="starlingx.build at cengn.ca"
COMPILER_BUILD_NUMBER="35"
COMPILER_BUILD_HOST="starlingx_mirror"
COMPILER_BUILD_DATE="2021-05-14 19:53:00 +0000"
Regards,
Shubham Gaur
________________________________
From: Gaur, Shubham
Sent: Friday, October 8, 2021 2:07 PM
To: starlingx-discuss at lists.starlingx.io <starlingx-discuss at lists.starlingx.io>
Subject: CPU Isolation over AIO Controller Nodes
Does the CPU isolation feature work over AIO controller nodes in an edge distributed cloud environment?
====================================================================
SYSTEM: edgecloud
====================================================================
controller-0:~$ cat /etc/build.info
###
### StarlingX
### Release 21.05
###
OS="centos"
SW_VERSION="21.05"
BUILD_TARGET="Host Installer"
BUILD_TYPE="Formal"
BUILD_ID="r/stx.5.0"
JOB="STX_5.0_build_layer_flock"
BUILD_BY="starlingx.build at cengn.ca"
BUILD_NUMBER="37"
BUILD_HOST="starlingx_mirror"
BUILD_DATE="2021-05-21 23:03:55 +0000"
FLOCK_OS="centos"
FLOCK_JOB="STX_5.0_build_layer_flock"
FLOCK_BUILD_BY="starlingx.build at cengn.ca"
FLOCK_BUILD_NUMBER="37"
FLOCK_BUILD_HOST="starlingx_mirror"
FLOCK_BUILD_DATE="2021-05-21 23:03:55 +0000"
DISTRO_OS="centos"
DISTRO_JOB="STX_5.0_build_layer_distro"
DISTRO_BUILD_BY="starlingx.build at cengn.ca"
DISTRO_BUILD_NUMBER="35"
DISTRO_BUILD_HOST="starlingx_mirror"
DISTRO_BUILD_DATE="2021-05-18 23:02:22 +0000"
COMPILER_OS="centos"
COMPILER_JOB="STX_5.0_build_layer_compiler"
COMPILER_BUILD_BY="starlingx.build at cengn.ca"
COMPILER_BUILD_NUMBER="35"
COMPILER_BUILD_HOST="starlingx_mirror"
COMPILER_BUILD_DATE="2021-05-14 19:53:00 +0000"
Regards,
Shubham Gaur
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20211025/9c4fd7be/attachment-0001.htm>
More information about the Starlingx-discuss
mailing list