Re: [Starlingx-discuss] questions about '[Feature] Huge page management' https://storyboard.openstack.org/#!/story/2004763

2 May 2019

      Hi Tao/Austin,

Thanks a lot for the information you provide!

BRs,
Marvin

From: Sun, Austin [mailto:austin.sun@intel.com]
Sent: Thursday, May 02, 2019 1:24 AM
To: Liu, Tao; Huang, Marvin
Cc: Eslimi, Dariush; Waheed, Numan; Xie, Cindy; starlingx-discuss@lists.starlingx.io
Subject: RE: questions about '[Feature] Huge page management' https://storyboard.openstack.org/#!/story/2004763

Hi Marvin :
The simplest way to check huge page feature is 'kubectl describe node compute-1 | grep hugepage' if compute is enabled hugepage and has pre-allocated 2M hugepage.

If compute did not enable hugepage , then the command should return empty.

For pod configuration you can refer to [1], but I did not tried trigger pod with hugepage.

[1] https://stupefied-goodall-e282f7.netlify.com/contributors/design-proposals/r...

Thanks.
BR
Austin Sun.

From: Liu, Tao [mailto:Tao.Liu@windriver.com]
Sent: Thursday, May 2, 2019 8:48 AM
To: Huang, Marvin <Marvin.Huang@windriver.com>; Sun, Austin <austin.sun@intel.com>
Cc: Eslimi, Dariush <Dariush.Eslimi@windriver.com>; Waheed, Numan <Numan.Waheed@windriver.com>; Xie, Cindy <cindy.xie@intel.com>; starlingx-discuss@lists.starlingx.io
Subject: RE: questions about '[Feature] Huge page management' https://storyboard.openstack.org/#!/story/2004763

Hi Marvin,

See inline comments below for the questions with regards  to platform huge pages management,
I will let  Austin to address your queries related to k8s huge page feature testing.

Tao

From: Huang, Marvin
Sent: Wednesday, May 1, 2019 5:20 PM
To: Liu, Tao; Sun, Austin
Cc: Eslimi, Dariush; Waheed, Numan; Xie, Cindy; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>
Subject: RE: questions about '[Feature] Huge page management' https://storyboard.openstack.org/#!/story/2004763

Hi Tao,

Regarding to Huge Page management rework, there's a US https://storyboard.openstack.org/#!/story/2004763, which has the following are the requirements:
- Remove automatic allocation of the 1G huge pages from the kick starts for vswitch
- Enable k8s huge page feature for worker nodes that do not have the openstack compute label. It should be disabled otherwise.
- Automatically defaults for worker nodes with openstack compute label. Changes will be applied on the unlock.
    - Current 2M huge page default settings
    - 1-1G huge page per numa node for vswitch
- Do not allocate any huge pages by default for worker nodes without the openstack compute label. Up to the customer to configure.

Here're some questions, please comment on them:

REQUIRMENT 1:
- Remove automatic allocation of the 1G huge pages from the kick starts for vswitch
    Is that meaning that, by default, no 1G huge pages for vswitch, no 'openstack compute label', and 'k8s huge page feature' IS ENABLED?
TL> Yes,  1G per NUMA node for vswitch will be allocated when the vswitch type is set. 'k8s huge page feature' will be disabled when 'openstack compute label' is set.

REQUIRMENT 2:
- Enable k8s huge page feature for worker nodes that do not have the openstack compute label. It should be disabled otherwise.

    q1:  What is it meaning by 'Enable k8s huge page feature'? How to verify it?
TL>Perhaps Austin could share how he tested it.

        1   Will any commands show the feature is enabled/disabled?
                e.g. Will it be shown in the options apiserver been started with, i.e. in the outputs of command
                     'sudo ps aux | grep apiserver | grep feature-gates'?

            And/or will any configuration files indicating it's enabled/disabled?
                I found currently, /var/lib/kubelet/config.yaml contains lines having 'featureGates'

    q2:  Will adding/removing the label 'openstack-compute-node' after initial provision/installation, enable/disable the 'k8s huge page feature'?
            And those the states of enabled/disabled can be verified using the any methods for q1.
TL> It should (after the host is unlocked).

REQUIRMENT 3:
- Automatically defaults for worker nodes with openstack compute label. Changes will be applied on the unlock.
    - Current 2M huge page default settings
    - 1-1G huge page per numa node for vswitch

- Do not allocate any huge pages by default for worker nodes without the openstack compute label. Up to the customer to configure.

    q1:  Does it mean if the label 'openstack-compute-node' is added to a node (hence 'k8s huge pages feature' is DISABLED),
after been unlocked (it must be locked in order to add the label),  certain numbers of 2M-pages and 1 1G-page will be allocated on the node,
and can be verified with above mentioned methods?
TL> Only 2M huge pages will be allocated for applications by default, and they are shown in CLI and Horizon. The remaining available memory is calculated as 4K when the host is unlocked.
You can also cat /sys/devices/system/node/nodex/hugepages/hugepages-2048kB/nr_hugepages to see the allocation in Linux.
The users can configure 1G pages for applications via CLI/Horizon.

Thanks,
Marvin