[Starlingx-discuss] Question - Using 1Gb mem pages
Hello, I'm trying to execute a test, which his goal is to made use of huge pages (2M / 1G) launching a VM. I have already verified the configuration of the controllers GRUB options and we have defined the variable hugepagesz=1G, however, inside the same parameters list, there is a "default_hugepagesz=2M". The default hugepagesz, has been modified to 1G, and the compute has been restarted with no issues. The flavor has been modified, with the following option: " hw:mem_page_size=104875" (1G) page. However when I launch a VM using the modified flavor it creates the VM, but if I check the field values of the VM using " openstack server show <vm_name>". I'm getting the following value for pgsize: wrs-res:topology | node:0, 512MB, pgsize:2M, vcpus:0, pol:sha | So, seems like it's still taking the 2M pagesize instead of the 1G. What I have already tried is to: * Change also the image metadata: hw:mem_page_size=large and also 104875. Result, VM launched successfully, but pgsize value still 2M. * Leave the GRUB default values, and use modified flavor to hw:mem_page_size=104875, launch the VM with ERROR status. * Leave the GRUB default values, and use modified flavor to hw:mem_page_size=2048, launch the VM with success, , pgsize value is 2M, which is expected, ok. * Leave the GRUB default values, and use modified flavor to hw:mem_page_size=small launch the VM with success, , pgsize value is 4k, which is expected, ok. * Leave the GRUB default values, and use modified flavor to hw:mem_page_size=any launch the VM with success, , pgsize value is 2M, which is expected, ok. Could be this an issue trying to use 1G mem pages or do I'm missing some other configuration parameter? Thanks in advance for your help
On 2018-10-19 1:45 p.m., Perez, Ricardo O wrote:
Hello,
I'm trying to execute a test, which his goal is to made use of huge pages (2M / 1G) launching a VM.
I have already verified the configuration of the controllers GRUB options and we have defined the variable hugepagesz=1G, however, inside the same parameters list, there is a "default_hugepagesz=2M". The default hugepagesz, has been modified to 1G, and the compute has been restarted with no issues.
Hi Ricardo, You wrote, "controllers GRUB options" then "the compute has been restarted with no issues". Is there a GRUB option one can configure on the controller that is for the computes? I don't remember seeing one. If you're actually changing /etc/default/grub, I doubt that's supported. I did not find a procedure for changing default_hugepagesz on controllers or computes grub configuration. It's supposed to be an option, for example through in horizon interface or 'system host-memory-modify'... but you didn't quote either of those.
The flavor has been modified, with the following option: " hw:mem_page_size=104875" (1G) page.
Is "104875" a typo? It looks a lot like 1024 * 1024 = 1048576, but not quite. M
However when I launch a VM using the modified flavor it creates the VM, but if I check the field values of the VM using " openstack server show <vm_name>". I'm getting the following value for pgsize: wrs-res:topology | node:0, 512MB, pgsize:2M, vcpus:0, pol:sha |
So, seems like it's still taking the 2M pagesize instead of the 1G.
What I have already tried is to:
* Change also the image metadata: hw:mem_page_size=large and also 104875. Result, VM launched successfully, but pgsize value still 2M.
There's the interesting "104875" again.
* Leave the GRUB default values, and use modified flavor to hw:mem_page_size=104875, launch the VM with ERROR status.
104875
* Leave the GRUB default values, and use modified flavor to hw:mem_page_size=2048, launch the VM with success, , pgsize value is 2M, which is expected, ok.
* Leave the GRUB default values, and use modified flavor to hw:mem_page_size=small launch the VM with success, , pgsize value is 4k, which is expected, ok.
* Leave the GRUB default values, and use modified flavor to hw:mem_page_size=any launch the VM with success, , pgsize value is 2M, which is expected, ok.
Could be this an issue trying to use 1G mem pages or do I'm missing some other configuration parameter?
Thanks in advance for your help
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Hi Michael, Thanks for your answer, I just forgot to put the computes information about memory configuration, here is the info: Compute-0 VM Pages 0 Reserved for Platform: 8000 MiB Usable Total: 86387 MiB Available: 84787 MiB 4K Pages: Total: 230656 2M Hugepages: Total: 1271 Available: 983 1G Hugepages: Total: 80 Available: 80 1 Reserved for Platform: 2000 MiB Usable Total: 93698 MiB Available: 92674 MiB 4K Pages: Total: 29696 2M Hugepages: Total: 5319 Available: 5319 1G Hugepages: Total: 80 Available: 80 *********************************** Compute-1 Processor Memory VM Pages 0 Reserved for Platform: 8000 MiB Usable Total: 86504 MiB Available: 84968 MiB 4K Pages: Total: 0 2M Hugepages: Total: 1780 Available: 1524 1G Hugepages: Total: 80 Available: 80 1 Reserved for Platform: 2000 MiB Usable Total: 93552 MiB Available: 92528 MiB 4K Pages: Total: 0 2M Hugepages: Total: 5304 Available: 5304 1G Hugepages: Total: 80 Available: 80 Am I missing something else to made use of 1G memory pages ? Thanks -Richo
-----Original Message----- From: Michel Thebeau [mailto:michel.thebeau@windriver.com] Sent: Friday, October 19, 2018 1:39 PM To: Perez, Ricardo O <ricardo.o.perez@intel.com>; starlingx- discuss@lists.starlingx.io Subject: Re: [Starlingx-discuss] Question - Using 1Gb mem pages
Hello,
I'm trying to execute a test, which his goal is to made use of huge pages (2M / 1G) launching a VM.
I have already verified the configuration of the controllers GRUB options and we have defined the variable hugepagesz=1G, however, inside the same
On 2018-10-19 1:45 p.m., Perez, Ricardo O wrote: parameters list, there is a "default_hugepagesz=2M". The default hugepagesz, has been modified to 1G, and the compute has been restarted with no issues.
Hi Ricardo,
You wrote, "controllers GRUB options" then "the compute has been restarted with no issues". Is there a GRUB option one can configure on the controller that is for the computes? I don't remember seeing one. If you're actually changing /etc/default/grub, I doubt that's supported. I did not find a procedure for changing default_hugepagesz on controllers or computes grub configuration.
It's supposed to be an option, for example through in horizon interface or 'system host-memory-modify'... but you didn't quote either of those.
The flavor has been modified, with the following option: "
hw:mem_page_size=104875" (1G) page.
Is "104875" a typo? It looks a lot like 1024 * 1024 = 1048576, but not quite.
M
However when I launch a VM using the modified flavor it creates the VM, but if
wrs-res:topology | node:0, 512MB, pgsize:2M, vcpus:0, pol:sha |
So, seems like it's still taking the 2M pagesize instead of the 1G.
What I have already tried is to:
* Change also the image metadata: hw:mem_page_size=large and also
I check the field values of the VM using " openstack server show <vm_name>". I'm getting the following value for pgsize: 104875. Result, VM launched successfully, but pgsize value still 2M.
There's the interesting "104875" again.
* Leave the GRUB default values, and use modified flavor to hw:mem_page_size=104875, launch the VM with ERROR status.
104875
* Leave the GRUB default values, and use modified flavor to hw:mem_page_size=2048, launch the VM with success, , pgsize value is 2M, which is expected, ok.
* Leave the GRUB default values, and use modified flavor to hw:mem_page_size=small launch the VM with success, , pgsize value is 4k, which is expected, ok.
* Leave the GRUB default values, and use modified flavor to hw:mem_page_size=any launch the VM with success, , pgsize value is 2M, which is expected, ok.
Could be this an issue trying to use 1G mem pages or do I'm missing some other configuration parameter?
Thanks in advance for your help
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Hi team, any idea about what is missing here ? Thanks _Richo
-----Original Message----- From: Perez, Ricardo O [mailto:ricardo.o.perez@intel.com] Sent: Friday, October 19, 2018 5:14 PM To: Michel Thebeau <michel.thebeau@windriver.com>; starlingx- discuss@lists.starlingx.io Subject: Re: [Starlingx-discuss] Question - Using 1Gb mem pages
Hi Michael,
Thanks for your answer, I just forgot to put the computes information about memory configuration, here is the info:
Compute-0
VM Pages 0 Reserved for Platform: 8000 MiB Usable Total: 86387 MiB Available: 84787 MiB 4K Pages: Total: 230656 2M Hugepages: Total: 1271 Available: 983 1G Hugepages: Total: 80 Available: 80
1 Reserved for Platform: 2000 MiB Usable Total: 93698 MiB Available: 92674 MiB 4K Pages: Total: 29696 2M Hugepages: Total: 5319 Available: 5319 1G Hugepages: Total: 80 Available: 80
*********************************** Compute-1 Processor Memory VM Pages 0 Reserved for Platform: 8000 MiB Usable Total: 86504 MiB Available: 84968 MiB 4K Pages: Total: 0 2M Hugepages: Total: 1780 Available: 1524 1G Hugepages: Total: 80 Available: 80
1 Reserved for Platform: 2000 MiB Usable Total: 93552 MiB Available: 92528 MiB 4K Pages: Total: 0 2M Hugepages: Total: 5304 Available: 5304 1G Hugepages: Total: 80 Available: 80
Am I missing something else to made use of 1G memory pages ?
Thanks -Richo
-----Original Message----- From: Michel Thebeau [mailto:michel.thebeau@windriver.com] Sent: Friday, October 19, 2018 1:39 PM To: Perez, Ricardo O <ricardo.o.perez@intel.com>; starlingx- discuss@lists.starlingx.io Subject: Re: [Starlingx-discuss] Question - Using 1Gb mem pages
Hello,
I'm trying to execute a test, which his goal is to made use of huge pages (2M / 1G) launching a VM.
I have already verified the configuration of the controllers GRUB options and we have defined the variable hugepagesz=1G, however, inside the same
On 2018-10-19 1:45 p.m., Perez, Ricardo O wrote: parameters list, there is a "default_hugepagesz=2M". The default hugepagesz, has been modified to 1G, and the compute has been restarted with no issues.
Hi Ricardo,
You wrote, "controllers GRUB options" then "the compute has been restarted with no issues". Is there a GRUB option one can configure on the controller that is for the computes? I don't remember seeing one. If you're actually changing /etc/default/grub, I doubt that's supported. I did not find a procedure for changing default_hugepagesz on controllers or computes grub configuration.
It's supposed to be an option, for example through in horizon interface or 'system host-memory-modify'... but you didn't quote either of those.
The flavor has been modified, with the following option: "
hw:mem_page_size=104875" (1G) page.
Is "104875" a typo? It looks a lot like 1024 * 1024 = 1048576, but not quite.
M
However when I launch a VM using the modified flavor it creates the VM, but if
wrs-res:topology | node:0, 512MB, pgsize:2M, vcpus:0, pol:sha |
So, seems like it's still taking the 2M pagesize instead of the 1G.
What I have already tried is to:
* Change also the image metadata: hw:mem_page_size=large and also
I check the field values of the VM using " openstack server show <vm_name>". I'm getting the following value for pgsize: 104875. Result, VM launched successfully, but pgsize value still 2M.
There's the interesting "104875" again.
* Leave the GRUB default values, and use modified flavor to hw:mem_page_size=104875, launch the VM with ERROR status.
104875
* Leave the GRUB default values, and use modified flavor to hw:mem_page_size=2048, launch the VM with success, , pgsize value is 2M, which is expected, ok.
* Leave the GRUB default values, and use modified flavor to hw:mem_page_size=small launch the VM with success, , pgsize value is 4k, which is expected, ok.
* Leave the GRUB default values, and use modified flavor to hw:mem_page_size=any launch the VM with success, , pgsize value is 2M, which is expected, ok.
Could be this an issue trying to use 1G mem pages or do I'm missing some other configuration parameter?
Thanks in advance for your help
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Hi Ricardo, The documentation indicates to configure the the huge pages per numa node on the compute, and to configure the flavour extra specs for the VM. I am not aware of anything else in particular. There's some tips and caveats in the document, but I don't see anything in your message to think of those. If there's any chance you are trying to do this under virtualization, I am not aware that that is currently supported. M On 2018-10-19 6:14 p.m., Perez, Ricardo O wrote: > Hi Michael, > > Thanks for your answer, I just forgot to put the computes information about memory configuration, here is the info: > > Compute-0 > > VM Pages > 0 Reserved for Platform: 8000 MiB > Usable Total: 86387 MiB > Available: 84787 MiB > 4K Pages: > Total: 230656 > 2M Hugepages: > Total: 1271 > Available: 983 > 1G Hugepages: > Total: 80 > Available: 80 > > 1 Reserved for Platform: 2000 MiB > Usable Total: 93698 MiB > Available: 92674 MiB > 4K Pages: > Total: 29696 > 2M Hugepages: > Total: 5319 > Available: 5319 > 1G Hugepages: > Total: 80 > Available: 80 > > *********************************** > Compute-1 > Processor > Memory > VM Pages > 0 Reserved for Platform: 8000 MiB > Usable Total: 86504 MiB > Available: 84968 MiB > 4K Pages: > Total: 0 > 2M Hugepages: > Total: 1780 > Available: 1524 > 1G Hugepages: > Total: 80 > Available: 80 > > 1 Reserved for Platform: 2000 MiB > Usable Total: 93552 MiB > Available: 92528 MiB > 4K Pages: > Total: 0 > 2M Hugepages: > Total: 5304 > Available: 5304 > 1G Hugepages: > Total: 80 > Available: 80 > > Am I missing something else to made use of 1G memory pages ? > > Thanks > -Richo > >> -----Original Message----- >> From: Michel Thebeau [mailto:michel.thebeau@windriver.com] >> Sent: Friday, October 19, 2018 1:39 PM >> To: Perez, Ricardo O <ricardo.o.perez@intel.com>; starlingx- >> discuss@lists.starlingx.io >> Subject: Re: [Starlingx-discuss] Question - Using 1Gb mem pages >> >> On 2018-10-19 1:45 p.m., Perez, Ricardo O wrote: >>> Hello, >>> >>> I'm trying to execute a test, which his goal is to made use of huge pages (2M / >> 1G) launching a VM. >>> I have already verified the configuration of the controllers GRUB options and >> we have defined the variable hugepagesz=1G, however, inside the same >> parameters list, there is a "default_hugepagesz=2M". The default hugepagesz, >> has been modified to 1G, and the compute has been restarted with no issues. >> >> >> Hi Ricardo, >> >> You wrote, "controllers GRUB options" then "the compute has been restarted >> with no issues". Is there a GRUB option one can configure on the controller that >> is for the computes? I don't remember seeing one. If you're actually changing >> /etc/default/grub, I doubt that's supported. >> I did not find a procedure for changing default_hugepagesz on controllers or >> computes grub configuration. >> >> It's supposed to be an option, for example through in horizon interface or >> 'system host-memory-modify'... but you didn't quote either of those. >> >> >>> The flavor has been modified, with the following option: " >> hw:mem_page_size=104875" (1G) page. >> >> >> Is "104875" a typo? It looks a lot like 1024 * 1024 = 1048576, but not >> quite. >> >> >> M >> >> >>> However when I launch a VM using the modified flavor it creates the VM, but if >> I check the field values of the VM using " openstack server show <vm_name>". >> I'm getting the following value for pgsize: >>> wrs-res:topology | node:0, 512MB, pgsize:2M, vcpus:0, pol:sha | >>> >>> So, seems like it's still taking the 2M pagesize instead of the 1G. >>> >>> What I have already tried is to: >>> >>> * Change also the image metadata: hw:mem_page_size=large and also >> 104875. Result, VM launched successfully, but pgsize value still 2M. >> >> There's the interesting "104875" again. >> >> >>> * Leave the GRUB default values, and use modified flavor to >> hw:mem_page_size=104875, launch the VM with ERROR status. >> >> 104875 >> >>> * Leave the GRUB default values, and use modified flavor to >> hw:mem_page_size=2048, launch the VM with success, , pgsize value is 2M, >> which is expected, ok. >>> * Leave the GRUB default values, and use modified flavor to >> hw:mem_page_size=small launch the VM with success, , pgsize value is 4k, >> which is expected, ok. >>> * Leave the GRUB default values, and use modified flavor to >> hw:mem_page_size=any launch the VM with success, , pgsize value is 2M, which >> is expected, ok. >>> Could be this an issue trying to use 1G mem pages or do I'm missing some >> other configuration parameter? >>> Thanks in advance for your help >>> >>> >>> _______________________________________________ >>> Starlingx-discuss mailing list >>> Starlingx-discuss@lists.starlingx.io >>> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Hi Ricardo, I was able to select 4K and 2M pages in the following way (attached txt file). I'll try to see if I can get a system up with 1G pages, and I'll follow up with you. M On 2018-10-22 10:59 a.m., Michel Thebeau wrote: > Hi Ricardo, > > The documentation indicates to configure the the huge pages per numa > node on the compute, and to configure the flavour extra specs for the > VM. I am not aware of anything else in particular. There's some tips > and caveats in the document, but I don't see anything in your message > to think of those. > > If there's any chance you are trying to do this under virtualization, > I am not aware that that is currently supported. > > M > > > On 2018-10-19 6:14 p.m., Perez, Ricardo O wrote: >> Hi Michael, >> >> Thanks for your answer, I just forgot to put the computes information >> about memory configuration, here is the info: >> >> Compute-0 >> >> VM Pages >> 0 Reserved for Platform: 8000 MiB >> Usable Total: 86387 MiB >> Available: 84787 MiB >> 4K Pages: >> Total: 230656 >> 2M Hugepages: >> Total: 1271 >> Available: 983 >> 1G Hugepages: >> Total: 80 >> Available: 80 >> >> 1 Reserved for Platform: 2000 MiB >> Usable Total: 93698 MiB >> Available: 92674 MiB >> 4K Pages: >> Total: 29696 >> 2M Hugepages: >> Total: 5319 >> Available: 5319 >> 1G Hugepages: >> Total: 80 >> Available: 80 >> >> *********************************** >> Compute-1 >> Processor >> Memory >> VM Pages >> 0 Reserved for Platform: 8000 MiB >> Usable Total: 86504 MiB >> Available: 84968 MiB >> 4K Pages: >> Total: 0 >> 2M Hugepages: >> Total: 1780 >> Available: 1524 >> 1G Hugepages: >> Total: 80 >> Available: 80 >> >> 1 Reserved for Platform: 2000 MiB >> Usable Total: 93552 MiB >> Available: 92528 MiB >> 4K Pages: >> Total: 0 >> 2M Hugepages: >> Total: 5304 >> Available: 5304 >> 1G Hugepages: >> Total: 80 >> Available: 80 >> >> Am I missing something else to made use of 1G memory pages ? >> >> Thanks >> -Richo >> >>> -----Original Message----- >>> From: Michel Thebeau [mailto:michel.thebeau@windriver.com] >>> Sent: Friday, October 19, 2018 1:39 PM >>> To: Perez, Ricardo O <ricardo.o.perez@intel.com>; starlingx- >>> discuss@lists.starlingx.io >>> Subject: Re: [Starlingx-discuss] Question - Using 1Gb mem pages >>> >>> On 2018-10-19 1:45 p.m., Perez, Ricardo O wrote: >>>> Hello, >>>> >>>> I'm trying to execute a test, which his goal is to made use of huge >>>> pages (2M / >>> 1G) launching a VM. >>>> I have already verified the configuration of the controllers GRUB >>>> options and >>> we have defined the variable hugepagesz=1G, however, inside the same >>> parameters list, there is a "default_hugepagesz=2M". The default >>> hugepagesz, >>> has been modified to 1G, and the compute has been restarted with no >>> issues. >>> >>> >>> Hi Ricardo, >>> >>> You wrote, "controllers GRUB options" then "the compute has been >>> restarted >>> with no issues". Is there a GRUB option one can configure on the >>> controller that >>> is for the computes? I don't remember seeing one. If you're >>> actually changing >>> /etc/default/grub, I doubt that's supported. >>> I did not find a procedure for changing default_hugepagesz on >>> controllers or >>> computes grub configuration. >>> >>> It's supposed to be an option, for example through in horizon >>> interface or >>> 'system host-memory-modify'... but you didn't quote either of those. >>> >>> >>>> The flavor has been modified, with the following option: " >>> hw:mem_page_size=104875" (1G) page. >>> >>> >>> Is "104875" a typo? It looks a lot like 1024 * 1024 = 1048576, but not >>> quite. >>> >>> >>> M >>> >>> >>>> However when I launch a VM using the modified flavor it creates the >>>> VM, but if >>> I check the field values of the VM using " openstack server show >>> <vm_name>". >>> I'm getting the following value for pgsize: >>>> wrs-res:topology | node:0, 512MB, pgsize:2M, >>>> vcpus:0, pol:sha | >>>> >>>> So, seems like it's still taking the 2M pagesize instead of the 1G. >>>> >>>> What I have already tried is to: >>>> >>>> * Change also the image metadata: hw:mem_page_size=large and >>>> also >>> 104875. Result, VM launched successfully, but pgsize value still 2M. >>> >>> There's the interesting "104875" again. >>> >>> >>>> * Leave the GRUB default values, and use modified flavor to >>> hw:mem_page_size=104875, launch the VM with ERROR status. >>> >>> 104875 >>> >>>> * Leave the GRUB default values, and use modified flavor to >>> hw:mem_page_size=2048, launch the VM with success, , pgsize value is >>> 2M, >>> which is expected, ok. >>>> * Leave the GRUB default values, and use modified flavor to >>> hw:mem_page_size=small launch the VM with success, , pgsize value is >>> 4k, >>> which is expected, ok. >>>> * Leave the GRUB default values, and use modified flavor to >>> hw:mem_page_size=any launch the VM with success, , pgsize value is >>> 2M, which >>> is expected, ok. >>>> Could be this an issue trying to use 1G mem pages or do I'm missing >>>> some >>> other configuration parameter? >>>> Thanks in advance for your help >>>> >>>> >>>> _______________________________________________ >>>> Starlingx-discuss mailing list >>>> Starlingx-discuss@lists.starlingx.io >>>> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss >
It's working now: wrsroot@compute-0 ~(keystone_admin)]$ openstack flavor set m1.small --property hw:mem_page_size=1GB [wrsroot@compute-0 ~(keystone_admin)]$ openstack server create --image cirros --flavor m1.small --nic net-id=net3 testvm [wrsroot@compute-0 ~(keystone_admin)]$ openstack server show testvm +-------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +-------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | wrs-res:pci_devices | | | wrs-res:topology | node:0, 1024MB, pgsize:1G, vcpus:0-3, pol:sha | | wrs-res:vcpus | [4, 4, 4] | | wrs-sg:server_group | | +-------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ Thanks ! -Richo > -----Original Message----- > From: Michel Thebeau [mailto:michel.thebeau@windriver.com] > Sent: Monday, October 22, 2018 11:18 AM > To: Perez, Ricardo O <ricardo.o.perez@intel.com>; starlingx- > discuss@lists.starlingx.io > Subject: Re: [Starlingx-discuss] Question - Using 1Gb mem pages > > Hi Ricardo, > > I was able to select 4K and 2M pages in the following way (attached txt file). I'll > try to see if I can get a system up with 1G pages, and I'll follow up with you. > > > > M > > > On 2018-10-22 10:59 a.m., Michel Thebeau wrote: > > Hi Ricardo, > > > > The documentation indicates to configure the the huge pages per numa > > node on the compute, and to configure the flavour extra specs for the > > VM. I am not aware of anything else in particular. There's some tips > > and caveats in the document, but I don't see anything in your message > > to think of those. > > > > If there's any chance you are trying to do this under virtualization, > > I am not aware that that is currently supported. > > > > M > > > > > > On 2018-10-19 6:14 p.m., Perez, Ricardo O wrote: > >> Hi Michael, > >> > >> Thanks for your answer, I just forgot to put the computes information > >> about memory configuration, here is the info: > >> > >> Compute-0 > >> > >> VM Pages > >> 0 Reserved for Platform: 8000 MiB > >> Usable Total: 86387 MiB > >> Available: 84787 MiB > >> 4K Pages: > >> Total: 230656 > >> 2M Hugepages: > >> Total: 1271 > >> Available: 983 > >> 1G Hugepages: > >> Total: 80 > >> Available: 80 > >> > >> 1 Reserved for Platform: 2000 MiB > >> Usable Total: 93698 MiB > >> Available: 92674 MiB > >> 4K Pages: > >> Total: 29696 > >> 2M Hugepages: > >> Total: 5319 > >> Available: 5319 > >> 1G Hugepages: > >> Total: 80 > >> Available: 80 > >> > >> *********************************** > >> Compute-1 > >> Processor > >> Memory > >> VM Pages > >> 0 Reserved for Platform: 8000 MiB > >> Usable Total: 86504 MiB > >> Available: 84968 MiB > >> 4K Pages: > >> Total: 0 > >> 2M Hugepages: > >> Total: 1780 > >> Available: 1524 > >> 1G Hugepages: > >> Total: 80 > >> Available: 80 > >> > >> 1 Reserved for Platform: 2000 MiB > >> Usable Total: 93552 MiB > >> Available: 92528 MiB > >> 4K Pages: > >> Total: 0 > >> 2M Hugepages: > >> Total: 5304 > >> Available: 5304 > >> 1G Hugepages: > >> Total: 80 > >> Available: 80 > >> > >> Am I missing something else to made use of 1G memory pages ? > >> > >> Thanks > >> -Richo > >> > >>> -----Original Message----- > >>> From: Michel Thebeau [mailto:michel.thebeau@windriver.com] > >>> Sent: Friday, October 19, 2018 1:39 PM > >>> To: Perez, Ricardo O <ricardo.o.perez@intel.com>; starlingx- > >>> discuss@lists.starlingx.io > >>> Subject: Re: [Starlingx-discuss] Question - Using 1Gb mem pages > >>> > >>> On 2018-10-19 1:45 p.m., Perez, Ricardo O wrote: > >>>> Hello, > >>>> > >>>> I'm trying to execute a test, which his goal is to made use of huge > >>>> pages (2M / > >>> 1G) launching a VM. > >>>> I have already verified the configuration of the controllers GRUB > >>>> options and > >>> we have defined the variable hugepagesz=1G, however, inside the same > >>> parameters list, there is a "default_hugepagesz=2M". The default > >>> hugepagesz, has been modified to 1G, and the compute has been > >>> restarted with no issues. > >>> > >>> > >>> Hi Ricardo, > >>> > >>> You wrote, "controllers GRUB options" then "the compute has been > >>> restarted with no issues". Is there a GRUB option one can configure > >>> on the controller that is for the computes? I don't remember seeing > >>> one. If you're actually changing /etc/default/grub, I doubt that's > >>> supported. > >>> I did not find a procedure for changing default_hugepagesz on > >>> controllers or computes grub configuration. > >>> > >>> It's supposed to be an option, for example through in horizon > >>> interface or 'system host-memory-modify'... but you didn't quote > >>> either of those. > >>> > >>> > >>>> The flavor has been modified, with the following option: " > >>> hw:mem_page_size=104875" (1G) page. > >>> > >>> > >>> Is "104875" a typo? It looks a lot like 1024 * 1024 = 1048576, but > >>> not quite. > >>> > >>> > >>> M > >>> > >>> > >>>> However when I launch a VM using the modified flavor it creates the > >>>> VM, but if > >>> I check the field values of the VM using " openstack server show > >>> <vm_name>". > >>> I'm getting the following value for pgsize: > >>>> wrs-res:topology | node:0, 512MB, pgsize:2M, > >>>> vcpus:0, pol:sha | > >>>> > >>>> So, seems like it's still taking the 2M pagesize instead of the 1G. > >>>> > >>>> What I have already tried is to: > >>>> > >>>> * Change also the image metadata: hw:mem_page_size=large and > >>>> also > >>> 104875. Result, VM launched successfully, but pgsize value still 2M. > >>> > >>> There's the interesting "104875" again. > >>> > >>> > >>>> * Leave the GRUB default values, and use modified flavor to > >>> hw:mem_page_size=104875, launch the VM with ERROR status. > >>> > >>> 104875 > >>> > >>>> * Leave the GRUB default values, and use modified flavor to > >>> hw:mem_page_size=2048, launch the VM with success, , pgsize value is > >>> 2M, which is expected, ok. > >>>> * Leave the GRUB default values, and use modified flavor to > >>> hw:mem_page_size=small launch the VM with success, , pgsize value is > >>> 4k, which is expected, ok. > >>>> * Leave the GRUB default values, and use modified flavor to > >>> hw:mem_page_size=any launch the VM with success, , pgsize value is > >>> 2M, which is expected, ok. > >>>> Could be this an issue trying to use 1G mem pages or do I'm missing > >>>> some > >>> other configuration parameter? > >>>> Thanks in advance for your help > >>>> > >>>> > >>>> _______________________________________________ > >>>> Starlingx-discuss mailing list > >>>> Starlingx-discuss@lists.starlingx.io > >>>> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discus > >>>> s > >
Hi Ricardo, That's cool. I had added cpu flag 'pdpe1gb' to compute under-virtualization, otherwise as persented by the virtualization wiki, to see if it would work: <cpu match='exact'> <model fallback='forbid'>Nehalem</model> <topology sockets='1' cores='4' threads='1'/> <feature policy='require' name='vmx'/> + <feature policy='require' name='pdpe1gb'/> <feature policy='optional' name='svm'/> </cpu> <clock offset='utc'/> And I was able to configure the compute and launch VMs with 1G pages (per the attached example). ... I found the memory accounting or display to be a bit wonky though. I'm not sure what to make of that. I'll mention it to my bosses, but I'm sure my method is not a supported. ;) M On 2018-10-22 12:54 p.m., Perez, Ricardo O wrote: > It's working now: > > wrsroot@compute-0 ~(keystone_admin)]$ openstack flavor set m1.small --property hw:mem_page_size=1GB > [wrsroot@compute-0 ~(keystone_admin)]$ openstack server create --image cirros --flavor m1.small --nic net-id=net3 testvm > [wrsroot@compute-0 ~(keystone_admin)]$ openstack server show testvm > +-------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ > | Field | Value | > +-------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ > | wrs-res:pci_devices | | > | wrs-res:topology | node:0, 1024MB, pgsize:1G, vcpus:0-3, pol:sha | > | wrs-res:vcpus | [4, 4, 4] | > | wrs-sg:server_group | | > +-------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ > > > Thanks ! > -Richo > >> -----Original Message----- >> From: Michel Thebeau [mailto:michel.thebeau@windriver.com] >> Sent: Monday, October 22, 2018 11:18 AM >> To: Perez, Ricardo O <ricardo.o.perez@intel.com>; starlingx- >> discuss@lists.starlingx.io >> Subject: Re: [Starlingx-discuss] Question - Using 1Gb mem pages >> >> Hi Ricardo, >> >> I was able to select 4K and 2M pages in the following way (attached txt file). I'll >> try to see if I can get a system up with 1G pages, and I'll follow up with you. >> >> >> >> M >> >> >> On 2018-10-22 10:59 a.m., Michel Thebeau wrote: >>> Hi Ricardo, >>> >>> The documentation indicates to configure the the huge pages per numa >>> node on the compute, and to configure the flavour extra specs for the >>> VM. I am not aware of anything else in particular. There's some tips >>> and caveats in the document, but I don't see anything in your message >>> to think of those. >>> >>> If there's any chance you are trying to do this under virtualization, >>> I am not aware that that is currently supported. >>> >>> M >>> >>> >>> On 2018-10-19 6:14 p.m., Perez, Ricardo O wrote: >>>> Hi Michael, >>>> >>>> Thanks for your answer, I just forgot to put the computes information >>>> about memory configuration, here is the info: >>>> >>>> Compute-0 >>>> >>>> VM Pages >>>> 0 Reserved for Platform: 8000 MiB >>>> Usable Total: 86387 MiB >>>> Available: 84787 MiB >>>> 4K Pages: >>>> Total: 230656 >>>> 2M Hugepages: >>>> Total: 1271 >>>> Available: 983 >>>> 1G Hugepages: >>>> Total: 80 >>>> Available: 80 >>>> >>>> 1 Reserved for Platform: 2000 MiB >>>> Usable Total: 93698 MiB >>>> Available: 92674 MiB >>>> 4K Pages: >>>> Total: 29696 >>>> 2M Hugepages: >>>> Total: 5319 >>>> Available: 5319 >>>> 1G Hugepages: >>>> Total: 80 >>>> Available: 80 >>>> >>>> *********************************** >>>> Compute-1 >>>> Processor >>>> Memory >>>> VM Pages >>>> 0 Reserved for Platform: 8000 MiB >>>> Usable Total: 86504 MiB >>>> Available: 84968 MiB >>>> 4K Pages: >>>> Total: 0 >>>> 2M Hugepages: >>>> Total: 1780 >>>> Available: 1524 >>>> 1G Hugepages: >>>> Total: 80 >>>> Available: 80 >>>> >>>> 1 Reserved for Platform: 2000 MiB >>>> Usable Total: 93552 MiB >>>> Available: 92528 MiB >>>> 4K Pages: >>>> Total: 0 >>>> 2M Hugepages: >>>> Total: 5304 >>>> Available: 5304 >>>> 1G Hugepages: >>>> Total: 80 >>>> Available: 80 >>>> >>>> Am I missing something else to made use of 1G memory pages ? >>>> >>>> Thanks >>>> -Richo >>>> >>>>> -----Original Message----- >>>>> From: Michel Thebeau [mailto:michel.thebeau@windriver.com] >>>>> Sent: Friday, October 19, 2018 1:39 PM >>>>> To: Perez, Ricardo O <ricardo.o.perez@intel.com>; starlingx- >>>>> discuss@lists.starlingx.io >>>>> Subject: Re: [Starlingx-discuss] Question - Using 1Gb mem pages >>>>> >>>>> On 2018-10-19 1:45 p.m., Perez, Ricardo O wrote: >>>>>> Hello, >>>>>> >>>>>> I'm trying to execute a test, which his goal is to made use of huge >>>>>> pages (2M / >>>>> 1G) launching a VM. >>>>>> I have already verified the configuration of the controllers GRUB >>>>>> options and >>>>> we have defined the variable hugepagesz=1G, however, inside the same >>>>> parameters list, there is a "default_hugepagesz=2M". The default >>>>> hugepagesz, has been modified to 1G, and the compute has been >>>>> restarted with no issues. >>>>> >>>>> >>>>> Hi Ricardo, >>>>> >>>>> You wrote, "controllers GRUB options" then "the compute has been >>>>> restarted with no issues". Is there a GRUB option one can configure >>>>> on the controller that is for the computes? I don't remember seeing >>>>> one. If you're actually changing /etc/default/grub, I doubt that's >>>>> supported. >>>>> I did not find a procedure for changing default_hugepagesz on >>>>> controllers or computes grub configuration. >>>>> >>>>> It's supposed to be an option, for example through in horizon >>>>> interface or 'system host-memory-modify'... but you didn't quote >>>>> either of those. >>>>> >>>>> >>>>>> The flavor has been modified, with the following option: " >>>>> hw:mem_page_size=104875" (1G) page. >>>>> >>>>> >>>>> Is "104875" a typo? It looks a lot like 1024 * 1024 = 1048576, but >>>>> not quite. >>>>> >>>>> >>>>> M >>>>> >>>>> >>>>>> However when I launch a VM using the modified flavor it creates the >>>>>> VM, but if >>>>> I check the field values of the VM using " openstack server show >>>>> <vm_name>". >>>>> I'm getting the following value for pgsize: >>>>>> wrs-res:topology | node:0, 512MB, pgsize:2M, >>>>>> vcpus:0, pol:sha | >>>>>> >>>>>> So, seems like it's still taking the 2M pagesize instead of the 1G. >>>>>> >>>>>> What I have already tried is to: >>>>>> >>>>>> * Change also the image metadata: hw:mem_page_size=large and >>>>>> also >>>>> 104875. Result, VM launched successfully, but pgsize value still 2M. >>>>> >>>>> There's the interesting "104875" again. >>>>> >>>>> >>>>>> * Leave the GRUB default values, and use modified flavor to >>>>> hw:mem_page_size=104875, launch the VM with ERROR status. >>>>> >>>>> 104875 >>>>> >>>>>> * Leave the GRUB default values, and use modified flavor to >>>>> hw:mem_page_size=2048, launch the VM with success, , pgsize value is >>>>> 2M, which is expected, ok. >>>>>> * Leave the GRUB default values, and use modified flavor to >>>>> hw:mem_page_size=small launch the VM with success, , pgsize value is >>>>> 4k, which is expected, ok. >>>>>> * Leave the GRUB default values, and use modified flavor to >>>>> hw:mem_page_size=any launch the VM with success, , pgsize value is >>>>> 2M, which is expected, ok. >>>>>> Could be this an issue trying to use 1G mem pages or do I'm missing >>>>>> some >>>>> other configuration parameter? >>>>>> Thanks in advance for your help >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Starlingx-discuss mailing list >>>>>> Starlingx-discuss@lists.starlingx.io >>>>>> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discus >>>>>> s
If you want to use 1G huge pages, you need to provision 1G huge pages in sysinv for the compute and then specify the page size extra spec when launching the VM. system host-memory-modify usage: system host-memory-modify [-m <Platform Reserved MiB>] [-2M <2M hugepages number>] [-1G <1G hugepages number>] <hostname or id> <processor> You will have to reduce the number of 2M huge pages to accommodate the 1G requirements. You should not be manually modifying grub args. Brent From: Perez, Ricardo O [mailto:ricardo.o.perez@intel.com] Sent: Friday, October 19, 2018 1:45 PM To: starlingx-discuss@lists.starlingx.io Subject: [Starlingx-discuss] Question - Using 1Gb mem pages Hello, I'm trying to execute a test, which his goal is to made use of huge pages (2M / 1G) launching a VM. I have already verified the configuration of the controllers GRUB options and we have defined the variable hugepagesz=1G, however, inside the same parameters list, there is a "default_hugepagesz=2M". The default hugepagesz, has been modified to 1G, and the compute has been restarted with no issues. The flavor has been modified, with the following option: " hw:mem_page_size=104875" (1G) page. However when I launch a VM using the modified flavor it creates the VM, but if I check the field values of the VM using " openstack server show <vm_name>". I'm getting the following value for pgsize: wrs-res:topology | node:0, 512MB, pgsize:2M, vcpus:0, pol:sha | So, seems like it's still taking the 2M pagesize instead of the 1G. What I have already tried is to: * Change also the image metadata: hw:mem_page_size=large and also 104875. Result, VM launched successfully, but pgsize value still 2M. * Leave the GRUB default values, and use modified flavor to hw:mem_page_size=104875, launch the VM with ERROR status. * Leave the GRUB default values, and use modified flavor to hw:mem_page_size=2048, launch the VM with success, , pgsize value is 2M, which is expected, ok. * Leave the GRUB default values, and use modified flavor to hw:mem_page_size=small launch the VM with success, , pgsize value is 4k, which is expected, ok. * Leave the GRUB default values, and use modified flavor to hw:mem_page_size=any launch the VM with success, , pgsize value is 2M, which is expected, ok. Could be this an issue trying to use 1G mem pages or do I'm missing some other configuration parameter? Thanks in advance for your help
participants (3)
-
Michel Thebeau
-
Perez, Ricardo O
-
Rowsell, Brent