[Starlingx-discuss] Question - Using 1Gb mem pages

Michel Thebeau michel.thebeau at windriver.com
Mon Oct 22 16:18:02 UTC 2018


Hi Ricardo,

I was able to select 4K and 2M pages in the following way (attached txt 
file).  I'll try to see if I can get a system up with 1G pages, and I'll 
follow up with you.



M


On 2018-10-22 10:59 a.m., Michel Thebeau wrote:
> Hi Ricardo,
>
> The documentation indicates to configure the the huge pages per numa 
> node on the compute, and to configure the flavour extra specs for the 
> VM.  I am not aware of anything else in particular. There's some tips 
> and caveats in the document, but I don't see anything in your message 
> to think of those.
>
> If there's any chance you are trying to do this under virtualization, 
> I am not aware that that is currently supported.
>
> M
>
>
> On 2018-10-19 6:14 p.m., Perez, Ricardo O wrote:
>> Hi Michael,
>>
>> Thanks for your answer, I just forgot to put the computes information 
>> about memory configuration, here is the info:
>>
>> Compute-0
>>
>>     VM Pages
>> 0   Reserved for Platform: 8000 MiB
>> Usable Total: 86387 MiB
>> Available: 84787 MiB
>>      4K Pages:
>>      Total: 230656
>> 2M Hugepages:
>>      Total: 1271
>>      Available: 983
>> 1G Hugepages:
>>      Total: 80
>>      Available: 80
>>
>> 1   Reserved for Platform: 2000 MiB
>> Usable Total: 93698 MiB
>> Available: 92674 MiB
>>      4K Pages:
>>      Total: 29696
>> 2M Hugepages:
>>      Total: 5319
>>      Available: 5319
>> 1G Hugepages:
>>      Total: 80
>>      Available: 80
>>
>> ***********************************
>> Compute-1
>> Processor
>>      Memory
>>      VM Pages
>> 0   Reserved for Platform: 8000 MiB
>> Usable Total: 86504 MiB
>> Available: 84968 MiB
>>      4K Pages:
>>      Total: 0
>> 2M Hugepages:
>>      Total: 1780
>>      Available: 1524
>> 1G Hugepages:
>>      Total: 80
>>      Available: 80
>>
>> 1   Reserved for Platform: 2000 MiB
>> Usable Total: 93552 MiB
>> Available: 92528 MiB
>>      4K Pages:
>>      Total: 0
>> 2M Hugepages:
>>      Total: 5304
>>      Available: 5304
>> 1G Hugepages:
>>      Total: 80
>>      Available: 80
>>
>> Am I missing something else to made use of 1G memory pages ?
>>
>> Thanks
>> -Richo
>>
>>> -----Original Message-----
>>> From: Michel Thebeau [mailto:michel.thebeau at windriver.com]
>>> Sent: Friday, October 19, 2018 1:39 PM
>>> To: Perez, Ricardo O <ricardo.o.perez at intel.com>; starlingx-
>>> discuss at lists.starlingx.io
>>> Subject: Re: [Starlingx-discuss] Question - Using 1Gb mem pages
>>>
>>> On 2018-10-19 1:45 p.m., Perez, Ricardo O wrote:
>>>> Hello,
>>>>
>>>> I'm trying to execute a test, which his goal is to made use of huge 
>>>> pages (2M /
>>> 1G) launching a VM.
>>>> I have already verified the configuration of the controllers GRUB 
>>>> options and
>>> we have defined the variable hugepagesz=1G, however, inside the same
>>> parameters list, there is a "default_hugepagesz=2M". The default 
>>> hugepagesz,
>>> has been modified to 1G, and the compute has been restarted with no 
>>> issues.
>>>
>>>
>>> Hi Ricardo,
>>>
>>> You wrote, "controllers GRUB options" then "the compute has been 
>>> restarted
>>> with no issues".  Is there a GRUB option one can configure on the 
>>> controller that
>>> is for the computes?  I don't remember seeing one. If you're 
>>> actually changing
>>> /etc/default/grub, I doubt that's supported.
>>> I did not find a procedure for changing default_hugepagesz  on 
>>> controllers or
>>> computes grub configuration.
>>>
>>> It's supposed to be an option, for example through in horizon 
>>> interface or
>>> 'system host-memory-modify'... but you didn't quote either of those.
>>>
>>>
>>>> The flavor has been modified, with the following option: "
>>> hw:mem_page_size=104875"  (1G) page.
>>>
>>>
>>> Is "104875" a typo?  It looks a lot like 1024 * 1024 = 1048576, but not
>>> quite.
>>>
>>>
>>> M
>>>
>>>
>>>> However when I launch a VM using the modified flavor it creates the 
>>>> VM, but if
>>> I check the field values of the VM using " openstack server show 
>>> <vm_name>".
>>> I'm getting the following value for pgsize:
>>>> wrs-res:topology                    | node:0,   512MB, pgsize:2M, 
>>>> vcpus:0, pol:sha |
>>>>
>>>> So, seems like it's still taking the 2M pagesize instead of the 1G.
>>>>
>>>> What I have already tried is to:
>>>>
>>>> *        Change also the image metadata: hw:mem_page_size=large and 
>>>> also
>>> 104875. Result, VM launched successfully, but pgsize value still 2M.
>>>
>>> There's the interesting "104875" again.
>>>
>>>
>>>> *        Leave the GRUB default values, and use modified flavor to
>>> hw:mem_page_size=104875, launch the VM with ERROR status.
>>>
>>> 104875
>>>
>>>> *        Leave the GRUB default values, and use modified flavor to
>>> hw:mem_page_size=2048, launch the VM with success, , pgsize value is 
>>> 2M,
>>> which is expected, ok.
>>>> *        Leave the GRUB default values, and use modified flavor to
>>> hw:mem_page_size=small launch the VM with success, , pgsize value is 
>>> 4k,
>>> which is  expected, ok.
>>>> *        Leave the GRUB default values, and use modified flavor to
>>> hw:mem_page_size=any launch the VM with success, , pgsize value is 
>>> 2M, which
>>> is  expected, ok.
>>>> Could be this an issue trying to use 1G mem pages or do I'm missing 
>>>> some
>>> other configuration parameter?
>>>> Thanks in advance for your help
>>>>
>>>>
>>>> _______________________________________________
>>>> Starlingx-discuss mailing list
>>>> Starlingx-discuss at lists.starlingx.io
>>>> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
>

-------------- next part --------------
# nova flavor-key s.f1.256 set hw:mem_page_size=4
# nova flavor-key s.f1     set hw:mem_page_size=2048


# openstack flavor show s.f1.256 | grep extra
| properties                 | aggregate_instance_extra_specs:storage='local_image', hw:mem_page_size='4' |

# openstack flavor show s.f1 | grep extra
| properties                 | aggregate_instance_extra_specs:storage='local_image', hw:mem_page_size='2048' |

# launch 2 vms with either flavor
# ...

# nova show shouldbe4k | grep -i "pgsize\|hyper"
| OS-EXT-SRV-ATTR:hypervisor_hostname  | compute-0 |
| wrs-res:topology                     | node:0,   256MB, pgsize:4K, vcpus:0, pol:sha

# nova show shouldbe2M | grep -i "pgsize\|hyper"
| OS-EXT-SRV-ATTR:hypervisor_hostname  | compute-0 |
| wrs-res:topology                     | node:0,   500MB, pgsize:2M, vcpus:0, pol:sha




# system host-memory-show compute-0 0
+--------------------------------+--------------------------------------+
| Property                       | Value                                |
+--------------------------------+--------------------------------------+
| Memory: Usable Total (MiB)     | 14190                                |
|         Platform     (MiB)     | 1600                                 |
|         Available    (MiB)     | 13178                                |
| Huge Pages Configured          | True                                 |
| vSwitch Huge Pages: Size (MiB) | 2                                    |
|                     Total      | 256                                  |
|                     Available  | 0                                    |
| VM  Pages (4K): Total          | 1824768                              |
| VM  Huge Pages (2M): Total     | 3275                                 |
|                 Available      | 3025                                 |
| VM  Huge Pages (1G): Total     | 0                                    |
|                 Available      | None                                 |



More information about the Starlingx-discuss mailing list