[Starlingx-discuss] cpu mode

Fri Jan 18 15:47:23 UTC 2019

Hi Volker,

Thanks, yeah it does not include cpu_host flag, I was searching
about that and I found this:
https://wiki.openstack.org/wiki/LibvirtXMLCPUModel 

It seems that you need to change libvirt_cpu_mode option in your 
 /etc/nova/nova.conf file by:

libvirt_cpu_mode = host-passthrough

Best regards.
Mario.
________________________________________
From: von Hoesslin, Volker [Volker.Hoesslin at swsn.de]
Sent: Friday, January 18, 2019 1:30 AM
To: Arevalo, Mario Alfredo C; starlingx-discuss at lists.starlingx.io
Subject: AW: cpu mode

hi,
this is an "ps aux | grep qemu" output (for better reading, a little bit formated by me):

/usr/libexec/qemu-kvm
-name guest=instance-0000000a,debug-threads=on
-S
-object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-13-instance-0000000a/master-key.aes
-machine pc-i440fx-rhel7.4.0,accel=kvm,usb=off,dump-guest-core=off
-m 8192
-realtime mlock=off
-smp 4,sockets=4,cores=1,threads=1
-object memory-backend-file,id=ram-node0,prealloc=yes,mem-path=/mnt/huge-2048kB/libvirt/qemu/13-instance-0000000a,share=yes,size=8589934592,host-nodes=0,policy=bind
-numa node,nodeid=0,cpus=0-3,memdev=ram-node0
-uuid 3bce281c-db91-4b6f-aa48-832c27c2338f
-smbios type=1,manufacturer=Fedora Project,product=OpenStack Nova,version=16.0.2-1.tis.11,serial=f88367ca-9cf6-4678-bc69-69ed97297bb5,uuid=3bce281c-db91-4b6f-aa48-832c27c2338f,family=Virtual Machine
-no-user-config
-nodefaults
-chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-13-instance-0000000a/monitor.sock,server,nowait
-mon chardev=charmonitor,id=monitor,mode=control
-rtc base=utc,driftfix=slew
-global kvm-pit.lost_tick_policy=delay
-no-hpet
-no-shutdown
-boot reboot-timeout=5000,strict=on
-global i440FX-pcihost.pci-hole64-size=67108864K
-device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2
-drive file=rbd:cinder-volumes/volume-ef431cc7-2964-4335-8138-2d2c6642be6c:auth_supported=none:mon_host=192.168.204.3\:6789\;192.168.204.4\:6789\;192.168.204.112\:6789,format=raw,if=none,id=drive-virtio-disk0,cache=none,discard=unmap
-device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1,write-cache=on,serial=ef431cc7-2964-4335-8138-2d2c6642be6c
-chardev socket,id=charnet0,path=/var/run/openvswitch/vhu74250cb0-40,server
-netdev vhost-user,chardev=charnet0,id=hostnet0
-device virtio-net-pci,netdev=hostnet0,id=net0,mac=fa:16:3e:ff:cc:ac,bus=pci.0,addr=0x3
-add-fd set=0,fd=81
-chardev pty,id=charserial0,logfile=/dev/fdset/0,logappend=on
-device isa-serial,chardev=charserial0,id=serial0
-device usb-tablet,id=input0,bus=usb.0,port=1
-vnc 0.0.0.0:1
-k en-us
-device cirrus-vga,id=video0,bus=pci.0,addr=0x2
-incoming defer
-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
-msg timestamp=on

=> btw, how can this cmd-line handle the spaces in argument "-smbios" or just "ps aux" remove the ' " ' ?

this is an running VM without any changes, created and started on top starlingx (2x controller, 2x compute, 3x storage). and yes, there is no "-cpu host" argument !?

this ends in something like this in guest view:

$ lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              4
On-line CPU(s) list: 0-3
Thread(s) per core:  1
Core(s) per socket:  1
Socket(s):           4
NUMA node(s):        1
Vendor ID:           AuthenticAMD
CPU family:          6
Model:               13
Model name:          QEMU Virtual CPU version 2.5+
Stepping:            3
CPU MHz:             2199.996
BogoMIPS:            4399.99
Hypervisor vendor:   KVM
Virtualization type: full
L1d cache:           64K
L1i cache:           64K
L2 cache:            512K
L3 cache:            16384K
NUMA node0 CPU(s):   0-3
Flags:               fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx fxsr_opt pdpe1gb lm nopl cpuid pni cx16 x2apic popcnt hypervisor lahf_lm 3dnowprefetch vmmcall

here are some outputs from my flavor metadata trys:

hw:cpu_model = SandyBridge
=> No valid host was found. There are not enough hosts available. compute-0: (VCpuModelFilter) Host VCPU model EPYC-IBPB required SandyBridge, compute-1: (VCpuModelFilter) Host VCPU model EPYC-IBPB required SandyBridge
of course, SandyBridge is an intel CPU, but all of allowed CPU architectures for meta extra specs "hw:cpu_model" are Intel devices, so i need AMD! do you remember:

$ openstack flavor set --property hw:cpu_model=EPYC-IBPB 76609f7b-f0c7-48ca-8c8a-f78481e62cd4
Failed to set flavor property: Invalid hw:cpu_model 'EPYC-IBPB', must be one of: Passthrough, Conroe, Penryn, Nehalem, Westmere, SandyBridge, IvyBridge, Haswell, Broadwell-noTSX, Broadwell, Skylake-Client, Skylake-Server.

how can i edit this allowed CPU-Model-List?!

the nova.conf settings define no CPU config, so the hyperviser has to handle this:

# cat /etc/nova/nova.conf | grep cpu_mode
libvirt_cpu_mode = none
cpu_mode=none

so this is an deadlock for me, i do not know how to fix this :(

plz help, volker...

________________________________________
Von: Arevalo, Mario Alfredo C [mario.alfredo.c.arevalo at intel.com]
Gesendet: Donnerstag, 17. Januar 2019 17:40
An: von Hoesslin, Volker; starlingx-discuss at lists.starlingx.io
Betreff: RE: cpu mode

Hi Volker,

Could you please send me the QEMU command line used to launch your VM, this is in
order to check the QEMU arguments/flags, possibly it requires "-cpu host" argument.

Thanks.

Best regards.
Mario.

From: von Hoesslin, Volker [Volker.Hoesslin at swsn.de]

Sent: Thursday, January 17, 2019 8:23 AM

To: starlingx-discuss at lists.starlingx.io

Subject: Re: [Starlingx-discuss] cpu mode

it is impossible to set a EPIC (or any other AMD) as guest CPU?

$ openstack flavor set --property hw:cpu_model=EPYC-IBPB 76609f7b-f0c7-48ca-8c8a-f78481e62cd4
Failed to set flavor property: Invalid hw:cpu_model 'EPYC-IBPB', must be one of: Passthrough, Conroe, Penryn, Nehalem, Westmere, SandyBridge, IvyBridge, Haswell, Broadwell-noTSX, Broadwell, Skylake-Client, Skylake-Server. (HTTP 400) (Request-ID: req-2fda19cc-8e0e-4be8-a8ea-b58fc00358ce)
Command Failed: One or more of the operations failed

but my compute node seems to support EPIC CPUs?

cat /usr/share/libvirt/cpu_map/x86_EPYC-IBRS.xml

<cpus>
  <model name='EPYC-IBPB'>
    <signature family='23' model='1'/>
    <vendor name='AMD'/>
    <feature name='3dnowprefetch'/>
 ....
    <feature name='xsavec'/>
    <feature name='xsaveopt'/>
  </model>
</cpus>

some tips for me how to handle this?

volker...

Von: von Hoesslin, Volker [Volker.Hoesslin at swsn.de]

Gesendet: Donnerstag, 17. Januar 2019 15:50

An: starlingx-discuss at lists.starlingx.io

Betreff: [Starlingx-discuss] cpu mode

hi,
my setup has two computes nodes, every node has a dual AMD EPYC 7601 CPU config. how can i bring all the CPU features (AES, SSSE3, ...) to the guest VMs. i have tryed with some flavor-metadata but nothing realy helps, the VMs getting just a little subset
 of cpu-features.
some investigations to the kvm-settings hit me to the facts that my nova config has "cpu_model=none" !? how can i fix that and bring my AMD EPIC CPU to my nova-config?!

here is the host /proc/cpuinfo

processor       : 127
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7601 32-Core Processor
stepping        : 2
microcode       : 0x8001227
cpu MHz         : 1200.000
cache size      : 512 KB
physical id     : 1
siblings        : 64
core id         : 31
cpu cores       : 32
apicid          : 127
initial apicid  : 127
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq
 monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate retpoline_amd ssbd
 ibpb vmmcall fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 clzero irperf xsaveerptr arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif overflow_recov
 succor smca
bogomips        : 4400.08
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

greez & thx,

volker...