Specifying "hw:mem_page_size=large" means that it will only use hugepages, not 4K pages. On the compute node do you have enough CPU and free hugepages on the same NUMA node to satisfy the request? The 'NUMATopologyFilter: (start: 1, end: 0)' part of the log says that this filter ruled out your only compute node. What does "virsh capabilities|grep -C5 pages" show on the compute node? What does /etc/nova/nova.conf show inside the nova-compute container? It's possible to enable debug mode for nova.conf but it's a little tricky to get the syntax right. Basically you'd be using the "system helm-override-update" command. Can you give the output of "system helm-override-show" for the nova chart in the openstack application? Chris On 9/24/2019 6:16 AM, Mariano Ucha wrote:
Hi all!
I'm having a trouble now trying to allocate a VM con Openstack. I'm using STX 2.0 AIO-SX with DPDK so when i'm trying to allocate a VM with a flavor that have hw:mem_page_size=large metadata the system gives me *error 500 **No valid host was found. There are not enough hosts available.* * * File "/var/lib/openstack/lib/python2.7/site-packages/nova/conductor/manager.py", line 1346, in schedule_and_build_instances instance_uuids, return_alternates=True) File "/var/lib/openstack/lib/python2.7/site-packages/nova/conductor/manager.py", line 800, in _schedule_instances return_alternates=return_alternates) File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations instance_uuids, return_objects, return_alternates) File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations return cctxt.call(ctxt, 'select_destinations', **msg_args) File "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 178, in call retry=self.retry) File "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/transport.py", line 128, in _send retry=retry) File "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 645, in send call_monitor_timeout, retry=retry) File "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 636, in _send raise result
* * *Logs from Nova Scheduler*
2019-09-23 17:37:56.537 1 INFO nova.filters [req-f57095ac-6539-4efb-a6a5-f27aebb0a27c edbcf8808d8a4f48a39b8eebb3951292 8ae61d9905d94b8aa66193b20a3ab973 - default default] Filter NUMATopologyFilter returned 0 hosts 2019-09-23 17:37:56.537 1 INFO nova.filters [req-f57095ac-6539-4efb-a6a5-f27aebb0a27c edbcf8808d8a4f48a39b8eebb3951292 8ae61d9905d94b8aa66193b20a3ab973 - default default] Filter NUMATopologyFilter returned 0 hosts 2019-09-23 17:37:56.538 1 INFO nova.filters [req-f57095ac-6539-4efb-a6a5-f27aebb0a27c edbcf8808d8a4f48a39b8eebb3951292 8ae61d9905d94b8aa66193b20a3ab973 - default default] Filtering removed all hosts for the request with instance ID '66175096-37a4-4a2c-b23a-e7a4a6f10ca1'. Filter results: ['RetryFilter: (start: 1, end: 1)', 'ComputeFilter: (start: 1, end: 1)', 'AvailabilityZoneFilter: (start: 1, end: 1)', 'AggregateInstanceExtraSpecsFilter: (start: 1, end: 1)', 'ComputeCapabilitiesFilter: (start: 1, end: 1)', 'ImagePropertiesFilter: (start: 1, end: 1)', 'NUMATopologyFilter: (start: 1, end: 0)'] 2019-09-23 17:37:56.538 1 INFO nova.filters [req-f57095ac-6539-4efb-a6a5-f27aebb0a27c edbcf8808d8a4f48a39b8eebb3951292 8ae61d9905d94b8aa66193b20a3ab973 - default default] Filtering removed all hosts for the request with instance ID '66175096-37a4-4a2c-b23a-e7a4a6f10ca1'. Filter results: ['RetryFilter: (start: 1, end: 1)', 'ComputeFilter: (start: 1, end: 1)', 'AvailabilityZoneFilter: (start: 1, end: 1)', 'AggregateInstanceExtraSpecsFilter: (start: 1, end: 1)', 'ComputeCapabilitiesFilter: (start: 1, end: 1)', 'ImagePropertiesFilter: (start: 1, end: 1)', 'NUMATopologyFilter: (start: 1, end: 0)']
*Logs from Nova Conductor*
2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager Traceback (most recent call last): 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/nova/conductor/manager.py", line 1346, in schedule_and_build_instances 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager instance_uuids, return_alternates=True) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/nova/conductor/manager.py", line 800, in _schedule_instances 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager return_alternates=return_alternates) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager instance_uuids, return_objects, return_alternates) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 178, in call 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager retry=self.retry) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/transport.py", line 128, in _send 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager retry=retry) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 645, in send 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager call_monitor_timeout, retry=retry) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 636, in _send 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager raise result 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager NoValidHost_Remote: No valid host was found. There are not enough hosts available. 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager Traceback (most recent call last): 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 229, in inner 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager return func(*args, **kwargs) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/manager.py", line 168, in select_destinations 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager allocation_request_version, return_alternates) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 96, in select_destinations 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager allocation_request_version, return_alternates) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 265, in _schedule 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager claimed_instance_uuids) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 302, in _ensure_sufficient_hosts 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager NoValidHost: No valid host was found. There are not enough hosts available. 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager [req-f57095ac-6539-4efb-a6a5-f27aebb0a27c edbcf8808d8a4f48a39b8eebb3951292 8ae61d9905d94b8aa66193b20a3ab973 - default default] Failed to schedule instances: NoValidHost_Remote: No valid host was found. There are not enough hosts available. Traceback (most recent call last):
File "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 229, in inner return func(*args, **kwargs)
File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/manager.py", line 168, in select_destinations allocation_request_version, return_alternates)
File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 96, in select_destinations allocation_request_version, return_alternates)
File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 265, in _schedule claimed_instance_uuids)
File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 302, in _ensure_sufficient_hosts raise exception.NoValidHost(reason=reason)
NoValidHost: No valid host was found. There are not enough hosts available. 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager Traceback (most recent call last): 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/nova/conductor/manager.py", line 1346, in schedule_and_build_instances 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager instance_uuids, return_alternates=True) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/nova/conductor/manager.py", line 800, in _schedule_instances 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager return_alternates=return_alternates) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager instance_uuids, return_objects, return_alternates) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 178, in call 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager retry=self.retry) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/transport.py", line 128, in _send 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager retry=retry) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 645, in send 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager call_monitor_timeout, retry=retry) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 636, in _send 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager raise result 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager NoValidHost_Remote: No valid host was found. There are not enough hosts available. 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager Traceback (most recent call last): 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 229, in inner 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager return func(*args, **kwargs) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/manager.py", line 168, in select_destinations 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager allocation_request_version, return_alternates) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 96, in select_destinations 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager allocation_request_version, return_alternates) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 265, in _schedule 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager claimed_instance_uuids) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 302, in _ensure_sufficient_hosts 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager NoValidHost: No valid host was found. There are not enough hosts available. 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 2019-09-23 17:37:56.627 1 WARNING nova.scheduler.utils [req-f57095ac-6539-4efb-a6a5-f27aebb0a27c edbcf8808d8a4f48a39b8eebb3951292 8ae61d9905d94b8aa66193b20a3ab973 - default default] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. Traceback (most recent call last):
File "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 229, in inner return func(*args, **kwargs)
File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/manager.py", line 168, in select_destinations allocation_request_version, return_alternates)
File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 96, in select_destinations allocation_request_version, return_alternates)
File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 265, in _schedule claimed_instance_uuids)
File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 302, in _ensure_sufficient_hosts raise exception.NoValidHost(reason=reason)
NoValidHost: No valid host was found. There are not enough hosts available. : NoValidHost_Remote: No valid host was found. There are not enough hosts available. 2019-09-23 17:37:56.627 1 WARNING nova.scheduler.utils [req-f57095ac-6539-4efb-a6a5-f27aebb0a27c edbcf8808d8a4f48a39b8eebb3951292 8ae61d9905d94b8aa66193b20a3ab973 - default default] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. Traceback (most recent call last):
File "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 229, in inner return func(*args, **kwargs)
File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/manager.py", line 168, in select_destinations allocation_request_version, return_alternates)
File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 96, in select_destinations allocation_request_version, return_alternates)
File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 265, in _schedule claimed_instance_uuids)
File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 302, in _ensure_sufficient_hosts raise exception.NoValidHost(reason=reason)
NoValidHost: No valid host was found. There are not enough hosts available. : NoValidHost_Remote: No valid host was found. There are not enough hosts available. 2019-09-23 17:37:56.630 1 WARNING nova.scheduler.utils [req-f57095ac-6539-4efb-a6a5-f27aebb0a27c edbcf8808d8a4f48a39b8eebb3951292 8ae61d9905d94b8aa66193b20a3ab973 - default default] [instance: 66175096-37a4-4a2c-b23a-e7a4a6f10ca1] Setting instance to ERROR state.: NoValidHost_Remote: No valid host was found. There are not enough hosts available. 2019-09-23 17:37:56.630 1 WARNING nova.scheduler.utils [req-f57095ac-6539-4efb-a6a5-f27aebb0a27c edbcf8808d8a4f48a39b8eebb3951292 8ae61d9905d94b8aa66193b20a3ab973 - default default] [instance: 66175096-37a4-4a2c-b23a-e7a4a6f10ca1] Setting instance to ERROR state.
How can i change Nova log mode to DEBUG?
Anything more i can check?
Regards, Mariano
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss