[Starlingx-discuss] Cannot allocate a VM - STX2.0-AIO-SX - Error 500

Chris Friesen chris.friesen at windriver.com
Fri Sep 27 19:45:29 UTC 2019 

Specifying "hw:mem_page_size=large" means that it will only use 
hugepages, not 4K pages.   On the compute node do you have enough CPU 
and free hugepages on the same NUMA node to satisfy the request?  The 
'NUMATopologyFilter: (start: 1, end: 0)' part of the log says that this 
filter ruled out your only compute node.

What does "virsh capabilities|grep -C5 pages" show on the compute node?

What does /etc/nova/nova.conf show inside the nova-compute container?

It's possible to enable debug mode for nova.conf but it's a little 
tricky to get the syntax right. Basically you'd be using the "system 
helm-override-update" command.  Can you give the output of "system 
helm-override-show" for the nova chart in the openstack application?

Chris

On 9/24/2019 6:16 AM, Mariano Ucha wrote:
> Hi all!
>
> I'm having a trouble now trying to allocate a VM con Openstack. I'm 
> using STX 2.0 AIO-SX with DPDK so when i'm trying to allocate a VM 
> with a flavor that have hw:mem_page_size=large metadata the system 
> gives me *error 500 **No valid host was found. There are not enough 
> hosts available.*
> *
> *
> File 
> "/var/lib/openstack/lib/python2.7/site-packages/nova/conductor/manager.py", 
> line 1346, in schedule_and_build_instances instance_uuids, 
> return_alternates=True) File 
> "/var/lib/openstack/lib/python2.7/site-packages/nova/conductor/manager.py", 
> line 800, in _schedule_instances return_alternates=return_alternates) 
> File 
> "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/client/query.py", 
> line 42, in select_destinations instance_uuids, return_objects, 
> return_alternates) File 
> "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/rpcapi.py", 
> line 160, in select_destinations return cctxt.call(ctxt, 
> 'select_destinations', **msg_args) File 
> "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", 
> line 178, in call retry=self.retry) File 
> "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/transport.py", 
> line 128, in _send retry=retry) File 
> "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", 
> line 645, in send call_monitor_timeout, retry=retry) File 
> "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", 
> line 636, in _send raise result
>
> *
> *
> *Logs from Nova Scheduler*
>
> 2019-09-23 17:37:56.537 1 INFO nova.filters 
> [req-f57095ac-6539-4efb-a6a5-f27aebb0a27c 
> edbcf8808d8a4f48a39b8eebb3951292 8ae61d9905d94b8aa66193b20a3ab973 - 
> default default] Filter NUMATopologyFilter returned 0 hosts
> 2019-09-23 17:37:56.537 1 INFO nova.filters 
> [req-f57095ac-6539-4efb-a6a5-f27aebb0a27c 
> edbcf8808d8a4f48a39b8eebb3951292 8ae61d9905d94b8aa66193b20a3ab973 - 
> default default] Filter NUMATopologyFilter returned 0 hosts
> 2019-09-23 17:37:56.538 1 INFO nova.filters 
> [req-f57095ac-6539-4efb-a6a5-f27aebb0a27c 
> edbcf8808d8a4f48a39b8eebb3951292 8ae61d9905d94b8aa66193b20a3ab973 - 
> default default] Filtering removed all hosts for the request with 
> instance ID '66175096-37a4-4a2c-b23a-e7a4a6f10ca1'. Filter results: 
> ['RetryFilter: (start: 1, end: 1)', 'ComputeFilter: (start: 1, end: 
> 1)', 'AvailabilityZoneFilter: (start: 1, end: 1)', 
> 'AggregateInstanceExtraSpecsFilter: (start: 1, end: 1)', 
> 'ComputeCapabilitiesFilter: (start: 1, end: 1)', 
> 'ImagePropertiesFilter: (start: 1, end: 1)', 'NUMATopologyFilter: 
> (start: 1, end: 0)']
> 2019-09-23 17:37:56.538 1 INFO nova.filters 
> [req-f57095ac-6539-4efb-a6a5-f27aebb0a27c 
> edbcf8808d8a4f48a39b8eebb3951292 8ae61d9905d94b8aa66193b20a3ab973 - 
> default default] Filtering removed all hosts for the request with 
> instance ID '66175096-37a4-4a2c-b23a-e7a4a6f10ca1'. Filter results: 
> ['RetryFilter: (start: 1, end: 1)', 'ComputeFilter: (start: 1, end: 
> 1)', 'AvailabilityZoneFilter: (start: 1, end: 1)', 
> 'AggregateInstanceExtraSpecsFilter: (start: 1, end: 1)', 
> 'ComputeCapabilitiesFilter: (start: 1, end: 1)', 
> 'ImagePropertiesFilter: (start: 1, end: 1)', 'NUMATopologyFilter: 
> (start: 1, end: 0)']
>
> *Logs from Nova Conductor*
>
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager Traceback (most 
> recent call last):
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager   File 
> "/var/lib/openstack/lib/python2.7/site-packages/nova/conductor/manager.py", 
> line 1346, in schedule_and_build_instances
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager instance_uuids, 
> return_alternates=True)
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager   File 
> "/var/lib/openstack/lib/python2.7/site-packages/nova/conductor/manager.py", 
> line 800, in _schedule_instances
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 
> return_alternates=return_alternates)
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager   File 
> "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/client/query.py", 
> line 42, in select_destinations
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager instance_uuids, 
> return_objects, return_alternates)
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager   File 
> "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/rpcapi.py", 
> line 160, in select_destinations
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager return 
> cctxt.call(ctxt, 'select_destinations', **msg_args)
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager   File 
> "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", 
> line 178, in call
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager retry=self.retry)
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager   File 
> "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/transport.py", 
> line 128, in _send
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager retry=retry)
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager   File 
> "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", 
> line 645, in send
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 
> call_monitor_timeout, retry=retry)
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager   File 
> "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", 
> line 636, in _send
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager raise result
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 
> NoValidHost_Remote: No valid host was found. There are not enough 
> hosts available.
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager Traceback (most 
> recent call last):
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager   File 
> "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", 
> line 229, in inner
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager return 
> func(*args, **kwargs)
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager   File 
> "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/manager.py", 
> line 168, in select_destinations
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 
> allocation_request_version, return_alternates)
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager   File 
> "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", 
> line 96, in select_destinations
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 
> allocation_request_version, return_alternates)
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager   File 
> "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", 
> line 265, in _schedule
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 
> claimed_instance_uuids)
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager   File 
> "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", 
> line 302, in _ensure_sufficient_hosts
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager raise 
> exception.NoValidHost(reason=reason)
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager NoValidHost: No 
> valid host was found. There are not enough hosts available.
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 
> [req-f57095ac-6539-4efb-a6a5-f27aebb0a27c 
> edbcf8808d8a4f48a39b8eebb3951292 8ae61d9905d94b8aa66193b20a3ab973 - 
> default default] Failed to schedule instances: NoValidHost_Remote: No 
> valid host was found. There are not enough hosts available.
> Traceback (most recent call last):
>
>   File 
> "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", 
> line 229, in inner
>     return func(*args, **kwargs)
>
>   File 
> "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/manager.py", 
> line 168, in select_destinations
>     allocation_request_version, return_alternates)
>
>   File 
> "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", 
> line 96, in select_destinations
>     allocation_request_version, return_alternates)
>
>   File 
> "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", 
> line 265, in _schedule
>     claimed_instance_uuids)
>
>   File 
> "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", 
> line 302, in _ensure_sufficient_hosts
>     raise exception.NoValidHost(reason=reason)
>
> NoValidHost: No valid host was found. There are not enough hosts 
> available.
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager Traceback (most 
> recent call last):
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager   File 
> "/var/lib/openstack/lib/python2.7/site-packages/nova/conductor/manager.py", 
> line 1346, in schedule_and_build_instances
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager instance_uuids, 
> return_alternates=True)
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager   File 
> "/var/lib/openstack/lib/python2.7/site-packages/nova/conductor/manager.py", 
> line 800, in _schedule_instances
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 
> return_alternates=return_alternates)
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager   File 
> "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/client/query.py", 
> line 42, in select_destinations
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager instance_uuids, 
> return_objects, return_alternates)
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager   File 
> "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/rpcapi.py", 
> line 160, in select_destinations
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager return 
> cctxt.call(ctxt, 'select_destinations', **msg_args)
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager   File 
> "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", 
> line 178, in call
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager retry=self.retry)
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager   File 
> "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/transport.py", 
> line 128, in _send
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager retry=retry)
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager   File 
> "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", 
> line 645, in send
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 
> call_monitor_timeout, retry=retry)
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager   File 
> "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", 
> line 636, in _send
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager raise result
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 
> NoValidHost_Remote: No valid host was found. There are not enough 
> hosts available.
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager Traceback (most 
> recent call last):
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager   File 
> "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", 
> line 229, in inner
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager return 
> func(*args, **kwargs)
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager   File 
> "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/manager.py", 
> line 168, in select_destinations
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 
> allocation_request_version, return_alternates)
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager   File 
> "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", 
> line 96, in select_destinations
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 
> allocation_request_version, return_alternates)
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager   File 
> "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", 
> line 265, in _schedule
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 
> claimed_instance_uuids)
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager   File 
> "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", 
> line 302, in _ensure_sufficient_hosts
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager raise 
> exception.NoValidHost(reason=reason)
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager NoValidHost: No 
> valid host was found. There are not enough hosts available.
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager
> 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager
> 2019-09-23 17:37:56.627 1 WARNING nova.scheduler.utils 
> [req-f57095ac-6539-4efb-a6a5-f27aebb0a27c 
> edbcf8808d8a4f48a39b8eebb3951292 8ae61d9905d94b8aa66193b20a3ab973 - 
> default default] Failed to compute_task_build_instances: No valid host 
> was found. There are not enough hosts available.
> Traceback (most recent call last):
>
>   File 
> "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", 
> line 229, in inner
>     return func(*args, **kwargs)
>
>   File 
> "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/manager.py", 
> line 168, in select_destinations
>     allocation_request_version, return_alternates)
>
>   File 
> "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", 
> line 96, in select_destinations
>     allocation_request_version, return_alternates)
>
>   File 
> "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", 
> line 265, in _schedule
>     claimed_instance_uuids)
>
>   File 
> "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", 
> line 302, in _ensure_sufficient_hosts
>     raise exception.NoValidHost(reason=reason)
>
> NoValidHost: No valid host was found. There are not enough hosts 
> available.
> : NoValidHost_Remote: No valid host was found. There are not enough 
> hosts available.
> 2019-09-23 17:37:56.627 1 WARNING nova.scheduler.utils 
> [req-f57095ac-6539-4efb-a6a5-f27aebb0a27c 
> edbcf8808d8a4f48a39b8eebb3951292 8ae61d9905d94b8aa66193b20a3ab973 - 
> default default] Failed to compute_task_build_instances: No valid host 
> was found. There are not enough hosts available.
> Traceback (most recent call last):
>
>   File 
> "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", 
> line 229, in inner
>     return func(*args, **kwargs)
>
>   File 
> "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/manager.py", 
> line 168, in select_destinations
>     allocation_request_version, return_alternates)
>
>   File 
> "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", 
> line 96, in select_destinations
>     allocation_request_version, return_alternates)
>
>   File 
> "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", 
> line 265, in _schedule
>     claimed_instance_uuids)
>
>   File 
> "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", 
> line 302, in _ensure_sufficient_hosts
>     raise exception.NoValidHost(reason=reason)
>
> NoValidHost: No valid host was found. There are not enough hosts 
> available.
> : NoValidHost_Remote: No valid host was found. There are not enough 
> hosts available.
> 2019-09-23 17:37:56.630 1 WARNING nova.scheduler.utils 
> [req-f57095ac-6539-4efb-a6a5-f27aebb0a27c 
> edbcf8808d8a4f48a39b8eebb3951292 8ae61d9905d94b8aa66193b20a3ab973 - 
> default default] [instance: 66175096-37a4-4a2c-b23a-e7a4a6f10ca1] 
> Setting instance to ERROR state.: NoValidHost_Remote: No valid host 
> was found. There are not enough hosts available.
> 2019-09-23 17:37:56.630 1 WARNING nova.scheduler.utils 
> [req-f57095ac-6539-4efb-a6a5-f27aebb0a27c 
> edbcf8808d8a4f48a39b8eebb3951292 8ae61d9905d94b8aa66193b20a3ab973 - 
> default default] [instance: 66175096-37a4-4a2c-b23a-e7a4a6f10ca1] 
> Setting instance to ERROR state.
>
> How can i change Nova log mode to DEBUG?
>
> Anything more i can check?
>
> Regards,
> Mariano
>
>
>
> _______________________________________________
> Starlingx-discuss mailing list
> Starlingx-discuss at lists.starlingx.io
> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20190927/b2281f64/attachment-0001.html>

More information about the Starlingx-discuss mailing list