Has anyone got any ideas on how to further debug this? I have also seen an issue similar to this and would like help getting further. Thanks Sau! On 9/24/19 5:16 AM, Mariano Ucha wrote:
Hi all!
I'm having a trouble now trying to allocate a VM con Openstack. I'm using STX 2.0 AIO-SX with DPDK so when i'm trying to allocate a VM with a flavor that have hw:mem_page_size=large metadata the system gives me *error 500 **No valid host was found. There are not enough hosts available.* * * File "/var/lib/openstack/lib/python2.7/site-packages/nova/conductor/manager.py", line 1346, in schedule_and_build_instances instance_uuids, return_alternates=True) File "/var/lib/openstack/lib/python2.7/site-packages/nova/conductor/manager.py", line 800, in _schedule_instances return_alternates=return_alternates) File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations instance_uuids, return_objects, return_alternates) File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations return cctxt.call(ctxt, 'select_destinations', **msg_args) File "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 178, in call retry=self.retry) File "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/transport.py", line 128, in _send retry=retry) File "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 645, in send call_monitor_timeout, retry=retry) File "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 636, in _send raise result
* * *Logs from Nova Scheduler*
2019-09-23 17:37:56.537 1 INFO nova.filters [req-f57095ac-6539-4efb-a6a5-f27aebb0a27c edbcf8808d8a4f48a39b8eebb3951292 8ae61d9905d94b8aa66193b20a3ab973 - default default] Filter NUMATopologyFilter returned 0 hosts 2019-09-23 17:37:56.537 1 INFO nova.filters [req-f57095ac-6539-4efb-a6a5-f27aebb0a27c edbcf8808d8a4f48a39b8eebb3951292 8ae61d9905d94b8aa66193b20a3ab973 - default default] Filter NUMATopologyFilter returned 0 hosts 2019-09-23 17:37:56.538 1 INFO nova.filters [req-f57095ac-6539-4efb-a6a5-f27aebb0a27c edbcf8808d8a4f48a39b8eebb3951292 8ae61d9905d94b8aa66193b20a3ab973 - default default] Filtering removed all hosts for the request with instance ID '66175096-37a4-4a2c-b23a-e7a4a6f10ca1'. Filter results: ['RetryFilter: (start: 1, end: 1)', 'ComputeFilter: (start: 1, end: 1)', 'AvailabilityZoneFilter: (start: 1, end: 1)', 'AggregateInstanceExtraSpecsFilter: (start: 1, end: 1)', 'ComputeCapabilitiesFilter: (start: 1, end: 1)', 'ImagePropertiesFilter: (start: 1, end: 1)', 'NUMATopologyFilter: (start: 1, end: 0)'] 2019-09-23 17:37:56.538 1 INFO nova.filters [req-f57095ac-6539-4efb-a6a5-f27aebb0a27c edbcf8808d8a4f48a39b8eebb3951292 8ae61d9905d94b8aa66193b20a3ab973 - default default] Filtering removed all hosts for the request with instance ID '66175096-37a4-4a2c-b23a-e7a4a6f10ca1'. Filter results: ['RetryFilter: (start: 1, end: 1)', 'ComputeFilter: (start: 1, end: 1)', 'AvailabilityZoneFilter: (start: 1, end: 1)', 'AggregateInstanceExtraSpecsFilter: (start: 1, end: 1)', 'ComputeCapabilitiesFilter: (start: 1, end: 1)', 'ImagePropertiesFilter: (start: 1, end: 1)', 'NUMATopologyFilter: (start: 1, end: 0)']
*Logs from Nova Conductor*
2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager Traceback (most recent call last): 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/nova/conductor/manager.py", line 1346, in schedule_and_build_instances 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager instance_uuids, return_alternates=True) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/nova/conductor/manager.py", line 800, in _schedule_instances 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager return_alternates=return_alternates) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager instance_uuids, return_objects, return_alternates) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 178, in call 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager retry=self.retry) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/transport.py", line 128, in _send 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager retry=retry) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 645, in send 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager call_monitor_timeout, retry=retry) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 636, in _send 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager raise result 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager NoValidHost_Remote: No valid host was found. There are not enough hosts available. 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager Traceback (most recent call last): 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 229, in inner 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager return func(*args, **kwargs) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/manager.py", line 168, in select_destinations 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager allocation_request_version, return_alternates) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 96, in select_destinations 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager allocation_request_version, return_alternates) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 265, in _schedule 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager claimed_instance_uuids) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 302, in _ensure_sufficient_hosts 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager NoValidHost: No valid host was found. There are not enough hosts available. 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager [req-f57095ac-6539-4efb-a6a5-f27aebb0a27c edbcf8808d8a4f48a39b8eebb3951292 8ae61d9905d94b8aa66193b20a3ab973 - default default] Failed to schedule instances: NoValidHost_Remote: No valid host was found. There are not enough hosts available. Traceback (most recent call last):
File "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 229, in inner return func(*args, **kwargs)
File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/manager.py", line 168, in select_destinations allocation_request_version, return_alternates)
File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 96, in select_destinations allocation_request_version, return_alternates)
File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 265, in _schedule claimed_instance_uuids)
File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 302, in _ensure_sufficient_hosts raise exception.NoValidHost(reason=reason)
NoValidHost: No valid host was found. There are not enough hosts available. 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager Traceback (most recent call last): 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/nova/conductor/manager.py", line 1346, in schedule_and_build_instances 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager instance_uuids, return_alternates=True) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/nova/conductor/manager.py", line 800, in _schedule_instances 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager return_alternates=return_alternates) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager instance_uuids, return_objects, return_alternates) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 178, in call 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager retry=self.retry) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/transport.py", line 128, in _send 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager retry=retry) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 645, in send 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager call_monitor_timeout, retry=retry) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 636, in _send 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager raise result 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager NoValidHost_Remote: No valid host was found. There are not enough hosts available. 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager Traceback (most recent call last): 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 229, in inner 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager return func(*args, **kwargs) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/manager.py", line 168, in select_destinations 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager allocation_request_version, return_alternates) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 96, in select_destinations 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager allocation_request_version, return_alternates) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 265, in _schedule 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager claimed_instance_uuids) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 302, in _ensure_sufficient_hosts 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager NoValidHost: No valid host was found. There are not enough hosts available. 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 2019-09-23 17:37:56.542 1 ERROR nova.conductor.manager 2019-09-23 17:37:56.627 1 WARNING nova.scheduler.utils [req-f57095ac-6539-4efb-a6a5-f27aebb0a27c edbcf8808d8a4f48a39b8eebb3951292 8ae61d9905d94b8aa66193b20a3ab973 - default default] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. Traceback (most recent call last):
File "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 229, in inner return func(*args, **kwargs)
File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/manager.py", line 168, in select_destinations allocation_request_version, return_alternates)
File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 96, in select_destinations allocation_request_version, return_alternates)
File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 265, in _schedule claimed_instance_uuids)
File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 302, in _ensure_sufficient_hosts raise exception.NoValidHost(reason=reason)
NoValidHost: No valid host was found. There are not enough hosts available. : NoValidHost_Remote: No valid host was found. There are not enough hosts available. 2019-09-23 17:37:56.627 1 WARNING nova.scheduler.utils [req-f57095ac-6539-4efb-a6a5-f27aebb0a27c edbcf8808d8a4f48a39b8eebb3951292 8ae61d9905d94b8aa66193b20a3ab973 - default default] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. Traceback (most recent call last):
File "/var/lib/openstack/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 229, in inner return func(*args, **kwargs)
File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/manager.py", line 168, in select_destinations allocation_request_version, return_alternates)
File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 96, in select_destinations allocation_request_version, return_alternates)
File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 265, in _schedule claimed_instance_uuids)
File "/var/lib/openstack/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 302, in _ensure_sufficient_hosts raise exception.NoValidHost(reason=reason)
NoValidHost: No valid host was found. There are not enough hosts available. : NoValidHost_Remote: No valid host was found. There are not enough hosts available. 2019-09-23 17:37:56.630 1 WARNING nova.scheduler.utils [req-f57095ac-6539-4efb-a6a5-f27aebb0a27c edbcf8808d8a4f48a39b8eebb3951292 8ae61d9905d94b8aa66193b20a3ab973 - default default] [instance: 66175096-37a4-4a2c-b23a-e7a4a6f10ca1] Setting instance to ERROR state.: NoValidHost_Remote: No valid host was found. There are not enough hosts available. 2019-09-23 17:37:56.630 1 WARNING nova.scheduler.utils [req-f57095ac-6539-4efb-a6a5-f27aebb0a27c edbcf8808d8a4f48a39b8eebb3951292 8ae61d9905d94b8aa66193b20a3ab973 - default default] [instance: 66175096-37a4-4a2c-b23a-e7a4a6f10ca1] Setting instance to ERROR state.
How can i change Nova log mode to DEBUG?
Anything more i can check?
Regards, Mariano
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss