Thanks. Here it is :

https://bugs.launchpad.net/starlingx/+bug/1875914

Thanks.

BR
Austin Sun.

From: Ning, Antai (Andy) <Andy.Ning@windriver.com>
Sent: Wednesday, April 29, 2020 11:05 PM
To: Sun, Austin <austin.sun@intel.com>; starlingx-discuss@lists.starlingx.io
Cc: Jones, Bruce E <bruce.e.jones@intel.com>
Subject: Re: [Starlingx-discuss] [starlingx] cinder bootstrap failed on subcloud

It looks like a dns issue. "keystone-admin.openstack.svc.cluster.subcloud1" is not resolvable.

coredns should be able to resolve this FQDN, but looks like it can't. So it can be bug. You may want to raise a LP for it.

Thanks,

Andy

From: Sun, Austin <austin.sun@intel.com>
Sent: Wednesday, April 29, 2020 10:15 AM
To: Ning, Antai (Andy) <Andy.Ning@windriver.com>; starlingx-discuss@lists.starlingx.io <starlingx-discuss@lists.starlingx.io>
Cc: Jones, Bruce E <bruce.e.jones@intel.com>
Subject: RE: [Starlingx-discuss] [starlingx] cinder bootstrap failed on subcloud

Hi Andy :

Thanks your reply.

Some output

currently , the result of ‘openstack endpoint list’ in cinder-bootstrap is

[root@cinder-bootstrap-rtxf7 /]# openstack endpoint list | grep identity

controller-0:~$ kubectl get ingress --all-namespaces

NAMESPACE NAME HOSTS ADDRESS PORTS AGE

openstack cinder cinder,cinder.openstack,cinder.openstack.svc.cluster.local 80 31h

openstack glance glance,glance.openstack,glance.openstack.svc.cluster.local 80 31h

openstack keystone keystone,keystone.openstack,keystone.openstack.svc.cluster.local 80 31h

openstack keystone-admin keystone-admin,keystone-admin.openstack,keystone-admin.openstack.svc.cluster.local 80 31h

openstack keystone-admin-cluster-fqdn keystone-admin.openstack.svc.cluster.subcloud1 80 31h

openstack keystone-admin-namespace-fqdn keystone-admin.openstack.svc.cluster.subcloud1 80 31h

openstack openstack-osh-openstack-ingress *.openstack.svc.cluster.local 80 31h

openstack osh-openstac-mgr-e50215 osh-openstac-mgr-e50215,osh-openstac-mgr-e50215.openstack,osh-openstac-mgr-e50215.openstack.svc.cluster.local 80 31h

‘ kubectl get ingress –openstack’ run with error

But

controller-0:~$ kubectl get ingress -n openstack

NAME HOSTS ADDRESS PORTS AGE

cinder cinder,cinder.openstack,cinder.openstack.svc.cluster.local 80 31h

glance glance,glance.openstack,glance.openstack.svc.cluster.local 80 31h

keystone keystone,keystone.openstack,keystone.openstack.svc.cluster.local 80 31h

keystone-admin keystone-admin,keystone-admin.openstack,keystone-admin.openstack.svc.cluster.local 80 31h

keystone-admin-cluster-fqdn keystone-admin.openstack.svc.cluster.subcloud1 80 31h

keystone-admin-namespace-fqdn keystone-admin.openstack.svc.cluster.subcloud1 80 31h

openstack-osh-openstack-ingress *.openstack.svc.cluster.local 80 31h

osh-openstac-mgr-e50215 osh-openstac-mgr-e50215,osh-openstac-mgr-e50215.openstack,osh-openstac-mgr-e50215.openstack.svc.cluster.local 80 31h

But keystone-admin.openstack.svc.cluster.subcloud1 cannot be found from host or container :

controller-0:~$ curl http://keystone-admin.openstack.svc.cluster.subcloud1

curl: (6) Could not resolve host: keystone-admin.openstack.svc.cluster.subcloud1; Name or service not known

[root@cinder-bootstrap-rtxf7 /]# curl http://keystone-admin.openstack.svc.cluster.subcloud1

curl: (6) Could not resolve host: keystone-admin.openstack.svc.cluster.subcloud1; Unknown error

Thanks.

BR
Austin Sun.

From: Ning, Antai (Andy) <Andy.Ning@windriver.com>
Sent: Wednesday, April 29, 2020 9:07 PM
To: Sun, Austin <austin.sun@intel.com>; starlingx-discuss@lists.starlingx.io
Cc: Jones, Bruce E <bruce.e.jones@intel.com>
Subject: Re: [Starlingx-discuss] [starlingx] cinder bootstrap failed on subcloud

Hi Austin,

What's the output of "openstack endpoint list" on containerized openstack like?

keystone-admin should be show up as identity's admin endpoint .... if not, that could be a bug.

Also what's the output of "kubectl get ingress --all-namespaces" or "kubectl get ingress --openstack"? I want to see whether the ingress for keystone-admin is created or not.

Andy

From: Sun, Austin <austin.sun@intel.com>
Sent: Tuesday, April 28, 2020 4:34 AM
To: Ning, Antai (Andy) <Andy.Ning@windriver.com>; starlingx-discuss@lists.starlingx.io <starlingx-discuss@lists.starlingx.io>
Cc: Jones, Bruce E <bruce.e.jones@intel.com>
Subject: FW: [Starlingx-discuss] [starlingx] cinder bootstrap failed on subcloud

Hi Andy:

I deployed dist-cloud in virtual env based on 20200414, but openstack application on subcloud deployed failed .

From cinder-api po logs:

>: Failed to establish a new connection: [Errno -2] Name or service not known',)): ConnectFailure: Unable to establish connection to http://keystone-admin.openstack.svc.cluster.subcloud1:80/v3/auth/tokens: HTTPConnectionPool(host='keystone-admin.openstack.svc.cluster.subcloud1', port=80): Max retries exceeded with url: /v3/auth/tokens (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc6beac6d10>: Failed to establish a new connection: [Errno -2] Name or service not known',))

This keystone-admin.openstack.svc.cluster.subcloud1 can not be found in subcloud1 controllers

This url seems override from helm keystone:

host_fqdn_override:

admin:

host: keystone-admin.openstack.svc.cluster.subcloud1

public: {}

and introduced in commit [1] by you.

is these any additional config needed to bring up openstack on subcloud besides the deploy guide [2]?

[1] https://review.opendev.org/#/c/697361/1

[2] https://docs.starlingx.io/deploy_install_guides/r3_release/distributed_cloud/index.html

Thanks.

BR
Austin Sun.

From: Sun, Austin <austin.sun@intel.com>
Sent: Saturday, April 18, 2020 9:06 AM
To: Jones, Bruce E <bruce.e.jones@intel.com>; starlingx-discuss@lists.starlingx.io
Subject: Re: [Starlingx-discuss] [starlingx] cinder bootstrap failed on subcloud

Hi Bruce:

No, for stx.3.0 the openstack on subcloud works well . central cloud does not support openstack .

Thanks.

BR
Austin Sun.

From: Jones, Bruce E <bruce.e.jones@intel.com>
Sent: Friday, April 17, 2020 11:01 PM
To: Sun, Austin <austin.sun@intel.com>; starlingx-discuss@lists.starlingx.io
Subject: RE: [starlingx] cinder bootstrap failed on subcloud

I thought that dist-cloud sub-nodes are Kubernetes only – without OpenStack support…

brucej

From: Sun, Austin [mailto:austin.sun@intel.com]
Sent: Friday, April 17, 2020 5:45 AM
To: starlingx-discuss@lists.starlingx.io
Subject: [Starlingx-discuss] [starlingx] cinder bootstrap failed on subcloud

Hi community:

I deployed dist-cloud in virtual env (20200414) , but the openstack application applied failed. The reason is cinder-bootstrap job failed.

The logs are

[root@cinder-bootstrap-rkp7j /]# openstack volume type show ceph-store

The server is currently unavailable. Please try again at a later time.<br /><br />

The Keystone service is temporarily unavailable.

(HTTP 503)

But openstack endpoint list works fine on this pod

Does any one meet same issue on recently ?

Thanks.

BR
Austin Sun.