Hi All, We installed the 4.0.1 from (http://mirror.starlingx.cengn.ca/mirror/starlingx/release/4.0.1/centos/flock...) on a Virtual Standard configuration in two scenarios: downloading the images from internet and using our mirror registry that already has the images. We managed to fully install and execute sanity on both cases. We are not able to replicate your issue. Regards, Nicolae Jascanu, Ph.D. Software Engineer IOTG Galati, Romania From: Kalvala, Haridhar <haridhar.kalvala@intel.com> Sent: Thursday, February 4, 2021 05:16 To: starlingx-discuss@lists.starlingx.io Cc: Kumar, Sharath <sharath.kumar@intel.com> Subject: Re: [Starlingx-discuss] Default Helm-chart apply failing (4.0.1) Hello All, Any update/comment on below issue. It still persists with any version stx-openstack app(Ex: stx-openstack-1.0-49-centos-stable-versioned.tgz) ? Thank you, Haridhar Kalvala Pasting error log again : ksysadmin@controller-0 ~(keystone_admin)]$ system application-apply stx-openstack +---------------+----------------------------------+ | Property | Value | +---------------+----------------------------------+ | active | False | | app_version | 1.0-49-centos-stable-versioned | | created_at | 2021-02-04T11:08:55.408108+00:00 | | manifest_file | stx-openstack.yaml | | manifest_name | armada-manifest | | name | stx-openstack | | progress | None | | status | applying | | updated_at | 2021-02-04T11:09:20.815085+00:00 | +---------------+----------------------------------+ Please use 'system application-list' or 'system application-show stx-openstack' to view the current progress. [sysadmin@controller-0 ~(keystone_admin)]$ tail -f /var/log/sysinv.log 2021-02-04 11:09:39.231 99042 ERROR sysinv.openstack.common.rpc.amqp app_applied = self._app.perform_app_apply(rpc_app, mode, lifecycle_hook_info_app_apply) 2021-02-04 11:09:39.231 99042 ERROR sysinv.openstack.common.rpc.amqp File "/usr/lib64/python2.7/site-packages/sysinv/conductor/kube_app.py", line 1995, in perform_app_apply 2021-02-04 11:09:39.231 99042 ERROR sysinv.openstack.common.rpc.amqp name=app.name, version=app.version, reason=e) 2021-02-04 11:09:39.231 99042 ERROR sysinv.openstack.common.rpc.amqp KubeAppApplyFailure: Deployment of application stx-openstack (1.0-49-centos-stable-versioned) failed: (404) 2021-02-04 11:09:39.231 99042 ERROR sysinv.openstack.common.rpc.amqp Reason: Not Found 2021-02-04 11:09:39.231 99042 ERROR sysinv.openstack.common.rpc.amqp HTTP response headers: HTTPHeaderDict({'Date': 'Thu, 04 Feb 2021 11:09:38 GMT', 'Content-Length': '210', 'Content-Type': 'application/json'}) 2021-02-04 11:09:39.231 99042 ERROR sysinv.openstack.common.rpc.amqp HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \"ceph-pool-kube-rbd\" not found","reason":"NotFound","details":{"name":"ceph-pool-kube-rbd","kind":"secrets"},"code":404} 2021-02-04 11:09:39.231 99042 ERROR sysinv.openstack.common.rpc.amqp 2021-02-04 11:09:39.231 99042 ERROR sysinv.openstack.common.rpc.amqp 2021-02-04 11:09:39.231 99042 ERROR sysinv.openstack.common.rpc.amqp sysinv 2021-02-04 11:10:20.070 99042 INFO sysinv.api.controllers.v1.rest_api [-] GET cmd:http://localhost:30001/nfvi-plugins/v1/sw-update hdr:{'Content-type': 'application/json', 'User-Agent': 'sysinv/1.0'} payload:None sysinv 2021-02-04 11:10:20.071 99042 INFO sysinv.api.controllers.v1.rest_api [-] Response={u'status': u'success', u'in-progress': None, u'sw-update-type': None} sysinv 2021-02-04 11:10:20.117 99042 INFO sysinv.conductor.kube_app [-] lifecycle hook for application platform-integ-apps (1.0-9) started {'lifecycle_type': u'check', 'relative_timing': u'pre', 'mode': u'auto', 'operation': u'apply', 'extra': {}}. sysinv 2021-02-04 11:10:20.117 99042 INFO sysinv.conductor.manager [-] Auto-apply failed prerequisites for platform-integ-apps: Automatic apply is disabled for platform-integ-apps. sysinv 2021-02-04 11:10:20.122 99042 INFO sysinv.conductor.kube_app [-] lifecycle hook for application oidc-auth-apps (1.0-29) started {'lifecycle_type': u'check', 'relative_timing': u'pre', 'mode': u'auto', 'operation': u'apply', 'extra': {}}. sysinv 2021-02-04 11:10:20.123 99042 INFO sysinv.conductor.manager [-] Auto-apply failed prerequisites for oidc-auth-apps: Automatic apply is disabled for oidc-auth-apps. sysinv 2021-02-04 11:10:20.131 99042 INFO sysinv.api.controllers.v1.rest_api [-] GET cmd:http://localhost:30001/nfvi-plugins/v1/sw-update hdr:{'Content-type': 'application/json', 'User-Agent': 'sysinv/1.0'} payload:None sysinv 2021-02-04 11:10:20.132 99042 INFO sysinv.api.controllers.v1.rest_api [-] Response={u'status': u'success', u'in-progress': None, u'sw-update-type': None} From: Kalvala, Haridhar Sent: Wednesday, February 3, 2021 4:39 PM To: Kumar, Sharath <sharath.kumar@intel.com<mailto:sharath.kumar@intel.com>>; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: RE: Default Helm-chart apply failing (4.0.1) Even I am observing the same since morning (IST time zone). Error log: 2021-02-03 18:37:19.530 99042 ERROR sysinv.openstack.common.rpc.amqp app_applied = self._app.perform_app_apply(rpc_app, mode, lifecycle_hook_info_app_apply) 2021-02-03 18:37:19.530 99042 ERROR sysinv.openstack.common.rpc.amqp File "/usr/lib64/python2.7/site-packages/sysinv/conductor/kube_app.py", line 1995, in perform_app_apply 2021-02-03 18:37:19.530 99042 ERROR sysinv.openstack.common.rpc.amqp name=app.name, version=app.version, reason=e) 2021-02-03 18:37:19.530 99042 ERROR sysinv.openstack.common.rpc.amqp KubeAppApplyFailure: Deployment of application stx-openstack (1.0-49-centos-stable-versioned) failed: (404) 2021-02-03 18:37:19.530 99042 ERROR sysinv.openstack.common.rpc.amqp Reason: Not Found 2021-02-03 18:37:19.530 99042 ERROR sysinv.openstack.common.rpc.amqp HTTP response headers: HTTPHeaderDict({'Date': 'Wed, 03 Feb 2021 18:37:19 GMT', 'Content-Length': '210', 'Content-Type': 'application/json'}) 2021-02-03 18:37:19.530 99042 ERROR sysinv.openstack.common.rpc.amqp HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"secrets \"ceph-pool-kube-rbd\" not found","reason":"NotFound","details":{"name":"ceph-pool-kube-rbd ","kind":"secrets"},"code":404} 2021-02-03 18:37:19.530 99042 ERROR sysinv.openstack.common.rpc.amqp 2021-02-03 18:37:19.530 99042 ERROR sysinv.openstack.common.rpc.amqp 2021-02-03 18:37:19.530 99042 ERROR sysinv.openstack.common.rpc.amqp Regards, Haridhar Kalvala From: Kumar, Sharath <sharath.kumar@intel.com<mailto:sharath.kumar@intel.com>> Sent: Wednesday, February 3, 2021 4:14 PM To: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: [Starlingx-discuss] Default Helm-chart apply failing (4.0.1) Hi All, We are seeing stx-openstack Armanda helm chart getting failed during apply. Initial docker images are not downloading and throwing error. It was working till last night and since morning am seeing this issue. Kindly fix the issue and do the needful. Thank you in advance. Regards, Sharath