[Starlingx-discuss] [stx-config] platform-integ-apps was applying with a long time
Hi guys, When I use sysinv to bring up the containerized services, platform-integ-apps is stay applying for a long time. I wonder it may be caused by recent poweroff. Now I have reactived the controllers, and can you tell me how to re-apply it? [root@controller-0 log(keystone_admin)]# system host-list +----+--------------+-------------+----------------+-------------+--------------+ | id | hostname | personality | administrative | operational | availability | +----+--------------+-------------+----------------+-------------+--------------+ | 1 | controller-0 | controller | unlocked | enabled | available | | 2 | controller-1 | controller | unlocked | enabled | available | +----+--------------+-------------+----------------+-------------+--------------+ [root@controller-0 log(keystone_admin)]# system application-list +---------------------+---------------------------+-------------------------------+--------------------+-----------+------------------------------------------------------------------+ | application | version | manifest name | manifest file | status | progress | +---------------------+---------------------------+-------------------------------+--------------------+-----------+------------------------------------------------------------------+ | platform-integ-apps | 1.0-7 | platform-integration-manifest | manifest.yaml | applying | processing chart: stx-rbd-provisioner, overall completion: 50.0% | | stx-openstack | 1.0-16-centos-stable- | armada-manifest | stx-openstack.yaml | uploading | validating and uploading charts | | | latest | | | | | | | | | | | | +---------------------+---------------------------+-------------------------------+--------------------+-----------+—————————————————————————————————+ Thanks Kunpeng
Dear Kunpeng, Your progress status is applying. If it shows apply-failed status, then do re-apply. Command: # system application-apply platform-integ-apps Thanks 张鲲鹏 <zhang.kunpeng@99cloud.net> 於 2019年6月28日 週五 上午11:48寫道：
Hi guys,
When I use sysinv to bring up the containerized services, platform-integ-apps is stay applying for a long time. I wonder it may be caused by recent poweroff. Now I have reactived the controllers, and can you tell me how to re-apply it?
[root@controller-0 log(keystone_admin)]# system host-list
+----+--------------+-------------+----------------+-------------+--------------+ | id | hostname | personality | administrative | operational | availability |
+----+--------------+-------------+----------------+-------------+--------------+ | 1 | controller-0 | controller | unlocked | enabled | available | | 2 | controller-1 | controller | unlocked | enabled | available |
+----+--------------+-------------+----------------+-------------+--------------+ [root@controller-0 log(keystone_admin)]# system application-list
+---------------------+---------------------------+-------------------------------+--------------------+-----------+------------------------------------------------------------------+ | application | version | manifest name | manifest file | status | progress |
+---------------------+---------------------------+-------------------------------+--------------------+-----------+------------------------------------------------------------------+ | platform-integ-apps | 1.0-7 | platform-integration-manifest | manifest.yaml | applying | processing chart: stx-rbd-provisioner, overall completion: 50.0% | | stx-openstack | 1.0-16-centos-stable- | armada-manifest | stx-openstack.yaml | uploading | validating and uploading charts | | | latest | | | | | | | | | | | |
+---------------------+---------------------------+-------------------------------+--------------------+-----------+—————————————————————————————————+
Thanks Kunpeng _______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Hi Kunpeng, You could check /var/log/armada/platform-integ-apps-apply.log, to get details where the process stuck at. Best Regards Shuicheng From: Ezpeer Chen [mailto:ezpeerchen@gmail.com] Sent: Friday, June 28, 2019 3:30 PM To: 张鲲鹏 <zhang.kunpeng@99cloud.net> Cc: starlingx-discuss@lists.starlingx.io Subject: Re: [Starlingx-discuss] [stx-config] platform-integ-apps was applying with a long time Dear Kunpeng, Your progress status is applying. If it shows apply-failed status, then do re-apply. Command: # system application-apply platform-integ-apps Thanks 张鲲鹏 <zhang.kunpeng@99cloud.net<mailto:zhang.kunpeng@99cloud.net>> 於 2019年6月28日 週五 上午11:48寫道： Hi guys, When I use sysinv to bring up the containerized services, platform-integ-apps is stay applying for a long time. I wonder it may be caused by recent poweroff. Now I have reactived the controllers, and can you tell me how to re-apply it? [root@controller-0 log(keystone_admin)]# system host-list +----+--------------+-------------+----------------+-------------+--------------+ | id | hostname | personality | administrative | operational | availability | +----+--------------+-------------+----------------+-------------+--------------+ | 1 | controller-0 | controller | unlocked | enabled | available | | 2 | controller-1 | controller | unlocked | enabled | available | +----+--------------+-------------+----------------+-------------+--------------+ [root@controller-0 log(keystone_admin)]# system application-list +---------------------+---------------------------+-------------------------------+--------------------+-----------+------------------------------------------------------------------+ | application | version | manifest name | manifest file | status | progress | +---------------------+---------------------------+-------------------------------+--------------------+-----------+------------------------------------------------------------------+ | platform-integ-apps | 1.0-7 | platform-integration-manifest | manifest.yaml | applying | processing chart: stx-rbd-provisioner, overall completion: 50.0% | | stx-openstack | 1.0-16-centos-stable- | armada-manifest | stx-openstack.yaml | uploading | validating and uploading charts | | | latest | | | | | | | | | | | | +---------------------+---------------------------+-------------------------------+--------------------+-----------+—————————————————————————————————+ Thanks Kunpeng _______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io<mailto:Starlingx-discuss@lists.starlingx.io> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Hi Kunpeng, I met the same issue before. As a result of poweroff, the status will always be applying because the job has been killed and the status in the database won’t be changed from applying to apply-failed. What’s more, you can’t re-apply because the status is applying. I changed the status in the database and then re-apply. Hope the following commands can be useful: cat /etc/sysinv/sysinv.conf | grep sql connection=postgresql+psycopg2://admin-sysinv:d0df960de712Ti0*@192.168.204.2/sysinv psql -h 192.168.204.2 -U admin-sysinv -W sysinv d0df960de712Ti0* select * from kube_app; update kube_app set status=’apply-failed’ where name=’platform-integ-apps’; select * from kube_app; \q system application-apply platform-integ-apps Best Regards, Xu, Chenjie From: Lin, Shuicheng [mailto:shuicheng.lin@intel.com] Sent: Friday, June 28, 2019 3:43 PM To: Ezpeer Chen <ezpeerchen@gmail.com>; 张鲲鹏 <zhang.kunpeng@99cloud.net> Cc: starlingx-discuss@lists.starlingx.io Subject: Re: [Starlingx-discuss] [stx-config] platform-integ-apps was applying with a long time Hi Kunpeng, You could check /var/log/armada/platform-integ-apps-apply.log, to get details where the process stuck at. Best Regards Shuicheng From: Ezpeer Chen [mailto:ezpeerchen@gmail.com] Sent: Friday, June 28, 2019 3:30 PM To: 张鲲鹏 <zhang.kunpeng@99cloud.net<mailto:zhang.kunpeng@99cloud.net>> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] [stx-config] platform-integ-apps was applying with a long time Dear Kunpeng, Your progress status is applying. If it shows apply-failed status, then do re-apply. Command: # system application-apply platform-integ-apps Thanks 张鲲鹏 <zhang.kunpeng@99cloud.net<mailto:zhang.kunpeng@99cloud.net>> 於 2019年6月28日 週五 上午11:48寫道： Hi guys, When I use sysinv to bring up the containerized services, platform-integ-apps is stay applying for a long time. I wonder it may be caused by recent poweroff. Now I have reactived the controllers, and can you tell me how to re-apply it? [root@controller-0 log(keystone_admin)]# system host-list +----+--------------+-------------+----------------+-------------+--------------+ | id | hostname | personality | administrative | operational | availability | +----+--------------+-------------+----------------+-------------+--------------+ | 1 | controller-0 | controller | unlocked | enabled | available | | 2 | controller-1 | controller | unlocked | enabled | available | +----+--------------+-------------+----------------+-------------+--------------+ [root@controller-0 log(keystone_admin)]# system application-list +---------------------+---------------------------+-------------------------------+--------------------+-----------+------------------------------------------------------------------+ | application | version | manifest name | manifest file | status | progress | +---------------------+---------------------------+-------------------------------+--------------------+-----------+------------------------------------------------------------------+ | platform-integ-apps | 1.0-7 | platform-integration-manifest | manifest.yaml | applying | processing chart: stx-rbd-provisioner, overall completion: 50.0% | | stx-openstack | 1.0-16-centos-stable- | armada-manifest | stx-openstack.yaml | uploading | validating and uploading charts | | | latest | | | | | | | | | | | | +---------------------+---------------------------+-------------------------------+--------------------+-----------+—————————————————————————————————+ Thanks Kunpeng _______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io<mailto:Starlingx-discuss@lists.starlingx.io> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
There is an existing LP with the same cause: https://bugs.launchpad.net/starlingx/+bug/1833323 Until it is resolved, issue the following commands if the application status are stuck in applying/removing because sysinv-conductor was abruptly terminated (power outage, process restarted by sm due to missing audits, process killed/system boot due to OOM, etc…) sudo -u postgres psql -d sysinv -c "update kube_app set status='apply-failed' where status='applying';" sudo -u postgres psql -d sysinv -c "update kube_app set status='remove-failed' where status='removing';" You can then retrigger the application apply/remove. Tee From: Xu, Chenjie [mailto:chenjie.xu@intel.com] Sent: June-28-19 3:53 AM To: Lin, Shuicheng; Ezpeer Chen; 张鲲鹏 Cc: starlingx-discuss@lists.starlingx.io Subject: Re: [Starlingx-discuss] [stx-config] platform-integ-apps was applying with a long time Hi Kunpeng, I met the same issue before. As a result of poweroff, the status will always be applying because the job has been killed and the status in the database won’t be changed from applying to apply-failed. What’s more, you can’t re-apply because the status is applying. I changed the status in the database and then re-apply. Hope the following commands can be useful: cat /etc/sysinv/sysinv.conf | grep sql connection=postgresql+psycopg2://admin-sysinv:d0df960de712Ti0*@192.168.204.2/sysinv psql -h 192.168.204.2 -U admin-sysinv -W sysinv d0df960de712Ti0* select * from kube_app; update kube_app set status=’apply-failed’ where name=’platform-integ-apps’; select * from kube_app; \q system application-apply platform-integ-apps Best Regards, Xu, Chenjie From: Lin, Shuicheng [mailto:shuicheng.lin@intel.com] Sent: Friday, June 28, 2019 3:43 PM To: Ezpeer Chen <ezpeerchen@gmail.com>; 张鲲鹏 <zhang.kunpeng@99cloud.net> Cc: starlingx-discuss@lists.starlingx.io Subject: Re: [Starlingx-discuss] [stx-config] platform-integ-apps was applying with a long time Hi Kunpeng, You could check /var/log/armada/platform-integ-apps-apply.log, to get details where the process stuck at. Best Regards Shuicheng From: Ezpeer Chen [mailto:ezpeerchen@gmail.com] Sent: Friday, June 28, 2019 3:30 PM To: 张鲲鹏 <zhang.kunpeng@99cloud.net<mailto:zhang.kunpeng@99cloud.net>> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] [stx-config] platform-integ-apps was applying with a long time Dear Kunpeng, Your progress status is applying. If it shows apply-failed status, then do re-apply. Command: # system application-apply platform-integ-apps Thanks 张鲲鹏 <zhang.kunpeng@99cloud.net<mailto:zhang.kunpeng@99cloud.net>> 於 2019年6月28日 週五 上午11:48寫道： Hi guys, When I use sysinv to bring up the containerized services, platform-integ-apps is stay applying for a long time. I wonder it may be caused by recent poweroff. Now I have reactived the controllers, and can you tell me how to re-apply it? [root@controller-0 log(keystone_admin)]# system host-list +----+--------------+-------------+----------------+-------------+--------------+ | id | hostname | personality | administrative | operational | availability | +----+--------------+-------------+----------------+-------------+--------------+ | 1 | controller-0 | controller | unlocked | enabled | available | | 2 | controller-1 | controller | unlocked | enabled | available | +----+--------------+-------------+----------------+-------------+--------------+ [root@controller-0 log(keystone_admin)]# system application-list +---------------------+---------------------------+-------------------------------+--------------------+-----------+------------------------------------------------------------------+ | application | version | manifest name | manifest file | status | progress | +---------------------+---------------------------+-------------------------------+--------------------+-----------+------------------------------------------------------------------+ | platform-integ-apps | 1.0-7 | platform-integration-manifest | manifest.yaml | applying | processing chart: stx-rbd-provisioner, overall completion: 50.0% | | stx-openstack | 1.0-16-centos-stable- | armada-manifest | stx-openstack.yaml | uploading | validating and uploading charts | | | latest | | | | | | | | | | | | +---------------------+---------------------------+-------------------------------+--------------------+-----------+—————————————————————————————————+ Thanks Kunpeng _______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io<mailto:Starlingx-discuss@lists.starlingx.io> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
participants (5)
-
Ezpeer Chen
-
Lin, Shuicheng
-
Ngo, Tee
-
Xu, Chenjie
-
张鲲鹏