Hi I checked this issue, alex reproduced. https://bugs.launchpad.net/starlingx/+bug/1918420 - Alarm 750.003 - Application remove failure for stx-openstack for multi node, remove stx-openstack will trigger hook _delete_app_specific_resources_post_remove. In this function, it will remove namespace openstack. But when there is one controller offline, the pod on this node in etcd is terminating status. So delete namespace will fail with timeout, exception log with "unexpected items still remain in namespace: openstack" kube-controller-manager-controller-0_kube-system_kube-controller-manager-649d78fb4a9150091be9f0f5cf0110d76efadd6b15c5b099b0fc530b66a90776.log:234:2021-05-11T11:09:22.880044863Z stderr F E0511 11:09:22.879841 1 namespace_controller.go:162] deletion of namespace openstack failed: unexpected items still remain in namespace: openstack for gvr: /v1, Resource=pods So I propose we forbit application-remove stx-openstack, request all nodes must be available. Thanks! Martin, Chen IOTG, Software Engineer 021-61164330 -----Original Message----- From: starlingx-discuss-request@lists.starlingx.io <starlingx-discuss-request@lists.starlingx.io> Sent: Tuesday, May 11, 2021 8:30 PM To: starlingx-discuss@lists.starlingx.io Subject: Starlingx-discuss Digest, Vol 36, Issue 30 Send Starlingx-discuss mailing list submissions to starlingx-discuss@lists.starlingx.io To subscribe or unsubscribe via the World Wide Web, visit http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss or, via email, send a message with subject or body 'help' to starlingx-discuss-request@lists.starlingx.io You can reach the person managing the list at starlingx-discuss-owner@lists.starlingx.io When replying, please edit your Subject line so it is more specific than "Re: Contents of Starlingx-discuss digest..." Today's Topics: 1. Sanity Master Test LAYERED build ISO 20210511T040437Z (Dimofte, Alexandru) ---------------------------------------------------------------------- Message: 1 Date: Tue, 11 May 2021 12:22:02 +0000 From: "Dimofte, Alexandru" <alexandru.dimofte@intel.com> To: "starlingx-discuss@lists.starlingx.io" <starlingx-discuss@lists.starlingx.io> Subject: [Starlingx-discuss] Sanity Master Test LAYERED build ISO 20210511T040437Z Message-ID: <BYAPR11MB2742384B8F6F867D7E89ACBAE5539@BYAPR11MB2742.namprd11.prod.outlook.com> Content-Type: text/plain; charset="utf-8" Sanity Test from 2021-May-11 (http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/flock/202105...) Status: YELLOW The Setup and Provision are PASSING. This might be related to: https://bugs.launchpad.net/starlingx/+bug/1918420 - Alarm 750.003 - Application remove failure for stx-openstack Helm-Chart used: http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/flock/202105... Regards, Alexandru Dimofte [Logo Description automatically generated] Dimofte Alexandru Software Engineer STARLINGX TEAM Skype no: +40 336403734 Personal Mobile: +40 743167456 alexandru.dimofte@intel.com<mailto:alexandru.dimofte@intel.com> Intel Romania -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20210511/4bd87fee/attachment-0001.html> -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 8413 bytes Desc: image001.png URL: <http://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20210511/4bd87fee/attachment-0001.png> ------------------------------ Subject: Digest Footer _______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss ------------------------------ End of Starlingx-discuss Digest, Vol 36, Issue 30 *************************************************