Re: [Starlingx-discuss] Sanity Master Test LAYERED build ISO 20200506T013246Z bug#1877035
Hi All: After further analysis this issue with Yan. Suspect there is bug hiding in cengn layer build. Scott : would you like to check cengn build env ? The details analysis : Latest puppet-keystone rpm built from cengn should include change [1] which was merged by April 28th.
From distro layer build 0505 [3] and latest distro output , puppet-keystone includes change [1]. But if we extract puppet-keystone in 0506 iso built from layer build [2] or latest iso [5] , it is surprised this change [1] was not included. so bug#1877035 shows-up based on 0506 and 0507 iso. If we extract puppet-keystone from 0506 iso built from monolithic [4], the change [1] was included in puppet-keystone, and ansible provision pass it.
BTW: layer build in my local setup (cross build flock or flock layer only build ) did not show this issue, so it should not be related to layer build script. [1] https://review.opendev.org/#/c/716842/ [2] http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/flock/202005... [3] http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/distro/20200... [4] http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/monolithic/2... [5] http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/flock/202005... [6] http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/distro/lates... Thanks. BR Austin Sun. From: Jascanu, Nicolae <nicolae.jascanu@intel.com> Sent: Wednesday, May 6, 2020 10:06 PM To: starlingx-discuss@lists.starlingx.io Subject: [Starlingx-discuss] Sanity Master Test LAYERED build ISO 20200506T013246Z Sanity Test from 2020-May-06 (http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/flock/202005...) Status: RED Provisioning is failing for all configurations. I've added a comment to the bug: https://bugs.launchpad.net/starlingx/+bug/1877035 We used the helm chart from http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/latest_docke... Regards, STX Validation Team
Thanks Austin for the very good analysis. The initial theory discussed on today's build call is the tis version was not updated for package [1]. Saul/Scott took the action to verify this. In the meantime, Nicolae can you run a monolithic build sanity so we can confirm if the ISO is sane or has other issues as well? Frank From: Sun, Austin <austin.sun@intel.com> Sent: Thursday, May 07, 2020 7:59 AM To: Jascanu, Nicolae <nicolae.jascanu@intel.com>; starlingx-discuss@lists.starlingx.io; Little, Scott <Scott.Little@windriver.com> Subject: Re: [Starlingx-discuss] Sanity Master Test LAYERED build ISO 20200506T013246Z bug#1877035 Hi All: After further analysis this issue with Yan. Suspect there is bug hiding in cengn layer build. Scott : would you like to check cengn build env ? The details analysis : Latest puppet-keystone rpm built from cengn should include change [1] which was merged by April 28th.
From distro layer build 0505 [3] and latest distro output , puppet-keystone includes change [1]. But if we extract puppet-keystone in 0506 iso built from layer build [2] or latest iso [5] , it is surprised this change [1] was not included. so bug#1877035 shows-up based on 0506 and 0507 iso. If we extract puppet-keystone from 0506 iso built from monolithic [4], the change [1] was included in puppet-keystone, and ansible provision pass it.
BTW: layer build in my local setup (cross build flock or flock layer only build ) did not show this issue, so it should not be related to layer build script. [1] https://review.opendev.org/#/c/716842/ [2] http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/flock/202005... [3] http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/distro/20200... [4] http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/monolithic/2... [5] http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/flock/202005... [6] http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/distro/lates... Thanks. BR Austin Sun. From: Jascanu, Nicolae <nicolae.jascanu@intel.com<mailto:nicolae.jascanu@intel.com>> Sent: Wednesday, May 6, 2020 10:06 PM To: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: [Starlingx-discuss] Sanity Master Test LAYERED build ISO 20200506T013246Z Sanity Test from 2020-May-06 (http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/flock/202005...) Status: RED Provisioning is failing for all configurations. I've added a comment to the bug: https://bugs.launchpad.net/starlingx/+bug/1877035 We used the helm chart from http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/latest_docke... Regards, STX Validation Team
Hi Frank, Scott and Saul : Does it means any changes are need update tis version to take effect in cengn build ? I just found another similar issue , change [1] was merged on april 30th , but if take containers layer build openstack tarball, the change wasn't there. The monolithic build [3] on same day included the change [1]. From my feeling , any changes in layer build w/o update tis version and needed for up layer may not be included in layer build iso or container output. [1] https://review.opendev.org/#/c/724385/1 [2] http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/containers/2... [3] http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/monolithic/2... Thanks. BR Austin Sun. From: Miller, Frank <Frank.Miller@windriver.com> Sent: Thursday, May 7, 2020 10:41 PM To: Sun, Austin <austin.sun@intel.com>; Jascanu, Nicolae <nicolae.jascanu@intel.com>; starlingx-discuss@lists.starlingx.io; Little, Scott <Scott.Little@windriver.com> Subject: RE: [Starlingx-discuss] Sanity Master Test LAYERED build ISO 20200506T013246Z bug#1877035 Thanks Austin for the very good analysis. The initial theory discussed on today's build call is the tis version was not updated for package [1]. Saul/Scott took the action to verify this. In the meantime, Nicolae can you run a monolithic build sanity so we can confirm if the ISO is sane or has other issues as well? Frank From: Sun, Austin <austin.sun@intel.com<mailto:austin.sun@intel.com>> Sent: Thursday, May 07, 2020 7:59 AM To: Jascanu, Nicolae <nicolae.jascanu@intel.com<mailto:nicolae.jascanu@intel.com>>; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>; Little, Scott <Scott.Little@windriver.com<mailto:Scott.Little@windriver.com>> Subject: Re: [Starlingx-discuss] Sanity Master Test LAYERED build ISO 20200506T013246Z bug#1877035 Hi All: After further analysis this issue with Yan. Suspect there is bug hiding in cengn layer build. Scott : would you like to check cengn build env ? The details analysis : Latest puppet-keystone rpm built from cengn should include change [1] which was merged by April 28th.
From distro layer build 0505 [3] and latest distro output , puppet-keystone includes change [1]. But if we extract puppet-keystone in 0506 iso built from layer build [2] or latest iso [5] , it is surprised this change [1] was not included. so bug#1877035 shows-up based on 0506 and 0507 iso. If we extract puppet-keystone from 0506 iso built from monolithic [4], the change [1] was included in puppet-keystone, and ansible provision pass it.
BTW: layer build in my local setup (cross build flock or flock layer only build ) did not show this issue, so it should not be related to layer build script. [1] https://review.opendev.org/#/c/716842/ [2] http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/flock/202005... [3] http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/distro/20200... [4] http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/monolithic/2... [5] http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/flock/202005... [6] http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/distro/lates... Thanks. BR Austin Sun. From: Jascanu, Nicolae <nicolae.jascanu@intel.com<mailto:nicolae.jascanu@intel.com>> Sent: Wednesday, May 6, 2020 10:06 PM To: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: [Starlingx-discuss] Sanity Master Test LAYERED build ISO 20200506T013246Z Sanity Test from 2020-May-06 (http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/flock/202005...) Status: RED Provisioning is failing for all configurations. I've added a comment to the bug: https://bugs.launchpad.net/starlingx/+bug/1877035 We used the helm chart from http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/latest_docke... Regards, STX Validation Team
Oops, ,I forgot to send email that we fixed the TIS_PATCH_VER issue in openstacklib and kerystone.! https://review.opendev.org/#/c/726219/ Hopefully we will pass sanity test tomorrow. Sau! On 5/7/20 8:35 PM, Sun, Austin wrote:
Hi Frank, Scott and Saul :
Does it means any changes are need update tis version to take effect in cengn build ?
I am not following the question here.
I just found another similar issue , change [1] was merged on april 30^th , but if take containers layer build openstack tarball, the change wasn’t there.
This change is to the manifest, not the RPM/Build version which TIS_PATCH_VER affects, essencially what happened is we had 2 packages both with the same version, but one was newer than the other. The Developer build likely worked because it did not see the older version, but on Cengn it picked the older version from the mirror. Sau!
The monolithic build [3] on same day included the change [1].
From my feeling , any changes in layer build w/o update tis version and needed for up layer may not be included in layer build iso or container output.
[1] https://review.opendev.org/#/c/724385/1
[2] http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/containers/2...
[3] http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/monolithic/2...
Thanks.
BR Austin Sun.
*From:* Miller, Frank <Frank.Miller@windriver.com> *Sent:* Thursday, May 7, 2020 10:41 PM *To:* Sun, Austin <austin.sun@intel.com>; Jascanu, Nicolae <nicolae.jascanu@intel.com>; starlingx-discuss@lists.starlingx.io; Little, Scott <Scott.Little@windriver.com> *Subject:* RE: [Starlingx-discuss] Sanity Master Test LAYERED build ISO 20200506T013246Z bug#1877035
Thanks Austin for the very good analysis. The initial theory discussed on today’s build call is the tis version was not updated for package [1]. Saul/Scott took the action to verify this.
In the meantime, Nicolae can you run a monolithic build sanity so we can confirm if the ISO is sane or has other issues as well?
Frank
*From:* Sun, Austin <austin.sun@intel.com <mailto:austin.sun@intel.com>> *Sent:* Thursday, May 07, 2020 7:59 AM *To:* Jascanu, Nicolae <nicolae.jascanu@intel.com <mailto:nicolae.jascanu@intel.com>>; starlingx-discuss@lists.starlingx.io <mailto:starlingx-discuss@lists.starlingx.io>; Little, Scott <Scott.Little@windriver.com <mailto:Scott.Little@windriver.com>> *Subject:* Re: [Starlingx-discuss] Sanity Master Test LAYERED build ISO 20200506T013246Z bug#1877035
Hi All:
After further analysis this issue with Yan. Suspect there is bug hiding in cengn layer build.
Scott : would you like to check cengn build env ?
The details analysis :
Latest puppet-keystone rpm built from cengn should include change [1] which was merged by April 28^th .
From distro layer build 0505 [3] and latest distro output , puppet-keystone includes change [1].
But if we extract puppet-keystone in 0506 iso built from layer build [2] or latest iso [5] , it is surprised this change [1] was not included. so bug#1877035 shows-up based on 0506 and 0507 iso.
If we extract puppet-keystone from 0506 iso built from monolithic [4], the change [1] was included in puppet-keystone, and ansible provision pass it.
BTW: layer build in my local setup (cross build flock or flock layer only build ) did not show this issue, so it should not be related to layer build script.
[1] https://review.opendev.org/#/c/716842/
[2] http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/flock/202005...
[3] http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/distro/20200...
[4] http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/monolithic/2...
[5] http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/flock/202005...
[6] http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/distro/lates...
Thanks.
BR Austin Sun.
*From:* Jascanu, Nicolae <nicolae.jascanu@intel.com <mailto:nicolae.jascanu@intel.com>> *Sent:* Wednesday, May 6, 2020 10:06 PM *To:* starlingx-discuss@lists.starlingx.io <mailto:starlingx-discuss@lists.starlingx.io> *Subject:* [Starlingx-discuss] Sanity Master Test LAYERED build ISO 20200506T013246Z
Sanity Test from 2020-May-06 (http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/flock/202005...)
Status: RED
*//*
Provisioning is failing for all configurations. I’ve added a comment to the bug: https://bugs.launchpad.net/starlingx/+bug/1877035
We used the helm chart from http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/latest_docke...
Regards,
STX Validation Team
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
participants (3)
-
Miller, Frank
-
Saul Wold
-
Sun, Austin