[Starlingx-discuss] CentOS 7.5 upgrade status update

Cordoba Malibran, Erich erich.cordoba.malibran at intel.com
Tue Sep 11 15:17:18 UTC 2018


Hi

You can add "debug loglevel=8" to the kernel boot option to get a more
detailed output of the crash (if any)

-Erich

On Tue, 2018-09-11 at 13:56 +0000, Xie, Cindy wrote:
> Shuicheng saw the symbol table error issue as well but it recovers in
> 8s, he is still working on debugging it. But 601204 is just kernel
> upgrade but not intended to solve this issues. Are you running bare
> metal? I am wondering if this is HW config and driver issue.
>  
> Is it possible that you try to deploy it in virtual environment?
>  
> Thx. - cindy
>  
> From: McKenna, Jason [mailto:Jason.McKenna at windriver.com] 
> Sent: Tuesday, September 11, 2018 9:50 PM
> To: Xie, Cindy <cindy.xie at intel.com>; Rasa, Nimalini <Nimalini.Rasa at w
> indriver.com>; Waheed, Numan <Numan.Waheed at windriver.com>; Lin,
> Shuicheng <shuicheng.lin at intel.com>; Liu, ZhipengS <zhipengs.liu at inte
> l.com>; Khalil, Ghada <Ghada.Khalil at windriver.com>; 'starlingx-discus
> s at lists.starlingx.io' <starlingx-discuss at lists.starlingx.io>
> Subject: RE: CentOS 7.5 upgrade status update
>  
> Hi Cindy,
>  
> We got past the installation device error, but are coming up against
> the symbol table error.  Our system hangs after this message.  I
> suspect a kernel issue.  I see there is an open code review to
> refactor some kernel patches to a rebased kernel (https://review.open
> stack.org/#/c/601204/) but I do not know if that is expected to
> resolve the issue
>  
> -J
>  
> From: Xie, Cindy <cindy.xie at intel.com> 
> Sent: September 11, 2018 9:32 AM
> To: Rasa, Nimalini <Nimalini.Rasa at windriver.com>; Waheed, Numan <Numa
> n.Waheed at windriver.com>; Lin, Shuicheng <shuicheng.lin at intel.com>;
> McKenna, Jason <Jason.McKenna at windriver.com>; Liu, ZhipengS <zhipengs
> .liu at intel.com>; Khalil, Ghada <Ghada.Khalil at windriver.com>; 'starlin
> gx-discuss at lists.starlingx.io' <starlingx-discuss at lists.starlingx.io>
> Subject: RE: CentOS 7.5 upgrade status update
>  
> But as Jason reported yesterday, the build cannot be installed with
> error messages:
>  
> Installation failed.
> ERROR: Specified installation (sda) or boot (sda) device is invalid.
>  
> We cannot debug without more information, the ISO installs in Intel’s
> HW. Thus Shuicheng created a new build and attached a manifest in
> this email thread. Did WindRiver has a chance to create a new ISO
> using today’s manifest file and re-run the test?
>  
> Thx. - cindy
>  
> From: Rasa, Nimalini [mailto:Nimalini.Rasa at windriver.com] 
> Sent: Tuesday, September 11, 2018 9:25 PM
> To: Xie, Cindy <cindy.xie at intel.com>; Waheed, Numan <Numan.Waheed at win
> driver.com>; Lin, Shuicheng <shuicheng.lin at intel.com>; McKenna, Jason
> <Jason.McKenna at windriver.com>; Liu, ZhipengS <zhipengs.liu at intel.com>
> ; Khalil, Ghada <Ghada.Khalil at windriver.com>; 'starlingx-discuss at list
> s.starlingx.io' <starlingx-discuss at lists.starlingx.io>
> Subject: RE: CentOS 7.5 upgrade status update
>  
> This is on the same build (yesterday’s build).
>  
> From: Xie, Cindy [mailto:cindy.xie at intel.com] 
> Sent: Tuesday, September 11, 2018 9:22 AM
> To: Rasa, Nimalini; Waheed, Numan; Lin, Shuicheng; McKenna, Jason;
> Liu, ZhipengS; Khalil, Ghada; 'starlingx-discuss at lists.starlingx.io'
> Subject: RE: CentOS 7.5 upgrade status update
>  
> Is this the same issue as yesterday’s build? or this is different
> issue now?
>  
> From: Rasa, Nimalini [mailto:Nimalini.Rasa at windriver.com] 
> Sent: Tuesday, September 11, 2018 9:06 PM
> To: Waheed, Numan <Numan.Waheed at windriver.com>; Lin, Shuicheng <shuic
> heng.lin at intel.com>; McKenna, Jason <Jason.McKenna at windriver.com>;
> Liu, ZhipengS <zhipengs.liu at intel.com>; Khalil, Ghada <Ghada.Khalil at w
> indriver.com>; 'starlingx-discuss at lists.starlingx.io' <starlingx-disc
> uss at lists.starlingx.io>; Xie, Cindy <cindy.xie at intel.com>
> Subject: RE: CentOS 7.5 upgrade status update
>  
> It is not booting in my case. After the  following , it just hangs
> there:
> “Error: no symbol table.
> Press any key to continue…”
>  
> -Nimalini
>  
> From: Waheed, Numan 
> Sent: Tuesday, September 11, 2018 8:50 AM
> To: Lin, Shuicheng; McKenna, Jason; Liu, ZhipengS; Khalil, Ghada; 'st
> arlingx-discuss at lists.starlingx.io'; Xie, Cindy; Rasa, Nimalini
> Subject: RE: CentOS 7.5 upgrade status update
>  
> Adding Nimalini
>  
> From: Lin, Shuicheng [mailto:shuicheng.lin at intel.com] 
> Sent: September-11-18 2:17 AM
> To: McKenna, Jason; Liu, ZhipengS; Khalil, Ghada; 'starlingx-discuss@
> lists.starlingx.io'; Xie, Cindy; Waheed, Numan
> Subject: RE: CentOS 7.5 upgrade status update
>  
> Hi Jason,
> Just finished deploy image with latest code, I could config
> controller successfully in bare metal environment.
> Manifest is created for you reference. There are two patches I add
> manually:
> https://review.openstack.org/596855            for python-horizon
> from master
> https://review.openstack.org/599892            for build-iso
>  
>  
> There is an issue found when reboot controller node.
> After the OS selection menu, it will show below error. And it will
> auto continue boot after around 8 seconds. I will check the issue
> later.
>> Error: no symbol table.
> Press any key to continue…
>>  
> Best Regards
> Shuicheng
>  
> From: Lin, Shuicheng [mailto:shuicheng.lin at intel.com] 
> Sent: Tuesday, September 11, 2018 9:35 AM
> To: McKenna, Jason <Jason.McKenna at windriver.com>; Liu, ZhipengS <zhip
> engs.liu at intel.com>; Khalil, Ghada <Ghada.Khalil at windriver.com>; 'sta
> rlingx-discuss at lists.starlingx.io' <starlingx-discuss at lists.starlingx
> .io>; Xie, Cindy <cindy.xie at intel.com>; Waheed, Numan <Numan.Waheed at w
> indriver.com>
> Subject: Re: [Starlingx-discuss] CentOS 7.5 upgrade status update
>  
> Hi Jason,
> I just tried my last Friday’s image at bare metal machine. There is
> no issue to install controller node.
> Will you help check what cause the install error in your environment?
>  
> I am syncing my code to latest to double confirm it.
>  
> Best Regards
> Shuicheng
>  
> From: McKenna, Jason [mailto:Jason.McKenna at windriver.com] 
> Sent: Tuesday, September 11, 2018 12:13 AM
> To: Liu, ZhipengS <zhipengs.liu at intel.com>; Lin, Shuicheng <shuicheng
> .lin at intel.com>; Khalil, Ghada <Ghada.Khalil at windriver.com>; 'starlin
> gx-discuss at lists.starlingx.io' <starlingx-discuss at lists.starlingx.io>
> ; Xie, Cindy <cindy.xie at intel.com>; Waheed, Numan <Numan.Waheed at windr
> iver.com>
> Subject: RE: CentOS 7.5 upgrade status update
>  
> Status update.  The build from the centos75 branch does not install
> on our hardware.  The error message we are seeing is:
>  
> Installation failed.
>  
> ERROR: Specified installation (sda) or boot (sda) device is invalid.
>  
> -Jason
>  
> From: Liu, ZhipengS <zhipengs.liu at intel.com> 
> Sent: September 8, 2018 9:56 AM
> To: Lin, Shuicheng <shuicheng.lin at intel.com>; McKenna, Jason <Jason.M
> cKenna at windriver.com>; Khalil, Ghada <Ghada.Khalil at windriver.com>; 's
> tarlingx-discuss at lists.starlingx.io' <starlingx-discuss at lists.starlin
> gx.io>; Xie, Cindy <cindy.xie at intel.com>; Waheed, Numan <Numan.Waheed
> @windriver.com>
> Subject: RE: CentOS 7.5 upgrade status update
>  
> Hi Jason,
>  
> I have fixed the issue you mentioned below.
> -        Furthermore, the tpm2-tools package was requesting version
> tpm2-tss-1.3.0-2.el7.centos (rather than tpm2-tss-1.3.0-2.el7).  I
> hacked the spec file for tpm2-tools and rebuilt it to require the
> version mentioned, but there is some rework to be done there.
> [slin14]: I have the same issue also. Let’s further check it to find
> the root cause. I use rebuild tpm2-tools/tss2 to workaround it
> without any code.
>  
> https://review.openstack.org/#/c/600920/
>  
> Thanks!
> Zhipeng
> From: Lin, Shuicheng [mailto:shuicheng.lin at intel.com] 
> Sent: 2018年9月8日 21:14
> To: McKenna, Jason <Jason.McKenna at windriver.com>; Khalil, Ghada <Ghad
> a.Khalil at windriver.com>; 'starlingx-discuss at lists.starlingx.io' <star
> lingx-discuss at lists.starlingx.io>; Xie, Cindy <cindy.xie at intel.com>;
> Waheed, Numan <Numan.Waheed at windriver.com>
> Subject: Re: [Starlingx-discuss] CentOS 7.5 upgrade status update
>  
> Hi Jason,
> I have fixed the error in the openstack-aodh. Patch is updated:
> https://review.openstack.org/#/c/598388/
>  
> It is due to my local version is not the latest one.
> Sorry for the trouble caused, and thanks for your test.
>  
> Best Regards
> Shuicheng
>  
> From: Lin, Shuicheng [mailto:shuicheng.lin at intel.com] 
> Sent: Saturday, September 8, 2018 9:23 AM
> To: McKenna, Jason <Jason.McKenna at windriver.com>; Khalil, Ghada <Ghad
> a.Khalil at windriver.com>; 'starlingx-discuss at lists.starlingx.io' <star
> lingx-discuss at lists.starlingx.io>; Xie, Cindy <cindy.xie at intel.com>;
> Waheed, Numan <Numan.Waheed at windriver.com>
> Subject: Re: [Starlingx-discuss] CentOS 7.5 upgrade status update
>  
> Hi Jason,
> Thanks very much for such detail feedback.
> I added some comments inline, and will check the issue in my
> environment also.
> Feel free to submit patch if you have to fix them. J
>  
>  
> Best Regards
> Shuicheng
>  
> From: McKenna, Jason [mailto:Jason.McKenna at windriver.com] 
> Sent: Saturday, September 8, 2018 3:47 AM
> To: McKenna, Jason <Jason.McKenna at windriver.com>; Khalil, Ghada <Ghad
> a.Khalil at windriver.com>; Lin, Shuicheng <shuicheng.lin at intel.com>; 's
> tarlingx-discuss at lists.starlingx.io' <starlingx-discuss at lists.starlin
> gx.io>; Xie, Cindy <cindy.xie at intel.com>; Waheed, Numan <Numan.Waheed
> @windriver.com>
> Subject: RE: CentOS 7.5 upgrade status update
>  
> Hi Shuicheng,
>  
> I’ve completed a build of the code.  We have not yet run run-time
> tests, but here are the build-time issues I encountered:
>  
> -        The .lst file contains lighttpd-1.4.49, but this is no
> longer available for download, and the download_mirrors.sh step
> fails.  It appears than CentOS has updated to lighttpd-1.4.50.  I
> manually built a lighttpd-1.4.49-1.src.rpm to satisfy the .lst
> -        Even though the .lst file specifies lighttpd-1.4.49, the
> code still references 1.4.45
> -        At build-pkgs time (the build-srpms step), the meta-patches
> to openstack-aodh do not apply, as two of them are malformed.  https:
> //review.openstack.org/#/c/598388/6/openstack/openstack-
> aodh/centos/meta_patches/0001-Modify-service-files-and-create-
> expirer-cron-script.patch -- In the new version, on line 190, the
> patch states that the old code covered 6 lines, and the new code
> covers 9 lines (-283,6 +286,9).  But there are only two lines
> inserted by the patch.  There is also a new blank line in the patch
> at line 194.  Removing this blank line and editing the hunk
> description to only insert 2 lines resolves the issue for this hunk
> (there are other similar issues in this patch, and in 0001-meta-
> modify-aodh-api.patch)
> [slin14]: Let me double check it.
> -        At binary build time, I had build failures on python-django-
> horizon and some magnum packages.  I had to pull in https://review.op
> enstack.org/#/c/596855/2 (from the master branch, but not in the
> centos75 branch of stx-upstream) to get past this.
> [slin14]: It is due to I have not synced latest master code, so I
> don’t have this issue. I used last week’s master code.
> -        After building all packages, I attempted to build the ISO. 
> The tpm2-tss-1.3.0-2.el7.x86_64.rpm package were missing, and should
> be added to the .lst files.
> [slin14]: This package is already in the lst.
> -        Furthermore, the tpm2-tools package was requesting version
> tpm2-tss-1.3.0-2.el7.centos (rather than tpm2-tss-1.3.0-2.el7).  I
> hacked the spec file for tpm2-tools and rebuilt it to require the
> version mentioned, but there is some rework to be done there.
> [slin14]: I have the same issue also. Let’s further check it to find
> the root cause. I use rebuild tpm2-tools/tss2 to workaround it
> without any code.
>  
> After these changes, I was able to produce a .iso
>  
> -Jason
>  
> From: McKenna, Jason <Jason.McKenna at windriver.com> 
> Sent: September 7, 2018 10:08 AM
> To: Khalil, Ghada <Ghada.Khalil at windriver.com>; Lin, Shuicheng <shuic
> heng.lin at intel.com>; 'starlingx-discuss at lists.starlingx.io' <starling
> x-discuss at lists.starlingx.io>; Xie, Cindy <cindy.xie at intel.com>;
> Waheed, Numan <Numan.Waheed at windriver.com>
> Subject: Re: [Starlingx-discuss] CentOS 7.5 upgrade status update
>  
> Hi Shuicheng,
>  
> As a download_mirrors.sh time issue, the version of lighttpd has been
> changed from 1.4.45 to 1.4.49.  The lighttpd-1.4.49-1.el7.src.rpm
> file no longer exists on EPEL or CentOS servers, as they’ve moved to
> 1.4.50.  I am attempting to locate an archived version of the 1.4.49
> .src rpm.
>  
> -Jason
>  
> From: Khalil, Ghada <Ghada.Khalil at windriver.com> 
> Sent: September 7, 2018 9:36 AM
> To: Lin, Shuicheng <shuicheng.lin at intel.com>; 'starlingx-discuss at list
> s.starlingx.io' <starlingx-discuss at lists.starlingx.io>; Xie, Cindy <c
> indy.xie at intel.com>; Waheed, Numan <Numan.Waheed at windriver.com>
> Subject: Re: [Starlingx-discuss] CentOS 7.5 upgrade status update
>  
> Thanks Shuicheng. We’ll run some tests in Wind River. Numan will be
> priming this activity. He’ll share the results with the team sometime
> next week.
>  
> Regards,
> Ghada
>  
> From: Lin, Shuicheng [mailto:shuicheng.lin at intel.com] 
> Sent: Friday, September 07, 2018 1:32 AM
> To: 'starlingx-discuss at lists.starlingx.io'; Khalil, Ghada; Xie, Cindy
> Subject: CentOS 7.5 upgrade status update
>  
> Hi all,
> Most task of CentOS7.5 upgrade in story 2003389 is done. Please go
> ahead to have a try and verify it.
> I expect there should be some issue due to we just did limited deploy
> test yet.
>  
> Here is the detail status update of this week.
> For the total 44 srpm we plan to upgrade:
> 38 already merged in feature branch.
> 3 kept as original version.  Anaconda upgrade is not needed. python-
> os-vif has higher version than 7.5. python-eventlet kept due to new
> src rpm itself have known issue.
> 2 under review with code ready. Puppet-haproxy and openstack-aodh.
> 1 under debug. Lighttpd.
>  
> Besides srpm, we also upgraded 260+ rpm packages to 7.5 version.
>  
> Here is the build instruction:
> 1.      Switch to f/centos75 branch for stx-tools/stx-integ/stx-
> upstream/stx-root.
> Cherry-pick below un-merged patch:
> For puppet-haproxy:
> https://review.openstack.org/600369
> https://review.openstack.org/600359
> For openstack-aodh:
> https://review.openstack.org/598388
> https://review.openstack.org/600591
> For build-iso:
> https://review.openstack.org/599892
> For build-pkgs:
> https://review.openstack.org/599901
> https://review.openstack.org/600639
>  
> 2.      Run mirror script to get the upgraded src rpm and related
> rpm.
> 3.      Try to build packages and ISO.
>  
> For the detail status of CentOS7.5 upgrade, please check below story:
> https://storyboard.openstack.org/#!/story/2003389
>  
> Feel free to contact me if you have trouble on it.
>  
> Best Regards
> Shuicheng
> _______________________________________________
> Starlingx-discuss mailing list
> Starlingx-discuss at lists.starlingx.io
> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss


More information about the Starlingx-discuss mailing list