[Starlingx-discuss] CentOS 7.5 upgrade status update

McKenna, Jason Jason.McKenna at windriver.com
Tue Sep 11 16:21:27 UTC 2018


Hi Cindy,

With the latest centos75 branch (and the cherry-picks you mentioned) I am unable to produce a build.  There seems to be several source rpm files missing from the currently pushed version of the .lst files (or the .lst files pull down the wrong version).  Most of the missing files I can download manually, however the checked in patches for dhcp and centos-release packages do not apply against the version of the source RPM which I can manually find on the Internet.  For example, the version of centos-release called out in the .lst is centos-release-7-5.1804.el7.centos.x86_64.rpm, the version called out in the checked in code base is the slightly different centos-release-7-5.1804.el7.centos.1.x86_64.rpm, and the patches checked in do not apply against either package.

Until the centos75 branch of the lst files pulls down the correct rpms, I will be unable to continue the validation effort.

-J

> -----Original Message-----
> From: Cordoba Malibran, Erich <erich.cordoba.malibran at intel.com>
> Sent: September 11, 2018 11:17 AM
> To: Waheed, Numan <Numan.Waheed at windriver.com>; Rasa, Nimalini
> <Nimalini.Rasa at windriver.com>; starlingx-discuss at lists.starlingx.io; Liu,
> ZhipengS <zhipengs.liu at intel.com>; Khalil, Ghada
> <Ghada.Khalil at windriver.com>; Lin, Shuicheng <shuicheng.lin at intel.com>;
> McKenna, Jason <Jason.McKenna at windriver.com>; Xie, Cindy
> <cindy.xie at intel.com>
> Subject: Re: [Starlingx-discuss] CentOS 7.5 upgrade status update
> 
> Hi
> 
> You can add "debug loglevel=8" to the kernel boot option to get a more
> detailed output of the crash (if any)
> 
> -Erich
> 
> On Tue, 2018-09-11 at 13:56 +0000, Xie, Cindy wrote:
> > Shuicheng saw the symbol table error issue as well but it recovers in
> > 8s, he is still working on debugging it. But 601204 is just kernel
> > upgrade but not intended to solve this issues. Are you running bare
> > metal? I am wondering if this is HW config and driver issue.
> >
> > Is it possible that you try to deploy it in virtual environment?
> >
> > Thx. - cindy
> >
> > From: McKenna, Jason [mailto:Jason.McKenna at windriver.com]
> > Sent: Tuesday, September 11, 2018 9:50 PM
> > To: Xie, Cindy <cindy.xie at intel.com>; Rasa, Nimalini <Nimalini.Rasa at w
> > indriver.com>; Waheed, Numan <Numan.Waheed at windriver.com>; Lin,
> > Shuicheng <shuicheng.lin at intel.com>; Liu, ZhipengS <zhipengs.liu at inte
> > l.com>; Khalil, Ghada <Ghada.Khalil at windriver.com>; 'starlingx-discus
> > s at lists.starlingx.io' <starlingx-discuss at lists.starlingx.io>
> > Subject: RE: CentOS 7.5 upgrade status update
> >
> > Hi Cindy,
> >
> > We got past the installation device error, but are coming up against
> > the symbol table error.  Our system hangs after this message.  I
> > suspect a kernel issue.  I see there is an open code review to
> > refactor some kernel patches to a rebased kernel (https://review.open
> > stack.org/#/c/601204/) but I do not know if that is expected to
> > resolve the issue
> >
> > -J
> >
> > From: Xie, Cindy <cindy.xie at intel.com>
> > Sent: September 11, 2018 9:32 AM
> > To: Rasa, Nimalini <Nimalini.Rasa at windriver.com>; Waheed, Numan
> <Numa
> > n.Waheed at windriver.com>; Lin, Shuicheng <shuicheng.lin at intel.com>;
> > McKenna, Jason <Jason.McKenna at windriver.com>; Liu, ZhipengS
> <zhipengs
> > .liu at intel.com>; Khalil, Ghada <Ghada.Khalil at windriver.com>; 'starlin
> > gx-discuss at lists.starlingx.io' <starlingx-discuss at lists.starlingx.io>
> > Subject: RE: CentOS 7.5 upgrade status update
> >
> > But as Jason reported yesterday, the build cannot be installed with
> > error messages:
> >
> > Installation failed.
> > ERROR: Specified installation (sda) or boot (sda) device is invalid.
> >
> > We cannot debug without more information, the ISO installs in Intel’s
> > HW. Thus Shuicheng created a new build and attached a manifest in this
> > email thread. Did WindRiver has a chance to create a new ISO using
> > today’s manifest file and re-run the test?
> >
> > Thx. - cindy
> >
> > From: Rasa, Nimalini [mailto:Nimalini.Rasa at windriver.com]
> > Sent: Tuesday, September 11, 2018 9:25 PM
> > To: Xie, Cindy <cindy.xie at intel.com>; Waheed, Numan
> <Numan.Waheed at win
> > driver.com>; Lin, Shuicheng <shuicheng.lin at intel.com>; McKenna, Jason
> > <Jason.McKenna at windriver.com>; Liu, ZhipengS
> <zhipengs.liu at intel.com>
> > ; Khalil, Ghada <Ghada.Khalil at windriver.com>; 'starlingx-discuss at list
> > s.starlingx.io' <starlingx-discuss at lists.starlingx.io>
> > Subject: RE: CentOS 7.5 upgrade status update
> >
> > This is on the same build (yesterday’s build).
> >
> > From: Xie, Cindy [mailto:cindy.xie at intel.com]
> > Sent: Tuesday, September 11, 2018 9:22 AM
> > To: Rasa, Nimalini; Waheed, Numan; Lin, Shuicheng; McKenna, Jason;
> > Liu, ZhipengS; Khalil, Ghada; 'starlingx-discuss at lists.starlingx.io'
> > Subject: RE: CentOS 7.5 upgrade status update
> >
> > Is this the same issue as yesterday’s build? or this is different
> > issue now?
> >
> > From: Rasa, Nimalini [mailto:Nimalini.Rasa at windriver.com]
> > Sent: Tuesday, September 11, 2018 9:06 PM
> > To: Waheed, Numan <Numan.Waheed at windriver.com>; Lin, Shuicheng
> <shuic
> > heng.lin at intel.com>; McKenna, Jason <Jason.McKenna at windriver.com>;
> > Liu, ZhipengS <zhipengs.liu at intel.com>; Khalil, Ghada <Ghada.Khalil at w
> > indriver.com>; 'starlingx-discuss at lists.starlingx.io' <starlingx-disc
> > uss at lists.starlingx.io>; Xie, Cindy <cindy.xie at intel.com>
> > Subject: RE: CentOS 7.5 upgrade status update
> >
> > It is not booting in my case. After the  following , it just hangs
> > there:
> > “Error: no symbol table.
> > Press any key to continue…”
> >
> > -Nimalini
> >
> > From: Waheed, Numan
> > Sent: Tuesday, September 11, 2018 8:50 AM
> > To: Lin, Shuicheng; McKenna, Jason; Liu, ZhipengS; Khalil, Ghada; 'st
> > arlingx-discuss at lists.starlingx.io'; Xie, Cindy; Rasa, Nimalini
> > Subject: RE: CentOS 7.5 upgrade status update
> >
> > Adding Nimalini
> >
> > From: Lin, Shuicheng [mailto:shuicheng.lin at intel.com]
> > Sent: September-11-18 2:17 AM
> > To: McKenna, Jason; Liu, ZhipengS; Khalil, Ghada; 'starlingx-discuss@
> > lists.starlingx.io'; Xie, Cindy; Waheed, Numan
> > Subject: RE: CentOS 7.5 upgrade status update
> >
> > Hi Jason,
> > Just finished deploy image with latest code, I could config controller
> > successfully in bare metal environment.
> > Manifest is created for you reference. There are two patches I add
> > manually:
> > https://review.openstack.org/596855            for python-horizon from
> > master
> > https://review.openstack.org/599892            for build-iso
> >
> >
> > There is an issue found when reboot controller node.
> > After the OS selection menu, it will show below error. And it will
> > auto continue boot after around 8 seconds. I will check the issue
> > later.
> > “
> > Error: no symbol table.
> > Press any key to continue…
> > “
> >
> > Best Regards
> > Shuicheng
> >
> > From: Lin, Shuicheng [mailto:shuicheng.lin at intel.com]
> > Sent: Tuesday, September 11, 2018 9:35 AM
> > To: McKenna, Jason <Jason.McKenna at windriver.com>; Liu, ZhipengS
> <zhip
> > engs.liu at intel.com>; Khalil, Ghada <Ghada.Khalil at windriver.com>; 'sta
> > rlingx-discuss at lists.starlingx.io' <starlingx-discuss at lists.starlingx
> > .io>; Xie, Cindy <cindy.xie at intel.com>; Waheed, Numan
> <Numan.Waheed at w
> > indriver.com>
> > Subject: Re: [Starlingx-discuss] CentOS 7.5 upgrade status update
> >
> > Hi Jason,
> > I just tried my last Friday’s image at bare metal machine. There is no
> > issue to install controller node.
> > Will you help check what cause the install error in your environment?
> >
> > I am syncing my code to latest to double confirm it.
> >
> > Best Regards
> > Shuicheng
> >
> > From: McKenna, Jason [mailto:Jason.McKenna at windriver.com]
> > Sent: Tuesday, September 11, 2018 12:13 AM
> > To: Liu, ZhipengS <zhipengs.liu at intel.com>; Lin, Shuicheng <shuicheng
> > .lin at intel.com>; Khalil, Ghada <Ghada.Khalil at windriver.com>; 'starlin
> > gx-discuss at lists.starlingx.io' <starlingx-discuss at lists.starlingx.io>
> > ; Xie, Cindy <cindy.xie at intel.com>; Waheed, Numan
> <Numan.Waheed at windr
> > iver.com>
> > Subject: RE: CentOS 7.5 upgrade status update
> >
> > Status update.  The build from the centos75 branch does not install on
> > our hardware.  The error message we are seeing is:
> >
> > Installation failed.
> >
> > ERROR: Specified installation (sda) or boot (sda) device is invalid.
> >
> > -Jason
> >
> > From: Liu, ZhipengS <zhipengs.liu at intel.com>
> > Sent: September 8, 2018 9:56 AM
> > To: Lin, Shuicheng <shuicheng.lin at intel.com>; McKenna, Jason <Jason.M
> > cKenna at windriver.com>; Khalil, Ghada <Ghada.Khalil at windriver.com>; 's
> > tarlingx-discuss at lists.starlingx.io' <starlingx-discuss at lists.starlin
> > gx.io>; Xie, Cindy <cindy.xie at intel.com>; Waheed, Numan
> <Numan.Waheed
> > @windriver.com>
> > Subject: RE: CentOS 7.5 upgrade status update
> >
> > Hi Jason,
> >
> > I have fixed the issue you mentioned below.
> > -        Furthermore, the tpm2-tools package was requesting version
> > tpm2-tss-1.3.0-2.el7.centos (rather than tpm2-tss-1.3.0-2.el7).  I
> > hacked the spec file for tpm2-tools and rebuilt it to require the
> > version mentioned, but there is some rework to be done there.
> > [slin14]: I have the same issue also. Let’s further check it to find
> > the root cause. I use rebuild tpm2-tools/tss2 to workaround it without
> > any code.
> >
> > https://review.openstack.org/#/c/600920/
> >
> > Thanks!
> > Zhipeng
> > From: Lin, Shuicheng [mailto:shuicheng.lin at intel.com]
> > Sent: 2018年9月8日 21:14
> > To: McKenna, Jason <Jason.McKenna at windriver.com>; Khalil, Ghada
> <Ghad
> > a.Khalil at windriver.com>; 'starlingx-discuss at lists.starlingx.io' <star
> > lingx-discuss at lists.starlingx.io>; Xie, Cindy <cindy.xie at intel.com>;
> > Waheed, Numan <Numan.Waheed at windriver.com>
> > Subject: Re: [Starlingx-discuss] CentOS 7.5 upgrade status update
> >
> > Hi Jason,
> > I have fixed the error in the openstack-aodh. Patch is updated:
> > https://review.openstack.org/#/c/598388/
> >
> > It is due to my local version is not the latest one.
> > Sorry for the trouble caused, and thanks for your test.
> >
> > Best Regards
> > Shuicheng
> >
> > From: Lin, Shuicheng [mailto:shuicheng.lin at intel.com]
> > Sent: Saturday, September 8, 2018 9:23 AM
> > To: McKenna, Jason <Jason.McKenna at windriver.com>; Khalil, Ghada
> <Ghad
> > a.Khalil at windriver.com>; 'starlingx-discuss at lists.starlingx.io' <star
> > lingx-discuss at lists.starlingx.io>; Xie, Cindy <cindy.xie at intel.com>;
> > Waheed, Numan <Numan.Waheed at windriver.com>
> > Subject: Re: [Starlingx-discuss] CentOS 7.5 upgrade status update
> >
> > Hi Jason,
> > Thanks very much for such detail feedback.
> > I added some comments inline, and will check the issue in my
> > environment also.
> > Feel free to submit patch if you have to fix them. J
> >
> >
> > Best Regards
> > Shuicheng
> >
> > From: McKenna, Jason [mailto:Jason.McKenna at windriver.com]
> > Sent: Saturday, September 8, 2018 3:47 AM
> > To: McKenna, Jason <Jason.McKenna at windriver.com>; Khalil, Ghada
> <Ghad
> > a.Khalil at windriver.com>; Lin, Shuicheng <shuicheng.lin at intel.com>; 's
> > tarlingx-discuss at lists.starlingx.io' <starlingx-discuss at lists.starlin
> > gx.io>; Xie, Cindy <cindy.xie at intel.com>; Waheed, Numan
> <Numan.Waheed
> > @windriver.com>
> > Subject: RE: CentOS 7.5 upgrade status update
> >
> > Hi Shuicheng,
> >
> > I’ve completed a build of the code.  We have not yet run run-time
> > tests, but here are the build-time issues I encountered:
> >
> > -        The .lst file contains lighttpd-1.4.49, but this is no
> > longer available for download, and the download_mirrors.sh step fails.
> > It appears than CentOS has updated to lighttpd-1.4.50.  I manually
> > built a lighttpd-1.4.49-1.src.rpm to satisfy the .lst
> > -        Even though the .lst file specifies lighttpd-1.4.49, the
> > code still references 1.4.45
> > -        At build-pkgs time (the build-srpms step), the meta-patches
> > to openstack-aodh do not apply, as two of them are malformed.  https:
> > //review.openstack.org/#/c/598388/6/openstack/openstack-
> > aodh/centos/meta_patches/0001-Modify-service-files-and-create-
> > expirer-cron-script.patch -- In the new version, on line 190, the
> > patch states that the old code covered 6 lines, and the new code
> > covers 9 lines (-283,6 +286,9).  But there are only two lines inserted
> > by the patch.  There is also a new blank line in the patch at line
> > 194.  Removing this blank line and editing the hunk description to
> > only insert 2 lines resolves the issue for this hunk (there are other
> > similar issues in this patch, and in 0001-meta-
> > modify-aodh-api.patch)
> > [slin14]: Let me double check it.
> > -        At binary build time, I had build failures on python-django-
> > horizon and some magnum packages.  I had to pull in https://review.op
> > enstack.org/#/c/596855/2 (from the master branch, but not in the
> > centos75 branch of stx-upstream) to get past this.
> > [slin14]: It is due to I have not synced latest master code, so I
> > don’t have this issue. I used last week’s master code.
> > -        After building all packages, I attempted to build the ISO.
> > The tpm2-tss-1.3.0-2.el7.x86_64.rpm package were missing, and should
> > be added to the .lst files.
> > [slin14]: This package is already in the lst.
> > -        Furthermore, the tpm2-tools package was requesting version
> > tpm2-tss-1.3.0-2.el7.centos (rather than tpm2-tss-1.3.0-2.el7).  I
> > hacked the spec file for tpm2-tools and rebuilt it to require the
> > version mentioned, but there is some rework to be done there.
> > [slin14]: I have the same issue also. Let’s further check it to find
> > the root cause. I use rebuild tpm2-tools/tss2 to workaround it without
> > any code.
> >
> > After these changes, I was able to produce a .iso
> >
> > -Jason
> >
> > From: McKenna, Jason <Jason.McKenna at windriver.com>
> > Sent: September 7, 2018 10:08 AM
> > To: Khalil, Ghada <Ghada.Khalil at windriver.com>; Lin, Shuicheng <shuic
> > heng.lin at intel.com>; 'starlingx-discuss at lists.starlingx.io' <starling
> > x-discuss at lists.starlingx.io>; Xie, Cindy <cindy.xie at intel.com>;
> > Waheed, Numan <Numan.Waheed at windriver.com>
> > Subject: Re: [Starlingx-discuss] CentOS 7.5 upgrade status update
> >
> > Hi Shuicheng,
> >
> > As a download_mirrors.sh time issue, the version of lighttpd has been
> > changed from 1.4.45 to 1.4.49.  The lighttpd-1.4.49-1.el7.src.rpm file
> > no longer exists on EPEL or CentOS servers, as they’ve moved to
> > 1.4.50.  I am attempting to locate an archived version of the 1.4.49
> > .src rpm.
> >
> > -Jason
> >
> > From: Khalil, Ghada <Ghada.Khalil at windriver.com>
> > Sent: September 7, 2018 9:36 AM
> > To: Lin, Shuicheng <shuicheng.lin at intel.com>; 'starlingx-discuss at list
> > s.starlingx.io' <starlingx-discuss at lists.starlingx.io>; Xie, Cindy <c
> > indy.xie at intel.com>; Waheed, Numan <Numan.Waheed at windriver.com>
> > Subject: Re: [Starlingx-discuss] CentOS 7.5 upgrade status update
> >
> > Thanks Shuicheng. We’ll run some tests in Wind River. Numan will be
> > priming this activity. He’ll share the results with the team sometime
> > next week.
> >
> > Regards,
> > Ghada
> >
> > From: Lin, Shuicheng [mailto:shuicheng.lin at intel.com]
> > Sent: Friday, September 07, 2018 1:32 AM
> > To: 'starlingx-discuss at lists.starlingx.io'; Khalil, Ghada; Xie, Cindy
> > Subject: CentOS 7.5 upgrade status update
> >
> > Hi all,
> > Most task of CentOS7.5 upgrade in story 2003389 is done. Please go
> > ahead to have a try and verify it.
> > I expect there should be some issue due to we just did limited deploy
> > test yet.
> >
> > Here is the detail status update of this week.
> > For the total 44 srpm we plan to upgrade:
> > 38 already merged in feature branch.
> > 3 kept as original version.  Anaconda upgrade is not needed. python-
> > os-vif has higher version than 7.5. python-eventlet kept due to new
> > src rpm itself have known issue.
> > 2 under review with code ready. Puppet-haproxy and openstack-aodh.
> > 1 under debug. Lighttpd.
> >
> > Besides srpm, we also upgraded 260+ rpm packages to 7.5 version.
> >
> > Here is the build instruction:
> > 1.      Switch to f/centos75 branch for stx-tools/stx-integ/stx-
> > upstream/stx-root.
> > Cherry-pick below un-merged patch:
> > For puppet-haproxy:
> > https://review.openstack.org/600369
> > https://review.openstack.org/600359
> > For openstack-aodh:
> > https://review.openstack.org/598388
> > https://review.openstack.org/600591
> > For build-iso:
> > https://review.openstack.org/599892
> > For build-pkgs:
> > https://review.openstack.org/599901
> > https://review.openstack.org/600639
> >
> > 2.      Run mirror script to get the upgraded src rpm and related
> > rpm.
> > 3.      Try to build packages and ISO.
> >
> > For the detail status of CentOS7.5 upgrade, please check below story:
> > https://storyboard.openstack.org/#!/story/2003389
> >
> > Feel free to contact me if you have trouble on it.
> >
> > Best Regards
> > Shuicheng
> > _______________________________________________
> > Starlingx-discuss mailing list
> > Starlingx-discuss at lists.starlingx.io
> > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss


More information about the Starlingx-discuss mailing list