[Starlingx-discuss] CentOS 7.5 upgrade status update
McKenna, Jason
Jason.McKenna at windriver.com
Wed Sep 12 20:23:54 UTC 2018
> -----Original Message-----
> From: Lin, Shuicheng <shuicheng.lin at intel.com>
> Sent: September 11, 2018 8:44 PM
> To: McKenna, Jason <Jason.McKenna at windriver.com>; Cordoba Malibran,
> Erich <erich.cordoba.malibran at intel.com>; Waheed, Numan
> <Numan.Waheed at windriver.com>; Rasa, Nimalini
> <Nimalini.Rasa at windriver.com>; starlingx-discuss at lists.starlingx.io; Liu,
> ZhipengS <zhipengs.liu at intel.com>; Khalil, Ghada
> <Ghada.Khalil at windriver.com>; Xie, Cindy <cindy.xie at intel.com>
> Subject: RE: [Starlingx-discuss] CentOS 7.5 upgrade status update
>
> Hi Jason,
> Here is the dhcp src rpm we are using:
> [slin14 at f998ec953c85 /]$ sudo -E yumdownloader -C --url --source dhcp
> http://vault.centos.org/7.5.1804/updates/Source/SPackages/dhcp-4.2.5-
> 68.el7.centos.1.src.rpm
>
> For centos-release, there are two packages in the mirror lst, 1 src rpm for
> build, and 1 rpm for the mock environment. And it has the "slightly
> difference" in the names as you mentioned.
> Both of them could be found in the centos repo with the exact name.
> ./rpms_centos.lst:135:centos-release-7-5.1804.1.el7.centos.src.rpm
> ./rpms_centos.lst:136:centos-release-7-5.1804.el7.centos.x86_64.rpm
> [slin14 at f998ec953c85 /]$ sudo -E yumdownloader -C --url --source centos-
> release-7-5.1804.1.el7.centos
> http://vault.centos.org/7.5.1804/updates/Source/SPackages/centos-
> release-7-5.1804.1.el7.centos.src.rpm
> [slin14 at f998ec953c85 /]$ sudo -E yumdownloader -C --url centos-release-7-
> 5.1804.el7.centos.x86_64
> http://mirror.centos.org/centos/7.5.1804/os/x86_64/Packages/centos-
> release-7-5.1804.el7.centos.x86_64.rpm
>
> If there is still some other rpm missing, you could try to find them from the
> repo.
> I have try my best to keep my local mirror the same as the lst. But it is still
> possible some package missed to be added to the repo lst. Please feel free
> to notify me or correct the error directly.
> Thanks.
With the latest commits to stx-tools I am now able to download the rpms and produce a build. I'll let you know how testing goes
>
> Best Regards
> Shuicheng
>
>
> -----Original Message-----
> From: McKenna, Jason [mailto:Jason.McKenna at windriver.com]
> Sent: Wednesday, September 12, 2018 12:21 AM
> To: Cordoba Malibran, Erich <erich.cordoba.malibran at intel.com>; Waheed,
> Numan <Numan.Waheed at windriver.com>; Rasa, Nimalini
> <Nimalini.Rasa at windriver.com>; starlingx-discuss at lists.starlingx.io; Liu,
> ZhipengS <zhipengs.liu at intel.com>; Khalil, Ghada
> <Ghada.Khalil at windriver.com>; Lin, Shuicheng <shuicheng.lin at intel.com>;
> Xie, Cindy <cindy.xie at intel.com>
> Subject: RE: [Starlingx-discuss] CentOS 7.5 upgrade status update
>
> Hi Cindy,
>
> With the latest centos75 branch (and the cherry-picks you mentioned) I am
> unable to produce a build. There seems to be several source rpm files
> missing from the currently pushed version of the .lst files (or the .lst files pull
> down the wrong version). Most of the missing files I can download manually,
> however the checked in patches for dhcp and centos-release packages do
> not apply against the version of the source RPM which I can manually find on
> the Internet. For example, the version of centos-release called out in the .lst
> is centos-release-7-5.1804.el7.centos.x86_64.rpm, the version called out in
> the checked in code base is the slightly different centos-release-7-
> 5.1804.el7.centos.1.x86_64.rpm, and the patches checked in do not apply
> against either package.
>
> Until the centos75 branch of the lst files pulls down the correct rpms, I will be
> unable to continue the validation effort.
>
> -J
>
> > -----Original Message-----
> > From: Cordoba Malibran, Erich <erich.cordoba.malibran at intel.com>
> > Sent: September 11, 2018 11:17 AM
> > To: Waheed, Numan <Numan.Waheed at windriver.com>; Rasa, Nimalini
> > <Nimalini.Rasa at windriver.com>; starlingx-discuss at lists.starlingx.io;
> > Liu, ZhipengS <zhipengs.liu at intel.com>; Khalil, Ghada
> > <Ghada.Khalil at windriver.com>; Lin, Shuicheng
> > <shuicheng.lin at intel.com>; McKenna, Jason
> > <Jason.McKenna at windriver.com>; Xie, Cindy <cindy.xie at intel.com>
> > Subject: Re: [Starlingx-discuss] CentOS 7.5 upgrade status update
> >
> > Hi
> >
> > You can add "debug loglevel=8" to the kernel boot option to get a more
> > detailed output of the crash (if any)
> >
> > -Erich
> >
> > On Tue, 2018-09-11 at 13:56 +0000, Xie, Cindy wrote:
> > > Shuicheng saw the symbol table error issue as well but it recovers
> > > in 8s, he is still working on debugging it. But 601204 is just
> > > kernel upgrade but not intended to solve this issues. Are you
> > > running bare metal? I am wondering if this is HW config and driver issue.
> > >
> > > Is it possible that you try to deploy it in virtual environment?
> > >
> > > Thx. - cindy
> > >
> > > From: McKenna, Jason [mailto:Jason.McKenna at windriver.com]
> > > Sent: Tuesday, September 11, 2018 9:50 PM
> > > To: Xie, Cindy <cindy.xie at intel.com>; Rasa, Nimalini
> > > <Nimalini.Rasa at w indriver.com>; Waheed, Numan
> > > <Numan.Waheed at windriver.com>; Lin, Shuicheng
> > > <shuicheng.lin at intel.com>; Liu, ZhipengS <zhipengs.liu at inte l.com>;
> > > Khalil, Ghada <Ghada.Khalil at windriver.com>; 'starlingx-discus
> > > s at lists.starlingx.io' <starlingx-discuss at lists.starlingx.io>
> > > Subject: RE: CentOS 7.5 upgrade status update
> > >
> > > Hi Cindy,
> > >
> > > We got past the installation device error, but are coming up against
> > > the symbol table error. Our system hangs after this message. I
> > > suspect a kernel issue. I see there is an open code review to
> > > refactor some kernel patches to a rebased kernel
> > > (https://review.open
> > > stack.org/#/c/601204/) but I do not know if that is expected to
> > > resolve the issue
> > >
> > > -J
> > >
> > > From: Xie, Cindy <cindy.xie at intel.com>
> > > Sent: September 11, 2018 9:32 AM
> > > To: Rasa, Nimalini <Nimalini.Rasa at windriver.com>; Waheed, Numan
> > <Numa
> > > n.Waheed at windriver.com>; Lin, Shuicheng <shuicheng.lin at intel.com>;
> > > McKenna, Jason <Jason.McKenna at windriver.com>; Liu, ZhipengS
> > <zhipengs
> > > .liu at intel.com>; Khalil, Ghada <Ghada.Khalil at windriver.com>;
> > > 'starlin gx-discuss at lists.starlingx.io'
> > > <starlingx-discuss at lists.starlingx.io>
> > > Subject: RE: CentOS 7.5 upgrade status update
> > >
> > > But as Jason reported yesterday, the build cannot be installed with
> > > error messages:
> > >
> > > Installation failed.
> > > ERROR: Specified installation (sda) or boot (sda) device is invalid.
> > >
> > > We cannot debug without more information, the ISO installs in
> > > Intel’s HW. Thus Shuicheng created a new build and attached a
> > > manifest in this email thread. Did WindRiver has a chance to create
> > > a new ISO using today’s manifest file and re-run the test?
> > >
> > > Thx. - cindy
> > >
> > > From: Rasa, Nimalini [mailto:Nimalini.Rasa at windriver.com]
> > > Sent: Tuesday, September 11, 2018 9:25 PM
> > > To: Xie, Cindy <cindy.xie at intel.com>; Waheed, Numan
> > <Numan.Waheed at win
> > > driver.com>; Lin, Shuicheng <shuicheng.lin at intel.com>; McKenna,
> > > Jason <Jason.McKenna at windriver.com>; Liu, ZhipengS
> > <zhipengs.liu at intel.com>
> > > ; Khalil, Ghada <Ghada.Khalil at windriver.com>;
> > > 'starlingx-discuss at list s.starlingx.io'
> > > <starlingx-discuss at lists.starlingx.io>
> > > Subject: RE: CentOS 7.5 upgrade status update
> > >
> > > This is on the same build (yesterday’s build).
> > >
> > > From: Xie, Cindy [mailto:cindy.xie at intel.com]
> > > Sent: Tuesday, September 11, 2018 9:22 AM
> > > To: Rasa, Nimalini; Waheed, Numan; Lin, Shuicheng; McKenna, Jason;
> > > Liu, ZhipengS; Khalil, Ghada; 'starlingx-discuss at lists.starlingx.io'
> > > Subject: RE: CentOS 7.5 upgrade status update
> > >
> > > Is this the same issue as yesterday’s build? or this is different
> > > issue now?
> > >
> > > From: Rasa, Nimalini [mailto:Nimalini.Rasa at windriver.com]
> > > Sent: Tuesday, September 11, 2018 9:06 PM
> > > To: Waheed, Numan <Numan.Waheed at windriver.com>; Lin, Shuicheng
> > <shuic
> > > heng.lin at intel.com>; McKenna, Jason
> <Jason.McKenna at windriver.com>;
> > > Liu, ZhipengS <zhipengs.liu at intel.com>; Khalil, Ghada
> > > <Ghada.Khalil at w indriver.com>;
> > > 'starlingx-discuss at lists.starlingx.io' <starlingx-disc
> > > uss at lists.starlingx.io>; Xie, Cindy <cindy.xie at intel.com>
> > > Subject: RE: CentOS 7.5 upgrade status update
> > >
> > > It is not booting in my case. After the following , it just hangs
> > > there:
> > > “Error: no symbol table.
> > > Press any key to continue…”
> > >
> > > -Nimalini
> > >
> > > From: Waheed, Numan
> > > Sent: Tuesday, September 11, 2018 8:50 AM
> > > To: Lin, Shuicheng; McKenna, Jason; Liu, ZhipengS; Khalil, Ghada;
> > > 'st arlingx-discuss at lists.starlingx.io'; Xie, Cindy; Rasa, Nimalini
> > > Subject: RE: CentOS 7.5 upgrade status update
> > >
> > > Adding Nimalini
> > >
> > > From: Lin, Shuicheng [mailto:shuicheng.lin at intel.com]
> > > Sent: September-11-18 2:17 AM
> > > To: McKenna, Jason; Liu, ZhipengS; Khalil, Ghada;
> > > 'starlingx-discuss@ lists.starlingx.io'; Xie, Cindy; Waheed, Numan
> > > Subject: RE: CentOS 7.5 upgrade status update
> > >
> > > Hi Jason,
> > > Just finished deploy image with latest code, I could config
> > > controller successfully in bare metal environment.
> > > Manifest is created for you reference. There are two patches I add
> > > manually:
> > > https://review.openstack.org/596855 for python-horizon
> > > from master
> > > https://review.openstack.org/599892 for build-iso
> > >
> > >
> > > There is an issue found when reboot controller node.
> > > After the OS selection menu, it will show below error. And it will
> > > auto continue boot after around 8 seconds. I will check the issue
> > > later.
> > > “
> > > Error: no symbol table.
> > > Press any key to continue…
> > > “
> > >
> > > Best Regards
> > > Shuicheng
> > >
> > > From: Lin, Shuicheng [mailto:shuicheng.lin at intel.com]
> > > Sent: Tuesday, September 11, 2018 9:35 AM
> > > To: McKenna, Jason <Jason.McKenna at windriver.com>; Liu, ZhipengS
> > <zhip
> > > engs.liu at intel.com>; Khalil, Ghada <Ghada.Khalil at windriver.com>;
> > > 'sta rlingx-discuss at lists.starlingx.io'
> > > <starlingx-discuss at lists.starlingx
> > > .io>; Xie, Cindy <cindy.xie at intel.com>; Waheed, Numan
> > <Numan.Waheed at w
> > > indriver.com>
> > > Subject: Re: [Starlingx-discuss] CentOS 7.5 upgrade status update
> > >
> > > Hi Jason,
> > > I just tried my last Friday’s image at bare metal machine. There is
> > > no issue to install controller node.
> > > Will you help check what cause the install error in your environment?
> > >
> > > I am syncing my code to latest to double confirm it.
> > >
> > > Best Regards
> > > Shuicheng
> > >
> > > From: McKenna, Jason [mailto:Jason.McKenna at windriver.com]
> > > Sent: Tuesday, September 11, 2018 12:13 AM
> > > To: Liu, ZhipengS <zhipengs.liu at intel.com>; Lin, Shuicheng
> > > <shuicheng .lin at intel.com>; Khalil, Ghada
> > > <Ghada.Khalil at windriver.com>; 'starlin
> > > gx-discuss at lists.starlingx.io'
> > > <starlingx-discuss at lists.starlingx.io>
> > > ; Xie, Cindy <cindy.xie at intel.com>; Waheed, Numan
> > <Numan.Waheed at windr
> > > iver.com>
> > > Subject: RE: CentOS 7.5 upgrade status update
> > >
> > > Status update. The build from the centos75 branch does not install
> > > on our hardware. The error message we are seeing is:
> > >
> > > Installation failed.
> > >
> > > ERROR: Specified installation (sda) or boot (sda) device is invalid.
> > >
> > > -Jason
> > >
> > > From: Liu, ZhipengS <zhipengs.liu at intel.com>
> > > Sent: September 8, 2018 9:56 AM
> > > To: Lin, Shuicheng <shuicheng.lin at intel.com>; McKenna, Jason
> > > <Jason.M cKenna at windriver.com>; Khalil, Ghada
> > > <Ghada.Khalil at windriver.com>; 's
> > > tarlingx-discuss at lists.starlingx.io'
> > > <starlingx-discuss at lists.starlin gx.io>; Xie, Cindy
> > > <cindy.xie at intel.com>; Waheed, Numan
> > <Numan.Waheed
> > > @windriver.com>
> > > Subject: RE: CentOS 7.5 upgrade status update
> > >
> > > Hi Jason,
> > >
> > > I have fixed the issue you mentioned below.
> > > - Furthermore, the tpm2-tools package was requesting version
> > > tpm2-tss-1.3.0-2.el7.centos (rather than tpm2-tss-1.3.0-2.el7). I
> > > hacked the spec file for tpm2-tools and rebuilt it to require the
> > > version mentioned, but there is some rework to be done there.
> > > [slin14]: I have the same issue also. Let’s further check it to find
> > > the root cause. I use rebuild tpm2-tools/tss2 to workaround it
> > > without any code.
> > >
> > > https://review.openstack.org/#/c/600920/
> > >
> > > Thanks!
> > > Zhipeng
> > > From: Lin, Shuicheng [mailto:shuicheng.lin at intel.com]
> > > Sent: 2018年9月8日 21:14
> > > To: McKenna, Jason <Jason.McKenna at windriver.com>; Khalil, Ghada
> > <Ghad
> > > a.Khalil at windriver.com>; 'starlingx-discuss at lists.starlingx.io'
> > > <star lingx-discuss at lists.starlingx.io>; Xie, Cindy
> > > <cindy.xie at intel.com>; Waheed, Numan
> <Numan.Waheed at windriver.com>
> > > Subject: Re: [Starlingx-discuss] CentOS 7.5 upgrade status update
> > >
> > > Hi Jason,
> > > I have fixed the error in the openstack-aodh. Patch is updated:
> > > https://review.openstack.org/#/c/598388/
> > >
> > > It is due to my local version is not the latest one.
> > > Sorry for the trouble caused, and thanks for your test.
> > >
> > > Best Regards
> > > Shuicheng
> > >
> > > From: Lin, Shuicheng [mailto:shuicheng.lin at intel.com]
> > > Sent: Saturday, September 8, 2018 9:23 AM
> > > To: McKenna, Jason <Jason.McKenna at windriver.com>; Khalil, Ghada
> > <Ghad
> > > a.Khalil at windriver.com>; 'starlingx-discuss at lists.starlingx.io'
> > > <star lingx-discuss at lists.starlingx.io>; Xie, Cindy
> > > <cindy.xie at intel.com>; Waheed, Numan
> <Numan.Waheed at windriver.com>
> > > Subject: Re: [Starlingx-discuss] CentOS 7.5 upgrade status update
> > >
> > > Hi Jason,
> > > Thanks very much for such detail feedback.
> > > I added some comments inline, and will check the issue in my
> > > environment also.
> > > Feel free to submit patch if you have to fix them. J
> > >
> > >
> > > Best Regards
> > > Shuicheng
> > >
> > > From: McKenna, Jason [mailto:Jason.McKenna at windriver.com]
> > > Sent: Saturday, September 8, 2018 3:47 AM
> > > To: McKenna, Jason <Jason.McKenna at windriver.com>; Khalil, Ghada
> > <Ghad
> > > a.Khalil at windriver.com>; Lin, Shuicheng <shuicheng.lin at intel.com>;
> > > 's tarlingx-discuss at lists.starlingx.io'
> > > <starlingx-discuss at lists.starlin gx.io>; Xie, Cindy
> > > <cindy.xie at intel.com>; Waheed, Numan
> > <Numan.Waheed
> > > @windriver.com>
> > > Subject: RE: CentOS 7.5 upgrade status update
> > >
> > > Hi Shuicheng,
> > >
> > > I’ve completed a build of the code. We have not yet run run-time
> > > tests, but here are the build-time issues I encountered:
> > >
> > > - The .lst file contains lighttpd-1.4.49, but this is no
> > > longer available for download, and the download_mirrors.sh step fails.
> > > It appears than CentOS has updated to lighttpd-1.4.50. I manually
> > > built a lighttpd-1.4.49-1.src.rpm to satisfy the .lst
> > > - Even though the .lst file specifies lighttpd-1.4.49, the
> > > code still references 1.4.45
> > > - At build-pkgs time (the build-srpms step), the meta-patches
> > > to openstack-aodh do not apply, as two of them are malformed. https:
> > > //review.openstack.org/#/c/598388/6/openstack/openstack-
> > > aodh/centos/meta_patches/0001-Modify-service-files-and-create-
> > > expirer-cron-script.patch -- In the new version, on line 190, the
> > > patch states that the old code covered 6 lines, and the new code
> > > covers 9 lines (-283,6 +286,9). But there are only two lines
> > > inserted by the patch. There is also a new blank line in the patch
> > > at line 194. Removing this blank line and editing the hunk
> > > description to only insert 2 lines resolves the issue for this hunk
> > > (there are other similar issues in this patch, and in 0001-meta-
> > > modify-aodh-api.patch)
> > > [slin14]: Let me double check it.
> > > - At binary build time, I had build failures on python-django-
> > > horizon and some magnum packages. I had to pull in
> > > https://review.op
> > > enstack.org/#/c/596855/2 (from the master branch, but not in the
> > > centos75 branch of stx-upstream) to get past this.
> > > [slin14]: It is due to I have not synced latest master code, so I
> > > don’t have this issue. I used last week’s master code.
> > > - After building all packages, I attempted to build the ISO.
> > > The tpm2-tss-1.3.0-2.el7.x86_64.rpm package were missing, and should
> > > be added to the .lst files.
> > > [slin14]: This package is already in the lst.
> > > - Furthermore, the tpm2-tools package was requesting version
> > > tpm2-tss-1.3.0-2.el7.centos (rather than tpm2-tss-1.3.0-2.el7). I
> > > hacked the spec file for tpm2-tools and rebuilt it to require the
> > > version mentioned, but there is some rework to be done there.
> > > [slin14]: I have the same issue also. Let’s further check it to find
> > > the root cause. I use rebuild tpm2-tools/tss2 to workaround it
> > > without any code.
> > >
> > > After these changes, I was able to produce a .iso
> > >
> > > -Jason
> > >
> > > From: McKenna, Jason <Jason.McKenna at windriver.com>
> > > Sent: September 7, 2018 10:08 AM
> > > To: Khalil, Ghada <Ghada.Khalil at windriver.com>; Lin, Shuicheng
> > > <shuic heng.lin at intel.com>; 'starlingx-discuss at lists.starlingx.io'
> > > <starling x-discuss at lists.starlingx.io>; Xie, Cindy
> > > <cindy.xie at intel.com>; Waheed, Numan
> <Numan.Waheed at windriver.com>
> > > Subject: Re: [Starlingx-discuss] CentOS 7.5 upgrade status update
> > >
> > > Hi Shuicheng,
> > >
> > > As a download_mirrors.sh time issue, the version of lighttpd has
> > > been changed from 1.4.45 to 1.4.49. The
> > > lighttpd-1.4.49-1.el7.src.rpm file no longer exists on EPEL or
> > > CentOS servers, as they’ve moved to 1.4.50. I am attempting to
> > > locate an archived version of the 1.4.49 .src rpm.
> > >
> > > -Jason
> > >
> > > From: Khalil, Ghada <Ghada.Khalil at windriver.com>
> > > Sent: September 7, 2018 9:36 AM
> > > To: Lin, Shuicheng <shuicheng.lin at intel.com>;
> > > 'starlingx-discuss at list s.starlingx.io'
> > > <starlingx-discuss at lists.starlingx.io>; Xie, Cindy <c
> > > indy.xie at intel.com>; Waheed, Numan
> <Numan.Waheed at windriver.com>
> > > Subject: Re: [Starlingx-discuss] CentOS 7.5 upgrade status update
> > >
> > > Thanks Shuicheng. We’ll run some tests in Wind River. Numan will be
> > > priming this activity. He’ll share the results with the team
> > > sometime next week.
> > >
> > > Regards,
> > > Ghada
> > >
> > > From: Lin, Shuicheng [mailto:shuicheng.lin at intel.com]
> > > Sent: Friday, September 07, 2018 1:32 AM
> > > To: 'starlingx-discuss at lists.starlingx.io'; Khalil, Ghada; Xie,
> > > Cindy
> > > Subject: CentOS 7.5 upgrade status update
> > >
> > > Hi all,
> > > Most task of CentOS7.5 upgrade in story 2003389 is done. Please go
> > > ahead to have a try and verify it.
> > > I expect there should be some issue due to we just did limited
> > > deploy test yet.
> > >
> > > Here is the detail status update of this week.
> > > For the total 44 srpm we plan to upgrade:
> > > 38 already merged in feature branch.
> > > 3 kept as original version. Anaconda upgrade is not needed. python-
> > > os-vif has higher version than 7.5. python-eventlet kept due to new
> > > src rpm itself have known issue.
> > > 2 under review with code ready. Puppet-haproxy and openstack-aodh.
> > > 1 under debug. Lighttpd.
> > >
> > > Besides srpm, we also upgraded 260+ rpm packages to 7.5 version.
> > >
> > > Here is the build instruction:
> > > 1. Switch to f/centos75 branch for stx-tools/stx-integ/stx-
> > > upstream/stx-root.
> > > Cherry-pick below un-merged patch:
> > > For puppet-haproxy:
> > > https://review.openstack.org/600369
> > > https://review.openstack.org/600359
> > > For openstack-aodh:
> > > https://review.openstack.org/598388
> > > https://review.openstack.org/600591
> > > For build-iso:
> > > https://review.openstack.org/599892
> > > For build-pkgs:
> > > https://review.openstack.org/599901
> > > https://review.openstack.org/600639
> > >
> > > 2. Run mirror script to get the upgraded src rpm and related
> > > rpm.
> > > 3. Try to build packages and ISO.
> > >
> > > For the detail status of CentOS7.5 upgrade, please check below story:
> > > https://storyboard.openstack.org/#!/story/2003389
> > >
> > > Feel free to contact me if you have trouble on it.
> > >
> > > Best Regards
> > > Shuicheng
> > > _______________________________________________
> > > Starlingx-discuss mailing list
> > > Starlingx-discuss at lists.starlingx.io
> > > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
More information about the Starlingx-discuss
mailing list