[Starlingx-discuss] CentOS 7.5 upgrade status update
Rasa, Nimalini
Nimalini.Rasa at windriver.com
Fri Sep 14 16:46:04 UTC 2018
Hi Cindy,
Here is the last test update, I am reporting the failed test cases only here:
From Sanity:
FAIL 20180914 04:32:46 test_evacuate_vms --> Ping Vms failed and this is seen intermittently on STX load as well (can be ignored for this testing puprpose)
From Nightly:
FAIL 20180914 06:54:06 test_schedule_vm_mempage_config[1048576]
LOG:
====================== Setup Step 6: (module) Execute modify function: <function add_1g_and_4k_pages.<locals>._modify at 0x7fc023a03d08>
[2018-09-14 06:55:20,035] 1300 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_1_4
[2018-09-14 06:55:20,035] 419 DEBUG MainThread ssh.exec_cmd:: Executing command...
[2018-09-14 06:55:20,036] 262 DEBUG MainThread ssh.send :: Send 'system --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://192.168.204.2:5000/v3 --os-user-domain-name Default --os-project-domain-name Default --os-region-name RegionOne host-memory-list --nowrap compute-1'
[2018-09-14 06:55:21,883] 382 DEBUG MainThread ssh.expect :: Output:
+-----------+----------------+-------------------+----------------+--------------------------+-----------------+-------------+-------------+-------------+----------------+----------------+------------------+----------------+----------------+------------------+--------------+
| processor | mem_total(MiB) | mem_platform(MiB) | mem_avail(MiB) | hugepages(hp)_configured | vs_hp_size(MiB) | vs_hp_total | vs_hp_avail | vm_total_4K | vm_hp_total_2M | vm_hp_avail_2M | vm_hp_pending_2M | vm_hp_total_1G | vm_hp_avail_1G | vm_hp_pending_1G | vm_hp_use_1G |
+-----------+----------------+-------------------+----------------+--------------------------+-----------------+-------------+-------------+-------------+----------------+----------------+------------------+----------------+----------------+------------------+--------------+
| 0 | 7540 | 8000 | 2420 | True | 1024 | 1 | 0 | 20992 | 3258 | 1210 | None | 0 | 0 | None | True |
| 1 | 13770 | 2000 | 12746 | True | 1024 | 1 | 0 | 0 | 6373 | 6373 | None | 0 | 0 | None | True |
+-----------+----------------+-------------------+----------------+--------------------------+-----------------+-------------+-------------+-------------+----------------+----------------+------------------+----------------+----------------+------------------+--------------+
controller-1:~$
[2018-09-14 06:55:21,884] 262 DEBUG MainThread ssh.send :: Send 'echo $?'
[2018-09-14 06:55:21,986] 382 DEBUG MainThread ssh.expect :: Output:
0
controller-1:~$
[2018-09-14 06:55:21,987] 366 DEBUG MainThread table_parser.__get_row_indexes_string:: row index list for processor: 0: [0]
[2018-09-14 06:55:21,987] 498 DEBUG MainThread table_parser.get_values:: Returning matching mem_avail(MiB) value(s): ['2420']
[2018-09-14 06:55:21,987] 366 DEBUG MainThread table_parser.__get_row_indexes_string:: row index list for processor: 0: [0]
[2018-09-14 06:55:21,988] 498 DEBUG MainThread table_parser.get_values:: Returning matching vm_hp_total_1G value(s): ['0']
[2018-09-14 06:55:21,988] 366 DEBUG MainThread table_parser.__get_row_indexes_string:: row index list for processor: 0: [0]
[2018-09-14 06:55:21,988] 498 DEBUG MainThread table_parser.get_values:: Returning matching vm_hp_pending_1G value(s): ['None']
[2018-09-14 06:55:21,988] 366 DEBUG MainThread table_parser.__get_row_indexes_string:: row index list for processor: 1: [1]
[2018-09-14 06:55:21,989] 498 DEBUG MainThread table_parser.get_values:: Returning matching mem_avail(MiB) value(s): ['12746']
[2018-09-14 06:55:21,989] 366 DEBUG MainThread table_parser.__get_row_indexes_string:: row index list for processor: 1: [1]
[2018-09-14 06:55:21,989] 498 DEBUG MainThread table_parser.get_values:: Returning matching vm_hp_total_1G value(s): ['0']
[2018-09-14 06:55:21,989] 366 DEBUG MainThread table_parser.__get_row_indexes_string:: row index list for processor: 1: [1]
[2018-09-14 06:55:21,990] 498 DEBUG MainThread table_parser.get_values:: Returning matching vm_hp_pending_1G value(s): ['None']
[2018-09-14 06:55:21,990] 1300 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_1_4
[2018-09-14 06:55:21,990] 419 DEBUG MainThread ssh.exec_cmd:: Executing command...
[2018-09-14 06:55:21,990] 262 DEBUG MainThread ssh.send :: Send 'system --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://192.168.204.2:5000/v3 --os-user-domain-name Default --os-project-domain-name Default --os-region-name RegionOne host-memory-modify -2M 2420 compute-1 0'
[2018-09-14 06:55:24,362] 382 DEBUG MainThread ssh.expect :: Output:
+--------------------------------+--------------------------------------+
| Property | Value |
+--------------------------------+--------------------------------------+
| Memory: Usable Total (MiB) | 7540 |
| Platform (MiB) | 8000 |
| Available (MiB) | 2420 |
| Huge Pages Configured | True |
| vSwitch Huge Pages: Size (MiB) | 1024 |
| Total | 1 |
| Available | 0 |
| VM Pages (4K): Total | 20992 |
| VM Huge Pages (2M): Total | 3258 |
| Total Pending | 2420 |
| Available | 1210 |
| VM Huge Pages (1G): Total | 0 |
| Available | 0 |
| uuid | 8f317d7a-3bcd-4b2d-bd1e-704429d0e0c5 |
| ihost_uuid | 4189d5c3-0c85-49b9-9fc9-1d54540c54ea |
| inode_uuid | 96430065-9a83-4e33-ab78-c7de77cf8bc9 |
| created_at | 2018-09-14T02:30:56.591922+00:00 |
| updated_at | 2018-09-14T06:48:29.882522+00:00 |
+--------------------------------+--------------------------------------+
controller-1:~$
[2018-09-14 06:55:24,362] 262 DEBUG MainThread ssh.send :: Send 'echo $?'
[2018-09-14 06:55:24,465] 382 DEBUG MainThread ssh.expect :: Output:
0
EXPECTED TO REJECT: HERE IS AN EXAMPLE FROM PASSED LOG ON DIFF LOAD:
[2018-09-14 12:37:33,741] 262 DEBUG MainThread ssh.send :: Send 'system --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://192.168.204.2:5000/v3 --os-user-domain-name Default --os-project-domain-name Default --os-region-name RegionOne host-memory-modify -2M 53818 compute-1 0'
[2018-09-14 12:37:36,023] 382 DEBUG MainThread ssh.expect :: Output:
Processor 0:No available space for 2M huge page allocation, max 2M pages: 26848
FAIL 20180914 07:00:37 test_schedule_vm_mempage_config[small]
This test failed b/c of the above test failure (Alarm was present, compute was in locked state)
FAIL 20180914 07:57:34 test_kernel_module_signatures
LOG
[2018-09-14 07:57:37,946] 24 INFO MainThread test_kernel_modules.test_kernel_module_signatures::
====================== Test Step 1: Check for unassigned kernel modules on controller-0
[2018-09-14 07:57:37,947] 419 DEBUG MainThread ssh.exec_cmd:: Executing command...
[2018-09-14 07:57:37,947] 262 DEBUG MainThread ssh.send :: Send 'cat /proc/sys/kernel/tainted'
[2018-09-14 07:57:38,050] 382 DEBUG MainThread ssh.expect :: Output:
12288
controller-0:~$
[2018-09-14 07:57:38,050] 262 DEBUG MainThread ssh.send :: Send 'echo $?'
[2018-09-14 07:57:38,152] 382 DEBUG MainThread ssh.expect :: Output:
0
controller-0:~$
[2018-09-14 07:57:38,153] 32 ERROR MainThread test_kernel_modules.test_kernel_module_signatures:: Kernel module verification(s) failed on controller-0. Collecting more info
[2018-09-14 07:57:38,153] 34 INFO MainThread test_kernel_modules.test_kernel_module_signatures::
====================== Test Step 2: Check kern.log for modules with failed verification
[2018-09-14 07:57:38,153] 419 DEBUG MainThread ssh.exec_cmd:: Executing command...
[2018-09-14 07:57:38,153] 262 DEBUG MainThread ssh.send :: Send 'grep --color=never -i "module verification failed" /var/log/kern.log'
[2018-09-14 07:57:38,265] 382 DEBUG MainThread ssh.expect :: Output:
2018-09-13T19:09:51.212 localhost kernel: notice [ 6.271466] e1000e: module verification failed: signature and/or required key missing - tainting kernel
2018-09-13T19:51:42.174 controller-0 kernel: notice [ 6.133741] e1000e: module verification failed: signature and/or required key missing - tainting kernel
2018-09-14T03:33:29.054 controller-0 kernel: notice [ 6.183120] e1000e: module verification failed: signature and/or required key missing - tainting kernel
controller-0:~$
[2018-09-14 07:57:38,265] 262 DEBUG MainThread ssh.send :: Send 'echo $?'
[2018-09-14 07:57:38,367] 382 DEBUG MainThread ssh.expect :: Output:
0
controller-0:~$
[2018-09-14 07:57:38,368] 43 INFO MainThread test_kernel_modules.test_kernel_module_signatures::
====================== Test Step 3: Display signing info for controller-0 failed kernel modules: ['e1000e']
[2018-09-14 07:57:38,368] 419 DEBUG MainThread ssh.exec_cmd:: Executing command...
[2018-09-14 07:57:38,368] 262 DEBUG MainThread ssh.send :: Send 'modinfo e1000e | grep --color=never -E "sig|filename"'
[2018-09-14 07:57:38,474] 382 DEBUG MainThread ssh.expect :: Output:
filename: /lib/modules/3.10.0-862.11.6.el7.36.tis.x86_64/weak-updates/e1000e/e1000e.ko
signer: CentOS Linux kernel signing key
sig_key: 58:7B:E7:83:22:7A:67:CA:49:54:D2:3B:57:AF:99:45:D3:AC:99:AC
sig_hashalgo: sha256
controller-0:~$
[2018-09-14 07:57:38,475] 262 DEBUG MainThread ssh.send :: Send 'echo $?'
[2018-09-14 07:57:38,577] 382 DEBUG MainThread ssh.expect :: Output:
Thanks,
Nimalini
-----Original Message-----
From: Xie, Cindy [mailto:cindy.xie at intel.com]
Sent: Thursday, September 13, 2018 10:05 AM
To: McKenna, Jason; Martinez Monroy, Elio; Waheed, Numan; Rasa, Nimalini; starlingx-discuss at lists.starlingx.io
Cc: Khalil, Ghada; Lin, Shuicheng
Subject: RE: [Starlingx-discuss] CentOS 7.5 upgrade status update
Thanks Jason for the update. Appreciate you can have a new build generated according to Shuicheng's email earlier. @Nimalini, please report any finding back.
The failure test cases in your both reports, is this the bug link? https://bugs.launchpad.net/starlingx/+bug/1791818
Thanks. - cindy
-----Original Message-----
From: McKenna, Jason [mailto:Jason.McKenna at windriver.com]
Sent: Thursday, September 13, 2018 9:35 PM
To: Xie, Cindy <cindy.xie at intel.com>; Martinez Monroy, Elio <elio.martinez.monroy at intel.com>; Waheed, Numan <Numan.Waheed at windriver.com>; Rasa, Nimalini <Nimalini.Rasa at windriver.com>; starlingx-discuss at lists.starlingx.io
Cc: Khalil, Ghada <Ghada.Khalil at windriver.com>; Lin, Shuicheng <shuicheng.lin at intel.com>
Subject: RE: [Starlingx-discuss] CentOS 7.5 upgrade status update
Hi Cindy,
Will do. In the meantime, Nimalini (one of our validation gurus) was able to boot a build which I had done a couple of days ago, and perform sanity and our nightly regression suite. The results (attached) align with the 7.4 based master branch of StarlingX.
-Jason
> -----Original Message-----
> From: Xie, Cindy <cindy.xie at intel.com>
> Sent: September 13, 2018 2:44 AM
> To: Martinez Monroy, Elio <elio.martinez.monroy at intel.com>; McKenna,
> Jason <Jason.McKenna at windriver.com>; Waheed, Numan
> <Numan.Waheed at windriver.com>; Rasa, Nimalini
> <Nimalini.Rasa at windriver.com>; starlingx-discuss at lists.starlingx.io
> Cc: Khalil, Ghada <Ghada.Khalil at windriver.com>; Lin, Shuicheng
> <shuicheng.lin at intel.com>
> Subject: RE: [Starlingx-discuss] CentOS 7.5 upgrade status update
>
> Elio,
> We will provide you an updated ISO later today with the following:
> - kernel upgrade to 3.10.0-862.11.6
> - symbol issue fixed:
> https://bugs.launchpad.net/starlingx/+bug/1792257
>
> Please help to do a sanity test based on this new ISO from Shuicheng on VE.
>
> @Jason,
> Please re-generate a build using today's manifest with above fix.
> Shuicheng shall be able to provide manifest with exact commitID.
> @Numan, please abort your current testing w/ old ISO once you get a
> new one from Jason and continue your sanity and regression on
> bare-metal. I am not sure how long it takes, but please take priority
> as we want merge back to mainline early next week.
>
> Thanks all for the support.
> - cindy
>
> -----Original Message-----
> From: McKenna, Jason [mailto:Jason.McKenna at windriver.com]
> Sent: Thursday, September 13, 2018 4:24 AM
> To: Lin, Shuicheng <shuicheng.lin at intel.com>; Cordoba Malibran, Erich
> <erich.cordoba.malibran at intel.com>; Waheed, Numan
> <Numan.Waheed at windriver.com>; Rasa, Nimalini
> <Nimalini.Rasa at windriver.com>; starlingx-discuss at lists.starlingx.io;
> Liu, ZhipengS <zhipengs.liu at intel.com>; Khalil, Ghada
> <Ghada.Khalil at windriver.com>; Xie, Cindy <cindy.xie at intel.com>
> Subject: RE: [Starlingx-discuss] CentOS 7.5 upgrade status update
>
> > -----Original Message-----
> > From: Lin, Shuicheng <shuicheng.lin at intel.com>
> > Sent: September 11, 2018 8:44 PM
> > To: McKenna, Jason <Jason.McKenna at windriver.com>; Cordoba Malibran,
> > Erich <erich.cordoba.malibran at intel.com>; Waheed, Numan
> > <Numan.Waheed at windriver.com>; Rasa, Nimalini
> > <Nimalini.Rasa at windriver.com>; starlingx-discuss at lists.starlingx.io;
> > Liu, ZhipengS <zhipengs.liu at intel.com>; Khalil, Ghada
> > <Ghada.Khalil at windriver.com>; Xie, Cindy <cindy.xie at intel.com>
> > Subject: RE: [Starlingx-discuss] CentOS 7.5 upgrade status update
> >
> > Hi Jason,
> > Here is the dhcp src rpm we are using:
> > [slin14 at f998ec953c85 /]$ sudo -E yumdownloader -C --url --source
> > dhcp
> > http://vault.centos.org/7.5.1804/updates/Source/SPackages/dhcp-4.2.5
> > -
> > 68.el7.centos.1.src.rpm
> >
> > For centos-release, there are two packages in the mirror lst, 1 src
> > rpm for build, and 1 rpm for the mock environment. And it has the
> > "slightly difference" in the names as you mentioned.
> > Both of them could be found in the centos repo with the exact name.
> > ./rpms_centos.lst:135:centos-release-7-5.1804.1.el7.centos.src.rpm
> > ./rpms_centos.lst:136:centos-release-7-5.1804.el7.centos.x86_64.rpm
> > [slin14 at f998ec953c85 /]$ sudo -E yumdownloader -C --url --source
> > centos- release-7-5.1804.1.el7.centos
> > http://vault.centos.org/7.5.1804/updates/Source/SPackages/centos-
> > release-7-5.1804.1.el7.centos.src.rpm
> > [slin14 at f998ec953c85 /]$ sudo -E yumdownloader -C --url
> > centos-release-7-
> > 5.1804.el7.centos.x86_64
> > http://mirror.centos.org/centos/7.5.1804/os/x86_64/Packages/centos-
> > release-7-5.1804.el7.centos.x86_64.rpm
> >
> > If there is still some other rpm missing, you could try to find them
> > from the repo.
> > I have try my best to keep my local mirror the same as the lst. But
> > it is still possible some package missed to be added to the repo lst.
> > Please feel free to notify me or correct the error directly.
> > Thanks.
>
> With the latest commits to stx-tools I am now able to download the
> rpms and produce a build. I'll let you know how testing goes
>
> >
> > Best Regards
> > Shuicheng
> >
> >
> > -----Original Message-----
> > From: McKenna, Jason [mailto:Jason.McKenna at windriver.com]
> > Sent: Wednesday, September 12, 2018 12:21 AM
> > To: Cordoba Malibran, Erich <erich.cordoba.malibran at intel.com>;
> > Waheed, Numan <Numan.Waheed at windriver.com>; Rasa, Nimalini
> > <Nimalini.Rasa at windriver.com>; starlingx-discuss at lists.starlingx.io;
> > Liu, ZhipengS <zhipengs.liu at intel.com>; Khalil, Ghada
> > <Ghada.Khalil at windriver.com>; Lin, Shuicheng
> > <shuicheng.lin at intel.com>; Xie, Cindy <cindy.xie at intel.com>
> > Subject: RE: [Starlingx-discuss] CentOS 7.5 upgrade status update
> >
> > Hi Cindy,
> >
> > With the latest centos75 branch (and the cherry-picks you mentioned)
> > I am unable to produce a build. There seems to be several source
> > rpm files missing from the currently pushed version of the .lst
> > files (or the .lst files pull down the wrong version). Most of the
> > missing files I can download manually, however the checked in
> > patches for dhcp and centos-release packages do not apply against
> > the version of the source RPM which I can manually find on the
> > Internet. For example, the version of centos-release called out in
> > the .lst is centos-release-7-5.1804.el7.centos.x86_64.rpm, the
> > version called out in the checked in code base is the slightly
> > different
> > centos-release-7- 5.1804.el7.centos.1.x86_64.rpm, and the patches
> checked in do not apply against either package.
> >
> > Until the centos75 branch of the lst files pulls down the correct
> > rpms, I will be unable to continue the validation effort.
> >
> > -J
> >
> > > -----Original Message-----
> > > From: Cordoba Malibran, Erich <erich.cordoba.malibran at intel.com>
> > > Sent: September 11, 2018 11:17 AM
> > > To: Waheed, Numan <Numan.Waheed at windriver.com>; Rasa, Nimalini
> > > <Nimalini.Rasa at windriver.com>;
> > > starlingx-discuss at lists.starlingx.io;
> > > Liu, ZhipengS <zhipengs.liu at intel.com>; Khalil, Ghada
> > > <Ghada.Khalil at windriver.com>; Lin, Shuicheng
> > > <shuicheng.lin at intel.com>; McKenna, Jason
> > > <Jason.McKenna at windriver.com>; Xie, Cindy <cindy.xie at intel.com>
> > > Subject: Re: [Starlingx-discuss] CentOS 7.5 upgrade status update
> > >
> > > Hi
> > >
> > > You can add "debug loglevel=8" to the kernel boot option to get a
> > > more detailed output of the crash (if any)
> > >
> > > -Erich
> > >
> > > On Tue, 2018-09-11 at 13:56 +0000, Xie, Cindy wrote:
> > > > Shuicheng saw the symbol table error issue as well but it
> > > > recovers in 8s, he is still working on debugging it. But 601204
> > > > is just kernel upgrade but not intended to solve this issues.
> > > > Are you running bare metal? I am wondering if this is HW config and driver issue.
> > > >
> > > > Is it possible that you try to deploy it in virtual environment?
> > > >
> > > > Thx. - cindy
> > > >
> > > > From: McKenna, Jason [mailto:Jason.McKenna at windriver.com]
> > > > Sent: Tuesday, September 11, 2018 9:50 PM
> > > > To: Xie, Cindy <cindy.xie at intel.com>; Rasa, Nimalini
> > > > <Nimalini.Rasa at w indriver.com>; Waheed, Numan
> > > > <Numan.Waheed at windriver.com>; Lin, Shuicheng
> > > > <shuicheng.lin at intel.com>; Liu, ZhipengS <zhipengs.liu at inte
> > > > l.com>; Khalil, Ghada <Ghada.Khalil at windriver.com>;
> > > > 'starlingx-discus s at lists.starlingx.io'
> > > > <starlingx-discuss at lists.starlingx.io>
> > > > Subject: RE: CentOS 7.5 upgrade status update
> > > >
> > > > Hi Cindy,
> > > >
> > > > We got past the installation device error, but are coming up
> > > > against the symbol table error. Our system hangs after this
> > > > message. I suspect a kernel issue. I see there is an open code
> > > > review to refactor some kernel patches to a rebased kernel
> > > > (https://review.open
> > > > stack.org/#/c/601204/) but I do not know if that is expected to
> > > > resolve the issue
> > > >
> > > > -J
> > > >
> > > > From: Xie, Cindy <cindy.xie at intel.com>
> > > > Sent: September 11, 2018 9:32 AM
> > > > To: Rasa, Nimalini <Nimalini.Rasa at windriver.com>; Waheed, Numan
> > > <Numa
> > > > n.Waheed at windriver.com>; Lin, Shuicheng
> <shuicheng.lin at intel.com>;
> > > > McKenna, Jason <Jason.McKenna at windriver.com>; Liu, ZhipengS
> > > <zhipengs
> > > > .liu at intel.com>; Khalil, Ghada <Ghada.Khalil at windriver.com>;
> > > > 'starlin gx-discuss at lists.starlingx.io'
> > > > <starlingx-discuss at lists.starlingx.io>
> > > > Subject: RE: CentOS 7.5 upgrade status update
> > > >
> > > > But as Jason reported yesterday, the build cannot be installed
> > > > with error messages:
> > > >
> > > > Installation failed.
> > > > ERROR: Specified installation (sda) or boot (sda) device is invalid.
> > > >
> > > > We cannot debug without more information, the ISO installs in
> > > > Intel’s HW. Thus Shuicheng created a new build and attached a
> > > > manifest in this email thread. Did WindRiver has a chance to
> > > > create a new ISO using today’s manifest file and re-run the test?
> > > >
> > > > Thx. - cindy
> > > >
> > > > From: Rasa, Nimalini [mailto:Nimalini.Rasa at windriver.com]
> > > > Sent: Tuesday, September 11, 2018 9:25 PM
> > > > To: Xie, Cindy <cindy.xie at intel.com>; Waheed, Numan
> > > <Numan.Waheed at win
> > > > driver.com>; Lin, Shuicheng <shuicheng.lin at intel.com>; McKenna,
> > > > Jason <Jason.McKenna at windriver.com>; Liu, ZhipengS
> > > <zhipengs.liu at intel.com>
> > > > ; Khalil, Ghada <Ghada.Khalil at windriver.com>;
> > > > 'starlingx-discuss at list s.starlingx.io'
> > > > <starlingx-discuss at lists.starlingx.io>
> > > > Subject: RE: CentOS 7.5 upgrade status update
> > > >
> > > > This is on the same build (yesterday’s build).
> > > >
> > > > From: Xie, Cindy [mailto:cindy.xie at intel.com]
> > > > Sent: Tuesday, September 11, 2018 9:22 AM
> > > > To: Rasa, Nimalini; Waheed, Numan; Lin, Shuicheng; McKenna,
> > > > Jason; Liu, ZhipengS; Khalil, Ghada; 'starlingx-discuss at lists.starlingx.io'
> > > > Subject: RE: CentOS 7.5 upgrade status update
> > > >
> > > > Is this the same issue as yesterday’s build? or this is
> > > > different issue now?
> > > >
> > > > From: Rasa, Nimalini [mailto:Nimalini.Rasa at windriver.com]
> > > > Sent: Tuesday, September 11, 2018 9:06 PM
> > > > To: Waheed, Numan <Numan.Waheed at windriver.com>; Lin,
> Shuicheng
> > > <shuic
> > > > heng.lin at intel.com>; McKenna, Jason
> > <Jason.McKenna at windriver.com>;
> > > > Liu, ZhipengS <zhipengs.liu at intel.com>; Khalil, Ghada
> > > > <Ghada.Khalil at w indriver.com>;
> > > > 'starlingx-discuss at lists.starlingx.io' <starlingx-disc
> > > > uss at lists.starlingx.io>; Xie, Cindy <cindy.xie at intel.com>
> > > > Subject: RE: CentOS 7.5 upgrade status update
> > > >
> > > > It is not booting in my case. After the following , it just
> > > > hangs
> > > > there:
> > > > “Error: no symbol table.
> > > > Press any key to continue…”
> > > >
> > > > -Nimalini
> > > >
> > > > From: Waheed, Numan
> > > > Sent: Tuesday, September 11, 2018 8:50 AM
> > > > To: Lin, Shuicheng; McKenna, Jason; Liu, ZhipengS; Khalil,
> > > > Ghada; 'st arlingx-discuss at lists.starlingx.io'; Xie, Cindy;
> > > > Rasa, Nimalini
> > > > Subject: RE: CentOS 7.5 upgrade status update
> > > >
> > > > Adding Nimalini
> > > >
> > > > From: Lin, Shuicheng [mailto:shuicheng.lin at intel.com]
> > > > Sent: September-11-18 2:17 AM
> > > > To: McKenna, Jason; Liu, ZhipengS; Khalil, Ghada;
> > > > 'starlingx-discuss@ lists.starlingx.io'; Xie, Cindy; Waheed,
> > > > Numan
> > > > Subject: RE: CentOS 7.5 upgrade status update
> > > >
> > > > Hi Jason,
> > > > Just finished deploy image with latest code, I could config
> > > > controller successfully in bare metal environment.
> > > > Manifest is created for you reference. There are two patches I
> > > > add
> > > > manually:
> > > > https://review.openstack.org/596855 for
> > > > python-horizon from master
> > > > https://review.openstack.org/599892 for build-iso
> > > >
> > > >
> > > > There is an issue found when reboot controller node.
> > > > After the OS selection menu, it will show below error. And it
> > > > will auto continue boot after around 8 seconds. I will check the
> > > > issue later.
> > > > “
> > > > Error: no symbol table.
> > > > Press any key to continue…
> > > > “
> > > >
> > > > Best Regards
> > > > Shuicheng
> > > >
> > > > From: Lin, Shuicheng [mailto:shuicheng.lin at intel.com]
> > > > Sent: Tuesday, September 11, 2018 9:35 AM
> > > > To: McKenna, Jason <Jason.McKenna at windriver.com>; Liu, ZhipengS
> > > <zhip
> > > > engs.liu at intel.com>; Khalil, Ghada <Ghada.Khalil at windriver.com>;
> > > > 'sta rlingx-discuss at lists.starlingx.io'
> > > > <starlingx-discuss at lists.starlingx
> > > > .io>; Xie, Cindy <cindy.xie at intel.com>; Waheed, Numan
> > > <Numan.Waheed at w
> > > > indriver.com>
> > > > Subject: Re: [Starlingx-discuss] CentOS 7.5 upgrade status
> > > > update
> > > >
> > > > Hi Jason,
> > > > I just tried my last Friday’s image at bare metal machine. There
> > > > is no issue to install controller node.
> > > > Will you help check what cause the install error in your environment?
> > > >
> > > > I am syncing my code to latest to double confirm it.
> > > >
> > > > Best Regards
> > > > Shuicheng
> > > >
> > > > From: McKenna, Jason [mailto:Jason.McKenna at windriver.com]
> > > > Sent: Tuesday, September 11, 2018 12:13 AM
> > > > To: Liu, ZhipengS <zhipengs.liu at intel.com>; Lin, Shuicheng
> > > > <shuicheng .lin at intel.com>; Khalil, Ghada
> > > > <Ghada.Khalil at windriver.com>; 'starlin
> > > > gx-discuss at lists.starlingx.io'
> > > > <starlingx-discuss at lists.starlingx.io>
> > > > ; Xie, Cindy <cindy.xie at intel.com>; Waheed, Numan
> > > <Numan.Waheed at windr
> > > > iver.com>
> > > > Subject: RE: CentOS 7.5 upgrade status update
> > > >
> > > > Status update. The build from the centos75 branch does not
> > > > install on our hardware. The error message we are seeing is:
> > > >
> > > > Installation failed.
> > > >
> > > > ERROR: Specified installation (sda) or boot (sda) device is invalid.
> > > >
> > > > -Jason
> > > >
> > > > From: Liu, ZhipengS <zhipengs.liu at intel.com>
> > > > Sent: September 8, 2018 9:56 AM
> > > > To: Lin, Shuicheng <shuicheng.lin at intel.com>; McKenna, Jason
> > > > <Jason.M cKenna at windriver.com>; Khalil, Ghada
> > > > <Ghada.Khalil at windriver.com>; 's
> > > > tarlingx-discuss at lists.starlingx.io'
> > > > <starlingx-discuss at lists.starlin gx.io>; Xie, Cindy
> > > > <cindy.xie at intel.com>; Waheed, Numan
> > > <Numan.Waheed
> > > > @windriver.com>
> > > > Subject: RE: CentOS 7.5 upgrade status update
> > > >
> > > > Hi Jason,
> > > >
> > > > I have fixed the issue you mentioned below.
> > > > - Furthermore, the tpm2-tools package was requesting version
> > > > tpm2-tss-1.3.0-2.el7.centos (rather than tpm2-tss-1.3.0-2.el7).
> > > > I hacked the spec file for tpm2-tools and rebuilt it to require
> > > > the version mentioned, but there is some rework to be done there.
> > > > [slin14]: I have the same issue also. Let’s further check it to
> > > > find the root cause. I use rebuild tpm2-tools/tss2 to workaround
> > > > it without any code.
> > > >
> > > > https://review.openstack.org/#/c/600920/
> > > >
> > > > Thanks!
> > > > Zhipeng
> > > > From: Lin, Shuicheng [mailto:shuicheng.lin at intel.com]
> > > > Sent: 2018年9月8日 21:14
> > > > To: McKenna, Jason <Jason.McKenna at windriver.com>; Khalil, Ghada
> > > <Ghad
> > > > a.Khalil at windriver.com>; 'starlingx-discuss at lists.starlingx.io'
> > > > <star lingx-discuss at lists.starlingx.io>; Xie, Cindy
> > > > <cindy.xie at intel.com>; Waheed, Numan
> > <Numan.Waheed at windriver.com>
> > > > Subject: Re: [Starlingx-discuss] CentOS 7.5 upgrade status
> > > > update
> > > >
> > > > Hi Jason,
> > > > I have fixed the error in the openstack-aodh. Patch is updated:
> > > > https://review.openstack.org/#/c/598388/
> > > >
> > > > It is due to my local version is not the latest one.
> > > > Sorry for the trouble caused, and thanks for your test.
> > > >
> > > > Best Regards
> > > > Shuicheng
> > > >
> > > > From: Lin, Shuicheng [mailto:shuicheng.lin at intel.com]
> > > > Sent: Saturday, September 8, 2018 9:23 AM
> > > > To: McKenna, Jason <Jason.McKenna at windriver.com>; Khalil, Ghada
> > > <Ghad
> > > > a.Khalil at windriver.com>; 'starlingx-discuss at lists.starlingx.io'
> > > > <star lingx-discuss at lists.starlingx.io>; Xie, Cindy
> > > > <cindy.xie at intel.com>; Waheed, Numan
> > <Numan.Waheed at windriver.com>
> > > > Subject: Re: [Starlingx-discuss] CentOS 7.5 upgrade status
> > > > update
> > > >
> > > > Hi Jason,
> > > > Thanks very much for such detail feedback.
> > > > I added some comments inline, and will check the issue in my
> > > > environment also.
> > > > Feel free to submit patch if you have to fix them. J
> > > >
> > > >
> > > > Best Regards
> > > > Shuicheng
> > > >
> > > > From: McKenna, Jason [mailto:Jason.McKenna at windriver.com]
> > > > Sent: Saturday, September 8, 2018 3:47 AM
> > > > To: McKenna, Jason <Jason.McKenna at windriver.com>; Khalil, Ghada
> > > <Ghad
> > > > a.Khalil at windriver.com>; Lin, Shuicheng
> > > > <shuicheng.lin at intel.com>; 's tarlingx-discuss at lists.starlingx.io'
> > > > <starlingx-discuss at lists.starlin gx.io>; Xie, Cindy
> > > > <cindy.xie at intel.com>; Waheed, Numan
> > > <Numan.Waheed
> > > > @windriver.com>
> > > > Subject: RE: CentOS 7.5 upgrade status update
> > > >
> > > > Hi Shuicheng,
> > > >
> > > > I’ve completed a build of the code. We have not yet run
> > > > run-time tests, but here are the build-time issues I encountered:
> > > >
> > > > - The .lst file contains lighttpd-1.4.49, but this is no
> > > > longer available for download, and the download_mirrors.sh step fails.
> > > > It appears than CentOS has updated to lighttpd-1.4.50. I
> > > > manually built a lighttpd-1.4.49-1.src.rpm to satisfy the .lst
> > > > - Even though the .lst file specifies lighttpd-1.4.49, the
> > > > code still references 1.4.45
> > > > - At build-pkgs time (the build-srpms step), the meta-patches
> > > > to openstack-aodh do not apply, as two of them are malformed. https:
> > > > //review.openstack.org/#/c/598388/6/openstack/openstack-
> > > > aodh/centos/meta_patches/0001-Modify-service-files-and-create-
> > > > expirer-cron-script.patch -- In the new version, on line 190,
> > > > the patch states that the old code covered 6 lines, and the new
> > > > code covers 9 lines (-283,6 +286,9). But there are only two
> > > > lines inserted by the patch. There is also a new blank line in
> > > > the patch at line 194. Removing this blank line and editing the
> > > > hunk description to only insert 2 lines resolves the issue for
> > > > this hunk (there are other similar issues in this patch, and in
> > > > 0001-meta-
> > > > modify-aodh-api.patch)
> > > > [slin14]: Let me double check it.
> > > > - At binary build time, I had build failures on python-django-
> > > > horizon and some magnum packages. I had to pull in
> > > > https://review.op
> > > > enstack.org/#/c/596855/2 (from the master branch, but not in the
> > > > centos75 branch of stx-upstream) to get past this.
> > > > [slin14]: It is due to I have not synced latest master code, so
> > > > I don’t have this issue. I used last week’s master code.
> > > > - After building all packages, I attempted to build the ISO.
> > > > The tpm2-tss-1.3.0-2.el7.x86_64.rpm package were missing, and
> > > > should be added to the .lst files.
> > > > [slin14]: This package is already in the lst.
> > > > - Furthermore, the tpm2-tools package was requesting version
> > > > tpm2-tss-1.3.0-2.el7.centos (rather than tpm2-tss-1.3.0-2.el7).
> > > > I hacked the spec file for tpm2-tools and rebuilt it to require
> > > > the version mentioned, but there is some rework to be done there.
> > > > [slin14]: I have the same issue also. Let’s further check it to
> > > > find the root cause. I use rebuild tpm2-tools/tss2 to workaround
> > > > it without any code.
> > > >
> > > > After these changes, I was able to produce a .iso
> > > >
> > > > -Jason
> > > >
> > > > From: McKenna, Jason <Jason.McKenna at windriver.com>
> > > > Sent: September 7, 2018 10:08 AM
> > > > To: Khalil, Ghada <Ghada.Khalil at windriver.com>; Lin, Shuicheng
> > > > <shuic heng.lin at intel.com>; 'starlingx-discuss at lists.starlingx.io'
> > > > <starling x-discuss at lists.starlingx.io>; Xie, Cindy
> > > > <cindy.xie at intel.com>; Waheed, Numan
> > <Numan.Waheed at windriver.com>
> > > > Subject: Re: [Starlingx-discuss] CentOS 7.5 upgrade status
> > > > update
> > > >
> > > > Hi Shuicheng,
> > > >
> > > > As a download_mirrors.sh time issue, the version of lighttpd has
> > > > been changed from 1.4.45 to 1.4.49. The
> > > > lighttpd-1.4.49-1.el7.src.rpm file no longer exists on EPEL or
> > > > CentOS servers, as they’ve moved to 1.4.50. I am attempting to
> > > > locate an archived version of the 1.4.49 .src rpm.
> > > >
> > > > -Jason
> > > >
> > > > From: Khalil, Ghada <Ghada.Khalil at windriver.com>
> > > > Sent: September 7, 2018 9:36 AM
> > > > To: Lin, Shuicheng <shuicheng.lin at intel.com>;
> > > > 'starlingx-discuss at list s.starlingx.io'
> > > > <starlingx-discuss at lists.starlingx.io>; Xie, Cindy <c
> > > > indy.xie at intel.com>; Waheed, Numan
> > <Numan.Waheed at windriver.com>
> > > > Subject: Re: [Starlingx-discuss] CentOS 7.5 upgrade status
> > > > update
> > > >
> > > > Thanks Shuicheng. We’ll run some tests in Wind River. Numan will
> > > > be priming this activity. He’ll share the results with the team
> > > > sometime next week.
> > > >
> > > > Regards,
> > > > Ghada
> > > >
> > > > From: Lin, Shuicheng [mailto:shuicheng.lin at intel.com]
> > > > Sent: Friday, September 07, 2018 1:32 AM
> > > > To: 'starlingx-discuss at lists.starlingx.io'; Khalil, Ghada; Xie,
> > > > Cindy
> > > > Subject: CentOS 7.5 upgrade status update
> > > >
> > > > Hi all,
> > > > Most task of CentOS7.5 upgrade in story 2003389 is done. Please
> > > > go ahead to have a try and verify it.
> > > > I expect there should be some issue due to we just did limited
> > > > deploy test yet.
> > > >
> > > > Here is the detail status update of this week.
> > > > For the total 44 srpm we plan to upgrade:
> > > > 38 already merged in feature branch.
> > > > 3 kept as original version. Anaconda upgrade is not needed.
> > > > python- os-vif has higher version than 7.5. python-eventlet kept
> > > > due to new src rpm itself have known issue.
> > > > 2 under review with code ready. Puppet-haproxy and openstack-aodh.
> > > > 1 under debug. Lighttpd.
> > > >
> > > > Besides srpm, we also upgraded 260+ rpm packages to 7.5 version.
> > > >
> > > > Here is the build instruction:
> > > > 1. Switch to f/centos75 branch for stx-tools/stx-integ/stx-
> > > > upstream/stx-root.
> > > > Cherry-pick below un-merged patch:
> > > > For puppet-haproxy:
> > > > https://review.openstack.org/600369
> > > > https://review.openstack.org/600359
> > > > For openstack-aodh:
> > > > https://review.openstack.org/598388
> > > > https://review.openstack.org/600591
> > > > For build-iso:
> > > > https://review.openstack.org/599892
> > > > For build-pkgs:
> > > > https://review.openstack.org/599901
> > > > https://review.openstack.org/600639
> > > >
> > > > 2. Run mirror script to get the upgraded src rpm and related
> > > > rpm.
> > > > 3. Try to build packages and ISO.
> > > >
> > > > For the detail status of CentOS7.5 upgrade, please check below story:
> > > > https://storyboard.openstack.org/#!/story/2003389
> > > >
> > > > Feel free to contact me if you have trouble on it.
> > > >
> > > > Best Regards
> > > > Shuicheng
> > > > _______________________________________________
> > > > Starlingx-discuss mailing list
> > > > Starlingx-discuss at lists.starlingx.io
> > > > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-dis
> > > > cu
> > > > ss
More information about the Starlingx-discuss
mailing list