[Starlingx-discuss] CentOS7.6 testing status - blocked

Fri Feb 15 02:08:41 UTC 2019

Hi Scott,
What do you mean “installer rebuild”?
Grub2 package is not included in “update-pxe-network-installer”. Could you share me more details?
Thanks.

Best Regards
Shuicheng

From: Scott Little [mailto:scott.little at windriver.com]
Sent: Thursday, February 14, 2019 10:56 PM
To: Liu, Yang <yang.liu at windriver.com>; Lin, Shuicheng <shuicheng.lin at intel.com>; Penney, Don <Don.Penney at windriver.com>; Xie, Cindy <cindy.xie at intel.com>; Chen, Haochuan Z <haochuan.z.chen at intel.com>; Sun, Austin <austin.sun at intel.com>
Cc: starlingx-discuss at lists.starlingx.io; Cabrales, Ada <ada.cabrales at intel.com>; Waheed, Numan <Numan.Waheed at windriver.com>
Subject: Re: CentOS7.6 testing status - blocked

I think it's on our side.

The in house jenkins script I cloned for the 76 build does not include an installer rebuild.

Scott

On 2019-02-14 8:36 a.m., Liu, Yang wrote:
Hi Shuicheng,

I checked the 3 items as per your instructions, build log does contain the expected step, however the results are different.
@ Scott/Don, any thoughts on this?

1.      + install -D -m 755 grubx64.efi /builddir/build/BUILDROOT/grub2-2.02-0.76.el7.centos.tis.12.x86_64/pxeboot/EFI/grubx64.efi

2.      -rwx------ 1 yliu12 users 1233016 Feb 12 16:37 ./pxeboot/EFI/grubx64.efi

3.      -rwx------ 1 jenkins mock 1233016 Feb 12 16:37 grubx64.efi

BR,
Yang

From: Lin, Shuicheng [mailto:shuicheng.lin at intel.com]
Sent: February-14-19 2:32 AM
To: Penney, Don; Xie, Cindy; Liu, Yang; Chen, Haochuan Z; Sun, Austin
Cc: starlingx-discuss at lists.starlingx.io<mailto:starlingx-discuss at lists.starlingx.io>; Cabrales, Ada; Waheed, Numan; Little, Scott
Subject: RE: CentOS7.6 testing status - blocked

Hi Yang/Don,
We double checked the issue today. Here is our finding:

1.      I try to revert the fix [0], then do build-pkgs and build-iso, the “grubx64.efi” in “export/dist/isolinux/pxeboot/EFI/” is with 700 permission mode.

Add the fix [0] back, then build-pkgs and build-iso, the “grubx64.efi” is changed to 755 permission mode.

I also checked the grubx64.efi file in both ISO image, it has the same mode as upper file.

2.      Martin confirmed there is tftp log in the deployment:

“
2019-02-11T00:36:52.000 controller-0 dnsmasq-tftp[8262]: info sent /pxeboot/EFI/grubx64.efi to 169.254.202.76
controller-0:/var/log$ ls /pxeboot/EFI/grubx64.efi  -l
-rwxr-xr-x. 1 root root 1234192 Feb  3 06:52 /pxeboot/EFI/grubx64.efi

“

3.      Austin confirmed “install -D -m 755” will set the grubx64.efi with 755 permission mode.

“

       -m, --mode=MODE

              set permission mode (as in chmod), instead of rwxr-xr-x

“

4.      I try to go through the build log. Here is the log from grub2’s build.log

“

+ install -m 700 grubx64.efi /builddir/build/BUILDROOT/grub2-2.02-0.76.el7.centos.tis.12.x86_64/boot/efi/EFI/centos/grubx64.efi.unsigned

+ install -m 700 gcdx64.efi /builddir/build/BUILDROOT/grub2-2.02-0.76.el7.centos.tis.12.x86_64/boot/efi/EFI/centos/gcdx64.efi.unsigned

+ install -D -m 755 grubx64.efi /builddir/build/BUILDROOT/grub2-2.02-0.76.el7.centos.tis.12.x86_64/pxeboot/EFI/grubx64.efi

+ install -m 700 grubx64.efi /builddir/build/BUILDROOT/grub2-2.02-0.76.el7.centos.tis.12.x86_64/boot/efi/EFI/centos/grubx64.efi

+ install -m 700 gcdx64.efi /builddir/build/BUILDROOT/grub2-2.02-0.76.el7.centos.tis.12.x86_64/boot/efi/EFI/centos/gcdx64.efi

“

And in build-iso script, the file will be extracted and copied to the EFI folder:

“

extract_pkg_from_local_repo ${MY_YUM_CONF} ${STD_REPO_ID} grub2-efi-x64-pxeboot

…

\cp --preserve=all pxeboot/EFI/grubx64.efi $OUTPUT_DIST_DIR/isolinux/pxeboot/EFI/

“
Due to we cannot reproduce the issue, we are not sure which step cause the issue yet.
So could you help me have a check with below step to narrow down the issue? Thanks.

1.      Please help check whether there is “install -D -m 755 grubx64.efi” in the “loadbuild/std/results/slin14-starlingx-tis-r5-pike-std/grub2-2.02-0.76.el7.centos.tis.12/build.log” or not.

2.      Please help extract “grub2-efi-x64-pxeboot-2.02-0.76.el7.centos.tis.12.x86_64.rpm” in “loadbuild/std/rpmbuild/RPMS”, and check whether the grubx64.efi file is with 755 mode or not.

Extract cmd: rpm2cpio grub2-efi-x64-pxeboot-2.02-0.76.el7.centos.tis.12.x86_64.rpm | cpio -idmv

3.      Please help check the “grubx64.efi” in “export/dist/isolinux/pxeboot/EFI/” folder is with 755 mode or not.

[0]: https://review.openstack.org/634559

Best Regards
Shuicheng

From: Penney, Don [mailto:Don.Penney at windriver.com]
Sent: Thursday, February 14, 2019 6:01 AM
To: Xie, Cindy <cindy.xie at intel.com><mailto:cindy.xie at intel.com>; Liu, Yang <yang.liu at windriver.com><mailto:yang.liu at windriver.com>
Cc: starlingx-discuss at lists.starlingx.io<mailto:starlingx-discuss at lists.starlingx.io>; Cabrales, Ada <ada.cabrales at intel.com><mailto:ada.cabrales at intel.com>; Waheed, Numan <Numan.Waheed at windriver.com><mailto:Numan.Waheed at windriver.com>; Little, Scott <Scott.Little at windriver.com><mailto:Scott.Little at windriver.com>
Subject: Re: [Starlingx-discuss] CentOS7.6 testing status - blocked

Hi Cindy,

In a successful case, you should see a TFTP log in daemon.log on the active controller indicating the file was transferred, such as:
2019-02-12T13:20:45.000 controller-0 dnsmasq-tftp[200877]: info sent /pxeboot/EFI/grubx64.efi to 192.168.204.4

I would suggest doing something like “tail -f /var/log/daemon.log | grep -i tftp” while doing the installation of nodes from the active controller, to verify the expected file is getting transferred. If the host installs and you don’t see this file transferred, I’d recommend reconfirming that the node is installing via UEFI.

From: Xie, Cindy [mailto:cindy.xie at intel.com]
Sent: Wednesday, February 13, 2019 4:46 PM
To: Liu, Yang
Cc: starlingx-discuss at lists.starlingx.io<mailto:starlingx-discuss at lists.starlingx.io>; Cabrales, Ada; Waheed, Numan; Little, Scott
Subject: Re: [Starlingx-discuss] CentOS7.6 testing status - blocked

Hi, Yang
Sorry about the issue! It’s interesting as I did have my engineer tested the scenarios. There must be something missing from my side.

We will redo the patch and test. In the same time, can you manually change the file permissions as temporarily workaround and unblock the test cycle?

Thanks! Cindy
Sent from my iPhone

On Feb 14, 2019, at 3:02 AM, Liu, Yang <yang.liu at windriver.com<mailto:yang.liu at windriver.com>> wrote:
Hi Cindy,

We are still seeing the same file permission issue for grubx64.efi under pxeboot/EFI, causing UEFI pxeboot to fail.
We need the grubx64.efi to be readable by others as well.

../pxeboot/EFI/
total 1220
drwxrwsr-x 3 jenkins mock    4096 Feb 12 16:48 .
drwxrwsr-x 3 jenkins mock    4096 Feb 12 16:48 ..
drwxrwsr-x 3 jenkins mock    4096 Feb 12 16:48 centos
-rwx------ 1 jenkins mock 1233016 Feb 12 16:37 grubx64.efi

The patch seems to have changed the dir permission for centos from 700 to 755, but not grubx64.efi.
For the dir permission for centos, I believe the original 700 should be sufficient (@Scott, please correct me if it’s wrong).

BR,
Yang

From: Liu, Yang
Sent: February-12-19 9:00 AM
To: 'Xie, Cindy'; starlingx-discuss at lists.starlingx.io<mailto:starlingx-discuss at lists.starlingx.io>; Cabrales, Ada; Waheed, Numan
Subject: RE: CentOS7.6 testing status - blocked

Thanks Cindy. Will do.

BR,
Yang

From: Xie, Cindy [mailto:cindy.xie at intel.com]
Sent: February-11-19 8:10 PM
To: Liu, Yang; starlingx-discuss at lists.starlingx.io<mailto:starlingx-discuss at lists.starlingx.io>; Cabrales, Ada; Waheed, Numan
Subject: RE: CentOS7.6 testing status - blocked

Hi, Numan/Yang,
The last pending patch (https://review.openstack.org/#/c/634559/) which was blocking your testing (#1814360) was just merged. Please get new build ISO from Jason so you can continue the testing.

Thx. - cindy

From: Liu, Yang [mailto:yang.liu at windriver.com]
Sent: Saturday, February 9, 2019 10:18 AM
To: Xie, Cindy <cindy.xie at intel.com<mailto:cindy.xie at intel.com>>; starlingx-discuss at lists.starlingx.io<mailto:starlingx-discuss at lists.starlingx.io>
Subject: RE: CentOS7.6 testing status - blocked

Correct.

BR,
Yang

From: Xie, Cindy [mailto:cindy.xie at intel.com]
Sent: February-08-19 8:28 PM
To: Liu, Yang; starlingx-discuss at lists.starlingx.io<mailto:starlingx-discuss at lists.starlingx.io>
Subject: RE: CentOS7.6 testing status - blocked

Hi, Yang,
Thanks for the report.
Are the “two node system” below referring to Duplex? Just want to confirm because #1814360 we have a patch pending and we do want to ensure it works on Duplex as well.

Th.x - cindy

From: Liu, Yang [mailto:yang.liu at windriver.com]
Sent: Saturday, February 9, 2019 2:09 AM
To: starlingx-discuss at lists.starlingx.io<mailto:starlingx-discuss at lists.starlingx.io>
Subject: [Starlingx-discuss] CentOS7.6 testing status - blocked

Hi folks,

Here’s an update for CentOS7.6 testing.
We are currently blocked due to pxeboot from controller-0 does not work for EFI. (#1814360)
We will continue after that issue is resolved.

System

NICs
Mgmt;infra;data

Special Configs

Test coverage after Install and Config

Status/Issues

Dedicated storage

X540-AT2; X540-AT2; fortville

IPv6

Sanity, nova

Completed. New issues logged.
#1814336 CentOS7.6: Unable to launch vm directly from virsh<https://bugs.launchpad.net/starlingx/+bug/1814336>
#1814335 CentOS7.6: Unable to launch vm with UEFI boot<https://bugs.launchpad.net/starlingx/+bug/1814335>

One node system

none; none; X522/X577-AT

Sanity, basic regression

Completed. Passed.

Two node system

fortville; fortville; fortville

tboot, tpm, https,
extended security profile

Sanity, security

Blocked by #1814360

Multi-node system

BCM5720; Niantic; Niantic

Sriov(niantic),pcipt(niantic)

Sanity, networking

Completed. Passed.

Two node system

Fortville; none; Fortville

Low latency, UEFI

Sanity, basic regression, cyclictest

Blocked by #1814360

Two node system

Fortville; none; Fortville

Secure boot

Sanity, security

Blocked by #1814360

Multi-node system

I350; Niantic/cx3; cx3

Pxeboot script

Sanity

Completed. Passed.
Only compute-0 was used, since compute-1 has CX3 data nic.

??

CX4 on infra or mgmt, but NOT data

Won’t test. We don’t have a system have required nics.

BR,
yang

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20190215/2a2fe975/attachment-0001.html>