Hi Scott, What do you mean “installer rebuild”? Grub2 package is not included in “update-pxe-network-installer”. Could you share me more details? Thanks. Best Regards Shuicheng From: Scott Little [mailto:scott.little@windriver.com] Sent: Thursday, February 14, 2019 10:56 PM To: Liu, Yang <yang.liu@windriver.com>; Lin, Shuicheng <shuicheng.lin@intel.com>; Penney, Don <Don.Penney@windriver.com>; Xie, Cindy <cindy.xie@intel.com>; Chen, Haochuan Z <haochuan.z.chen@intel.com>; Sun, Austin <austin.sun@intel.com> Cc: starlingx-discuss@lists.starlingx.io; Cabrales, Ada <ada.cabrales@intel.com>; Waheed, Numan <Numan.Waheed@windriver.com> Subject: Re: CentOS7.6 testing status - blocked I think it's on our side. The in house jenkins script I cloned for the 76 build does not include an installer rebuild. Scott On 2019-02-14 8:36 a.m., Liu, Yang wrote: Hi Shuicheng, I checked the 3 items as per your instructions, build log does contain the expected step, however the results are different. @ Scott/Don, any thoughts on this? 1. + install -D -m 755 grubx64.efi /builddir/build/BUILDROOT/grub2-2.02-0.76.el7.centos.tis.12.x86_64/pxeboot/EFI/grubx64.efi 2. -rwx------ 1 yliu12 users 1233016 Feb 12 16:37 ./pxeboot/EFI/grubx64.efi 3. -rwx------ 1 jenkins mock 1233016 Feb 12 16:37 grubx64.efi BR, Yang From: Lin, Shuicheng [mailto:shuicheng.lin@intel.com] Sent: February-14-19 2:32 AM To: Penney, Don; Xie, Cindy; Liu, Yang; Chen, Haochuan Z; Sun, Austin Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>; Cabrales, Ada; Waheed, Numan; Little, Scott Subject: RE: CentOS7.6 testing status - blocked Hi Yang/Don, We double checked the issue today. Here is our finding: 1. I try to revert the fix [0], then do build-pkgs and build-iso, the “grubx64.efi” in “export/dist/isolinux/pxeboot/EFI/” is with 700 permission mode. Add the fix [0] back, then build-pkgs and build-iso, the “grubx64.efi” is changed to 755 permission mode. I also checked the grubx64.efi file in both ISO image, it has the same mode as upper file. 2. Martin confirmed there is tftp log in the deployment: “ 2019-02-11T00:36:52.000 controller-0 dnsmasq-tftp[8262]: info sent /pxeboot/EFI/grubx64.efi to 169.254.202.76 controller-0:/var/log$ ls /pxeboot/EFI/grubx64.efi -l -rwxr-xr-x. 1 root root 1234192 Feb 3 06:52 /pxeboot/EFI/grubx64.efi “ 3. Austin confirmed “install -D -m 755” will set the grubx64.efi with 755 permission mode. “ -m, --mode=MODE set permission mode (as in chmod), instead of rwxr-xr-x “ 4. I try to go through the build log. Here is the log from grub2’s build.log “ + install -m 700 grubx64.efi /builddir/build/BUILDROOT/grub2-2.02-0.76.el7.centos.tis.12.x86_64/boot/efi/EFI/centos/grubx64.efi.unsigned + install -m 700 gcdx64.efi /builddir/build/BUILDROOT/grub2-2.02-0.76.el7.centos.tis.12.x86_64/boot/efi/EFI/centos/gcdx64.efi.unsigned + install -D -m 755 grubx64.efi /builddir/build/BUILDROOT/grub2-2.02-0.76.el7.centos.tis.12.x86_64/pxeboot/EFI/grubx64.efi + install -m 700 grubx64.efi /builddir/build/BUILDROOT/grub2-2.02-0.76.el7.centos.tis.12.x86_64/boot/efi/EFI/centos/grubx64.efi + install -m 700 gcdx64.efi /builddir/build/BUILDROOT/grub2-2.02-0.76.el7.centos.tis.12.x86_64/boot/efi/EFI/centos/gcdx64.efi “ And in build-iso script, the file will be extracted and copied to the EFI folder: “ extract_pkg_from_local_repo ${MY_YUM_CONF} ${STD_REPO_ID} grub2-efi-x64-pxeboot … \cp --preserve=all pxeboot/EFI/grubx64.efi $OUTPUT_DIST_DIR/isolinux/pxeboot/EFI/ “ Due to we cannot reproduce the issue, we are not sure which step cause the issue yet. So could you help me have a check with below step to narrow down the issue? Thanks. 1. Please help check whether there is “install -D -m 755 grubx64.efi” in the “loadbuild/std/results/slin14-starlingx-tis-r5-pike-std/grub2-2.02-0.76.el7.centos.tis.12/build.log” or not. 2. Please help extract “grub2-efi-x64-pxeboot-2.02-0.76.el7.centos.tis.12.x86_64.rpm” in “loadbuild/std/rpmbuild/RPMS”, and check whether the grubx64.efi file is with 755 mode or not. Extract cmd: rpm2cpio grub2-efi-x64-pxeboot-2.02-0.76.el7.centos.tis.12.x86_64.rpm | cpio -idmv 3. Please help check the “grubx64.efi” in “export/dist/isolinux/pxeboot/EFI/” folder is with 755 mode or not. [0]: https://review.openstack.org/634559 Best Regards Shuicheng From: Penney, Don [mailto:Don.Penney@windriver.com] Sent: Thursday, February 14, 2019 6:01 AM To: Xie, Cindy <cindy.xie@intel.com><mailto:cindy.xie@intel.com>; Liu, Yang <yang.liu@windriver.com><mailto:yang.liu@windriver.com> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>; Cabrales, Ada <ada.cabrales@intel.com><mailto:ada.cabrales@intel.com>; Waheed, Numan <Numan.Waheed@windriver.com><mailto:Numan.Waheed@windriver.com>; Little, Scott <Scott.Little@windriver.com><mailto:Scott.Little@windriver.com> Subject: Re: [Starlingx-discuss] CentOS7.6 testing status - blocked Hi Cindy, In a successful case, you should see a TFTP log in daemon.log on the active controller indicating the file was transferred, such as: 2019-02-12T13:20:45.000 controller-0 dnsmasq-tftp[200877]: info sent /pxeboot/EFI/grubx64.efi to 192.168.204.4 I would suggest doing something like “tail -f /var/log/daemon.log | grep -i tftp” while doing the installation of nodes from the active controller, to verify the expected file is getting transferred. If the host installs and you don’t see this file transferred, I’d recommend reconfirming that the node is installing via UEFI. From: Xie, Cindy [mailto:cindy.xie@intel.com] Sent: Wednesday, February 13, 2019 4:46 PM To: Liu, Yang Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>; Cabrales, Ada; Waheed, Numan; Little, Scott Subject: Re: [Starlingx-discuss] CentOS7.6 testing status - blocked Hi, Yang Sorry about the issue! It’s interesting as I did have my engineer tested the scenarios. There must be something missing from my side. We will redo the patch and test. In the same time, can you manually change the file permissions as temporarily workaround and unblock the test cycle? Thanks! Cindy Sent from my iPhone On Feb 14, 2019, at 3:02 AM, Liu, Yang <yang.liu@windriver.com<mailto:yang.liu@windriver.com>> wrote: Hi Cindy, We are still seeing the same file permission issue for grubx64.efi under pxeboot/EFI, causing UEFI pxeboot to fail. We need the grubx64.efi to be readable by others as well. ../pxeboot/EFI/ total 1220 drwxrwsr-x 3 jenkins mock 4096 Feb 12 16:48 . drwxrwsr-x 3 jenkins mock 4096 Feb 12 16:48 .. drwxrwsr-x 3 jenkins mock 4096 Feb 12 16:48 centos -rwx------ 1 jenkins mock 1233016 Feb 12 16:37 grubx64.efi The patch seems to have changed the dir permission for centos from 700 to 755, but not grubx64.efi. For the dir permission for centos, I believe the original 700 should be sufficient (@Scott, please correct me if it’s wrong). BR, Yang From: Liu, Yang Sent: February-12-19 9:00 AM To: 'Xie, Cindy'; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>; Cabrales, Ada; Waheed, Numan Subject: RE: CentOS7.6 testing status - blocked Thanks Cindy. Will do. BR, Yang From: Xie, Cindy [mailto:cindy.xie@intel.com] Sent: February-11-19 8:10 PM To: Liu, Yang; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>; Cabrales, Ada; Waheed, Numan Subject: RE: CentOS7.6 testing status - blocked Hi, Numan/Yang, The last pending patch (https://review.openstack.org/#/c/634559/) which was blocking your testing (#1814360) was just merged. Please get new build ISO from Jason so you can continue the testing. Thx. - cindy From: Liu, Yang [mailto:yang.liu@windriver.com] Sent: Saturday, February 9, 2019 10:18 AM To: Xie, Cindy <cindy.xie@intel.com<mailto:cindy.xie@intel.com>>; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: RE: CentOS7.6 testing status - blocked Correct. BR, Yang From: Xie, Cindy [mailto:cindy.xie@intel.com] Sent: February-08-19 8:28 PM To: Liu, Yang; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: RE: CentOS7.6 testing status - blocked Hi, Yang, Thanks for the report. Are the “two node system” below referring to Duplex? Just want to confirm because #1814360 we have a patch pending and we do want to ensure it works on Duplex as well. Th.x - cindy From: Liu, Yang [mailto:yang.liu@windriver.com] Sent: Saturday, February 9, 2019 2:09 AM To: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: [Starlingx-discuss] CentOS7.6 testing status - blocked Hi folks, Here’s an update for CentOS7.6 testing. We are currently blocked due to pxeboot from controller-0 does not work for EFI. (#1814360) We will continue after that issue is resolved. System NICs Mgmt;infra;data Special Configs Test coverage after Install and Config Status/Issues Dedicated storage X540-AT2; X540-AT2; fortville IPv6 Sanity, nova Completed. New issues logged. #1814336 CentOS7.6: Unable to launch vm directly from virsh<https://bugs.launchpad.net/starlingx/+bug/1814336> #1814335 CentOS7.6: Unable to launch vm with UEFI boot<https://bugs.launchpad.net/starlingx/+bug/1814335> One node system none; none; X522/X577-AT Sanity, basic regression Completed. Passed. Two node system fortville; fortville; fortville tboot, tpm, https, extended security profile Sanity, security Blocked by #1814360 Multi-node system BCM5720; Niantic; Niantic Sriov(niantic),pcipt(niantic) Sanity, networking Completed. Passed. Two node system Fortville; none; Fortville Low latency, UEFI Sanity, basic regression, cyclictest Blocked by #1814360 Two node system Fortville; none; Fortville Secure boot Sanity, security Blocked by #1814360 Multi-node system I350; Niantic/cx3; cx3 Pxeboot script Sanity Completed. Passed. Only compute-0 was used, since compute-1 has CX3 data nic. ?? CX4 on infra or mgmt, but NOT data Won’t test. We don’t have a system have required nics. BR, yang