Thanks Don – that takes care of #2 so we won’t see this issue again once Shuicheng updates the kata version and completes #1. Frank From: Penney, Don Sent: Tuesday, January 14, 2020 3:24 PM To: Miller, Frank <Frank.Miller@windriver.com>; Little, Scott <Scott.Little@windriver.com>; starlingx-discuss@lists.starlingx.io Subject: RE: [Starlingx-discuss] Kata containers update breaking build - failure to download RPMs It may not actually cause a build failure, now that I think about it… the download-mirror will see that the file exists and skip it. So it’s only because the file no longer existed the first time it tried to download that we had a failure. From: Miller, Frank Sent: Tuesday, January 14, 2020 3:22 PM To: Penney, Don; Little, Scott; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: RE: [Starlingx-discuss] Kata containers update breaking build - failure to download RPMs I think the best 2 actions would be: 1) Revert the kata commit(s) and Shuicheng to rebase on the new kata version, run his own testing to ensure the new version did not break anything and then push the kata commits along with the new version of kata. 2) Identify how we can prevent a build failure in the future when kata puts out a new version and deletes the 1.9.3-8.1 version. Is there a way the StarlingX build or CENGN mirror can cache this version so we can control when we move to a new version? Frank From: Penney, Don Sent: Tuesday, January 14, 2020 3:09 PM To: Miller, Frank <Frank.Miller@windriver.com<mailto:Frank.Miller@windriver.com>>; Little, Scott <Scott.Little@windriver.com<mailto:Scott.Little@windriver.com>>; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: RE: [Starlingx-discuss] Kata containers update breaking build - failure to download RPMs Option B here would be to revert the kata updates so the build can proceed, and ask for it to be rebased and retested: https://review.opendev.org/#/q/status:merged+projects:starlingx+branch:maste... From: Penney, Don Sent: Tuesday, January 14, 2020 3:05 PM To: Miller, Frank; Little, Scott; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: RE: [Starlingx-discuss] Kata containers update breaking build - failure to download RPMs Unfortunately, the upstream repo where we get the RPMs deletes the older versions when they release a new one, it seems. So our build will break any time they do so. From: Miller, Frank Sent: Tuesday, January 14, 2020 3:03 PM To: Penney, Don; Little, Scott; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: RE: [Starlingx-discuss] Kata containers update breaking build - failure to download RPMs Thank-you Don. Did Shuicheng not use a specific version of kata containers in his commits? Will your commit ensure that we are locked down on the 1.9.3-8.1 version? We need to make sure the next time kata releases a new version that we do not automatically pick up the new version. Going to a new version should be done in a controlled way where the feature owner does testing with the new version in his own environment before posting a commit to upversion to the newer version. Frank From: Penney, Don [mailto:Don.Penney@windriver.com] Sent: Tuesday, January 14, 2020 2:39 PM To: Little, Scott <Scott.Little@windriver.com<mailto:Scott.Little@windriver.com>>; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] Kata containers update breaking build - failure to download RPMs A review is posted to update the versions of the kata RPMs: https://review.opendev.org/702506 From: Scott Little [mailto:scott.little@windriver.com] Sent: Tuesday, January 14, 2020 2:23 PM To: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] Kata containers update breaking build - failure to download RPMs 1) The rpms listed in the lst files are way out of date relative to what's available from the new kata repo. [ ] kata-containers-image-1.9.3-8.1.x86_64.rpm 10-Dec-2019 09:11 36M Details [ ] kata-ksm-throttler-1.9.3-8.1.x86_64.rpm 10-Dec-2019 09:33 5.9M Details [ ] kata-linux-container-4.19.75.54-8.1.x86_64.rpm 10-Dec-2019 09:22 8.3M Details [ ] kata-linux-container-debug-4.19.75.54-8.1.x86_64.rpm 10-Dec-2019 09:22 335K Details [ ] kata-proxy-1.9.3-8.1.x86_64.rpm 10-Dec-2019 09:11 2.4K Details [ ] kata-proxy-bin-1.9.3-8.1.x86_64.rpm 10-Dec-2019 09:11 1.8M Details [ ] kata-runtime-1.9.3-8.1.x86_64.rpm 10-Dec-2019 09:39 20M Details [ ] kata-shim-1.9.3-8.1.x86_64.rpm 10-Dec-2019 09:20 2.4K Details [ ] kata-shim-bin-1.9.3-8.1.x86_64.rpm 10-Dec-2019 09:20 7.3M Details [ ] qemu-lite-2.11.0+git.87517afd72-8.1.x86_64.rpm 10-Dec-2019 09:29 2.5K Details [ ] qemu-lite-bin-2.11.0+git.87517afd72-8.1.x86_64.rpm 10-Dec-2019 09:29 2.6M Details [ ] qemu-lite-data-2.11.0+git.87517afd72-8.1.x86_64.rpm 10-Dec-2019 09:29 3.0M Details [ ] qemu-vanilla-4.1.0+git.9e06029aea-8.1.x86_64.rpm 10-Dec-2019 09:44 2.6K Details [ ] qemu-vanilla-bin-4.1.0+git.9e06029aea-8.1.x86_64.rpm 10-Dec-2019 09:44 2.5M Details [ ] qemu-vanilla-data-4.1.0+git.9e06029aea-8.1.x86_64.rpm 10-Dec-2019 09:44 13M Details 2) The kata repo appears to be for SUSE, not CentOS. 3) The new repos url <http://download.opensuse.org/repositories/home:/katacontainers:/releases:/x86_64:/stable-1.9/CentOS_7/> points to a location with a lot of extra 'stuff' when viewed from a browser. Not sure if yum is being served a cleaner view, but there is certainly the potential of confusing tools. IS there an alternate UR that would be cleaner? I recommend that the update (stx-tools:431885231ae41256188a7c32f0f5351c4455707b) be reverted. There is likely other updates dependent on this that also need to be reverted. Scott On 2020-01-14 12:47 p.m., Penney, Don wrote: The latest CENGN build attempt fails on the RPM download stage, with the following errors: Looking for kata-containers-image-1.9.2-7.1.x86_64.rpm --> run: sudo -E yumdownloader -q -C -c /tmp/stx_mirror_vHj2Gp/yum.conf --releasever=7 --archlist=noarch,x86_64 kata-containers-image-1.9.2-7.1 No Match for argument kata-containers-image-1.9.2-7.1 Nothing to download Warning: kata-containers-image-1.9.2-7.1.x86_64.rpm not found Looking for kata-containers-image-1.9.2-7.1.x86_64.rpm --> run: sudo -E yumdownloader -q -C -c /tmp/stx_mirror_vHj2Gp/yum.conf --releasever=7 --archlist=noarch,x86_64 kata-containers-image-1.9.2-7.1 No Match for argument kata-containers-image-1.9.2-7.1 Nothing to download Warning: kata-containers-image-1.9.2-7.1.x86_64.rpm not found Error: kata-containers-image-1.9.2-7.1.x86_64.rpm not found Looking for kata-ksm-throttler-1.9.2-7.1.x86_64.rpm --> run: sudo -E yumdownloader -q -C -c /tmp/stx_mirror_vHj2Gp/yum.conf --releasever=7 --archlist=noarch,x86_64 kata-ksm-throttler-1.9.2-7.1 No Match for argument kata-ksm-throttler-1.9.2-7.1 Nothing to download Warning: kata-ksm-throttler-1.9.2-7.1.x86_64.rpm not found Looking for kata-ksm-throttler-1.9.2-7.1.x86_64.rpm --> run: sudo -E yumdownloader -q -C -c /tmp/stx_mirror_vHj2Gp/yum.conf --releasever=7 --archlist=noarch,x86_64 kata-ksm-throttler-1.9.2-7.1 No Match for argument kata-ksm-throttler-1.9.2-7.1 Nothing to download Warning: kata-ksm-throttler-1.9.2-7.1.x86_64.rpm not found Error: kata-ksm-throttler-1.9.2-7.1.x86_64.rpm not found Looking for kata-linux-container-4.19.75.54-7.1.x86_64.rpm --> run: sudo -E yumdownloader -q -C -c /tmp/stx_mirror_vHj2Gp/yum.conf --releasever=7 --archlist=noarch,x86_64 kata-linux-container-4.19.75.54-7.1 No Match for argument kata-linux-container-4.19.75.54-7.1 Nothing to download Warning: kata-linux-container-4.19.75.54-7.1.x86_64.rpm not found Looking for kata-linux-container-4.19.75.54-7.1.x86_64.rpm --> run: sudo -E yumdownloader -q -C -c /tmp/stx_mirror_vHj2Gp/yum.conf --releasever=7 --archlist=noarch,x86_64 kata-linux-container-4.19.75.54-7.1 No Match for argument kata-linux-container-4.19.75.54-7.1 Nothing to download Warning: kata-linux-container-4.19.75.54-7.1.x86_64.rpm not found Error: kata-linux-container-4.19.75.54-7.1.x86_64.rpm not found Looking for kata-linux-container-debug-4.19.75.54-7.1.x86_64.rpm --> run: sudo -E yumdownloader -q -C -c /tmp/stx_mirror_vHj2Gp/yum.conf --releasever=7 --archlist=noarch,x86_64 kata-linux-container-debug-4.19.75.54-7.1 No Match for argument kata-linux-container-debug-4.19.75.54-7.1 Nothing to download Warning: kata-linux-container-debug-4.19.75.54-7.1.x86_64.rpm not found Looking for kata-linux-container-debug-4.19.75.54-7.1.x86_64.rpm --> run: sudo -E yumdownloader -q -C -c /tmp/stx_mirror_vHj2Gp/yum.conf --releasever=7 --archlist=noarch,x86_64 kata-linux-container-debug-4.19.75.54-7.1 No Match for argument kata-linux-container-debug-4.19.75.54-7.1 Nothing to download Warning: kata-linux-container-debug-4.19.75.54-7.1.x86_64.rpm not found Error: kata-linux-container-debug-4.19.75.54-7.1.x86_64.rpm not found Looking for kata-proxy-1.9.2-7.1.x86_64.rpm --> run: sudo -E yumdownloader -q -C -c /tmp/stx_mirror_vHj2Gp/yum.conf --releasever=7 --archlist=noarch,x86_64 kata-proxy-1.9.2-7.1 No Match for argument kata-proxy-1.9.2-7.1 Nothing to download Warning: kata-proxy-1.9.2-7.1.x86_64.rpm not found Looking for kata-proxy-1.9.2-7.1.x86_64.rpm --> run: sudo -E yumdownloader -q -C -c /tmp/stx_mirror_vHj2Gp/yum.conf --releasever=7 --archlist=noarch,x86_64 kata-proxy-1.9.2-7.1 No Match for argument kata-proxy-1.9.2-7.1 Nothing to download Warning: kata-proxy-1.9.2-7.1.x86_64.rpm not found Error: kata-proxy-1.9.2-7.1.x86_64.rpm not found Looking for kata-proxy-bin-1.9.2-7.1.x86_64.rpm --> run: sudo -E yumdownloader -q -C -c /tmp/stx_mirror_vHj2Gp/yum.conf --releasever=7 --archlist=noarch,x86_64 kata-proxy-bin-1.9.2-7.1 No Match for argument kata-proxy-bin-1.9.2-7.1 Nothing to download Warning: kata-proxy-bin-1.9.2-7.1.x86_64.rpm not found Looking for kata-proxy-bin-1.9.2-7.1.x86_64.rpm --> run: sudo -E yumdownloader -q -C -c /tmp/stx_mirror_vHj2Gp/yum.conf --releasever=7 --archlist=noarch,x86_64 kata-proxy-bin-1.9.2-7.1 No Match for argument kata-proxy-bin-1.9.2-7.1 Nothing to download Warning: kata-proxy-bin-1.9.2-7.1.x86_64.rpm not found Error: kata-proxy-bin-1.9.2-7.1.x86_64.rpm not found Looking for kata-runtime-1.9.2-7.1.x86_64.rpm --> run: sudo -E yumdownloader -q -C -c /tmp/stx_mirror_vHj2Gp/yum.conf --releasever=7 --archlist=noarch,x86_64 kata-runtime-1.9.2-7.1 No Match for argument kata-runtime-1.9.2-7.1 Nothing to download Warning: kata-runtime-1.9.2-7.1.x86_64.rpm not found Looking for kata-runtime-1.9.2-7.1.x86_64.rpm --> run: sudo -E yumdownloader -q -C -c /tmp/stx_mirror_vHj2Gp/yum.conf --releasever=7 --archlist=noarch,x86_64 kata-runtime-1.9.2-7.1 No Match for argument kata-runtime-1.9.2-7.1 Nothing to download Warning: kata-runtime-1.9.2-7.1.x86_64.rpm not found Error: kata-runtime-1.9.2-7.1.x86_64.rpm not found Looking for kata-shim-1.9.2-7.1.x86_64.rpm --> run: sudo -E yumdownloader -q -C -c /tmp/stx_mirror_vHj2Gp/yum.conf --releasever=7 --archlist=noarch,x86_64 kata-shim-1.9.2-7.1 No Match for argument kata-shim-1.9.2-7.1 Nothing to download Warning: kata-shim-1.9.2-7.1.x86_64.rpm not found Looking for kata-shim-1.9.2-7.1.x86_64.rpm --> run: sudo -E yumdownloader -q -C -c /tmp/stx_mirror_vHj2Gp/yum.conf --releasever=7 --archlist=noarch,x86_64 kata-shim-1.9.2-7.1 No Match for argument kata-shim-1.9.2-7.1 Nothing to download Warning: kata-shim-1.9.2-7.1.x86_64.rpm not found Error: kata-shim-1.9.2-7.1.x86_64.rpm not found Looking for kata-shim-bin-1.9.2-7.1.x86_64.rpm --> run: sudo -E yumdownloader -q -C -c /tmp/stx_mirror_vHj2Gp/yum.conf --releasever=7 --archlist=noarch,x86_64 kata-shim-bin-1.9.2-7.1 No Match for argument kata-shim-bin-1.9.2-7.1 Nothing to download Warning: kata-shim-bin-1.9.2-7.1.x86_64.rpm not found Looking for kata-shim-bin-1.9.2-7.1.x86_64.rpm --> run: sudo -E yumdownloader -q -C -c /tmp/stx_mirror_vHj2Gp/yum.conf --releasever=7 --archlist=noarch,x86_64 kata-shim-bin-1.9.2-7.1 No Match for argument kata-shim-bin-1.9.2-7.1 Nothing to download Warning: kata-shim-bin-1.9.2-7.1.x86_64.rpm not found Error: kata-shim-bin-1.9.2-7.1.x86_64.rpm not found -----Original Message----- From: build.starlingx@gmail.com<mailto:build.starlingx@gmail.com> [mailto:build.starlingx@gmail.com] Sent: Tuesday, January 14, 2020 12:40 PM To: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: [Starlingx-discuss] [build-report] STX_build_master_master - Build # 382 - Still Failing! Project: STX_build_master_master Build #: 382 Status: Still Failing Timestamp: 20200114T172515Z Check logs at: http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/20200114T172... -------------------------------------------------------------------------------- Parameters BUILD_CONTAINERS_DEV: false BUILD_CONTAINERS_STABLE: false _______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io<mailto:Starlingx-discuss@lists.starlingx.io> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss