Looks like centos decided to move 7.7.1908 repos from mirror.centos.org to vault.centos.org while I was on vacation.  This broke mirror updates when we tried to update from the old location.  Extra unhelpfull was that the error report was sent exclusively to me, and wasn't being forwarded to starlingx-discuss.

The repo urls have been corrected by Don's update https://review.opendev.org/#/c/729739

I've modified the mirroring job to forward an error report to starlingx-discuss

I've also modified the underlying mirroring scripts to continue when confronted with this class of failure, and only report the problem only after mirroring the remaining repos.

An additional fix was made by Don to another jenkins job that processes rpms_3rdparties.lst files from the layered config.  All the underlying scripting had been written and tested, but somehow not enabled within jenkins. 

Looking at the RPM inputs of tonights build, and comparing vs recent lst modifications, I can report that all rpms are available to the build, and the build completed successfully.

Scott





-------- Forwarded Message --------
Subject: Build failed in Jenkins: mirror_repo_sync #1435
Date: Wed, 20 May 2020 08:10:42 -0400
From: build.starlingx@gmail.com
Reply-To: build.starlingx@gmail.com
To: scott.little@windriver.com


See <http://127.0.0.1:8080/job/mirror_repo_sync/1435/display/redirect>

------------------------------------------
[...truncated 87.78 KB...]
UPSTREAM_REPO_ID=StarlingX-C7.6.1810-cloud-pike
UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
UPSTREAM_REPO_URL=http://vault.centos.org/centos/7.6.1810/cloud/x86_64/openstack-pike/
UPSTREAM_REPO_NAME=StarlingX-CentOS-7.6.1810 - Cloud-pike
Processing: REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo REPO_ID=StarlingX-C7.6.1810-cloud-pike REPO_URL=http://vault.centos.org/7.6.1810/centosplus/Source/ DOWNLOAD_PATH=/export/mirror/centos/centos/vault.centos.org/7.6.1810/centosplus/Source UPSTREAM_REPO_ID=StarlingX-C7.6.1810-cloud-pike UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
REPO_URL=http://vault.centos.org/centos/7.6.1810/cloud/x86_64/openstack-pike/
REPO_NAME=StarlingX-CentOS-7.6.1810 - Cloud-pike
Already have 'StarlingX-C7.6.1810-cloud-pike' from '/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo'
UPSTREAM_YUM_CONF=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.conf.sample
UPSTREAM_REPO_ID=StarlingX-C7.6.1810-cloud-pike-source
UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
UPSTREAM_REPO_URL=http://vault.centos.org/7.6.1810/cloud/Source/openstack-pike/
UPSTREAM_REPO_NAME=StarlingX-CentOS-7.6.1810 - cloud-pike-source
Processing: REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo REPO_ID=StarlingX-C7.6.1810-cloud-pike-source REPO_URL=http://vault.centos.org/centos/7.6.1810/cloud/x86_64/openstack-pike/ DOWNLOAD_PATH=/export/mirror/centos/centos/vault.centos.org/centos/7.6.1810/cloud/x86_64/openstack-pike UPSTREAM_REPO_ID=StarlingX-C7.6.1810-cloud-pike-source UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
REPO_URL=http://vault.centos.org/7.6.1810/cloud/Source/openstack-pike/
REPO_NAME=StarlingX-CentOS-7.6.1810 - cloud-pike-source
Already have 'StarlingX-C7.6.1810-cloud-pike-source' from '/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo'
UPSTREAM_YUM_CONF=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.conf.sample
UPSTREAM_REPO_ID=StarlingX-C7.6.1810-extras
UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
UPSTREAM_REPO_URL=http://vault.centos.org/centos/7.6.1810/extras/x86_64/
UPSTREAM_REPO_NAME=StarlingX-CentOS-7.6.1810 - extras
Processing: REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo REPO_ID=StarlingX-C7.6.1810-extras REPO_URL=http://vault.centos.org/7.6.1810/cloud/Source/openstack-pike/ DOWNLOAD_PATH=/export/mirror/centos/centos/vault.centos.org/7.6.1810/cloud/Source/openstack-pike UPSTREAM_REPO_ID=StarlingX-C7.6.1810-extras UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
REPO_URL=http://vault.centos.org/centos/7.6.1810/extras/x86_64/
REPO_NAME=StarlingX-CentOS-7.6.1810 - extras
Already have 'StarlingX-C7.6.1810-extras' from '/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo'
UPSTREAM_YUM_CONF=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.conf.sample
UPSTREAM_REPO_ID=StarlingX-C7.6.1810-extras-source
UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
UPSTREAM_REPO_URL=http://vault.centos.org/7.6.1810/extras/Source/
UPSTREAM_REPO_NAME=StarlingX-CentOS-7.6.1810 - extras-source
Processing: REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo REPO_ID=StarlingX-C7.6.1810-extras-source REPO_URL=http://vault.centos.org/centos/7.6.1810/extras/x86_64/ DOWNLOAD_PATH=/export/mirror/centos/centos/vault.centos.org/centos/7.6.1810/extras/x86_64 UPSTREAM_REPO_ID=StarlingX-C7.6.1810-extras-source UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
REPO_URL=http://vault.centos.org/7.6.1810/extras/Source/
REPO_NAME=StarlingX-CentOS-7.6.1810 - extras-source
Already have 'StarlingX-C7.6.1810-extras-source' from '/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo'
UPSTREAM_YUM_CONF=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.conf.sample
UPSTREAM_REPO_ID=StarlingX-C7.6.1810-nfv-fdio
UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
UPSTREAM_REPO_URL=http://vault.centos.org/centos/7.6.1810/nfv/x86_64/fdio/
UPSTREAM_REPO_NAME=StarlingX-CentOS-7.6.1810 - nfv
Processing: REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo REPO_ID=StarlingX-C7.6.1810-nfv-fdio REPO_URL=http://vault.centos.org/7.6.1810/extras/Source/ DOWNLOAD_PATH=/export/mirror/centos/centos/vault.centos.org/7.6.1810/extras/Source UPSTREAM_REPO_ID=StarlingX-C7.6.1810-nfv-fdio UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
REPO_URL=http://vault.centos.org/centos/7.6.1810/nfv/x86_64/fdio/
REPO_NAME=StarlingX-CentOS-7.6.1810 - nfv
Already have 'StarlingX-C7.6.1810-nfv-fdio' from '/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo'
UPSTREAM_YUM_CONF=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.conf.sample
UPSTREAM_REPO_ID=StarlingX-C7.6.1810-opstools
UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
UPSTREAM_REPO_URL=http://vault.centos.org/centos/7.6.1810/opstools/x86_64/
UPSTREAM_REPO_NAME=StarlingX-CentOS-7.6.1810 - opstools
Processing: REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo REPO_ID=StarlingX-C7.6.1810-opstools REPO_URL=http://vault.centos.org/centos/7.6.1810/nfv/x86_64/fdio/ DOWNLOAD_PATH=/export/mirror/centos/centos/vault.centos.org/centos/7.6.1810/nfv/x86_64/fdio UPSTREAM_REPO_ID=StarlingX-C7.6.1810-opstools UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
REPO_URL=http://vault.centos.org/centos/7.6.1810/opstools/x86_64/
REPO_NAME=StarlingX-CentOS-7.6.1810 - opstools
Already have 'StarlingX-C7.6.1810-opstools' from '/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo'
UPSTREAM_YUM_CONF=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.conf.sample
UPSTREAM_REPO_ID=StarlingX-C7.6.1810-opstools-source
UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
UPSTREAM_REPO_URL=http://vault.centos.org/7.6.1810/opstools/Source/
UPSTREAM_REPO_NAME=StarlingX-CentOS-7.6.1810 - opstools-source
Processing: REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo REPO_ID=StarlingX-C7.6.1810-opstools-source REPO_URL=http://vault.centos.org/centos/7.6.1810/opstools/x86_64/ DOWNLOAD_PATH=/export/mirror/centos/centos/vault.centos.org/centos/7.6.1810/opstools/x86_64 UPSTREAM_REPO_ID=StarlingX-C7.6.1810-opstools-source UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
REPO_URL=http://vault.centos.org/7.6.1810/opstools/Source/
REPO_NAME=StarlingX-CentOS-7.6.1810 - opstools-source
Already have 'StarlingX-C7.6.1810-opstools-source' from '/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo'
UPSTREAM_YUM_CONF=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.conf.sample
UPSTREAM_REPO_ID=StarlingX-C7.6.1810-os
UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
UPSTREAM_REPO_URL=http://vault.centos.org/centos/7.6.1810/os/x86_64/
UPSTREAM_REPO_NAME=StarlingX-CentOS-7.6.1810 - os
Processing: REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo REPO_ID=StarlingX-C7.6.1810-os REPO_URL=http://vault.centos.org/7.6.1810/opstools/Source/ DOWNLOAD_PATH=/export/mirror/centos/centos/vault.centos.org/7.6.1810/opstools/Source UPSTREAM_REPO_ID=StarlingX-C7.6.1810-os UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
REPO_URL=http://vault.centos.org/centos/7.6.1810/os/x86_64/
REPO_NAME=StarlingX-CentOS-7.6.1810 - os
Already have 'StarlingX-C7.6.1810-os' from '/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo'
UPSTREAM_YUM_CONF=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.conf.sample
UPSTREAM_REPO_ID=StarlingX-C7.6.1810-os-source
UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
UPSTREAM_REPO_URL=http://vault.centos.org/7.6.1810/os/Source/
UPSTREAM_REPO_NAME=StarlingX-CentOS-7.6.1810 - os-source
Processing: REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo REPO_ID=StarlingX-C7.6.1810-os-source REPO_URL=http://vault.centos.org/centos/7.6.1810/os/x86_64/ DOWNLOAD_PATH=/export/mirror/centos/centos/vault.centos.org/centos/7.6.1810/os/x86_64 UPSTREAM_REPO_ID=StarlingX-C7.6.1810-os-source UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
REPO_URL=http://vault.centos.org/7.6.1810/os/Source/
REPO_NAME=StarlingX-CentOS-7.6.1810 - os-source
Already have 'StarlingX-C7.6.1810-os-source' from '/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo'
UPSTREAM_YUM_CONF=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.conf.sample
UPSTREAM_REPO_ID=StarlingX-C7.6.1810-paas-openshift-origin
UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
UPSTREAM_REPO_URL=http://vault.centos.org/centos/7.6.1810/paas/x86_64/openshift-origin/
UPSTREAM_REPO_NAME=StarlingX-CentOS-7.6.1810 - paas-openshift-origin
Processing: REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo REPO_ID=StarlingX-C7.6.1810-paas-openshift-origin REPO_URL=http://vault.centos.org/7.6.1810/os/Source/ DOWNLOAD_PATH=/export/mirror/centos/centos/vault.centos.org/7.6.1810/os/Source UPSTREAM_REPO_ID=StarlingX-C7.6.1810-paas-openshift-origin UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
REPO_URL=http://vault.centos.org/centos/7.6.1810/paas/x86_64/openshift-origin/
REPO_NAME=StarlingX-CentOS-7.6.1810 - paas-openshift-origin
Already have 'StarlingX-C7.6.1810-paas-openshift-origin' from '/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo'
UPSTREAM_YUM_CONF=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.conf.sample
UPSTREAM_REPO_ID=StarlingX-C7.6.1810-paas-openshift-origin311
UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
UPSTREAM_REPO_URL=http://vault.centos.org/centos/7.6.1810/paas/x86_64/openshift-origin311/
UPSTREAM_REPO_NAME=StarlingX-CentOS-7.6.1810 - paas-openshift-origin311
Processing: REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo REPO_ID=StarlingX-C7.6.1810-paas-openshift-origin311 REPO_URL=http://vault.centos.org/centos/7.6.1810/paas/x86_64/openshift-origin/ DOWNLOAD_PATH=/export/mirror/centos/centos/vault.centos.org/centos/7.6.1810/paas/x86_64/openshift-origin UPSTREAM_REPO_ID=StarlingX-C7.6.1810-paas-openshift-origin311 UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
REPO_URL=http://vault.centos.org/centos/7.6.1810/paas/x86_64/openshift-origin311/
REPO_NAME=StarlingX-CentOS-7.6.1810 - paas-openshift-origin311
Already have 'StarlingX-C7.6.1810-paas-openshift-origin311' from '/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo'
UPSTREAM_YUM_CONF=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.conf.sample
UPSTREAM_REPO_ID=StarlingX-C7.6.1810-rt
UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
UPSTREAM_REPO_URL=http://vault.centos.org/centos/7.6.1810/rt/x86_64/
UPSTREAM_REPO_NAME=StarlingX-CentOS-7.6.1810 - rt
Processing: REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo REPO_ID=StarlingX-C7.6.1810-rt REPO_URL=http://vault.centos.org/centos/7.6.1810/paas/x86_64/openshift-origin311/ DOWNLOAD_PATH=/export/mirror/centos/centos/vault.centos.org/centos/7.6.1810/paas/x86_64/openshift-origin311 UPSTREAM_REPO_ID=StarlingX-C7.6.1810-rt UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
REPO_URL=http://vault.centos.org/centos/7.6.1810/rt/x86_64/
REPO_NAME=StarlingX-CentOS-7.6.1810 - rt
Already have 'StarlingX-C7.6.1810-rt' from '/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo'
UPSTREAM_YUM_CONF=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.conf.sample
UPSTREAM_REPO_ID=StarlingX-C7.6.1810-sclo-sclo
UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
UPSTREAM_REPO_URL=http://vault.centos.org/centos/7.6.1810/sclo/x86_64/sclo/
UPSTREAM_REPO_NAME=StarlingX-CentOS-7.6.1810 - sclo-sclo
Processing: REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo REPO_ID=StarlingX-C7.6.1810-sclo-sclo REPO_URL=http://vault.centos.org/centos/7.6.1810/rt/x86_64/ DOWNLOAD_PATH=/export/mirror/centos/centos/vault.centos.org/centos/7.6.1810/rt/x86_64 UPSTREAM_REPO_ID=StarlingX-C7.6.1810-sclo-sclo UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
REPO_URL=http://vault.centos.org/centos/7.6.1810/sclo/x86_64/sclo/
REPO_NAME=StarlingX-CentOS-7.6.1810 - sclo-sclo
Already have 'StarlingX-C7.6.1810-sclo-sclo' from '/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo'
UPSTREAM_YUM_CONF=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.conf.sample
UPSTREAM_REPO_ID=StarlingX-C7.6.1810-storage-ceph-jewel
UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
UPSTREAM_REPO_URL=http://vault.centos.org/centos/7.6.1810/storage/x86_64/ceph-jewel/
UPSTREAM_REPO_NAME=StarlingX-CentOS-7.6.1810 - storage-ceph-jewel
Processing: REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo REPO_ID=StarlingX-C7.6.1810-storage-ceph-jewel REPO_URL=http://vault.centos.org/centos/7.6.1810/sclo/x86_64/sclo/ DOWNLOAD_PATH=/export/mirror/centos/centos/vault.centos.org/centos/7.6.1810/sclo/x86_64/sclo UPSTREAM_REPO_ID=StarlingX-C7.6.1810-storage-ceph-jewel UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
REPO_URL=http://vault.centos.org/centos/7.6.1810/storage/x86_64/ceph-jewel/
REPO_NAME=StarlingX-CentOS-7.6.1810 - storage-ceph-jewel
Already have 'StarlingX-C7.6.1810-storage-ceph-jewel' from '/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo'
UPSTREAM_YUM_CONF=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.conf.sample
UPSTREAM_REPO_ID=StarlingX-C7.6.1810-storage-ceph-luminous
UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
UPSTREAM_REPO_URL=http://vault.centos.org/centos/7.6.1810/storage/x86_64/ceph-luminous/
UPSTREAM_REPO_NAME=StarlingX-CentOS-7.6.1810 - storage-ceph-luminous
Processing: REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo REPO_ID=StarlingX-C7.6.1810-storage-ceph-luminous REPO_URL=http://vault.centos.org/centos/7.6.1810/storage/x86_64/ceph-jewel/ DOWNLOAD_PATH=/export/mirror/centos/centos/vault.centos.org/centos/7.6.1810/storage/x86_64/ceph-jewel UPSTREAM_REPO_ID=StarlingX-C7.6.1810-storage-ceph-luminous UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
REPO_URL=http://vault.centos.org/centos/7.6.1810/storage/x86_64/ceph-luminous/
REPO_NAME=StarlingX-CentOS-7.6.1810 - storage-ceph-luminous
Already have 'StarlingX-C7.6.1810-storage-ceph-luminous' from '/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo'
UPSTREAM_YUM_CONF=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.conf.sample
UPSTREAM_REPO_ID=StarlingX-C7.6.1810-storage-ceph-nautilus
UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
UPSTREAM_REPO_URL=http://vault.centos.org/centos/7.6.1810/storage/x86_64/ceph-nautilus/
UPSTREAM_REPO_NAME=StarlingX-CentOS-7.6.1810 - storage-ceph-nautilus
Processing: REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo REPO_ID=StarlingX-C7.6.1810-storage-ceph-nautilus REPO_URL=http://vault.centos.org/centos/7.6.1810/storage/x86_64/ceph-luminous/ DOWNLOAD_PATH=/export/mirror/centos/centos/vault.centos.org/centos/7.6.1810/storage/x86_64/ceph-luminous UPSTREAM_REPO_ID=StarlingX-C7.6.1810-storage-ceph-nautilus UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
REPO_URL=http://vault.centos.org/centos/7.6.1810/storage/x86_64/ceph-nautilus/
REPO_NAME=StarlingX-CentOS-7.6.1810 - storage-ceph-nautilus
Already have 'StarlingX-C7.6.1810-storage-ceph-nautilus' from '/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo'
UPSTREAM_YUM_CONF=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.conf.sample
UPSTREAM_REPO_ID=StarlingX-C7.6.1810-storage-gluster-5
UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
UPSTREAM_REPO_URL=http://vault.centos.org/centos/7.6.1810/storage/x86_64/gluster-5/
UPSTREAM_REPO_NAME=StarlingX-CentOS-7.6.1810 - storage-gluster-5
Processing: REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo REPO_ID=StarlingX-C7.6.1810-storage-gluster-5 REPO_URL=http://vault.centos.org/centos/7.6.1810/storage/x86_64/ceph-nautilus/ DOWNLOAD_PATH=/export/mirror/centos/centos/vault.centos.org/centos/7.6.1810/storage/x86_64/ceph-nautilus UPSTREAM_REPO_ID=StarlingX-C7.6.1810-storage-gluster-5 UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
REPO_URL=http://vault.centos.org/centos/7.6.1810/storage/x86_64/gluster-5/
REPO_NAME=StarlingX-CentOS-7.6.1810 - storage-gluster-5
Already have 'StarlingX-C7.6.1810-storage-gluster-5' from '/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo'
UPSTREAM_YUM_CONF=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.conf.sample
UPSTREAM_REPO_ID=StarlingX-C7.6.1810-updates
UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
UPSTREAM_REPO_URL=http://vault.centos.org/centos/7.6.1810/updates/x86_64/
UPSTREAM_REPO_NAME=StarlingX-CentOS-7.6.1810 - updates
Processing: REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo REPO_ID=StarlingX-C7.6.1810-updates REPO_URL=http://vault.centos.org/centos/7.6.1810/storage/x86_64/gluster-5/ DOWNLOAD_PATH=/export/mirror/centos/centos/vault.centos.org/centos/7.6.1810/storage/x86_64/gluster-5 UPSTREAM_REPO_ID=StarlingX-C7.6.1810-updates UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
REPO_URL=http://vault.centos.org/centos/7.6.1810/updates/x86_64/
REPO_NAME=StarlingX-CentOS-7.6.1810 - updates
Already have 'StarlingX-C7.6.1810-updates' from '/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo'
UPSTREAM_YUM_CONF=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.conf.sample
UPSTREAM_REPO_ID=StarlingX-C7.6.1810-updates-source
UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
UPSTREAM_REPO_URL=http://vault.centos.org/7.6.1810/updates/Source/
UPSTREAM_REPO_NAME=StarlingX-CentOS-7.6.1810 - updates-source
Processing: REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo REPO_ID=StarlingX-C7.6.1810-updates-source REPO_URL=http://vault.centos.org/centos/7.6.1810/updates/x86_64/ DOWNLOAD_PATH=/export/mirror/centos/centos/vault.centos.org/centos/7.6.1810/updates/x86_64 UPSTREAM_REPO_ID=StarlingX-C7.6.1810-updates-source UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
REPO_URL=http://vault.centos.org/7.6.1810/updates/Source/
REPO_NAME=StarlingX-CentOS-7.6.1810 - updates-source
Already have 'StarlingX-C7.6.1810-updates-source' from '/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo'
UPSTREAM_YUM_CONF=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.conf.sample
UPSTREAM_REPO_ID=StarlingX-C7.6.1810-virt-kvm
UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
UPSTREAM_REPO_URL=http://vault.centos.org/centos/7.6.1810/virt/x86_64/kvm-common/
UPSTREAM_REPO_NAME=StarlingX-CentOS-7.6.1810 - virt-kvm
Processing: REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo REPO_ID=StarlingX-C7.6.1810-virt-kvm REPO_URL=http://vault.centos.org/7.6.1810/updates/Source/ DOWNLOAD_PATH=/export/mirror/centos/centos/vault.centos.org/7.6.1810/updates/Source UPSTREAM_REPO_ID=StarlingX-C7.6.1810-virt-kvm UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
REPO_URL=http://vault.centos.org/centos/7.6.1810/virt/x86_64/kvm-common/
REPO_NAME=StarlingX-CentOS-7.6.1810 - virt-kvm
Already have 'StarlingX-C7.6.1810-virt-kvm' from '/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo'
UPSTREAM_YUM_CONF=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.conf.sample
UPSTREAM_REPO_ID=StarlingX-C7.6.1810-virt-libvirt
UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
UPSTREAM_REPO_URL=http://vault.centos.org/centos/7.6.1810/virt/x86_64/libvirt-latest/
UPSTREAM_REPO_NAME=StarlingX-CentOS-7.6.1810 - virt-libvirt
Processing: REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo REPO_ID=StarlingX-C7.6.1810-virt-libvirt REPO_URL=http://vault.centos.org/centos/7.6.1810/virt/x86_64/kvm-common/ DOWNLOAD_PATH=/export/mirror/centos/centos/vault.centos.org/centos/7.6.1810/virt/x86_64/kvm-common UPSTREAM_REPO_ID=StarlingX-C7.6.1810-virt-libvirt UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
REPO_URL=http://vault.centos.org/centos/7.6.1810/virt/x86_64/libvirt-latest/
REPO_NAME=StarlingX-CentOS-7.6.1810 - virt-libvirt
Already have 'StarlingX-C7.6.1810-virt-libvirt' from '/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo'
UPSTREAM_YUM_CONF=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.conf.sample
UPSTREAM_REPO_ID=StarlingX-C7.6.1810-sclo-rh
UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
UPSTREAM_REPO_URL=http://vault.centos.org/centos/7.6.1810/sclo/x86_64/rh/
UPSTREAM_REPO_NAME=StarlingX-CentOS-7.6.1810 - sclo-rh
Processing: REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo REPO_ID=StarlingX-C7.6.1810-sclo-rh REPO_URL=http://vault.centos.org/centos/7.6.1810/virt/x86_64/libvirt-latest/ DOWNLOAD_PATH=/export/mirror/centos/centos/vault.centos.org/centos/7.6.1810/virt/x86_64/libvirt-latest UPSTREAM_REPO_ID=StarlingX-C7.6.1810-sclo-rh UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo
REPO_URL=http://vault.centos.org/centos/7.6.1810/sclo/x86_64/rh/
REPO_NAME=StarlingX-CentOS-7.6.1810 - sclo-rh
Already have 'StarlingX-C7.6.1810-sclo-rh' from '/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.6.repo'
UPSTREAM_YUM_CONF=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.conf.sample
UPSTREAM_REPO_ID=StarlingX-C7.7.1908-atomic
UPSTREAM_REPO=/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.7.repo
http://mirror.centos.org/centos/7.7.1908/atomic/x86_64/repodata/repomd.xml: [Errno 14] HTTP Error 404 - Not Found
Trying other mirror.
To address this issue please refer to the below wiki article
https://wiki.centos.org/yum-errors

If above article doesn't help to resolve this issue please use https://bugs.centos.org/.

http://mirror.centos.org/centos/7.7.1908/atomic/x86_64/repodata/1ffdf5a5a4e5484d903c8228f77440ebd00ae7f0e7d75a7883e8208b51204545-primary.sqlite.bz2: [Errno 14] HTTP Error 404 - Not Found
Trying other mirror.
http://mirror.centos.org/centos/7.7.1908/atomic/x86_64/repodata/1ffdf5a5a4e5484d903c8228f77440ebd00ae7f0e7d75a7883e8208b51204545-primary.sqlite.bz2: [Errno 14] HTTP Error 404 - Not Found
Trying other mirror.


One of the configured repositories failed (StarlingX-CentOS-7.7.1908 - atomic),
and yum doesn't have enough cached data to continue. At this point the only
safe thing yum can do is fail. There are a few ways to work "fix" this:

1. Contact the upstream for the repository and get them to fix the problem.

2. Reconfigure the baseurl/etc. for the repository, to point to a working
upstream. This is most often useful if you are using a newer
distribution release than is supported by the repository (and the
packages for the previous distribution release still work).

3. Run the command with the repository temporarily disabled
yum --disablerepo=StarlingX-C7.7.1908-atomic ...

4. Disable the repository permanently, so yum won't use it by default. Yum
will then just ignore the repository until you permanently enable it
again or use --enablerepo for temporary usage:

yum-config-manager --disable StarlingX-C7.7.1908-atomic
or
subscription-manager repos --disable=StarlingX-C7.7.1908-atomic

5. Configure the failing repository to be skipped, if it is unavailable.
Note that yum will try to contact the repo. when it runs most commands,
so will have to try and fail each time (and thus. yum will be be much
slower). If it is a very temporary problem though, this is often a nice
compromise:

yum-config-manager --save --setopt=StarlingX-C7.7.1908-atomic.skip_if_unavailable=true

failure: repodata/1ffdf5a5a4e5484d903c8228f77440ebd00ae7f0e7d75a7883e8208b51204545-primary.sqlite.bz2 from StarlingX-C7.7.1908-atomic: [Errno 256] No more mirrors to try.
http://mirror.centos.org/centos/7.7.1908/atomic/x86_64/repodata/1ffdf5a5a4e5484d903c8228f77440ebd00ae7f0e7d75a7883e8208b51204545-primary.sqlite.bz2: [Errno 14] HTTP Error 404 - Not Found
ERROR: yum repoinfo --config='/var/lib/jenkins/stx-repo/master/stx-tools/centos-mirror-tools/yum.conf.sample' --disablerepo='*' --enablerepo='StarlingX-C7.7.1908-atomic'
Error: Failed in update_yum_repos_d. Can't continue.
Build step 'Execute shell' marked build as failure