There are 3 repos in StarlingX_3rd.repo that are unreachable currently from our site, due to firewall concerns. Disabling these repos allows me to complete the download_mirror.sh
$ git diff StarlingX_3rd.repo
diff --git a/centos-mirror-tools/StarlingX_3rd.repo b/centos-mirror-tools/StarlingX_3rd.repo
index d9c5b37..d669784 100644
--- a/centos-mirror-tools/StarlingX_3rd.repo
+++ b/centos-mirror-tools/StarlingX_3rd.repo
@@ -11,17 +11,17 @@ enabled=1
[Starlingx-epel.blizoo.mk_epel_7Server_x86_64]
name=Starlingx-epel.blizoo.mk_epel_7Server_x86_64 - epel.blizoo.mk_epel_7Server_x86_64
baseurl=http://epel.blizoo.mk/epel/7Server/x86_64
-enabled=1
+enabled=0
[Starlingx-epel.blizoo.mk_epel_7_SRPMS]
name=Starlingx-epel.blizoo.mk_epel_7_SRPMS - epel.blizoo.mk_epel_7_SRPMS
baseurl=http://epel.blizoo.mk/epel/7/SRPMS
-enabled=1
+enabled=0
[Starlingx-ftp.iij.ad.jp_pub_linux_centos_7_rt_x86_64]
name=Starlingx-ftp.iij.ad.jp_pub_linux_centos_7_rt_x86_64 - ftp.iij.ad.jp_pub_linux_centos_7_rt_x86_64
baseurl=http://ftp.iij.ad.jp/pub/linux/centos/7/rt/x86_64
-enabled=1
+enabled=0
[Starlingx-ftp.riken.jp_Linux_cern_centos_7_rt_Sources]
name=Starlingx-ftp.riken.jp_Linux_cern_centos_7_rt_Sources - ftp.riken.jp_Linux_cern_centos_7_rt_Sources
Without these disabled, the “yum makecache” aborts with an error. With them disabled, the only missing RPMs I have now are those covered by this pending update:
https://review.openstack.org/#/c/586393/
From: Penney, Don [mailto:Don.Penney@windriver.com]
Sent: Friday, July 27, 2018 8:56 AM
To: Little, Scott; starlingx-discuss@lists.starlingx.io
Subject: Re: [Starlingx-discuss] download_mirror.sh using bash -e causes early exit
We need to drop the -e from these scripts. Proper error checking would need to be added to commands that need it.
The failure to access the blizoo repo we’ve been experiencing caused the 1st stage download to silently abort when the “yum makecache” failed.
failure: repodata/repomd.xml from Starlingx-epel.blizoo.mk_epel_7Server_x86_64: [Errno 256] No more mirrors to try.
http://epel.blizoo.mk/epel/7Server/x86_64/repodata/repomd.xml: [Errno 14] HTTP Error 403 - Forbidden
step #2: start 1st round of downloading RPMs and SRPMs with L1 match criteria...
using ./rpms_from_centos_repo.lst as the download name lists
[root@fd7ddd238144 localdisk]# cat logs/log_download_rpms_from_3rd_party.txt
using ./rpms_from_3rd_parties.lst as the download name lists
2018-07-26_2009
Loaded plugins: fastestmirror, ovl
[root@fd7ddd238144 localdisk]#
From: Scott Little [mailto:scott.little@windriver.com]
Sent: Thursday, July 26, 2018 3:53 PM
To: starlingx-discuss@lists.starlingx.io
Subject: Re: [Starlingx-discuss] download_mirror.sh using bash -e causes early exit
find ./output -type f -name "*.rpm" | xargs rpm -K | { grep -i "MISSING KEYS" || true } > ./rpm-gpg-key-missing.txt
Scott
On 18-07-26 03:46 PM, Penney, Don wrote:
Hi folks,
I was just running a fresh download_mirror.sh execution and ran into a problem. I run it as follows, to get timing info and record output:
time ./download_mirror.sh 2>&1 | tee dpenney.out
The script exited here:
--> run: sudo -E yumdownloader -q -C --source python-voluptuous-0.8.9-1.el7
Looking for python-voluptuous-0.8.9-1.el7.src.rpm
done!!
finish 1st round of RPM downloading successfully!
ERROR: -------RPMs missing 6 in yumdownloader with L1 match ---------------
real 84m3.238s
user 62m49.391s
sys 9m46.608s
It seemed like an odd place to finish “1st round”, so I took a look around there, checking various timestamps to try to see where it quit, and found the last command executed was this line:
## verify all RPMs SRPMs we download for the GPG keys
find ./output -type f -name "*.rpm" | xargs rpm -K | grep -i "MISSING KEYS" > ./rpm-gpg-key-missing.txt
Because I had no missing keys, the last piped command returned a non-zero status. And since this script is launched with “bash -e” on the first line, and I ran the script directly instead of “bash download_mirror.sh”, the script exited at this point.
The other download scripts use “bash -e” here as well. Since these are called directly from download_mirror.sh, they could be exiting early silently.
Cheers,
Don.
Don Penney, Developer, Wind River
_______________________________________________Starlingx-discuss mailing listStarlingx-discuss@lists.starlingx.iohttp://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss