[Starlingx-discuss] download_mirror.sh using bash -e causes early exit

[Penney, Don]

We need to drop the -e from these scripts. Proper error checking would need to be added to commands that need it.

The failure to access the blizoo repo we've been experiencing caused the 1st stage download to silently abort when the "yum makecache" failed.

failure: repodata/repomd.xml from Starlingx-epel.blizoo.mk_epel_7Server_x86_64: [Errno 256] No more mirrors to try.
http://epel.blizoo.mk/epel/7Server/x86_64/repodata/repomd.xml: [Errno 14] HTTP Error 403 - Forbidden
step #2: start 1st round of downloading RPMs and SRPMs with L1 match criteria...
using ./rpms_from_centos_repo.lst as the download name lists


[root at fd7ddd238144 localdisk]# cat logs/log_download_rpms_from_3rd_party.txt
using ./rpms_from_3rd_parties.lst as the download name lists
2018-07-26_2009
Loaded plugins: fastestmirror, ovl
[root at fd7ddd238144 localdisk]#



From: Scott Little [mailto:scott.little at windriver.com]
Sent: Thursday, July 26, 2018 3:53 PM
To: starlingx-discuss at lists.starlingx.io
Subject: Re: [Starlingx-discuss] download_mirror.sh using bash -e causes early exit

find ./output -type f -name "*.rpm" | xargs rpm -K | { grep -i "MISSING KEYS" || true } > ./rpm-gpg-key-missing.txt

Scott

On 18-07-26 03:46 PM, Penney, Don wrote:
Hi folks,

I was just running a fresh download_mirror.sh execution and ran into a problem. I run it as follows, to get timing info and record output:
time ./download_mirror.sh 2>&1 | tee dpenney.out

The script exited here:
--> run: sudo -E yumdownloader -q -C --source python-voluptuous-0.8.9-1.el7
Looking for python-voluptuous-0.8.9-1.el7.src.rpm
done!!
finish 1st round of RPM downloading successfully!
ERROR:  -------RPMs missing 6 in yumdownloader with L1 match ---------------

real    84m3.238s
user    62m49.391s
sys     9m46.608s

It seemed like an odd place to finish "1st round", so I took a look around there, checking various timestamps to try to see where it quit, and found the last command executed was this line:

## verify all RPMs SRPMs we download for the GPG keys
find ./output -type f -name "*.rpm" | xargs rpm -K | grep -i "MISSING KEYS" > ./rpm-gpg-key-missing.txt

Because I had no missing keys, the last piped command returned a non-zero status. And since this script is launched with "bash -e" on the first line, and I ran the script directly instead of "bash download_mirror.sh", the script exited at this point.

The other download scripts use "bash -e" here as well. Since these are called directly from download_mirror.sh, they could be exiting early silently.

Cheers,
Don.

Don Penney, Developer, Wind River







_______________________________________________

Starlingx-discuss mailing list

Starlingx-discuss at lists.starlingx.io<mailto:Starlingx-discuss at lists.starlingx.io>

http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20180727/f6156f99/attachment.html>