Comments inline -----Original Message----- From: Cordoba Malibran, Erich [mailto:erich.cordoba.malibran@intel.com] Sent: Wednesday, July 11, 2018 3:18 PM To: Little, Scott <Scott.Little@windriver.com>; starlingx-discuss@lists.starlingx.io Subject: Re: [Starlingx-discuss] Download packages issues Hi Scott. Indeed, the mirror downloading is very fragile, actually some of the packages can't be downloaded with yumdownloader and direct wget is used. Every time that a package has a new patch a new release version is generated and our mirror is broken. Not sure about hosting our own mirror, although it will ease our process in a short term, the path to be able to run with latest CentOS (or another distro) will be harder to reach. So, this could be a first step going there. Probably, in the future we can remove from the build system the dependency on specific srpm versions (1.2.3-10 <- moving from 10 to 11 shouldn't be an issue) and just update patches when is necessary. [BR] We are not going to be able to remove dependencies on specific rpm versions for packages with patches against them which is currently a significant number. We need predictable builds and finding out at build time that patches don’t apply is not really acceptable. Once this has been eliminated/significantly reduced then perhaps that becomes an option. Anyway, in this particular problem, kernel-3.10.0-862.3.2.el7.src.rpm was added here[0] but the proper repository wasn't added. The fix for this was already merged[1] The issue about not finding git was fixed last week, so maybe a git pull may help. -Erich [0] https://review.openstack.org/#/c/580689/ [1] https://review.openstack.org/#/c/581471/ From: Scott Little <scott.little@windriver.com> Date: Wednesday, July 11, 2018 at 9:09 AM To: "starlingx-discuss@lists.starlingx.io" <starlingx-discuss@lists.starlingx.io> Subject: [Starlingx-discuss] Download packages issues The 'bash download_mirror.sh' step seems to be very fragile.  e.g. this morning I tried to start fresh and it failed at several points.    ------ using kernel-3.10.0-862.3.2.el7 to search kernel-3.10.0-862.3.2.el7.src.rpm ------    --> run: sudo -E yumdownloader -q -C --source kernel-3.10.0-862.3.2.el7    Looking for kernel-3.10.0-862.3.2.el7.src.rpm    No source RPM found for kernel-3.10.0-862.3.2.el7.x86_64    Nothing to download     ------ using epel-rpm-macros-7-18 to search epel-rpm-macros-7-18.noarch.rpm ------    --> run: sudo -E yumdownloader -q -C epel-rpm-macros-7-18 --archlist=noarch,x86_64    Looking for epel-rpm-macros-7-18.noarch.rpm...    No Match for argument epel-rpm-macros-7-18    Nothing to download and the tarball downloads failed on.    --2018-07-11 13:13:28--  http://git.infradead.org/users/jjs/linux-tpmdd.git/snapshot/fd5c78694f3f1c87...    Resolving git.infradead.org (git.infradead.org)... 198.137.202.133, 2607:7c80:54:e::133    Connecting to git.infradead.org (git.infradead.org)|198.137.202.133|:80... connected.    HTTP request sent, awaiting response... 200 OK    Length: unspecified [application/x-gzip]    Saving to: 'fd5c78694f3f1c875e293de7a641ba8a3d60d00d.tar.gz'    2018-07-11 13:14:34 (0.00 B/s) - 'fd5c78694f3f1c875e293de7a641ba8a3d60d00d.tar.gz' saved [0]    tar: This does not look like a tar archive    gzip: stdin: unexpected end of file    tar: Child returned status 1    tar: Error is not recoverable: exiting now    mv: cannot stat 'linux-tpmdd-fd5c786/drivers/char/tpm': No such file or directory    tar: tpm: Cannot stat: No such file or directory    tar: Exiting with failure status due to previous errors    /localdisk    tss2-930.tar.gz    /localdisk/output/stx-r1/CentOS/pike/downloads /localdisk    ./tarball-dl.sh: line 156: git: command not found    ./tarball-dl.sh: line 157: pushd: ibmtpm20tss-tss: No such file or directory    ./tarball-dl.sh: line 158: git: command not found    /localdisk    mv: cannot stat 'ibmtpm20tss-tss': No such file or directory    tar: tss2-930: Cannot stat: No such file or directory    tar: Exiting with failure status due to previous errors    ./tarball-dl.sh: line 165: popd: directory stack empty Is this really going to be our default operating mode?   Never knowing, day by day, when the next package will be dropped from an upstream source.  Is it possible for strarlingX to host it's own mirrors... of at least the needed subset of packages? Scott _______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss