Re: [Starlingx-discuss] [build][meetings] Meeting minutes 11/1/2018
Cesar, A quick update on the mirror. The tooling to enable the mirror was merged on Friday. The Wind River used the mirror over the week-end for its builds with no issues. How have your builds gone since Friday? /KenY From: "Lara, Cesar" <cesar.lara@intel.com> Date: Thursday, November 1, 2018 at 6:58 PM To: "starlingx-discuss@lists.starlingx.io" <starlingx-discuss@lists.starlingx.io> Subject: [Starlingx-discuss] [build][meetings] Meeting minutes 11/1/2018 Build team meeting minutes 11/1/2018 Attendees Ken, Jason, Scott, Memo, Erich, Marcela, Victor, Chuy, Cesar, Luis Cengn Status -Mirror is now completed and operational, the wiki has been updated with the details for Cengn and our partnership, the details and specs are already there. -The scripts that are related with mirror download are now pointing to Cengn repos and are waiting in the queue to be reviewed tested and merged, the desired date for fully operational mirror is November 5th - Now changes to the packages required to build will be caught by Cengn mirror within one day, the scripts will try to download first from Cengn and if is still not available will try to go get the packages from upstream. Koji follow up -presentation on a possible scenario hosting Koji and demo on using Koji based built packages with our current build scripts. -discussion followed up by implementation details led to different points of view about the problem we have to solve if there’s any, the amount of effort to integrate tooling as we know Koji doesn’t solve every use case by itself and where to host a Koji service(we are striving to host a centralized service for the whole community). -The statement we are using to follow thru with Koji versus our current script is that we will require a lot of effort to support multi OS with the current tools. We had to defer the topic of mirror script refactored using Python. We will start a thread on the goals this team has for next release on the mailing list. Regards Cesar Lara Software Engineering Manager OpenSource Technology Center
Hi Ken! We tried mirror_downloader.sh on our Jenkins automated environment and also on my local environment and when it finished running we noticed that the following files were moved: this ones were sent to: “/localdisk/output/stx-r1/CentOS/pike/Binary/x86_64” -rpms_3rdparties.lst -rpms_centos.lst -rpms_centos3rdparties.lst -tarball-dl.lst Same with “yum.repos.d” were sent to: “CentOS/pike/Binary/noarch/” CentOS/pike/Binary/noarch/yum.repos.d/ CentOS/pike/Binary/noarch/yum.repos.d/StarlingX-Centos-7.2.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_buildlogs_centos_org.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_download_docker_com.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_CentOS-OpenStack-queens.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_epel_blizoo_mk.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_linuxsoft_cern_ch.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX-Centos-7.3.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_buildlogs-seed_centos_org.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_www_rpmfind_net.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX-Centos-7.4.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX-EPEL-7.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_ftp_iij_ad_jp.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX-Centos-7.5.repo And finally rpm-gpg-keys sent to: “CentOS/pike/Binary/x86_64” CentOS/pike/Binary/x86_64/rpm-gpg-keys/ CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-FC27-primary CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-SIG-SCLo CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-Debug-6 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-6 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-EPEL-6 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-Security-6 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-SIG-OpsTools CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-FC29 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-FC27 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-EPEL-7 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-5 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-DOCKER CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-cern CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-SIG-Virtualization CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-FC27-second CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-FC28 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-Testing-6 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-Cloud-SIG Is this the expected behavior? I’m currently finishing the build with this mirror, will update once it’s done. Best -Hayde From: "Young, Ken" <Ken.Young@windriver.com> Date: Monday, November 5, 2018 at 2:43 PM To: "Lara, Cesar" <cesar.lara@intel.com> Cc: "starlingx-discuss@lists.starlingx.io" <starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] [build][meetings] Meeting minutes 11/1/2018 Cesar, A quick update on the mirror. The tooling to enable the mirror was merged on Friday. The Wind River used the mirror over the week-end for its builds with no issues. How have your builds gone since Friday? /KenY From: "Lara, Cesar" <cesar.lara@intel.com> Date: Thursday, November 1, 2018 at 6:58 PM To: "starlingx-discuss@lists.starlingx.io" <starlingx-discuss@lists.starlingx.io> Subject: [Starlingx-discuss] [build][meetings] Meeting minutes 11/1/2018 Build team meeting minutes 11/1/2018 Attendees Ken, Jason, Scott, Memo, Erich, Marcela, Victor, Chuy, Cesar, Luis Cengn Status -Mirror is now completed and operational, the wiki has been updated with the details for Cengn and our partnership, the details and specs are already there. -The scripts that are related with mirror download are now pointing to Cengn repos and are waiting in the queue to be reviewed tested and merged, the desired date for fully operational mirror is November 5th - Now changes to the packages required to build will be caught by Cengn mirror within one day, the scripts will try to download first from Cengn and if is still not available will try to go get the packages from upstream. Koji follow up -presentation on a possible scenario hosting Koji and demo on using Koji based built packages with our current build scripts. -discussion followed up by implementation details led to different points of view about the problem we have to solve if there’s any, the amount of effort to integrate tooling as we know Koji doesn’t solve every use case by itself and where to host a Koji service(we are striving to host a centralized service for the whole community). -The statement we are using to follow thru with Koji versus our current script is that we will require a lot of effort to support multi OS with the current tools. We had to defer the topic of mirror script refactored using Python. We will start a thread on the goals this team has for next release on the mailing list. Regards Cesar Lara Software Engineering Manager OpenSource Technology Center
I can't reproduce this, with or without https://review.openstack.org/#/c/615316 Where you in the container when you ran mirror_downloader.sh? In directory /localdisk? Scott On 18-11-06 11:51 AM, Martinez Landa, Hayde wrote:
Hi Ken!
We tried mirror_downloader.sh on our Jenkins automated environment and also on
my local environment and when it finished running we noticed that the following files were moved:
this ones were sent to: “/localdisk/output/stx-r1/CentOS/pike/Binary/x86_64”
-rpms_3rdparties.lst
-rpms_centos.lst
-rpms_centos3rdparties.lst
-tarball-dl.lst
Same with “yum.repos.d” were sent to: “CentOS/pike/Binary/noarch/”
CentOS/pike/Binary/noarch/yum.repos.d/
CentOS/pike/Binary/noarch/yum.repos.d/StarlingX-Centos-7.2.repo
CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_buildlogs_centos_org.repo
CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_download_docker_com.repo
CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_CentOS-OpenStack-queens.repo
CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_epel_blizoo_mk.repo
CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_linuxsoft_cern_ch.repo
CentOS/pike/Binary/noarch/yum.repos.d/StarlingX-Centos-7.3.repo
CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_buildlogs-seed_centos_org.repo
CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_www_rpmfind_net.repo
CentOS/pike/Binary/noarch/yum.repos.d/StarlingX-Centos-7.4.repo
CentOS/pike/Binary/noarch/yum.repos.d/StarlingX-EPEL-7.repo
CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_ftp_iij_ad_jp.repo
CentOS/pike/Binary/noarch/yum.repos.d/StarlingX-Centos-7.5.repo
And finally rpm-gpg-keys sent to: “CentOS/pike/Binary/x86_64”
CentOS/pike/Binary/x86_64/rpm-gpg-keys/
CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-FC27-primary
CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-SIG-SCLo
CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-Debug-6
CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-6
CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-EPEL-6
CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-Security-6
CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-SIG-OpsTools
CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-FC29
CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-FC27
CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-EPEL-7
CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-5
CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-DOCKER
CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-cern
CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-SIG-Virtualization
CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-FC27-second
CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-FC28
CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-Testing-6
CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-Cloud-SIG
Is this the expected behavior?
I’m currently finishing the build with this mirror, will update once it’s done.
Best
-Hayde
*From: *"Young, Ken" <Ken.Young@windriver.com> *Date: *Monday, November 5, 2018 at 2:43 PM *To: *"Lara, Cesar" <cesar.lara@intel.com> *Cc: *"starlingx-discuss@lists.starlingx.io" <starlingx-discuss@lists.starlingx.io> *Subject: *Re: [Starlingx-discuss] [build][meetings] Meeting minutes 11/1/2018
Cesar,
A quick update on the mirror. The tooling to enable the mirror was merged on Friday. The Wind River used the mirror over the week-end for its builds with no issues.
How have your builds gone since Friday?
/KenY
*From: *"Lara, Cesar" <cesar.lara@intel.com> *Date: *Thursday, November 1, 2018 at 6:58 PM *To: *"starlingx-discuss@lists.starlingx.io" <starlingx-discuss@lists.starlingx.io> *Subject: *[Starlingx-discuss] [build][meetings] Meeting minutes 11/1/2018
Build team meeting minutes 11/1/2018
Attendees
Ken, Jason, Scott, Memo, Erich, Marcela, Victor, Chuy, Cesar, Luis
Cengn Status
-Mirror is now completed and operational, the wiki has been updated with the details for Cengn and our partnership, the details and specs are already there.
-The scripts that are related with mirror download are now pointing to Cengn repos and are waiting in the queue to be reviewed tested and merged, the desired date for fully operational mirror is November 5^th
- Now changes to the packages required to build will be caught by Cengn mirror within one day, the scripts will try to download first from Cengn and if is still not available will try to go get the packages from upstream.
Koji follow up
-presentation on a possible scenario hosting Koji and demo on using Koji based built packages with our current build scripts.
-discussion followed up by implementation details led to different points of view about the problem we have to solve if there’s any, the amount of effort to integrate tooling as we know Koji doesn’t solve every use case by itself and where to host a Koji service(we are striving to host a centralized service for the whole community).
-The statement we are using to follow thru with Koji versus our current script is that we will require a lot of effort to support multi OS with the current tools.
We had to defer the topic of mirror script refactored using Python.
We will start a thread on the goals this team has for next release on the mailing list.
Regards
*//*
*/Cesar Lara/*
Software Engineering Manager
OpenSource Technology Center
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
We have seen this behavior in two different systems. The script is run in this way: cd stx-tools/centos-mirror-tools docker build -t stx-mirror -f Dockerfile . docker run -ti -v $(pwd):/localdisk stx-mirror ./download_mirror.sh After run the script, this is what git reports: On branch master Your branch is up to date with 'origin/master'. Changes not staged for commit: (use "git add/rm <file>..." to update what will be committed) (use "git checkout -- <file>..." to discard changes in working directory) deleted: centos-mirror-tools/make_stx_mirror_yum_conf.sh deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-CentOS-5 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-CentOS-6 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-CentOS-Debug-6 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-CentOS-SIG-OpsTools deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-CentOS-SIG-SCLo deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-CentOS-SIG-Virtualization deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-CentOS-Security-6 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-CentOS-Testing-6 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-Cloud-SIG deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-DOCKER deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-EPEL-6 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-EPEL-7 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-FC27 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-FC27-primary deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-FC27-second deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-FC28 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-FC29 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-cern deleted: centos-mirror-tools/rpms_3rdparties.lst deleted: centos-mirror-tools/rpms_centos.lst deleted: centos-mirror-tools/rpms_centos3rdparties.lst deleted: centos-mirror-tools/tarball-dl.lst deleted: centos-mirror-tools/yum.conf.sample deleted: centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.2.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.3.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.4.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.5.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX-EPEL-7.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX_3rd_buildlogs-seed_centos_org.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX_3rd_buildlogs_centos_org.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX_3rd_download_docker_com.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX_3rd_epel_blizoo_mk.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX_3rd_ftp_iij_ad_jp.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX_3rd_linuxsoft_cern_ch.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX_3rd_www_rpmfind_net.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX_CentOS-OpenStack-queens.repo On Tue, 2018-11-06 at 16:33 -0500, Scott Little wrote:
I can't reproduce this, with or without https://review.openstack.org/ #/c/615316
Where you in the container when you ran mirror_downloader.sh? In directory /localdisk?
Scott
On 18-11-06 11:51 AM, Martinez Landa, Hayde wrote:
Hi Ken!
We tried mirror_downloader.sh on our Jenkins automated environment and also on my local environment and when it finished running we noticed that the following files were moved:
this ones were sent to: “/localdisk/output/stx- r1/CentOS/pike/Binary/x86_64” -rpms_3rdparties.lst -rpms_centos.lst -rpms_centos3rdparties.lst -tarball-dl.lst Same with “yum.repos.d” were sent to: “CentOS/pike/Binary/noarch/” CentOS/pike/Binary/noarch/yum.repos.d/ CentOS/pike/Binary/noarch/yum.repos.d/StarlingX-Centos-7.2.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_buildlogs_cento s_org.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_download_docker _com.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_CentOS-OpenStack- queens.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_epel_blizoo_mk. repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_linuxsoft_cern_ ch.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX-Centos-7.3.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_buildlogs- seed_centos_org.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_www_rpmfind_net .repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX-Centos-7.4.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX-EPEL-7.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_ftp_iij_ad_jp.r epo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX-Centos-7.5.repo
And finally rpm-gpg-keys sent to: “CentOS/pike/Binary/x86_64” CentOS/pike/Binary/x86_64/rpm-gpg-keys/ CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-FC27-primary CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-SIG-SCLo CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-Debug-6 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-6 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-EPEL-6 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-Security- 6 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-SIG- OpsTools CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-FC29 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-FC27 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-EPEL-7 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-5 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-DOCKER CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-cern CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-SIG- Virtualization CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-FC27-second CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-FC28 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-Testing-6 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-Cloud-SIG
Is this the expected behavior?
I’m currently finishing the build with this mirror, will update once it’s done.
Best -Hayde
From: "Young, Ken" <Ken.Young@windriver.com> Date: Monday, November 5, 2018 at 2:43 PM To: "Lara, Cesar" <cesar.lara@intel.com> Cc: "starlingx-discuss@lists.starlingx.io" <starlingx-discuss@lists .starlingx.io> Subject: Re: [Starlingx-discuss] [build][meetings] Meeting minutes 11/1/2018
Cesar,
A quick update on the mirror. The tooling to enable the mirror was merged on Friday. The Wind River used the mirror over the week-end for its builds with no issues.
How have your builds gone since Friday?
/KenY
From: "Lara, Cesar" <cesar.lara@intel.com> Date: Thursday, November 1, 2018 at 6:58 PM To: "starlingx-discuss@lists.starlingx.io" <starlingx-discuss@lists .starlingx.io> Subject: [Starlingx-discuss] [build][meetings] Meeting minutes 11/1/2018
Build team meeting minutes 11/1/2018
Attendees Ken, Jason, Scott, Memo, Erich, Marcela, Victor, Chuy, Cesar, Luis
Cengn Status
-Mirror is now completed and operational, the wiki has been updated with the details for Cengn and our partnership, the details and specs are already there. -The scripts that are related with mirror download are now pointing to Cengn repos and are waiting in the queue to be reviewed tested and merged, the desired date for fully operational mirror is November 5th - Now changes to the packages required to build will be caught by Cengn mirror within one day, the scripts will try to download first from Cengn and if is still not available will try to go get the packages from upstream.
Koji follow up
-presentation on a possible scenario hosting Koji and demo on using Koji based built packages with our current build scripts. -discussion followed up by implementation details led to different points of view about the problem we have to solve if there’s any, the amount of effort to integrate tooling as we know Koji doesn’t solve every use case by itself and where to host a Koji service(we are striving to host a centralized service for the whole community). -The statement we are using to follow thru with Koji versus our current script is that we will require a lot of effort to support multi OS with the current tools.
We had to defer the topic of mirror script refactored using Python.
We will start a thread on the goals this team has for next release on the mailing list.
Regards
Cesar Lara Software Engineering Manager OpenSource Technology Center
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discus s
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Ok, I'll repo sync and try a new Docker. I'll run these exact commands. My previous docker was over a month old. Perhaps the difference lies there. For the first pass I'll run using the bug fix in https://review.openstack.org/#/c/615316/ . I would like your eyes on that review. Scott On 18-11-07 10:16 AM, Cordoba Malibran, Erich wrote:
We have seen this behavior in two different systems.
The script is run in this way:
cd stx-tools/centos-mirror-tools docker build -t stx-mirror -f Dockerfile . docker run -ti -v $(pwd):/localdisk stx-mirror ./download_mirror.sh
After run the script, this is what git reports:
On branch master Your branch is up to date with 'origin/master'.
Changes not staged for commit: (use "git add/rm <file>..." to update what will be committed) (use "git checkout -- <file>..." to discard changes in working directory)
deleted: centos-mirror-tools/make_stx_mirror_yum_conf.sh deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-CentOS-5 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-CentOS-6 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-CentOS-Debug-6 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-CentOS-SIG-OpsTools deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-CentOS-SIG-SCLo deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-CentOS-SIG-Virtualization deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-CentOS-Security-6 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-CentOS-Testing-6 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-Cloud-SIG deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-DOCKER deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-EPEL-6 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-EPEL-7 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-FC27 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-FC27-primary deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-FC27-second deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-FC28 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-FC29 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-cern deleted: centos-mirror-tools/rpms_3rdparties.lst deleted: centos-mirror-tools/rpms_centos.lst deleted: centos-mirror-tools/rpms_centos3rdparties.lst deleted: centos-mirror-tools/tarball-dl.lst deleted: centos-mirror-tools/yum.conf.sample deleted: centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.2.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.3.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.4.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.5.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX-EPEL-7.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX_3rd_buildlogs-seed_centos_org.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX_3rd_buildlogs_centos_org.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX_3rd_download_docker_com.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX_3rd_epel_blizoo_mk.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX_3rd_ftp_iij_ad_jp.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX_3rd_linuxsoft_cern_ch.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX_3rd_www_rpmfind_net.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX_CentOS-OpenStack-queens.repo
On Tue, 2018-11-06 at 16:33 -0500, Scott Little wrote:
I can't reproduce this, with or without https://review.openstack.org/ #/c/615316
Where you in the container when you ran mirror_downloader.sh? In directory /localdisk?
Scott
On 18-11-06 11:51 AM, Martinez Landa, Hayde wrote:
Hi Ken!
We tried mirror_downloader.sh on our Jenkins automated environment and also on my local environment and when it finished running we noticed that the following files were moved:
this ones were sent to: “/localdisk/output/stx- r1/CentOS/pike/Binary/x86_64” -rpms_3rdparties.lst -rpms_centos.lst -rpms_centos3rdparties.lst -tarball-dl.lst Same with “yum.repos.d” were sent to: “CentOS/pike/Binary/noarch/” CentOS/pike/Binary/noarch/yum.repos.d/ CentOS/pike/Binary/noarch/yum.repos.d/StarlingX-Centos-7.2.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_buildlogs_cento s_org.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_download_docker _com.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_CentOS-OpenStack- queens.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_epel_blizoo_mk. repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_linuxsoft_cern_ ch.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX-Centos-7.3.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_buildlogs- seed_centos_org.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_www_rpmfind_net .repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX-Centos-7.4.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX-EPEL-7.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_ftp_iij_ad_jp.r epo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX-Centos-7.5.repo
And finally rpm-gpg-keys sent to: “CentOS/pike/Binary/x86_64” CentOS/pike/Binary/x86_64/rpm-gpg-keys/ CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-FC27-primary CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-SIG-SCLo CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-Debug-6 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-6 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-EPEL-6 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-Security- 6 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-SIG- OpsTools CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-FC29 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-FC27 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-EPEL-7 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-5 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-DOCKER CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-cern CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-SIG- Virtualization CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-FC27-second CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-FC28 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-Testing-6 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-Cloud-SIG
Is this the expected behavior?
I’m currently finishing the build with this mirror, will update once it’s done.
Best -Hayde
From: "Young, Ken" <Ken.Young@windriver.com> Date: Monday, November 5, 2018 at 2:43 PM To: "Lara, Cesar" <cesar.lara@intel.com> Cc: "starlingx-discuss@lists.starlingx.io" <starlingx-discuss@lists .starlingx.io> Subject: Re: [Starlingx-discuss] [build][meetings] Meeting minutes 11/1/2018
Cesar,
A quick update on the mirror. The tooling to enable the mirror was merged on Friday. The Wind River used the mirror over the week-end for its builds with no issues.
How have your builds gone since Friday?
/KenY
From: "Lara, Cesar" <cesar.lara@intel.com> Date: Thursday, November 1, 2018 at 6:58 PM To: "starlingx-discuss@lists.starlingx.io" <starlingx-discuss@lists .starlingx.io> Subject: [Starlingx-discuss] [build][meetings] Meeting minutes 11/1/2018
Build team meeting minutes 11/1/2018
Attendees Ken, Jason, Scott, Memo, Erich, Marcela, Victor, Chuy, Cesar, Luis
Cengn Status
-Mirror is now completed and operational, the wiki has been updated with the details for Cengn and our partnership, the details and specs are already there. -The scripts that are related with mirror download are now pointing to Cengn repos and are waiting in the queue to be reviewed tested and merged, the desired date for fully operational mirror is November 5th - Now changes to the packages required to build will be caught by Cengn mirror within one day, the scripts will try to download first from Cengn and if is still not available will try to go get the packages from upstream.
Koji follow up
-presentation on a possible scenario hosting Koji and demo on using Koji based built packages with our current build scripts. -discussion followed up by implementation details led to different points of view about the problem we have to solve if there’s any, the amount of effort to integrate tooling as we know Koji doesn’t solve every use case by itself and where to host a Koji service(we are striving to host a centralized service for the whole community). -The statement we are using to follow thru with Koji versus our current script is that we will require a lot of effort to support multi OS with the current tools.
We had to defer the topic of mirror script refactored using Python.
We will start a thread on the goals this team has for next release on the mailing list.
Regards
Cesar Lara Software Engineering Manager OpenSource Technology Center
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discus s
Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Can't reproduce with a fresh container and https://review.openstack.org/#/c/615316/. I'll try without the update. Scott On 18-11-07 10:43 AM, Scott Little wrote:
Ok, I'll repo sync and try a new Docker. I'll run these exact commands. My previous docker was over a month old. Perhaps the difference lies there.
For the first pass I'll run using the bug fix in https://review.openstack.org/#/c/615316/ . I would like your eyes on that review.
Scott
On 18-11-07 10:16 AM, Cordoba Malibran, Erich wrote:
We have seen this behavior in two different systems.
The script is run in this way:
cd stx-tools/centos-mirror-tools docker build -t stx-mirror -f Dockerfile . docker run -ti -v $(pwd):/localdisk stx-mirror ./download_mirror.sh
After run the script, this is what git reports:
On branch master Your branch is up to date with 'origin/master'.
Changes not staged for commit: (use "git add/rm <file>..." to update what will be committed) (use "git checkout -- <file>..." to discard changes in working directory)
deleted: centos-mirror-tools/make_stx_mirror_yum_conf.sh deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-CentOS-5 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-CentOS-6 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-CentOS-Debug-6 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-CentOS-SIG-OpsTools deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-CentOS-SIG-SCLo deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-CentOS-SIG-Virtualization deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-CentOS-Security-6 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-CentOS-Testing-6 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-Cloud-SIG deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-DOCKER deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-EPEL-6 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-EPEL-7 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-FC27 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-FC27-primary deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-FC27-second deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-FC28 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-FC29 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-cern deleted: centos-mirror-tools/rpms_3rdparties.lst deleted: centos-mirror-tools/rpms_centos.lst deleted: centos-mirror-tools/rpms_centos3rdparties.lst deleted: centos-mirror-tools/tarball-dl.lst deleted: centos-mirror-tools/yum.conf.sample deleted: centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.2.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.3.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.4.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.5.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX-EPEL-7.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX_3rd_buildlogs-seed_centos_org.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX_3rd_buildlogs_centos_org.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX_3rd_download_docker_com.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX_3rd_epel_blizoo_mk.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX_3rd_ftp_iij_ad_jp.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX_3rd_linuxsoft_cern_ch.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX_3rd_www_rpmfind_net.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX_CentOS-OpenStack-queens.repo
On Tue, 2018-11-06 at 16:33 -0500, Scott Little wrote:
I can't reproduce this, with or without https://review.openstack.org/ #/c/615316
Where you in the container when you ran mirror_downloader.sh? In directory /localdisk?
Scott
On 18-11-06 11:51 AM, Martinez Landa, Hayde wrote:
Hi Ken! We tried mirror_downloader.sh on our Jenkins automated environment and also on my local environment and when it finished running we noticed that the following files were moved: this ones were sent to: “/localdisk/output/stx- r1/CentOS/pike/Binary/x86_64” -rpms_3rdparties.lst -rpms_centos.lst -rpms_centos3rdparties.lst -tarball-dl.lst Same with “yum.repos.d” were sent to: “CentOS/pike/Binary/noarch/” CentOS/pike/Binary/noarch/yum.repos.d/ CentOS/pike/Binary/noarch/yum.repos.d/StarlingX-Centos-7.2.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_buildlogs_cento s_org.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_download_docker _com.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_CentOS-OpenStack- queens.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_epel_blizoo_mk. repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_linuxsoft_cern_ ch.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX-Centos-7.3.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_buildlogs- seed_centos_org.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_www_rpmfind_net .repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX-Centos-7.4.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX-EPEL-7.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_ftp_iij_ad_jp.r epo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX-Centos-7.5.repo And finally rpm-gpg-keys sent to: “CentOS/pike/Binary/x86_64” CentOS/pike/Binary/x86_64/rpm-gpg-keys/ CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-FC27-primary CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-SIG-SCLo CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-Debug-6 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-6 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-EPEL-6 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-Security- 6 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-SIG- OpsTools CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-FC29 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-FC27 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-EPEL-7 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-5 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-DOCKER CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-cern CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-SIG- Virtualization CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-FC27-second CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-FC28 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-Testing-6 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-Cloud-SIG Is this the expected behavior? I’m currently finishing the build with this mirror, will update once it’s done. Best -Hayde From: "Young, Ken" <Ken.Young@windriver.com> Date: Monday, November 5, 2018 at 2:43 PM To: "Lara, Cesar" <cesar.lara@intel.com> Cc: "starlingx-discuss@lists.starlingx.io" <starlingx-discuss@lists .starlingx.io> Subject: Re: [Starlingx-discuss] [build][meetings] Meeting minutes 11/1/2018 Cesar, A quick update on the mirror. The tooling to enable the mirror was merged on Friday. The Wind River used the mirror over the week-end for its builds with no issues. How have your builds gone since Friday? /KenY From: "Lara, Cesar" <cesar.lara@intel.com> Date: Thursday, November 1, 2018 at 6:58 PM To: "starlingx-discuss@lists.starlingx.io" <starlingx-discuss@lists .starlingx.io> Subject: [Starlingx-discuss] [build][meetings] Meeting minutes 11/1/2018 Build team meeting minutes 11/1/2018 Attendees Ken, Jason, Scott, Memo, Erich, Marcela, Victor, Chuy, Cesar, Luis Cengn Status -Mirror is now completed and operational, the wiki has been updated with the details for Cengn and our partnership, the details and specs are already there. -The scripts that are related with mirror download are now pointing to Cengn repos and are waiting in the queue to be reviewed tested and merged, the desired date for fully operational mirror is November 5th - Now changes to the packages required to build will be caught by Cengn mirror within one day, the scripts will try to download first from Cengn and if is still not available will try to go get the packages from upstream. Koji follow up -presentation on a possible scenario hosting Koji and demo on using Koji based built packages with our current build scripts. -discussion followed up by implementation details led to different points of view about the problem we have to solve if there’s any, the amount of effort to integrate tooling as we know Koji doesn’t solve every use case by itself and where to host a Koji service(we are striving to host a centralized service for the whole community). -The statement we are using to follow thru with Koji versus our current script is that we will require a lot of effort to support multi OS with the current tools. We had to defer the topic of mirror script refactored using Python. We will start a thread on the goals this team has for next release on the mailing list. Regards Cesar Lara Software Engineering Manager OpenSource Technology Center
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discus s
Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
I can reproduce without https://review.openstack.org/#/c/615316/. and no pre-existing output. So the fix is in covered by https://review.openstack.org/#/c/615316/ Scott On 18-11-07 01:39 PM, Scott Little wrote:
Can't reproduce with a fresh container and https://review.openstack.org/#/c/615316/. I'll try without the update.
Scott
On 18-11-07 10:43 AM, Scott Little wrote:
Ok, I'll repo sync and try a new Docker. I'll run these exact commands. My previous docker was over a month old. Perhaps the difference lies there.
For the first pass I'll run using the bug fix in https://review.openstack.org/#/c/615316/ . I would like your eyes on that review.
Scott
On 18-11-07 10:16 AM, Cordoba Malibran, Erich wrote:
We have seen this behavior in two different systems.
The script is run in this way:
cd stx-tools/centos-mirror-tools docker build -t stx-mirror -f Dockerfile . docker run -ti -v $(pwd):/localdisk stx-mirror ./download_mirror.sh
After run the script, this is what git reports:
On branch master Your branch is up to date with 'origin/master'.
Changes not staged for commit: (use "git add/rm <file>..." to update what will be committed) (use "git checkout -- <file>..." to discard changes in working directory)
deleted: centos-mirror-tools/make_stx_mirror_yum_conf.sh deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-CentOS-5 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-CentOS-6 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-CentOS-Debug-6 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-CentOS-SIG-OpsTools deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-CentOS-SIG-SCLo deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-CentOS-SIG-Virtualization deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-CentOS-Security-6 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-CentOS-Testing-6 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-Cloud-SIG deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-DOCKER deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-EPEL-6 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-EPEL-7 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-FC27 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-FC27-primary deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-FC27-second deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-FC28 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-FC29 deleted: centos-mirror-tools/rpm-gpg-keys/RPM-GPG-KEY-cern deleted: centos-mirror-tools/rpms_3rdparties.lst deleted: centos-mirror-tools/rpms_centos.lst deleted: centos-mirror-tools/rpms_centos3rdparties.lst deleted: centos-mirror-tools/tarball-dl.lst deleted: centos-mirror-tools/yum.conf.sample deleted: centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.2.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.3.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.4.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX-Centos-7.5.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX-EPEL-7.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX_3rd_buildlogs-seed_centos_org.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX_3rd_buildlogs_centos_org.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX_3rd_download_docker_com.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX_3rd_epel_blizoo_mk.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX_3rd_ftp_iij_ad_jp.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX_3rd_linuxsoft_cern_ch.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX_3rd_www_rpmfind_net.repo deleted: centos-mirror-tools/yum.repos.d/StarlingX_CentOS-OpenStack-queens.repo
On Tue, 2018-11-06 at 16:33 -0500, Scott Little wrote:
I can't reproduce this, with or without https://review.openstack.org/ #/c/615316
Where you in the container when you ran mirror_downloader.sh? In directory /localdisk?
Scott
On 18-11-06 11:51 AM, Martinez Landa, Hayde wrote:
Hi Ken! We tried mirror_downloader.sh on our Jenkins automated environment and also on my local environment and when it finished running we noticed that the following files were moved: this ones were sent to: “/localdisk/output/stx- r1/CentOS/pike/Binary/x86_64” -rpms_3rdparties.lst -rpms_centos.lst -rpms_centos3rdparties.lst -tarball-dl.lst Same with “yum.repos.d” were sent to: “CentOS/pike/Binary/noarch/” CentOS/pike/Binary/noarch/yum.repos.d/ CentOS/pike/Binary/noarch/yum.repos.d/StarlingX-Centos-7.2.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_buildlogs_cento s_org.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_download_docker _com.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_CentOS-OpenStack- queens.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_epel_blizoo_mk. repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_linuxsoft_cern_ ch.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX-Centos-7.3.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_buildlogs- seed_centos_org.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_www_rpmfind_net .repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX-Centos-7.4.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX-EPEL-7.repo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX_3rd_ftp_iij_ad_jp.r epo CentOS/pike/Binary/noarch/yum.repos.d/StarlingX-Centos-7.5.repo And finally rpm-gpg-keys sent to: “CentOS/pike/Binary/x86_64” CentOS/pike/Binary/x86_64/rpm-gpg-keys/ CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-FC27-primary CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-SIG-SCLo CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-Debug-6 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-6 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-EPEL-6 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-Security- 6 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-SIG- OpsTools CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-FC29 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-FC27 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-EPEL-7 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-5 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-DOCKER CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-cern CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-SIG- Virtualization CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-FC27-second CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-FC28 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-CentOS-Testing-6 CentOS/pike/Binary/x86_64/rpm-gpg-keys/RPM-GPG-KEY-Cloud-SIG Is this the expected behavior? I’m currently finishing the build with this mirror, will update once it’s done. Best -Hayde From: "Young, Ken" <Ken.Young@windriver.com> Date: Monday, November 5, 2018 at 2:43 PM To: "Lara, Cesar" <cesar.lara@intel.com> Cc: "starlingx-discuss@lists.starlingx.io" <starlingx-discuss@lists .starlingx.io> Subject: Re: [Starlingx-discuss] [build][meetings] Meeting minutes 11/1/2018 Cesar, A quick update on the mirror. The tooling to enable the mirror was merged on Friday. The Wind River used the mirror over the week-end for its builds with no issues. How have your builds gone since Friday? /KenY From: "Lara, Cesar" <cesar.lara@intel.com> Date: Thursday, November 1, 2018 at 6:58 PM To: "starlingx-discuss@lists.starlingx.io" <starlingx-discuss@lists .starlingx.io> Subject: [Starlingx-discuss] [build][meetings] Meeting minutes 11/1/2018 Build team meeting minutes 11/1/2018 Attendees Ken, Jason, Scott, Memo, Erich, Marcela, Victor, Chuy, Cesar, Luis Cengn Status -Mirror is now completed and operational, the wiki has been updated with the details for Cengn and our partnership, the details and specs are already there. -The scripts that are related with mirror download are now pointing to Cengn repos and are waiting in the queue to be reviewed tested and merged, the desired date for fully operational mirror is November 5th - Now changes to the packages required to build will be caught by Cengn mirror within one day, the scripts will try to download first from Cengn and if is still not available will try to go get the packages from upstream. Koji follow up -presentation on a possible scenario hosting Koji and demo on using Koji based built packages with our current build scripts. -discussion followed up by implementation details led to different points of view about the problem we have to solve if there’s any, the amount of effort to integrate tooling as we know Koji doesn’t solve every use case by itself and where to host a Koji service(we are striving to host a centralized service for the whole community). -The statement we are using to follow thru with Koji versus our current script is that we will require a lot of effort to support multi OS with the current tools. We had to defer the topic of mirror script refactored using Python. We will start a thread on the goals this team has for next release on the mailing list. Regards Cesar Lara Software Engineering Manager OpenSource Technology Center
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discus s
Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
participants (4)
-
Cordoba Malibran, Erich
-
Martinez Landa, Hayde
-
Scott Little
-
Young, Ken