[Starlingx-discuss] stx-tools: setup stx build environment
Hu, Yong
yong.hu at intel.com
Mon Nov 26 01:10:19 UTC 2018
Your no_proxy env setting should resolve the following error, are you still seeing it?
21:10:47 http://127.0.0.1:8088/localdisk/designer/nstcc1/starlingx/cgcs-root/cgcs-centos-repo/Binary/repodata/repomd.xml: [Errno 14] HTTP Error 503 - Service Unavailable
From: "Liu, Changcheng" <changcheng.liu at intel.com>
Date: Sunday, 25 November 2018 at 3:24 PM
To: Erich Cordoba <erich.cm.lists at yandex.com>, "'starlingx-discuss at lists.starlingx.io'" <starlingx-discuss at lists.starlingx.io>, "Cordoba Malibran, Erich" <erich.cordoba.malibran at intel.com>
Subject: Re: [Starlingx-discuss] stx-tools: setup stx build environment
Hi Erich,
Is there anything wrong with proxy env in container as below?
[nstcc1 at d8ae703a2a91 /]$ env | grep -i proxy
NO_PROXY=localhost,.intel.com,127.0.0.0/8,172,16.0.0/20,192.168.0.0/16,10.0.0.0/8
http_proxy=http://child-prc.intel.com:913
FTP_PROXY=ftp://child-prc.intel.com:913
ftp_proxy=ftp://child-prc.intel.com:913
https_proxy=http://child-prc.intel.com:913
HTTPS_PROXY=http://child-prc.intel.com:913
no_proxy=localhost,.intel.com,127.0.0.0/8,172,16.0.0/20,192.168.0.0/16,10.0.0.0/8
HTTP_PROXY=http://child-prc.intel.com:913
B.R.
Changcheng
From: Erich Cordoba [mailto:erich.cm.lists at yandex.com]
Sent: Friday, November 23, 2018 9:37 PM
To: 'starlingx-discuss at lists.starlingx.io' <starlingx-discuss at lists.starlingx.io>; Cordoba Malibran, Erich <erich.cordoba.malibran at intel.com>; Liu, Changcheng <changcheng.liu at intel.com>
Subject: Re: [Starlingx-discuss] stx-tools: setup stx build environment
Hi Liu,
As you need to setup a proxy, you need also to set the no_proxy=127.0.0.1 env variable, so mock can find the internal repository.
This is something that we need to do in our internal build job.
-Erich
--
Sent from Yandex.Mail for mobile
23.11.2018, 07:27, "Liu, Changcheng" <changcheng.liu at intel.com<mailto:changcheng.liu at intel.com>>:
Hi Erich,
I hit lots of error when executing “build-pkgs”. Please help check it.
https://bugs.launchpad.net/starlingx/+bug/1804412
1. get stx-tools
cd $HOME
git clone https://git.starlingx.io/stx-tools
cd stx-tools
git checkout -b erich_base 1d2ecbb0be2
2. apply below patch
nstcc1 at nstcloudcc1:stx-tools$ git diff Dockerfile tb.sh Makefile
diff --git a/Dockerfile b/Dockerfile
index 87ef00b..dd730f9 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -13,9 +13,9 @@
FROM centos:7.4.1708
# Proxy configuration
-#ENV http_proxy "http://your.actual_http_proxy.com<http://your.actual_http_proxy.com/>:your_port"
-#ENV https_proxy "https://your.actual_https_proxy.com<https://your.actual_https_proxy.com/>:your_port"
-#ENV ftp_proxy "http://your.actual_ftp_proxy.com<http://your.actual_ftp_proxy.com/>:your_port"
+ENV http_proxy "http://child-prc.intel.com:913<http://child-prc.intel.com:913/>"
+ENV https_proxy "http://child-prc.intel.com:913<http://child-prc.intel.com:913/>"
+ENV ftp_proxy "ftp://child-prc.intel.com:913<ftp://child-prc.intel.com:913/>"
# username you will docker exec into the container as.
@@ -26,6 +26,8 @@ ARG MYUID=1000
ENV container=docker
+RUN echo "proxy=$http_proxy" >> /etc/yum.conf && \
+echo -e "export http_proxy=$http_proxy\nexport https_proxy=$https_proxy\nexport ftp_proxy=$ftp_proxy" >> /root/.bashrc
# Download required dependencies by mirror/build processes.
# Notice there are 3 invocations to yum package manage.
# 1) Enable EPEL repository.
diff --git a/Makefile b/Makefile
index c4ebff4..73f5f3a 100644
--- a/Makefile
+++ b/Makefile
@@ -20,6 +20,8 @@ all:
docker build \
--build-arg MYUID=$(UID) \
--build-arg MYUNAME=$(MYUNAME) \
+ --build-arg https_proxy=http://child-prc.intel.com:913<http://child-prc.intel.com:913/> \
+ --build-arg http_proxy=http://child-prc.intel.com:913<http://child-prc.intel.com:913/> \
--ulimit core=0 \
--network host \
-t $(TC_CONTAINER_TAG) \
diff --git a/tb.sh b/tb.sh
index e0508dd..9530efd 100755
--- a/tb.sh
+++ b/tb.sh
@@ -38,11 +38,14 @@ function run_container {
docker run -it --rm \
--name ${TC_CONTAINER_NAME} \
--detach \
+ -v /etc/localtime:/etc/localtime:ro \
-v ${LOCALDISK}:/${GUEST_LOCALDISK} \
-v ${HOST_MIRROR_DIR}:/import/mirrors:ro \
-v /sys/fs/cgroup:/sys/fs/cgroup:ro \
-v ~/.ssh:/mySSH:ro \
-e "container=docker" \
+ --env https_proxy=http://child-prc.intel.com:913<http://child-prc.intel.com:913/> \
+ --env http_proxy=http://child-prc.intel.com:913<http://child-prc.intel.com:913/> \
-e MYUNAME=${MYUNAME} \
--privileged=true \
--security-opt seccomp=unconfined \
3. build docker image
make all
4. run docker image
bash tb.sh run
5. execute into docker container
bash tb.sh exec
6.
[nstcc1 at d8ae703a2a91 /]$ eval $(ssh-agent)
Agent pid 15524
[nstcc1 at d8ae703a2a91 /]$ ssh-add
Identity added: /home/nstcc1/.ssh/id_rsa (/home/nstcc1/.ssh/id_rsa)
[nstcc1 at d8ae703a2a91 /]$ cd $MY_REPO_ROOT_DIR
[nstcc1 at d8ae703a2a91 starlingx]$ repo init -u https://git.starlingx.io/stx-manifest -m default.xml
[nstcc1 at d8ae703a2a91 starlingx]$ repo sync -j4
[nstcc1 at d8ae703a2a91 starlingx]$ ln -s /import/mirrors/CentOS/stx-r1/CentOS/pike/downloads/ $MY_REPO/stx/
[nstcc1 at d8ae703a2a91 starlingx]$ populate_downloads.sh /import/mirrors/CentOS/stx-r1/CentOS/pike/
7.
nstcc1 at nstcloudcc1:~$ mkdir -p $HOME/starlingx/mirror/CentOS/tis-installer
nstcc1 at nstcloudcc1:~$ cp $HOME/starlingx/mirror/CentOS/stx-r1/CentOS/pike/Binary/images/pxeboot/initrd.img $HOME/starlingx/mirror/CentOS/tis-installer/initrd.img-stx-0.2
nstcc1 at nstcloudcc1:~$ cp $HOME/starlingx/mirror/CentOS/stx-r1/CentOS/pike/Binary/images/pxeboot/vmlinuz $HOME/starlingx/mirror/CentOS/tis-installer/vmlinuz-stx-0.2
nstcc1 at nstcloudcc1:~$ cp $HOME/starlingx/mirror/CentOS/stx-r1/CentOS/pike/Binary/LiveOS/squashfs.img $HOME/starlingx/mirror/CentOS/tis-installer/squashfs.img-stx-0.2
8.
[nstcc1 at d8ae703a2a91 starlingx]$ generate-cgcs-centos-repo.sh /import/mirrors/CentOS/stx-r1/CentOS/pike/
9.
[nstcc1 at d8ae703a2a91 starlingx]$ build-pkgs
Then, hit lots of error, the log is build-pkgs.log
21:10:47 failure: repodata/repomd.xml from TisCentos7Distro: [Errno 256] No more mirrors to try.
21:10:47 http://127.0.0.1:8088/localdisk/designer/nstcc1/starlingx/cgcs-root/cgcs-centos-repo/Binary/repodata/repomd.xml: [Errno 14] HTTP Error 503 - Service Unavailable
21:10:47 http://127.0.0.1:8088/localdisk/designer/nstcc1/starlingx/cgcs-root/cgcs-centos-repo/Binary/repodata/repomd.xml: [Errno 14] HTTP Error 503 - Service Unavailable
21:10:47 http://127.0.0.1:8088/localdisk/designer/nstcc1/starlingx/cgcs-root/cgcs-centos-repo/Binary/repodata/repomd.xml: [Errno 14] HTTP Error 503 - Service Unavailable
21:10:47 http://127.0.0.1:8088/localdisk/designer/nstcc1/starlingx/cgcs-root/cgcs-centos-repo/Binary/repodata/repomd.xml: [Errno 14] HTTP Error 503 - Service Unavailable
21:10:47 http://127.0.0.1:8088/localdisk/designer/nstcc1/starlingx/cgcs-root/cgcs-centos-repo/Binary/repodata/repomd.xml: [Errno 14] HTTP Error 503 - Service Unavailable
21:10:47 http://127.0.0.1:8088/localdisk/designer/nstcc1/starlingx/cgcs-root/cgcs-centos-repo/Binary/repodata/repomd.xml: [Errno 14] HTTP Error 503 - Service Unavailable
21:10:47 http://127.0.0.1:8088/localdisk/designer/nstcc1/starlingx/cgcs-root/cgcs-centos-repo/Binary/repodata/repomd.xml: [Errno 14] HTTP Error 503 - Service Unavailable
21:10:47 http://127.0.0.1:8088/localdisk/designer/nstcc1/starlingx/cgcs-root/cgcs-centos-repo/Binary/repodata/repomd.xml: [Errno 14] HTTP Error 503 - Service Unavailable
21:10:47 http://127.0.0.1:8088/localdisk/designer/nstcc1/starlingx/cgcs-root/cgcs-centos-repo/Binary/repodata/repomd.xml: [Errno 14] HTTP Error 503 - Service Unavailable
21:10:47 http://127.0.0.1:8088/localdisk/designer/nstcc1/starlingx/cgcs-root/cgcs-centos-repo/Binary/repodata/repomd.xml: [Errno 14] HTTP Error 503 - Service Unavailable
21:10:47 http://127.0.0.1:8088/localdisk/designer/nstcc1/starlingx/cgcs-root/cgcs-centos-repo/Binary/repodata/repomd.xml: [Errno 14] HTTP Error 503 - Service Unavailable
21:10:47 http://127.0.0.1:8088/localdisk/designer/nstcc1/starlingx/cgcs-root/cgcs-centos-repo/Binary/repodata/repomd.xml: [Errno 14] HTTP Error 503 - Service Unavailable
21:10:47 http://127.0.0.1:8088/localdisk/designer/nstcc1/starlingx/cgcs-root/cgcs-centos-repo/Binary/repodata/repomd.xml: [Errno 14] HTTP Error 503 - Service Unavailable
21:10:47 http://127.0.0.1:8088/localdisk/designer/nstcc1/starlingx/cgcs-root/cgcs-centos-repo/Binary/repodata/repomd.xml: [Errno 14] HTTP Error 503 - Service Unavailable
21:10:47 http://127.0.0.1:8088/localdisk/designer/nstcc1/starlingx/cgcs-root/cgcs-centos-repo/Binary/repodata/repomd.xml: [Errno 14] HTTP Error 503 - Service Unavailable
21:10:47 http://127.0.0.1:8088/localdisk/designer/nstcc1/starlingx/cgcs-root/cgcs-centos-repo/Binary/repodata/repomd.xml: [Errno 14] HTTP Error 503 - Service Unavailable
21:10:47 http://127.0.0.1:8088/localdisk/designer/nstcc1/starlingx/cgcs-root/cgcs-centos-repo/Binary/repodata/repomd.xml: [Errno 14] HTTP Error 503 - Service Unavailable
21:10:47 http://127.0.0.1:8088/localdisk/designer/nstcc1/starlingx/cgcs-root/cgcs-centos-repo/Binary/repodata/repomd.xml: [Errno 14] HTTP Error 503 - Service Unavailable
21:10:47 http://127.0.0.1:8088/localdisk/designer/nstcc1/starlingx/cgcs-root/cgcs-centos-repo/Binary/repodata/repomd.xml: [Errno 14] HTTP Error 503 - Service Unavailable
21:10:47 http://127.0.0.1:8088/localdisk/designer/nstcc1/starlingx/cgcs-root/cgcs-centos-repo/Binary/repodata/repomd.xml: [Errno 14] HTTP Error 503 - Service Unavailable
21:10:47
21:10:47 End build on 'b0': /localdisk/loadbuild/nstcc1/starlingx/std/rpmbuild/SRPMS/mariadb-10.1.28-1.el7.tis.15.src.rpm
21:10:47 Error building mariadb-10.1.28-1.el7.tis.15.src.rpm on 'b0'.
--Thanks
Changcheng
From: Liu, Changcheng
Sent: Friday, November 23, 2018 12:16 AM
To: 'starlingx-discuss at lists.starlingx.io'<mailto:'starlingx-discuss at lists.starlingx.io'> <starlingx-discuss at lists.starlingx.io<mailto:starlingx-discuss at lists.starlingx.io>>; Cordoba Malibran, Erich <erich.cordoba.malibran at intel.com<mailto:erich.cordoba.malibran at intel.com>>
Subject: RE: stx-tools: setup stx build environment
Hi Erich,
I still met with some problems after following your method to run “download_mirror.sh”.
Please check the comment in https://bugs.launchpad.net/starlingx/+bug/1804412
[root at 90104f541da1 logs]# wc -l *
0 K1_failmoved_url_centos.log
0 K1_rpms_found_centos.log
0 K1_rpms_missing_centos.log
0 K1_srpms_found_centos.log
0 K1_srpms_missing_centos.log
46 L1_failmoved_url_3rdparties.log
5306 L1_failmoved_url_centos.log
394 L1_failmoved_url_centos3rdparties.log
0 L1_rpms_found_3rdparties.log
0 L1_rpms_found_centos.log
0 L1_rpms_found_centos3rdparties.log
0 L1_rpms_missing_3rdparties.log
0 L1_rpms_missing_centos.log
0 L1_rpms_missing_centos3rdparties.log
23 L1_srpms_found_3rdparties.log
1936 L1_srpms_found_centos.log
148 L1_srpms_found_centos3rdparties.log
0 L1_srpms_missing_3rdparties.log
0 L1_srpms_missing_centos.log
0 L1_srpms_missing_centos3rdparties.log
590 all_i686.txt
9858 otherfiles_centos_download.log
1 rpm-gpg-key-missing.txt
80 rpms_3rdparties_download_L1.log
490 rpms_centos3rdparties_download_L1.log
2 rpms_centos_download_K1.log
6735 rpms_centos_download_L1.log
57768 tarballs_download.log
83377 total
[root at 90104f541da1 logs]#
I don’t know whether the “failmoved” packages will affect the ISO image build or not.
I’m building another docker image for building ISO image.
--Changcheng
From: Liu, Changcheng
Sent: Wednesday, November 21, 2018 11:24 PM
To: 'starlingx-discuss at lists.starlingx.io'<mailto:'starlingx-discuss at lists.starlingx.io'> <starlingx-discuss at lists.starlingx.io<mailto:starlingx-discuss at lists.starlingx.io>>; Cordoba Malibran, Erich <erich.cordoba.malibran at intel.com<mailto:erich.cordoba.malibran at intel.com>>
Subject: RE: stx-tools: setup stx build environment
Hi Erich,
Please help check the problem. Tell me if you need more information.
I’m trying to run “download_mirror.sh” again.
B.R.
Changcheng
From: Liu, Changcheng
Sent: Wednesday, November 21, 2018 6:35 PM
To: 'starlingx-discuss at lists.starlingx.io'<mailto:'starlingx-discuss at lists.starlingx.io'> <starlingx-discuss at lists.starlingx.io<mailto:starlingx-discuss at lists.starlingx.io>>
Subject: stx-tools: setup stx build environment
Could someone help check below bug?
Title: stx-tools: failed to download required packaged
Link: https://bugs.launchpad.net/starlingx/+bug/1804412
B.R.
Changcheng
From: Liu, Changcheng
Sent: Friday, November 16, 2018 8:31 PM
To: starlingx-discuss at lists.starlingx.io<mailto:starlingx-discuss at lists.starlingx.io>
Subject: setup stx build environment
Hi all,
I have one question about setting up stx build environment.
Why not move lots of many basic program installation process from mirror-check.sh to Dockerfile directly?
Then stx community could supply one basic docker image as the base image to let developer customize it. Currently, I still need install lots of program after building Dockerfile into image.
B.R.
Changcheng
_______________________________________________
Starlingx-discuss mailing list
Starlingx-discuss at lists.starlingx.io<mailto:Starlingx-discuss at lists.starlingx.io>
http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20181126/b5cdbb2d/attachment-0001.html>
More information about the Starlingx-discuss
mailing list