[Starlingx-discuss] StarlingX CI/CD setup on Arm platform
Scott Little
scott.little at windriver.com
Wed Aug 30 20:20:02 UTC 2023
The Jenkins setup at https://github.com/starlingx-staging/jenkins has
99% of what is needed to populate a StarlingX mirror.
The Jenkins jobs assume that writing to /export/mirror/ is populating
https://mirror.starlingx.windriver.com/mirror/ ... formerly
http://mirror.starlingx.cengn.ca/mirror/ .
The webserver imports the same filesystem as /starlingx/mirror/
The webserver nginx config is ... /etc/nginx/sites-available/default
server {
listen 80;
listen 443 ssl default_server;
listen [::]:443 ssl default_server;
server_name mirror.starlingx.windriver.com;
ssl_certificate /etc/ssl/certs/mirror_starlingx_windriver_com-full.crt;
ssl_certificate_key
/etc/ssl/private/mirror_starlingx_windriver_com.key;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
ssl_ciphers HIGH:!aNULL:!MD5;
root /var/www/html;
index index.html index.htm index.nginx-debian.html;
location / {
try_files $uri $uri/ =404;
}
location /mirror/ {
alias /starlingx/mirror/;
autoindex on;
autoindex_exact_size off;
autoindex_format html;
autoindex_localtime off;
}
}
You'll need to substitute your own server name and ssl certificate.
Tracking down and substituting all the references to
mirror.starlingx.cengn.ca in the StarlingX source code is the trick. I
need to do something similar to replace mirror.starlingx.cengn.ca with
mirror.starlingx.windriver.com.
Attached is a tarfile with the content of /export/config/debian/
It drives which debian repo's are mirrored. You'll need to adapt it for
arm.
The last piece you need is a /export/config/debian/mirrorkeyring for the
keys validate the debian sources you are pulling down.
Scott
On 2023-08-24 08:57, Huang, Jie (Jackie) wrote:
>
> Hi Scott / Davlet,
>
> As we discussed on the OS Distro call, a local CENGEN like server is
> needed for the local CI/CD setup,
>
> Could you please share the info of how to setup a CENGEN like server,
> specifically how to prepare the
>
> package mirror, we need to get it setup and prepare the mirror for ARM
> deb packages, and for the
>
> LAT-SDK for ARM as well. Thanks!
>
> Thanks,
>
> Jackie
>
> *From:*Huang, Jie (Jackie) <Jackie.Huang at windriver.com>
> *Sent:* Friday, August 18, 2023 1:33 PM
> *To:* Little, Scott <Scott.Little at windriver.com>; Panech, Davlet
> <Davlet.Panech at windriver.com>; Tianyu Li <Tianyu.Li at arm.com>
> *Cc:* starlingx-discuss at lists.starlingx.io; Scott Kamp
> <scott.kamp at xunison.com>; Trevor Tao <Trevor.Tao at arm.com>; Geary,
> Stephen (Steve) <Steve.Geary at windriver.com>; Wang, Linda
> <Linda.Wang at windriver.com>
> *Subject:* Re: [Starlingx-discuss] StarlingX CI/CD setup on Arm platform
>
> Hi Scott,
>
> Thanks for sharing, it’s very helpful.
>
> Thanks,
>
> Jackie
>
> *From:*Little, Scott <Scott.Little at windriver.com>
> *Sent:* Thursday, August 17, 2023 11:48 PM
> *To:* Huang, Jie (Jackie) <Jackie.Huang at windriver.com>; Panech, Davlet
> <Davlet.Panech at windriver.com>; Tianyu Li <Tianyu.Li at arm.com>
> *Cc:* starlingx-discuss at lists.starlingx.io; Scott Kamp
> <scott.kamp at xunison.com>; Trevor Tao <Trevor.Tao at arm.com>; Geary,
> Stephen (Steve) <Steve.Geary at windriver.com>; Wang, Linda
> <Linda.Wang at windriver.com>
> *Subject:* Re: StarlingX CI/CD setup on Arm platform
>
> There is also ...
>
> https://github.com/starlingx-staging/jenkins
> <https://urldefense.com/v3/__https:/github.com/starlingx-staging/jenkins__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6exWFUmA$>
>
> Which contains the current state of the StarlingX jenkins setup, less
> a few credentials.
>
> It includes 'Debian_build_server.txt' with crude documentation of how
> the server was setup.
>
> Scott
>
> On 2023-08-16 21:04, Huang, Jie (Jackie) wrote:
>
> Hi Davlet,
>
> Thanks for the detailed answers, which are very helpful, Tianyu
> and I will start by familiarizing ourselves and check if there are
> any more questions.
>
> Thanks,
>
> Jackie
>
> *From:*Panech, Davlet <Davlet.Panech at windriver.com>
> <mailto:Davlet.Panech at windriver.com>
> *Sent:* Wednesday, August 16, 2023 11:30 PM
> *To:* Huang, Jie (Jackie) <Jackie.Huang at windriver.com>
> <mailto:Jackie.Huang at windriver.com>; Little, Scott
> <Scott.Little at windriver.com> <mailto:Scott.Little at windriver.com>;
> Tianyu Li <Tianyu.Li at arm.com> <mailto:Tianyu.Li at arm.com>; Wang,
> Linda <Linda.Wang at windriver.com> <mailto:Linda.Wang at windriver.com>
> *Cc:* starlingx-discuss at lists.starlingx.io; Scott Kamp
> <scott.kamp at xunison.com> <mailto:scott.kamp at xunison.com>; Trevor
> Tao <Trevor.Tao at arm.com> <mailto:Trevor.Tao at arm.com>; Geary,
> Stephen (Steve) <Steve.Geary at windriver.com>
> <mailto:Steve.Geary at windriver.com>
> *Subject:* Re: StarlingX CI/CD setup on Arm platform
>
> Hi Jackie,
>
> CI/CD can be thought of as 2 somewhat disconnected parts - build &
> automated testing. I know little about test automation, the
> current build set up is outlined below.
>
> You would need a dedicated server:
>
> 1.Linux : we currently use Debian 11, but it shouldn't matter
> since most build steps run in docker containers
>
> 2.Docker
>
> 3.Kubernetes
>
> 4.Jenkins: integrated with k8s and able to run k8s pipelines
>
> 5.git
>
> 6.git-repo https://gerrit.googlesource.com/git-repo
> <https://urldefense.com/v3/__https:/gerrit.googlesource.com/git-repo__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6Y5TFnTg$>
>
> Other prerequisites:
>
> 1.You would need some DockerHub account to avoid DockerHub's rate
> limits when pulling images
>
> 2.Once your ARM set up is ready we can replace that with a
> controlled account that we manage in order to push the images to
> Docker Hub.
>
> Jenkins job definitions (Jenkinsfiles) are source-controlled here:
> https://opendev.org/starlingx/jenkins-pipelines
> <https://urldefense.com/v3/__https:/opendev.org/starlingx/jenkins-pipelines__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6EmOjdNg$>
> . You would need to define Jenkins jobs manually in the GUI, and
> point them to the corresponding Jenkinsfiles in this git repo.
>
> 1.The repo itself acts as a pipeline library in Jenkins with
> shared functions under vars/
> <https://urldefense.com/v3/__https:/opendev.org/starlingx/jenkins-pipelines/src/branch/master/vars__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6wk4ZyVg$>
> . The library must be defined at Jenkins' folder level.
>
> 2.pipelines/monolithic.Jenkinsfile
> <https://urldefense.com/v3/__https:/opendev.org/starlingx/jenkins-pipelines/src/branch/master/pipelines/monolithic.Jenkinsfile__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6qaNL9Is$>
> : top-level job that calls other jobs under parts/
>
> 3.pipelines/parts/
> <https://urldefense.com/v3/__https:/opendev.org/starlingx/jenkins-pipelines/src/branch/master/pipelines/parts__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp616q0Kfk$>
> : directory containing sub-job Jenkinsfiles. You would create a
> Jenkins folder in the GUI called "parts" and define individual
> jobs pointing to individual Jenkinsfiles from this subdirectory in
> git repo
>
> 4.scripts/
> <https://urldefense.com/v3/__https:/opendev.org/starlingx/jenkins-pipelines/src/branch/master/scripts__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6mfhen9o$>
> : shell scripts that implement individual build steps (called by
> Jenkinsfiles)
>
> Required directories on the server, owned by jenkins user:
>
> 1./localdisk/designer/jenkins/ root for per-project build areas
>
> 2./localdisk/loadbuild/jenkins/ root for per-project build archives
>
> At first execution the monolithic job will fail, but create an
> example configuration file in the project build area that you are
> expected to copy & edit:
>
> 1./localdisk/designer/jenkins/$PROJECT/build.cond.example
>
> There are many options with comments in it; the example file is
> also source controlled under
> scripts/templates/build.conf.example.in
> <https://urldefense.com/v3/__https:/opendev.org/starlingx/jenkins-pipelines/src/branch/master/scripts/templates/build.conf.example.in__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6S_cks2U$>
>
> Builds are slow and would greatly benefit from fast storage (SSDs
> etc).
>
> Please start by familiarizing yourself with this repository and
> ping me if you have any more questions.
>
> Hope this helps,
>
> D.
>
> ------------------------------------------------------------------------
>
> *From:*Huang, Jie (Jackie) <Jackie.Huang at windriver.com>
> *Sent:* August 9, 2023 10:28 AM
> *To:* Little, Scott <Scott.Little at windriver.com>; Panech, Davlet
> <Davlet.Panech at windriver.com>; Tianyu Li <Tianyu.Li at arm.com>;
> Wang, Linda <Linda.Wang at windriver.com>
> *Cc:* starlingx-discuss at lists.starlingx.io
> <starlingx-discuss at lists.starlingx.io>; Scott Kamp
> <scott.kamp at xunison.com>; Trevor Tao <Trevor.Tao at arm.com>; Geary,
> Stephen (Steve) <Steve.Geary at windriver.com>
> *Subject:* RE: StarlingX CI/CD setup on Arm platform
>
> Hi Scott and Davlet,
>
> As mentioned in the Multi-OS call, I’m re-sending this with all
> the questions about StarlingX CI/CD, please help answer as many as
> you can, thanks!
>
> There would be 6 Arm servers contributed to our community for
> CI/CD (2 for build, 4 for testing of different deployment
> configs), what we want to know is:
>
> 1.What’s the current status of StarlingX CICD for x86-64?
>
> 2.What needs to be done for the CICD workflow setup and how after
> the 6 ARM servers are ready and hosted in the lab?
>
> a.E.g. how to setup build servers and testing env?
>
> b.What CICD tools need to be setup? Zuul/Jenkins/etc.
>
> c.Are there any docs or info of the steps for these setups?
>
> 3.How many efforts and how long will it take?
>
> 4.Can this only be done by specific person? Or anyone have
> permission to access to the servers? If the later, I think Tianyu
> from Arm team and I can also help
>
> on that if we got enough information.
>
> Tianyu, please add if you have any other questions.
>
> Thanks,
>
> Jackie
>
> *From:*Huang, Jie (Jackie)
> *Sent:* Monday, July 31, 2023 12:57 PM
> *To:* Trevor Tao <Trevor.Tao at arm.com>; Geary, Stephen (Steve)
> <Steve.Geary at windriver.com>; Little, Scott
> <Scott.Little at windriver.com>; Scott Kamp <scott.kamp at xunison.com>
> *Cc:* starlingx-discuss at lists.starlingx.io
> *Subject:* RE: StarlingX CI/CD setup on Arm platform
>
> Hi, Scott Kamp and Scott Little,
>
> I found the following actions in the meeting minutes in stx-status
> | OpenDev Etherpad
> <https://urldefense.com/v3/__https:/etherpad.opendev.org/p/stx-status__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6bH3qX9g$>:
>
> oARM Support
>
> §_Action:_ Scott Kamp to respond to Jackie/the mailing list to
> provide assistance/access to some arm machines
>
> §_Action:_ Scott Little to respond regarding build questions
>
> ·Cannot mirror more files on CENGN; will need to wait until we
> transition away from CENGN
>
> ·_Action:_ Scott Kamp to explore providing a hosting env temporarily
>
> So I think you might be able to help on Trevor’s and my questions:
>
> There would be 6 Arm servers contributed to our community for
> CI/CD (2 for build, 4 for testing of different deployment
> configs), what we want to know is:
>
> 1.What needs to be done for the CICD workflow setup and how?
>
> a.E.g. how to setup build servers and testing env?
>
> b.What CICD tools need to be setup? Zuul/Jenkins/etc.
>
> c.Are there any docs for these setups?
>
> 2.How many efforts and how long will it take?
>
> 3.Is there a CICD team or someone that can help on the setup or
> just providing useful information?
>
> Thanks,
>
> Jackie
>
> *From:*Trevor Tao <Trevor.Tao at arm.com>
> *Sent:* Thursday, July 27, 2023 5:06 PM
> *To:* Geary, Stephen (Steve) <Steve.Geary at windriver.com>;
> starlingx-discuss at lists.starlingx.io
> *Cc:* Huang, Jie (Jackie) <Jackie.Huang at windriver.com>
> *Subject:* StarlingX CI/CD setup on Arm platform
>
> Hi Steve, StarlingX:
>
> From the OS distro meeting
> <https://urldefense.com/v3/__https:/etherpad.opendev.org/p/stx-distro-other__;!!AjveYdw8EvQ!e07vSdClUS6f5DohQ4EQeAEzsC4SnjZH2Vb3mjeau1FjCuAnkaGqf1_1vuARZRPcPxdw8dZJN-D3pMmWKHLvhCs$>
> yesterday, we knew that there would be 6 Arm servers contributed
> to our community to do the CI/CD related work.
>
> So we would like to ask who can help on the setup of CI/CD
> environment on the arm platform, and any other useful
> information/details for the CI/CD issue here.
>
> Thanks,
>
> Best Regards,
>
> Zijin Tao(Trevor Tao, 陶孜谨)
>
> ARM Electronic Technology (Shanghai) Co., Ltd
>
> 安谋电子科技（上海）有限公司
>
> Building 11, Shanghai Busininess ParkⅢ,
>
> No.1016 Tianlin Rd, Minhang District, Shanghai, 200233 China
>
> 上海市闵行区田林路1016号科技绿洲三期2号楼10楼，200233
>
> Cell: +86-153 7109 6192
>
> IMPORTANT NOTICE: The contents of this email and any attachments
> are confidential and may also be privileged. If you are not the
> intended recipient, please notify the sender immediately and do
> not disclose the contents to any other person, use it for any
> purpose, or store or copy the information in any medium. Thank you.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20230830/65eadc7a/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mirror.cfg.tgz
Type: application/x-compressed-tar
Size: 754 bytes
Desc: not available
URL: <https://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20230830/65eadc7a/attachment-0001.bin>
More information about the Starlingx-discuss
mailing list