Hi Scott,
Thanks for sharing, it’s very helpful.
Thanks,
Jackie
From: Little, Scott <Scott.Little@windriver.com>
Sent: Thursday, August 17, 2023 11:48 PM
To: Huang, Jie (Jackie) <Jackie.Huang@windriver.com>; Panech, Davlet <Davlet.Panech@windriver.com>; Tianyu Li <Tianyu.Li@arm.com>
Cc: starlingx-discuss@lists.starlingx.io; Scott Kamp <scott.kamp@xunison.com>; Trevor Tao <Trevor.Tao@arm.com>; Geary, Stephen (Steve) <Steve.Geary@windriver.com>; Wang, Linda <Linda.Wang@windriver.com>
Subject: Re: StarlingX CI/CD setup on Arm platform
There is also ...
Which contains the current state of the StarlingX jenkins setup, less a few credentials.
It includes 'Debian_build_server.txt' with crude documentation of how the server was setup.
Scott
On 2023-08-16 21:04, Huang, Jie (Jackie) wrote:
Hi Davlet,
Thanks for the detailed answers, which are very helpful, Tianyu and I will start by familiarizing ourselves and check if there are any more questions.
Thanks,
Jackie
From: Panech, Davlet <Davlet.Panech@windriver.com>
Sent: Wednesday, August 16, 2023 11:30 PM
To: Huang, Jie (Jackie) <Jackie.Huang@windriver.com>; Little, Scott <Scott.Little@windriver.com>; Tianyu Li <Tianyu.Li@arm.com>; Wang, Linda <Linda.Wang@windriver.com>
Cc: starlingx-discuss@lists.starlingx.io; Scott Kamp <scott.kamp@xunison.com>; Trevor Tao <Trevor.Tao@arm.com>; Geary, Stephen (Steve) <Steve.Geary@windriver.com>
Subject: Re: StarlingX CI/CD setup on Arm platform
Hi Jackie,
CI/CD can be thought of as 2 somewhat disconnected parts - build & automated testing. I know little about test automation, the current build set up is outlined below.
You would need a dedicated server:
1. Linux : we currently use Debian 11, but it shouldn't matter since most build steps run in docker containers
2. Docker
3. Kubernetes
4. Jenkins: integrated with k8s and able to run k8s pipelines
5. git
6. git-repo https://gerrit.googlesource.com/git-repo
Other prerequisites:
1. You would need some DockerHub account to avoid DockerHub's rate limits when pulling images
2. Once your ARM set up is ready we can replace that with a controlled account that we manage in order to push the images to Docker Hub.
Jenkins job definitions (Jenkinsfiles) are source-controlled here: https://opendev.org/starlingx/jenkins-pipelines . You would need to define Jenkins jobs manually in the GUI, and point them to the corresponding Jenkinsfiles in this git repo.
1. The repo itself acts as a pipeline library in Jenkins with shared functions under vars/ . The library must be defined at Jenkins' folder level.
2. pipelines/monolithic.Jenkinsfile : top-level job that calls other jobs under parts/
3. pipelines/parts/ : directory containing sub-job Jenkinsfiles. You would create a Jenkins folder in the GUI called "parts" and define individual jobs pointing to individual Jenkinsfiles from this subdirectory in git repo
4. scripts/ : shell scripts that implement individual build steps (called by Jenkinsfiles)
Required directories on the server, owned by jenkins user:
1. /localdisk/designer/jenkins/ root for per-project build areas
2. /localdisk/loadbuild/jenkins/ root for per-project build archives
At first execution the monolithic job will fail, but create an example configuration file in the project build area that you are expected to copy & edit:
1. /localdisk/designer/jenkins/$PROJECT/build.cond.example
There are many options with comments in it; the example file is also source controlled under scripts/templates/build.conf.example.in
Builds are slow and would greatly benefit from fast storage (SSDs etc).
Please start by familiarizing yourself with this repository and ping me if you have any more questions.
Hope this helps,
D.
From: Huang, Jie (Jackie) <Jackie.Huang@windriver.com>
Sent: August 9, 2023 10:28 AM
To: Little, Scott <Scott.Little@windriver.com>; Panech, Davlet <Davlet.Panech@windriver.com>; Tianyu Li <Tianyu.Li@arm.com>; Wang, Linda <Linda.Wang@windriver.com>
Cc: starlingx-discuss@lists.starlingx.io <starlingx-discuss@lists.starlingx.io>; Scott Kamp <scott.kamp@xunison.com>; Trevor Tao <Trevor.Tao@arm.com>; Geary, Stephen (Steve) <Steve.Geary@windriver.com>
Subject: RE: StarlingX CI/CD setup on Arm platform
Hi Scott and Davlet,
As mentioned in the Multi-OS call, I’m re-sending this with all the questions about StarlingX CI/CD, please help answer as many as you can, thanks!
There would be 6 Arm servers contributed to our community for CI/CD (2 for build, 4 for testing of different deployment configs), what we want to know is:
1. What’s the current status of StarlingX CICD for x86-64?
2. What needs to be done for the CICD workflow setup and how after the 6 ARM servers are ready and hosted in the lab?
a. E.g. how to setup build servers and testing env?
b. What CICD tools need to be setup? Zuul/Jenkins/etc.
c. Are there any docs or info of the steps for these setups?
3. How many efforts and how long will it take?
4. Can this only be done by specific person? Or anyone have permission to access to the servers? If the later, I think Tianyu from Arm team and I can also help
on that if we got enough information.
Tianyu, please add if you have any other questions.
Thanks,
Jackie
From: Huang, Jie (Jackie)
Sent: Monday, July 31, 2023 12:57 PM
To: Trevor Tao <Trevor.Tao@arm.com>; Geary, Stephen (Steve) <Steve.Geary@windriver.com>; Little, Scott <Scott.Little@windriver.com>; Scott Kamp <scott.kamp@xunison.com>
Cc: starlingx-discuss@lists.starlingx.io
Subject: RE: StarlingX CI/CD setup on Arm platform
Hi, Scott Kamp and Scott Little,
I found the following actions in the meeting minutes in stx-status | OpenDev Etherpad:
o ARM Support
§ Action: Scott Kamp to respond to Jackie/the mailing list to provide assistance/access to some arm machines
§ Action: Scott Little to respond regarding build questions
· Cannot mirror more files on CENGN; will need to wait until we transition away from CENGN
· Action: Scott Kamp to explore providing a hosting env temporarily
So I think you might be able to help on Trevor’s and my questions:
There would be 6 Arm servers contributed to our community for CI/CD (2 for build, 4 for testing of different deployment configs), what we want to know is:
1. What needs to be done for the CICD workflow setup and how?
a. E.g. how to setup build servers and testing env?
b. What CICD tools need to be setup? Zuul/Jenkins/etc.
c. Are there any docs for these setups?
2. How many efforts and how long will it take?
3. Is there a CICD team or someone that can help on the setup or just providing useful information?
Thanks,
Jackie
From: Trevor Tao <Trevor.Tao@arm.com>
Sent: Thursday, July 27, 2023 5:06 PM
To: Geary, Stephen (Steve) <Steve.Geary@windriver.com>; starlingx-discuss@lists.starlingx.io
Cc: Huang, Jie (Jackie) <Jackie.Huang@windriver.com>
Subject: StarlingX CI/CD setup on Arm platform
Hi Steve, StarlingX:
From the OS distro meeting yesterday, we knew that there would be 6 Arm servers contributed to our community to do the CI/CD related work.
So we would like to ask who can help on the setup of CI/CD environment on the arm platform, and any other useful information/details for the CI/CD issue here.
Thanks,
Best Regards,
Zijin Tao(Trevor Tao, 陶孜谨)
ARM Electronic Technology (Shanghai) Co., Ltd
安谋电子科技（上海）有限公司
Building 11, Shanghai Busininess ParkⅢ ,
No.1016 Tianlin Rd, Minhang District, Shanghai, 200233 China
上海市闵行区田林路1016号科技绿洲三期2号楼10楼，200233
Cell: +86-153 7109 6192
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.