[Starlingx-discuss] StarlingX CI/CD setup on Arm platform

Thu Aug 17 15:47:47 UTC 2023

There is also ...

https://github.com/starlingx-staging/jenkins

Which contains the current state of the StarlingX jenkins setup, less a 
few credentials.

It includes 'Debian_build_server.txt' with crude documentation of how 
the server was setup.

Scott

On 2023-08-16 21:04, Huang, Jie (Jackie) wrote:
>
> Hi Davlet,
>
> Thanks for the detailed answers, which are very helpful, Tianyu and I 
> will start by familiarizing ourselves and check if there are any more 
> questions.
>
> Thanks,
>
> Jackie
>
> *From:*Panech, Davlet <Davlet.Panech at windriver.com>
> *Sent:* Wednesday, August 16, 2023 11:30 PM
> *To:* Huang, Jie (Jackie) <Jackie.Huang at windriver.com>; Little, Scott 
> <Scott.Little at windriver.com>; Tianyu Li <Tianyu.Li at arm.com>; Wang, 
> Linda <Linda.Wang at windriver.com>
> *Cc:* starlingx-discuss at lists.starlingx.io; Scott Kamp 
> <scott.kamp at xunison.com>; Trevor Tao <Trevor.Tao at arm.com>; Geary, 
> Stephen (Steve) <Steve.Geary at windriver.com>
> *Subject:* Re: StarlingX CI/CD setup on Arm platform
>
> Hi Jackie,
>
> CI/CD can be thought of as 2 somewhat disconnected parts - build & 
> automated testing. I know little about test automation, the current 
> build set up is outlined below.
>
> You would need a dedicated server:
>
> ·Linux : we currently use Debian 11, but it shouldn't matter since 
> most build steps run in docker containers
>
> ·Docker
>
> ·Kubernetes
>
> ·Jenkins: integrated with k8s and able to run k8s pipelines
>
> ·git
>
> ·git-repo https://gerrit.googlesource.com/git-repo
>
> Other prerequisites:
>
> ·You would need some DockerHub account to avoid DockerHub's rate 
> limits when pulling images
>
> ·Once your ARM set up is ready we can replace that with a controlled 
> account that we manage in order to push the images to Docker Hub.
>
> Jenkins job definitions (Jenkinsfiles) are source-controlled here: 
> https://opendev.org/starlingx/jenkins-pipelines . You would need to 
> define Jenkins jobs manually in the GUI, and point them to the 
> corresponding Jenkinsfiles in this git repo.
>
> ·The repo itself acts as a pipeline library in Jenkins with shared 
> functions under vars/ 
> <https://opendev.org/starlingx/jenkins-pipelines/src/branch/master/vars> 
> . The library must be defined at Jenkins' folder level.
>
> ·pipelines/monolithic.Jenkinsfile 
> <https://opendev.org/starlingx/jenkins-pipelines/src/branch/master/pipelines/monolithic.Jenkinsfile> 
> : top-level job that calls other jobs under parts/
>
> ·pipelines/parts/ 
> <https://opendev.org/starlingx/jenkins-pipelines/src/branch/master/pipelines/parts> 
> : directory containing sub-job Jenkinsfiles. You would create a 
> Jenkins folder in the GUI called "parts" and define individual jobs 
> pointing to individual Jenkinsfiles from this subdirectory in git repo
>
> ·scripts/ 
> <https://opendev.org/starlingx/jenkins-pipelines/src/branch/master/scripts> 
> : shell scripts that implement individual build steps (called by 
> Jenkinsfiles)
>
> Required directories on the server, owned by jenkins user:
>
> ·/localdisk/designer/jenkins/ root for per-project build areas
>
> ·/localdisk/loadbuild/jenkins/ root for per-project build archives
>
> At first execution the monolithic job will fail, but create an example 
> configuration file in the project build area that you are expected to 
> copy & edit:
>
> ·/localdisk/designer/jenkins/$PROJECT/build.cond.example
>
> There are many options with comments in it; the example file is also 
> source controlled under scripts/templates/build.conf.example.in 
> <https://opendev.org/starlingx/jenkins-pipelines/src/branch/master/scripts/templates/build.conf.example.in>
>
> Builds are slow and would greatly benefit from fast storage (SSDs etc).
>
> Please start by familiarizing yourself with this repository and ping 
> me if you have any more questions.
>
> Hope this helps,
>
> D.
>
> ------------------------------------------------------------------------
>
> *From:*Huang, Jie (Jackie) <Jackie.Huang at windriver.com>
> *Sent:* August 9, 2023 10:28 AM
> *To:* Little, Scott <Scott.Little at windriver.com>; Panech, Davlet 
> <Davlet.Panech at windriver.com>; Tianyu Li <Tianyu.Li at arm.com>; Wang, 
> Linda <Linda.Wang at windriver.com>
> *Cc:* starlingx-discuss at lists.starlingx.io 
> <starlingx-discuss at lists.starlingx.io>; Scott Kamp 
> <scott.kamp at xunison.com>; Trevor Tao <Trevor.Tao at arm.com>; Geary, 
> Stephen (Steve) <Steve.Geary at windriver.com>
> *Subject:* RE: StarlingX CI/CD setup on Arm platform
>
> Hi Scott and Davlet,
>
> As mentioned in the Multi-OS call, I’m re-sending this with all the 
> questions about StarlingX CI/CD, please help answer as many as you 
> can, thanks!
>
> There would be 6 Arm servers contributed to our community for CI/CD (2 
> for build, 4 for testing of different deployment configs), what we 
> want to know is:
>
> 1.What’s the current status of StarlingX CICD for x86-64?
>
> 2.What needs to be done for the CICD workflow setup and how after the 
> 6 ARM servers are ready and hosted in the lab?
>
> a.E.g. how to setup build servers and testing env?
>
> b.What CICD tools need to be setup? Zuul/Jenkins/etc.
>
> c.Are there any docs or info of the steps for these setups?
>
> 3.How many efforts and how long will it take?
>
> 4.Can this only be done by specific person? Or anyone have permission 
> to access to the servers? If the later, I think Tianyu from Arm team 
> and I can also help
>
> on that if we got enough information.
>
> Tianyu, please add if you have any other questions.
>
> Thanks,
>
> Jackie
>
> *From:*Huang, Jie (Jackie)
> *Sent:* Monday, July 31, 2023 12:57 PM
> *To:* Trevor Tao <Trevor.Tao at arm.com>; Geary, Stephen (Steve) 
> <Steve.Geary at windriver.com>; Little, Scott 
> <Scott.Little at windriver.com>; Scott Kamp <scott.kamp at xunison.com>
> *Cc:* starlingx-discuss at lists.starlingx.io
> *Subject:* RE: StarlingX CI/CD setup on Arm platform
>
> Hi, Scott Kamp and Scott Little,
>
> I found the following actions in the meeting minutes in stx-status | 
> OpenDev Etherpad <https://etherpad.opendev.org/p/stx-status>:
>
> oARM Support
>
> §_Action:_ Scott Kamp to respond to Jackie/the mailing list to provide 
> assistance/access to some arm machines
>
> §_Action:_ Scott Little to respond regarding build questions
>
> ·Cannot mirror more files on CENGN; will need to wait until we 
> transition away from CENGN
>
> ·_Action:_ Scott Kamp to explore providing a hosting env temporarily
>
> So I think you might be able to help on Trevor’s and my questions:
>
> There would be 6 Arm servers contributed to our community for CI/CD (2 
> for build, 4 for testing of different deployment configs), what we 
> want to know is:
>
> 1.What needs to be done for the CICD workflow setup and how?
>
> a.E.g. how to setup build servers and testing env?
>
> b.What CICD tools need to be setup? Zuul/Jenkins/etc.
>
> c.Are there any docs for these setups?
>
> 2.How many efforts and how long will it take?
>
> 3.Is there a CICD team or someone that can help on the setup or just 
> providing useful information?
>
> Thanks,
>
> Jackie
>
> *From:*Trevor Tao <Trevor.Tao at arm.com>
> *Sent:* Thursday, July 27, 2023 5:06 PM
> *To:* Geary, Stephen (Steve) <Steve.Geary at windriver.com>; 
> starlingx-discuss at lists.starlingx.io
> *Cc:* Huang, Jie (Jackie) <Jackie.Huang at windriver.com>
> *Subject:* StarlingX CI/CD setup on Arm platform
>
> Hi Steve, StarlingX:
>
> From the OS distro meeting 
> <https://urldefense.com/v3/__https:/etherpad.opendev.org/p/stx-distro-other__;!!AjveYdw8EvQ!e07vSdClUS6f5DohQ4EQeAEzsC4SnjZH2Vb3mjeau1FjCuAnkaGqf1_1vuARZRPcPxdw8dZJN-D3pMmWKHLvhCs$> 
> yesterday, we knew that there would be 6 Arm servers contributed to 
> our community to do the CI/CD related work.
>
> So we would like to ask who can help on the setup of CI/CD environment 
> on the arm platform, and any other useful information/details for the 
> CI/CD issue here.
>
> Thanks,
>
> Best Regards,
>
> Zijin Tao(Trevor Tao, 陶孜谨)
>
> ARM Electronic Technology (Shanghai) Co., Ltd
>
> 安谋电子科技（上海）有限公司
>
> Building 11, Shanghai Busininess ParkⅢ,
>
> No.1016 Tianlin Rd, Minhang District, Shanghai, 200233 China
>
> 上海市闵行区田林路1016号科技绿洲三期2号楼10楼，200233
>
> Cell:   +86-153 7109 6192
>
> IMPORTANT NOTICE: The contents of this email and any attachments are 
> confidential and may also be privileged. If you are not the intended 
> recipient, please notify the sender immediately and do not disclose 
> the contents to any other person, use it for any purpose, or store or 
> copy the information in any medium. Thank you.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20230817/f94ef983/attachment-0001.htm>