[Starlingx-discuss] StarlingX CI/CD setup on Arm platform
Hi Steve, StarlingX: From the OS distro meeting<https://etherpad.opendev.org/p/stx-distro-other> yesterday, we knew that there would be 6 Arm servers contributed to our community to do the CI/CD related work. So we would like to ask who can help on the setup of CI/CD environment on the arm platform, and any other useful information/details for the CI/CD issue here. Thanks, Best Regards, Zijin Tao(Trevor Tao, 陶孜谨) ARM Electronic Technology (Shanghai) Co., Ltd 安谋电子科技（上海）有限公司 Building 11, Shanghai Busininess ParkⅢ , No.1016 Tianlin Rd, Minhang District, Shanghai, 200233 China 上海市闵行区田林路1016号科技绿洲三期2号楼10楼，200233 Cell: +86-153 7109 6192 IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Hi, Scott Kamp and Scott Little, I found the following actions in the meeting minutes in stx-status | OpenDev Etherpad<https://etherpad.opendev.org/p/stx-status>: o ARM Support § Action: Scott Kamp to respond to Jackie/the mailing list to provide assistance/access to some arm machines § Action: Scott Little to respond regarding build questions ・ Cannot mirror more files on CENGN; will need to wait until we transition away from CENGN ・ Action: Scott Kamp to explore providing a hosting env temporarily So I think you might be able to help on Trevor’s and my questions: There would be 6 Arm servers contributed to our community for CI/CD (2 for build, 4 for testing of different deployment configs), what we want to know is: 1. What needs to be done for the CICD workflow setup and how? * E.g. how to setup build servers and testing env? * What CICD tools need to be setup? Zuul/Jenkins/etc. * Are there any docs for these setups? 2. How many efforts and how long will it take? 3. Is there a CICD team or someone that can help on the setup or just providing useful information? Thanks, Jackie From: Trevor Tao <Trevor.Tao@arm.com> Sent: Thursday, July 27, 2023 5:06 PM To: Geary, Stephen (Steve) <Steve.Geary@windriver.com>; starlingx-discuss@lists.starlingx.io Cc: Huang, Jie (Jackie) <Jackie.Huang@windriver.com> Subject: StarlingX CI/CD setup on Arm platform Hi Steve, StarlingX: From the OS distro meeting<https://urldefense.com/v3/__https:/etherpad.opendev.org/p/stx-distro-other__;!!AjveYdw8EvQ!e07vSdClUS6f5DohQ4EQeAEzsC4SnjZH2Vb3mjeau1FjCuAnkaGqf1_1vuARZRPcPxdw8dZJN-D3pMmWKHLvhCs$> yesterday, we knew that there would be 6 Arm servers contributed to our community to do the CI/CD related work. So we would like to ask who can help on the setup of CI/CD environment on the arm platform, and any other useful information/details for the CI/CD issue here. Thanks, Best Regards, Zijin Tao(Trevor Tao, 陶孜谨) ARM Electronic Technology (Shanghai) Co., Ltd 安谋电子科技（上海）有限公司 Building 11, Shanghai Busininess ParkⅢ , No.1016 Tianlin Rd, Minhang District, Shanghai, 200233 China 上海市闵行区田林路1016号科技绿洲三期2号楼10楼，200233 Cell: +86-153 7109 6192 IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Hi Scott and Davlet, As mentioned in the Multi-OS call, I’m re-sending this with all the questions about StarlingX CI/CD, please help answer as many as you can, thanks! There would be 6 Arm servers contributed to our community for CI/CD (2 for build, 4 for testing of different deployment configs), what we want to know is: 1. What’s the current status of StarlingX CICD for x86-64? 2. What needs to be done for the CICD workflow setup and how after the 6 ARM servers are ready and hosted in the lab? a. E.g. how to setup build servers and testing env? b. What CICD tools need to be setup? Zuul/Jenkins/etc. c. Are there any docs or info of the steps for these setups? 3. How many efforts and how long will it take? 4. Can this only be done by specific person? Or anyone have permission to access to the servers? If the later, I think Tianyu from Arm team and I can also help on that if we got enough information. Tianyu, please add if you have any other questions. Thanks, Jackie From: Huang, Jie (Jackie) Sent: Monday, July 31, 2023 12:57 PM To: Trevor Tao <Trevor.Tao@arm.com>; Geary, Stephen (Steve) <Steve.Geary@windriver.com>; Little, Scott <Scott.Little@windriver.com>; Scott Kamp <scott.kamp@xunison.com> Cc: starlingx-discuss@lists.starlingx.io Subject: RE: StarlingX CI/CD setup on Arm platform Hi, Scott Kamp and Scott Little, I found the following actions in the meeting minutes in stx-status | OpenDev Etherpad<https://etherpad.opendev.org/p/stx-status>: o ARM Support § Action: Scott Kamp to respond to Jackie/the mailing list to provide assistance/access to some arm machines § Action: Scott Little to respond regarding build questions ・ Cannot mirror more files on CENGN; will need to wait until we transition away from CENGN ・ Action: Scott Kamp to explore providing a hosting env temporarily So I think you might be able to help on Trevor’s and my questions: There would be 6 Arm servers contributed to our community for CI/CD (2 for build, 4 for testing of different deployment configs), what we want to know is: 1. What needs to be done for the CICD workflow setup and how? a. E.g. how to setup build servers and testing env? b. What CICD tools need to be setup? Zuul/Jenkins/etc. c. Are there any docs for these setups? 2. How many efforts and how long will it take? 3. Is there a CICD team or someone that can help on the setup or just providing useful information? Thanks, Jackie From: Trevor Tao <Trevor.Tao@arm.com<mailto:Trevor.Tao@arm.com>> Sent: Thursday, July 27, 2023 5:06 PM To: Geary, Stephen (Steve) <Steve.Geary@windriver.com<mailto:Steve.Geary@windriver.com>>; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Cc: Huang, Jie (Jackie) <Jackie.Huang@windriver.com<mailto:Jackie.Huang@windriver.com>> Subject: StarlingX CI/CD setup on Arm platform Hi Steve, StarlingX: From the OS distro meeting<https://urldefense.com/v3/__https:/etherpad.opendev.org/p/stx-distro-other__;!!AjveYdw8EvQ!e07vSdClUS6f5DohQ4EQeAEzsC4SnjZH2Vb3mjeau1FjCuAnkaGqf1_1vuARZRPcPxdw8dZJN-D3pMmWKHLvhCs$> yesterday, we knew that there would be 6 Arm servers contributed to our community to do the CI/CD related work. So we would like to ask who can help on the setup of CI/CD environment on the arm platform, and any other useful information/details for the CI/CD issue here. Thanks, Best Regards, Zijin Tao(Trevor Tao, 陶孜谨) ARM Electronic Technology (Shanghai) Co., Ltd 安谋电子科技（上海）有限公司 Building 11, Shanghai Busininess ParkⅢ , No.1016 Tianlin Rd, Minhang District, Shanghai, 200233 China 上海市闵行区田林路1016号科技绿洲三期2号楼10楼，200233 Cell: +86-153 7109 6192 IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Hi Jackie, CI/CD can be thought of as 2 somewhat disconnected parts - build & automated testing. I know little about test automation, the current build set up is outlined below. You would need a dedicated server: * Linux : we currently use Debian 11, but it shouldn't matter since most build steps run in docker containers * Docker * Kubernetes * Jenkins: integrated with k8s and able to run k8s pipelines * git * git-repo https://gerrit.googlesource.com/git-repo Other prerequisites: * You would need some DockerHub account to avoid DockerHub's rate limits when pulling images * Once your ARM set up is ready we can replace that with a controlled account that we manage in order to push the images to Docker Hub. Jenkins job definitions (Jenkinsfiles) are source-controlled here: https://opendev.org/starlingx/jenkins-pipelines . You would need to define Jenkins jobs manually in the GUI, and point them to the corresponding Jenkinsfiles in this git repo. * The repo itself acts as a pipeline library in Jenkins with shared functions under vars/<https://opendev.org/starlingx/jenkins-pipelines/src/branch/master/vars> . The library must be defined at Jenkins' folder level. * pipelines/monolithic.Jenkinsfile<https://opendev.org/starlingx/jenkins-pipelines/src/branch/master/pipelines/monolithic.Jenkinsfile> : top-level job that calls other jobs under parts/ * pipelines/parts/<https://opendev.org/starlingx/jenkins-pipelines/src/branch/master/pipelines/parts> : directory containing sub-job Jenkinsfiles. You would create a Jenkins folder in the GUI called "parts" and define individual jobs pointing to individual Jenkinsfiles from this subdirectory in git repo * scripts/<https://opendev.org/starlingx/jenkins-pipelines/src/branch/master/scripts> : shell scripts that implement individual build steps (called by Jenkinsfiles) Required directories on the server, owned by jenkins user: * /localdisk/designer/jenkins/ root for per-project build areas * /localdisk/loadbuild/jenkins/ root for per-project build archives At first execution the monolithic job will fail, but create an example configuration file in the project build area that you are expected to copy & edit: * /localdisk/designer/jenkins/$PROJECT/build.cond.example There are many options with comments in it; the example file is also source controlled under scripts/templates/build.conf.example.in<https://opendev.org/starlingx/jenkins-pipelines/src/branch/master/scripts/templates/build.conf.example.in> Builds are slow and would greatly benefit from fast storage (SSDs etc). Please start by familiarizing yourself with this repository and ping me if you have any more questions. Hope this helps, D. ________________________________ From: Huang, Jie (Jackie) <Jackie.Huang@windriver.com> Sent: August 9, 2023 10:28 AM To: Little, Scott <Scott.Little@windriver.com>; Panech, Davlet <Davlet.Panech@windriver.com>; Tianyu Li <Tianyu.Li@arm.com>; Wang, Linda <Linda.Wang@windriver.com> Cc: starlingx-discuss@lists.starlingx.io <starlingx-discuss@lists.starlingx.io>; Scott Kamp <scott.kamp@xunison.com>; Trevor Tao <Trevor.Tao@arm.com>; Geary, Stephen (Steve) <Steve.Geary@windriver.com> Subject: RE: StarlingX CI/CD setup on Arm platform Hi Scott and Davlet, As mentioned in the Multi-OS call, I’m re-sending this with all the questions about StarlingX CI/CD, please help answer as many as you can, thanks! There would be 6 Arm servers contributed to our community for CI/CD (2 for build, 4 for testing of different deployment configs), what we want to know is: 1. What’s the current status of StarlingX CICD for x86-64? 2. What needs to be done for the CICD workflow setup and how after the 6 ARM servers are ready and hosted in the lab? a. E.g. how to setup build servers and testing env? b. What CICD tools need to be setup? Zuul/Jenkins/etc. c. Are there any docs or info of the steps for these setups? 3. How many efforts and how long will it take? 4. Can this only be done by specific person? Or anyone have permission to access to the servers? If the later, I think Tianyu from Arm team and I can also help on that if we got enough information. Tianyu, please add if you have any other questions. Thanks, Jackie From: Huang, Jie (Jackie) Sent: Monday, July 31, 2023 12:57 PM To: Trevor Tao <Trevor.Tao@arm.com>; Geary, Stephen (Steve) <Steve.Geary@windriver.com>; Little, Scott <Scott.Little@windriver.com>; Scott Kamp <scott.kamp@xunison.com> Cc: starlingx-discuss@lists.starlingx.io Subject: RE: StarlingX CI/CD setup on Arm platform Hi, Scott Kamp and Scott Little, I found the following actions in the meeting minutes in stx-status | OpenDev Etherpad<https://etherpad.opendev.org/p/stx-status>: o ARM Support § Action: Scott Kamp to respond to Jackie/the mailing list to provide assistance/access to some arm machines § Action: Scott Little to respond regarding build questions ・ Cannot mirror more files on CENGN; will need to wait until we transition away from CENGN ・ Action: Scott Kamp to explore providing a hosting env temporarily So I think you might be able to help on Trevor’s and my questions: There would be 6 Arm servers contributed to our community for CI/CD (2 for build, 4 for testing of different deployment configs), what we want to know is: 1. What needs to be done for the CICD workflow setup and how? a. E.g. how to setup build servers and testing env? b. What CICD tools need to be setup? Zuul/Jenkins/etc. c. Are there any docs for these setups? 2. How many efforts and how long will it take? 3. Is there a CICD team or someone that can help on the setup or just providing useful information? Thanks, Jackie From: Trevor Tao <Trevor.Tao@arm.com<mailto:Trevor.Tao@arm.com>> Sent: Thursday, July 27, 2023 5:06 PM To: Geary, Stephen (Steve) <Steve.Geary@windriver.com<mailto:Steve.Geary@windriver.com>>; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Cc: Huang, Jie (Jackie) <Jackie.Huang@windriver.com<mailto:Jackie.Huang@windriver.com>> Subject: StarlingX CI/CD setup on Arm platform Hi Steve, StarlingX: From the OS distro meeting<https://urldefense.com/v3/__https:/etherpad.opendev.org/p/stx-distro-other__;!!AjveYdw8EvQ!e07vSdClUS6f5DohQ4EQeAEzsC4SnjZH2Vb3mjeau1FjCuAnkaGqf1_1vuARZRPcPxdw8dZJN-D3pMmWKHLvhCs$> yesterday, we knew that there would be 6 Arm servers contributed to our community to do the CI/CD related work. So we would like to ask who can help on the setup of CI/CD environment on the arm platform, and any other useful information/details for the CI/CD issue here. Thanks, Best Regards, Zijin Tao(Trevor Tao, 陶孜谨) ARM Electronic Technology (Shanghai) Co., Ltd 安谋电子科技（上海）有限公司 Building 11, Shanghai Busininess ParkⅢ , No.1016 Tianlin Rd, Minhang District, Shanghai, 200233 China 上海市闵行区田林路1016号科技绿洲三期2号楼10楼，200233 Cell: +86-153 7109 6192 IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Hi Davlet, Thanks for the detailed answers, which are very helpful, Tianyu and I will start by familiarizing ourselves and check if there are any more questions. Thanks, Jackie From: Panech, Davlet <Davlet.Panech@windriver.com> Sent: Wednesday, August 16, 2023 11:30 PM To: Huang, Jie (Jackie) <Jackie.Huang@windriver.com>; Little, Scott <Scott.Little@windriver.com>; Tianyu Li <Tianyu.Li@arm.com>; Wang, Linda <Linda.Wang@windriver.com> Cc: starlingx-discuss@lists.starlingx.io; Scott Kamp <scott.kamp@xunison.com>; Trevor Tao <Trevor.Tao@arm.com>; Geary, Stephen (Steve) <Steve.Geary@windriver.com> Subject: Re: StarlingX CI/CD setup on Arm platform Hi Jackie, CI/CD can be thought of as 2 somewhat disconnected parts - build & automated testing. I know little about test automation, the current build set up is outlined below. You would need a dedicated server: ・ Linux : we currently use Debian 11, but it shouldn't matter since most build steps run in docker containers ・ Docker ・ Kubernetes ・ Jenkins: integrated with k8s and able to run k8s pipelines ・ git ・ git-repo https://gerrit.googlesource.com/git-repo Other prerequisites: ・ You would need some DockerHub account to avoid DockerHub's rate limits when pulling images ・ Once your ARM set up is ready we can replace that with a controlled account that we manage in order to push the images to Docker Hub. Jenkins job definitions (Jenkinsfiles) are source-controlled here: https://opendev.org/starlingx/jenkins-pipelines . You would need to define Jenkins jobs manually in the GUI, and point them to the corresponding Jenkinsfiles in this git repo. ・ The repo itself acts as a pipeline library in Jenkins with shared functions under vars/<https://opendev.org/starlingx/jenkins-pipelines/src/branch/master/vars> . The library must be defined at Jenkins' folder level. ・ pipelines/monolithic.Jenkinsfile<https://opendev.org/starlingx/jenkins-pipelines/src/branch/master/pipelines/monolithic.Jenkinsfile> : top-level job that calls other jobs under parts/ ・ pipelines/parts/<https://opendev.org/starlingx/jenkins-pipelines/src/branch/master/pipelines/parts> : directory containing sub-job Jenkinsfiles. You would create a Jenkins folder in the GUI called "parts" and define individual jobs pointing to individual Jenkinsfiles from this subdirectory in git repo ・ scripts/<https://opendev.org/starlingx/jenkins-pipelines/src/branch/master/scripts> : shell scripts that implement individual build steps (called by Jenkinsfiles) Required directories on the server, owned by jenkins user: ・ /localdisk/designer/jenkins/ root for per-project build areas ・ /localdisk/loadbuild/jenkins/ root for per-project build archives At first execution the monolithic job will fail, but create an example configuration file in the project build area that you are expected to copy & edit: ・ /localdisk/designer/jenkins/$PROJECT/build.cond.example There are many options with comments in it; the example file is also source controlled under scripts/templates/build.conf.example.in<https://opendev.org/starlingx/jenkins-pipelines/src/branch/master/scripts/templates/build.conf.example.in> Builds are slow and would greatly benefit from fast storage (SSDs etc). Please start by familiarizing yourself with this repository and ping me if you have any more questions. Hope this helps, D. ________________________________ From: Huang, Jie (Jackie) <Jackie.Huang@windriver.com<mailto:Jackie.Huang@windriver.com>> Sent: August 9, 2023 10:28 AM To: Little, Scott <Scott.Little@windriver.com<mailto:Scott.Little@windriver.com>>; Panech, Davlet <Davlet.Panech@windriver.com<mailto:Davlet.Panech@windriver.com>>; Tianyu Li <Tianyu.Li@arm.com<mailto:Tianyu.Li@arm.com>>; Wang, Linda <Linda.Wang@windriver.com<mailto:Linda.Wang@windriver.com>> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> <starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>>; Scott Kamp <scott.kamp@xunison.com<mailto:scott.kamp@xunison.com>>; Trevor Tao <Trevor.Tao@arm.com<mailto:Trevor.Tao@arm.com>>; Geary, Stephen (Steve) <Steve.Geary@windriver.com<mailto:Steve.Geary@windriver.com>> Subject: RE: StarlingX CI/CD setup on Arm platform Hi Scott and Davlet, As mentioned in the Multi-OS call, I’m re-sending this with all the questions about StarlingX CI/CD, please help answer as many as you can, thanks! There would be 6 Arm servers contributed to our community for CI/CD (2 for build, 4 for testing of different deployment configs), what we want to know is: 1. What’s the current status of StarlingX CICD for x86-64? 2. What needs to be done for the CICD workflow setup and how after the 6 ARM servers are ready and hosted in the lab? a. E.g. how to setup build servers and testing env? b. What CICD tools need to be setup? Zuul/Jenkins/etc. c. Are there any docs or info of the steps for these setups? 3. How many efforts and how long will it take? 4. Can this only be done by specific person? Or anyone have permission to access to the servers? If the later, I think Tianyu from Arm team and I can also help on that if we got enough information. Tianyu, please add if you have any other questions. Thanks, Jackie From: Huang, Jie (Jackie) Sent: Monday, July 31, 2023 12:57 PM To: Trevor Tao <Trevor.Tao@arm.com<mailto:Trevor.Tao@arm.com>>; Geary, Stephen (Steve) <Steve.Geary@windriver.com<mailto:Steve.Geary@windriver.com>>; Little, Scott <Scott.Little@windriver.com<mailto:Scott.Little@windriver.com>>; Scott Kamp <scott.kamp@xunison.com<mailto:scott.kamp@xunison.com>> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: RE: StarlingX CI/CD setup on Arm platform Hi, Scott Kamp and Scott Little, I found the following actions in the meeting minutes in stx-status | OpenDev Etherpad<https://etherpad.opendev.org/p/stx-status>: o ARM Support * Action: Scott Kamp to respond to Jackie/the mailing list to provide assistance/access to some arm machines * Action: Scott Little to respond regarding build questions * Cannot mirror more files on CENGN; will need to wait until we transition away from CENGN * Action: Scott Kamp to explore providing a hosting env temporarily So I think you might be able to help on Trevor’s and my questions: There would be 6 Arm servers contributed to our community for CI/CD (2 for build, 4 for testing of different deployment configs), what we want to know is: 1. What needs to be done for the CICD workflow setup and how? a. E.g. how to setup build servers and testing env? b. What CICD tools need to be setup? Zuul/Jenkins/etc. c. Are there any docs for these setups? 2. How many efforts and how long will it take? 3. Is there a CICD team or someone that can help on the setup or just providing useful information? Thanks, Jackie From: Trevor Tao <Trevor.Tao@arm.com<mailto:Trevor.Tao@arm.com>> Sent: Thursday, July 27, 2023 5:06 PM To: Geary, Stephen (Steve) <Steve.Geary@windriver.com<mailto:Steve.Geary@windriver.com>>; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Cc: Huang, Jie (Jackie) <Jackie.Huang@windriver.com<mailto:Jackie.Huang@windriver.com>> Subject: StarlingX CI/CD setup on Arm platform Hi Steve, StarlingX: From the OS distro meeting<https://urldefense.com/v3/__https:/etherpad.opendev.org/p/stx-distro-other__;!!AjveYdw8EvQ!e07vSdClUS6f5DohQ4EQeAEzsC4SnjZH2Vb3mjeau1FjCuAnkaGqf1_1vuARZRPcPxdw8dZJN-D3pMmWKHLvhCs$> yesterday, we knew that there would be 6 Arm servers contributed to our community to do the CI/CD related work. So we would like to ask who can help on the setup of CI/CD environment on the arm platform, and any other useful information/details for the CI/CD issue here. Thanks, Best Regards, Zijin Tao(Trevor Tao, 陶孜谨) ARM Electronic Technology (Shanghai) Co., Ltd 安谋电子科技（上海）有限公司 Building 11, Shanghai Busininess ParkⅢ , No.1016 Tianlin Rd, Minhang District, Shanghai, 200233 China 上海市闵行区田林路1016号科技绿洲三期2号楼10楼，200233 Cell: +86-153 7109 6192 IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
There is also ... https://github.com/starlingx-staging/jenkins Which contains the current state of the StarlingX jenkins setup, less a few credentials. It includes 'Debian_build_server.txt' with crude documentation of how the server was setup. Scott On 2023-08-16 21:04, Huang, Jie (Jackie) wrote:
Hi Davlet,
Thanks for the detailed answers, which are very helpful, Tianyu and I will start by familiarizing ourselves and check if there are any more questions.
Thanks,
Jackie
*From:*Panech, Davlet <Davlet.Panech@windriver.com> *Sent:* Wednesday, August 16, 2023 11:30 PM *To:* Huang, Jie (Jackie) <Jackie.Huang@windriver.com>; Little, Scott <Scott.Little@windriver.com>; Tianyu Li <Tianyu.Li@arm.com>; Wang, Linda <Linda.Wang@windriver.com> *Cc:* starlingx-discuss@lists.starlingx.io; Scott Kamp <scott.kamp@xunison.com>; Trevor Tao <Trevor.Tao@arm.com>; Geary, Stephen (Steve) <Steve.Geary@windriver.com> *Subject:* Re: StarlingX CI/CD setup on Arm platform
Hi Jackie,
CI/CD can be thought of as 2 somewhat disconnected parts - build & automated testing. I know little about test automation, the current build set up is outlined below.
You would need a dedicated server:
·Linux : we currently use Debian 11, but it shouldn't matter since most build steps run in docker containers
·Docker
·Kubernetes
·Jenkins: integrated with k8s and able to run k8s pipelines
·git
·git-repo https://gerrit.googlesource.com/git-repo
Other prerequisites:
·You would need some DockerHub account to avoid DockerHub's rate limits when pulling images
·Once your ARM set up is ready we can replace that with a controlled account that we manage in order to push the images to Docker Hub.
Jenkins job definitions (Jenkinsfiles) are source-controlled here: https://opendev.org/starlingx/jenkins-pipelines . You would need to define Jenkins jobs manually in the GUI, and point them to the corresponding Jenkinsfiles in this git repo.
·The repo itself acts as a pipeline library in Jenkins with shared functions under vars/ <https://opendev.org/starlingx/jenkins-pipelines/src/branch/master/vars> . The library must be defined at Jenkins' folder level.
·pipelines/monolithic.Jenkinsfile <https://opendev.org/starlingx/jenkins-pipelines/src/branch/master/pipelines/monolithic.Jenkinsfile> : top-level job that calls other jobs under parts/
·pipelines/parts/ <https://opendev.org/starlingx/jenkins-pipelines/src/branch/master/pipelines/parts> : directory containing sub-job Jenkinsfiles. You would create a Jenkins folder in the GUI called "parts" and define individual jobs pointing to individual Jenkinsfiles from this subdirectory in git repo
·scripts/ <https://opendev.org/starlingx/jenkins-pipelines/src/branch/master/scripts> : shell scripts that implement individual build steps (called by Jenkinsfiles)
Required directories on the server, owned by jenkins user:
·/localdisk/designer/jenkins/ root for per-project build areas
·/localdisk/loadbuild/jenkins/ root for per-project build archives
At first execution the monolithic job will fail, but create an example configuration file in the project build area that you are expected to copy & edit:
·/localdisk/designer/jenkins/$PROJECT/build.cond.example
There are many options with comments in it; the example file is also source controlled under scripts/templates/build.conf.example.in <https://opendev.org/starlingx/jenkins-pipelines/src/branch/master/scripts/templates/build.conf.example.in>
Builds are slow and would greatly benefit from fast storage (SSDs etc).
Please start by familiarizing yourself with this repository and ping me if you have any more questions.
Hope this helps,
D.
------------------------------------------------------------------------
*From:*Huang, Jie (Jackie) <Jackie.Huang@windriver.com> *Sent:* August 9, 2023 10:28 AM *To:* Little, Scott <Scott.Little@windriver.com>; Panech, Davlet <Davlet.Panech@windriver.com>; Tianyu Li <Tianyu.Li@arm.com>; Wang, Linda <Linda.Wang@windriver.com> *Cc:* starlingx-discuss@lists.starlingx.io <starlingx-discuss@lists.starlingx.io>; Scott Kamp <scott.kamp@xunison.com>; Trevor Tao <Trevor.Tao@arm.com>; Geary, Stephen (Steve) <Steve.Geary@windriver.com> *Subject:* RE: StarlingX CI/CD setup on Arm platform
Hi Scott and Davlet,
As mentioned in the Multi-OS call, I’m re-sending this with all the questions about StarlingX CI/CD, please help answer as many as you can, thanks!
There would be 6 Arm servers contributed to our community for CI/CD (2 for build, 4 for testing of different deployment configs), what we want to know is:
1.What’s the current status of StarlingX CICD for x86-64?
2.What needs to be done for the CICD workflow setup and how after the 6 ARM servers are ready and hosted in the lab?
a.E.g. how to setup build servers and testing env?
b.What CICD tools need to be setup? Zuul/Jenkins/etc.
c.Are there any docs or info of the steps for these setups?
3.How many efforts and how long will it take?
4.Can this only be done by specific person? Or anyone have permission to access to the servers? If the later, I think Tianyu from Arm team and I can also help
on that if we got enough information.
Tianyu, please add if you have any other questions.
Thanks,
Jackie
*From:*Huang, Jie (Jackie) *Sent:* Monday, July 31, 2023 12:57 PM *To:* Trevor Tao <Trevor.Tao@arm.com>; Geary, Stephen (Steve) <Steve.Geary@windriver.com>; Little, Scott <Scott.Little@windriver.com>; Scott Kamp <scott.kamp@xunison.com> *Cc:* starlingx-discuss@lists.starlingx.io *Subject:* RE: StarlingX CI/CD setup on Arm platform
Hi, Scott Kamp and Scott Little,
I found the following actions in the meeting minutes in stx-status | OpenDev Etherpad <https://etherpad.opendev.org/p/stx-status>:
oARM Support
§_Action:_ Scott Kamp to respond to Jackie/the mailing list to provide assistance/access to some arm machines
§_Action:_ Scott Little to respond regarding build questions
·Cannot mirror more files on CENGN; will need to wait until we transition away from CENGN
·_Action:_ Scott Kamp to explore providing a hosting env temporarily
So I think you might be able to help on Trevor’s and my questions:
There would be 6 Arm servers contributed to our community for CI/CD (2 for build, 4 for testing of different deployment configs), what we want to know is:
1.What needs to be done for the CICD workflow setup and how?
a.E.g. how to setup build servers and testing env?
b.What CICD tools need to be setup? Zuul/Jenkins/etc.
c.Are there any docs for these setups?
2.How many efforts and how long will it take?
3.Is there a CICD team or someone that can help on the setup or just providing useful information?
Thanks,
Jackie
*From:*Trevor Tao <Trevor.Tao@arm.com> *Sent:* Thursday, July 27, 2023 5:06 PM *To:* Geary, Stephen (Steve) <Steve.Geary@windriver.com>; starlingx-discuss@lists.starlingx.io *Cc:* Huang, Jie (Jackie) <Jackie.Huang@windriver.com> *Subject:* StarlingX CI/CD setup on Arm platform
Hi Steve, StarlingX:
From the OS distro meeting <https://urldefense.com/v3/__https:/etherpad.opendev.org/p/stx-distro-other__;!!AjveYdw8EvQ!e07vSdClUS6f5DohQ4EQeAEzsC4SnjZH2Vb3mjeau1FjCuAnkaGqf1_1vuARZRPcPxdw8dZJN-D3pMmWKHLvhCs$> yesterday, we knew that there would be 6 Arm servers contributed to our community to do the CI/CD related work.
So we would like to ask who can help on the setup of CI/CD environment on the arm platform, and any other useful information/details for the CI/CD issue here.
Thanks,
Best Regards,
Zijin Tao(Trevor Tao, 陶孜谨)
ARM Electronic Technology (Shanghai) Co., Ltd
安谋电子科技（上海）有限公司
Building 11, Shanghai Busininess ParkⅢ,
No.1016 Tianlin Rd, Minhang District, Shanghai, 200233 China
上海市闵行区田林路1016号科技绿洲三期2号楼10楼，200233
Cell: +86-153 7109 6192
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Hi Scott, Thanks for sharing, it’s very helpful. Thanks, Jackie From: Little, Scott <Scott.Little@windriver.com> Sent: Thursday, August 17, 2023 11:48 PM To: Huang, Jie (Jackie) <Jackie.Huang@windriver.com>; Panech, Davlet <Davlet.Panech@windriver.com>; Tianyu Li <Tianyu.Li@arm.com> Cc: starlingx-discuss@lists.starlingx.io; Scott Kamp <scott.kamp@xunison.com>; Trevor Tao <Trevor.Tao@arm.com>; Geary, Stephen (Steve) <Steve.Geary@windriver.com>; Wang, Linda <Linda.Wang@windriver.com> Subject: Re: StarlingX CI/CD setup on Arm platform There is also ... https://github.com/starlingx-staging/jenkins Which contains the current state of the StarlingX jenkins setup, less a few credentials. It includes 'Debian_build_server.txt' with crude documentation of how the server was setup. Scott On 2023-08-16 21:04, Huang, Jie (Jackie) wrote: Hi Davlet, Thanks for the detailed answers, which are very helpful, Tianyu and I will start by familiarizing ourselves and check if there are any more questions. Thanks, Jackie From: Panech, Davlet <Davlet.Panech@windriver.com><mailto:Davlet.Panech@windriver.com> Sent: Wednesday, August 16, 2023 11:30 PM To: Huang, Jie (Jackie) <Jackie.Huang@windriver.com><mailto:Jackie.Huang@windriver.com>; Little, Scott <Scott.Little@windriver.com><mailto:Scott.Little@windriver.com>; Tianyu Li <Tianyu.Li@arm.com><mailto:Tianyu.Li@arm.com>; Wang, Linda <Linda.Wang@windriver.com><mailto:Linda.Wang@windriver.com> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>; Scott Kamp <scott.kamp@xunison.com><mailto:scott.kamp@xunison.com>; Trevor Tao <Trevor.Tao@arm.com><mailto:Trevor.Tao@arm.com>; Geary, Stephen (Steve) <Steve.Geary@windriver.com><mailto:Steve.Geary@windriver.com> Subject: Re: StarlingX CI/CD setup on Arm platform Hi Jackie, CI/CD can be thought of as 2 somewhat disconnected parts - build & automated testing. I know little about test automation, the current build set up is outlined below. You would need a dedicated server: 1. Linux : we currently use Debian 11, but it shouldn't matter since most build steps run in docker containers 2. Docker 3. Kubernetes 4. Jenkins: integrated with k8s and able to run k8s pipelines 5. git 6. git-repo https://gerrit.googlesource.com/git-repo Other prerequisites: 1. You would need some DockerHub account to avoid DockerHub's rate limits when pulling images 2. Once your ARM set up is ready we can replace that with a controlled account that we manage in order to push the images to Docker Hub. Jenkins job definitions (Jenkinsfiles) are source-controlled here: https://opendev.org/starlingx/jenkins-pipelines . You would need to define Jenkins jobs manually in the GUI, and point them to the corresponding Jenkinsfiles in this git repo. 1. The repo itself acts as a pipeline library in Jenkins with shared functions under vars/<https://opendev.org/starlingx/jenkins-pipelines/src/branch/master/vars> . The library must be defined at Jenkins' folder level. 2. pipelines/monolithic.Jenkinsfile<https://opendev.org/starlingx/jenkins-pipelines/src/branch/master/pipelines/monolithic.Jenkinsfile> : top-level job that calls other jobs under parts/ 3. pipelines/parts/<https://opendev.org/starlingx/jenkins-pipelines/src/branch/master/pipelines/parts> : directory containing sub-job Jenkinsfiles. You would create a Jenkins folder in the GUI called "parts" and define individual jobs pointing to individual Jenkinsfiles from this subdirectory in git repo 4. scripts/<https://opendev.org/starlingx/jenkins-pipelines/src/branch/master/scripts> : shell scripts that implement individual build steps (called by Jenkinsfiles) Required directories on the server, owned by jenkins user: 1. /localdisk/designer/jenkins/ root for per-project build areas 2. /localdisk/loadbuild/jenkins/ root for per-project build archives At first execution the monolithic job will fail, but create an example configuration file in the project build area that you are expected to copy & edit: 1. /localdisk/designer/jenkins/$PROJECT/build.cond.example There are many options with comments in it; the example file is also source controlled under scripts/templates/build.conf.example.in<https://opendev.org/starlingx/jenkins-pipelines/src/branch/master/scripts/templates/build.conf.example.in> Builds are slow and would greatly benefit from fast storage (SSDs etc). Please start by familiarizing yourself with this repository and ping me if you have any more questions. Hope this helps, D. ________________________________ From: Huang, Jie (Jackie) <Jackie.Huang@windriver.com<mailto:Jackie.Huang@windriver.com>> Sent: August 9, 2023 10:28 AM To: Little, Scott <Scott.Little@windriver.com<mailto:Scott.Little@windriver.com>>; Panech, Davlet <Davlet.Panech@windriver.com<mailto:Davlet.Panech@windriver.com>>; Tianyu Li <Tianyu.Li@arm.com<mailto:Tianyu.Li@arm.com>>; Wang, Linda <Linda.Wang@windriver.com<mailto:Linda.Wang@windriver.com>> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> <starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>>; Scott Kamp <scott.kamp@xunison.com<mailto:scott.kamp@xunison.com>>; Trevor Tao <Trevor.Tao@arm.com<mailto:Trevor.Tao@arm.com>>; Geary, Stephen (Steve) <Steve.Geary@windriver.com<mailto:Steve.Geary@windriver.com>> Subject: RE: StarlingX CI/CD setup on Arm platform Hi Scott and Davlet, As mentioned in the Multi-OS call, I’m re-sending this with all the questions about StarlingX CI/CD, please help answer as many as you can, thanks! There would be 6 Arm servers contributed to our community for CI/CD (2 for build, 4 for testing of different deployment configs), what we want to know is: 1. What’s the current status of StarlingX CICD for x86-64? 2. What needs to be done for the CICD workflow setup and how after the 6 ARM servers are ready and hosted in the lab? a. E.g. how to setup build servers and testing env? b. What CICD tools need to be setup? Zuul/Jenkins/etc. c. Are there any docs or info of the steps for these setups? 3. How many efforts and how long will it take? 4. Can this only be done by specific person? Or anyone have permission to access to the servers? If the later, I think Tianyu from Arm team and I can also help on that if we got enough information. Tianyu, please add if you have any other questions. Thanks, Jackie From: Huang, Jie (Jackie) Sent: Monday, July 31, 2023 12:57 PM To: Trevor Tao <Trevor.Tao@arm.com<mailto:Trevor.Tao@arm.com>>; Geary, Stephen (Steve) <Steve.Geary@windriver.com<mailto:Steve.Geary@windriver.com>>; Little, Scott <Scott.Little@windriver.com<mailto:Scott.Little@windriver.com>>; Scott Kamp <scott.kamp@xunison.com<mailto:scott.kamp@xunison.com>> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: RE: StarlingX CI/CD setup on Arm platform Hi, Scott Kamp and Scott Little, I found the following actions in the meeting minutes in stx-status | OpenDev Etherpad<https://etherpad.opendev.org/p/stx-status>: o ARM Support • Action: Scott Kamp to respond to Jackie/the mailing list to provide assistance/access to some arm machines • Action: Scott Little to respond regarding build questions • Cannot mirror more files on CENGN; will need to wait until we transition away from CENGN • Action: Scott Kamp to explore providing a hosting env temporarily So I think you might be able to help on Trevor’s and my questions: There would be 6 Arm servers contributed to our community for CI/CD (2 for build, 4 for testing of different deployment configs), what we want to know is: 1. What needs to be done for the CICD workflow setup and how? a. E.g. how to setup build servers and testing env? b. What CICD tools need to be setup? Zuul/Jenkins/etc. c. Are there any docs for these setups? 2. How many efforts and how long will it take? 3. Is there a CICD team or someone that can help on the setup or just providing useful information? Thanks, Jackie From: Trevor Tao <Trevor.Tao@arm.com<mailto:Trevor.Tao@arm.com>> Sent: Thursday, July 27, 2023 5:06 PM To: Geary, Stephen (Steve) <Steve.Geary@windriver.com<mailto:Steve.Geary@windriver.com>>; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Cc: Huang, Jie (Jackie) <Jackie.Huang@windriver.com<mailto:Jackie.Huang@windriver.com>> Subject: StarlingX CI/CD setup on Arm platform Hi Steve, StarlingX: From the OS distro meeting<https://urldefense.com/v3/__https:/etherpad.opendev.org/p/stx-distro-other__;!!AjveYdw8EvQ!e07vSdClUS6f5DohQ4EQeAEzsC4SnjZH2Vb3mjeau1FjCuAnkaGqf1_1vuARZRPcPxdw8dZJN-D3pMmWKHLvhCs$> yesterday, we knew that there would be 6 Arm servers contributed to our community to do the CI/CD related work. So we would like to ask who can help on the setup of CI/CD environment on the arm platform, and any other useful information/details for the CI/CD issue here. Thanks, Best Regards, Zijin Tao(Trevor Tao, 陶孜谨) ARM Electronic Technology (Shanghai) Co., Ltd 安谋电子科技（上海）有限公司 Building 11, Shanghai Busininess ParkⅢ , No.1016 Tianlin Rd, Minhang District, Shanghai, 200233 China 上海市闵行区田林路1016号科技绿洲三期2号楼10楼，200233 Cell: +86-153 7109 6192 IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Hi Scott / Davlet, As we discussed on the OS Distro call, a local CENGEN like server is needed for the local CI/CD setup, Could you please share the info of how to setup a CENGEN like server, specifically how to prepare the package mirror, we need to get it setup and prepare the mirror for ARM deb packages, and for the LAT-SDK for ARM as well. Thanks! Thanks, Jackie From: Huang, Jie (Jackie) <Jackie.Huang@windriver.com> Sent: Friday, August 18, 2023 1:33 PM To: Little, Scott <Scott.Little@windriver.com>; Panech, Davlet <Davlet.Panech@windriver.com>; Tianyu Li <Tianyu.Li@arm.com> Cc: starlingx-discuss@lists.starlingx.io; Scott Kamp <scott.kamp@xunison.com>; Trevor Tao <Trevor.Tao@arm.com>; Geary, Stephen (Steve) <Steve.Geary@windriver.com>; Wang, Linda <Linda.Wang@windriver.com> Subject: Re: [Starlingx-discuss] StarlingX CI/CD setup on Arm platform Hi Scott, Thanks for sharing, it’s very helpful. Thanks, Jackie From: Little, Scott <Scott.Little@windriver.com<mailto:Scott.Little@windriver.com>> Sent: Thursday, August 17, 2023 11:48 PM To: Huang, Jie (Jackie) <Jackie.Huang@windriver.com<mailto:Jackie.Huang@windriver.com>>; Panech, Davlet <Davlet.Panech@windriver.com<mailto:Davlet.Panech@windriver.com>>; Tianyu Li <Tianyu.Li@arm.com<mailto:Tianyu.Li@arm.com>> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>; Scott Kamp <scott.kamp@xunison.com<mailto:scott.kamp@xunison.com>>; Trevor Tao <Trevor.Tao@arm.com<mailto:Trevor.Tao@arm.com>>; Geary, Stephen (Steve) <Steve.Geary@windriver.com<mailto:Steve.Geary@windriver.com>>; Wang, Linda <Linda.Wang@windriver.com<mailto:Linda.Wang@windriver.com>> Subject: Re: StarlingX CI/CD setup on Arm platform There is also ... https://github.com/starlingx-staging/jenkins<https://urldefense.com/v3/__https:/github.com/starlingx-staging/jenkins__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6exWFUmA$> Which contains the current state of the StarlingX jenkins setup, less a few credentials. It includes 'Debian_build_server.txt' with crude documentation of how the server was setup. Scott On 2023-08-16 21:04, Huang, Jie (Jackie) wrote: Hi Davlet, Thanks for the detailed answers, which are very helpful, Tianyu and I will start by familiarizing ourselves and check if there are any more questions. Thanks, Jackie From: Panech, Davlet <Davlet.Panech@windriver.com><mailto:Davlet.Panech@windriver.com> Sent: Wednesday, August 16, 2023 11:30 PM To: Huang, Jie (Jackie) <Jackie.Huang@windriver.com><mailto:Jackie.Huang@windriver.com>; Little, Scott <Scott.Little@windriver.com><mailto:Scott.Little@windriver.com>; Tianyu Li <Tianyu.Li@arm.com><mailto:Tianyu.Li@arm.com>; Wang, Linda <Linda.Wang@windriver.com><mailto:Linda.Wang@windriver.com> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>; Scott Kamp <scott.kamp@xunison.com><mailto:scott.kamp@xunison.com>; Trevor Tao <Trevor.Tao@arm.com><mailto:Trevor.Tao@arm.com>; Geary, Stephen (Steve) <Steve.Geary@windriver.com><mailto:Steve.Geary@windriver.com> Subject: Re: StarlingX CI/CD setup on Arm platform Hi Jackie, CI/CD can be thought of as 2 somewhat disconnected parts - build & automated testing. I know little about test automation, the current build set up is outlined below. You would need a dedicated server: 1. Linux : we currently use Debian 11, but it shouldn't matter since most build steps run in docker containers 2. Docker 3. Kubernetes 4. Jenkins: integrated with k8s and able to run k8s pipelines 5. git 6. git-repo https://gerrit.googlesource.com/git-repo<https://urldefense.com/v3/__https:/gerrit.googlesource.com/git-repo__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6Y5TFnTg$> Other prerequisites: 1. You would need some DockerHub account to avoid DockerHub's rate limits when pulling images 2. Once your ARM set up is ready we can replace that with a controlled account that we manage in order to push the images to Docker Hub. Jenkins job definitions (Jenkinsfiles) are source-controlled here: https://opendev.org/starlingx/jenkins-pipelines<https://urldefense.com/v3/__https:/opendev.org/starlingx/jenkins-pipelines__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6EmOjdNg$> . You would need to define Jenkins jobs manually in the GUI, and point them to the corresponding Jenkinsfiles in this git repo. 1. The repo itself acts as a pipeline library in Jenkins with shared functions under vars/<https://urldefense.com/v3/__https:/opendev.org/starlingx/jenkins-pipelines/src/branch/master/vars__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6wk4ZyVg$> . The library must be defined at Jenkins' folder level. 2. pipelines/monolithic.Jenkinsfile<https://urldefense.com/v3/__https:/opendev.org/starlingx/jenkins-pipelines/src/branch/master/pipelines/monolithic.Jenkinsfile__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6qaNL9Is$> : top-level job that calls other jobs under parts/ 3. pipelines/parts/<https://urldefense.com/v3/__https:/opendev.org/starlingx/jenkins-pipelines/src/branch/master/pipelines/parts__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp616q0Kfk$> : directory containing sub-job Jenkinsfiles. You would create a Jenkins folder in the GUI called "parts" and define individual jobs pointing to individual Jenkinsfiles from this subdirectory in git repo 4. scripts/<https://urldefense.com/v3/__https:/opendev.org/starlingx/jenkins-pipelines/src/branch/master/scripts__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6mfhen9o$> : shell scripts that implement individual build steps (called by Jenkinsfiles) Required directories on the server, owned by jenkins user: 1. /localdisk/designer/jenkins/ root for per-project build areas 2. /localdisk/loadbuild/jenkins/ root for per-project build archives At first execution the monolithic job will fail, but create an example configuration file in the project build area that you are expected to copy & edit: 1. /localdisk/designer/jenkins/$PROJECT/build.cond.example There are many options with comments in it; the example file is also source controlled under scripts/templates/build.conf.example.in<https://urldefense.com/v3/__https:/opendev.org/starlingx/jenkins-pipelines/src/branch/master/scripts/templates/build.conf.example.in__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6S_cks2U$> Builds are slow and would greatly benefit from fast storage (SSDs etc). Please start by familiarizing yourself with this repository and ping me if you have any more questions. Hope this helps, D. ________________________________ From: Huang, Jie (Jackie) <Jackie.Huang@windriver.com<mailto:Jackie.Huang@windriver.com>> Sent: August 9, 2023 10:28 AM To: Little, Scott <Scott.Little@windriver.com<mailto:Scott.Little@windriver.com>>; Panech, Davlet <Davlet.Panech@windriver.com<mailto:Davlet.Panech@windriver.com>>; Tianyu Li <Tianyu.Li@arm.com<mailto:Tianyu.Li@arm.com>>; Wang, Linda <Linda.Wang@windriver.com<mailto:Linda.Wang@windriver.com>> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> <starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>>; Scott Kamp <scott.kamp@xunison.com<mailto:scott.kamp@xunison.com>>; Trevor Tao <Trevor.Tao@arm.com<mailto:Trevor.Tao@arm.com>>; Geary, Stephen (Steve) <Steve.Geary@windriver.com<mailto:Steve.Geary@windriver.com>> Subject: RE: StarlingX CI/CD setup on Arm platform Hi Scott and Davlet, As mentioned in the Multi-OS call, I’m re-sending this with all the questions about StarlingX CI/CD, please help answer as many as you can, thanks! There would be 6 Arm servers contributed to our community for CI/CD (2 for build, 4 for testing of different deployment configs), what we want to know is: 1. What’s the current status of StarlingX CICD for x86-64? 2. What needs to be done for the CICD workflow setup and how after the 6 ARM servers are ready and hosted in the lab? a. E.g. how to setup build servers and testing env? b. What CICD tools need to be setup? Zuul/Jenkins/etc. c. Are there any docs or info of the steps for these setups? 3. How many efforts and how long will it take? 4. Can this only be done by specific person? Or anyone have permission to access to the servers? If the later, I think Tianyu from Arm team and I can also help on that if we got enough information. Tianyu, please add if you have any other questions. Thanks, Jackie From: Huang, Jie (Jackie) Sent: Monday, July 31, 2023 12:57 PM To: Trevor Tao <Trevor.Tao@arm.com<mailto:Trevor.Tao@arm.com>>; Geary, Stephen (Steve) <Steve.Geary@windriver.com<mailto:Steve.Geary@windriver.com>>; Little, Scott <Scott.Little@windriver.com<mailto:Scott.Little@windriver.com>>; Scott Kamp <scott.kamp@xunison.com<mailto:scott.kamp@xunison.com>> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: RE: StarlingX CI/CD setup on Arm platform Hi, Scott Kamp and Scott Little, I found the following actions in the meeting minutes in stx-status | OpenDev Etherpad<https://urldefense.com/v3/__https:/etherpad.opendev.org/p/stx-status__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6bH3qX9g$>: o ARM Support • Action: Scott Kamp to respond to Jackie/the mailing list to provide assistance/access to some arm machines • Action: Scott Little to respond regarding build questions • Cannot mirror more files on CENGN; will need to wait until we transition away from CENGN • Action: Scott Kamp to explore providing a hosting env temporarily So I think you might be able to help on Trevor’s and my questions: There would be 6 Arm servers contributed to our community for CI/CD (2 for build, 4 for testing of different deployment configs), what we want to know is: 1. What needs to be done for the CICD workflow setup and how? a. E.g. how to setup build servers and testing env? b. What CICD tools need to be setup? Zuul/Jenkins/etc. c. Are there any docs for these setups? 2. How many efforts and how long will it take? 3. Is there a CICD team or someone that can help on the setup or just providing useful information? Thanks, Jackie From: Trevor Tao <Trevor.Tao@arm.com<mailto:Trevor.Tao@arm.com>> Sent: Thursday, July 27, 2023 5:06 PM To: Geary, Stephen (Steve) <Steve.Geary@windriver.com<mailto:Steve.Geary@windriver.com>>; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Cc: Huang, Jie (Jackie) <Jackie.Huang@windriver.com<mailto:Jackie.Huang@windriver.com>> Subject: StarlingX CI/CD setup on Arm platform Hi Steve, StarlingX: From the OS distro meeting<https://urldefense.com/v3/__https:/etherpad.opendev.org/p/stx-distro-other__;!!AjveYdw8EvQ!e07vSdClUS6f5DohQ4EQeAEzsC4SnjZH2Vb3mjeau1FjCuAnkaGqf1_1vuARZRPcPxdw8dZJN-D3pMmWKHLvhCs$> yesterday, we knew that there would be 6 Arm servers contributed to our community to do the CI/CD related work. So we would like to ask who can help on the setup of CI/CD environment on the arm platform, and any other useful information/details for the CI/CD issue here. Thanks, Best Regards, Zijin Tao(Trevor Tao, 陶孜谨) ARM Electronic Technology (Shanghai) Co., Ltd 安谋电子科技（上海）有限公司 Building 11, Shanghai Busininess ParkⅢ , No.1016 Tianlin Rd, Minhang District, Shanghai, 200233 China 上海市闵行区田林路1016号科技绿洲三期2号楼10楼，200233 Cell: +86-153 7109 6192 IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Thanks Davlet and Scott for information, I tried to setup the Jenkins pipeline jobs on local Arm server, after applied Jackie's Arm porting patches[0] with environment variable PATCH_LIST, the build job passed some stage and failed at 'download-prerequisites' with following log[1], looks like the downloader still trying to download some x86 packages. Any idea what could cause this? I have another general question regarding testing stage in CI/CD process, What is the minimum automation test case should be done after image is done? If Davlet and Scott are not familiar with this area, who can help on this? FYI, this is our plan to support StartingX CI/CD on Arm. 1. Reproduce the build CI/CD on arm in our internal lab 2. Verify procedure to trigger test cases for CI/CD on arm machine (internal lab) with few selected cases 3. Discuss with community to merge code changes to enable CI/CD on arm in community lab once arm servers are in place. 4. Fix build and “minimal test set” errors by all maintainers of community 5. Enable voting right by community per necessary [0] https://review.opendev.org/q/topic:arm64/20230725-stx-master-native [1] downloaderERROR "The cache has no package named 'gcc-alpha-linux-gnu'" Thanks, Tianyu ________________________________ From: Huang, Jie (Jackie) <Jackie.Huang@windriver.com> Sent: Thursday, August 24, 2023 8:57 PM To: Huang, Jie (Jackie) <Jackie.Huang@windriver.com>; Little, Scott <Scott.Little@windriver.com>; Panech, Davlet <Davlet.Panech@windriver.com>; Tianyu Li <Tianyu.Li@arm.com> Cc: starlingx-discuss@lists.starlingx.io <starlingx-discuss@lists.starlingx.io>; Scott Kamp <scott.kamp@xunison.com>; Trevor Tao <Trevor.Tao@arm.com>; Geary, Stephen (Steve) <Steve.Geary@windriver.com>; Wang, Linda <Linda.Wang@windriver.com> Subject: RE: StarlingX CI/CD setup on Arm platform Hi Scott / Davlet, As we discussed on the OS Distro call, a local CENGEN like server is needed for the local CI/CD setup, Could you please share the info of how to setup a CENGEN like server, specifically how to prepare the package mirror, we need to get it setup and prepare the mirror for ARM deb packages, and for the LAT-SDK for ARM as well. Thanks! Thanks, Jackie From: Huang, Jie (Jackie) <Jackie.Huang@windriver.com> Sent: Friday, August 18, 2023 1:33 PM To: Little, Scott <Scott.Little@windriver.com>; Panech, Davlet <Davlet.Panech@windriver.com>; Tianyu Li <Tianyu.Li@arm.com> Cc: starlingx-discuss@lists.starlingx.io; Scott Kamp <scott.kamp@xunison.com>; Trevor Tao <Trevor.Tao@arm.com>; Geary, Stephen (Steve) <Steve.Geary@windriver.com>; Wang, Linda <Linda.Wang@windriver.com> Subject: Re: [Starlingx-discuss] StarlingX CI/CD setup on Arm platform Hi Scott, Thanks for sharing, it’s very helpful. Thanks, Jackie From: Little, Scott <Scott.Little@windriver.com<mailto:Scott.Little@windriver.com>> Sent: Thursday, August 17, 2023 11:48 PM To: Huang, Jie (Jackie) <Jackie.Huang@windriver.com<mailto:Jackie.Huang@windriver.com>>; Panech, Davlet <Davlet.Panech@windriver.com<mailto:Davlet.Panech@windriver.com>>; Tianyu Li <Tianyu.Li@arm.com<mailto:Tianyu.Li@arm.com>> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>; Scott Kamp <scott.kamp@xunison.com<mailto:scott.kamp@xunison.com>>; Trevor Tao <Trevor.Tao@arm.com<mailto:Trevor.Tao@arm.com>>; Geary, Stephen (Steve) <Steve.Geary@windriver.com<mailto:Steve.Geary@windriver.com>>; Wang, Linda <Linda.Wang@windriver.com<mailto:Linda.Wang@windriver.com>> Subject: Re: StarlingX CI/CD setup on Arm platform There is also ... https://github.com/starlingx-staging/jenkins<https://urldefense.com/v3/__https:/github.com/starlingx-staging/jenkins__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6exWFUmA$> Which contains the current state of the StarlingX jenkins setup, less a few credentials. It includes 'Debian_build_server.txt' with crude documentation of how the server was setup. Scott On 2023-08-16 21:04, Huang, Jie (Jackie) wrote: Hi Davlet, Thanks for the detailed answers, which are very helpful, Tianyu and I will start by familiarizing ourselves and check if there are any more questions. Thanks, Jackie From: Panech, Davlet <Davlet.Panech@windriver.com><mailto:Davlet.Panech@windriver.com> Sent: Wednesday, August 16, 2023 11:30 PM To: Huang, Jie (Jackie) <Jackie.Huang@windriver.com><mailto:Jackie.Huang@windriver.com>; Little, Scott <Scott.Little@windriver.com><mailto:Scott.Little@windriver.com>; Tianyu Li <Tianyu.Li@arm.com><mailto:Tianyu.Li@arm.com>; Wang, Linda <Linda.Wang@windriver.com><mailto:Linda.Wang@windriver.com> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>; Scott Kamp <scott.kamp@xunison.com><mailto:scott.kamp@xunison.com>; Trevor Tao <Trevor.Tao@arm.com><mailto:Trevor.Tao@arm.com>; Geary, Stephen (Steve) <Steve.Geary@windriver.com><mailto:Steve.Geary@windriver.com> Subject: Re: StarlingX CI/CD setup on Arm platform Hi Jackie, CI/CD can be thought of as 2 somewhat disconnected parts - build & automated testing. I know little about test automation, the current build set up is outlined below. You would need a dedicated server: 1. Linux : we currently use Debian 11, but it shouldn't matter since most build steps run in docker containers 2. Docker 3. Kubernetes 4. Jenkins: integrated with k8s and able to run k8s pipelines 5. git 6. git-repo https://gerrit.googlesource.com/git-repo<https://urldefense.com/v3/__https:/gerrit.googlesource.com/git-repo__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6Y5TFnTg$> Other prerequisites: 1. You would need some DockerHub account to avoid DockerHub's rate limits when pulling images 2. Once your ARM set up is ready we can replace that with a controlled account that we manage in order to push the images to Docker Hub. Jenkins job definitions (Jenkinsfiles) are source-controlled here: https://opendev.org/starlingx/jenkins-pipelines<https://urldefense.com/v3/__https:/opendev.org/starlingx/jenkins-pipelines__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6EmOjdNg$> . You would need to define Jenkins jobs manually in the GUI, and point them to the corresponding Jenkinsfiles in this git repo. 1. The repo itself acts as a pipeline library in Jenkins with shared functions under vars/<https://urldefense.com/v3/__https:/opendev.org/starlingx/jenkins-pipelines/src/branch/master/vars__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6wk4ZyVg$> . The library must be defined at Jenkins' folder level. 2. pipelines/monolithic.Jenkinsfile<https://urldefense.com/v3/__https:/opendev.org/starlingx/jenkins-pipelines/src/branch/master/pipelines/monolithic.Jenkinsfile__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6qaNL9Is$> : top-level job that calls other jobs under parts/ 3. pipelines/parts/<https://urldefense.com/v3/__https:/opendev.org/starlingx/jenkins-pipelines/src/branch/master/pipelines/parts__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp616q0Kfk$> : directory containing sub-job Jenkinsfiles. You would create a Jenkins folder in the GUI called "parts" and define individual jobs pointing to individual Jenkinsfiles from this subdirectory in git repo 4. scripts/<https://urldefense.com/v3/__https:/opendev.org/starlingx/jenkins-pipelines/src/branch/master/scripts__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6mfhen9o$> : shell scripts that implement individual build steps (called by Jenkinsfiles) Required directories on the server, owned by jenkins user: 1. /localdisk/designer/jenkins/ root for per-project build areas 2. /localdisk/loadbuild/jenkins/ root for per-project build archives At first execution the monolithic job will fail, but create an example configuration file in the project build area that you are expected to copy & edit: 1. /localdisk/designer/jenkins/$PROJECT/build.cond.example There are many options with comments in it; the example file is also source controlled under scripts/templates/build.conf.example.in<https://urldefense.com/v3/__https:/opendev.org/starlingx/jenkins-pipelines/src/branch/master/scripts/templates/build.conf.example.in__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6S_cks2U$> Builds are slow and would greatly benefit from fast storage (SSDs etc). Please start by familiarizing yourself with this repository and ping me if you have any more questions. Hope this helps, D. ________________________________ From: Huang, Jie (Jackie) <Jackie.Huang@windriver.com<mailto:Jackie.Huang@windriver.com>> Sent: August 9, 2023 10:28 AM To: Little, Scott <Scott.Little@windriver.com<mailto:Scott.Little@windriver.com>>; Panech, Davlet <Davlet.Panech@windriver.com<mailto:Davlet.Panech@windriver.com>>; Tianyu Li <Tianyu.Li@arm.com<mailto:Tianyu.Li@arm.com>>; Wang, Linda <Linda.Wang@windriver.com<mailto:Linda.Wang@windriver.com>> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> <starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>>; Scott Kamp <scott.kamp@xunison.com<mailto:scott.kamp@xunison.com>>; Trevor Tao <Trevor.Tao@arm.com<mailto:Trevor.Tao@arm.com>>; Geary, Stephen (Steve) <Steve.Geary@windriver.com<mailto:Steve.Geary@windriver.com>> Subject: RE: StarlingX CI/CD setup on Arm platform Hi Scott and Davlet, As mentioned in the Multi-OS call, I’m re-sending this with all the questions about StarlingX CI/CD, please help answer as many as you can, thanks! There would be 6 Arm servers contributed to our community for CI/CD (2 for build, 4 for testing of different deployment configs), what we want to know is: 1. What’s the current status of StarlingX CICD for x86-64? 2. What needs to be done for the CICD workflow setup and how after the 6 ARM servers are ready and hosted in the lab? a. E.g. how to setup build servers and testing env? b. What CICD tools need to be setup? Zuul/Jenkins/etc. c. Are there any docs or info of the steps for these setups? 3. How many efforts and how long will it take? 4. Can this only be done by specific person? Or anyone have permission to access to the servers? If the later, I think Tianyu from Arm team and I can also help on that if we got enough information. Tianyu, please add if you have any other questions. Thanks, Jackie From: Huang, Jie (Jackie) Sent: Monday, July 31, 2023 12:57 PM To: Trevor Tao <Trevor.Tao@arm.com<mailto:Trevor.Tao@arm.com>>; Geary, Stephen (Steve) <Steve.Geary@windriver.com<mailto:Steve.Geary@windriver.com>>; Little, Scott <Scott.Little@windriver.com<mailto:Scott.Little@windriver.com>>; Scott Kamp <scott.kamp@xunison.com<mailto:scott.kamp@xunison.com>> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: RE: StarlingX CI/CD setup on Arm platform Hi, Scott Kamp and Scott Little, I found the following actions in the meeting minutes in stx-status | OpenDev Etherpad<https://urldefense.com/v3/__https:/etherpad.opendev.org/p/stx-status__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6bH3qX9g$>: o ARM Support * Action: Scott Kamp to respond to Jackie/the mailing list to provide assistance/access to some arm machines * Action: Scott Little to respond regarding build questions * Cannot mirror more files on CENGN; will need to wait until we transition away from CENGN * Action: Scott Kamp to explore providing a hosting env temporarily So I think you might be able to help on Trevor’s and my questions: There would be 6 Arm servers contributed to our community for CI/CD (2 for build, 4 for testing of different deployment configs), what we want to know is: 1. What needs to be done for the CICD workflow setup and how? a. E.g. how to setup build servers and testing env? b. What CICD tools need to be setup? Zuul/Jenkins/etc. c. Are there any docs for these setups? 2. How many efforts and how long will it take? 3. Is there a CICD team or someone that can help on the setup or just providing useful information? Thanks, Jackie From: Trevor Tao <Trevor.Tao@arm.com<mailto:Trevor.Tao@arm.com>> Sent: Thursday, July 27, 2023 5:06 PM To: Geary, Stephen (Steve) <Steve.Geary@windriver.com<mailto:Steve.Geary@windriver.com>>; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Cc: Huang, Jie (Jackie) <Jackie.Huang@windriver.com<mailto:Jackie.Huang@windriver.com>> Subject: StarlingX CI/CD setup on Arm platform Hi Steve, StarlingX: From the OS distro meeting<https://urldefense.com/v3/__https:/etherpad.opendev.org/p/stx-distro-other__;!!AjveYdw8EvQ!e07vSdClUS6f5DohQ4EQeAEzsC4SnjZH2Vb3mjeau1FjCuAnkaGqf1_1vuARZRPcPxdw8dZJN-D3pMmWKHLvhCs$> yesterday, we knew that there would be 6 Arm servers contributed to our community to do the CI/CD related work. So we would like to ask who can help on the setup of CI/CD environment on the arm platform, and any other useful information/details for the CI/CD issue here. Thanks, Best Regards, Zijin Tao(Trevor Tao, 陶孜谨) ARM Electronic Technology (Shanghai) Co., Ltd 安谋电子科技（上海）有限公司 Building 11, Shanghai Busininess ParkⅢ , No.1016 Tianlin Rd, Minhang District, Shanghai, 200233 China 上海市闵行区田林路1016号科技绿洲三期2号楼10楼，200233 Cell: +86-153 7109 6192 IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
+ Steve in the loop. Hi Steve, As you mentioned that Davlet or Scott may be on vacation, would you please help to see who could answer Tianyu’s question regarding the test cases? Thanks Chris From: Tianyu Li <Tianyu.Li@arm.com> Sent: Wednesday, August 30, 2023 3:32 PM To: Huang, Jie (Jackie) <Jackie.Huang@windriver.com>; Little, Scott <Scott.Little@windriver.com>; Panech, Davlet <Davlet.Panech@windriver.com> Cc: starlingx-discuss@lists.starlingx.io; Scott Kamp <scott.kamp@xunison.com>; Chris Wang <Chris.Wang@arm.com>; Lijian Zhang <Lijian.Zhang@arm.com>; Geary, Stephen (Steve) <Steve.Geary@windriver.com>; Wang, Linda <Linda.Wang@windriver.com>; nd <nd@arm.com> Subject: Re: StarlingX CI/CD setup on Arm platform Thanks Davlet and Scott for information, I tried to setup the Jenkins pipeline jobs on local Arm server, after applied Jackie's Arm porting patches[0] with environment variable PATCH_LIST, the build job passed some stage and failed at 'download-prerequisites' with following log[1], looks like the downloader still trying to download some x86 packages. Any idea what could cause this? I have another general question regarding testing stage in CI/CD process, What is the minimum automation test case should be done after image is done? If Davlet and Scott are not familiar with this area, who can help on this? FYI, this is our plan to support StartingX CI/CD on Arm. 1. Reproduce the build CI/CD on arm in our internal lab 2. Verify procedure to trigger test cases for CI/CD on arm machine (internal lab) with few selected cases 3. Discuss with community to merge code changes to enable CI/CD on arm in community lab once arm servers are in place. 4. Fix build and “minimal test set” errors by all maintainers of community 5. Enable voting right by community per necessary [0] https://review.opendev.org/q/topic:arm64/20230725-stx-master-native [1] downloaderERROR "The cache has no package named 'gcc-alpha-linux-gnu'" Thanks, Tianyu ________________________________ From: Huang, Jie (Jackie) <Jackie.Huang@windriver.com<mailto:Jackie.Huang@windriver.com>> Sent: Thursday, August 24, 2023 8:57 PM To: Huang, Jie (Jackie) <Jackie.Huang@windriver.com<mailto:Jackie.Huang@windriver.com>>; Little, Scott <Scott.Little@windriver.com<mailto:Scott.Little@windriver.com>>; Panech, Davlet <Davlet.Panech@windriver.com<mailto:Davlet.Panech@windriver.com>>; Tianyu Li <Tianyu.Li@arm.com<mailto:Tianyu.Li@arm.com>> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> <starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>>; Scott Kamp <scott.kamp@xunison.com<mailto:scott.kamp@xunison.com>>; Trevor Tao <Trevor.Tao@arm.com<mailto:Trevor.Tao@arm.com>>; Geary, Stephen (Steve) <Steve.Geary@windriver.com<mailto:Steve.Geary@windriver.com>>; Wang, Linda <Linda.Wang@windriver.com<mailto:Linda.Wang@windriver.com>> Subject: RE: StarlingX CI/CD setup on Arm platform Hi Scott / Davlet, As we discussed on the OS Distro call, a local CENGEN like server is needed for the local CI/CD setup, Could you please share the info of how to setup a CENGEN like server, specifically how to prepare the package mirror, we need to get it setup and prepare the mirror for ARM deb packages, and for the LAT-SDK for ARM as well. Thanks! Thanks, Jackie From: Huang, Jie (Jackie) <Jackie.Huang@windriver.com<mailto:Jackie.Huang@windriver.com>> Sent: Friday, August 18, 2023 1:33 PM To: Little, Scott <Scott.Little@windriver.com<mailto:Scott.Little@windriver.com>>; Panech, Davlet <Davlet.Panech@windriver.com<mailto:Davlet.Panech@windriver.com>>; Tianyu Li <Tianyu.Li@arm.com<mailto:Tianyu.Li@arm.com>> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>; Scott Kamp <scott.kamp@xunison.com<mailto:scott.kamp@xunison.com>>; Trevor Tao <Trevor.Tao@arm.com<mailto:Trevor.Tao@arm.com>>; Geary, Stephen (Steve) <Steve.Geary@windriver.com<mailto:Steve.Geary@windriver.com>>; Wang, Linda <Linda.Wang@windriver.com<mailto:Linda.Wang@windriver.com>> Subject: Re: [Starlingx-discuss] StarlingX CI/CD setup on Arm platform Hi Scott, Thanks for sharing, it’s very helpful. Thanks, Jackie From: Little, Scott <Scott.Little@windriver.com<mailto:Scott.Little@windriver.com>> Sent: Thursday, August 17, 2023 11:48 PM To: Huang, Jie (Jackie) <Jackie.Huang@windriver.com<mailto:Jackie.Huang@windriver.com>>; Panech, Davlet <Davlet.Panech@windriver.com<mailto:Davlet.Panech@windriver.com>>; Tianyu Li <Tianyu.Li@arm.com<mailto:Tianyu.Li@arm.com>> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>; Scott Kamp <scott.kamp@xunison.com<mailto:scott.kamp@xunison.com>>; Trevor Tao <Trevor.Tao@arm.com<mailto:Trevor.Tao@arm.com>>; Geary, Stephen (Steve) <Steve.Geary@windriver.com<mailto:Steve.Geary@windriver.com>>; Wang, Linda <Linda.Wang@windriver.com<mailto:Linda.Wang@windriver.com>> Subject: Re: StarlingX CI/CD setup on Arm platform There is also ... https://github.com/starlingx-staging/jenkins<https://urldefense.com/v3/__https:/github.com/starlingx-staging/jenkins__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6exWFUmA$> Which contains the current state of the StarlingX jenkins setup, less a few credentials. It includes 'Debian_build_server.txt' with crude documentation of how the server was setup. Scott On 2023-08-16 21:04, Huang, Jie (Jackie) wrote: Hi Davlet, Thanks for the detailed answers, which are very helpful, Tianyu and I will start by familiarizing ourselves and check if there are any more questions. Thanks, Jackie From: Panech, Davlet <Davlet.Panech@windriver.com><mailto:Davlet.Panech@windriver.com> Sent: Wednesday, August 16, 2023 11:30 PM To: Huang, Jie (Jackie) <Jackie.Huang@windriver.com><mailto:Jackie.Huang@windriver.com>; Little, Scott <Scott.Little@windriver.com><mailto:Scott.Little@windriver.com>; Tianyu Li <Tianyu.Li@arm.com><mailto:Tianyu.Li@arm.com>; Wang, Linda <Linda.Wang@windriver.com><mailto:Linda.Wang@windriver.com> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>; Scott Kamp <scott.kamp@xunison.com><mailto:scott.kamp@xunison.com>; Trevor Tao <Trevor.Tao@arm.com><mailto:Trevor.Tao@arm.com>; Geary, Stephen (Steve) <Steve.Geary@windriver.com><mailto:Steve.Geary@windriver.com> Subject: Re: StarlingX CI/CD setup on Arm platform Hi Jackie, CI/CD can be thought of as 2 somewhat disconnected parts - build & automated testing. I know little about test automation, the current build set up is outlined below. You would need a dedicated server: 1. Linux : we currently use Debian 11, but it shouldn't matter since most build steps run in docker containers 2. Docker 3. Kubernetes 4. Jenkins: integrated with k8s and able to run k8s pipelines 5. git 6. git-repo https://gerrit.googlesource.com/git-repo<https://urldefense.com/v3/__https:/gerrit.googlesource.com/git-repo__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6Y5TFnTg$> Other prerequisites: 1. You would need some DockerHub account to avoid DockerHub's rate limits when pulling images 2. Once your ARM set up is ready we can replace that with a controlled account that we manage in order to push the images to Docker Hub. Jenkins job definitions (Jenkinsfiles) are source-controlled here: https://opendev.org/starlingx/jenkins-pipelines<https://urldefense.com/v3/__https:/opendev.org/starlingx/jenkins-pipelines__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6EmOjdNg$> . You would need to define Jenkins jobs manually in the GUI, and point them to the corresponding Jenkinsfiles in this git repo. 1. The repo itself acts as a pipeline library in Jenkins with shared functions under vars/<https://urldefense.com/v3/__https:/opendev.org/starlingx/jenkins-pipelines/src/branch/master/vars__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6wk4ZyVg$> . The library must be defined at Jenkins' folder level. 2. pipelines/monolithic.Jenkinsfile<https://urldefense.com/v3/__https:/opendev.org/starlingx/jenkins-pipelines/src/branch/master/pipelines/monolithic.Jenkinsfile__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6qaNL9Is$> : top-level job that calls other jobs under parts/ 3. pipelines/parts/<https://urldefense.com/v3/__https:/opendev.org/starlingx/jenkins-pipelines/src/branch/master/pipelines/parts__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp616q0Kfk$> : directory containing sub-job Jenkinsfiles. You would create a Jenkins folder in the GUI called "parts" and define individual jobs pointing to individual Jenkinsfiles from this subdirectory in git repo 4. scripts/<https://urldefense.com/v3/__https:/opendev.org/starlingx/jenkins-pipelines/src/branch/master/scripts__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6mfhen9o$> : shell scripts that implement individual build steps (called by Jenkinsfiles) Required directories on the server, owned by jenkins user: 1. /localdisk/designer/jenkins/ root for per-project build areas 2. /localdisk/loadbuild/jenkins/ root for per-project build archives At first execution the monolithic job will fail, but create an example configuration file in the project build area that you are expected to copy & edit: 1. /localdisk/designer/jenkins/$PROJECT/build.cond.example There are many options with comments in it; the example file is also source controlled under scripts/templates/build.conf.example.in<https://urldefense.com/v3/__https:/opendev.org/starlingx/jenkins-pipelines/src/branch/master/scripts/templates/build.conf.example.in__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6S_cks2U$> Builds are slow and would greatly benefit from fast storage (SSDs etc). Please start by familiarizing yourself with this repository and ping me if you have any more questions. Hope this helps, D. ________________________________ From: Huang, Jie (Jackie) <Jackie.Huang@windriver.com<mailto:Jackie.Huang@windriver.com>> Sent: August 9, 2023 10:28 AM To: Little, Scott <Scott.Little@windriver.com<mailto:Scott.Little@windriver.com>>; Panech, Davlet <Davlet.Panech@windriver.com<mailto:Davlet.Panech@windriver.com>>; Tianyu Li <Tianyu.Li@arm.com<mailto:Tianyu.Li@arm.com>>; Wang, Linda <Linda.Wang@windriver.com<mailto:Linda.Wang@windriver.com>> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> <starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>>; Scott Kamp <scott.kamp@xunison.com<mailto:scott.kamp@xunison.com>>; Trevor Tao <Trevor.Tao@arm.com<mailto:Trevor.Tao@arm.com>>; Geary, Stephen (Steve) <Steve.Geary@windriver.com<mailto:Steve.Geary@windriver.com>> Subject: RE: StarlingX CI/CD setup on Arm platform Hi Scott and Davlet, As mentioned in the Multi-OS call, I’m re-sending this with all the questions about StarlingX CI/CD, please help answer as many as you can, thanks! There would be 6 Arm servers contributed to our community for CI/CD (2 for build, 4 for testing of different deployment configs), what we want to know is: 1. What’s the current status of StarlingX CICD for x86-64? 2. What needs to be done for the CICD workflow setup and how after the 6 ARM servers are ready and hosted in the lab? a. E.g. how to setup build servers and testing env? b. What CICD tools need to be setup? Zuul/Jenkins/etc. c. Are there any docs or info of the steps for these setups? 3. How many efforts and how long will it take? 4. Can this only be done by specific person? Or anyone have permission to access to the servers? If the later, I think Tianyu from Arm team and I can also help on that if we got enough information. Tianyu, please add if you have any other questions. Thanks, Jackie From: Huang, Jie (Jackie) Sent: Monday, July 31, 2023 12:57 PM To: Trevor Tao <Trevor.Tao@arm.com<mailto:Trevor.Tao@arm.com>>; Geary, Stephen (Steve) <Steve.Geary@windriver.com<mailto:Steve.Geary@windriver.com>>; Little, Scott <Scott.Little@windriver.com<mailto:Scott.Little@windriver.com>>; Scott Kamp <scott.kamp@xunison.com<mailto:scott.kamp@xunison.com>> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: RE: StarlingX CI/CD setup on Arm platform Hi, Scott Kamp and Scott Little, I found the following actions in the meeting minutes in stx-status | OpenDev Etherpad<https://urldefense.com/v3/__https:/etherpad.opendev.org/p/stx-status__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6bH3qX9g$>: o ARM Support • Action: Scott Kamp to respond to Jackie/the mailing list to provide assistance/access to some arm machines • Action: Scott Little to respond regarding build questions • Cannot mirror more files on CENGN; will need to wait until we transition away from CENGN • Action: Scott Kamp to explore providing a hosting env temporarily So I think you might be able to help on Trevor’s and my questions: There would be 6 Arm servers contributed to our community for CI/CD (2 for build, 4 for testing of different deployment configs), what we want to know is: 1. What needs to be done for the CICD workflow setup and how? a. E.g. how to setup build servers and testing env? b. What CICD tools need to be setup? Zuul/Jenkins/etc. c. Are there any docs for these setups? 2. How many efforts and how long will it take? 3. Is there a CICD team or someone that can help on the setup or just providing useful information? Thanks, Jackie From: Trevor Tao <Trevor.Tao@arm.com<mailto:Trevor.Tao@arm.com>> Sent: Thursday, July 27, 2023 5:06 PM To: Geary, Stephen (Steve) <Steve.Geary@windriver.com<mailto:Steve.Geary@windriver.com>>; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Cc: Huang, Jie (Jackie) <Jackie.Huang@windriver.com<mailto:Jackie.Huang@windriver.com>> Subject: StarlingX CI/CD setup on Arm platform Hi Steve, StarlingX: From the OS distro meeting<https://urldefense.com/v3/__https:/etherpad.opendev.org/p/stx-distro-other__;!!AjveYdw8EvQ!e07vSdClUS6f5DohQ4EQeAEzsC4SnjZH2Vb3mjeau1FjCuAnkaGqf1_1vuARZRPcPxdw8dZJN-D3pMmWKHLvhCs$> yesterday, we knew that there would be 6 Arm servers contributed to our community to do the CI/CD related work. So we would like to ask who can help on the setup of CI/CD environment on the arm platform, and any other useful information/details for the CI/CD issue here. Thanks, Best Regards, Zijin Tao(Trevor Tao, 陶孜谨) ARM Electronic Technology (Shanghai) Co., Ltd 安谋电子科技（上海）有限公司 Building 11, Shanghai Busininess ParkⅢ , No.1016 Tianlin Rd, Minhang District, Shanghai, 200233 China 上海市闵行区田林路1016号科技绿洲三期2号楼10楼，200233 Cell: +86-153 7109 6192 IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Hi, To have a better understanding process of the CI/CD setup on Arm platform, I summarize and come up with the following simple topology diagram. With action items list below. Welcome any comments and suggestions. The proposed topology [cid:5c2ebdb3-30c3-4019-b6b0-62c3feaa1be2] Work steps 1. Reproduce the build CI/CD on arm in our internal lab 2. Verify procedure to trigger test cases for CI/CD on arm machine (internal lab) with few selected cases 3. Discuss with community to merge code changes to enable CI/CD on arm in community lab once arm servers are in place. 4. Fix build and “minimal test set” errors by all maintainers of community 5. Enable voting right by community per necessary Current status I am trying step 1, build ISO image with Jackie's porting patch series in our internal lab to understand the build process. The previous build packages error is because I missed some patches from Jackie. I still need to figure out an error during generating ISO image with Jackie. The auto test part in step 2 still needs some input. What is test repo used? What is the minimal test set? Any quick get started guide? Thanks, Tianyu ________________________________ From: Chris Wang <Chris.Wang@arm.com> Sent: Friday, September 8, 2023 10:06 AM To: Tianyu Li <Tianyu.Li@arm.com>; Huang, Jie (Jackie) <Jackie.Huang@windriver.com>; Little, Scott <Scott.Little@windriver.com>; Panech, Davlet <Davlet.Panech@windriver.com>; Geary, Stephen (Steve) <Steve.Geary@windriver.com> Cc: starlingx-discuss@lists.starlingx.io <starlingx-discuss@lists.starlingx.io>; Scott Kamp <scott.kamp@xunison.com>; Lijian Zhang <Lijian.Zhang@arm.com>; Geary, Stephen (Steve) <Steve.Geary@windriver.com>; Wang, Linda <Linda.Wang@windriver.com>; nd <nd@arm.com> Subject: RE: StarlingX CI/CD setup on Arm platform + Steve in the loop. Hi Steve, As you mentioned that Davlet or Scott may be on vacation, would you please help to see who could answer Tianyu’s question regarding the test cases? Thanks Chris From: Tianyu Li <Tianyu.Li@arm.com> Sent: Wednesday, August 30, 2023 3:32 PM To: Huang, Jie (Jackie) <Jackie.Huang@windriver.com>; Little, Scott <Scott.Little@windriver.com>; Panech, Davlet <Davlet.Panech@windriver.com> Cc: starlingx-discuss@lists.starlingx.io; Scott Kamp <scott.kamp@xunison.com>; Chris Wang <Chris.Wang@arm.com>; Lijian Zhang <Lijian.Zhang@arm.com>; Geary, Stephen (Steve) <Steve.Geary@windriver.com>; Wang, Linda <Linda.Wang@windriver.com>; nd <nd@arm.com> Subject: Re: StarlingX CI/CD setup on Arm platform Thanks Davlet and Scott for information, I tried to setup the Jenkins pipeline jobs on local Arm server, after applied Jackie's Arm porting patches[0] with environment variable PATCH_LIST, the build job passed some stage and failed at 'download-prerequisites' with following log[1], looks like the downloader still trying to download some x86 packages. Any idea what could cause this? I have another general question regarding testing stage in CI/CD process, What is the minimum automation test case should be done after image is done? If Davlet and Scott are not familiar with this area, who can help on this? FYI, this is our plan to support StartingX CI/CD on Arm. 1. Reproduce the build CI/CD on arm in our internal lab 2. Verify procedure to trigger test cases for CI/CD on arm machine (internal lab) with few selected cases 3. Discuss with community to merge code changes to enable CI/CD on arm in community lab once arm servers are in place. 4. Fix build and “minimal test set” errors by all maintainers of community 5. Enable voting right by community per necessary [0] https://review.opendev.org/q/topic:arm64/20230725-stx-master-native [1] downloaderERROR "The cache has no package named 'gcc-alpha-linux-gnu'" Thanks, Tianyu ________________________________ From: Huang, Jie (Jackie) <Jackie.Huang@windriver.com<mailto:Jackie.Huang@windriver.com>> Sent: Thursday, August 24, 2023 8:57 PM To: Huang, Jie (Jackie) <Jackie.Huang@windriver.com<mailto:Jackie.Huang@windriver.com>>; Little, Scott <Scott.Little@windriver.com<mailto:Scott.Little@windriver.com>>; Panech, Davlet <Davlet.Panech@windriver.com<mailto:Davlet.Panech@windriver.com>>; Tianyu Li <Tianyu.Li@arm.com<mailto:Tianyu.Li@arm.com>> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> <starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>>; Scott Kamp <scott.kamp@xunison.com<mailto:scott.kamp@xunison.com>>; Trevor Tao <Trevor.Tao@arm.com<mailto:Trevor.Tao@arm.com>>; Geary, Stephen (Steve) <Steve.Geary@windriver.com<mailto:Steve.Geary@windriver.com>>; Wang, Linda <Linda.Wang@windriver.com<mailto:Linda.Wang@windriver.com>> Subject: RE: StarlingX CI/CD setup on Arm platform Hi Scott / Davlet, As we discussed on the OS Distro call, a local CENGEN like server is needed for the local CI/CD setup, Could you please share the info of how to setup a CENGEN like server, specifically how to prepare the package mirror, we need to get it setup and prepare the mirror for ARM deb packages, and for the LAT-SDK for ARM as well. Thanks! Thanks, Jackie From: Huang, Jie (Jackie) <Jackie.Huang@windriver.com<mailto:Jackie.Huang@windriver.com>> Sent: Friday, August 18, 2023 1:33 PM To: Little, Scott <Scott.Little@windriver.com<mailto:Scott.Little@windriver.com>>; Panech, Davlet <Davlet.Panech@windriver.com<mailto:Davlet.Panech@windriver.com>>; Tianyu Li <Tianyu.Li@arm.com<mailto:Tianyu.Li@arm.com>> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>; Scott Kamp <scott.kamp@xunison.com<mailto:scott.kamp@xunison.com>>; Trevor Tao <Trevor.Tao@arm.com<mailto:Trevor.Tao@arm.com>>; Geary, Stephen (Steve) <Steve.Geary@windriver.com<mailto:Steve.Geary@windriver.com>>; Wang, Linda <Linda.Wang@windriver.com<mailto:Linda.Wang@windriver.com>> Subject: Re: [Starlingx-discuss] StarlingX CI/CD setup on Arm platform Hi Scott, Thanks for sharing, it’s very helpful. Thanks, Jackie From: Little, Scott <Scott.Little@windriver.com<mailto:Scott.Little@windriver.com>> Sent: Thursday, August 17, 2023 11:48 PM To: Huang, Jie (Jackie) <Jackie.Huang@windriver.com<mailto:Jackie.Huang@windriver.com>>; Panech, Davlet <Davlet.Panech@windriver.com<mailto:Davlet.Panech@windriver.com>>; Tianyu Li <Tianyu.Li@arm.com<mailto:Tianyu.Li@arm.com>> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>; Scott Kamp <scott.kamp@xunison.com<mailto:scott.kamp@xunison.com>>; Trevor Tao <Trevor.Tao@arm.com<mailto:Trevor.Tao@arm.com>>; Geary, Stephen (Steve) <Steve.Geary@windriver.com<mailto:Steve.Geary@windriver.com>>; Wang, Linda <Linda.Wang@windriver.com<mailto:Linda.Wang@windriver.com>> Subject: Re: StarlingX CI/CD setup on Arm platform There is also ... https://github.com/starlingx-staging/jenkins<https://urldefense.com/v3/__https:/github.com/starlingx-staging/jenkins__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6exWFUmA$> Which contains the current state of the StarlingX jenkins setup, less a few credentials. It includes 'Debian_build_server.txt' with crude documentation of how the server was setup. Scott On 2023-08-16 21:04, Huang, Jie (Jackie) wrote: Hi Davlet, Thanks for the detailed answers, which are very helpful, Tianyu and I will start by familiarizing ourselves and check if there are any more questions. Thanks, Jackie From: Panech, Davlet <Davlet.Panech@windriver.com><mailto:Davlet.Panech@windriver.com> Sent: Wednesday, August 16, 2023 11:30 PM To: Huang, Jie (Jackie) <Jackie.Huang@windriver.com><mailto:Jackie.Huang@windriver.com>; Little, Scott <Scott.Little@windriver.com><mailto:Scott.Little@windriver.com>; Tianyu Li <Tianyu.Li@arm.com><mailto:Tianyu.Li@arm.com>; Wang, Linda <Linda.Wang@windriver.com><mailto:Linda.Wang@windriver.com> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>; Scott Kamp <scott.kamp@xunison.com><mailto:scott.kamp@xunison.com>; Trevor Tao <Trevor.Tao@arm.com><mailto:Trevor.Tao@arm.com>; Geary, Stephen (Steve) <Steve.Geary@windriver.com><mailto:Steve.Geary@windriver.com> Subject: Re: StarlingX CI/CD setup on Arm platform Hi Jackie, CI/CD can be thought of as 2 somewhat disconnected parts - build & automated testing. I know little about test automation, the current build set up is outlined below. You would need a dedicated server: 1. Linux : we currently use Debian 11, but it shouldn't matter since most build steps run in docker containers 2. Docker 3. Kubernetes 4. Jenkins: integrated with k8s and able to run k8s pipelines 5. git 6. git-repo https://gerrit.googlesource.com/git-repo<https://urldefense.com/v3/__https:/gerrit.googlesource.com/git-repo__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6Y5TFnTg$> Other prerequisites: 1. You would need some DockerHub account to avoid DockerHub's rate limits when pulling images 2. Once your ARM set up is ready we can replace that with a controlled account that we manage in order to push the images to Docker Hub. Jenkins job definitions (Jenkinsfiles) are source-controlled here: https://opendev.org/starlingx/jenkins-pipelines<https://urldefense.com/v3/__https:/opendev.org/starlingx/jenkins-pipelines__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6EmOjdNg$> . You would need to define Jenkins jobs manually in the GUI, and point them to the corresponding Jenkinsfiles in this git repo. 1. The repo itself acts as a pipeline library in Jenkins with shared functions under vars/<https://urldefense.com/v3/__https:/opendev.org/starlingx/jenkins-pipelines/src/branch/master/vars__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6wk4ZyVg$> . The library must be defined at Jenkins' folder level. 2. pipelines/monolithic.Jenkinsfile<https://urldefense.com/v3/__https:/opendev.org/starlingx/jenkins-pipelines/src/branch/master/pipelines/monolithic.Jenkinsfile__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6qaNL9Is$> : top-level job that calls other jobs under parts/ 3. pipelines/parts/<https://urldefense.com/v3/__https:/opendev.org/starlingx/jenkins-pipelines/src/branch/master/pipelines/parts__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp616q0Kfk$> : directory containing sub-job Jenkinsfiles. You would create a Jenkins folder in the GUI called "parts" and define individual jobs pointing to individual Jenkinsfiles from this subdirectory in git repo 4. scripts/<https://urldefense.com/v3/__https:/opendev.org/starlingx/jenkins-pipelines/src/branch/master/scripts__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6mfhen9o$> : shell scripts that implement individual build steps (called by Jenkinsfiles) Required directories on the server, owned by jenkins user: 1. /localdisk/designer/jenkins/ root for per-project build areas 2. /localdisk/loadbuild/jenkins/ root for per-project build archives At first execution the monolithic job will fail, but create an example configuration file in the project build area that you are expected to copy & edit: 1. /localdisk/designer/jenkins/$PROJECT/build.cond.example There are many options with comments in it; the example file is also source controlled under scripts/templates/build.conf.example.in<https://urldefense.com/v3/__https:/opendev.org/starlingx/jenkins-pipelines/src/branch/master/scripts/templates/build.conf.example.in__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6S_cks2U$> Builds are slow and would greatly benefit from fast storage (SSDs etc). Please start by familiarizing yourself with this repository and ping me if you have any more questions. Hope this helps, D. ________________________________ From: Huang, Jie (Jackie) <Jackie.Huang@windriver.com<mailto:Jackie.Huang@windriver.com>> Sent: August 9, 2023 10:28 AM To: Little, Scott <Scott.Little@windriver.com<mailto:Scott.Little@windriver.com>>; Panech, Davlet <Davlet.Panech@windriver.com<mailto:Davlet.Panech@windriver.com>>; Tianyu Li <Tianyu.Li@arm.com<mailto:Tianyu.Li@arm.com>>; Wang, Linda <Linda.Wang@windriver.com<mailto:Linda.Wang@windriver.com>> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> <starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>>; Scott Kamp <scott.kamp@xunison.com<mailto:scott.kamp@xunison.com>>; Trevor Tao <Trevor.Tao@arm.com<mailto:Trevor.Tao@arm.com>>; Geary, Stephen (Steve) <Steve.Geary@windriver.com<mailto:Steve.Geary@windriver.com>> Subject: RE: StarlingX CI/CD setup on Arm platform Hi Scott and Davlet, As mentioned in the Multi-OS call, I’m re-sending this with all the questions about StarlingX CI/CD, please help answer as many as you can, thanks! There would be 6 Arm servers contributed to our community for CI/CD (2 for build, 4 for testing of different deployment configs), what we want to know is: 1. What’s the current status of StarlingX CICD for x86-64? 2. What needs to be done for the CICD workflow setup and how after the 6 ARM servers are ready and hosted in the lab? a. E.g. how to setup build servers and testing env? b. What CICD tools need to be setup? Zuul/Jenkins/etc. c. Are there any docs or info of the steps for these setups? 3. How many efforts and how long will it take? 4. Can this only be done by specific person? Or anyone have permission to access to the servers? If the later, I think Tianyu from Arm team and I can also help on that if we got enough information. Tianyu, please add if you have any other questions. Thanks, Jackie From: Huang, Jie (Jackie) Sent: Monday, July 31, 2023 12:57 PM To: Trevor Tao <Trevor.Tao@arm.com<mailto:Trevor.Tao@arm.com>>; Geary, Stephen (Steve) <Steve.Geary@windriver.com<mailto:Steve.Geary@windriver.com>>; Little, Scott <Scott.Little@windriver.com<mailto:Scott.Little@windriver.com>>; Scott Kamp <scott.kamp@xunison.com<mailto:scott.kamp@xunison.com>> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: RE: StarlingX CI/CD setup on Arm platform Hi, Scott Kamp and Scott Little, I found the following actions in the meeting minutes in stx-status | OpenDev Etherpad<https://urldefense.com/v3/__https:/etherpad.opendev.org/p/stx-status__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6bH3qX9g$>: o ARM Support * Action: Scott Kamp to respond to Jackie/the mailing list to provide assistance/access to some arm machines * Action: Scott Little to respond regarding build questions * Cannot mirror more files on CENGN; will need to wait until we transition away from CENGN * Action: Scott Kamp to explore providing a hosting env temporarily So I think you might be able to help on Trevor’s and my questions: There would be 6 Arm servers contributed to our community for CI/CD (2 for build, 4 for testing of different deployment configs), what we want to know is: 1. What needs to be done for the CICD workflow setup and how? a. E.g. how to setup build servers and testing env? b. What CICD tools need to be setup? Zuul/Jenkins/etc. c. Are there any docs for these setups? 2. How many efforts and how long will it take? 3. Is there a CICD team or someone that can help on the setup or just providing useful information? Thanks, Jackie From: Trevor Tao <Trevor.Tao@arm.com<mailto:Trevor.Tao@arm.com>> Sent: Thursday, July 27, 2023 5:06 PM To: Geary, Stephen (Steve) <Steve.Geary@windriver.com<mailto:Steve.Geary@windriver.com>>; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Cc: Huang, Jie (Jackie) <Jackie.Huang@windriver.com<mailto:Jackie.Huang@windriver.com>> Subject: StarlingX CI/CD setup on Arm platform Hi Steve, StarlingX: From the OS distro meeting<https://urldefense.com/v3/__https:/etherpad.opendev.org/p/stx-distro-other__;!!AjveYdw8EvQ!e07vSdClUS6f5DohQ4EQeAEzsC4SnjZH2Vb3mjeau1FjCuAnkaGqf1_1vuARZRPcPxdw8dZJN-D3pMmWKHLvhCs$> yesterday, we knew that there would be 6 Arm servers contributed to our community to do the CI/CD related work. So we would like to ask who can help on the setup of CI/CD environment on the arm platform, and any other useful information/details for the CI/CD issue here. Thanks, Best Regards, Zijin Tao(Trevor Tao, 陶孜谨) ARM Electronic Technology (Shanghai) Co., Ltd 安谋电子科技（上海）有限公司 Building 11, Shanghai Busininess ParkⅢ , No.1016 Tianlin Rd, Minhang District, Shanghai, 200233 China 上海市闵行区田林路1016号科技绿洲三期2号楼10楼，200233 Cell: +86-153 7109 6192 IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
The Jenkins setup at https://github.com/starlingx-staging/jenkins has 99% of what is needed to populate a StarlingX mirror. The Jenkins jobs assume that writing to /export/mirror/ is populating https://mirror.starlingx.windriver.com/mirror/ ... formerly http://mirror.starlingx.cengn.ca/mirror/ . The webserver imports the same filesystem as /starlingx/mirror/ The webserver nginx config is ... /etc/nginx/sites-available/default server { listen 80; listen 443 ssl default_server; listen [::]:443 ssl default_server; server_name mirror.starlingx.windriver.com; ssl_certificate /etc/ssl/certs/mirror_starlingx_windriver_com-full.crt; ssl_certificate_key /etc/ssl/private/mirror_starlingx_windriver_com.key; ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; ssl_ciphers HIGH:!aNULL:!MD5; root /var/www/html; index index.html index.htm index.nginx-debian.html; location / { try_files $uri $uri/ =404; } location /mirror/ { alias /starlingx/mirror/; autoindex on; autoindex_exact_size off; autoindex_format html; autoindex_localtime off; } } You'll need to substitute your own server name and ssl certificate. Tracking down and substituting all the references to mirror.starlingx.cengn.ca in the StarlingX source code is the trick. I need to do something similar to replace mirror.starlingx.cengn.ca with mirror.starlingx.windriver.com. Attached is a tarfile with the content of /export/config/debian/ It drives which debian repo's are mirrored. You'll need to adapt it for arm. The last piece you need is a /export/config/debian/mirrorkeyring for the keys validate the debian sources you are pulling down. Scott On 2023-08-24 08:57, Huang, Jie (Jackie) wrote:
Hi Scott / Davlet,
As we discussed on the OS Distro call, a local CENGEN like server is needed for the local CI/CD setup,
Could you please share the info of how to setup a CENGEN like server, specifically how to prepare the
package mirror, we need to get it setup and prepare the mirror for ARM deb packages, and for the
LAT-SDK for ARM as well. Thanks!
Thanks,
Jackie
*From:*Huang, Jie (Jackie) <Jackie.Huang@windriver.com> *Sent:* Friday, August 18, 2023 1:33 PM *To:* Little, Scott <Scott.Little@windriver.com>; Panech, Davlet <Davlet.Panech@windriver.com>; Tianyu Li <Tianyu.Li@arm.com> *Cc:* starlingx-discuss@lists.starlingx.io; Scott Kamp <scott.kamp@xunison.com>; Trevor Tao <Trevor.Tao@arm.com>; Geary, Stephen (Steve) <Steve.Geary@windriver.com>; Wang, Linda <Linda.Wang@windriver.com> *Subject:* Re: [Starlingx-discuss] StarlingX CI/CD setup on Arm platform
Hi Scott,
Thanks for sharing, it’s very helpful.
Thanks,
Jackie
*From:*Little, Scott <Scott.Little@windriver.com> *Sent:* Thursday, August 17, 2023 11:48 PM *To:* Huang, Jie (Jackie) <Jackie.Huang@windriver.com>; Panech, Davlet <Davlet.Panech@windriver.com>; Tianyu Li <Tianyu.Li@arm.com> *Cc:* starlingx-discuss@lists.starlingx.io; Scott Kamp <scott.kamp@xunison.com>; Trevor Tao <Trevor.Tao@arm.com>; Geary, Stephen (Steve) <Steve.Geary@windriver.com>; Wang, Linda <Linda.Wang@windriver.com> *Subject:* Re: StarlingX CI/CD setup on Arm platform
There is also ...
https://github.com/starlingx-staging/jenkins <https://urldefense.com/v3/__https:/github.com/starlingx-staging/jenkins__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6exWFUmA$>
Which contains the current state of the StarlingX jenkins setup, less a few credentials.
It includes 'Debian_build_server.txt' with crude documentation of how the server was setup.
Scott
On 2023-08-16 21:04, Huang, Jie (Jackie) wrote:
Hi Davlet,
Thanks for the detailed answers, which are very helpful, Tianyu and I will start by familiarizing ourselves and check if there are any more questions.
Thanks,
Jackie
*From:*Panech, Davlet <Davlet.Panech@windriver.com> <mailto:Davlet.Panech@windriver.com> *Sent:* Wednesday, August 16, 2023 11:30 PM *To:* Huang, Jie (Jackie) <Jackie.Huang@windriver.com> <mailto:Jackie.Huang@windriver.com>; Little, Scott <Scott.Little@windriver.com> <mailto:Scott.Little@windriver.com>; Tianyu Li <Tianyu.Li@arm.com> <mailto:Tianyu.Li@arm.com>; Wang, Linda <Linda.Wang@windriver.com> <mailto:Linda.Wang@windriver.com> *Cc:* starlingx-discuss@lists.starlingx.io; Scott Kamp <scott.kamp@xunison.com> <mailto:scott.kamp@xunison.com>; Trevor Tao <Trevor.Tao@arm.com> <mailto:Trevor.Tao@arm.com>; Geary, Stephen (Steve) <Steve.Geary@windriver.com> <mailto:Steve.Geary@windriver.com> *Subject:* Re: StarlingX CI/CD setup on Arm platform
Hi Jackie,
CI/CD can be thought of as 2 somewhat disconnected parts - build & automated testing. I know little about test automation, the current build set up is outlined below.
You would need a dedicated server:
1.Linux : we currently use Debian 11, but it shouldn't matter since most build steps run in docker containers
2.Docker
3.Kubernetes
4.Jenkins: integrated with k8s and able to run k8s pipelines
5.git
6.git-repo https://gerrit.googlesource.com/git-repo <https://urldefense.com/v3/__https:/gerrit.googlesource.com/git-repo__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6Y5TFnTg$>
Other prerequisites:
1.You would need some DockerHub account to avoid DockerHub's rate limits when pulling images
2.Once your ARM set up is ready we can replace that with a controlled account that we manage in order to push the images to Docker Hub.
Jenkins job definitions (Jenkinsfiles) are source-controlled here: https://opendev.org/starlingx/jenkins-pipelines <https://urldefense.com/v3/__https:/opendev.org/starlingx/jenkins-pipelines__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6EmOjdNg$> . You would need to define Jenkins jobs manually in the GUI, and point them to the corresponding Jenkinsfiles in this git repo.
1.The repo itself acts as a pipeline library in Jenkins with shared functions under vars/ <https://urldefense.com/v3/__https:/opendev.org/starlingx/jenkins-pipelines/src/branch/master/vars__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6wk4ZyVg$> . The library must be defined at Jenkins' folder level.
2.pipelines/monolithic.Jenkinsfile <https://urldefense.com/v3/__https:/opendev.org/starlingx/jenkins-pipelines/src/branch/master/pipelines/monolithic.Jenkinsfile__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6qaNL9Is$> : top-level job that calls other jobs under parts/
3.pipelines/parts/ <https://urldefense.com/v3/__https:/opendev.org/starlingx/jenkins-pipelines/src/branch/master/pipelines/parts__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp616q0Kfk$> : directory containing sub-job Jenkinsfiles. You would create a Jenkins folder in the GUI called "parts" and define individual jobs pointing to individual Jenkinsfiles from this subdirectory in git repo
4.scripts/ <https://urldefense.com/v3/__https:/opendev.org/starlingx/jenkins-pipelines/src/branch/master/scripts__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6mfhen9o$> : shell scripts that implement individual build steps (called by Jenkinsfiles)
Required directories on the server, owned by jenkins user:
1./localdisk/designer/jenkins/ root for per-project build areas
2./localdisk/loadbuild/jenkins/ root for per-project build archives
At first execution the monolithic job will fail, but create an example configuration file in the project build area that you are expected to copy & edit:
1./localdisk/designer/jenkins/$PROJECT/build.cond.example
There are many options with comments in it; the example file is also source controlled under scripts/templates/build.conf.example.in <https://urldefense.com/v3/__https:/opendev.org/starlingx/jenkins-pipelines/src/branch/master/scripts/templates/build.conf.example.in__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6S_cks2U$>
Builds are slow and would greatly benefit from fast storage (SSDs etc).
Please start by familiarizing yourself with this repository and ping me if you have any more questions.
Hope this helps,
D.
------------------------------------------------------------------------
*From:*Huang, Jie (Jackie) <Jackie.Huang@windriver.com> *Sent:* August 9, 2023 10:28 AM *To:* Little, Scott <Scott.Little@windriver.com>; Panech, Davlet <Davlet.Panech@windriver.com>; Tianyu Li <Tianyu.Li@arm.com>; Wang, Linda <Linda.Wang@windriver.com> *Cc:* starlingx-discuss@lists.starlingx.io <starlingx-discuss@lists.starlingx.io>; Scott Kamp <scott.kamp@xunison.com>; Trevor Tao <Trevor.Tao@arm.com>; Geary, Stephen (Steve) <Steve.Geary@windriver.com> *Subject:* RE: StarlingX CI/CD setup on Arm platform
Hi Scott and Davlet,
As mentioned in the Multi-OS call, I’m re-sending this with all the questions about StarlingX CI/CD, please help answer as many as you can, thanks!
There would be 6 Arm servers contributed to our community for CI/CD (2 for build, 4 for testing of different deployment configs), what we want to know is:
1.What’s the current status of StarlingX CICD for x86-64?
2.What needs to be done for the CICD workflow setup and how after the 6 ARM servers are ready and hosted in the lab?
a.E.g. how to setup build servers and testing env?
b.What CICD tools need to be setup? Zuul/Jenkins/etc.
c.Are there any docs or info of the steps for these setups?
3.How many efforts and how long will it take?
4.Can this only be done by specific person? Or anyone have permission to access to the servers? If the later, I think Tianyu from Arm team and I can also help
on that if we got enough information.
Tianyu, please add if you have any other questions.
Thanks,
Jackie
*From:*Huang, Jie (Jackie) *Sent:* Monday, July 31, 2023 12:57 PM *To:* Trevor Tao <Trevor.Tao@arm.com>; Geary, Stephen (Steve) <Steve.Geary@windriver.com>; Little, Scott <Scott.Little@windriver.com>; Scott Kamp <scott.kamp@xunison.com> *Cc:* starlingx-discuss@lists.starlingx.io *Subject:* RE: StarlingX CI/CD setup on Arm platform
Hi, Scott Kamp and Scott Little,
I found the following actions in the meeting minutes in stx-status | OpenDev Etherpad <https://urldefense.com/v3/__https:/etherpad.opendev.org/p/stx-status__;!!AjveYdw8EvQ!fy4riVoRaMARLJhys1pQ-2-dUqlguu-vZxfBGYtlrpyF-ePItaqQE41k4sgTk4wmc85A4kt56sPOPZPfcsKpBrp6bH3qX9g$>:
oARM Support
§_Action:_ Scott Kamp to respond to Jackie/the mailing list to provide assistance/access to some arm machines
§_Action:_ Scott Little to respond regarding build questions
·Cannot mirror more files on CENGN; will need to wait until we transition away from CENGN
·_Action:_ Scott Kamp to explore providing a hosting env temporarily
So I think you might be able to help on Trevor’s and my questions:
There would be 6 Arm servers contributed to our community for CI/CD (2 for build, 4 for testing of different deployment configs), what we want to know is:
1.What needs to be done for the CICD workflow setup and how?
a.E.g. how to setup build servers and testing env?
b.What CICD tools need to be setup? Zuul/Jenkins/etc.
c.Are there any docs for these setups?
2.How many efforts and how long will it take?
3.Is there a CICD team or someone that can help on the setup or just providing useful information?
Thanks,
Jackie
*From:*Trevor Tao <Trevor.Tao@arm.com> *Sent:* Thursday, July 27, 2023 5:06 PM *To:* Geary, Stephen (Steve) <Steve.Geary@windriver.com>; starlingx-discuss@lists.starlingx.io *Cc:* Huang, Jie (Jackie) <Jackie.Huang@windriver.com> *Subject:* StarlingX CI/CD setup on Arm platform
Hi Steve, StarlingX:
From the OS distro meeting <https://urldefense.com/v3/__https:/etherpad.opendev.org/p/stx-distro-other__;!!AjveYdw8EvQ!e07vSdClUS6f5DohQ4EQeAEzsC4SnjZH2Vb3mjeau1FjCuAnkaGqf1_1vuARZRPcPxdw8dZJN-D3pMmWKHLvhCs$> yesterday, we knew that there would be 6 Arm servers contributed to our community to do the CI/CD related work.
So we would like to ask who can help on the setup of CI/CD environment on the arm platform, and any other useful information/details for the CI/CD issue here.
Thanks,
Best Regards,
Zijin Tao(Trevor Tao, 陶孜谨)
ARM Electronic Technology (Shanghai) Co., Ltd
安谋电子科技（上海）有限公司
Building 11, Shanghai Busininess ParkⅢ,
No.1016 Tianlin Rd, Minhang District, Shanghai, 200233 China
上海市闵行区田林路1016号科技绿洲三期2号楼10楼，200233
Cell: +86-153 7109 6192
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
participants (6)
-
Chris Wang
-
Huang, Jie (Jackie)
-
Panech, Davlet
-
Scott Little
-
Tianyu Li
-
Trevor Tao