[Starlingx-discuss] MultiOS: Integration of OpenSUSE build system (OBS) in StarlingX CI process
Hello, having received some requests for information, about the best way to integrate the OpenSUSE Build System (OBS) in StartlingX CI system, I am happy to share my finding. My explanations are written using osc command lines, in order to ease the checking of the described concepts, but obviously a CI integration would be easier/safer made using the OBS API equivalences. For more detail son the OBS API see: https://build.opensuse.org/apidocs/index A) Testing the change submission ============================ 1) Test via Local build ----------------------------------- It is obviously possible for the CI, to create a local checkout (osc co ProjectName PackageName) of the project/package on which the change submission is targetted, update the _service to point toward the change submission to be tested, on git source repo. Then follow by a local build for one or multiple distro target (repository). osc checkout --output-dir=CIlocalTestDir ProjectName PackageName # copy a _service file pointing to the change submission to be tested. osc build --trust-all-projects Repository It would require to parse the osc build output to extract the final status and potential a failure cause. The local build present the advantage to be potentially quicker as we would control the power allocated to the best build as well as the scheduling but requires network and CPU resources allocation. 2) Test via build in public OBS ------------------------------------------------ In that model, we want to create a branch of the project to test our change submission. The osc branch command does not well fit the requirement, because it immediately starts a build after it creation, what would slow the process a lot. We want to change the source download origin, before starting any build process. So I propose to create a 'branch' manually to overcome that current OBS limitation. We need to create a project that will have an identical configuration to the final target project. To do so, we need to copy the meta prj and prjconf as well as creating a test build project. # Extracting the target project configs osc meta prj TargetProjectName >TargetProjectName.prj osc meta prjconf TargetProjectName >TargetProjectName.prjconf # Create the test project on OBS with an easy to track name osc meta prj -F TargetProjectName.prj TargetProjectName-submissionXYZ # Copy target prjconf in the newly created test project osc meta prjconf -F TargetProjectName.prjconf TargetProjectName-submissionXYZpraget # Clean local tmp tiles rm -f TargetProjectName.prj TargetProjectName.prjconf # Installing the _service pointing to the commit submission # API based implementation is highly advise to better manage possible errors osc co TargetProjectName-submissionXYZ # install the _service pointing to the commit submission osc ci -m "test build for submissionXYZ" # spec files and rpmlint files will soon direclty been download by the _service. I assume that to be already the case in that process description. # getting build results (xml is easier to parse but other return forTargetProjectName-submissionXYZmats are available if desired) # Note: my test shows the -w option is not that reliable and retries handling would required. osc results -w --xml TargetProjectName-submissionXYZhttps://rabbit.opensuse.org/ # Alternative method # OBS provides an event channel which provides many valuable information for an integration activity. https://rabbit.opensuse.org/ # list of all available events https://openbuildservice.org/help/manuals/obs-admin-guide/obs.cha.administra... 3) case of update via new patchset ---------------------------------------------------------- It's not uncommon, that a new patchset is created during the review process before acceptance. This can be done before the finalisation of any validation of the test build. In that case we would need to abort the current build and restart a new one to speed up the process. # Install an updated _service pointing to the new patch set (depending of git config that might not be required). # trigger an abort rebuild. osc restartbuild TargetProjectName-submissionXYZ # restart the build result monitoring process. 4) Using a local OBS ---------------------------------- Creating a local OBS linked to the public OBS is quite easy and likely a better method that doing local build in the CI, would the control of the speed of validation, via the public OBS, be an issue . B) Accepting the submission ======================= The process would be the same whatever is the test build method (local, public/private OBS). Once the submission is accepted you just need to retrigger a remote service run on the public OBS. This would download the source, spec and rpmlint from the git repo, trigger a new build of the target package as well as all the depending packages and publish results in public repo upon completion. We can see here, one of the weakness of the proposed process. A change in a package may be acceptable for that package but may break dependant packages. There is a model in OBS to tackle that issue via the use of staging project, but it would slow down the acceptance of all change submissions, what might be an overkill. Up the community to decide what is the best compromise. I would favour to stick to single package check, at least, in a first step. Once the submission request has been accepted, remember to delete all the temporary projects where ever they are located. It is good idea to keep them present until acceptance, as OBS as well as osc use caches internally. Enjoy your day. -- Dominig ar Foll Senior Software Architect Intel Open Source Technology Centre
participants (1)
-
Dominig ar Foll (Intel Open Source)