<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<div class="moz-text-html" lang="x-unicode">It was a really bad week
for build issues on CENGN. Below I try to summarize the we've
been working on. <br>
</div>
<div class="moz-text-html" lang="x-unicode"><br>
</div>
<div class="moz-text-html" lang="x-unicode">Scott</div>
<div class="moz-text-html" lang="x-unicode"><br>
</div>
<div class="moz-text-html" lang="x-unicode">PS: use a wide window to
view! <br>
</div>
<div class="moz-text-html" lang="x-unicode"><br>
<pre>Branch OS Type Issue
====== == ==== =====
</pre>
<pre>Master Centos Layered Build failures Oct 28-31
Issue: 'Repo' pulled down a non-backward compatible upgrade from upstream that breaks non-containerized uses of 'repo'.
Fix: Updated the version of 'repo' on CENGN bare metal.
Manual build successful: Oct 31
Master Centos Monolithic Build Failure Oct 29
Issue: Corruption of the jenkins build script. Cause of corruption unknown.
Fix: Repaired build script and restarted jenkins
Next build attempt Nov 5
Master Debian Monolithic Build Failures Oct 28-Nov 1
Issue 1: Attempt to optimize the 'publish packages' step had undetected bugs when merged Friday.
Fix 1: corrected on path, and prevented script from throwing error when a grep fails to find a match.
Benefit: 'publish packages' step takes minutes, not hours.
Issue 2: STX_build_debian_master jenkins job was deleted. Unclear how/why but CENGN jenkins was very slow and acting strangely.
Fix 2: restored STX_build_debian_master from git, but we've lost build history. Restarted Jenkins.
Issue 3: Activation of 'reuse' accelerated package build, ~8 hrs -> ~2hrs. Works for ISO, but breaks container image build.
Misunderstood the intent of one recent fix to this feature, thought it would address the container image build issues.
Clearly it did not.
Fix 3: Deactivated 'reuse' feature once again. Longer term, a LP will be raised vs docker images. Docker images
need to pull debs from the aptly published repo, not the traditional repo.
Issue 4: A jenkins job was create to test changes to jenkins pipeline scripts. That job had a name that was
similar to but not identical to the main job. This exposed a defect in how minikube finds containers using
fuzzy rather than exact matches. Cross-talk between main and dev minikube environments broke both builds.
<a href="https://bugs.launchpad.net/starlingx/+bug/1995400" class="external-link moz-txt-link-freetext" target="_blank" rel="nofollow noopener" title="Follow link">https://bugs.launchpad.net/starlingx/+bug/1995400</a>
Fix 4: Quick fix was to change the dev build name to avoid the 'fuzzy' match, and to nuke/rebuild both correpted
minikube environments. Longer term, we need to use exect matches within minikube.
Success: Build with docker images, TIMESTAMP 20221102T060000Z, Available 20221102T160000Z
stx.6.0 Centos Layered Build Failed Oct 29-Nov 1
Issue: The builder container can't build. pip install pulled in unconstrained packages that have dropped py2.7 compatibility.
<a class="moz-txt-link-freetext" href="https://bugs.launchpad.net/starlingx/+bug/1995484">https://bugs.launchpad.net/starlingx/+bug/1995484</a>
Fix: Add constraints to maintain py2.7 compatibility in the pip installed packages.
<a class="moz-txt-link-freetext" href="https://review.opendev.org/c/starlingx/tools/+/863265">https://review.opendev.org/c/starlingx/tools/+/863265</a>
Test build started
stx.7.0 Centos Layered Build Failed Oct 30
Issue: Failed to build containers locationservice-base, notificationclient-base, notificationservice-base
<a class="moz-txt-link-freetext" href="https://bugs.launchpad.net/starlingx/+bug/1995486">https://bugs.launchpad.net/starlingx/+bug/1995486</a>
One candidate fix back-ported
<a class="moz-txt-link-freetext" href="https://review.opendev.org/c/starlingx/ptp-notification-armada-app/+/856185">https://review.opendev.org/c/starlingx/ptp-notification-armada-app/+/856185</a>
Awaiting a free time slot for a trial build.
</pre>
</div>
<p></p>
</body>
</html>