<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
  </head>
  <body>
    <div class="moz-text-html" lang="x-unicode">It was a really bad week
      for build issues on CENGN.  Below I try to summarize the we've
      been working on.  <br>
    </div>
    <div class="moz-text-html" lang="x-unicode"><br>
    </div>
    <div class="moz-text-html" lang="x-unicode">Scott</div>
    <div class="moz-text-html" lang="x-unicode"><br>
    </div>
    <div class="moz-text-html" lang="x-unicode">PS: use a wide window to
      view!  <br>
    </div>
    <div class="moz-text-html" lang="x-unicode"><br>
      <pre>Branch OS      Type            Issue
======  ==      ====            =====
</pre>
      <pre>Master Centos  Layered         Build failures Oct 28-31
                                Issue: 'Repo' pulled down a non-backward compatible upgrade from upstream that breaks non-containerized uses of 'repo'.
                                Fix: Updated the version of 'repo' on CENGN bare metal.
                                Manual build successful: Oct 31

Master  Centos  Monolithic      Build Failure Oct 29
                                Issue: Corruption of the jenkins build script.  Cause of corruption unknown.
                                Fix: Repaired build script and restarted jenkins
                                Next build attempt Nov 5

Master  Debian  Monolithic      Build Failures Oct 28-Nov 1
                                Issue 1: Attempt to optimize the 'publish packages' step had undetected bugs when merged Friday.
                                Fix 1: corrected on path, and prevented script from throwing error when a grep fails to find a match.
                                Benefit: 'publish packages' step takes minutes, not hours.
                                Issue 2: STX_build_debian_master jenkins job was deleted.  Unclear how/why but CENGN jenkins was very slow and acting strangely.
                                Fix 2: restored STX_build_debian_master from git, but we've lost build history.  Restarted Jenkins.
                                Issue 3: Activation of 'reuse' accelerated package build, ~8 hrs -> ~2hrs. Works for ISO, but breaks container image build.
                                        Misunderstood the intent of one recent fix to this feature, thought it would address the container image build issues.
                                        Clearly it did not.
                                Fix 3: Deactivated 'reuse' feature once again.  Longer term, a LP will be raised vs docker images.  Docker images
                                        need to pull debs from the aptly published repo, not the traditional repo. 
                                Issue 4: A jenkins job was create to test changes to jenkins pipeline scripts. That job had a name that was
                                        similar to but not identical to the main job. This exposed a defect in how minikube finds containers using
                                        fuzzy rather than exact matches.  Cross-talk between main and dev minikube environments broke both builds.
                                        <a href="https://bugs.launchpad.net/starlingx/+bug/1995400" class="external-link moz-txt-link-freetext" target="_blank" rel="nofollow noopener" title="Follow link">https://bugs.launchpad.net/starlingx/+bug/1995400</a>
                                Fix 4: Quick fix was to change the dev build name to avoid the 'fuzzy' match, and to nuke/rebuild both correpted
                                        minikube environments. Longer term, we need to use exect matches within minikube.
                                Success: Build with docker images, TIMESTAMP 20221102T060000Z, Available 20221102T160000Z

stx.6.0 Centos  Layered         Build Failed Oct 29-Nov 1
                                Issue: The builder container can't build.  pip install pulled in unconstrained packages that have dropped py2.7 compatibility.
                                        <a class="moz-txt-link-freetext" href="https://bugs.launchpad.net/starlingx/+bug/1995484">https://bugs.launchpad.net/starlingx/+bug/1995484</a>
                                Fix: Add constraints to maintain py2.7 compatibility in the pip installed packages. 
                                        <a class="moz-txt-link-freetext" href="https://review.opendev.org/c/starlingx/tools/+/863265">https://review.opendev.org/c/starlingx/tools/+/863265</a>
                                Test build started

stx.7.0 Centos  Layered         Build Failed Oct 30
                                Issue: Failed to build containers locationservice-base, notificationclient-base, notificationservice-base
                                        <a class="moz-txt-link-freetext" href="https://bugs.launchpad.net/starlingx/+bug/1995486">https://bugs.launchpad.net/starlingx/+bug/1995486</a>
                                One candidate fix back-ported
                                        <a class="moz-txt-link-freetext" href="https://review.opendev.org/c/starlingx/ptp-notification-armada-app/+/856185">https://review.opendev.org/c/starlingx/ptp-notification-armada-app/+/856185</a>
                                Awaiting a free time slot for a trial build.


                                
                                



</pre>
    </div>
    <p></p>
  </body>
</html>