November 2022 - Starlingx-discuss

[Starlingx-discuss] [build-report] r/stx.6.0 STX_6.0_build_layer_containers - Build # 8 - Failure!
by starlingx.build＠gmail.com 03 Nov '22

03 Nov '22

Project: STX_6.0_build_layer_containers Build #: 8 Status: Failure Timestamp: 20221102T231513Z Branch: r/stx.6.0 Check logs at: http://mirror.starlingx.cengn.ca/mirror/starlingx/rc/6.0/centos/containers/… -------------------------------------------------------------------------------- Parameters BUILD_CONTAINERS_DEV: false BUILD_CONTAINERS_STABLE: true FORCE_BUILD: true

1 0

[Starlingx-discuss] [stable] [build-report] r/stx.6.0 STX_build_docker_images_layered - Build # 363 - Failure!
by starlingx.build＠gmail.com 03 Nov '22

03 Nov '22

Project: STX_build_docker_images_layered Build #: 363 Status: Failure Timestamp: 20221103T000026Z Branch: r/stx.6.0 Check logs at: http://mirror.starlingx.cengn.ca/mirror/starlingx/rc/6.0/centos/containers/… -------------------------------------------------------------------------------- Parameters BRANCH: r/stx.6.0 MY_WORKSPACE: /localdisk/loadbuild/jenkins/rc-6.0-containers/20221102T231513Z OS: centos MUNGED_BRANCH: rc-6.0 MY_REPO: /localdisk/designer/jenkins/rc-6.0-containers/cgcs-root PUBLISH_LOGS_URL: http://mirror.starlingx.cengn.ca/mirror/starlingx/rc/6.0/centos/containers/… MASTER_BUILD_NUMBER: 8 PUBLISH_LOGS_BASE: /export/mirror/starlingx/rc/6.0/centos/containers/20221102T231513Z/logs MASTER_JOB_NAME: STX_6.0_build_layer_containers MY_REPO_ROOT: /localdisk/designer/jenkins/rc-6.0-containers PUBLISH_DISTRO_BASE: /export/mirror/starlingx/rc/6.0/centos/containers PUBLISH_TIMESTAMP: 20221102T231513Z DOCKER_BUILD_ID: jenkins-rc-6.0-containers-20221102T231513Z-builder TIMESTAMP: 20221102T231513Z OS_VERSION: 7.5.1804 BUILD_STREAM: stable PUBLISH_INPUTS_BASE: /export/mirror/starlingx/rc/6.0/centos/containers/20221102T231513Z/inputs LAYER: containers PUBLISH_OUTPUTS_BASE: /export/mirror/starlingx/rc/6.0/centos/containers/20221102T231513Z/outputs MAIN_HELM_CHART_RPM_NAME: stx-openstack-helm

1 0

[Starlingx-discuss] [stable] [build-report] STX_build_docker_flock_images - Build # 607 - Failure!
by starlingx.build＠gmail.com 03 Nov '22

03 Nov '22

Project: STX_build_docker_flock_images Build #: 607 Status: Failure Timestamp: 20221103T003510Z Branch: Check logs at: http://mirror.starlingx.cengn.ca/mirror/starlingx/rc/6.0/centos/containers/… -------------------------------------------------------------------------------- Parameters WEB_HOST: mirror.starlingx.cengn.ca MY_WORKSPACE: /localdisk/loadbuild/jenkins/rc-6.0-containers/20221102T231513Z OS: centos MY_REPO: /localdisk/designer/jenkins/rc-6.0-containers/cgcs-root BASE_VERSION: rc-6.0-stable-20221102T231513Z PUBLISH_LOGS_URL: http://mirror.starlingx.cengn.ca/mirror/starlingx/rc/6.0/centos/containers/… REGISTRY_USERID: slittlewrs LATEST_PREFIX: rc-6.0 PUBLISH_LOGS_BASE: /export/mirror/starlingx/rc/6.0/centos/containers/20221102T231513Z/logs PUBLISH_TIMESTAMP: 20221102T231513Z FLOCK_VERSION: rc-6.0-centos-stable-20221102T231513Z WEB_HOST_PORT: 80 PREFIX: rc-6.0 TIMESTAMP: 20221102T231513Z BUILD_STREAM: stable REGISTRY_ORG: starlingx PUBLISH_OUTPUTS_BASE: /export/mirror/starlingx/rc/6.0/centos/containers/20221102T231513Z/outputs REGISTRY: docker.io

1 0

[Starlingx-discuss] K8s upgrade to 1.24 not quite functional yet
by Chris Friesen 03 Nov '22

03 Nov '22

Hey all, Just a heads-up that there are some issues with K8s upgrades from 1.23 to 1.24. First, there's what appears to be a timing issue with orchestrated upgrades on AIO-SX, where puppet seems to be trying to update the bind mount before kubelet is stopped all the way. There are also some issues with preserving some of our modified kubelet options over the upgrade. So unless you have a really good reason to play with it, I suggest avoiding any K8s upgrades to 1.24 for now. Thanks, Chris Friesen

1 0

[Starlingx-discuss] Summary of the numerous build issues to strike CENGN in the past week
by Scott Little 03 Nov '22

03 Nov '22

It was a really bad week for build issues on CENGN. Below I try to summarize the we've been working on. Scott PS: use a wide window to view! Branch OS Type Issue ====== == ==== ===== Master Centos Layered Build failures Oct 28-31 Issue: 'Repo' pulled down a non-backward compatible upgrade from upstream that breaks non-containerized uses of 'repo'. Fix: Updated the version of 'repo' on CENGN bare metal. Manual build successful: Oct 31 Master Centos Monolithic Build Failure Oct 29 Issue: Corruption of the jenkins build script. Cause of corruption unknown. Fix: Repaired build script and restarted jenkins Next build attempt Nov 5 Master Debian Monolithic Build Failures Oct 28-Nov 1 Issue 1: Attempt to optimize the 'publish packages' step had undetected bugs when merged Friday. Fix 1: corrected on path, and prevented script from throwing error when a grep fails to find a match. Benefit: 'publish packages' step takes minutes, not hours. Issue 2: STX_build_debian_master jenkins job was deleted. Unclear how/why but CENGN jenkins was very slow and acting strangely. Fix 2: restored STX_build_debian_master from git, but we've lost build history. Restarted Jenkins. Issue 3: Activation of 'reuse' accelerated package build, ~8 hrs -> ~2hrs. Works for ISO, but breaks container image build. Misunderstood the intent of one recent fix to this feature, thought it would address the container image build issues. Clearly it did not. Fix 3: Deactivated 'reuse' feature once again. Longer term, a LP will be raised vs docker images. Docker images need to pull debs from the aptly published repo, not the traditional repo. Issue 4: A jenkins job was create to test changes to jenkins pipeline scripts. That job had a name that was similar to but not identical to the main job. This exposed a defect in how minikube finds containers using fuzzy rather than exact matches. Cross-talk between main and dev minikube environments broke both builds. https://bugs.launchpad.net/starlingx/+bug/1995400 <https://bugs.launchpad.net/starlingx/+bug/1995400> Fix 4: Quick fix was to change the dev build name to avoid the 'fuzzy' match, and to nuke/rebuild both correpted minikube environments. Longer term, we need to use exect matches within minikube. Success: Build with docker images, TIMESTAMP 20221102T060000Z, Available 20221102T160000Z stx.6.0 Centos Layered Build Failed Oct 29-Nov 1 Issue: The builder container can't build. pip install pulled in unconstrained packages that have dropped py2.7 compatibility. https://bugs.launchpad.net/starlingx/+bug/1995484 Fix: Add constraints to maintain py2.7 compatibility in the pip installed packages. https://review.opendev.org/c/starlingx/tools/+/863265 Test build started stx.7.0 Centos Layered Build Failed Oct 30 Issue: Failed to build containers locationservice-base, notificationclient-base, notificationservice-base https://bugs.launchpad.net/starlingx/+bug/1995486 One candidate fix back-ported https://review.opendev.org/c/starlingx/ptp-notification-armada-app/+/856185 Awaiting a free time slot for a trial build.

1 0

[Starlingx-discuss] Schedule change for the bi-weekly release & os-distro meetings
by Khalil, Ghada 02 Nov '22

02 Nov '22

Hello all, There is a change to the schedule for the Wednesday 6:30am Pacific / 9:30am Eastern StarlingX community meetings. This timeslot is using on alternative weeks for the StarlingX Release Meeting and the OS-Distro Subproject Meeting. Due to conflicts for the OS-Distro team, we will be switching the weeks around. The OS-Distro meeting will take place on Nov 2, Nov 16, Nov 30, ...etc. The Release meeting will take place on Nov 9, Nov 23, Dec 7, ...etc. Updated invites have been sent. Regards, Ghada

1 0

[Starlingx-discuss] Bi-weekly StarlingX Release Meeting
by Khalil, Ghada 02 Nov '22

02 Nov '22

Updated meeting series for the StarlingX Release Meeting Starting from Nov 9, 2022 and meeting every other week. Bi-weekly meeting on Wednesday 06:30AM PT / 09:30AM ET / 02:30PM UTC Zoom Link: https://zoom.us/j/342730236?pwd=N21CUXNXVlJXMlcyZjZ0SE96cVNjQT09 Meeting agenda/minutes: https://etherpad.openstack.org/p/stx-releases Regards, Ghada

1 0

[Starlingx-discuss] Bi-weekly StarlingX Release Meeting (new time)
by Khalil, Ghada 02 Nov '22

02 Nov '22

New meeting series has been sent to the stx-discuss mailing list. New meeting series for the StarlingX Release Meeting Bi-weekly meeting on Wednesday 06:30AM PT / 09:30AM ET / 02:30PM UTC Zoom Link: https://zoom.us/j/342730236?pwd=N21CUXNXVlJXMlcyZjZ0SE96cVNjQT09 Meeting agenda/minutes: https://etherpad.openstack.org/p/stx-releases

1 0

[Starlingx-discuss] Scott V. Kamp candidacy for StarlingX TSC role
by Outback Dingo 02 Nov '22

02 Nov '22

Hello, My name is Scott Kamp, I am nominating myself for election to StarlingX TSC. I am an avid user and advocate of the StarlingX community over the past years. My goals for the StarlingX community is to bring StarlingX to a larger global user base. I am intent on growing the community and helping StarlingX become a more community friendly solution for users to realize its potential allowing for a more global footprint and adoption into many industries including IOT, Smart Home, and Telecomm. I intend to also enable the community to create reproducible builds and provide a upgrade path to current deployments moving forward, while also contributing features that I feel are required. This will allow more users to become involved, and more feedback will come to the community via the extended user base as it grows which makes StarlingX a more versatile, stable, scalable and more secure system. Thank you for considering me as a contributor to the TSC.

1 0

[Starlingx-discuss] [build-report] r/stx.6.0 STX_6.0_build_layer_compiler - Build # 17 - Still Failing!
by starlingx.build＠gmail.com 02 Nov '22

02 Nov '22

Project: STX_6.0_build_layer_compiler Build #: 17 Status: Still Failing Timestamp: 20221101T194826Z Branch: r/stx.6.0 Check logs at: http://mirror.starlingx.cengn.ca/mirror/starlingx/rc/6.0/centos/compiler/20… -------------------------------------------------------------------------------- Parameters FULL_BUILD: false FORCE_BUILD: true

1 0