[Starlingx-discuss] Minutes: Community Call (May 3, 2023)

[Khalil, Ghada]

Etherpad: https://etherpad.opendev.org/p/stx-status

Minutes from the community call
May 3, 2023
Meeting is at 7pm PDT / 10pm EDT / +1 10am CST

Standing topics
- Build
     - Main Branch Debian Builds
          - Have some issues w/ disk space in CENGN. 
          - Tried to workaround this by setting up some hard links for the same files, but that caused the docker image builds to be really slow and ScottL aborted the build.
          - This change was rolled back.
     - stx.6.0 Weekly Builds 
          - Green
          - Build Output: http://mirror.starlingx.cengn.ca/mirror/starlingx/rc/6.0/
     - stx.7.0 Weekly Builds
          - Green
          - Build Output: http://mirror.starlingx.cengn.ca/mirror/starlingx/rc/7.0/
     - stx.8.0 Weekly Builds
          - Green
          - Build Output: https://mirror.starlingx.cengn.ca/mirror/starlingx/rc/8.0/
     - github Mirroring Issue
          - Email: https://lists.starlingx.io/pipermail/starlingx-discuss/2023-April/014000.html
          - The stx code is not being mirrored to github due to a change in the key.  
          - The key is hard-coded in each repo as part of zuul, so it's a pain to update the key in every repo. 
          - Davlet updated 40 repos with the correct key. The majority of the repos sync'd properly, but a few still failed due to other issues.
          - The correct / longer term solution is to setup our own zuul configuration project for StarlingX and not piggy-back on openstack
              - Estimate is about 2wks of work
              - Raised in the build project team to determine if there are resources to do this work
          - Action: Greg to review the requirements for github mirroring to confirm if it's required for all repos or just a subset to meet k8s conformance requirements.
     - Transition from CENGN
          - Nothing new; still in progress
          - As discussed in the PTG, CENGN will no longer be able to host the StarlingX builds.
          - Build team is working on transitioning the builds to WR infrastructure.
          - Timing is TBD
- Sanity
     - Debian Main Branch Platform Sanity
          - Last sanity email sent on May 2: https://lists.starlingx.io/pipermail/starlingx-discuss/2023-May/014041.html
          - Status: Green for SX. Green for DX.
     - Debian Main Branch stx-openstack Sanity
          - Last sanity email sent on Apr 11: https://lists.starlingx.io/pipermail/starlingx-discuss/2023-April/013968.html
          - No update
          - No sanity since Apr 11 due to the DX issue as well as the stx-nova image build failure noted above.
          - Sanity LPs
              - LP: https://bugs.launchpad.net/starlingx/+bug/2007303 -- intermittent issue / works on retest
              - LP: https://bugs.launchpad.net/starlingx/+bug/2012389 -- intermittent issue
              - LP: https://bugs.launchpad.net/starlingx/+bug/2012392 -- FIXED
          - Team has some resourcing challenges, so hasn't been able to investigate the failures further
- Gerrit Reviews in Need of Attention
     - Making good progress on doc reviews; list is getting smaller:
          - https://review.opendev.org/q/starlingx/docs+status:open
     - From previous meetings: Fixes to libvirt env: https://review.opendev.org/c/starlingx/tools/+/863735
          - Review comments provided in Nov; still waiting for author (Scott Kamp) to respond/address review comments
          - Jan 18: Some activity in the review as of Jan 15
          - Feb 1: Alternative fix proposed on Jan 18; waiting for ScottK's review
          - Feb 15: ScottK is going back to this
          - Mar 22: Next action is w/ ScottK; As per today's meeting, he'll be looking at the comments
          - Apr 11: Review is still open. Next action is w/ ScottK (the author)
          - May 4: ScottK lost his setup, so needs to recover the review
     - Reference Links:
          - Active Branch (open): https://review.opendev.org/q/projects:starlingx+is:open+branch:+master
          - Active Branch (merged): https://review.opendev.org/q/projects:starlingx+is:merged+branch:master

Topics for this week
- New Feature Requests
     - Raised by ScottK
     - Scenario: 
          - Deploying a 36-node StarlingX, but couldn't setup the connectivity between the nodes from the beginning. 
          - So StartlingX went into split-brain and was not able to recover.
          - In the end, had to deploy an alternative solution not using StarlingX
     - As a result, suggest the following enhancement/feature requests
          - Multiple Controller Support instead of just two controllers
              - StarlingX was designed for two controllers, so this requires revisiting the existing design.
              - This was discussed before and would be great to tackle.
          - Storage not requiring dedicated nodes
              - StarlingX would like to migrate to using rook to support storage anywhere
              - The key challenge/complexity is migration/upgrade from the existing ceph deployment
          - Free Range Routing (FRR) Support (or Quagga Support)
     - These all sound like good feature requests
     - ScottK has some resources, but not the detailed knowledge to implement these features.

ARs from Previous Meetings
- None

Open Requests for Help
- Issue w/ pxebooting a second node on StarlingX
     - Question raised on the community call (Apr 26) by Kevin
     - The initial node installs/boots correctly, but but an identical second node doesn't. This setup is using vmare to run StarlingX in VMs.
     - Status: Open / Greg has some information to share, but isn't sure how to get in touch w/ Kevin
- Build System Setup
     - Question was posted on IRC by Kevin
     - StarlingX documentation has a lot of outdated information. It took a lot of digging to find the right documentation for setting up a build env with minikube.
     - Requested that Kevin open an LP to the doc team and/or send an email to the mailing list w/ more details to the mailing list
     - Status: Open / No email sent to the mailing list yet. Unsure if LP was opened.
- Support for StarlingX 8 setup bring up
     - Reported by Prashanthi
     - Email thread: https://lists.starlingx.io/pipermail/starlingx-discuss/2023-April/014026.html
     - Status: Open / Greg spent some time with Prashanthi helping her w/ debugging. The issue seemed related to the system time being incorrectly setup on the server.
- stx.8.0 Interface Bonding issue
     - https://lists.starlingx.io/pipermail/starlingx-discuss/2023-March/013877.html
     - LP: https://bugs.launchpad.net/starlingx/+bug/2010119
     - Fixed in the stx main branch on April 28. Cherrypicked to r/stx.8.0 on May 3.
     - Status: Closed / Fixed