[Starlingx-discuss] stx.5.0 Launchpads

Zvonar, Bill Bill.Zvonar at windriver.com
Mon May 17 18:21:18 UTC 2021 

Hi guys - coming back to this now that the build issues have been resolved.

Martin/Austin - can you provide an update on https://bugs.launchpad.net/starlingx/+bug/1918420  (it's still seen in the latest sanity).

Austin - you've got the last comment on https://bugs.launchpad.net/starlingx/+bug/1918318 - do you think this still needs to hold up the release?

Alex  - can you update https://bugs.launchpad.net/starlingx/+bug/1928341 to confirm that it wasn't seen in the 5.0 sanity, then we'll remove the stx.5.0 tag.

Bill...  

-----Original Message-----
From: Zvonar, Bill <Bill.Zvonar at windriver.com> 
Sent: Thursday, May 13, 2021 7:19 AM
To: Sun, Austin <austin.sun at intel.com>; Dimofte, Alexandru <alexandru.dimofte at intel.com>; Chen, Haochuan Z <haochuan.z.chen at intel.com>
Cc: starlingx-discuss at lists.starlingx.io
Subject: Re: [Starlingx-discuss] stx.5.0 Launchpads

Thanks Austin.

Based on the discussion on the community call yesterday [0], we're holding the release for 2 issues: 

https://bugs.launchpad.net/starlingx/+bug/1918318

     - Alex, please keep us posted on your test results, as mentioned in [1]

https://bugs.launchpad.net/starlingx/+bug/1918420

     -  Martin/Austin, please let us know if you were able to determine whether this is a testcase issue or a real issue 

Thanks, Bill... 

[0] http://lists.starlingx.io/pipermail/starlingx-discuss/2021-May/011430.html
[1] http://lists.starlingx.io/pipermail/starlingx-discuss/2021-May/011438.html 

-----Original Message-----
From: Sun, Austin <austin.sun at intel.com> 
Sent: Tuesday, May 11, 2021 9:21 PM
To: Zvonar, Bill <Bill.Zvonar at windriver.com>; starlingx-discuss at lists.starlingx.io
Subject: RE: stx.5.0 Launchpads

[Please note: This e-mail is from an EXTERNAL e-mail address]

Hi Bill:
    Some comments inline.  We can discuss detail in community meeting.

Thanks.
BR
Austin Sun

-----Original Message-----
From: Zvonar, Bill <Bill.Zvonar at windriver.com>
Sent: Wednesday, May 12, 2021 12:29 AM
To: starlingx-discuss at lists.starlingx.io
Subject: Re: [Starlingx-discuss] stx.5.0 Launchpads

Hi all - we have 3 open High Importance launchpads for stx.5.0 - 2 have reviews up & one needs input from Alex.

I'm not sure if we should be considering 1918420 to be high importance as well, but I included it below - it has a review up as well.

Bill...

Two High importance bugs have reviews up, one needs more info:

     - 1922299 (Martin): Openstack-Helm mariadb crash recovery failed
     - https://bugs.launchpad.net/starlingx/+bug/1922299
     - Martin seems to have fixed it, needs his review to merge: https://review.opendev.org/c/starlingx/openstack-armada-app/+/790321
       [Austin]  After analysis ,  This issue only some errors in logs ,  the openstack function is not impacted,   Martin's fix will eliminate this logs.   Suggest downgrade the to Medium.
     - 1925668 (Zhipeng): Bootstrap replay fails when changing mgmt subnet
     - https://bugs.launchpad.net/starlingx/+bug/1925668
     - fix up for review https://review.opendev.org/c/starlingx/ansible-playbooks/+/787942
     - Zhipeng addressed latest comments, needs to be reviewed
       [Austin]   Zhipeng has uploaded new patchset , waiting new review.

     - 1918318 (Martin): controller-0 is degraded due to the failure of its 'pci-irq-affinity-agent' process. Auto recovery of this major process is in progress."
     - https://bugs.launchpad.net/starlingx/+bug/1918318
     - Alex to provide info on the frequency of this intermittent issue
       [Austin]   0507/0510/0511 not see this issue. 0505 see this issue.

And this one is Medium, not High, but gets a mention in the sanity report - we

     - 1918420 (Martin): Alarm 750.003 - Application remove failure for stx-openstack
     - not high, but mentioned in stx.5.0 sanity report
     - https://bugs.launchpad.net/starlingx/+bug/1918420
     - review is up: https://review.opendev.org/c/starlingx/ha/+/788897

[1] https://etherpad.opendev.org/p/stx-gating-bugs

-----Original Message-----
From: Zvonar, Bill
Sent: Monday, April 12, 2021 8:23 AM
To: starlingx-discuss at lists.starlingx.io
Subject: RE: stx.5.0 Launchpads

Hi all, as of now we have 5 high importance bugs tagged against stx.5.0 - 2 new ones came in recently.

From https://etherpad.opendev.org/p/stx-gating-bugs:

Build
        1891416 (Davlet):  stx-platformclients docker image build doesn't include the correct content
        https://bugs.launchpad.net/starlingx/+bug/1891416
        fix in progress
Security
        1921511 (Zhipeng):  kubernetes-ca cert/key are replaced by etcd-ca cert/key
        https://bugs.launchpad.net/starlingx/+bug/1921511
        fix is under review https://review.opendev.org/c/starlingx/ansible-playbooks/+/784006
Update
        1913455 (Adriano): Fail to  swact to controller-1 after upgrade
        https://bugs.launchpad.net/starlingx/+bug/1913455
        testing is underway, going well so far Dist Cloud
        19230793 (Tee): Distributed Cloud: Remote install failure during orchestrated subcloud upgrade due to non-encoded bmc password
        https://bugs.launchpad.net/starlingx/+bug/1923073
        new as of Friday
Helm
        1923185 (Mihnea): Restore fails when launching Armada with Helm v3
        https://bugs.launchpad.net/starlingx/+bug/1923185
        new as of Friday

-----Original Message-----
From: Zvonar, Bill
Sent: Wednesday, March 31, 2021 12:28 PM
To: starlingx-discuss at lists.starlingx.io
Subject: stx.5.0 Launchpads

Hi all,

As discussed in the Community & Release Team calls, as we near the end of stx.5.0, we need to focus on fixing stx.5.0 launchpads

Per the stx.5.0 release plan [0], we have 2 weeks until RC1 (4/14) and just over a month until the forecast release date (5/5).

Generally, we want to have fixes for all known issues in place by RC1, so let's say we have 2 weeks.

And this means all Critical & High importance Launchpads, and as many Medium importance Launchpads as possible.

We currently have 1 Critical, 5 High & ~33 Medium, as of a snapshot this morning.  The full list of stx.5.0 Launchpads is at [2].

So, let's put focus on fixing the bugs in our respective domains.

I created an etherpad [1] with a snapshot of the bugs per domain & copied that list below.

Bill...

[0] https://docs.google.com/spreadsheets/d/1JbOQELqXG_GDoP1jo6YoRDytpTkJEs89TVimzvrqc_A/edit#gid=0
[1] https://etherpad.opendev.org/p/stx-gating-bugs
[2] https://bugs.launchpad.net/starlingx/+bugs?field.tag=stx.5.0

*Importance = Critical

OpenStack
        1917308:  Stx-openstack apply-fail after swact standby controller, lock, unlock standby  controller  https://bugs.launchpad.net/starlingx/+bug/1917308

*Importance = High

Apps
        1917352: stx-sdo  apply fails to pull images from registry.local  https://bugs.launchpad.net/starlingx/+bug/1917352
Build
        1891416:  stx-platformclients docker image build doesn't include the correct content  https://bugs.launchpad.net/starlingx/+bug/1891416
Config
        1918943: Regression -  Bootstrap replay following a cluster_host_subnet reconfiguration fails  https://bugs.launchpad.net/starlingx/+bug/1918943
Security
        1921511:  kubernetes-ca cert/key are replaced by etcd-ca cert/key  https://bugs.launchpad.net/starlingx/+bug/1921511
Update
        1913455: Fail to  swact to controller-1 after upgrade  https://bugs.launchpad.net/starlingx/+bug/1913455

*Importance = Medium

Apps
        1888900: System  remove,apply fails to start the vault pods with TLS handshake error  https://bugs.launchpad.net/starlingx/+bug/1888900
Apps,  Containers
        1882485: cert-manager  application in apply-failed status since initial controller-1 unlock  https://bugs.launchpad.net/starlingx/+bug/1882485
        1883555: applying  custom system application on controller-1 failed  https://bugs.launchpad.net/starlingx/+bug/1883555
Build
        1904893: clarity of  Layered Build Reference in StarlingX  https://bugs.launchpad.net/starlingx/+bug/1904893
        1921149: centos7:  Python < 2.7.9 not supported by PyPI  https://bugs.launchpad.net/starlingx/+bug/1921149
Config
        1884335: DC: Worker  node went for second reboot after all nodes power down/up  https://bugs.launchpad.net/starlingx/+bug/1884335
        1889101: 100.114  alarm "NTP cannot reach external time source; syncing with peer  controller only " raised when other external NTP servers are available  https://bugs.launchpad.net/starlingx/+bug/1889101
        1890970: controller  unable to unlock after reinstall (inv_state stuck at reinstalling)  https://bugs.launchpad.net/starlingx/+bug/1890970
        1895555: OAM IP  change needs double lock/unlock controllers for IPV6 system  https://bugs.launchpad.net/starlingx/+bug/1895555
        1918139: On AIO  hosts, kuberenetes is starting before key resources are initialized  https://bugs.launchpad.net/starlingx/+bug/1918139
Containers
        1883791: DC subcloud  bootsrap failed - nginx-ingress-controller   apply  failed  https://bugs.launchpad.net/starlingx/+bug/1883791
        1883953: Cert-manager  app failed to apply after controller-1 of the system controller is  upgraded   https://bugs.launchpad.net/starlingx/+bug/1883953
        1887105: Using python  helm plugin to dynamically override helmchart will cause an issue   https://bugs.launchpad.net/starlingx/+bug/1887105
        1887438: Controller-0  Not Ready after force rebooting active controller (Controller-1)  https://bugs.launchpad.net/starlingx/+bug/1887438
        1901911:  platform-integration-manifest apply failed due to error Install release:  stx-rbd-provisioner  https://bugs.launchpad.net/starlingx/+bug/1901911
        1903090: DC: helm  operation failure during merging helm overrides  https://bugs.launchpad.net/starlingx/+bug/1903090
Containers, DC
        1902266: Failed  scheduling of armada pod resulting in subcloud bootstrap failure  https://bugs.launchpad.net/starlingx/+bug/1902266
DC, GUI
        1886536: Horizon:  error occurred when attempt to download subcloud openrc file  https://bugs.launchpad.net/starlingx/+bug/1886536
GUI
        1890503: starlingx  gui dc_admin deploy status field needs updating  https://bugs.launchpad.net/starlingx/+bug/1890503
HA
        1893669: swact is not  triggered after killing dnsmasq process within 90 seconds  https://bugs.launchpad.net/starlingx/+bug/1893669
Networking
        1911213: Global  network policy not found in default namespace  https://bugs.launchpad.net/starlingx/+bug/1911213
OpenStack
        1888178: Neturon vm  in error status after resize revert due to "Virtual Interface creation  failed"  https://bugs.launchpad.net/starlingx/+bug/1888178
        1904729: Deprecate  vcpu_pin_set for stx-openstack in favor of new configs  https://bugs.launchpad.net/starlingx/+bug/1904729
Storage
        1883975: Disk  partitions not aligned  https://bugs.launchpad.net/starlingx/+bug/1883975
        1884136: file system  modify give error and modifies fs  https://bugs.launchpad.net/starlingx/+bug/1884136
        1884184: Storage node  configuration failed after performing DOR  https://bugs.launchpad.net/starlingx/+bug/1884184
        1890293: ceph-osds  are down after swact operation generating a coredump  https://bugs.launchpad.net/starlingx/+bug/1890293
        1892907: Ceph storage  condition alarm was not cleared after lock and unlock controller and swact  https://bugs.launchpad.net/starlingx/+bug/1892907
        1896628:  rook-ceph-apps upload failed to find an application tarball  https://bugs.launchpad.net/starlingx/+bug/1896628
Tools
        1886868: Need  detailed resource cpu and memory info in collect  https://bugs.launchpad.net/starlingx/+bug/1886868
        1911935: collect is  missing etcdctl output  https://bugs.launchpad.net/starlingx/+bug/1911935
Update
        1894050: AIO-DX:  Backup and Restore: After Reinstall controller-1, auto apply cert-manager and  platform-integration failed  https://bugs.launchpad.net/starlingx/+bug/1894050
        1913465: AIODX  upgrade：cgts-vg partition space is decreased on controller-0  https://bugs.launchpad.net/starlingx/+bug/1913465
_______________________________________________
Starlingx-discuss mailing list
Starlingx-discuss at lists.starlingx.io
http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
_______________________________________________
Starlingx-discuss mailing list
Starlingx-discuss at lists.starlingx.io
http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss

More information about the Starlingx-discuss mailing list