[Starlingx-discuss] DRAFT release policy

Tue Apr 2 18:23:43 UTC 2019

Hi Bruce,

If none of the anchor features will be ready by MS-3, then we have no choice but to reforecast.

Then we get into the question of by how much, and which, if any anchor features can be excluded (which doesn't totally make sense to me - why would we have called it an anchor in the first place).  

This Release Churn is another heading that we need to add to the release policy, and whatever we come up with, we'll need to get it approved by the TSC, I think.

Bill... 

-----Original Message-----
From: Jones, Bruce E <bruce.e.jones at intel.com> 
Sent: Tuesday, April 2, 2019 1:02 PM
To: Khalil, Ghada <Ghada.Khalil at windriver.com>
Cc: starlingx-discuss <starlingx-discuss at lists.starlingx.io>; Dean Troyer <dtroyer at gmail.com>; Seiler, Glenn <glenn.seiler at windriver.com>
Subject: Re: [Starlingx-discuss] DRAFT release policy

Good feedback, Ghada, thank you.  I have updated the document accordingly.  I've also made a few other changes like adding test case readiness as a release criteria and proposed a method for handing anchor features that aren't completed by MS-3.   I've removed the previously resolved (and much appreciated) feedback in the interests of readability.

Link and updated text below.

       Brucej

This file is: https://etherpad.openstack.org/p/stx-release-policy-draft

This is a draft document for release planning.  Comments and feedback welcomed!

Openstack Release Policy
====================

The StarlingX project follows the release model defined in https://docs.openstack.org/project-team-guide/release-management.html using the "Trailing the common cycle" due to our dependency on upstream OpenStack projects.

Release Planning
==============

Initial release planning starts at the Open Infrastructure PTG meetings, where the TSC and community members discuss candidate features for the next release.  The TSC then will review and approve a feature list for the release, will identify any release gating "anchor" features, and set a target date.  The recommended target date is the date of the next OpenStack release plus 6 weeks.

The PTG meeting is also an opportunity to review the community's goals and to define goals for the release.

The overall Release Plan is created and managed by the Release sub-team by combining the TSC's input on content and target dates with input from the feature developers in the community and the Test team.  The plan will include a standard set of milestones as per the usual OpenStack release management process.  

The Release team will actively manage the plan over the course of the release, recommending any adjustments in content and dates to the community and to the TSC for approval.  We recognize that we are a new community working in a highly dynamic technology and that changes in our plans over time are normal and expected.  We will work as a community to be open and transparent about our release process, and to minimize change from the original plan.

Open issue: We should consider changing our release naming convention to something that isn't a date.

Defect Tracking
============

The release team shall review active and incoming bug reports and make an initial call as to whether or not the bug needs to be fixed in the next release.  If so, the bugs shall be tagged and tracked as the work on the release progresses.  The list of release gating bugs will be actively managed, reviewed and scrubbed by the Release team to ensure that bugs are properly categorized as release gating.

Release Policy
===========

The Release team, together with the Test team TLs/PLs, shall make a recommendation to the community and TSC that a release is ready to go.  Upon TSC approval, the release branches are tagged and the release documented.

That recommendation should be based on:
* Whether or not all anchor features in the release are complete, as per the input of the team(s) implementing the features and the results of Test team testing of the features
        * Proposal: All features identified as anchor features for a release need to be completed by the feature freeze milestone (MS-3).  In the event that an anchor feature is not complete before the release feature freeze milestone, the Release team will make a recommendation to the TSC to extend the milestone date or to defer the feature to the next release.

*  Whether or not all test cases planned for the release are complete and ready to run
    * Proposal: All planned test cases shall be ready before the start of formal release candidate testing (RC1 milestone)

* The completion and results of formal Testing performed by the Test team, measured by the percentage of planned tests attempted and the test pass rate
    * Proposal: 100% test cases attempted and 95% test cases passing in all configurations

* The status of release gating bugs
    * Proposal: All release gating bugs must be fixed prior to a StarlingX release, ideally before the RC1 milestone but certainly before the final release.  

Bugfix Releases
=============

The Release team, in conjunction with the community, can create a plan for a bug fix release.  This would be an update to a previous release to address important defects that are impacting our users.  The content would be fixes backported from master to the release branch, and would be based on both community and developer input regarding which fixes should be included.  Testing of a bug fix release should include at least verification testing of the fixes and any additional testing needed as determined by the Test team.

-----Original Message-----
From: Khalil, Ghada [mailto:Ghada.Khalil at windriver.com] 
Sent: Monday, April 1, 2019 5:19 PM
To: Jones, Bruce E <bruce.e.jones at intel.com>; Dean Troyer <dtroyer at gmail.com>; Seiler, Glenn <glenn.seiler at windriver.com>
Cc: starlingx-discuss <starlingx-discuss at lists.starlingx.io>
Subject: RE: [Starlingx-discuss] DRAFT release policy

Hi Bruce,
I have a comment regarding this point:

The severity and number of bugs open against the release
Proposal: No open Critical or High severity bugs against the release candidate.  Or maybe 1-3 Highs if we have a clear resolution plan (and a plan to release a patch against the release?)

         (ghada) We use an explicit tag to identify which bugs gate a particular release (regardless of severity). The whole list will have to be reviewed and scrubbed prior to reaching the release milestone. I don't feel it is sufficient to only review Critical / Major issues. [Example: On April 1/2019, there are 85 release gating bugs: only 13 are Critical/High. Yet it wouldn't be sufficient to only fix those 13 to ensure a quality release].
         In the Release Planning wiki[1] , we have previously stated this policy: 
All release gating issues are addressed or reviewed/accepted for deferral I feel we need to keep this.

[1] https://wiki.openstack.org/wiki/StarlingX/Release_Plan

-----Original Message-----
From: Jones, Bruce E [mailto:bruce.e.jones at intel.com]
Sent: Friday, March 29, 2019 2:10 PM
To: Jones, Bruce E; Dean Troyer; Seiler, Glenn
Cc: starlingx-discuss
Subject: Re: [Starlingx-discuss] DRAFT release policy

I've updated the etherpad with changes that reflect the current feedback.  Please review and add any additional feedback there.  Thank you!

https://etherpad.openstack.org/p/stx-release-policy-draft

      brucej

-----Original Message-----
From: Jones, Bruce E [mailto:bruce.e.jones at intel.com]
Sent: Friday, March 29, 2019 10:06 AM
To: Dean Troyer <dtroyer at gmail.com>; Seiler, Glenn <glenn.seiler at windriver.com>
Cc: starlingx-discuss <starlingx-discuss at lists.starlingx.io>
Subject: Re: [Starlingx-discuss] DRAFT release policy

Dean, Glenn - thank you for the feedback.  I agree with it.  

There is also some feedback in the etherpad.  I'm going to respond to both sets in the etherpad and try to improve the policy and the wording.

https://etherpad.openstack.org/p/stx-release-policy-draft

Thanks!

           brucej

-----Original Message-----
From: Dean Troyer [mailto:dtroyer at gmail.com]
Sent: Thursday, March 28, 2019 7:29 PM
To: Seiler, Glenn <glenn.seiler at windriver.com>
Cc: Jones, Bruce E <bruce.e.jones at intel.com>; starlingx-discuss <starlingx-discuss at lists.starlingx.io>
Subject: Re: [Starlingx-discuss] DRAFT release policy

On Thu, Mar 28, 2019 at 6:39 PM Seiler, Glenn <glenn.seiler at windriver.com> wrote:
> 1-     We need to move away from time-based releases
> 2-     We need to do twice a year releases.  This stmt, by definition, implies a time-gated release. Maybe it isn’t a specific date, but it is still time-gated.

The wording does need work, yes. After a short conversation with Bruce this afternoon (Bruce, correct me if I'm wrong here) I came away with the intention being more of increasing the lag from OpenStack releases rather than separating completely from the OpenStack release cycle which is likely to stay at approx 6 months for a while (that's a rabbit hole under the bike shed I'd like to avoid just now).

> As a nascent project, I think we need to show gradual and consistent progress.

++

> I did listen to much of the release team meeting today, and realize the trade-offs between big-rocks and timing are very difficult.
>
> Given the difficult choice of functionality versus timing, I personally think we need to show progress in getting to Stein and a container based distribution as major milestones in 1H and perhaps defer the Distributed Cloud capability to a 2H release.
>
> I don’t see anything intrinsically wrong with moving a specific date out; it happens all the time. But I also think a release should have some gate; i.e. we don’t move out of 1H. And if some functionality isn’t ready, then we move the functionality to another release in 2H.

We took a stab at estimating and missed, making adjustments now is normal and to be expected.

I agree with considering pushing distcloud to the next release because it is a) new functionality, and b) devs overlap with the container work and I think making the k8s infrastructure rock solid is much more important.  If we are too far off with system stability the ramifications will be harder to overcome than delaying a new feature.

> Anyway, that would be my vote, if I have one.

You totally have a voice as part of the community, I would like to hear from more folks here...

dt

--
Dean Troyer
dtroyer at gmail.com
_______________________________________________
Starlingx-discuss mailing list
Starlingx-discuss at lists.starlingx.io
http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
_______________________________________________
Starlingx-discuss mailing list
Starlingx-discuss at lists.starlingx.io
http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
_______________________________________________
Starlingx-discuss mailing list
Starlingx-discuss at lists.starlingx.io
http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss