[Starlingx-discuss] bug severity and priority

[Perez Carranza, Jose]

> -----Original Message-----
> From: Saul Wold [mailto:sgw at linux.intel.com]
> Sent: Thursday, July 18, 2019 10:57 PM
> To: starlingx-discuss at lists.starlingx.io
> Subject: Re: [Starlingx-discuss] bug severity and priority
> 
> 
> Folks,
> 
> As I mentioned in a prior email about a previous project (Yocto Project), we
> were also time-based (every 6 months).  We defined Importance [0] of the
> bug based on Severity (chosen by submitter) and Priority (assigned during a
> triage process). We had 5  Priory levels in
> Bugzilla: High, Medium+, Medium, Low and Undecided, these would map to
> our Critical, High, Medium, Low and Undecided.

Those triage meetings were very helpful because they were live discussions about the bugs with all the stockholders. I think we should consider to have a weekly meeting just to triage bugs.     

Regards,
José
> 
> This clearly frames it based on Milestones and releases due to the time
> based nature of the Yocto Project.  Notice that the High/Critical is the only
> one that is truly "gating" or milestone/release blocker, the
> Medium+, our High, won't block a milestone but be should be fixed for a
> release, but could be a dot.dot soon after the release.
> 
> > Importance
> > The Importance of the bug is defined by its Priority and Severity. The
> Priority classifies the bug's fixing order. In other words, how soon will it get
> fixed relative to other bugs? Priorities are set during the bug Triage meeting
> and cannot be changed by the user. The priority appears to the left of the
> Severity field. Here are the values that Priority can be set to during the Triage
> meeting:
> >
> > High -- Bug fixing is planned immediately for the target milestone.
> Milestone cannot be released if there is a high bug opened against the
> milestone. High priority issues cause major functional loss of a specific
> feature that is POR for the up-comping milestone. These issues are easily hit
> by the user and greatly impact the user experience or customer
> requirements. Finally, these issues could be urgent security fixes that need to
> be corrected in a prior release. The bug assignee is not to change the target
> milestones for High bugs without prior approval of the Triage team.
> > Medium+ -- Bug fixing is planned before the milestone and must be fixed or
> have a solution planned before the release is finalized. These issues are not
> show-stoppers but have somewhat significant impact to system functions
> and user experience.
> > Medium -- These are important issues we keep track and try to plan fixing
> for the release. They have limited impact for the system functions and
> releases.
> > Low -- Bug fixing is only done opportunistically. Generally not planned for
> the up-coming project release. Issues that are not a POR feature request, or
> are hard to reproduce fall into this category.
> > Undecided -- These issues are newly reported and are undecided before
> Triage. Issues that are a feature request, which isn't approved for future
> release yet. This issue will be changed to have an actual Priority after the
> Triage team approves it.
> > Note: High impact but Low Priority bugs can be documented in the release
> notes.
> >
> > The Severity indicates how much the issue impacted the person reporting
> the bug. Severity can be categorized into five areas.
> >
> > Critical -- Crashes, hang, loss of data, negative impact to other components,
> memory leak etc.
> > Major -- Major loss of functionality of POR.
> > Normal -- Regular issue, some loss of functionality under certain
> circumstance. This is the default Severity.
> > Minor -- Minor loss of functionality, or issues with easy workaround
> available.
> > Enhancement -- Request for enhancement or new feature to be worked.
> 
> I hope the helps by provide a different viewpoint from another project.
> 
> Sau!
> 
> [0]
> https://wiki.yoctoproject.org/wiki/Bugzilla_Configuration_and_Bug_Tracking
> #Importance
> 
> On 7/17/19 3:41 AM, Zvonar, Bill wrote:
> > Hi Cindy,
> >
> > Thought about this some more, sorry it took me so long to respond further.
> >
> > I agree with splitting out the definitions of release priority/importance
> (which is subjective) from the technical severity (which is I'd say much less
> subjective).
> >
> > Do we agree that one of the key next steps is to define the severity levels
> for defects in different domains?
> >
> > Once we have those agreed and written down somewhere, they can be
> used as guidance for people that are opening Launchpads, and for those that
> screen them.  Someone will note that some bugs cross domains, so it's not as
> simple as looking at one set of severity definitions, but let's cross that bridge
> next.
> >
> > Then, if we've got general alignment on the severity definitions per domain,
> we can sort out what to use as a QRC formula for a release, I think.
> >
> > Btw, it'd be nice if Launchpad had a field for Severity, so we could track that
> more easily - does anybody know if we can just request this & get it added as
> a custom field?
> >
> > Bill...
> >
> > -----Original Message-----
> > From: Xie, Cindy <cindy.xie at intel.com>
> > Sent: Wednesday, July 10, 2019 7:13 PM
> > To: Zvonar, Bill <Bill.Zvonar at windriver.com>; starlingx-
> discuss at lists.starlingx.io; Khalil, Ghada <Ghada.Khalil at windriver.com>
> > Subject: RE: bug severity and priority
> >
> > Bill,
> > I definitely agree that not all Medium shall be pushed to stx.3.0, this needs
> to be assessed carefully. But if we combine the severity and priority together,
> then this decision needs to put resource factor in consideration as well.
> >
> > Actually, I think it's confusing of calling individual LP "gating" - I understand
> that we want to get the product quality to a good shape and want to get bugs
> fixed as many as possible before we ship it. I will suggest to use defects# as
> part of release criteria (QRC). Example could be:
> >
> > 	Number of Critical P1 defects	Zero
> > 	Number of High P2 defects	< x
> > 	Number of Medium P3 defects	< y
> >
> > And the only thing we need to agree on is the "x" and "y". It makes TSC or
> release team to make decision easier. The QRC needs to be agreed earlier
> instead of right before the release decision shall be made. This way, we can
> really direct our engineering resource working on the most important items
> and we all have an agreed common goal.
> >
> > Thanks. - cindy
> >
> > -----Original Message-----
> > From: Zvonar, Bill [mailto:Bill.Zvonar at windriver.com]
> > Sent: Thursday, July 11, 2019 1:39 AM
> > To: Xie, Cindy <cindy.xie at intel.com>; starlingx-discuss at lists.starlingx.io;
> Khalil, Ghada <Ghada.Khalil at windriver.com>
> > Subject: RE: bug severity and priority
> >
> > Hi Cindy,
> >
> > Thanks for sending this, I think this gives us something to start the
> discussion.
> >
> > However we decide to align on severity/priority (I'll comment on that more
> later, need to think about it more), I think we need to be careful before we
> move all mediums to 3.0, it may be too much of a Gordian knot solution.
> >
> > I think we need to assess the mediums (as Yong suggested earlier) to say
> why they should or should not be in 2.0.  I also think this may help us sort
> out what our gating criteria are.
> >
> > Bill...
> >
> > -----Original Message-----
> > From: Xie, Cindy <cindy.xie at intel.com>
> > Sent: Wednesday, July 10, 2019 10:42 AM
> > To: starlingx-discuss at lists.starlingx.io; Zvonar, Bill
> <Bill.Zvonar at windriver.com>; Khalil, Ghada <Ghada.Khalil at windriver.com>
> > Subject: bug severity and priority
> >
> > Bill/Ghada,
> > I am sending out my definition of bug severity and priority:
> >
> > Bug Exposure or Severity	Definition
> > 1- Critical	Product or key feature is not usable for intended purpose.
> > 2- High		Product or key feature is not reliably usable for
> intended purpose or use is significantly impaired
> > 3 - Medium	Product or key feature is usable provided by a workaround
> > 4 - Low		Tolerable impact to user experience with minimal
> service and support costs
> >
> > Bug Priority	Definition
> > P1 - Stopper	Resolution of this defect takes precedence over other defects
> and most other development activities. This level is used to focus maximum
> development team resources to resolve a defect in the shortest possible
> timeframe.
> > P2 - High	Resolution of the defect has precedence over resolving other
> defects with lesser classifications of priority.  The urgency to fix a P2 priority
> defect is imminent. - P2 priority defects are intended to be resolved by the
> next planned external release of the software.
> > P3 - Medium	Resolution of the defect has precedence over resolving other
> defects with lesser classifications of priority.  - P3 priority defects must have a
> planned timeframe for a verified resolution.
> > P4 - Low	Resolution of the defect has least urgency to resolve, P4
> priority defects may or may not have plans to resolve.
> >
> > Let's discuss this and agree how we'd like to use them. My suggestion for
> current "Medium" is to we can mark them as "stx.3.0" and then in the
> beginning of stx.3, they can move Priority to "high" due to the fact they want
> to get them fixed in 3.0.
> >
> > But the bug severity should never change because they are standard.
> >
> > Thx. - cindy
> >
> >
> > _______________________________________________
> > Starlingx-discuss mailing list
> > Starlingx-discuss at lists.starlingx.io
> > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
> >
> 
> _______________________________________________
> Starlingx-discuss mailing list
> Starlingx-discuss at lists.starlingx.io
> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss