[Starlingx-discuss] bug severity and priority

[Saul Wold]

Folks,

As I mentioned in a prior email about a previous project (Yocto 
Project), we were also time-based (every 6 months).  We defined 
Importance [0] of the bug based on Severity (chosen by submitter) and 
Priority (assigned during a triage process). We had 5  Priory levels in 
Bugzilla: High, Medium+, Medium, Low and Undecided, these would map to 
our Critical, High, Medium, Low and Undecided.

This clearly frames it based on Milestones and releases due to the time 
based nature of the Yocto Project.  Notice that the High/Critical is the 
only one that is truly "gating" or milestone/release blocker, the 
Medium+, our High, won't block a milestone but be should be fixed for a 
release, but could be a dot.dot soon after the release.

> Importance
> The Importance of the bug is defined by its Priority and Severity. The Priority classifies the bug's fixing order. In other words, how soon will it get fixed relative to other bugs? Priorities are set during the bug Triage meeting and cannot be changed by the user. The priority appears to the left of the Severity field. Here are the values that Priority can be set to during the Triage meeting:
> 
> High -- Bug fixing is planned immediately for the target milestone. Milestone cannot be released if there is a high bug opened against the milestone. High priority issues cause major functional loss of a specific feature that is POR for the up-comping milestone. These issues are easily hit by the user and greatly impact the user experience or customer requirements. Finally, these issues could be urgent security fixes that need to be corrected in a prior release. The bug assignee is not to change the target milestones for High bugs without prior approval of the Triage team.
> Medium+ -- Bug fixing is planned before the milestone and must be fixed or have a solution planned before the release is finalized. These issues are not show-stoppers but have somewhat significant impact to system functions and user experience.
> Medium -- These are important issues we keep track and try to plan fixing for the release. They have limited impact for the system functions and releases.
> Low -- Bug fixing is only done opportunistically. Generally not planned for the up-coming project release. Issues that are not a POR feature request, or are hard to reproduce fall into this category.
> Undecided -- These issues are newly reported and are undecided before Triage. Issues that are a feature request, which isn't approved for future release yet. This issue will be changed to have an actual Priority after the Triage team approves it.
> Note: High impact but Low Priority bugs can be documented in the release notes.
> 
> The Severity indicates how much the issue impacted the person reporting the bug. Severity can be categorized into five areas.
> 
> Critical -- Crashes, hang, loss of data, negative impact to other components, memory leak etc.
> Major -- Major loss of functionality of POR.
> Normal -- Regular issue, some loss of functionality under certain circumstance. This is the default Severity.
> Minor -- Minor loss of functionality, or issues with easy workaround available.
> Enhancement -- Request for enhancement or new feature to be worked.

I hope the helps by provide a different viewpoint from another project.

Sau!

[0] 
https://wiki.yoctoproject.org/wiki/Bugzilla_Configuration_and_Bug_Tracking#Importance

On 7/17/19 3:41 AM, Zvonar, Bill wrote:
> Hi Cindy,
> 
> Thought about this some more, sorry it took me so long to respond further.
> 
> I agree with splitting out the definitions of release priority/importance (which is subjective) from the technical severity (which is I'd say much less subjective).
> 
> Do we agree that one of the key next steps is to define the severity levels for defects in different domains?
> 
> Once we have those agreed and written down somewhere, they can be used as guidance for people that are opening Launchpads, and for those that screen them.  Someone will note that some bugs cross domains, so it's not as simple as looking at one set of severity definitions, but let's cross that bridge next.
> 
> Then, if we've got general alignment on the severity definitions per domain, we can sort out what to use as a QRC formula for a release, I think.
> 
> Btw, it'd be nice if Launchpad had a field for Severity, so we could track that more easily - does anybody know if we can just request this & get it added as a custom field?
> 
> Bill...
> 
> -----Original Message-----
> From: Xie, Cindy <cindy.xie at intel.com>
> Sent: Wednesday, July 10, 2019 7:13 PM
> To: Zvonar, Bill <Bill.Zvonar at windriver.com>; starlingx-discuss at lists.starlingx.io; Khalil, Ghada <Ghada.Khalil at windriver.com>
> Subject: RE: bug severity and priority
> 
> Bill,
> I definitely agree that not all Medium shall be pushed to stx.3.0, this needs to be assessed carefully. But if we combine the severity and priority together, then this decision needs to put resource factor in consideration as well.
> 
> Actually, I think it's confusing of calling individual LP "gating" - I understand that we want to get the product quality to a good shape and want to get bugs fixed as many as possible before we ship it. I will suggest to use defects# as part of release criteria (QRC). Example could be:
> 
> 	Number of Critical P1 defects	Zero
> 	Number of High P2 defects	< x
> 	Number of Medium P3 defects	< y
> 
> And the only thing we need to agree on is the "x" and "y". It makes TSC or release team to make decision easier. The QRC needs to be agreed earlier instead of right before the release decision shall be made. This way, we can really direct our engineering resource working on the most important items and we all have an agreed common goal.
> 
> Thanks. - cindy
> 
> -----Original Message-----
> From: Zvonar, Bill [mailto:Bill.Zvonar at windriver.com]
> Sent: Thursday, July 11, 2019 1:39 AM
> To: Xie, Cindy <cindy.xie at intel.com>; starlingx-discuss at lists.starlingx.io; Khalil, Ghada <Ghada.Khalil at windriver.com>
> Subject: RE: bug severity and priority
> 
> Hi Cindy,
> 
> Thanks for sending this, I think this gives us something to start the discussion.
> 
> However we decide to align on severity/priority (I'll comment on that more later, need to think about it more), I think we need to be careful before we move all mediums to 3.0, it may be too much of a Gordian knot solution.
> 
> I think we need to assess the mediums (as Yong suggested earlier) to say why they should or should not be in 2.0.  I also think this may help us sort out what our gating criteria are.
> 
> Bill...
> 
> -----Original Message-----
> From: Xie, Cindy <cindy.xie at intel.com>
> Sent: Wednesday, July 10, 2019 10:42 AM
> To: starlingx-discuss at lists.starlingx.io; Zvonar, Bill <Bill.Zvonar at windriver.com>; Khalil, Ghada <Ghada.Khalil at windriver.com>
> Subject: bug severity and priority
> 
> Bill/Ghada,
> I am sending out my definition of bug severity and priority:
> 
> Bug Exposure or Severity	Definition
> 1- Critical	Product or key feature is not usable for intended purpose.
> 2- High		Product or key feature is not reliably usable for intended purpose or use is significantly impaired
> 3 - Medium	Product or key feature is usable provided by a workaround
> 4 - Low		Tolerable impact to user experience with minimal service and support costs
> 	
> Bug Priority	Definition
> P1 - Stopper	Resolution of this defect takes precedence over other defects and most other development activities. This level is used to focus maximum development team resources to resolve a defect in the shortest possible timeframe.
> P2 - High	Resolution of the defect has precedence over resolving other defects with lesser classifications of priority.  The urgency to fix a P2 priority  defect is imminent. - P2 priority defects are intended to be resolved by the next planned external release of the software.
> P3 - Medium	Resolution of the defect has precedence over resolving other defects with lesser classifications of priority.  - P3 priority defects must have a planned timeframe for a verified resolution.
> P4 - Low	Resolution of the defect has least urgency to resolve, P4 priority defects may or may not have plans to resolve.
> 
> Let's discuss this and agree how we'd like to use them. My suggestion for current "Medium" is to we can mark them as "stx.3.0" and then in the beginning of stx.3, they can move Priority to "high" due to the fact they want to get them fixed in 3.0.
> 
> But the bug severity should never change because they are standard.
> 
> Thx. - cindy
> 
> 
> _______________________________________________
> Starlingx-discuss mailing list
> Starlingx-discuss at lists.starlingx.io
> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
>