[Starlingx-discuss] bug severity and priority

Fri Jul 19 14:35:41 UTC 2019

On 7/19/19 5:15 AM, Perez Carranza, Jose wrote:
>> -----Original Message-----
>> From: Saul Wold [mailto:sgw at linux.intel.com]
>> Sent: Thursday, July 18, 2019 10:57 PM
>> To: starlingx-discuss at lists.starlingx.io
>> Subject: Re: [Starlingx-discuss] bug severity and priority
>>
>>
>> Folks,
>>
>> As I mentioned in a prior email about a previous project (Yocto Project), we
>> were also time-based (every 6 months).  We defined Importance [0] of the
>> bug based on Severity (chosen by submitter) and Priority (assigned during a
>> triage process). We had 5  Priory levels in
>> Bugzilla: High, Medium+, Medium, Low and Undecided, these would map to
>> our Critical, High, Medium, Low and Undecided.
> 
> Those triage meetings were very helpful because they were live discussions about the bugs with all the stockholders. I think we should consider to have a weekly meeting just to triage bugs.
> 

Not sure we need a bug triage meeting, that should be happening in the 
sub-team meetings.

Sau!

> Regards,
> José
>>
>> This clearly frames it based on Milestones and releases due to the time
>> based nature of the Yocto Project.  Notice that the High/Critical is the only
>> one that is truly "gating" or milestone/release blocker, the
>> Medium+, our High, won't block a milestone but be should be fixed for a
>> release, but could be a dot.dot soon after the release.
>>
>>> Importance
>>> The Importance of the bug is defined by its Priority and Severity. The
>> Priority classifies the bug's fixing order. In other words, how soon will it get
>> fixed relative to other bugs? Priorities are set during the bug Triage meeting
>> and cannot be changed by the user. The priority appears to the left of the
>> Severity field. Here are the values that Priority can be set to during the Triage
>> meeting:
>>>
>>> High -- Bug fixing is planned immediately for the target milestone.
>> Milestone cannot be released if there is a high bug opened against the
>> milestone. High priority issues cause major functional loss of a specific
>> feature that is POR for the up-comping milestone. These issues are easily hit
>> by the user and greatly impact the user experience or customer
>> requirements. Finally, these issues could be urgent security fixes that need to
>> be corrected in a prior release. The bug assignee is not to change the target
>> milestones for High bugs without prior approval of the Triage team.
>>> Medium+ -- Bug fixing is planned before the milestone and must be fixed or
>> have a solution planned before the release is finalized. These issues are not
>> show-stoppers but have somewhat significant impact to system functions
>> and user experience.
>>> Medium -- These are important issues we keep track and try to plan fixing
>> for the release. They have limited impact for the system functions and
>> releases.
>>> Low -- Bug fixing is only done opportunistically. Generally not planned for
>> the up-coming project release. Issues that are not a POR feature request, or
>> are hard to reproduce fall into this category.
>>> Undecided -- These issues are newly reported and are undecided before
>> Triage. Issues that are a feature request, which isn't approved for future
>> release yet. This issue will be changed to have an actual Priority after the
>> Triage team approves it.
>>> Note: High impact but Low Priority bugs can be documented in the release
>> notes.
>>>
>>> The Severity indicates how much the issue impacted the person reporting
>> the bug. Severity can be categorized into five areas.
>>>
>>> Critical -- Crashes, hang, loss of data, negative impact to other components,
>> memory leak etc.
>>> Major -- Major loss of functionality of POR.
>>> Normal -- Regular issue, some loss of functionality under certain
>> circumstance. This is the default Severity.
>>> Minor -- Minor loss of functionality, or issues with easy workaround
>> available.
>>> Enhancement -- Request for enhancement or new feature to be worked.
>>
>> I hope the helps by provide a different viewpoint from another project.
>>
>> Sau!
>>
>> [0]
>> https://wiki.yoctoproject.org/wiki/Bugzilla_Configuration_and_Bug_Tracking
>> #Importance
>>
>> On 7/17/19 3:41 AM, Zvonar, Bill wrote:
>>> Hi Cindy,
>>>
>>> Thought about this some more, sorry it took me so long to respond further.
>>>
>>> I agree with splitting out the definitions of release priority/importance
>> (which is subjective) from the technical severity (which is I'd say much less
>> subjective).
>>>
>>> Do we agree that one of the key next steps is to define the severity levels
>> for defects in different domains?
>>>
>>> Once we have those agreed and written down somewhere, they can be
>> used as guidance for people that are opening Launchpads, and for those that
>> screen them.  Someone will note that some bugs cross domains, so it's not as
>> simple as looking at one set of severity definitions, but let's cross that bridge
>> next.
>>>
>>> Then, if we've got general alignment on the severity definitions per domain,
>> we can sort out what to use as a QRC formula for a release, I think.
>>>
>>> Btw, it'd be nice if Launchpad had a field for Severity, so we could track that
>> more easily - does anybody know if we can just request this & get it added as
>> a custom field?
>>>
>>> Bill...
>>>
>>> -----Original Message-----
>>> From: Xie, Cindy <cindy.xie at intel.com>
>>> Sent: Wednesday, July 10, 2019 7:13 PM
>>> To: Zvonar, Bill <Bill.Zvonar at windriver.com>; starlingx-
>> discuss at lists.starlingx.io; Khalil, Ghada <Ghada.Khalil at windriver.com>
>>> Subject: RE: bug severity and priority
>>>
>>> Bill,
>>> I definitely agree that not all Medium shall be pushed to stx.3.0, this needs
>> to be assessed carefully. But if we combine the severity and priority together,
>> then this decision needs to put resource factor in consideration as well.
>>>
>>> Actually, I think it's confusing of calling individual LP "gating" - I understand
>> that we want to get the product quality to a good shape and want to get bugs
>> fixed as many as possible before we ship it. I will suggest to use defects# as
>> part of release criteria (QRC). Example could be:
>>>
>>> 	Number of Critical P1 defects	Zero
>>> 	Number of High P2 defects	< x
>>> 	Number of Medium P3 defects	< y
>>>
>>> And the only thing we need to agree on is the "x" and "y". It makes TSC or
>> release team to make decision easier. The QRC needs to be agreed earlier
>> instead of right before the release decision shall be made. This way, we can
>> really direct our engineering resource working on the most important items
>> and we all have an agreed common goal.
>>>
>>> Thanks. - cindy
>>>
>>> -----Original Message-----
>>> From: Zvonar, Bill [mailto:Bill.Zvonar at windriver.com]
>>> Sent: Thursday, July 11, 2019 1:39 AM
>>> To: Xie, Cindy <cindy.xie at intel.com>; starlingx-discuss at lists.starlingx.io;
>> Khalil, Ghada <Ghada.Khalil at windriver.com>
>>> Subject: RE: bug severity and priority
>>>
>>> Hi Cindy,
>>>
>>> Thanks for sending this, I think this gives us something to start the
>> discussion.
>>>
>>> However we decide to align on severity/priority (I'll comment on that more
>> later, need to think about it more), I think we need to be careful before we
>> move all mediums to 3.0, it may be too much of a Gordian knot solution.
>>>
>>> I think we need to assess the mediums (as Yong suggested earlier) to say
>> why they should or should not be in 2.0.  I also think this may help us sort
>> out what our gating criteria are.
>>>
>>> Bill...
>>>
>>> -----Original Message-----
>>> From: Xie, Cindy <cindy.xie at intel.com>
>>> Sent: Wednesday, July 10, 2019 10:42 AM
>>> To: starlingx-discuss at lists.starlingx.io; Zvonar, Bill
>> <Bill.Zvonar at windriver.com>; Khalil, Ghada <Ghada.Khalil at windriver.com>
>>> Subject: bug severity and priority
>>>
>>> Bill/Ghada,
>>> I am sending out my definition of bug severity and priority:
>>>
>>> Bug Exposure or Severity	Definition
>>> 1- Critical	Product or key feature is not usable for intended purpose.
>>> 2- High		Product or key feature is not reliably usable for
>> intended purpose or use is significantly impaired
>>> 3 - Medium	Product or key feature is usable provided by a workaround
>>> 4 - Low		Tolerable impact to user experience with minimal
>> service and support costs
>>>
>>> Bug Priority	Definition
>>> P1 - Stopper	Resolution of this defect takes precedence over other defects
>> and most other development activities. This level is used to focus maximum
>> development team resources to resolve a defect in the shortest possible
>> timeframe.
>>> P2 - High	Resolution of the defect has precedence over resolving other
>> defects with lesser classifications of priority.  The urgency to fix a P2 priority
>> defect is imminent. - P2 priority defects are intended to be resolved by the
>> next planned external release of the software.
>>> P3 - Medium	Resolution of the defect has precedence over resolving other
>> defects with lesser classifications of priority.  - P3 priority defects must have a
>> planned timeframe for a verified resolution.
>>> P4 - Low	Resolution of the defect has least urgency to resolve, P4
>> priority defects may or may not have plans to resolve.
>>>
>>> Let's discuss this and agree how we'd like to use them. My suggestion for
>> current "Medium" is to we can mark them as "stx.3.0" and then in the
>> beginning of stx.3, they can move Priority to "high" due to the fact they want
>> to get them fixed in 3.0.
>>>
>>> But the bug severity should never change because they are standard.
>>>
>>> Thx. - cindy
>>>
>>>
>>> _______________________________________________
>>> Starlingx-discuss mailing list
>>> Starlingx-discuss at lists.starlingx.io
>>> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
>>>
>>
>> _______________________________________________
>> Starlingx-discuss mailing list
>> Starlingx-discuss at lists.starlingx.io
>> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
> _______________________________________________
> Starlingx-discuss mailing list
> Starlingx-discuss at lists.starlingx.io
> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
> 