[Starlingx-discuss] bug severity and priority

Xie, Cindy cindy.xie at intel.com
Fri Jul 19 14:34:58 UTC 2019 

Jose,
Just to clarify: for the weekly bug triage meeting, you only ask to triage the new bugs, right?

My concern is about the triage frequency: right now, the new bugs are triaged almost on daily basis, mostly by Ghada by consulting technical expert. If we switch to a triage meeting, now sure how the new LP can be handled timely.

But agree that having a triage meeting is a good idea.
Thx. - cindy

-----Original Message-----
From: Perez Carranza, Jose [mailto:jose.perez.carranza at intel.com] 
Sent: Friday, July 19, 2019 8:16 PM
To: Saul Wold <sgw at linux.intel.com>; starlingx-discuss at lists.starlingx.io
Subject: Re: [Starlingx-discuss] bug severity and priority

> -----Original Message-----
> From: Saul Wold [mailto:sgw at linux.intel.com]
> Sent: Thursday, July 18, 2019 10:57 PM
> To: starlingx-discuss at lists.starlingx.io
> Subject: Re: [Starlingx-discuss] bug severity and priority
> 
> 
> Folks,
> 
> As I mentioned in a prior email about a previous project (Yocto 
> Project), we were also time-based (every 6 months).  We defined 
> Importance [0] of the bug based on Severity (chosen by submitter) and 
> Priority (assigned during a triage process). We had 5  Priory levels 
> in
> Bugzilla: High, Medium+, Medium, Low and Undecided, these would map to 
> our Critical, High, Medium, Low and Undecided.

Those triage meetings were very helpful because they were live discussions about the bugs with all the stockholders. I think we should consider to have a weekly meeting just to triage bugs.     

Regards,
José
> 
> This clearly frames it based on Milestones and releases due to the 
> time based nature of the Yocto Project.  Notice that the High/Critical 
> is the only one that is truly "gating" or milestone/release blocker, 
> the
> Medium+, our High, won't block a milestone but be should be fixed for 
> Medium+a
> release, but could be a dot.dot soon after the release.
> 
> > Importance
> > The Importance of the bug is defined by its Priority and Severity. 
> > The
> Priority classifies the bug's fixing order. In other words, how soon 
> will it get fixed relative to other bugs? Priorities are set during 
> the bug Triage meeting and cannot be changed by the user. The priority 
> appears to the left of the Severity field. Here are the values that 
> Priority can be set to during the Triage
> meeting:
> >
> > High -- Bug fixing is planned immediately for the target milestone.
> Milestone cannot be released if there is a high bug opened against the 
> milestone. High priority issues cause major functional loss of a 
> specific feature that is POR for the up-comping milestone. These 
> issues are easily hit by the user and greatly impact the user 
> experience or customer requirements. Finally, these issues could be 
> urgent security fixes that need to be corrected in a prior release. 
> The bug assignee is not to change the target milestones for High bugs without prior approval of the Triage team.
> > Medium+ -- Bug fixing is planned before the milestone and must be 
> > Medium+ fixed or
> have a solution planned before the release is finalized. These issues 
> are not show-stoppers but have somewhat significant impact to system 
> functions and user experience.
> > Medium -- These are important issues we keep track and try to plan 
> > fixing
> for the release. They have limited impact for the system functions and 
> releases.
> > Low -- Bug fixing is only done opportunistically. Generally not 
> > planned for
> the up-coming project release. Issues that are not a POR feature 
> request, or are hard to reproduce fall into this category.
> > Undecided -- These issues are newly reported and are undecided 
> > before
> Triage. Issues that are a feature request, which isn't approved for 
> future release yet. This issue will be changed to have an actual 
> Priority after the Triage team approves it.
> > Note: High impact but Low Priority bugs can be documented in the 
> > release
> notes.
> >
> > The Severity indicates how much the issue impacted the person 
> > reporting
> the bug. Severity can be categorized into five areas.
> >
> > Critical -- Crashes, hang, loss of data, negative impact to other 
> > components,
> memory leak etc.
> > Major -- Major loss of functionality of POR.
> > Normal -- Regular issue, some loss of functionality under certain
> circumstance. This is the default Severity.
> > Minor -- Minor loss of functionality, or issues with easy workaround
> available.
> > Enhancement -- Request for enhancement or new feature to be worked.
> 
> I hope the helps by provide a different viewpoint from another project.
> 
> Sau!
> 
> [0]
> https://wiki.yoctoproject.org/wiki/Bugzilla_Configuration_and_Bug_Trac
> king
> #Importance
> 
> On 7/17/19 3:41 AM, Zvonar, Bill wrote:
> > Hi Cindy,
> >
> > Thought about this some more, sorry it took me so long to respond further.
> >
> > I agree with splitting out the definitions of release 
> > priority/importance
> (which is subjective) from the technical severity (which is I'd say 
> much less subjective).
> >
> > Do we agree that one of the key next steps is to define the severity 
> > levels
> for defects in different domains?
> >
> > Once we have those agreed and written down somewhere, they can be
> used as guidance for people that are opening Launchpads, and for those 
> that screen them.  Someone will note that some bugs cross domains, so 
> it's not as simple as looking at one set of severity definitions, but 
> let's cross that bridge next.
> >
> > Then, if we've got general alignment on the severity definitions per 
> > domain,
> we can sort out what to use as a QRC formula for a release, I think.
> >
> > Btw, it'd be nice if Launchpad had a field for Severity, so we could 
> > track that
> more easily - does anybody know if we can just request this & get it 
> added as a custom field?
> >
> > Bill...
> >
> > -----Original Message-----
> > From: Xie, Cindy <cindy.xie at intel.com>
> > Sent: Wednesday, July 10, 2019 7:13 PM
> > To: Zvonar, Bill <Bill.Zvonar at windriver.com>; starlingx-
> discuss at lists.starlingx.io; Khalil, Ghada <Ghada.Khalil at windriver.com>
> > Subject: RE: bug severity and priority
> >
> > Bill,
> > I definitely agree that not all Medium shall be pushed to stx.3.0, 
> > this needs
> to be assessed carefully. But if we combine the severity and priority 
> together, then this decision needs to put resource factor in consideration as well.
> >
> > Actually, I think it's confusing of calling individual LP "gating" - 
> > I understand
> that we want to get the product quality to a good shape and want to 
> get bugs fixed as many as possible before we ship it. I will suggest 
> to use defects# as part of release criteria (QRC). Example could be:
> >
> > 	Number of Critical P1 defects	Zero
> > 	Number of High P2 defects	< x
> > 	Number of Medium P3 defects	< y
> >
> > And the only thing we need to agree on is the "x" and "y". It makes 
> > TSC or
> release team to make decision easier. The QRC needs to be agreed 
> earlier instead of right before the release decision shall be made. 
> This way, we can really direct our engineering resource working on the 
> most important items and we all have an agreed common goal.
> >
> > Thanks. - cindy
> >
> > -----Original Message-----
> > From: Zvonar, Bill [mailto:Bill.Zvonar at windriver.com]
> > Sent: Thursday, July 11, 2019 1:39 AM
> > To: Xie, Cindy <cindy.xie at intel.com>; 
> > starlingx-discuss at lists.starlingx.io;
> Khalil, Ghada <Ghada.Khalil at windriver.com>
> > Subject: RE: bug severity and priority
> >
> > Hi Cindy,
> >
> > Thanks for sending this, I think this gives us something to start 
> > the
> discussion.
> >
> > However we decide to align on severity/priority (I'll comment on 
> > that more
> later, need to think about it more), I think we need to be careful 
> before we move all mediums to 3.0, it may be too much of a Gordian knot solution.
> >
> > I think we need to assess the mediums (as Yong suggested earlier) to 
> > say
> why they should or should not be in 2.0.  I also think this may help 
> us sort out what our gating criteria are.
> >
> > Bill...
> >
> > -----Original Message-----
> > From: Xie, Cindy <cindy.xie at intel.com>
> > Sent: Wednesday, July 10, 2019 10:42 AM
> > To: starlingx-discuss at lists.starlingx.io; Zvonar, Bill
> <Bill.Zvonar at windriver.com>; Khalil, Ghada 
> <Ghada.Khalil at windriver.com>
> > Subject: bug severity and priority
> >
> > Bill/Ghada,
> > I am sending out my definition of bug severity and priority:
> >
> > Bug Exposure or Severity	Definition
> > 1- Critical	Product or key feature is not usable for intended purpose.
> > 2- High		Product or key feature is not reliably usable for
> intended purpose or use is significantly impaired
> > 3 - Medium	Product or key feature is usable provided by a workaround
> > 4 - Low		Tolerable impact to user experience with minimal
> service and support costs
> >
> > Bug Priority	Definition
> > P1 - Stopper	Resolution of this defect takes precedence over other defects
> and most other development activities. This level is used to focus 
> maximum development team resources to resolve a defect in the shortest 
> possible timeframe.
> > P2 - High	Resolution of the defect has precedence over resolving other
> defects with lesser classifications of priority.  The urgency to fix a 
> P2 priority defect is imminent. - P2 priority defects are intended to 
> be resolved by the next planned external release of the software.
> > P3 - Medium	Resolution of the defect has precedence over resolving other
> defects with lesser classifications of priority.  - P3 priority 
> defects must have a planned timeframe for a verified resolution.
> > P4 - Low	Resolution of the defect has least urgency to resolve, P4
> priority defects may or may not have plans to resolve.
> >
> > Let's discuss this and agree how we'd like to use them. My 
> > suggestion for
> current "Medium" is to we can mark them as "stx.3.0" and then in the 
> beginning of stx.3, they can move Priority to "high" due to the fact 
> they want to get them fixed in 3.0.
> >
> > But the bug severity should never change because they are standard.
> >
> > Thx. - cindy
> >
> >
> > _______________________________________________
> > Starlingx-discuss mailing list
> > Starlingx-discuss at lists.starlingx.io
> > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
> >
> 
> _______________________________________________
> Starlingx-discuss mailing list
> Starlingx-discuss at lists.starlingx.io
> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
_______________________________________________
Starlingx-discuss mailing list
Starlingx-discuss at lists.starlingx.io
http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss

More information about the Starlingx-discuss mailing list