[Starlingx-discuss] bug severity and priority

Fri Jul 19 15:01:39 UTC 2019

> -----Original Message-----
> From: Xie, Cindy
> Sent: Friday, July 19, 2019 9:35 AM
> To: Perez Carranza, Jose <jose.perez.carranza at intel.com>; Saul Wold
> <sgw at linux.intel.com>; starlingx-discuss at lists.starlingx.io
> Subject: RE: [Starlingx-discuss] bug severity and priority
> 
> Jose,
> Just to clarify: for the weekly bug triage meeting, you only ask to triage the
> new bugs, right?

Yes, only the new ones should be triaged. 

> 
> My concern is about the triage frequency: right now, the new bugs are
> triaged almost on daily basis, mostly by Ghada by consulting technical expert.
> If we switch to a triage meeting, now sure how the new LP can be handled
> timely.
> 
> But agree that having a triage meeting is a good idea.
> Thx. - cindy
> 

To mitigate this concern as Saul pointed out we should ensure to have a "triage section"section on subproject meeting but ensuring all the stakeholders for the specific bugs are online to provide feedback. 

> -----Original Message-----
> From: Perez Carranza, Jose [mailto:jose.perez.carranza at intel.com]
> Sent: Friday, July 19, 2019 8:16 PM
> To: Saul Wold <sgw at linux.intel.com>; starlingx-discuss at lists.starlingx.io
> Subject: Re: [Starlingx-discuss] bug severity and priority
> 
> > -----Original Message-----
> > From: Saul Wold [mailto:sgw at linux.intel.com]
> > Sent: Thursday, July 18, 2019 10:57 PM
> > To: starlingx-discuss at lists.starlingx.io
> > Subject: Re: [Starlingx-discuss] bug severity and priority
> >
> >
> > Folks,
> >
> > As I mentioned in a prior email about a previous project (Yocto
> > Project), we were also time-based (every 6 months).  We defined
> > Importance [0] of the bug based on Severity (chosen by submitter) and
> > Priority (assigned during a triage process). We had 5  Priory levels
> > in
> > Bugzilla: High, Medium+, Medium, Low and Undecided, these would map
> to
> > our Critical, High, Medium, Low and Undecided.
> 
> Those triage meetings were very helpful because they were live discussions
> about the bugs with all the stockholders. I think we should consider to have a
> weekly meeting just to triage bugs.
> 
> Regards,
> José
> >
> > This clearly frames it based on Milestones and releases due to the
> > time based nature of the Yocto Project.  Notice that the High/Critical
> > is the only one that is truly "gating" or milestone/release blocker,
> > the
> > Medium+, our High, won't block a milestone but be should be fixed for
> > Medium+a
> > release, but could be a dot.dot soon after the release.
> >
> > > Importance
> > > The Importance of the bug is defined by its Priority and Severity.
> > > The
> > Priority classifies the bug's fixing order. In other words, how soon
> > will it get fixed relative to other bugs? Priorities are set during
> > the bug Triage meeting and cannot be changed by the user. The priority
> > appears to the left of the Severity field. Here are the values that
> > Priority can be set to during the Triage
> > meeting:
> > >
> > > High -- Bug fixing is planned immediately for the target milestone.
> > Milestone cannot be released if there is a high bug opened against the
> > milestone. High priority issues cause major functional loss of a
> > specific feature that is POR for the up-comping milestone. These
> > issues are easily hit by the user and greatly impact the user
> > experience or customer requirements. Finally, these issues could be
> > urgent security fixes that need to be corrected in a prior release.
> > The bug assignee is not to change the target milestones for High bugs
> without prior approval of the Triage team.
> > > Medium+ -- Bug fixing is planned before the milestone and must be
> > > Medium+ fixed or
> > have a solution planned before the release is finalized. These issues
> > are not show-stoppers but have somewhat significant impact to system
> > functions and user experience.
> > > Medium -- These are important issues we keep track and try to plan
> > > fixing
> > for the release. They have limited impact for the system functions and
> > releases.
> > > Low -- Bug fixing is only done opportunistically. Generally not
> > > planned for
> > the up-coming project release. Issues that are not a POR feature
> > request, or are hard to reproduce fall into this category.
> > > Undecided -- These issues are newly reported and are undecided
> > > before
> > Triage. Issues that are a feature request, which isn't approved for
> > future release yet. This issue will be changed to have an actual
> > Priority after the Triage team approves it.
> > > Note: High impact but Low Priority bugs can be documented in the
> > > release
> > notes.
> > >
> > > The Severity indicates how much the issue impacted the person
> > > reporting
> > the bug. Severity can be categorized into five areas.
> > >
> > > Critical -- Crashes, hang, loss of data, negative impact to other
> > > components,
> > memory leak etc.
> > > Major -- Major loss of functionality of POR.
> > > Normal -- Regular issue, some loss of functionality under certain
> > circumstance. This is the default Severity.
> > > Minor -- Minor loss of functionality, or issues with easy workaround
> > available.
> > > Enhancement -- Request for enhancement or new feature to be worked.
> >
> > I hope the helps by provide a different viewpoint from another project.
> >
> > Sau!
> >
> > [0]
> > https://wiki.yoctoproject.org/wiki/Bugzilla_Configuration_and_Bug_Trac
> > king
> > #Importance
> >
> > On 7/17/19 3:41 AM, Zvonar, Bill wrote:
> > > Hi Cindy,
> > >
> > > Thought about this some more, sorry it took me so long to respond
> further.
> > >
> > > I agree with splitting out the definitions of release
> > > priority/importance
> > (which is subjective) from the technical severity (which is I'd say
> > much less subjective).
> > >
> > > Do we agree that one of the key next steps is to define the severity
> > > levels
> > for defects in different domains?
> > >
> > > Once we have those agreed and written down somewhere, they can be
> > used as guidance for people that are opening Launchpads, and for those
> > that screen them.  Someone will note that some bugs cross domains, so
> > it's not as simple as looking at one set of severity definitions, but
> > let's cross that bridge next.
> > >
> > > Then, if we've got general alignment on the severity definitions per
> > > domain,
> > we can sort out what to use as a QRC formula for a release, I think.
> > >
> > > Btw, it'd be nice if Launchpad had a field for Severity, so we could
> > > track that
> > more easily - does anybody know if we can just request this & get it
> > added as a custom field?
> > >
> > > Bill...
> > >
> > > -----Original Message-----
> > > From: Xie, Cindy <cindy.xie at intel.com>
> > > Sent: Wednesday, July 10, 2019 7:13 PM
> > > To: Zvonar, Bill <Bill.Zvonar at windriver.com>; starlingx-
> > discuss at lists.starlingx.io; Khalil, Ghada <Ghada.Khalil at windriver.com>
> > > Subject: RE: bug severity and priority
> > >
> > > Bill,
> > > I definitely agree that not all Medium shall be pushed to stx.3.0,
> > > this needs
> > to be assessed carefully. But if we combine the severity and priority
> > together, then this decision needs to put resource factor in consideration
> as well.
> > >
> > > Actually, I think it's confusing of calling individual LP "gating" -
> > > I understand
> > that we want to get the product quality to a good shape and want to
> > get bugs fixed as many as possible before we ship it. I will suggest
> > to use defects# as part of release criteria (QRC). Example could be:
> > >
> > > 	Number of Critical P1 defects	Zero
> > > 	Number of High P2 defects	< x
> > > 	Number of Medium P3 defects	< y
> > >
> > > And the only thing we need to agree on is the "x" and "y". It makes
> > > TSC or
> > release team to make decision easier. The QRC needs to be agreed
> > earlier instead of right before the release decision shall be made.
> > This way, we can really direct our engineering resource working on the
> > most important items and we all have an agreed common goal.
> > >
> > > Thanks. - cindy
> > >
> > > -----Original Message-----
> > > From: Zvonar, Bill [mailto:Bill.Zvonar at windriver.com]
> > > Sent: Thursday, July 11, 2019 1:39 AM
> > > To: Xie, Cindy <cindy.xie at intel.com>;
> > > starlingx-discuss at lists.starlingx.io;
> > Khalil, Ghada <Ghada.Khalil at windriver.com>
> > > Subject: RE: bug severity and priority
> > >
> > > Hi Cindy,
> > >
> > > Thanks for sending this, I think this gives us something to start
> > > the
> > discussion.
> > >
> > > However we decide to align on severity/priority (I'll comment on
> > > that more
> > later, need to think about it more), I think we need to be careful
> > before we move all mediums to 3.0, it may be too much of a Gordian knot
> solution.
> > >
> > > I think we need to assess the mediums (as Yong suggested earlier) to
> > > say
> > why they should or should not be in 2.0.  I also think this may help
> > us sort out what our gating criteria are.
> > >
> > > Bill...
> > >
> > > -----Original Message-----
> > > From: Xie, Cindy <cindy.xie at intel.com>
> > > Sent: Wednesday, July 10, 2019 10:42 AM
> > > To: starlingx-discuss at lists.starlingx.io; Zvonar, Bill
> > <Bill.Zvonar at windriver.com>; Khalil, Ghada
> > <Ghada.Khalil at windriver.com>
> > > Subject: bug severity and priority
> > >
> > > Bill/Ghada,
> > > I am sending out my definition of bug severity and priority:
> > >
> > > Bug Exposure or Severity	Definition
> > > 1- Critical	Product or key feature is not usable for intended purpose.
> > > 2- High		Product or key feature is not reliably usable for
> > intended purpose or use is significantly impaired
> > > 3 - Medium	Product or key feature is usable provided by a workaround
> > > 4 - Low		Tolerable impact to user experience with minimal
> > service and support costs
> > >
> > > Bug Priority	Definition
> > > P1 - Stopper	Resolution of this defect takes precedence over other
> defects
> > and most other development activities. This level is used to focus
> > maximum development team resources to resolve a defect in the shortest
> > possible timeframe.
> > > P2 - High	Resolution of the defect has precedence over resolving other
> > defects with lesser classifications of priority.  The urgency to fix a
> > P2 priority defect is imminent. - P2 priority defects are intended to
> > be resolved by the next planned external release of the software.
> > > P3 - Medium	Resolution of the defect has precedence over
> resolving other
> > defects with lesser classifications of priority.  - P3 priority
> > defects must have a planned timeframe for a verified resolution.
> > > P4 - Low	Resolution of the defect has least urgency to resolve, P4
> > priority defects may or may not have plans to resolve.
> > >
> > > Let's discuss this and agree how we'd like to use them. My
> > > suggestion for
> > current "Medium" is to we can mark them as "stx.3.0" and then in the
> > beginning of stx.3, they can move Priority to "high" due to the fact
> > they want to get them fixed in 3.0.
> > >
> > > But the bug severity should never change because they are standard.
> > >
> > > Thx. - cindy
> > >
> > >
> > > _______________________________________________
> > > Starlingx-discuss mailing list
> > > Starlingx-discuss at lists.starlingx.io
> > > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
> > >
> >
> > _______________________________________________
> > Starlingx-discuss mailing list
> > Starlingx-discuss at lists.starlingx.io
> > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
> _______________________________________________
> Starlingx-discuss mailing list
> Starlingx-discuss at lists.starlingx.io
> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss