Bug Gardening

From Foss2Serve
(Difference between revisions)
Jump to: navigation, search
(→Directions:)
m (fixed links)
 
(54 intermediate revisions by 5 users not shown)
Line 1: Line 1:
 
__NOTOC__
 
__NOTOC__
−
{| border="1"
 
−
|-
 
−
|'''Title''' || Bug Gardening / Bug Triage / Bug Grooming /
 
−
|-
 
−
|'''Overview''' || Most projects have a backlog of bugs that need to be periodically “gardened.”  Sometimes there are even old bugs that may have already been fixed that just haven’t been closed in the system.  This module familiarizes students with bug gardening (/bug triage/ bug grooming) techniques, *and* helps the community by doing some of that gardening.
 
−
|-
 
−
|'''Prerequisite Knowledge''' || Students should be familiar with how bug trackers work -- see [[http://foss2serve.org/index.php/Bug_Tracker_Activity| Bug Tracker Activity]]
 
−
|-
 
−
|'''Learning Objectives''' || What should the student be able to do after completing completed this activity?
 
−
|}
 
  
−
=== Background: ===
+
{{Learning Activity Overview
−
If the project you're working with has a guide for how to triage bugs, the students should read that first.
+
|title=
 +
Bug Grooming / Bug Triage / Bug Zapping / Bug Wrangling
 +
|overview=  
 +
Most projects have a backlog of bugs that need to be periodically "gardened".  Sometimes there are even old bugs that may have already been fixed that just haven’t been closed in the system.  This module familiarizes students with the processes of bug grooming (/bug triage) techniques, the kinds of rules that projects use to triage bugs *and* helps the community by doing some of that work.  
 +
|prerequisites=
 +
Students should be familiar with how bug trackers work -- see [[Intro to Bug Trackers (Activity)]].
 +
|objectives=
 +
# Review a bug to see if it still is relevant.
 +
# Explain the importance of "bug grooming".
 +
|process skills=
 +
}}
  
−
Here are two examples:
+
=== Background ===
−
* [http://www.gluster.org/community/documentation/index.php/Bug_triage Gluster Bug Triage Guidelines] and their accompanying [http://www.gluster.org/community/documentation/index.php/Bug_report_life_cycle Lifecycle of a Bug]
+
−
* [https://www.mediawiki.org/wiki/Bug_management/How_to_triage Bug Management - How to Triage (MediaWiki)]
+
  
−
The student should be familiar with how to use the bug tracker (see [[Bug_Tracker_Activity|Bug Tracker Activity]])
+
Bug Gardening (or Grooming or Triage) is the process by which projects:
 +
# confirm new bugs that have been submitted
 +
# gather more information about the bug when necessary
 +
# prioritize confirmed bugs
 +
# review bugs to ensure that they are being worked on
 +
# retire old bugs
  
−
If you're working with a specific open source project, you may want to ask the project (via IRC or mailing list) if there are a particular set of bugs that your students should focus on.  For example, in the Gluster project I noted above, if a bug is tagged POST it means that an initial patch for a bug has been put into the Gerrit code review tool.  If there
+
This activity, however, is focused on retiring bugs that are no longer relevant (#5), primarily because that is more clear-cut than full-fledged bug triage.
−
is a very old bug that's marked POST, there's a decent likelihood that the patch has been incorporated into the project and the bug wasn't closed for whatever reason.  If your students can verify this and close the bug, that's valuable.
+
  
−
Other projects may have similar situations that are worth asking about.
+
Bug Triage Guides explain the rules for how to treat bugs within a project.  Here are some examples of Bug Triage Guides:
 +
* [https://docs.gluster.org/en/latest/Contributors-Guide/Bug-Triage/] and their accompanying [https://docs.gluster.org/en/latest/Contributors-Guide/Bug-report-Life-Cycle/]
 +
* [https://www.mediawiki.org/wiki/Bug_management/How_to_triage Bug Management - How to Triage (MediaWiki)]
 +
* [https://wiki.gnome.org/Bugsquad/TriageGuide Gnome Bug Triage Guide]
 +
* [https://bugzilla.mozilla.org/page.cgi?id=etiquette.html Bugzilla "bug etiquette" guide]
  
−
=== Directions: ===
+
If your project has a triage guide, read it first.  If your project doesn't have a triage guide, I suggest contacting the project to find out if there is an established mailing list or IRC channel for discussing bugs.
−
This activity is best performed in groups of two or three students, who can work together to:
+
−
* identify a "suspect" bug in the bug tracker,  
+
−
* test it to see if it still exists, and
+
−
* if it doesn't, close the bug.  
+
  
−
Identifying the "suspect" bugs will be easiest if you have information from the project about their bug lifecycle or have information directly from the project about which bugs to focus on, but even in the absence of that information, there are some other clues that can be used to identify bugs that might no longer be bugs. 
+
=== Directions ===
  
−
One thing to look at is the version number of the software that the bug was reported against (versus the one that's current now). If the bug is for a version that's several releases out of date, there's a good likelihood that the issue may have been fixed (or may have just become irrelevant) in the meantime.
+
Though all bugs need to be triaged (see examples of Bug Triage Guides), this activity is focused on culling bugs that are no longer relevant, primarily because that is more clear-cut than full-fledged bug triage.
  
−
Another possible hint is the date on the bug: bugs that were reported years ago may have been fixed and are probably worth looking at first.
+
This activity is best performed in groups of two or three students.
  
−
=== Deliverables: ===
+
You should:
−
What will the student hand in?
+
  
−
=== Assessment: ===
+
# Read the bug triage guide for your project (if available).
−
How will the activity be graded?
+
# Join the IRC channel or mailing list for your project on which bugs are discussed.
−
   
+
# Identify a "suspect" bug in the bug tracker,
−
How will learning will be measured?
+
## Identifying the "suspect" bugs will be easier if you have information from the project about their bug lifecycle or which bugs they suggest you focus on.  However, even in the absence of that information, there are some other clues that can be used to identify bugs that may no longer exist. 
 +
## Look at the version number of the software that the bug was reported against (versus the one that's current now). If the bug is for a version that's several releases out of date, there's a good likelihood that the issue may have been fixed (or may have just become irrelevant) in the meantime.
 +
## Another possible hint is the date on the bug: bugs that were reported years ago may have been fixed and are probably worth looking at first.
 +
# Test the bug to see if it still exists. Once you have identified a bug to work on, try to verify the bug status.
 +
## At that point, if the bug is no longer relevant, you can take one of several courses of action, depending on how well-established your relationship with the project is:
 +
### If you have established a good working relationship with a project and have permissions to do so, you can close the bug (this is something a more advanced student might do).
 +
### You can add a comment to the bug with notes and a suggestion to close.  This is more appropriate for students with less project experience.
 +
## If, after testing, you cannot tell if a bug is still relevant -- that's ok. At that point you can:
 +
### Move on and try another bug, or
 +
### Ask for more information from the bug reporter or the person assigned to the bug.  This can be a valuable exposure to more of the project ecosystem.
  
−
Include sample assessment questions/rubrics.
+
=== Deliverables ===
  
−
=== Comments: ===
+
At the end of the exercise, the student teams should have a list of bugs that they investigated and the outcomes. In the ideal case, they will have been able to close a bug or two, but even if they haven't you should still receive a report out.  I suggest the following as a sample format (I've filled two lines as examples):
−
What should the instructor know before using this activity?
+
  
−
What are some likely difficulties that an instructor may encounter using this activity?
 
  
−
=== Additional Information: ===
+
{| class="wikitable"
−
{| border="1"
+
 
|-  
 
|-  
−
|'''Knowledge Area/Knowledge Unit''' || Software Development Fundamentals//Software Verification and Validation
+
| '''Bug Number''' ||'''Short Description'''||'''Bug Status'''||'''URL for Bug'''||'''Analysis'''||'''Actions Taken'''
 
|-
 
|-
−
|'''Topic''' || Defect tracking
+
| Bug 302338 || Machine Explodes on Disk Insertion || Active || http://example.com/fakebug.enter_bug.cgi?product=fake&bug=302338 || Bug submitted against version 2 of product, which was EOL'd in 2003  || Left comment suggesting that bug be closed   
 
|-
 
|-
−
|'''Level of Difficulty''' || Medium (requires some understanding of the intended functionality of the software, ability to use bug tracking software, and critical thinking skills)  
+
| Bug 42984 || Funny Noises || Active || http://example.com/fakebug.enter_bug.cgi?product=fake&bug=42984 || Bug submitted in 2010, patched in release 2 (2011) || Closed bug. 
 
|-
 
|-
−
|'''Estimated Time to Completion''' ||  How long should it take for the student to complete the activity?
+
|}
−
|-
+
 
−
|'''Materials/Environment''' || Student needs access to the project's bug tracker, internet access
+
=== Assessment ===
−
|-
+
 
−
|'''Author''' || Gina Likins
+
{| class="wikitable"
 +
|-
 +
|'''Criteria''' ||'''Level 1 (fail)'''||'''Level 2 (pass)'''||'''Level 3 (good)'''||'''Level 4 (exceptional)'''
 
|-
 
|-
−
|'''Source''' || [http://blog.smartbear.com/programming/ 14-ways-to-contribute-to-open-source-without-being-a-programming-genius-or-a-rock-star]
+
|Ability to evaluate/triage bugs || Unable to correctly evaluate bug status || Evaluates bug status correctly || Evaluates bug status correctly and recommends correct action  || Evaluates bug correctly, recommends correct action *and* suggests another bug that may be related (or makes another connection)     
 
|-
 
|-
−
|'''License''' || This work is licensed under a [http://creativecommons.org/licenses/by-sa/4.0/ Creative Commons Attribution-ShareAlike 4.0 International License]
 
 
|}
 
|}
  
−
=== Suggestions for the Open Source Project: ===
+
=== Comments ===
  
−
Your community should have specific expectations around [support] that are published [somewhere].  For example, if your code will only work on Fedora versions newer than 19,  then specify that.  
+
If you're working with a specific open source project:
 +
* ask the project (via IRC or mailing list) if there are a particular set of bugs that your students should focus on.  For example, in the Gluster project I noted above, if a bug is tagged POST it means that an initial patch for a bug has been put into the Gerrit code review tool.  If there is a very old bug that's marked POST, there's a decent likelihood that the patch has been incorporated into the project and the bug wasn't closed for whatever reason. If your students can verify this and close the bug, that's valuable.
 +
* let the project know that your students will be doing this -- an influx of activity on a bug tracker can be overwhelming if unexpected
  
−
If there are a set of bugs that it would be more helpful to have someone verify, then marking those in some way would help the instructor.
+
Other projects may have similar set-ups that are worth asking about.
  
 +
Also: Make sure your project has enough old bugs for all your student teams :-)
  
 +
'''Good Quiz Questions'''
 +
# Name some reasons that bugs that could have been closed might still be open?
 +
# Diagram the "normal" lifecycle of a bug for the project you're working on. Label the bug stages and who is involved at each stage. 
 +
# Find several bugs from your project:  one that should be closed according to the criteria you're using (that are in the project's Bug Triage Guide, for instance), one that needs more information, and one that has the in correct status. Have students triage them and evaluate their success based on the criteria they site in the assessment as well as the resolution they suggest.
  
−
--------------------
+
'''Variation'''
−
This work is licensed under a  
+
If you're working with a specific open source project '''and''' you have a block of time at least 1.5 hours long, you may want to run a bug sprint. You may be able to find a time when the regular "bug wrangler(s)" from the project can join you (via IRC or Google Hangout) and offer your students "live" assistance with triaging bugs.
−
[http://creativecommons.org/licenses/by-sa/4.0/ Creative Commons Attribution-ShareAlike 4.0 International License]
+
  
−
[[File:CC_license.png]]
+
{{Learning Activity Info
 +
|acm unit=
 +
Software Development Fundamentals (SDF) // Software Engineering (SE)
 +
|acm topic= Development Methods, Verification and Validation (Defect tracking)
 +
|difficulty=
 +
Medium (requires some understanding of the intended functionality of the software, ability to use bug tracking software, and critical thinking skills).
 +
|time=
 +
Not counting background reading, students should be able to find and "groom" a bug during an hour-long class.
 +
|environment=
 +
Student needs access to the project's bug tracker, internet access.  If your students will be making changes, they will also need to join the IRC channel or mailing list where bugs are discussed (the bug triage info should mention this), and if they are going to be making changes, they will need a login to the bug tracker.
 +
|author=
 +
Gina Likins
 +
|source=
 +
[http://blog.smartbear.com/programming/14-ways-to-contribute-to-open-source-without-being-a-programming-genius-or-a-rock-star 14 Ways to Contribute to Open Source without Being a Programming Genius or a Rock Star]
 +
|license=
 +
{{License CC BY SA}}
 +
}}
 +
 
 +
=== Suggestions for the Open Source Project ===
 +
 
 +
Your community should have specific expectations around [support] that are published [somewhere].  For example, if your code will only work on Fedora versions newer than 19,  then specify that.
 +
 
 +
If there are a set of bugs that it would be more helpful to have someone verify, then marking those in some way would help the instructor.
  
−
[[Category: Learning_Activity]]
+
[[Category:Learning Activity]]
−
[[Category: Quality_and_Testing]]
+
[[Category:Communication and Tools]]
 +
[[Category:Bugzilla]]
 +
[[Category:Quality and Testing]]
 +
[[Category:CS2]]
 +
[[Category:Good Draft]]

Latest revision as of 21:19, 3 July 2019


Title

Bug Grooming / Bug Triage / Bug Zapping / Bug Wrangling

Overview

Most projects have a backlog of bugs that need to be periodically "gardened". Sometimes there are even old bugs that may have already been fixed that just haven’t been closed in the system. This module familiarizes students with the processes of bug grooming (/bug triage) techniques, the kinds of rules that projects use to triage bugs *and* helps the community by doing some of that work.

Prerequisites

Students should be familiar with how bug trackers work -- see Intro to Bug Trackers (Activity).

Learning
Objectives 		After successfully completing this activity, the learner should be able to:
  1. Review a bug to see if it still is relevant.
  2. Explain the importance of "bug grooming".
Process Skills
Practiced


Background

Bug Gardening (or Grooming or Triage) is the process by which projects:

  1. confirm new bugs that have been submitted
  2. gather more information about the bug when necessary
  3. prioritize confirmed bugs
  4. review bugs to ensure that they are being worked on
  5. retire old bugs

This activity, however, is focused on retiring bugs that are no longer relevant (#5), primarily because that is more clear-cut than full-fledged bug triage.

Bug Triage Guides explain the rules for how to treat bugs within a project. Here are some examples of Bug Triage Guides:

If your project has a triage guide, read it first. If your project doesn't have a triage guide, I suggest contacting the project to find out if there is an established mailing list or IRC channel for discussing bugs.

Directions

Though all bugs need to be triaged (see examples of Bug Triage Guides), this activity is focused on culling bugs that are no longer relevant, primarily because that is more clear-cut than full-fledged bug triage.

This activity is best performed in groups of two or three students.

You should:

  1. Read the bug triage guide for your project (if available).
  2. Join the IRC channel or mailing list for your project on which bugs are discussed.
  3. Identify a "suspect" bug in the bug tracker,
    1. Identifying the "suspect" bugs will be easier if you have information from the project about their bug lifecycle or which bugs they suggest you focus on. However, even in the absence of that information, there are some other clues that can be used to identify bugs that may no longer exist.
    2. Look at the version number of the software that the bug was reported against (versus the one that's current now). If the bug is for a version that's several releases out of date, there's a good likelihood that the issue may have been fixed (or may have just become irrelevant) in the meantime.
    3. Another possible hint is the date on the bug: bugs that were reported years ago may have been fixed and are probably worth looking at first.
  4. Test the bug to see if it still exists. Once you have identified a bug to work on, try to verify the bug status.
    1. At that point, if the bug is no longer relevant, you can take one of several courses of action, depending on how well-established your relationship with the project is:
      1. If you have established a good working relationship with a project and have permissions to do so, you can close the bug (this is something a more advanced student might do).
      2. You can add a comment to the bug with notes and a suggestion to close. This is more appropriate for students with less project experience.
    2. If, after testing, you cannot tell if a bug is still relevant -- that's ok. At that point you can:
      1. Move on and try another bug, or
      2. Ask for more information from the bug reporter or the person assigned to the bug. This can be a valuable exposure to more of the project ecosystem.

Deliverables

At the end of the exercise, the student teams should have a list of bugs that they investigated and the outcomes. In the ideal case, they will have been able to close a bug or two, but even if they haven't you should still receive a report out. I suggest the following as a sample format (I've filled two lines as examples):


Bug Number Short Description Bug Status URL for Bug Analysis Actions Taken
Bug 302338 Machine Explodes on Disk Insertion Active http://example.com/fakebug.enter_bug.cgi?product=fake&bug=302338 Bug submitted against version 2 of product, which was EOL'd in 2003 Left comment suggesting that bug be closed
Bug 42984 Funny Noises Active http://example.com/fakebug.enter_bug.cgi?product=fake&bug=42984 Bug submitted in 2010, patched in release 2 (2011) Closed bug.

Assessment

Criteria Level 1 (fail) Level 2 (pass) Level 3 (good) Level 4 (exceptional)
Ability to evaluate/triage bugs Unable to correctly evaluate bug status Evaluates bug status correctly Evaluates bug status correctly and recommends correct action Evaluates bug correctly, recommends correct action *and* suggests another bug that may be related (or makes another connection)

Comments

If you're working with a specific open source project:

  • ask the project (via IRC or mailing list) if there are a particular set of bugs that your students should focus on. For example, in the Gluster project I noted above, if a bug is tagged POST it means that an initial patch for a bug has been put into the Gerrit code review tool. If there is a very old bug that's marked POST, there's a decent likelihood that the patch has been incorporated into the project and the bug wasn't closed for whatever reason. If your students can verify this and close the bug, that's valuable.
  • let the project know that your students will be doing this -- an influx of activity on a bug tracker can be overwhelming if unexpected

Other projects may have similar set-ups that are worth asking about.

Also: Make sure your project has enough old bugs for all your student teams :-)

Good Quiz Questions

  1. Name some reasons that bugs that could have been closed might still be open?
  2. Diagram the "normal" lifecycle of a bug for the project you're working on. Label the bug stages and who is involved at each stage.
  3. Find several bugs from your project: one that should be closed according to the criteria you're using (that are in the project's Bug Triage Guide, for instance), one that needs more information, and one that has the in correct status. Have students triage them and evaluate their success based on the criteria they site in the assessment as well as the resolution they suggest.

Variation If you're working with a specific open source project and you have a block of time at least 1.5 hours long, you may want to run a bug sprint. You may be able to find a time when the regular "bug wrangler(s)" from the project can join you (via IRC or Google Hangout) and offer your students "live" assistance with triaging bugs.

ACM BoK
Area & Unit(s)

Software Development Fundamentals (SDF) // Software Engineering (SE)

ACM BoK
Topic(s)

Development Methods, Verification and Validation (Defect tracking)

Difficulty

Medium (requires some understanding of the intended functionality of the software, ability to use bug tracking software, and critical thinking skills).

Estimated Time
to Complete

Not counting background reading, students should be able to find and "groom" a bug during an hour-long class.

Environment /
Materials

Student needs access to the project's bug tracker, internet access. If your students will be making changes, they will also need to join the IRC channel or mailing list where bugs are discussed (the bug triage info should mention this), and if they are going to be making changes, they will need a login to the bug tracker.

Author(s)

Gina Likins

Source

14 Ways to Contribute to Open Source without Being a Programming Genius or a Rock Star

License

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License

CC license.png


Suggestions for the Open Source Project

Your community should have specific expectations around [support] that are published [somewhere]. For example, if your code will only work on Fedora versions newer than 19, then specify that.

If there are a set of bugs that it would be more helpful to have someone verify, then marking those in some way would help the instructor.

Retrieved from "http://foss2serve.org/index.php?title=Bug_Gardening&oldid=13682"
Personal tools
Namespaces
Variants
Actions
Events
Learning Resources
HFOSS Projects
Evaluation
Navigation
Toolbox