Regression Test Assignment

From Foss2Serve
(Difference between revisions)
Jump to: navigation, search
m (added knowledge unit)
 
(4 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 
__NOTOC__
 
__NOTOC__
−
{| border="1"
+
 
−
|-
+
{{Learning Activity Overview
−
|'''Title''' || 
+
|title=
 
Regression Test Assignment
 
Regression Test Assignment
−
|-   
+
|overview=
−
|'''Overview''' || 
+
Participants write unit tests (in [[:Category:Google Test|Google Test]]) to establish reliable regression tests for a C++ class
−
Participants write unit tests (in GoogleTest) to establish reliable regression tests for a C++ class
+
|prerequisites=
−
|-
+
Students should:
−
|'''Prerequisite Knowledge''' ||
+
* Have installed and programmed unit tests with [[:Category:Google Test|Google Test]].  
−
Students need to have installed and programmed unit tests with GoogleTest. They should also know how to run a coverage analyzer and interpret its results. Intermediate knowledge of Git version control is required.
+
* Know how to run a coverage analyzer and interpret its results.  
−
|-
+
* Have intermediate knowledge of Git version control.
−
|'''Learning Objectives''' ||
+
|objectives=
−
Students should write unit tests and use a coverage analysis tool to identify test cases they may be missing
+
* Write unit tests.
−
|}
+
* Use a coverage analysis tool to identify test cases they may be missing.
 +
|process skills=
 +
}}
  
−
=== Directions: ===
+
=== Directions ===
  
−
==== Students: ====
+
==== Students ====
  
 
'''Overview'''
 
'''Overview'''
Line 25: Line 27:
 
'''Instructions'''
 
'''Instructions'''
  
−
*Fork the [https://github.com/ChicoState/GoalUndo GoalUndo repo] into your own GitHub space
+
# Fork the [https://github.com/ChicoState/GoalUndo GoalUndo repo] into your own GitHub space
−
*Clone your fork onto your own computer
+
# Clone your fork onto your own computer
−
*Set the upstream remote in case the instructor needs to give you changes to the code during the exam
+
# Set the upstream remote in case the instructor needs to give you changes to the code during the exam
−
*Working within a new branch, add unit tests that thoroughly test the functionality as described in the documentation
+
# Working within a new branch, add unit tests that thoroughly test the functionality as described in the documentation
−
*Use your judgment -- based on what you learned about testing thoroughly -- to decide when you have written sufficient tests to be confident that '''any correct implementation will pass your tests''' and that '''any incorrect implementation will fail.'''
+
# Use your judgment -- based on what you learned about testing thoroughly -- to decide when you have written sufficient tests to be confident that '''any correct implementation will pass your tests''' and that '''any incorrect implementation will fail.'''
−
*Once you are done testing, push your changes into your fork (keeping it in your branch)
+
# Once you are done testing, push your changes into your fork (keeping it in your branch)
−
*Submit the url and branch name of the branch on your fork with your tests
+
# Submit the url and branch name of the branch on your fork with your tests
−
*Make sure you have only pushed changes to the <code>GoalUndoTest.cpp</code> file. ''No other files'' should be changed on your GitHub fork. Any commits to your fork after the deadline will be considered late.
+
# Make sure you have only pushed changes to the <code>GoalUndoTest.cpp</code> file. ''No other files'' should be changed on your GitHub fork. Any commits to your fork after the deadline will be considered late.
  
−
==== Instructors: ====
+
==== Instructors ====
  
 
Make your own local clone of the GoalUndo repository and change the code so that it will work as documented in some cases but not in others (such as changing conditional operators or types of control structures). To grade each student, fetch the student's fork and then merge with the changes you made to inject intentional errors into the code. Run the student's tests and confirm that tests fail where they should be testing the functionality broken by the errors you introduced.
 
Make your own local clone of the GoalUndo repository and change the code so that it will work as documented in some cases but not in others (such as changing conditional operators or types of control structures). To grade each student, fetch the student's fork and then merge with the changes you made to inject intentional errors into the code. Run the student's tests and confirm that tests fail where they should be testing the functionality broken by the errors you introduced.
  
−
=== Deliverables: ===
+
=== Deliverables ===
  
 
A fork of the GoalUndo repository, with added tests
 
A fork of the GoalUndo repository, with added tests
  
−
=== Assessment: ===
+
=== Assessment ===
  
 
Grading Rubric
 
Grading Rubric
Line 52: Line 54:
 
  20% do your tests verify that each function acts as documented as regression tests?
 
  20% do your tests verify that each function acts as documented as regression tests?
  
−
=== Additional Information: ===
+
{| border="1" class="wikitable"
−
{| border="1"
+
! Criteria
−
|-
+
! Level 1 (poor)
−
|'''Knowledge Area/Knowledge Unit''' || 
+
! Level 2 (fair)
−
Software Engineering / SE Software Verification Validation
+
! Level 3 (excellent)
 
|-
 
|-
−
|'''Topic''' || 
+
| '''Instructions (10%)'''
−
Unit Testing, Test Coverage
+
| Instructions to fork the repo, only commit the test file, and submit the url were not followed
 +
| Some instructions were not followed precisely, such as committing unnecessary files
 +
| Instructions were followed accurately and precisely
 +
 
 
|-
 
|-
−
|'''Level of Difficulty''' ||
+
| '''Version Control (20%)'''
−
Writing thorough unit tests will take intermediate knowledge of object oriented programming and predicting possible future bugs will take advanced reflective cognition
+
| Several mistakes shown in demonstrating ability to fork, commit progress, and pushing to remote (forked) repository
 +
| Minor mistakes shown in demonstrating ability to fork, commit progress, and pushing to remote (forked) repository, such as only committing once or forgetting to push local changes to the remote
 +
| Demonstrated ability to fork, commit progress, and pushing to remote (forked) repository
 +
 
 
|-
 
|-
−
|'''Estimated Time to Completion''' ||
+
| '''Unit Tests (20%)'''
−
The assignment should take from 1-3 hours of work and is recommended as an equivalent to a relatively small programming assignment.
+
| Tests not adequately set up nor separated into individual test functions that each check one specific case
 +
| Unit tests generally follow the correct format but do not follow all conventions such as poorly-named functions or a test function really checking multiple cases
 +
| Unit tests are organized in a collection of test functions which each test specific cases and each test has appropriate input, output, and assertions
 +
 
 
|-
 
|-
−
|'''Materials/Environment''' ||
+
| '''Test Thoroughness (30%)'''
−
Need to have a working C++ development environment with GoogleTest and gcov installed
+
| Multiple lines of code and/or branches are not exercised by the tests
 +
| The tests cover most of the code according to gcov, but missed possible branches
 +
| The tests cover all possible branches as reported by gcov's "branch coverage"
 +
 
 
|-
 
|-
−
|'''Author''' || 
+
| '''Regression Test Effectiveness (20%)'''
−
Kevin Buffardi
+
| Changing the code with new (or fixed) bugs do not change the result of the tests
−
|-
+
| Some corrected bugs result in passed tests and injecting new bugs cause new failed tests but the tests do not correctly report all changes
−
|'''Source''' ||
+
| Corrected bugs result in passed tests and injecting new bugs cause new failed tests with appropriate names that identify the failing feature
−
Kevin Buffardi
+
 
−
|-
+
−
|'''License''' ||
+
−
Licensed CC BY-SA 4.0
+
 
|}
 
|}
  
 +
=== Additional Information ===
  
−
--------------------
+
{{Learning Activity Info
−
This work is licensed under a  
+
|acm unit=
−
[http://creativecommons.org/licenses/by-sa/4.0/ Creative Commons Attribution-ShareAlike 4.0 International License]
+
Software Engineering / SE Software Verification Validation
−
 
+
|acm topic=
−
[[File:CC_license.png]]
+
Unit Testing, Test Coverage
 +
|difficulty=
 +
Writing thorough unit tests will take intermediate knowledge of object oriented programming and predicting possible future bugs will take advanced reflective cognition
 +
|time=
 +
The assignment should take from 1-3 hours of work and is recommended as an equivalent to a relatively small programming assignment.
 +
|environment=
 +
Need to have a working C++ development environment with [[:Category:Google Test|Google Test]] and gcov installed
 +
|author=
 +
Kevin Buffardi
 +
|source=
 +
Kevin Buffardi
 +
|license=
 +
{{License CC BY SA}}
 +
}}
  
−
[[Category: Learning_Activity]]
+
[[Category:Learning Activity]]
−
[[Category: Quality_and_Testing]]
+
[[Category:Quality and Testing]]
 +
[[Category:Google Test]]
 +
[[Category:Good Draft]]

Latest revision as of 13:37, 8 September 2018


Title

Regression Test Assignment

Overview

Participants write unit tests (in Google Test) to establish reliable regression tests for a C++ class

Prerequisites

Students should:

  • Have installed and programmed unit tests with Google Test.
  • Know how to run a coverage analyzer and interpret its results.
  • Have intermediate knowledge of Git version control.
Learning
Objectives 		After successfully completing this activity, the learner should be able to:
  • Write unit tests.
  • Use a coverage analysis tool to identify test cases they may be missing.
Process Skills
Practiced


Directions

Students

Overview

Your job is to test the GoalUndo class thoroughly so that developers can verify that it is working correctly and have regression tests for when the code is changed during maintenance. That means that your job is not only to verify that the current solution code is working, but that any implementation of the existing functions work as described in the documentation.

Instructions

  1. Fork the GoalUndo repo into your own GitHub space
  2. Clone your fork onto your own computer
  3. Set the upstream remote in case the instructor needs to give you changes to the code during the exam
  4. Working within a new branch, add unit tests that thoroughly test the functionality as described in the documentation
  5. Use your judgment -- based on what you learned about testing thoroughly -- to decide when you have written sufficient tests to be confident that any correct implementation will pass your tests and that any incorrect implementation will fail.
  6. Once you are done testing, push your changes into your fork (keeping it in your branch)
  7. Submit the url and branch name of the branch on your fork with your tests
  8. Make sure you have only pushed changes to the GoalUndoTest.cpp file. No other files should be changed on your GitHub fork. Any commits to your fork after the deadline will be considered late.

Instructors

Make your own local clone of the GoalUndo repository and change the code so that it will work as documented in some cases but not in others (such as changing conditional operators or types of control structures). To grade each student, fetch the student's fork and then merge with the changes you made to inject intentional errors into the code. Run the student's tests and confirm that tests fail where they should be testing the functionality broken by the errors you introduced.

Deliverables

A fork of the GoalUndo repository, with added tests

Assessment

Grading Rubric 

10% did you follow the instructions accurately?
20% did you demonstrate recommended practices with git and GitHub?
20% were unit tests written and formatted following recommended practices?
30% how thoroughly did you test the existing code?
20% do your tests verify that each function acts as documented as regression tests?
Criteria Level 1 (poor) Level 2 (fair) Level 3 (excellent)
Instructions (10%) Instructions to fork the repo, only commit the test file, and submit the url were not followed Some instructions were not followed precisely, such as committing unnecessary files Instructions were followed accurately and precisely
Version Control (20%) Several mistakes shown in demonstrating ability to fork, commit progress, and pushing to remote (forked) repository Minor mistakes shown in demonstrating ability to fork, commit progress, and pushing to remote (forked) repository, such as only committing once or forgetting to push local changes to the remote Demonstrated ability to fork, commit progress, and pushing to remote (forked) repository
Unit Tests (20%) Tests not adequately set up nor separated into individual test functions that each check one specific case Unit tests generally follow the correct format but do not follow all conventions such as poorly-named functions or a test function really checking multiple cases Unit tests are organized in a collection of test functions which each test specific cases and each test has appropriate input, output, and assertions
Test Thoroughness (30%) Multiple lines of code and/or branches are not exercised by the tests The tests cover most of the code according to gcov, but missed possible branches The tests cover all possible branches as reported by gcov's "branch coverage"
Regression Test Effectiveness (20%) Changing the code with new (or fixed) bugs do not change the result of the tests Some corrected bugs result in passed tests and injecting new bugs cause new failed tests but the tests do not correctly report all changes Corrected bugs result in passed tests and injecting new bugs cause new failed tests with appropriate names that identify the failing feature

Additional Information

ACM BoK
Area & Unit(s)

Software Engineering / SE Software Verification Validation

ACM BoK
Topic(s)

Unit Testing, Test Coverage

Difficulty

Writing thorough unit tests will take intermediate knowledge of object oriented programming and predicting possible future bugs will take advanced reflective cognition

Estimated Time
to Complete

The assignment should take from 1-3 hours of work and is recommended as an equivalent to a relatively small programming assignment.

Environment /
Materials

Need to have a working C++ development environment with Google Test and gcov installed

Author(s)

Kevin Buffardi

Source

Kevin Buffardi

License

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License

CC license.png

Retrieved from "http://foss2serve.org/index.php?title=Regression_Test_Assignment&oldid=12711"
Personal tools
Namespaces
Variants
Actions
Events
Learning Resources
HFOSS Projects
Evaluation
Navigation
Toolbox