Regression Test Assignment

Title	Regression Test Assignment
Overview	Participants write unit tests (in GoogleTest) to establish reliable regression tests for a C++ class
Prerequisite Knowledge	Students need to have installed and programmed unit tests with GoogleTest. They should also know how to run a coverage analyzer and interpret its results. Intermediate knowledge of Git version control is required.
Learning Objectives	Students should write unit tests and use a coverage analysis tool to identify test cases they may be missing

Directions:

Students:

Overview

Your job is to test the GoalUndo class thoroughly so that developers can verify that it is working correctly and have regression tests for when the code is changed during maintenance. That means that your job is not only to verify that the current solution code is working, but that any implementation of the existing functions work as described in the documentation.

Instructions

• Fork the GoalUndo repo into your own GitHub space
• Clone your fork onto your own computer
• Set the upstream remote in case the instructor needs to give you changes to the code during the exam
• Working within a new branch, add unit tests that thoroughly test the functionality as described in the documentation
• Use your judgment -- based on what you learned about testing thoroughly -- to decide when you have written sufficient tests to be confident that any correct implementation will pass your tests and that any incorrect implementation will fail.
• Once you are done testing, push your changes into your fork (keeping it in your branch)
• Submit the url and branch name of the branch on your fork with your tests
• Make sure you have only pushed changes to the GoalUndoTest.cpp file. No other files should be changed on your GitHub fork. Any commits to your fork after the deadline will be considered late.

Instructors:

Make your own local clone of the GoalUndo repository and change the code so that it will work as documented in some cases but not in others (such as changing conditional operators or types of control structures). To grade each student, fetch the student's fork and then merge with the changes you made to inject intentional errors into the code. Run the student's tests and confirm that tests fail where they should be testing the functionality broken by the errors you introduced.

Deliverables:

A fork of the GoalUndo repository, with added tests

Assessment:

Grading Rubric

10% did you follow the instructions accurately?
20% did you demonstrate recommended practices with git and GitHub?
20% were unit tests written and formatted following recommended practices?
30% how thoroughly did you test the existing code?
20% do your tests verify that each function acts as documented as regression tests?

Criteria	Level 1 (poor)	Level 2 (fair)	Level 3 (excellent)
Instructions (10%)	Instructions to fork the repo, only commit the test file, and submit the url were not followed	Some instructions were not followed precisely, such as committing unnecessary files	Instructions were followed accurately and precisely
Version Control (20%)	Several mistakes shown in demonstrating ability to fork, commit progress, and pushing to remote (forked) repository	Minor mistakes shown in demonstrating ability to fork, commit progress, and pushing to remote (forked) repository, such as only committing once or forgetting to push local changes to the remote	Demonstrated ability to fork, commit progress, and pushing to remote (forked) repository
Unit Tests (20%)	Tests not adequately set up nor separated into individual test functions that each check one specific case	Unit tests generally follow the correct format but do not follow all conventions such as poorly-named functions or a test function really checking multiple cases	Unit tests are organized in a collection of test functions which each test specific cases and each test has appropriate input, output, and assertions
Test Thoroughness (30%)	Multiple lines of code and/or branches are not exercised by the tests	The tests cover most of the code according to gcov, but missed possible branches	The tests cover all possible branches as reported by gcov's "branch coverage"
Regression Test Effectiveness (20%)	Changing the code with new (or fixed) bugs do not change the result of the tests	Some corrected bugs result in passed tests and injecting new bugs cause new failed tests but the tests do not correctly report all changes	Corrected bugs result in passed tests and injecting new bugs cause new failed tests with appropriate names that identify the failing feature

Additional Information:

Knowledge Area/Knowledge Unit	Software Engineering / SE Software Verification Validation
Topic	Unit Testing, Test Coverage
Level of Difficulty	Writing thorough unit tests will take intermediate knowledge of object oriented programming and predicting possible future bugs will take advanced reflective cognition
Estimated Time to Completion	The assignment should take from 1-3 hours of work and is recommended as an equivalent to a relatively small programming assignment.
Materials/Environment	Need to have a working C++ development environment with GoogleTest and gcov installed
Author	Kevin Buffardi
Source	Kevin Buffardi
License	Licensed CC BY-SA 4.0

---

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License