Regression Test Assignment
Regression Test Assignment
Participants write unit tests (in GoogleTest) to establish reliable regression tests for a C++ class
Students need to have installed and programmed unit tests with GoogleTest. They should also know how to run a coverage analyzer and interpret its results. Intermediate knowledge of Git version control is required.
Students should write unit tests and use a coverage analysis tool to identify test cases they may be missing
Your job is to test the GoalUndo class thoroughly so that developers can verify that it is working correctly and have regression tests for when the code is changed during maintenance. That means that your job is not only to verify that the current solution code is working, but that any implementation of the existing functions work as described in the documentation.
- Fork the GoalUndo repo into your own GitHub space
- Clone your fork onto your own computer
- Set the upstream remote in case the instructor needs to give you changes to the code during the exam
- Working within a new branch, add unit tests that thoroughly test the functionality as described in the documentation
- Use your judgment -- based on what you learned about testing thoroughly -- to decide when you have written sufficient tests to be confident that any correct implementation will pass your tests and that any incorrect implementation will fail.
- Once you are done testing, push your changes into your fork (keeping it in your branch)
- Submit the url and branch name of the branch on your fork with your tests
- Make sure you have only pushed changes to the
GoalUndoTest.cppfile. No other files should be changed on your GitHub fork. Any commits to your fork after the deadline will be considered late.
Make your own local clone of the GoalUndo repository and change the code so that it will work as documented in some cases but not in others (such as changing conditional operators or types of control structures). To grade each student, fetch the student's fork and then merge with the changes you made to inject intentional errors into the code. Run the student's tests and confirm that tests fail where they should be testing the functionality broken by the errors you introduced.
A fork of the GoalUndo repository, with added tests
10% did you follow the instructions accurately? 20% did you demonstrate recommended practices with git and GitHub? 20% were unit tests written and formatted following recommended practices? 30% how thoroughly did you test the existing code? 20% do your tests verify that each function acts as documented as regression tests?
|Criteria||Level 1 (poor)||Level 2 (fair)||Level 3 (excellent)|
|Instructions (10%)||Instructions to fork the repo, only commit the test file, and submit the url were not followed||Some instructions were not followed precisely, such as committing unnecessary files||Instructions were followed accurately and precisely||Version Control (20%)||Several mistakes shown in demonstrating ability to fork, commit progress, and pushing to remote (forked) repository||Minor mistakes shown in demonstrating ability to fork, commit progress, and pushing to remote (forked) repository, such as only committing once or forgetting to push local changes to the remote||Demonstrated ability to fork, commit progress, and pushing to remote (forked) repository||Unit Tests (20%)||Tests not adequately set up nor separated into individual test functions that each check one specific case||Unit tests generally follow the correct format but do not follow all conventions such as poorly-named functions or a test function really checking multiple cases||Unit tests are organized in a collection of test functions which each test specific cases and each test has appropriate input, output, and assertions||Test Thoroughness (30%)||Multiple lines of code and/or branches are not exercised by the tests||The tests cover most of the code according to gcov, but missed possible branches||The tests cover all possible branches as reported by gcov's "branch coverage"||Regression Test Effectiveness (20%)||Changing the code with new (or fixed) bugs do not change the result of the tests||Some corrected bugs result in passed tests and injecting new bugs cause new failed tests but the tests do not correctly report all changes||Corrected bugs result in passed tests and injecting new bugs cause new failed tests with appropriate names that identify the failing feature|
|Knowledge Area/Knowledge Unit||
Software Engineering / SE Software Verification Validation
Unit Testing, Test Coverage
|Level of Difficulty||
Writing thorough unit tests will take intermediate knowledge of object oriented programming and predicting possible future bugs will take advanced reflective cognition
|Estimated Time to Completion||
The assignment should take from 1-3 hours of work and is recommended as an equivalent to a relatively small programming assignment.
Need to have a working C++ development environment with GoogleTest and gcov installed
Licensed CC BY-SA 4.0
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License