Mutation Testing

Nowadays everyone agrees that you should write tests and have good code coverage… You’re already doing that and still feel that things could be improved. Are you really sure that the tests are good? Unit tests guard the source code, but who guards the guardians?! Sure, code coverage helps you up to a point and you can go even further with mutation testing!

In this session we’ll show and discuss the difference to code coverage and see concrete examples. Here’s a taste of the session:

// production code
if (!a && !b) {
    doThis();
} else {
    doThat();
}

Your tests calls this code and does some asserts. Then you refactor the code (like any good developer using TDD does) and run the tests again.

If you did a good job and wrote all the tests then this code should fail since the De Morgan’s laws weren’t respected:

// production code
if (a && b) { // deliberate mistake: should have || instead of &&
    doThat(); // then and else branches reversed to improve logic
} else {
    doThis();
}

Using mutation testing you can find all the missing unit tests and write them before touching the production code again. This technique is especially useful when working on legacy code, but can also be used when working on new code to discover new test cases.

Join this session and learn more on how to improve your tests and increase your confidence while refactoring from legacy code to the bright code of the future that will make your maintainers happy!

Audience Learnings

Improving the reliability and confidence in your automated tests.

Structure

• Introduction – who I am and what I do?
• What is mutation testing?
• How to do it manually?
• How to do it automatically?
• Advantages vs Disadvantages
• Questions

Prerequisites

Basic knowledge of unit testing and code coverage. And maybe the Java programming language.