What is Mutation Testing?

Some time ago I worked in a project that was deeply concerned about percentage of code coverage. It is, indeed, a good thing, but something to be careful, because, even if you hit 100% of the lines, it still doesn’t mean your tests fully cover all possible mistakes that may happen. That occurs either because the logic can be, sometimes, hard to emulate or because the test was poorly written.

The second one is something that worries me. Tests are handy to give credit to code, in maintenance and in overrall system quality. In other words, tests assert the quality of your code (in the ideal world). But what do asserts the quality of tests?

That’s when mutation tests shine. Their objective is to make sure that, if something changes in your code, the tests will break. That’s incredible useful to show you where your tests are weak, and gives you confidence that all lines being executated in the code are there for a reason.


The full cycle of mutation


How does it happen

Roughly, what a mutation test suite does is: take your code, change something meaninfull (i.e create a mutant) and run the tests. They should break, and, if they don’t, it means that they are weak. Simple as that.

More specifically

The suite will get your code and make some interesting changes, like remove one line, change and operator (like “>” to “<”) or nullify an object. This should be enough to break some test, because, if it doesn’t happen, it means that your tests are hitting that line but not testing it. Putting it simple: if another developer makes that change, your tests should warn him about this.

Keep an eye for mutation test, it might be useful for your team.

Leave a message

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s