The vicious circle of bad test code and how to break it

SUMMARY

Compared to the great advances in programming languages and tools, the day-to-day practice of how we actually code is messier than it should be. Especially for long running and complex products building things right is just as important as building the right thing. Improving the quality of your test code is a great start to teach yourself more discipline, which is essential to keep technical debt under control.

If programmers were dentists…

Why do certain professions (pilots, doctors) have strict entry requirements, making it illegal to practice without a license that takes four to seven years of hard and expensive study to obtain? Well, it’s dangerous to fly an airplane or cut into living human flesh without some proof that you know what you’re doing.
I’ve had seven different dentists so far, and while some were better than others none of them did the kind of lasting damage to my mouth that could have resulted in a malpractice suit. I have been at the receiving end of technological advances in the field, which I won’t spell out because I haven’t a clue about dentistry. I can only imagine a lot has changed in a short time, considering that people have been fixing teeth for centuries.
Still it’s nothing compared to the frantic rate of progress in computing. Or is it? In terms of languages and development tools things are definitely better than before. I won’t get sentimental about the time I wrote Perl with Emacs on the LAMP stack. New languages correct old mistaken ideas — checked exceptions anyone? — in a way that Java 9 never can. Great tools and languages however don’t prevent us from making a mess in our daily practice of writing software. In my seventeen years of programming for pay I have met colleges who, had they been dentists:

• would have kept using amalgam fillings, because that was the only thing around when they were in school.
• would not use rubber gloves, or even wash their hands.
• would yank out a wisdom tooth without first taking an x-ray because they trusted their superior experience more. Anyway, what’s the worst that could happen?

Basically, had they been dentists they would have racked up enough nasty work to warrant, if not actual jail time, at least losing their license to ever practice as a software developer again. What license? My point exactly. Now I’m perfectly aware that there are any number of quality standards and certifications in the software industry and in some places they may actually be enforced. While I’m not in favour of a disciplinary board for developers, we could show much more discipline ourselves and stop building software without washing our hands. So what is it about human nature that entices us to code with greasy fingers and regret it afterwards?

Shiny on the outside, messy on the inside

Software is a black box to the user. It can be totally awesome on the outside until some poor developer has to crack it open and change it. Inside it’s a tangled, monolithic, untested abomination. There’s an ‘ility’ it doesn’t satisfy: maintainability.

Developers aim to please. We want to produce business value quickly and we are wary of more costly methods to ensure that we can provide that same value also in the long run, when the product expands. Aiming to please the customer is not the only unconscious incentive to let tidiness slip early on. Maintainability just isn’t much of an issue in the early days of a project, when the code base is still manageable and all developers of the first hour are involved in its inception and still know why a certain design choice was made. Improving maintainability takes time and discipline that doesn’t directly translate into user value. It’s is a crucial investment for the long run, but people are notoriously bad at appreciating the long run. Messy coding standards are like junk food and no exercise: it doesn’t seem to do much harm when you’re young, but the bad habit is hard to shed in your forties. Or so I’ve read…

Craftsmenship, not artistry

You’ve heard it before: programming is an art. I beg to differ, if only because the definition of art itself is a knotty problem. I like to think of programming as a highly creative craft. Calling yourself a software artist smacks too much of a misguided desire to affirm one’s uniqueness. It’s typical of the hacker mentality that sneers at design patterns because it “teaches you a cookie cutter, paint-by-numbers” approach to building software. And they sure don’t want to be tied down to something as bureaucratic as coding standards or test coverage. In an agile team these people get you nowhere. A major software product is a team effort. The artistic stance only breeds egotism. Excessive pride in one’s unique way of doing things gives rise to the Not Invented Here/By Me syndrome. True artists don’t strive for user friendliness, let alone neatness. They develop their own unique style. They don’t copy, they innovate. That’s fine for painting, poetry and jazz, but code of such ilk is likely to be an opaque mess full of ‘clever’ tricks. It has nothing to do with art.
Let me assure you by the way, as an amateur musician and playwright, that the training for any artistic discipline (like playing the cello) takes an excruciating, mind-numbing attention to detail. A professional symphony orchestra has a level of discipline that no software team can rival. There is nothing messy about art.

We need discipline

Agile methods have taught us to pay attention to user value. We want to build the right thing. That’s good, but systems tend towards messiness and the human brain is fixed on short-term benefits, as I explained. So we also need discipline. Yes, and I do mean that in the strictest dictionary sense of the word: the practice of training people to obey rules or a code of behaviour, using punishment to correct disobedience. That punishment need not go further than a group email that Jasper has broken the build (again!) and a moral obligation to provide a selection of treats at the coffee machine. But it should be very clear that neatness is not optional. If we want to build the thing right, it is our responsibility as developers first and foremost to make that happen. Don’t expect business to fully appreciate the importance of clean, maintainable code. They will expect it’s a given.

I believe the biggest benefit of TDD, and automatic software tests in general is the capacity to bring more discipline to a team. Tests don’t just encourage better design: they require it because badly designed systems are almost untestable. Writing tests puts a brake on the messy side of creativity: the lure of cutting corners, of quick-and-dirty, the appeal of banging something out on your keyboard with the satisfying hormonal rush when it somehow, actually works. Test code is a great focus for creating more discipline, because quality of test code is something that even the best teams let slip. I didn’t realise how pernicious it was until I read about the vicious circle of bad test code in Robert ‘Uncle Bob’ Martin’s Clean Code . This particular example, and really all the times we’re not coding cleanly means we sacrifice long-term code stability and programmer peace of mind for the short-term benefit of getting things done.

The vicious circle of bad test code

• Your test code has poor maintainability. Every change to production code requires lots of work, slowing down development efforts and making refactoring painful.
• You @Ignore tests altogether, or decide not to test new additions.
• You test coverage slips
• You don’t do necessary refactoring for fear of breaking anything.
• Congratulations: your production code is now unmaintainable.

So what is good test code? 100% line coverage of course! “Hang on: without the proper assertions that doesn’t mean anything”, as you rightly reply. So you build in mutation coverage with the excellent PIT framework . Say we can bump mutation coverage to 80% (an excellent figure), does that mean our test code is great? Possibly, but not definitely. The tests do a good job of catching regressions: you can’t make behaviour-changing modifications to your software without a failing test, which is good. Yet how much of an effort is it to adapt the when the production code changes? In other words: how maintainable is your test code?

Great test code must be maintainable. A change to production code should require a minimum of changes to the corresponding unit test while keeping the same coverage quality. Some first steps towards maintainable test code:

• apply similar coding standards to your test code with regard to length and complexity.
• avoid lengthy setup code. Make re-usable components for setup code. If setup is costly, the code under tests probably does too much and needs refactoring.
• Keep unit tests pure. Use stubs and mocks as much as possible, and do it always for networked dependencies like services and databases.
• Changes to a single source file that do not affect its public API should require changes only to its corresponding unit test. Of course if the behaviour changes then higher level tests (integration/end to end) should be changed as well. Changes to the API do take extra work, but if your class/interface is used all over the place that’s bad as well.
• and please, please do all this from day one.

The trouble is that you cannot make a test suite great if the production code was poorly designed to begin with, which probably happened because you didn’t take testing seriously. So you can’t really improve test quality without some serious refactoring of the production code. Face it: that’s not going to happen.
Your disillusioned brain is now a likely victim of the infamous broken window effect, also known as the ‘not cleaning up after your dog if the park is already strewn with turds’-effect. People of good will are reluctant to make a mess, but they won’t go to great lengths to make something already messy slightly less messy. Why should they? The serious refactoring required will likely not be contained to a single unit, making it difficult to be a good boy scout. It’s all downhill from there.

Are there cases where you can be less strict with your cleanliness? Can you sometimes forget about the rubber gloves because they slow down your typing? Sure, not all code is written for the long term, but you can’t always tell beforehand and it makes better business sense to assume that it is not throw-away. In some industries it makes good sense to build the right thing, build it quick, accept the technical debt and spend your resulting profits on building the thing right. In all other cases I think gloves are essential.

So sink your professionally maintained teeth into some messy code today. Oh, and if some smart aleck business person asks you how all this clean code stuff translates into business value you can answer: “Maybe it doesn’t, but I happen to take pride in my job”. And if they ask why you let things slip in the first place you can answer with a genuine 80’s pop quote .