Mocks or the real thing? Tips for better unit testing

Recently I had to bone up on some of the new features in Mockito 2 and Powermock , though more out of necessity than from genuine curiosity. Powermock and Mockito 2 let you fake static methods, final classes and even constructor calls, but this has been possible for some time with the JMockit framework, which handles these cases without special syntax or extra boilerplate code. This is not going to be a tutorial about how to exploit the advanced features in modern mocking frameworks. Rather, it’s a warning that these features work around design decisions that were wrong to begin with and that it is the class under test that needs fixing, not the unit test.

The code whose test coverage I was supposed to jack up – notice how wrong that sounds – was a tough nut to crack. The classes under test had many dependencies that were either invoked statically or instantiated through UserDao userDao = new UserDao(new Connection()); when they should have been injected. The test suite only stubbed or mocked handles to networked services. All database handles – how is that not a networked service? – used the real thing. So most of the existing tests were really integration tests posing as unit tests.

Here’s a little refresher. What’s wrong with static methods and constructors calls? The answer it tight coupling. If you explicitly create an instance of, say, an OracleDataSource in your code rather than a javax.sql.DataSource you introduce a compile-time dependency to that implementation. If you let the container (any dependency injection framework will do) create an instance of that datasource and inject the value you can treat it as a generic javax.sql.DataSource interface. Constructing and managing objects is a separate concern from using them, and especially with pooled resources like database connections that have their own lifecycle management you shouldn’t create them yourself. A more practical reason to delegate object construction though is that new(..) makes unit testing awkward.

21 per cent forever

Another refresher: why do we need a mocking framework at all? Why do we want to replace certain objects in our class under test with these weird proxies? It’s because we want to make our unit tests autistic. Any non-trivial class is likely to have dependencies on other classes. Some are part of the same source root, others belong to the core library (java.util.ArrayList, java.io.File) and a considerable bulk are in third-party libraries. These dependencies may be very trustworthy and behave  predictably, but they might also depend on more fickle external resources like the filesystem or network. Any object that uses the curent date/time or does other hardware read-outs is unpredictable by definition. That’s terrible for testing. In a unit test we want to make sure that the world outside the class under test behaves exactly the way we want. Suppose we’re testing an InvoiceBuilder that uses a service to retrieve the VAT rate for a country.

1double vatRate = VATService.getVATRateForCountry(countryCode);

The test case assumes that the VAT rate for the Netherlands is 21 per cent, but we can’t know that unless we peek into the internals of the VATService class. Perhaps it relies on a local file or maybe it accesses a VAT lookup REST server, which slows down the test by orders of magnitude. Unit tests should be fast as lightning and run without elaborate setting up of resources. If you want to reap the benefits of mutation testing then speed of execution is an essential. There is also a more principled reason. Unit tests are not there to make sure that your application as a whole works as intended. For that purpose you have your integration tests, your end-to-end tests, stress tests, and your stressed human testers. Unit tests are deliberately designed to be short-sighted. When a unit test runs a piece of code it must treat anything external to that code as a given, i.e. be oblivious to its internals. The test wants the Dutch VAT rate to be written in stone, long after the North Sea has reclaimed the Low Countries. For that we want to mock. Behold the mother of non-deterministic code:

1LocalDateTime now = LocalDateTime.now();
2File file = new File(PATH + formatDate(now));
3FileUtils.writeLines(file, 
4     Arrays.asList("This file was created on:", formatDate(now)));

No exception was thrown, so we know that some file was written, but we don’t know its name or contents, because that was based on the current date. I want to freeze time on 14 July 2017 11:45, so I can open a file named TEST_14_7_2017_11_15. Actually, forget I said that. I don’t want to open any file. I trust  that Apache FileUtils is a stable library. I only want to make sure that its writeToFile method is invoked with a File whose path and timestamp are correct. Powermock can do it, but it’s not as straightforward as mocking simple injectable instances:

• You have to explicitly specify the classes in a @PrepareForTest(MockableDependency.class) annotation, including the class containing the class you want to construct.
• You have to invoke PowerMockito.mockStatic(MockableDependency.class)
• The syntax for verification is different and constructors are cumbersome: whenNew(SomeClass.class).withArguments(..).thenReturn(..)

This somewhat contrived example shows you when you absolutely need a mocking framework to make things testable. Later I’ll show you there’s a simpler and better way than mocking statics and constructors.

To mock or not to mock?

Suppose our VATService is backed by a hard-coded enumeration. What if after an update by the VAT team the service suddenly returns a percentage of 20 rather than a fraction (0.2). What if our FileUtils example suddently has no write privileges? Had you used the real thing then your unit test would have stumbled over the illegal state. Now that it’s mocked it will go sour in production! Assuming there’s no test setup required and performance penalty in using the VAT service for real, would it not be better to use the real thing rather than a mock? NO IT WOULD NOT! A unit test is not responsible for anything but the class under test. No amount of testing on your laptop can prevent things from crashing under a production load. Maybe the VAT service is a memory hog. It’s still beyond the scope of the unit test. To prevent disasters we have integration and end-to-end tests. They’re no less important than unit tests, but of a different order. If the contract – the output given certain input — of your class changes, so should the mocked verifications. Imagine a data access object (DAO) that used to return null for an invalid key but now throws a ThingyNotFoundException.

Obligatory candidates for mocking are anything network related, database connections or anything to do with the file system. A special case is code that produces unpredictable results, like the current time. You should only ever use the real dependency if its behaviour is completely deterministic, there’s no setup involved and no performance hit. A good example is the standard collection library, or utilities such as Apache StringUtils. We can assume they work as designed. Anyway, if you feel you need to test the correctness of an external library, should you really use it in production? Note that the choice to mock FileUtils and use the real thing for StringUtils has nothing to do with the quality of the library: it’s the interaction with the file system that we want to keep out of the unit test.

Wrap it up

So static invocations and constructors can be a sign of bad design practice. While they are ‘mockable’, Powermock and JMockit work best with a dependency injection approach. The bytecode manipulation voodoo involved to make statics work are not without problems, especially with older or non-standard JDKs. It’s like most prescription medicine: a sub-optimal means with nauseating side effects to cure a disease often caused by bad habits. Powermock does things right, but it’s a cholesterol pill. We are not doing the right thing if we have to use it often. But sometimes we have no choice. FileUtils and other similar stateless libraries (e.g. LocalDate.now()) are built around static methods. What to do?
The Facade pattern to the rescue! Write a simple wrapper around it that exposes only the calls you want and inject instances of that. The advantages:

• it hides the implementation and doesn’t tie you in to Apache FileUtils.
• It hides what you don’t need and you can customise it to reduce error and make it more user-friendly.
• You can mock it much more easily.

Now that you have isolated all usages of FileUtils in very lean Facade, you can choose to unit test it using static mocks or forgo the unit test entirely and integration-test it, using real files. How about constructors? It depends on the kind of object you’re making. If it is an instance member containing logic and behaviour such as the VATService you should let the framework inject it. If it’s a typical Employee data object that gets ‘newed’ all over the place then it probably belongs in a dedicated factory, injected as a mockable dependency.

Train wrecks

One final warning to dampen your spirits should I have given you the wrong impression that mocks are always fun to work with. Here’s a little made-up yet painfully reminiscent example:

1String postCode = employeeDao
2     .getEmployeeById(employeeId)
3     .getPreviousEmployer()
4     .getAddress()
5     .getPostCode();

Forget about the absence of null-safety (if only everybody used Kotlin), this is just typical train wreck notation. It’s a bear to test with or without mocks.

1when(employeeDao.getEmployeeById(42)).thenReturn(employee);
2when(employee.getPreviousEmployer()).thenReturn(previousEmployer);
3when(previousEmployer.getAddress()).thenReturn(address);
4when(address.getPostCode()).thenReturn(“1234AB”);By Jove, we’re finally there!

Sniffing out code smells

Unit tests are great at sniffing out code smells. Code that is hard or impossible to unit test is always cause for concern. If you struggle with the mocking framework or can’t do without a monstrous base class or helpers to get the unit tests going, then it’s the class under test that has sprouted the evil green hairs and it is time for refactoring. Better late than never. Stick to a test-driven approach and this is much less likely to happen.