In projects facing end customers style matters, often more than less. While in internal apps it doesn’t matter if the UI changes after each release, there might be customers that want their app to follow a very strict style guide to integrate with their corporate identity.
If you ever worked with HTML and pixel perfect CSS you know that things can easily screw up, even with very strict layout guidelines.
Last Friday we brainstormed in our team on how to make this part of our nightly build.
We found a solution which was surprisingly simple..
The idea
When the developer created HTML and CSS for a component (like a login box) and made sure that it was looking fine in all required browsers, the developer would take a screenshot, and crop it to contain only the box. Like depicted here on the right, we would want to make sure that the logo inside the red box looks exactly like that.
Next the automatic build needs to be told on which page this box is expected to be (e.g. a page with a user that is not logged in). The test takes then a screenshot and searches it for the expected box.
We considered also telling at which position we expect it, but that would cause multiple issues: The dev needs to find the pixel position. It needs to be absolute on different screen sizes, changes are hard to track.
When thinking about this, we believe that finding the expected sub-image in the screenshot is sufficient. If you would want to make sure it is at a specific position you just include neighboring objects in your expectation image.
The Implementation
We wrote a very simple keyword that takes a screenshot of a given URL (and possibly after performing steps like putting stuff to a shopping cart) and checks the occurrences of template screenshots. The keyword is implemented in Robot BDD using Selenium.
1*** Keywords *** 2 3Match Styles ${Styles} 4 Execute Javascript window.resizeTo(100,100) 5 ${ID} = Get Unique ID 6 Capture Page Screenshot ${OUTPUT DIR}${/}StyleTest-${ID}.png css= 7 :FOR ${style} IN @{Styles} 8 \ Match Style ${CURDIR}${/}styles${/}${style}.png ${OUTPUT DIR}${/}StyleTest-${ID}.png 9 Remove File ${OUTPUT DIR}${/}StyleTest-${ID}.png
The Javascript helps in bringing down the file size of the screenshot and makes searching faster, as it compresses unused whitespace which occurs in many liquid layouts. Get Unique ID generates us an ID using Java UUID .
The keyword is simple:
1public static void matchStyle(String expectedFilename, String actualFilename) throws IOException { 2 boolean check = StyleChecker.check(expectedFilename, actualFilename); 3 if (!check) { 4 throw new IllegalStateException("could not find expected image in actual screenshot"); 5 } 6}
The actual picture searching is done in Java. The code is not superclever, but after a few rounds of optimization pretty fast (worst case: up to 300ms when expected part is not found, in real use cases averages about 200ms for existing sub-images). In fact the screenshot generation is the bottleneck.
Java code can be downloaded and is excluded here for brevity.
A test can look like this:
1*** Keyword *** 2Behaviour 3 [Arguments] ${Url} @{Styles} 4 5 Go To ${Url} 6 Match Styles @{Styles} 7 8| *Test Case* | | *Url* | *style* | 9| 1 | Behaviour | http://${BASE_URL}${CONTEXT} | Login | ShoppingCart-Empty|
Caveats
- The screenshots need to be lossless.
- Our code is pixel perfect. Which means that every tiny detail is important. Because of that Developers cannot take screenshots of their own to crop, but need to take the screenshot that is produced by selenium. There are differences, for example clear type, or available fonts.
- When implementing picture comparison in Java you can make it horribly wrong. I hope my code is not that bad. At least I benchmarked it and for the simplicity of the algorithm it performs fine.
Benefits
With those simple steps we are able to do regression testing on our user interfaces that have strict standards. It helps even more, because CSS is sometimes a bit fragile. And because of the simplicity of the testsetup it is also very easy to add new test withing seconds.
Debugging
What happens, if a style test fails? More often than not, the analysis is trivial, because style bugs that should be found with these kind of tests (wrong borders and margins, colors don’t match, wrong image, etc.) can be quickly detected by looking at the screenshot. But right now we had the situation, where this was not the case, and immediately suspected a refactoring gone bad. We were wrong! In order to detect differences in two images, you can use GIMP’s layer mode “difference” ;
- Load screenshot (1) in GIMP
- Load image (2), that should be found in the screenshot, in GIMP
- Lay image (2) as new, half-transparent layer over the screenshot
- Apply mode “difference” to the new layer
- Merge the visible layers
- Brighten up the darker image parts, to see the differences.
So far for the short tutorial for debugging. In case you run into troubles, please comment.
Was this post helpful?
More articles
fromFabian Lange
26.3.2015 Your job at codecentric?
Jobs
Agile Developer und Consultant (w/d/m)
Alle Standorte
More articles in this subject area
Discover exciting further topics and let the codecentric world inspire you.
Gemeinsam bessere Projekte umsetzen.
Wir helfen deinem Unternehmen.
Du stehst vor einer großen IT-Herausforderung? Wir sorgen für eine maßgeschneiderte Unterstützung. Informiere dich jetzt.
Hilf uns, noch besser zu werden.
Wir sind immer auf der Suche nach neuen Talenten. Auch für dich ist die passende Stelle dabei.
Blog author
Fabian Lange
Do you still have questions? Just send me a message.
Do you still have questions? Just send me a message.