Introduction
Nowadays Agile Teams can choose from a wide variety of tools for automating Acceptence Tests. Thereby a lot of quite different concepts show up. FitNesse for example is using an integrated Wiki to organise testcases, while the Robot Framework is using keyword-driven test development. These are examples of two well-established tools in this area. These tools have a rich scope of operation, but therefore they also require a longer period of vocational adjustment.
In comparison JBehave seems to be very lightweight on first sight (and also on second sight). There is the description of the testcases in BDD-format on one side and the implementation of the required test functionality in Java on the other side. Both parts can be easily connected using Maven .
As this article is a bit longer a short overview on the topics that follow is given here:
- Maven Configuration: Explains the required configuration for Maven to implement and execute tests using JBehave.
- Testcases: Shows how testcases are written in JBehave using the BDD-format.
- Implementation: Explains the required steps to implement the test functionality in Java that is matching the description of certain testcases.
- Reporting: Showing the results of a test execution with JBehave.
- Putting the pieces together: Maven, description of testcases and implementation of test functionality are glued together using certain naming and directory conventions.
- Conclusion: How suitable is JBehave for automating Acceptance Tests? And for whom?
- Download: Download the complete example as a ZIP-file.
Ship’s log supplement as of 16.06.2012: My collegue Andreas has written a very good article on the various configuration possibilities of JBehave. . This one could be surely of great help after starting with JBehave :-).
Maven Configuration
For projects that are developed using Java, JBehave has definitely the big advantage that the integration to the existing development environment of the team works very smoothly. This is especially the case if Maven is already used as the build tool. Of course this is working for other testing tools as well. But especially if these tools are not initially developed in and for Java this often gets more complicated than desired.
The following Maven configuration is sufficient to start with the development of Acceptance Tests with JBehave. Tests can be compiled, started and even artifacts required for the reporting can be downloaded with this:
1<?xml version="1.0" encoding="UTF-8"?> 2 3<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 4 xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"> 5 6 <url>http://maven.apache.org</url> 7 <modelVersion>4.0.0</modelVersion> 8 <name>JBehaveDemo</name> 9 <groupId>de.codecentric</groupId> 10 <artifactId>jbehave-demo</artifactId> 11 <packaging>jar</packaging> 12 <version>1.0-SNAPSHOT</version> 13 14 <dependencies> 15 <dependency> 16 <groupId>org.jbehave</groupId> 17 <artifactId>jbehave-core</artifactId> 18 <version>3.1.2</version> 19 </dependency> 20 <dependency> 21 <groupId>junit</groupId> 22 <artifactId>junit</artifactId> 23 <version>4.4</version> 24 </dependency> 25 </dependencies> 26 27 <build> 28 <plugins> 29 <plugin> 30 <groupId>org.jbehave</groupId> 31 <artifactId>jbehave-maven-plugin</artifactId> 32 <version>3.1.2</version> 33 <executions> 34 <execution> 35 <id>run-stories-as-embeddables</id> 36 <phase>integration-test</phase> 37 <configuration> 38 <includes> 39 <include>**/*Scenarios.java</include> 40 </includes> 41 <ignoreFailureInStories>true</ignoreFailureInStories> 42 <ignoreFailureInView>false</ignoreFailureInView> 43 </configuration> 44 <goals> 45 <goal>run-stories-as-embeddables</goal> 46 </goals> 47 </execution> 48 </executions> 49 </plugin> 50 51 <plugin> 52 <groupId>org.apache.maven.plugins</groupId> 53 <artifactId>maven-dependency-plugin</artifactId> 54 <executions> 55 <execution> 56 <id>unpack-jbehave-site-resources</id> 57 <phase>generate-resources</phase> 58 <goals> 59 <goal>unpack</goal> 60 </goals> 61 <configuration> 62 <overwriteReleases>false</overwriteReleases> 63 <overwriteSnapshots>true</overwriteSnapshots> 64 <artifactItems> 65 <artifactItem> 66 <groupId>org.jbehave.site</groupId> 67 <artifactId>jbehave-site-resources</artifactId> 68 <version>3.1.1</version> 69 <type>zip</type> 70 <outputDirectory> ${project.build.directory}/jbehave/view</outputDirectory> 71 </artifactItem> 72 </artifactItems> 73 </configuration> 74 </execution> 75 <execution> 76 <id>unpack-jbehave-reports-resources</id> 77 <phase>generate-resources</phase> 78 <goals> 79 <goal>unpack</goal> 80 </goals> 81 <configuration> 82 <overwriteReleases>false</overwriteReleases> 83 <overwriteSnapshots>true</overwriteSnapshots> 84 <artifactItems> 85 <artifactItem> 86 <groupId>org.jbehave</groupId> 87 <artifactId>jbehave-core</artifactId> 88 <version>3.1.2</version> 89 <outputDirectory>${project.build.directory}/jbehave/view</outputDirectory> 90 <includes>**\/*.css,**\/*.ftl,**\/*.js</includes> 91 </artifactItem> 92 </artifactItems> 93 </configuration> 94 </execution> 95 </executions> 96 </plugin> 97 </plugins> 98 </build> 99</project>
As said with this configuration all required Jar files are downloaded automatically and the tests can be started using the command: “mvn integration-test”. Of course we need some testcases and implementation of test functionality for this to work.
Testcases
JBehave supports the so-called BDD-format (Given/When/Then) for writing testcases. This format has the following syntax:
| Given: | a static precondition | |
| And: | another static precondition | |
| And: | … | |
| When: | tested behaviour | |
| And: | more tested behaviour | |
| And: | … | |
| Then: | the result of the tested behaviour under the given preconditions | |
| And: | … |
This format has the big advantage that it can be read quite easily also from non-technical people. This enables the team to establish a common understanding between the development team and members of the operating departments and/or customers.
Writing automated Acceptance Tests is a task that – in an ideal case – is always performed in close cooperation between the experts of the operating departments and the agile development team. This ensures that really all required compentences are available and that first results from the test execution can be analysed quickly together. A description of the testcases that is easily understood by all involved parties is of course a mandatory precondition for this to work.
In JBehave the testcases are written in co-called story-files. Thereby it is a meaningful convention that in one of these story-files all tests for one user story are described. It is already taken care of when writing user stories that those do not become too big. This ensures that the amount of testcases in these story-files are as well kept at a reasonable level. In addition this is supported by the clear structure of the story-files. Testcases are by the way called “scenarios” in JBehave.
To keep this example as simple as possible the tests are written for the existing stack implementation of Java. This way the use case is very simple and the focus can be fully put on the concepts and correlations of JBehave.
1Narrative: 2In order to develop an application that requires a stack efficiently 3As a development team 4I would like to use an interface and implementation in Java directly 5 6 7Scenario: Basic functionality of a Stack 8 9Given an empty stack 10When the string Java is added 11And the string C++ is added 12And the last element is removed again 13Then the resulting element should be Java 14 15Scenario: Stack search 16 17Given an empty stack 18When the string Java is added 19And the string C++ is added 20And the string PHP is added 21And the element Java is searched for 22Then the position returned should be 3
At the beginning of a story-file it is possible to explain the tested user story once again. This is happening after the keyword Narrative:. Of course it makes sense to repeat here – a hopefully already existing description – of the user story. But as this text is anyway completely ignored with respect to the testing functionality, any text can be put here in the end.
Especially for tests written in the BDD-format it makes much sense to write them in your native language. In the example above the tests are written in English language as this is supported out-of-the-box by JBehave. But it is possible to translate the used keywords (Given/When/Then) into other languages as well doing some additional coding in Java.
After this any amount of testcase (=Scenarios) can be written. Those are always started with the keyword Scenario: followed by a brief description of this testcase. Then the actual definition of the testcase is given.
Implementation
Mapping the description of the testcases to the corresponding implementation is really simple. Each row in the description of the testcases (in JBehave those are called steps) is mapped to a corresponding method in a Java class. The methods are marked with the proper annotations defined by JBehave. Passing parameters is handled using a naming convention ($variable) in the annotations. Of course those must then be as well defined as parameters in the method definitions.
1package de.codecentric.jbehave;
2
3import java.util.Stack;
4import org.jbehave.core.annotations.*;
5import org.jbehave.core.embedder.Embedder;
6import junit.framework.Assert;
7
8public class StackStories extends Embedder{
9
10 private Stack<String> testStack;
11 private String searchElement;
12
13 @Given("an empty stack")
14 public void anEmptyStack() {
15 testStack = new Stack<String>();
16 }
17
18 @When("the string $element is added")
19 public void anElementIsAdded(String element) {
20 testStack.push(element);
21 }
22
23 @When("the last element is removed again")
24 public void removeLastElement() {
25 testStack.pop();
26 }
27
28 @When("the element $element is searched for")
29 public void searchForElement(String element) {
30 searchElement = element;
31 }
32
33 @Then("the resulting element should be $result")
34 public void theResultingElementShouldBe(String result) {
35 Assert.assertEquals(testStack.pop(), result);
36 }
37
38 @Then("the position returned should be $pos")
39 public void thePositionReturnedShouldBe(int pos) {
40 Assert.assertEquals(testStack.search(searchElement), pos);
41 }
42}
The class shown above implements the so-called steps from the test descriptions. To be able to execute the tests from a story-file an additional Java class is required. This class is mapped to a story file using some naming conventions and furthermore some common settings (like reporting) are defined in this class:
1package de.codecentric.jbehave;
2
3import java.net.MalformedURLException;
4import java.net.URL;
5import java.util.List;
6
7import org.jbehave.core.configuration.Configuration;
8import org.jbehave.core.configuration.MostUsefulConfiguration;
9import org.jbehave.core.io.LoadFromRelativeFile;
10import org.jbehave.core.junit.JUnitStory;
11import org.jbehave.core.reporters.StoryReporterBuilder;
12import org.jbehave.core.reporters.StoryReporterBuilder.Format;
13import org.jbehave.core.steps.CandidateSteps;
14import org.jbehave.core.steps.InstanceStepsFactory;
15import org.junit.Test;
16
17public class StackScenarios extends JUnitStory {
18
19 @Override
20 public Configuration configuration() {
21 URL storyURL = null;
22 try {
23 // This requires you to start Maven from the project directory
24 storyURL = new URL("file://" + System.getProperty("user.dir")
25 + "/src/main/resources/stories/");
26 } catch (MalformedURLException e) {
27 e.printStackTrace();
28 }
29 return new MostUsefulConfiguration().useStoryLoader(
30 new LoadFromRelativeFile(storyURL)).useStoryReporterBuilder(
31 new StoryReporterBuilder().withFormats(Format.HTML));
32 }
33
34 @Override
35 public List<CandidateSteps> candidateSteps() {
36 return new InstanceStepsFactory(configuration(), new StackSteps())
37 .createCandidateSteps();
38 }
39
40 @Override
41 @Test
42 public void run() {
43 try {
44 super.run();
45 } catch (Throwable e) {
46 e.printStackTrace();
47 }
48 }
49}
The API of JBehave is offering some more possibilities, but these classes are sufficient to start testing. Unfortunately is the quality of the JavaDoc documentation of the API rather poor.
It should be noted that it is not mandatory to have an implementation for every step in the story-file right away. JBehave can handle situations where steps are missing and marks them then per default as “pending” in the report. This way tests can be written before the actual implementation must be there. The behaviour of JBehave with respect to this situation is configurable and if desired a missing step implementation can also result in an error.
Obviously it is very helpful being able to execute the tests directly from an IDE (e. g. Eclipse). This is working with JBehave very easily thanks to this unimposing method from the above shown class:
1...
2 @Override
3 @Test
4 public void run() {
5 try {
6 super.run();
7 } catch (Throwable e) {
8 e.printStackTrace();
9 }
10 }
11...
With this tests can be directly executed with JUnit . I found this possibility by chance when trying things out as it was not stated explicitly in the documentation (or I have just not seen it). But this way tests can for example be started in the debugger quite nicely.
Reporting
The following screenshot shows the report generated by JBehave when executing the tests from this example:
The amount of information is not really overwhelming but sufficient. Especially the – in this kind of report – important information about the execution date and time is shown in the footer of the report. The two links at the end of the row are leading to the definition of the related story-file and a page displaying the execution statistics directly. The whole configuration of the reporting has for sure still some possibilities to explore. But for this overview this should be enough for the time being.
The report itsself if located in the directory target/jbehave/view of the Eclipse workspace after execution. It has the name reports.html. To make the report look pretty additional style information is required. And as we luckily have already made the corresponding entries in the POM-file we can download the required files easily using Maven:
mvn generate-resources
Afterwards there should be additional directories (ftl, images …) showing up below the directory target/jbehave/view.
Putting the pieces together
There are three important parts for developing Acceptance Tests with JBehave:
- Maven Configuration: Once this one is working it is more or less static.
- Testcases: A new story-file is added for every new user story. Of course existing scenarios might be changed or extended in the course of a project.
- Implementation: The Java class required for executing the tests is more or less static. To add more stories it would make sense here to consider a generic super class. The implementation of the test functionality as such is happening in parallel to writing the testcases. These classes contain the real logic of the tests.
There are some naming conventions used in JBehave to glue certain things together. The following part of the POM-file is for example defining which classes are searched for to execute tests (=scenarios):
1... 2 <execution> 3 <id>run-stories-as-embeddables</id> 4 <phase>integration-test</phase> 5 <configuration> 6 <includes> 7 <include>**/*Scenarios.java</include> 8 </includes> 9 <ignoreFailureInStories>true</ignoreFailureInStories> 10 <ignoreFailureInView>false</ignoreFailureInView> 11 </configuration> 12 <goals> 13 <goal>run-stories-as-embeddables</goal> 14 </goals> 15 </execution> 16 ...
Based on the name of the “scenario class” – in this case StackScenarios.java – the corresponding story-file can be found. Therefore the story-file must have a name that matches the name of the “scenario class”, unless it is allowed to add additional underscores and the mapping is not case-sensitive. In our example the name of the story-file is stack_scenarios.story.
JBehave offers different possibilities to search for story-files. In our example an absolute path is used for this, which contains the current working directory. Therefore the tests must be executed from the project directory containing the POM-file. The name of the story-file is in addition prepended by the path that is mathcing the Java package of the scenario class. This must be considered when storing the story-files.
The scenario class contains a method that configures the class (or classes) that contain the required step definitions (methods):
1...
2 @Override
3 public List<CandidateSteps> candidateSteps() {
4 return new InstanceStepsFactory(configuration(), new StackSteps())
5 .createCandidateSteps();
6 }
7...
The corresponding classes must be reachable via the classpath, but probably it would be quite hard to make the implementation in a way that those classes can not be found.
Conclusion
Due to the length of this article one might think that JBehave is not as lightweight as promised in the introduction. But this is not true as the example in this article is covering for sure 80%-90% of the required implementation to get started with writing Acceptance Tests in a real project. The effort for this of course highly depends on the complexity of the tested software/system. But this is true for every tool used. An advantage here is that Java developers can probably estimate quite easily what is supported already by existing libraries and how much effort is thus required for writing the final test code.
The structure of the story-files is simple and clean. Furthermore there are additional features like the usage of tables with test-data to iterate over some scenarios with different test values. It is also possible to use some meta statements in the story-files to influence the reporting if this is desired. The reporting looks sufficient on first sight and can be extended still according to the documentation.
The good integration with Maven and Eclipse is the big strength of JBehave and makes it especially interesting for Java developers. The possibility to execute the tests easily using JUnit (e. g. for debugging purposes) is an additional advantage here.
Integrating JBehave into a CI-environment (Continuous Integration) is easily possible due to the usage of Maven. Testing Web-Applications is in addition supported with an own web component that is based on Selenium and of course one has the freedom to use the Java selenium library right away.
My very personal conclusion: Getting started with JBehave is fun and is having only very few moments of frustration (mainly when reading the API documentation). And even though I am not the biggest Maven-fan on this planet I have to admit that the integration with Maven is well done and helpful. I could well consider using the tool in some real-life project. Of course there are also still some more things to explore. For me most interesting here would be support of different languages in the story-files and testing web applications. Hopefully some of these aspects can be checked out in some forthcoming blog article.
Download
Here all the files required for the example can be downloaded as one ZIP-file. Afterwards the project can for example be imported to Eclipse.
The project can then be build issuing the following command:
mvn compile
The required style-files for the reporting can be downloaded using this command (required only once):
mvn generate-resources
Afterwards the tests can be executed using this command:
mvn integration-test
All commands must be executed from the project directory that is also containing the pom.xml-file.
Was this post helpful?
More articles
fromThomas Jaspers
22.2.2019 Your job at codecentric?
Jobs
Agile Developer und Consultant (w/d/m)
Alle Standorte
More articles in this subject area
Discover exciting further topics and let the codecentric world inspire you.
Gemeinsam bessere Projekte umsetzen.
Wir helfen deinem Unternehmen.
Du stehst vor einer großen IT-Herausforderung? Wir sorgen für eine maßgeschneiderte Unterstützung. Informiere dich jetzt.
Hilf uns, noch besser zu werden.
Wir sind immer auf der Suche nach neuen Talenten. Auch für dich ist die passende Stelle dabei.
Blog author
Thomas Jaspers
Do you still have questions? Just send me a message.
Do you still have questions? Just send me a message.