Robot Framework Tutorial
Part I: Robot Framework Tutorial – Overview
Part II: Robot Framework Tutorial – A complete example
Part III: Robot Framework IDE
Part IV: How to Structure a Scalable And Maintainable Acceptance Test Suite
Part V: Robot Framework Tutorial – Writing Keyword Libraries in Java
Part VI: Robot Framework Tutorial – Loops, Conditional Execution and more
Part VII: Robot Framework – Testing Windows Applications
Appendix A: Robot Framework – Compact Sheet
The new Robot Framework Tutorial 2016 series
Please take a look here for an updated version on testing web applications.
Update February 2019: It might be worth looking at this newer blog post that is utilizing Docker to install the Robot Framework:
https://blog.codecentric.de/en/2019/02/running-robot-framework-in-your-own-docker-container/
The biggest problem when starting a new project with the Robot Framework is probably the pure amount of possible options it is offering:
- Using Python, Jython, or Java installation?
- Which input format should be used to implement the tests (HTML, Text, BDD)?
- This has direct impact on the question whether or not the Robot IDE (RIDE) can be used?
- How to enable running the same tests locally and on some CI environment?
- How to exceute the tests at all (scripting, ANT, Maven)?
So what would be the best way to go? Well, I have seen quite some different Robot Framework setups and I sense a certain urge to overly complicate things. Of course it is cool to start the Robot Tests from Eclipse using Maven (we have blogged on this ). And writing tests in BDD-style is certainly more agile than using the HTML-format (funny thing, isn’t it, we have blogged on this as well :-)).
But there is one thing that is really cool: Simplicity! And this does not only mean it is simple once everything is setup and works, but it is also simple to understand a problem when things are not working. This is especially important in teams with different technical background.
So let’s start setting up a simple and bulletproof Robot Framework project …
Some initial Thoughts
But B-4 (any Trekkies around :)) starting to setup a new Robot Framework project it must always be analysed which Test-Libraries are needed for testing the System under Test (SuT). A few examples:
- Testing a web application? Then one will probably include the SeleniumLibrary or Selenium2Library .
- There is a database involved? Then one of the Database Libraries (Python or Java ) should be considered.
- Need to test SSH/SFTP? Then SSHLibrary is for sure worth a try.
The list could of course be continued for all the available Test Libraries, up to the point where no ready-made Test Library is available. Then you would need to plan writing your own one. (But this is something to cover in an article of its own.)
But why is this important? The Test Libraries used might have direct impact on whether the Python or Jython/Java installation of the Robot Framework must be used (compare Robot Framework Tutorial – Overview ). But again, only because some required Test Library is implemented in Java it could still be possible to use the pure Python installation if the Remote Library Interface is supported by that library. Thus these things should really be well thought of before starting.
Tip: The complete code example for this article can be found from GitHub.
Luckily I have done this thinking already for a hypothetical SuT :-). Let’s imagine that this is a web application that makes use of some database (quite typical use case from my experience). To keep things as simple as possible the tested web application will be Google and the database will simply be a local MySQL database instance. Of course there is then no real relation between Google and this local database, but that does not matter for demonstrating the tests. What is more important is to show how to mix different Test Libraries (in this example SeleniumLibrary and DatabaseLibrary) and make use of the Remote Test Library feature of the DatabaseLibrary at the same time. I cannot really tell why, but I tend to use the Java version from the available database libraries .
“Testing Architecture”
The following figure gives an overview on the “Testing Architecture” (sounds cool, doesn’t it?):
Overview on the “Testing Architecture”
Selenium always requires a running Selenium Server. That could be run on the same machine where the Robot Framework is running, but it can also be any other Server that can be reached via TCP/IP. The same holds true for the Database Library Server. DatabaseLibrary could be used without RPC, but as then we would be forced to use Jython let’s use it as a server as well. Of course locally all these servers will be running on the same machine (namely your PC). In some official testing environment that is typically setup for a project it is often the case that the Robot Framework is running on some Linux box together with the CI-server. Then the Selenium Server is running on some Windows-Server, as you would like to test with browser versions that are close to those used by the end users. The DatabaseLibrary Server could then again run on the CI-Server. That does not hurt.
Test Implementation & Organisation
Finally we are coming to the implementation of the tests. We will not go to all the details in the implementation here, as those can be seen from the sample project. But again let’s more look at some general considerations that should be made when implementing tests using the Robot Framework.
As mentioned in the beginning here a decision is needed which format (HTML, Text, BDD) to use and whether or not RIDE should be used. The usage of RIDE is having direct impact on the choice of the used format. Of course one should also consider the – probably different – technical knowledge of the team members and – one step further – of a maybe different team supporting a potential maintenance phase.
Tip: If there is already testdata available in a tool like Excel this data can be easily Copy & Pasted to your tests when using RIDE.
If I am in charge to make a choice – and in this article I am 🙂 – it is very simple: Use the HTML-format and use RIDE. Here is my reasoning for this:
- RIDE has made big progress since its beginning and its auto-completion for keywords and guidence in implementing Test Suites and Resource Files is extremely helpful.
- Using RIDE there is no need to consider writing tests in BDD-style anyway. But honestly there are some syntactical elements required here that I simply do not like. Furthermore I believe it is harder to write/maintain for less technical oriented team members, because there is not yet such a thing like a machine that can understand human language. That is an illusion. I also think that other Testing Frameworks that have started with BDD (and often only supporting BDD) are having an advantage if BDD is the one and most important requirement.
- The HTML-format is simply great. You can visualize the tests nicely in a Browser and I would say it is quite easy to understand by just looking at it. Especially less technical oriented team members feel at home with the table-style test descriptions as those are looking to some extend familiar from using tools like Excel.
- There is also a disadvantage in using HTML-format, as merging changes in Source Control Systems can be problematic. But this is a problem that can be tackled.
Of course one of the most important aspects for implementing the tests is to be able to execute the same tests locally and on the official test environment (CI environment). Luckily this can be achieved quite easily with the Robot Framework, as it is possible to pass in parameters to the tests that can then be used in the corresponding keywords. Some typical examples:
- The starting URL of the web-application to be tested.
- IP-address and port of the Selenium-Server.
- JDBC connection string to connect to the database using the Database Library.
These parameters can be collected in variable files . These variable files can then be passed to the Robot Framework in the command line. As we will anyway have different startup-scripts locally and on the CI-environment this can be easily steered. The parameters from those files are then used in the test implementation as variables.
This leads us to some central part of organizing the tests: The directory structure.
Project Directory Structure
Defining a common directory structure – that might even be re-used in different projects – is of course helpful as it is then much easier to work in different projects. The directory structure depicted above is working very well for me since quite some time already (with minor tweak every now and then).
Tip: All path information should be always given relative. For example references from Testsuites to Resource Files or a Testsuite given as a parameter to a robot start script.
First of all we bundle everything below one top-level directory (robot). Then we distinguish between the implementation and the execution of the tests. On the implementation side all Testsuites go to the testsuites-directory and all Resource Files go to the, well I guess you guessed it already. Of course in some really big projects some additional sub-directories might be created to further structure the tests. What is really important that – when referencing any other files (e. g. Resource Files from Testsuites) – all path information is given only relative. This way it does not matter at all on which operating system and in which directory the top-level robot-directory is located. This is especially important as all these files are under some Source Control System and every team member might check out files to different locations.
The execution-branch is used to handle the different target environments. Typically this means a local environment on the development machines of the team and some official CI environment. If more environments would be required additional branches can be implemented here. Quite obviously the scripts-directory is holding all scripts required for execution (robot itself, Selenium Server, Database Library Remote Server) and the settings-directory is holding the specific variable files. It should be noted that especially the scripts are only written once in the beginning of the project and are very rarely changed afterwards. The same is true for the settings as those should only cover differences in the execution environments.
Finally the is the lib-folder. Of course this one can be used to check-in own Test Libraries, but it might also make sense to store ready-made libraries here to ensure they are kept stable. It heavily depends on the project whether or not this folder is required.
Test Execution
When it comes to executing the tests I simply stick to shell-scripts. Easy to understand, working without problems since ages (do I have to feel old saying this?) and easy to use in any CI environment. Of course we will probably have to write two different versions of all those startup-scripts as it is often the case that locally the tests must be started on a Windows PC, but on the official CI-environment some bash – or csh -script is required. But as mentioned beforehand this is a “do once and forget” job and the scripts are not really that complicated.
For the beginning we need three scripts:
- One to start the robot tests as such.
- One to start the Selenium Server.
- One to start the Database Library Remote Server.
Again we could do some fancy stuff here having one script starting everything, but why should we do that. We anyway have to start the latter two scripts only once and afterwards only the tests are re-executed. Let’s better spend that time for implementing the tests. So basically that’s it for the execution.
Putting the pieces together
First of all we need to install the Robot Framework and required Libraries on the development machines. We assume this is Windows. Installation on Unix should be checked from the corresponding tool pages, but should also not be too complicated.
Tip: Robot Framework is not supporting 3.x versions of Python and wxPython (required for RIDE) is having problems with 2.7x version of Python, thus stick to Python 2.6x.
For this example the following versions of the required tools have been used:
- python-2.6.6.msi – Python 2.6.6
- robotframework-2.7.1.win32.exe – Robot Framework 2.7.1
- wxPython2.8-win32-unicode-2.8.12.1-py26.exe – wxPython 2.8.12.1 (Required unicode support for RIDE)
- robotframework-ride-0.43.3.win32.exe – RIDE 0.43.3
- robotframework-seleniumlibrary-2.8.1.win32.exe – Selenium2Library 2.8.1
- dblibrary-2.0-server.jar – Database Library Server 2.0
Just install the first five artefacts in the given order using the installers (I am using defaults for everything all the time). Then add the following part to your PATH variable: “C:\Python26;C:\Python26\Scripts”. Now you should be able to execute Robot Framework using “pybot –version” and to start RIDE from “C:\Python26\Scripts\ride.py”.
Robot Framework & RIDE after installation
Downloading the Database Library JAR was more like an exercise as this one is bundled (just copied) to the lib-folder of the sample project anyway :).
The Selenium Library is always bundled with a corresponding Selenium Server JAR. To be able to use this one independent from Robot (well, Python) installed to different folders by different team members you have to define a new environment variable RF_HOME pointing to your Python installation directory (default would be “C:\Python26”). This variable is used in the local scripts starting the selenium server.
To make use of a local MySQL database the same database settings are used as in the DBLibrary Sample Project . Thus in order for this example to work a local MySQL database must be installed and a testing schema and a corresponding user must be created as follows:
C:\xampp\mysql\bin>mysql -u root -p
mysql> create database databaselibrarydemo;
mysql> create user ‘dblib’@’localhost’ identified by ‘dblib’;
mysql> grant all privileges on databaselibrarydemo.* to ‘dblib’;
Of course this will look slightly different on different operating systems. Basically this is all that is required to start executing the sample tests.
All other required files are either already installed together with the Robot Framework (like for example the Selenium Server JAR) or are coming from the project that needs to be checked out from GitHub.
In the directory robot/execution/local/scripts are all the scripts needed to execute the tests. The test implementation can be found from robot/implementation/testsuites. The testcases can be best viewed/edited in RIDE by opening the implementation directory directly. Then all files can be viewed and edited directly.
To start the tests first the Selenium-Server and the DBLibrary-Server must be started. The the Sample Testsuite can be started. Windows batch scripts to start everything can be found from robot\execution\local\scripts. If everything works out as planned (hopefully it does) all scripts should work without any modifications, as all path information is given relative to the project directory structure. There are no server-side files given, but those can be easily adapted from the local files.
Conclusion & Impressions
We have seen that the Robot Framework is offering a lot of functionality and possibilities to do – maybe even the same – things in different ways. Thus it really makes sense to start with some basic analysis of the required test functionality before starting to hack some tests.
Editing Testsuites and Resource Files in RIDE
Using RIDE makes implementing test functionality easier, especially for less technical oriented team members. Keeping things simple will help in the long run maintaining the project (this is not only true for Robot Framework tests ;-)).
Selenium Server up and running
By the way, I think it was not explicitly noticed in this article yet, but the reporting and logging capabilities of the Robot Framework are really great and are helping a lot in troubleshooting problems in the test implementation.
Robot Framework Logfile with embedded screenshot
The Robot Framework is offering a lot of predefined testing functionality in various test libraries. This really helps a lot in starting to write tests once it is clarified which libraries are best to be used. Doing some pre-testing (prototyping) to check different test libraries will help a lot to decide which test libraries should be used productively.
Database Library Server running
Especially when doing project work in a lot of different projects (in opposite to testing one in-house product for example) a common project structure and common tooling will help a lot. When testing a product it makes of course much sense to create own keywords in some common resource file to encapsulate the testing functionality and making changes easier (centralized).
Hopefully this gives you some ideas how to start working with the Robot Framework and how to structure test functionality. Of course there are a lot of possibilities still to enhance this sample project. But hopefully it is a good starting point nonetheless.
Please leave comments and your own experiences in the comments section and I have the feeling there are more Robot Framework related articles to come on this channel :-).
Was this post helpful?
More articles
fromThomas Jaspers
22.2.2019 Your job at codecentric?
Jobs
Agile Developer und Consultant (w/d/m)
Alle Standorte
More articles in this subject area
Discover exciting further topics and let the codecentric world inspire you.
Gemeinsam bessere Projekte umsetzen.
Wir helfen deinem Unternehmen.
Du stehst vor einer großen IT-Herausforderung? Wir sorgen für eine maßgeschneiderte Unterstützung. Informiere dich jetzt.
Hilf uns, noch besser zu werden.
Wir sind immer auf der Suche nach neuen Talenten. Auch für dich ist die passende Stelle dabei.
Blog author
Thomas Jaspers
Do you still have questions? Just send me a message.
Do you still have questions? Just send me a message.