A Debugger for the Robot Framework (almost)

In my recent blog entry I wrote, that I would write a couple of lines about the “robotframework debuger [sic!]” (“debugger” or just short “RDB” in the following). Have I been hasty? The project’s download page at Google Code says “prototype”. Should that scare me off? The repository shows no activity for more than two years – beside the initial commit. Should I’ve been warned?

No, why should I?

Welcome to a journey into the open source world of the “not quite finished yet” kind. Learn why RDB is the tool you have always been missing from Robot and why it won’t help you anyway… 😉

A bumpy start

The start is encouraging. Beside the source code there is a distribution package and precisely one page of documentation. Setup isn’t working because one file is missing from the distribution package. Fortunately the source code is complete and the installation works when started from the source directory.

RDB is implemented as a Listener, using the approach I drafted . A conservative start with “pybot --listener rdb.Listener SimpleTest.txt” leads to a full stop immediately: pybot aborts with an import error because Robot modules required by RDB are not found.

Searching the repository leads me to two insights:

1. The last stable Robot release meeting the requirements of RDB must have been Robot 2.5.
2. The required Robot modules have been ultimately deleted during refactoring.

Hence there is no quick fix. Instead I pick release 2.5 from the Robot repository to test RDB with that version.

The first impression astounds: Without setting a breakpoint the test just rushes through despite the debugger being in place. Started with a valid breakpoint (“pybot --listener rdb.Listener:Log SimpleTest.txt” sets a breakpoint at the “Log” keyword) the test runs until the breakpoint is reached and stops there. RDB prints the URL for accessing the web GUI to the command line.

User Interface & Functionality

Using a web GUI is a simple but clever move to avoid any influence that using the debugger might have on the “system under test”. So no more unintended focus change when doing GUI tests!

The GUI itself is simple and clear. Information about the current keyword are displayed in the upper left corner. When using user definied keywords the whole stack of keywords is shown. Breakpoint management is in the upper right corner. You can define new breakpoints and (de)activate breakpoints. Below this the currently used variables and their values are displayed. You can add new variables and change the variables’ values. Plus there is an input field to enter keywords for immediate execution. The left center area shows some status information about the test and the debugger. At the lower border an information area for the Listener attributes is placed. This shows some information about the current keyword’s attributes as well.

Above the display area the buttons controlling the debugger are lined up. The offer is (almost) complete:

• “Run”: executes a test until the next breakpoint or the end of the test suite is reached.
• “Go into”: executes the current keyword; for user defined keywords the first keyword of the definition is executed.
• “Go over”: executes the current keyword; user defined keywords are executed in the aggregate.
• “Go return”: executes the test until the next [Return], breakpoint, or the end of the test is reached.
• “Pause”: pauses the running test at any time without a breakpoint.
• “Refresh”: updates the GUI

Beside information about the current keyword all currently used variables plus their values are displayed. New variables can be created and variable values can be changed.

Road Holes

Yet there are certain things that just don’t work. The web GUI works well with Firefox (v13) and Internet Explorer (v9) – just the very first few clicks are ignored. Using Chrome (v20) is no option – the GUI will be laid out correctly but no interaction will show any effect.

Furthermore the GUI doesn’t process entered white space right: all spaces will be replaced by plus signes. Therefore de facto no function requiring manual input (setting new breakpoint, changing variable values, executing keywords) is usable.

Wish List

What functions are missing but would be nice to have?

• Source Code View: a view of the complete test case file; navigation within the test suite
• Interactive breakpoint management
• Using test cases as breakpoint (only keywords are allowed as breakpoint right now)
• Skip: ommitting a test step
• Changing the test code
• Continue the test execution at any point within the test code include “back steps”
• …

Conclusion: Sorry, not usable right now

How to judge this project? It is a proper prototype. It shows the concept and provides good functionality. It generates benefit. On the other side functions are missing or implemented improperly, even functions important for everyday use. Decisive is the missing progress in development. As the debugger has never been updated to the current releases of the Robot framwork it is simply not usable.

My request to the project owner for some information about the project’s future remains unanswered as I’m writing these lines.

That’s calling for a fork, isn’t it? 😀

PS: I’m interested in knowing your opinion on the idea of an interactive debugger for Robot in general. How do you debug your tests? Or is good logging good enough? Comments welcome!