While working on my Sonar fork which allows to simulate refactoring without actually touching source files I have once again realized what a PITA it is to traverse the antlr-generated Abstract Syntax Tree (AST) for Java. The mechanism is absolutely cool, no doubt. But the final AST representation is not intuitive, and the corresponding traversal code always looks ugly.
While intensively working with Neo4j, I just asked myself: wouldn’t it be nice to use it as index for the Java-AST? You only need to jump to a relevant node and still can use the classic AST-traversal to get details out of it. Or you could wire the whole AST through an accompanying graph and thus use the graph to traverse the whole AST.
So, here comes Graphlr as the first result. It allows traversing the antlr3 generated Java AST using Neo4j. It’s of course open source and available through the same licence antlr is. Graphlr is nothing more than an antlr grammar/parser/lexer file based on the original one for Java and extending it with, well, code building the Neo4j graph. The graph is a sort of index, if you like, allowing quick jumps into the code spots you’re interested in.
Let’s look at an example, though I assume that you’re familiar with antlr, especially antlr3.
1package org.pbit.graphlr;
2
3import java.util.List;
4
5import org.antlr.runtime.ANTLRFileStream;
6import org.antlr.runtime.CharStream;
7import org.antlr.runtime.CommonTokenStream;
8import org.antlr.runtime.tree.Tree;
9
10
11public class GraphlrTest {
12 public static void main(String[] args) throws Exception {
13 try {
14 CharStream input = new ANTLRFileStream("/data/workspace/Graphlr/src/org/pbit/graphlr/GraphlrTest.java");
15 GraphlrJavaLexer lex = new GraphlrJavaLexer(input);
16 CommonTokenStream tokens = new CommonTokenStream(lex);
17 GraphlrJavaParser parser = new GraphlrJavaParser(tokens);
18 parser.compilationUnit();
19
20 //find compilation units
21 List<Tree> l = parser.runCypher("start ret=node:node_auto_index(type = 'class') return ret");
22 for (Tree t : l) {
23 System.out.println("Compilation unit found: " + t.getText());
24 }
25
26 //find methods of a particular class
27 l = parser.runCypher("start n=node:node_auto_index(name = 'GraphlrTest') match n-[:IMPLEMENTS]->ret return ret");
28 for (Tree t : l) {
29 System.out.println("GraphlrTest implements: " + t.getText());
30 }
31 } catch(Throwable t) {
32 t.printStackTrace();
33 }
34 }
35
36 protected void dummy () {}
37}
38
39final class Dummy {
40
41}
It’s very simple. First, it still works and feels like antlr3. So you use the generated classes to lex and parse your class (I’m doing it with the one including the test, so you can see the relation). Then you start with the compilation unit. From here, you can use available Neo4j nodes as well as indexes to traverse the AST. In my case, I use Cypher.
There are two traversals finding all the top level classes as well as all methods of the main class. That’s basically it.
So, there is still work to be done. Why I started this implementation is the main reason to extend the Neo4j based AST index to allow direct jumps into method calls, variable declaration etc. It’s a bit more tricky with antlr, but still doable.
If you want to check it out or maybe to contribute, feel free to get it from GitHub: https://github.com/pavlobaron/graphlr
Feedback is very welcome.
Was this post helpful?
More articles
fromPavlo Baron
10.1.2014 Your job at codecentric?
Jobs
Agile Developer und Consultant (w/d/m)
Alle Standorte
More articles in this subject area
Discover exciting further topics and let the codecentric world inspire you.
Gemeinsam bessere Projekte umsetzen.
Wir helfen deinem Unternehmen.
Du stehst vor einer großen IT-Herausforderung? Wir sorgen für eine maßgeschneiderte Unterstützung. Informiere dich jetzt.
Hilf uns, noch besser zu werden.
Wir sind immer auf der Suche nach neuen Talenten. Auch für dich ist die passende Stelle dabei.
Blog author
Pavlo Baron
Do you still have questions? Just send me a message.
Do you still have questions? Just send me a message.