SQLite parser

An ANTLR4 grammar for SQLite 3.8.x based on the official specs.

Install

To install this library, do the following:

git clone https://github.com/bkiers/sqlite-parser
cd sqlite-parser
mvn clean install -DskipTests=true

Test

The generated parser has been tested by approximately 30000 SQLite statements scraped from the test suite of the SQLite repository. Running these tests, which can take quite a few minutes, can be done as follows:

mvn clean test

If running the tests takes too long for your liking, try increasing the max heap space as follows:

export JAVA_TOOL_OPTIONS="-Xmx4096m"&& mvn clean test

Example

Let's say you would like to record all the names of functions used in an select-statement:

SELECT log AS x FROM t1
GROUP BY x
HAVINGcount(*) >=4ORDER BYmax(n) +0

This can be done by attaching a listener to the parse tree that listens when the parse tree enters an SQL expression, and the function name inside this expression is not null:

importorg.antlr.v4.runtime.ANTLRInputStream;
importorg.antlr.v4.runtime.CommonTokenStream;
importorg.antlr.v4.runtime.misc.NotNull;
importorg.antlr.v4.runtime.tree.ParseTree;
importorg.antlr.v4.runtime.tree.ParseTreeWalker;
importjava.util.ArrayList;
importjava.util.List;
publicclassMain {
publicstaticvoidmain(String[] args) throwsException {
// The list that will hold our function names.finalList<String> functionNames = newArrayList<String>();
// The select-statement to be parsed.Stringsql = "SELECT log AS x FROM t1 \n" +
"GROUP BY x \n" +
"HAVING count(*) >= 4 \n" +
"ORDER BY max(n) + 0 \n";
// Create a lexer and parser for the input.SQLiteLexerlexer = newSQLiteLexer(newANTLRInputStream(sql));
SQLiteParserparser = newSQLiteParser(newCommonTokenStream(lexer));
// Invoke the `select_stmt` production.ParseTreetree = parser.select_stmt();
// Walk the `select_stmt` production and listen when the parser// enters the `expr` production.ParseTreeWalker.DEFAULT.walk(newSQLiteBaseListener(){
@OverridepublicvoidenterExpr(@NotNullSQLiteParser.ExprContextctx) {
// Check if the expression is a function call.if (ctx.function_name() != null) {
// Yes, it was a function call: add the name of the function// to out list.functionNames.add(ctx.function_name().getText());
}
}
}, tree);
// Print the parsed functions.System.out.println("functionNames=" + functionNames);
}
}

which will print:

functionNames=[count, max]