added jdbc interpreter

[andrescelis]

I added a generalized JDBC interpreter for Zeppelin. It doesn't come packaged with any drivers to avoid licensing concerns (I saw a Mysql interpreter was rejected because of this and similar issues would arise with SQL Server).

In order to use, you download your JDBC driver, go to the interpreter settings menu, create a '%jdbc' interpreter, and set your driver's name, type, and location. Right now it supports SQL Server, Mysql, and Postgresql.

Since this interpreter uses java.sql.DriverManager, it supports all JDBC drivers with the same java code. However, JDBC driver names and connection url formats vary. The only thing needed to support a new backend is to modify JDBCConnectionUrlBuilder.java, and add a connection url method to use the right format for your backend.

Tests are coming, I am currently unsure what the best testing strategy for this is, but I welcome advice.

[Leemoonsoo]

Version of current master branch is '0.6.0-incubating-SNAPSHOT'

[Leemoonsoo]

You might also want to add JDBCInterpreter into the default configuration and conf/zeppelin-site.xml.template

[tzolov]

@andrescelis, could you please elaborate on the added-value of this interpreter?
It seems it tries to unify the JDBC URL syntax for different drivers Do you really want to do this? IMHO it is not a good practice to hide the driver specific options and conventions.
Also can't you achieve the same goal using one of the existing SQL interpreters like Postgresql or Phoenix to name some. If you add the desired driver jar to some of those interpreters and set the driver name and jdbc url through the provided properties would you be able to query any of the above mentioned DBs?

[randerzander]

a generic JDBC Interpreter with good property features may reduce the number of interpreters required in Zeppelin.

Looks like Postgres, Phoenix, Hive, Ignite, & Tajo could all work with a generic JDBC driver.

If custom code is needed for a JDBC driver, it could be added to a JDBC-extensions module instead of creating new top level Interpreters for every Zeppelin interface using JDBC. This may significantly reduce the number of PRs and necessary maintenance overall.

[andrescelis]

@Leemoonsoo Thanks for letting me know I made the changes.

@tzolov As @randerzander said, it would be nice to reduce the number of interpreters in Zeppelin and the JDBC module could be extended if necessary.

[tzolov]

@andrescelis, @randerzander i understand the benefits of unifying and abstracting common or related code but i fail to see how this jdbc interpreter helps with this? It still will be required to create a separate interpreter projects to cope different drivers specifics.
IMO the main pain point at the moment is the lack of generic Dep insertion functionality. This project doesn't help with this. Also trying to unify the JDBC driver's URL notations (correct me if i've misunderstood) is contraproductive. Usually those are well known. By abstracting them you force the devs to learn not only the driver specific notation but also the abstraction and the way to get around it...

[andrescelis]

@tzolov the java code to make a connection and execute a query on a jdbc backend is identical no matter what jdbc backend it is... the only differences are in the connection strings. I don't think that the connection strings would give the devs a lot of trouble because they would just need to provide the values and the connection string is made for them.
Even if this doesn't end up being the long term solution, in the short term it gives Zeppelin the ability to interact with SQL Server and MySQL in an easy way.

[djoelz]

NullArgumentException check for parameter d

[djoelz]

@andrescelis I would also love to see unit tests for:
JDBCInterpreter
DriverShim
JDBCConnectionUrlBuilder

[andrescelis]

@tzolov let's set up a call so we can discuss the design of the interpreter. Please feel free to message me at t-ancel@microsoft.com. I want to be able to understand your issue with the current design and work towards something that would help with what you mentioned about Dep insertion functionality.

[Leemoonsoo]

@andrescelis Can there be some test?

About configuration, my guess is many people will want to use different database name with single jdbc interpreter configuration. And like @tzolov suggested, dep insertion for loading jdbc driver will help a lot.

[andrescelis]

@Leemoonsoo yes I can add tests, but can you expand on what you mean by dep insertion? It is not a term I am familiar with. What specifically would you want to see? Thanks.

[tzolov]

@andrescelis , sorry for the delayed reply.
Design or any other project related discussions should be open for everyone. The apache mailing lists or source trackers are the best media for this.
I've expressed my concerns already. There is use of generic jdbc interpreter only if it improves over the specific such. The main challenge with the jdbc generalization at the moment is the lack to dynamically inject particularly jdbc jar dependencies. On the other hand artificial problems like generalizing the JDBC URL syntax seems useless and even harmful for me.

[jongyoul]

@andrescelis Hi, I'm Jongyoul Lee and trying to merge jdbc interpreters for Zeppelin. I saw this PR is the second trial for doing this. The first one is contributed by @1teed but I cannot contact him/her at all. Could you tell me that you are still interested in this issue to contributed to Zeppelin? If you are available to do this, I think we should fix some codes. Basically, I think generic jdbc interpreter would help people use it but your PR only supports some of jdbc interpreters. Could you please make this PR more generally?

[jongyoul]

@andrescelis Hi, I'll move to next PR. I hope you would understand this procedure.