THTTPException: Received a response with a bad HTTP status code: 400

[deugene]

Hi! I had TypeError: Converting circular structure to JSON" when calling client.openSession() error. After applying changes from #89 I managed to get the real error:

THTTPException: Received a response with a bad HTTP status code: 400
 at exports.HttpConnection.responseCallback (/workspace/node_modules/thrift/lib/nodejs/lib/thrift/http_connection.js:173:26)
 at ClientRequest.connection.responseCallback (/workspace/node_modules/@databricks/sql/dist/connection/connections/HttpConnection.js:67:30)
 at Object.onceWrapper (node:events:628:26)
 at ClientRequest.emit (node:events:513:28)
 at HTTPParser.parserOnIncomingClient [as onIncoming] (node:_http_client:693:27)
 at HTTPParser.parserOnHeadersComplete (node:_http_common:128:17)
 at TLSSocket.socketOnData (node:_http_client:534:22)
 at TLSSocket.emit (node:events:513:28)
 at addChunk (node:internal/streams/readable:315:12)
 at readableAddChunk (node:internal/streams/readable:289:9)

the credentials are right because I'm refactoring the code that uses odbc + simbaspark driver and it works with the same credentials.

[kravets-levko]

Hi @deugene! Can you please share credentials you use (or even better - the whole code snippet)? Obviously, you can hide sensitive parts like PAT. I just want to make sure everything is okay with your code in general before I try other ideas. Thank you!

[deugene]

Hi, @kravets-levko! Thank you for your quick response! Yep, sure. It looks something like this:

const {logger} = require('.logger');
const {DBSQLClient} = require('@databricks/sql');
const databricks = new DBSQLClient()
const connection = {
 token: 'token',
 host: '0000000000000000.0.gcp.databricks.com',
 path: `sql/protocolv1/o/0000000000000000/0000-000000-0a0aaa0a`
}

async function getQueryResult({id, options}) {
 let results = [];
 const query = getQueryById(id);
 const statement = query.compile(options);

 try {
 const client = await databricks.connect(connection);
 const session = await client.openSession();
 const queryOperation = await session.executeStatement(statement, {
 runAsync: true
 });

 results = await queryOperation.fetchAll();

 await queryOperation.close();
 await session.close();
 await client.close();
 } catch (e) {
 const message = [
 `getQueryResult error: ${e.message}`,
 `statement:\n${statement}`
 ].join('\n\n');

 logger.error(message);

 throw new Error(message);
 }

 return results;
}

[deugene]

By the way, I have an off-topic question. Do I really have to do the following:

const client = await databricks.connect(connection);

each time I make a request?
Or can I do it once on app init and just open/close sessions to make a new request?

[deugene]

I found the reason!)) The path must start with "/"!
But in databricks docs they have this line:

...
The HTTP Path value for http_path is similar to the following: for a cluster, sql/protocolv1/o/1234567890123456/1234-567890-abcdefgh; ...

and in cluster settings HTTP Path also doesn't have a trailing /. I think it would be a good idea to handle paths both with and without a trailing / :slightly_smiling_face:

[kravets-levko]

Hi @deugene! We already did fix this, but looks it got lost somehow (#27). I'll double-check it ASAP, thank you for reporting!

Upd.: yes, I accidentally removed that feature during one of refactoring 🤦‍♂️

By the way, I have an off-topic question. Do I really have to do the following:
const client = await databricks.connect(connection);
each time I make a request?
Or can I do it once on app init and just open/close sessions to make a new request?

Yes, the idea is that you have a connection for each set of host+path+PAT, and then just open session(s) and do requests. You can even have a single session - no need to open a new one for each new request (but of course you may open as much sessions as you need).

[kravets-levko]

@deugene I restored a feature I mentioned above (#91), it will become available with next release. Meanwhile, since you have a workaround, I'll close this issue. Feel free to re-open it, or open another one if you have other questions. Thank you!