If the token is null, the connection hangs

[wallystart]

Hi, today we were troubleshooting a process that uses databricks-sql-python. The connection hangs if the token is null.

Evidence to reproduce:

(The cell never ends)

Generating token it ends correctly:

We are using databricks-sql-connector==3.4.0

Thanks!

[kravets-levko]

Hi @wallystart! In v3.4.0, when you omit PAT, U2M OAuth will be used. It means that library will try to open browser tab and ask to log in. If browser wasn't open for some reason - it may look like the whole thing hung up. I'm curious how you run your script, and in which environment. Also, please enable debug mode (add following lines in beginning of your script) and share the full log you'll get from the library:

import logging

logging.basicConfig(level=logging.DEBUG)

[wallystart]

Hi @wallystart! In v3.4.0, when you omit PAT, U2M OAuth will be used. It means that library will try to open browser tab and ask to log in. If browser wasn't open for some reason - it may look like the whole thing hung up. I'm curious how you run your script, and in which environment. Also, please enable debug mode (add following lines in beginning of your script) and share the full log you'll get from the library:

import logging

logging.basicConfig(level=logging.DEBUG)

Hi @kravets-levko ! Thanks for your answer. Probably it wasn't open because i'm running this on a Databricks notebook.

Here the full log

DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): [REDACTED]
DEBUG:urllib3.connectionpool:[[REDACTED]:443]([REDACTED]) "GET /oidc/.well-known/oauth-authorization-server HTTP/1.1" 200 None
INFO:databricks.sql.auth.oauth:Opening [REDACTED]/oidc/oauth2/v2.0/authorize?response_type=code&client_id=databricks-sql-python&redirect_uri=http%3A%2F%2Flocalhost%3A8020&scope=sql+offline_access&state=gAS4QglCJ9EAXb6raUfcvQ&code_challenge=7DZzBPfS4S6bnhz5p-StczFmxsW5lxs217U0KnYPqTk&code_challenge_method=S256
INFO:databricks.sql.auth.oauth:Listening for OAuth authorization callback at http://localhost:8020/
Starting connection to Unity Catalog

Caching the list of root modules, please wait!
(This will only be done once - type '%rehashx' to reset cache!)

DEBUG:parso.python.diff:diff parser start
DEBUG:parso.python.diff:line_lengths old: 3; new: 3
DEBUG:parso.python.diff:-> code[equal] old[1:2] new[1:2]
DEBUG:parso.python.diff:parse_part from 1 to 4 (to 2 in part parser)
DEBUG:parso.python.diff:-> code[replace] old[3:3] new[3:3]
DEBUG:parso.python.diff:diff parser end
DEBUG:Comm:handle_msg[LSP_COMM_ID]({'header': {'msg_type': 'comm_msg', 'msg_id': '0fa6d0fd-71c1-4775-8920-a29cb15cec7a', 'session': '', 'username': '', 'date': datetime.datetime(2024, 10, 23, 11, 41, 34, 895000, tzinfo=tzlocal()), 'version': '5.4'}, 'msg_id': '0fa6d0fd-71c1-4775-8920-a29cb15cec7a', 'msg_type': 'comm_msg', 'parent_header': {'session': '', 'username': '', 'version': '5.3'}, 'metadata': {'enableLSPGoToDefinitionOnModule_v2': 'true', 'enableWSFSPythonModulePeek': 'false', 'jediAutoImportModulesList': 'asyncio,numpy,pandas,pyspark,pyspark.ml,pyspark.sql,seaborn,urllib3'}, 'content': {'comm_id': 'LSP_COMM_ID', 'data': {'header': {'lsm_protocol_version': 0, 'msg_type': 'lsp_msg', 'ls_instance_id': 'd36a70de-578c-4c6c-9b77-5de626e9ecc9'}, 'body': {'lsp_msg': {'jsonrpc': '2.0', 'method': 'textDocument/didChange', 'params': {'cellRanges': [{'startLine': 0, 'stopLine': 2, 'commandId': 4155759404121619}, {'startLine': 2, 'stopLine': 4, 'commandId': 4155759404121620}, {'startLine': 4, 'stopLine': 17, 'commandId': 4155759404121621}, {'startLine': 17, 'stopLine': 302, 'commandId': 4155759404121622}, {'startLine': 302, 'stopLine': 316, 'commandId': 4155759404121668}, {'startLine': 316, 'stopLine': 317, 'commandId': 4155759404121623}, {'startLine': 317, 'stopLine': 406, 'commandId': 4155759404121624}, {'startLine': 406, 'stopLine': 407, 'commandId': 4155759404121625}, {'startLine': 407, 'stopLine': 408, 'commandId': 4155759404121626}, {'startLine': 408, 'stopLine': 410, 'commandId': 4155759404121627}, {'startLine': 410, 'stopLine': 488, 'commandId': 4155759404121628}, {'startLine': 488, 'stopLine': 489, 'commandId': 4155759404121629}, {'startLine': 489, 'stopLine': 589, 'commandId': 4155759404121630}, {'startLine': 589, 'stopLine': 590, 'commandId': 4155759404121631}, {'startLine': 590, 'stopLine': 745, 'commandId': 4155759404121632}, {'startLine': 745, 'stopLine': 747, 'commandId': 1533995912379064}, {'startLine': 747, 'stopLine': 1000, 'commandId': 1533995912379065}, {'startLine': 1000, 'stopLine': 1001, 'commandId': 4155759404121633}, {'startLine': 1001, 'stopLine': 1096, 'commandId': 4155759404121634}, {'startLine': 1096, 'stopLine': 1110, 'commandId': 1788161016397273}, {'startLine': 1110, 'stopLine': 1125, 'commandId': 4289103047541716}, {'startLine': 1125, 'stopLine': 1127, 'commandId': 4289103047541718}, {'startLine': 1127, 'stopLine': 1129, 'commandId': 4289103047541717}, {'startLine': 1129, 'stopLine': 1131, 'commandId': 1788161016397274}, {'startLine': 1131, 'stopLine': 1132, 'commandId': 4155759404121635}, {'startLine': 1132, 'stopLine': 1138, 'commandId': 4155759404121636}, {'startLine': 1138, 'stopLine': 1139, 'commandId': 4155759404121637}, {'startLine': 1139, 'stopLine': 1140, 'commandId': 4155759404121638}, {'startLine': 1140, 'stopLine': 1153, 'commandId': 1578124208117059}, {'startLine': 1153, 'stopLine': 1170, 'commandId': 4155759404121639}, {'startLine': 1170, 'stopLine': 1171, 'commandId': 4155759404121640}, {'startLine': 1171, 'stopLine': 1174, 'commandId': 4155759404121641}, {'startLine': 1174, 'stopLine': 1175, 'commandId': 4155759404121644}, {'startLine': 1175, 'stopLine': 1182, 'commandId': 4155759404121645}, {'startLine': 1182, 'stopLine': 1184, 'commandId': 4289103047541711}, {'startLine': 1184, 'stopLine': 1187, 'commandId': 4289103047541710}, {'startLine': 1187, 'stopLine': 1188, 'commandId': 4155759404121646}, {'startLine': 1188, 'stopLine': 1193, 'commandId': 4155759404121647}, {'startLine': 1193, 'stopLine': 1194, 'commandId': 4155759404121648}, {'startLine': 1194, 'stopLine': 1195, 'commandId': 4155759404121649}, {'startLine': 1195, 'stopLine': 1487, 'commandId': 4155759404121650}, {'startLine': 1487, 'stopLine': 1488, 'commandId': 4155759404121651}, {'startLine': 1488, 'stopLine': 1489, 'commandId': 4155759404121652}, {'startLine': 1489, 'stopLine': 1505, 'commandId': 4155759404121653}, {'startLine': 1505, 'stopLine': 1506, 'commandId': 4155759404121654}, {'startLine': 1506, 'stopLine': 1522, 'commandId': 4155759404121655}, {'startLine': 1522, 'stopLine': 1523, 'commandId': 4155759404121656}, {'startLine': 1523, 'stopLine': 1611, 'commandId': 4155759404121657}, {'startLine': 1611, 'stopLine': 1612, 'commandId': 4155759404121658}, {'startLine': 1612, 'stopLine': 1613, 'commandId': 1788161016397275}, {'startLine': 1613, 'stopLine': 1759, 'commandId': 4155759404121659}, {'startLine': 1759, 'stopLine': 1760, 'commandId': 4155759404121660}, {'startLine': 1760, 'stopLine': 1764, 'commandId': 4155759404121661}, {'startLine': 1764, 'stopLine': 1765, 'commandId': 4155759404121662}, {'startLine': 1765, 'stopLine': 1766, 'commandId': 4155759404121663}, {'startLine': 1766, 'stopLine': 1769, 'commandId': 4155759404121664}], 'editedCommandId': 4155759404121639, 'contentChanges': [{'text': '\n'}], 'cursorPosition': {'line': 1153, 'character': 14}, 'textDocument': {'uri': '/notebook/ba35a6c6-fa71-460a-823e-66fbfe481e63/4155759404121618', 'version': 26}}}}}}, 'buffers': []})
DEBUG:Comm:handle_msg[LSP_COMM_ID]({'header': {'msg_type': 'comm_msg', 'msg_id': '031a1ab9-1521-48fe-809c-874e74334217', 'session': '', 'username': '', 'date': datetime.datetime(2024, 10, 23, 11, 41, 34, 896000, tzinfo=tzlocal()), 'version': '5.4'}, 'msg_id': '031a1ab9-1521-48fe-809c-874e74334217', 'msg_type': 'comm_msg', 'parent_header': {'session': '', 'username': '', 'version': '5.3'}, 'metadata': {'enableLSPGoToDefinitionOnModule_v2': 'true', 'enableWSFSPythonModulePeek': 'false', 'jediAutoImportModulesList': 'asyncio,numpy,pandas,pyspark,pyspark.ml,pyspark.sql,seaborn,urllib3'}, 'content': {'comm_id': 'LSP_COMM_ID', 'data': {'header': {'lsm_protocol_version': 0, 'msg_type': 'lsp_msg', 'ls_instance_id': 'd36a70de-578c-4c6c-9b77-5de626e9ecc9'}, 'body': {'lsp_msg': {'jsonrpc': '2.0', 'method': 'textDocument/didOpen', 'params': {'textDocument': {'uri': '/notebook/ba35a6c6-fa71-460a-823e-66fbfe481e63/4155759404121618', 'languageId': 'python', 'version': 1, 'text': '\n'}, 'cellRanges': [{'startLine': 0, 'stopLine': 2, 'commandId': 4155759404121619}, {'startLine': 2, 'stopLine': 4, 'commandId': 4155759404121620}, {'startLine': 4, 'stopLine': 17, 'commandId': 4155759404121621}, {'startLine': 17, 'stopLine': 302, 'commandId': 4155759404121622}, {'startLine': 302, 'stopLine': 316, 'commandId': 4155759404121668}, {'startLine': 316, 'stopLine': 317, 'commandId': 4155759404121623}, {'startLine': 317, 'stopLine': 406, 'commandId': 4155759404121624}, {'startLine': 406, 'stopLine': 407, 'commandId': 4155759404121625}, {'startLine': 407, 'stopLine': 408, 'commandId': 4155759404121626}, {'startLine': 408, 'stopLine': 410, 'commandId': 4155759404121627}, {'startLine': 410, 'stopLine': 488, 'commandId': 4155759404121628}, {'startLine': 488, 'stopLine': 489, 'commandId': 4155759404121629}, {'startLine': 489, 'stopLine': 589, 'commandId': 4155759404121630}, {'startLine': 589, 'stopLine': 590, 'commandId': 4155759404121631}, {'startLine': 590, 'stopLine': 745, 'commandId': 4155759404121632}, {'startLine': 745, 'stopLine': 747, 'commandId': 1533995912379064}, {'startLine': 747, 'stopLine': 1000, 'commandId': 1533995912379065}, {'startLine': 1000, 'stopLine': 1001, 'commandId': 4155759404121633}, {'startLine': 1001, 'stopLine': 1096, 'commandId': 4155759404121634}, {'startLine': 1096, 'stopLine': 1110, 'commandId': 1788161016397273}, {'startLine': 1110, 'stopLine': 1125, 'commandId': 4289103047541716}, {'startLine': 1125, 'stopLine': 1127, 'commandId': 4289103047541718}, {'startLine': 1127, 'stopLine': 1129, 'commandId': 4289103047541717}, {'startLine': 1129, 'stopLine': 1131, 'commandId': 1788161016397274}, {'startLine': 1131, 'stopLine': 1132, 'commandId': 4155759404121635}, {'startLine': 1132, 'stopLine': 1138, 'commandId': 4155759404121636}, {'startLine': 1138, 'stopLine': 1139, 'commandId': 4155759404121637}, {'startLine': 1139, 'stopLine': 1140, 'commandId': 4155759404121638}, {'startLine': 1140, 'stopLine': 1153, 'commandId': 1578124208117059}, {'startLine': 1153, 'stopLine': 1170, 'commandId': 4155759404121639}, {'startLine': 1170, 'stopLine': 1171, 'commandId': 4155759404121640}, {'startLine': 1171, 'stopLine': 1174, 'commandId': 4155759404121641}, {'startLine': 1174, 'stopLine': 1175, 'commandId': 4155759404121644}, {'startLine': 1175, 'stopLine': 1182, 'commandId': 4155759404121645}, {'startLine': 1182, 'stopLine': 1184, 'commandId': 4289103047541711}, {'startLine': 1184, 'stopLine': 1187, 'commandId': 4289103047541710}, {'startLine': 1187, 'stopLine': 1188, 'commandId': 4155759404121646}, {'startLine': 1188, 'stopLine': 1193, 'commandId': 4155759404121647}, {'startLine': 1193, 'stopLine': 1194, 'commandId': 4155759404121648}, {'startLine': 1194, 'stopLine': 1195, 'commandId': 4155759404121649}, {'startLine': 1195, 'stopLine': 1487, 'commandId': 4155759404121650}, {'startLine': 1487, 'stopLine': 1488, 'commandId': 4155759404121651}, {'startLine': 1488, 'stopLine': 1489, 'commandId': 4155759404121652}, {'startLine': 1489, 'stopLine': 1505, 'commandId': 4155759404121653}, {'startLine': 1505, 'stopLine': 1506, 'commandId': 4155759404121654}, {'startLine': 1506, 'stopLine': 1522, 'commandId': 4155759404121655}, {'startLine': 1522, 'stopLine': 1523, 'commandId': 4155759404121656}, {'startLine': 1523, 'stopLine': 1611, 'commandId': 4155759404121657}, {'startLine': 1611, 'stopLine': 1612, 'commandId': 4155759404121658}, {'startLine': 1612, 'stopLine': 1613, 'commandId': 1788161016397275}, {'startLine': 1613, 'stopLine': 1759, 'commandId': 4155759404121659}, {'startLine': 1759, 'stopLine': 1760, 'commandId': 4155759404121660}, {'startLine': 1760, 'stopLine': 1764, 'commandId': 4155759404121661}, {'startLine': 1764, 'stopLine': 1765, 'commandId': 4155759404121662}, {'startLine': 1765, 'stopLine': 1766, 'commandId': 4155759404121663}, {'startLine': 1766, 'stopLine': 1769, 'commandId': 4155759404121664}]}}}}}, 'buffers': []})

Regards

[kravets-levko]

Hi @wallystart! Yes, logs clearly indicate that library tries to use U2M OAuth flow, which implies opening link in a browser. Unfortunately, that's expected behavior, so if you need to use the library in a notebook - you have to provide access token.

Although, I will keep this issue open - we definitely need to handle this use case (maybe, throw an error if library cannot open an OAuth link).

PECO-2054

[wallystart]

Hi @kravets-levko! Thanks for your response. Yes, we do provide an access token, but in this case, it was due to our bug (generating a null access token). We’ve already fixed it, but I opened this issue so you can analyze and determine if it needs further action :)

Thanks again!

[susodapop]

@kravets-levko one way to handle this is to check whether the user passed a None value versus the None default being used. Easiest way to achieve this is with a singleton object which you could call NotPassed. If you then set the default value in .connect() to this value, you can assert that nothing was passed and bail to the oauth flow. Whereas if the user passed a None value you can raise a nice exception that tells them something in their code is out of whack.

[susodapop]

I opened a pull request that implements the pattern described above: #462

[peco-engineer-bot]

🤖 engineer-bot opened a fix PR: #876

Review before merge.