ConnectionError

[jsu]

Summary

1. Taboola API started to show OpenSSL.SSL.WantReadError. Let's add requests.exceptions.ConnectionError to retry.

Deployment Instructions

1. merge

2. add new release

3. change requirements.txt in OpenMail.

[jsu]

The Error

Airflow

Error

[2020-07-30 15:37:04,354] {docker_operator.py:242} INFO - ERROR:ingestor:Account qoolmedia-yahoo-sc failed.
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/urllib3/contrib/pyopenssl.py", line 313, in recv_into
return self.connection.recv_into(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/OpenSSL/SSL.py", line 1840, in recv_into
self._raise_ssl_error(self._ssl, result)
File "/usr/local/lib/python3.6/site-packages/OpenSSL/SSL.py", line 1646, in _raise_ssl_error
raise WantReadError()
OpenSSL.SSL.WantReadError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/urllib3/response.py", line 425, in _error_catcher
yield
File "/usr/local/lib/python3.6/site-packages/urllib3/response.py", line 752, in read_chunked
self._update_chunk_length()
File "/usr/local/lib/python3.6/site-packages/urllib3/response.py", line 682, in _update_chunk_length
line = self._fp.fp.readline()
File "/usr/local/lib/python3.6/socket.py", line 586, in readinto
return self._sock.recv_into(b)
File "/usr/local/lib/python3.6/site-packages/urllib3/contrib/pyopenssl.py", line 326, in recv_into
raise timeout("The read operation timed out")
socket.timeout: The read operation timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/requests/models.py", line 750, in generate
for chunk in self.raw.stream(chunk_size, decode_content=True):
File "/usr/local/lib/python3.6/site-packages/urllib3/response.py", line 560, in stream
for line in self.read_chunked(amt, decode_content=decode_content):
File "/usr/local/lib/python3.6/site-packages/urllib3/response.py", line 781, in read_chunked
self._original_response.close()
File "/usr/local/lib/python3.6/contextlib.py", line 99, in __exit__
self.gen.throw(type, value, traceback)
File "/usr/local/lib/python3.6/site-packages/urllib3/response.py", line 430, in _error_catcher
raise ReadTimeoutError(self._pool, None, "Read timed out.")
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='backstage.taboola.com', port=443): Read timed out.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/OpenMail/etl/ingestor/ingest.py", line 72, in run
worker.run(manager)
File "/OpenMail/etl/ingestor/framework/worker.py", line 369, in run
handle_from_download(manager)
File "/OpenMail/etl/ingestor/framework/worker.py", line 391, in handle_from_download
manager.task_row, f.name, parent_account_id=manager.parent_account_id
File "/OpenMail/etl/ingestor/adapter/taboola.py", line 165, in download_fn
common.gzip_json_write(tmp_file, results)
File "/OpenMail/etl/ingestor/adapter/common.py", line 930, in gzip_json_write
jsonlines.Writer(f).write_all(rows)
File "/usr/local/lib/python3.6/site-packages/jsonlines/jsonlines.py", line 296, in write_all
for obj in iterable:
File "/OpenMail/etl/ingestor/adapter/taboola.py", line 132, in get_campaign_items
client, account_id, campaign["id"]
File "/src/pytaboola/pytaboola/services/base.py", line 41, in list
return self.execute('GET', self.build_uri())['results']
File "/src/pytaboola/pytaboola/services/base.py", line 28, in execute
query_params=query_params, **payload)
File "/usr/local/lib/python3.6/site-packages/tenacity/__init__.py", line 311, in wrapped_f
return self.call(f, *args, **kw)
File "/usr/local/lib/python3.6/site-packages/tenacity/__init__.py", line 391, in call
do = self.iter(retry_state=retry_state)
File "/usr/local/lib/python3.6/site-packages/tenacity/__init__.py", line 338, in iter
return fut.result()
File "/usr/local/lib/python3.6/concurrent/futures/_base.py", line 425, in result
return self.__get_result()
File "/usr/local/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
raise self._exception
File "/usr/local/lib/python3.6/site-packages/tenacity/__init__.py", line 394, in call
result = fn(*args, **kwargs)
File "/src/pytaboola/pytaboola/client.py", line 116, in execute
timeout=self.timeout)
File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 60, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 686, in send
r.content
File "/usr/local/lib/python3.6/site-packages/requests/models.py", line 828, in content
self._content = b''.join(self.iter_content(CONTENT_CHUNK_SIZE)) or b''
File "/usr/local/lib/python3.6/site-packages/requests/models.py", line 757, in generate
raise ConnectionError(e)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='backstage.taboola.com', port=443): Read timed out.