Skip to content

Cleanup code for elasticsearch<8 - #35707

Merged
eladkal merged 3 commits into
apache:mainfrom
eladkal:elastic
Dec 1, 2023
Merged

Cleanup code for elasticsearch<8#35707
eladkal merged 3 commits into
apache:mainfrom
eladkal:elastic

Conversation

@eladkal

@eladkaleladkal commented Nov 17, 2023

Copy link
Copy Markdown
Contributor

Starting elasticsearch>=8 there is no more use_ssl
see https://github.com/apache/airflow/pull/33135/files#r1285347163

mentioning also open question about supporting elasticsearch<8
#33281 (comment)

cc @sunank200@Owen-CH-Leung


EDIT:
This PR practically reverts #33281 as provider setup is to support elasticsearch>=8

Thus users can't install older version of the library anyway.

Comment threaddocs/apache-airflow-providers-elasticsearch/logging/index.rst Outdated

@harishkesavaraoharishkesavarao left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree with @Taragolis to remove it altogether from the documentation, but also updating the doc here pending min version decision sounds good.

@eladkaleladkal changed the title update elasticsearch doc for use_ssl parameterRemove backport support for elasticsearch<8Nov 25, 2023
@eladkaleladkal changed the title Remove backport support for elasticsearch<8Cleanup code for elasticsearch<8Nov 25, 2023
Comment threadairflow/providers/elasticsearch/log/es_task_handler.py
"""
with conf_vars(
{
("elasticsearch_configs", "use_ssl"): "True",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to keep it in the tests for check that we do not use non-existed parameters

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which parameter are you worried about?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

("elasticsearch_configs", "use_ssl"): "True", This mostly for check that we remove it from resulting arguments here

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes but we no longer need it because we are on elasticsearch>7
the only reference to this parameter was in the docs which was wrong

Comment threadtests/providers/elasticsearch/log/test_es_task_handler.py Outdated
"""
with conf_vars(
{
("elasticsearch_configs", "use_ssl"): "True",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

("elasticsearch_configs", "use_ssl"): "True", This mostly for check that we remove it from resulting arguments here

@eladkal
eladkal merged commit 396b1ba into apache:mainDec 1, 2023
@eladkal
eladkal deleted the elastic branch December 1, 2023 09:28
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@eladkal@Taragolis@harishkesavarao