Remove deprecated "delegate_to" from GCP operators and hooks

[shahar1]

related: #9461, #29088

• This PR removes the deprecated delegate_to parameter from GCP operators, hooks, sensors, transfers, and triggers, as well as from firebase hook.
• It also removes the parameter from Microsoft Azure, Presto, Trino, and gsuite transfers that interact with Google Cloud.
• The delegate_to param will still be available only in gsuite and marketing platform hooks, operators, sensors, and transfers that don't interact with Google Cloud (see reasoning in Unclear documentation for the delegate_to parameter #9461).

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

[eladkal]

Need also the following:

1. Add to all relevant provider.yaml with new breaking change version (Except google as there it's already set)
2. Add entry in changelog of each provider that explains the breaking change for the specific provider. This is simply an entry of "Removed delegate_to parameter from operators x, y, z"

[shahar1]

Need also the following:

1. Add to all relevant provider.yaml with new breaking change version (Except google as there it's already set)
2. Add entry in changelog of each provider that explains the breaking change for the specific provider. This is simply an entry of "Removed delegate_to parameter from operators x, y, z"

Done

[eladkal]

This line will be added automatically by the release manager during release.
What we want to describe via the PR is information about that the commit message itself does not cover.
For example: what operators involve? what action users needs to take.. basically additional information that may help user to bridge the breaking changes faster

[shahar1]

Done

[potiuk]

I have to think about it, but this has potential of breaking much more than just one provider, i "request for changes" for now until we discuss it. More comments are coming

[potiuk]

I think we cannot just remove the parameter here, because this will break an important property of the providers we have that they can be upgraded independently.

While I see why removing such deprecated parameter is needed at some point in time doing it this way has rather profound consequences.

For example if somene would like to keep old AWS provider but would like to upgrade a google provider to the new version, this will break silently for AWS provider (which will not be upgraded) - even if delegate_to parameter has NOT been used.

The problem is that old providers are already passing this parameter even if it has not been used. It will default to None and None will passed down.

So for exmple if you have old AWS provider and new Google provider, and you do not have delegate_to parameter in your operator, the gcs_to_s3 provider will call the GCSHook with "delegate_to" parameter set to None and this will crash.

This is far too invasive. Yes. It's ok to remove that feature, but it's not ok to break another provider's usage of this hook even if the feature has not been used.

Proposal:

I am ok with removing the delegate to parameter, but this change should be done differently.

• In all Google hooks using delegate_to, remove the delegate_to parameter but add **kwargs . This will stop the Hooks from crashing even if delegate_to=None parameter has been passed to it from an older version of Amazon or other providers.

• In those hooks convert the warning into error. Smth like:

if kwargs.get('delegate_to') is not None:
raise RuntimeError("The `delegate_to` parameter has been deprecated before and finally removed in this version of Google Provider. You MUST convert it to `impersonate_chain`).

I think this should be left here pretty much indefinitely.

[shahar1]

Done, including tests for the if statements in all GCP hooks

[eladkal]

cc @potiuk

[HammadASiddiqui]

Hi!
Can someone please guide me on this discussion

googleapis/google-api-python-client#2228

[shahar1]

related: #9461, #29088

• This PR removes the deprecated delegate_to parameter from GCP operators, hooks, sensors, transfers, and triggers, as well as from firebase hook.
• It also removes the parameter from Microsoft Azure, Presto, Trino, and gsuite transfers that interact with Google Cloud.
• The delegate_to param will still be available only in gsuite and marketing platform hooks, operators, sensors, and transfers that don't interact with Google Cloud (see reasoning in Unclear documentation for the delegate_to parameter #9461).

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

[eladkal]

Need also the following:

1. Add to all relevant provider.yaml with new breaking change version (Except google as there it's already set)
2. Add entry in changelog of each provider that explains the breaking change for the specific provider. This is simply an entry of "Removed delegate_to parameter from operators x, y, z"

[shahar1]

Need also the following:

1. Add to all relevant provider.yaml with new breaking change version (Except google as there it's already set)
2. Add entry in changelog of each provider that explains the breaking change for the specific provider. This is simply an entry of "Removed delegate_to parameter from operators x, y, z"

Done

[eladkal]

This line will be added automatically by the release manager during release.
What we want to describe via the PR is information about that the commit message itself does not cover.
For example: what operators involve? what action users needs to take.. basically additional information that may help user to bridge the breaking changes faster

[shahar1]

Done

[potiuk]

I have to think about it, but this has potential of breaking much more than just one provider, i "request for changes" for now until we discuss it. More comments are coming

[potiuk]

I think we cannot just remove the parameter here, because this will break an important property of the providers we have that they can be upgraded independently.

While I see why removing such deprecated parameter is needed at some point in time doing it this way has rather profound consequences.

For example if somene would like to keep old AWS provider but would like to upgrade a google provider to the new version, this will break silently for AWS provider (which will not be upgraded) - even if delegate_to parameter has NOT been used.

The problem is that old providers are already passing this parameter even if it has not been used. It will default to None and None will passed down.

So for exmple if you have old AWS provider and new Google provider, and you do not have delegate_to parameter in your operator, the gcs_to_s3 provider will call the GCSHook with "delegate_to" parameter set to None and this will crash.

This is far too invasive. Yes. It's ok to remove that feature, but it's not ok to break another provider's usage of this hook even if the feature has not been used.

Proposal:

I am ok with removing the delegate to parameter, but this change should be done differently.

• In all Google hooks using delegate_to, remove the delegate_to parameter but add **kwargs . This will stop the Hooks from crashing even if delegate_to=None parameter has been passed to it from an older version of Amazon or other providers.

• In those hooks convert the warning into error. Smth like:

if kwargs.get('delegate_to') is not None:
raise RuntimeError("The `delegate_to` parameter has been deprecated before and finally removed in this version of Google Provider. You MUST convert it to `impersonate_chain`).

I think this should be left here pretty much indefinitely.

[shahar1]

Done, including tests for the if statements in all GCP hooks

[eladkal]

cc @potiuk

[HammadASiddiqui]

Hi!
Can someone please guide me on this discussion

googleapis/google-api-python-client#2228

[shahar1]

related: #9461, #29088

• This PR removes the deprecated delegate_to parameter from GCP operators, hooks, sensors, transfers, and triggers, as well as from firebase hook.
• It also removes the parameter from Microsoft Azure, Presto, Trino, and gsuite transfers that interact with Google Cloud.
• The delegate_to param will still be available only in gsuite and marketing platform hooks, operators, sensors, and transfers that don't interact with Google Cloud (see reasoning in Unclear documentation for the delegate_to parameter #9461).

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

[eladkal]

Need also the following:

1. Add to all relevant provider.yaml with new breaking change version (Except google as there it's already set)
2. Add entry in changelog of each provider that explains the breaking change for the specific provider. This is simply an entry of "Removed delegate_to parameter from operators x, y, z"

[shahar1]

Need also the following:

1. Add to all relevant provider.yaml with new breaking change version (Except google as there it's already set)
2. Add entry in changelog of each provider that explains the breaking change for the specific provider. This is simply an entry of "Removed delegate_to parameter from operators x, y, z"

Done

[eladkal]

This line will be added automatically by the release manager during release.
What we want to describe via the PR is information about that the commit message itself does not cover.
For example: what operators involve? what action users needs to take.. basically additional information that may help user to bridge the breaking changes faster

[shahar1]

Done

[potiuk]

I have to think about it, but this has potential of breaking much more than just one provider, i "request for changes" for now until we discuss it. More comments are coming

[potiuk]

I think we cannot just remove the parameter here, because this will break an important property of the providers we have that they can be upgraded independently.

While I see why removing such deprecated parameter is needed at some point in time doing it this way has rather profound consequences.

For example if somene would like to keep old AWS provider but would like to upgrade a google provider to the new version, this will break silently for AWS provider (which will not be upgraded) - even if delegate_to parameter has NOT been used.

The problem is that old providers are already passing this parameter even if it has not been used. It will default to None and None will passed down.

So for exmple if you have old AWS provider and new Google provider, and you do not have delegate_to parameter in your operator, the gcs_to_s3 provider will call the GCSHook with "delegate_to" parameter set to None and this will crash.

This is far too invasive. Yes. It's ok to remove that feature, but it's not ok to break another provider's usage of this hook even if the feature has not been used.

Proposal:

I am ok with removing the delegate to parameter, but this change should be done differently.

• In all Google hooks using delegate_to, remove the delegate_to parameter but add **kwargs . This will stop the Hooks from crashing even if delegate_to=None parameter has been passed to it from an older version of Amazon or other providers.

• In those hooks convert the warning into error. Smth like:

if kwargs.get('delegate_to') is not None:
raise RuntimeError("The `delegate_to` parameter has been deprecated before and finally removed in this version of Google Provider. You MUST convert it to `impersonate_chain`).

I think this should be left here pretty much indefinitely.

[shahar1]

Done, including tests for the if statements in all GCP hooks

[eladkal]

cc @potiuk

[HammadASiddiqui]

Hi!
Can someone please guide me on this discussion

googleapis/google-api-python-client#2228

[shahar1]

related: #9461, #29088

• This PR removes the deprecated delegate_to parameter from GCP operators, hooks, sensors, transfers, and triggers, as well as from firebase hook.
• It also removes the parameter from Microsoft Azure, Presto, Trino, and gsuite transfers that interact with Google Cloud.
• The delegate_to param will still be available only in gsuite and marketing platform hooks, operators, sensors, and transfers that don't interact with Google Cloud (see reasoning in Unclear documentation for the delegate_to parameter #9461).

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

[eladkal]

Need also the following:

1. Add to all relevant provider.yaml with new breaking change version (Except google as there it's already set)
2. Add entry in changelog of each provider that explains the breaking change for the specific provider. This is simply an entry of "Removed delegate_to parameter from operators x, y, z"

[shahar1]

Need also the following:

1. Add to all relevant provider.yaml with new breaking change version (Except google as there it's already set)
2. Add entry in changelog of each provider that explains the breaking change for the specific provider. This is simply an entry of "Removed delegate_to parameter from operators x, y, z"

Done

[eladkal]

This line will be added automatically by the release manager during release.
What we want to describe via the PR is information about that the commit message itself does not cover.
For example: what operators involve? what action users needs to take.. basically additional information that may help user to bridge the breaking changes faster

[shahar1]

Done

[potiuk]

I have to think about it, but this has potential of breaking much more than just one provider, i "request for changes" for now until we discuss it. More comments are coming

[potiuk]

I think we cannot just remove the parameter here, because this will break an important property of the providers we have that they can be upgraded independently.

While I see why removing such deprecated parameter is needed at some point in time doing it this way has rather profound consequences.

For example if somene would like to keep old AWS provider but would like to upgrade a google provider to the new version, this will break silently for AWS provider (which will not be upgraded) - even if delegate_to parameter has NOT been used.

The problem is that old providers are already passing this parameter even if it has not been used. It will default to None and None will passed down.

So for exmple if you have old AWS provider and new Google provider, and you do not have delegate_to parameter in your operator, the gcs_to_s3 provider will call the GCSHook with "delegate_to" parameter set to None and this will crash.

This is far too invasive. Yes. It's ok to remove that feature, but it's not ok to break another provider's usage of this hook even if the feature has not been used.

Proposal:

I am ok with removing the delegate to parameter, but this change should be done differently.

• In all Google hooks using delegate_to, remove the delegate_to parameter but add **kwargs . This will stop the Hooks from crashing even if delegate_to=None parameter has been passed to it from an older version of Amazon or other providers.

• In those hooks convert the warning into error. Smth like:

if kwargs.get('delegate_to') is not None:
raise RuntimeError("The `delegate_to` parameter has been deprecated before and finally removed in this version of Google Provider. You MUST convert it to `impersonate_chain`).

I think this should be left here pretty much indefinitely.

[shahar1]

Done, including tests for the if statements in all GCP hooks

[eladkal]

cc @potiuk

[HammadASiddiqui]

Hi!
Can someone please guide me on this discussion

googleapis/google-api-python-client#2228

[shahar1]

related: #9461, #29088

• This PR removes the deprecated delegate_to parameter from GCP operators, hooks, sensors, transfers, and triggers, as well as from firebase hook.
• It also removes the parameter from Microsoft Azure, Presto, Trino, and gsuite transfers that interact with Google Cloud.
• The delegate_to param will still be available only in gsuite and marketing platform hooks, operators, sensors, and transfers that don't interact with Google Cloud (see reasoning in Unclear documentation for the delegate_to parameter #9461).

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

[eladkal]

Need also the following:

1. Add to all relevant provider.yaml with new breaking change version (Except google as there it's already set)
2. Add entry in changelog of each provider that explains the breaking change for the specific provider. This is simply an entry of "Removed delegate_to parameter from operators x, y, z"

[shahar1]

Need also the following:

1. Add to all relevant provider.yaml with new breaking change version (Except google as there it's already set)
2. Add entry in changelog of each provider that explains the breaking change for the specific provider. This is simply an entry of "Removed delegate_to parameter from operators x, y, z"

Done

[eladkal]

This line will be added automatically by the release manager during release.
What we want to describe via the PR is information about that the commit message itself does not cover.
For example: what operators involve? what action users needs to take.. basically additional information that may help user to bridge the breaking changes faster

[shahar1]

Done

[potiuk]

I have to think about it, but this has potential of breaking much more than just one provider, i "request for changes" for now until we discuss it. More comments are coming

[potiuk]

I think we cannot just remove the parameter here, because this will break an important property of the providers we have that they can be upgraded independently.

While I see why removing such deprecated parameter is needed at some point in time doing it this way has rather profound consequences.

For example if somene would like to keep old AWS provider but would like to upgrade a google provider to the new version, this will break silently for AWS provider (which will not be upgraded) - even if delegate_to parameter has NOT been used.

The problem is that old providers are already passing this parameter even if it has not been used. It will default to None and None will passed down.

So for exmple if you have old AWS provider and new Google provider, and you do not have delegate_to parameter in your operator, the gcs_to_s3 provider will call the GCSHook with "delegate_to" parameter set to None and this will crash.

This is far too invasive. Yes. It's ok to remove that feature, but it's not ok to break another provider's usage of this hook even if the feature has not been used.

Proposal:

I am ok with removing the delegate to parameter, but this change should be done differently.

• In all Google hooks using delegate_to, remove the delegate_to parameter but add **kwargs . This will stop the Hooks from crashing even if delegate_to=None parameter has been passed to it from an older version of Amazon or other providers.

• In those hooks convert the warning into error. Smth like:

if kwargs.get('delegate_to') is not None:
raise RuntimeError("The `delegate_to` parameter has been deprecated before and finally removed in this version of Google Provider. You MUST convert it to `impersonate_chain`).

I think this should be left here pretty much indefinitely.

[shahar1]

Done, including tests for the if statements in all GCP hooks

[eladkal]

cc @potiuk

[HammadASiddiqui]

Hi!
Can someone please guide me on this discussion

googleapis/google-api-python-client#2228

[shahar1]

related: #9461, #29088

• This PR removes the deprecated delegate_to parameter from GCP operators, hooks, sensors, transfers, and triggers, as well as from firebase hook.
• It also removes the parameter from Microsoft Azure, Presto, Trino, and gsuite transfers that interact with Google Cloud.
• The delegate_to param will still be available only in gsuite and marketing platform hooks, operators, sensors, and transfers that don't interact with Google Cloud (see reasoning in Unclear documentation for the delegate_to parameter #9461).

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

[eladkal]

Need also the following:

1. Add to all relevant provider.yaml with new breaking change version (Except google as there it's already set)
2. Add entry in changelog of each provider that explains the breaking change for the specific provider. This is simply an entry of "Removed delegate_to parameter from operators x, y, z"

[shahar1]

Need also the following:

1. Add to all relevant provider.yaml with new breaking change version (Except google as there it's already set)
2. Add entry in changelog of each provider that explains the breaking change for the specific provider. This is simply an entry of "Removed delegate_to parameter from operators x, y, z"

Done

[eladkal]

This line will be added automatically by the release manager during release.
What we want to describe via the PR is information about that the commit message itself does not cover.
For example: what operators involve? what action users needs to take.. basically additional information that may help user to bridge the breaking changes faster

[shahar1]

Done

[potiuk]

I have to think about it, but this has potential of breaking much more than just one provider, i "request for changes" for now until we discuss it. More comments are coming

[potiuk]

I think we cannot just remove the parameter here, because this will break an important property of the providers we have that they can be upgraded independently.

While I see why removing such deprecated parameter is needed at some point in time doing it this way has rather profound consequences.

For example if somene would like to keep old AWS provider but would like to upgrade a google provider to the new version, this will break silently for AWS provider (which will not be upgraded) - even if delegate_to parameter has NOT been used.

The problem is that old providers are already passing this parameter even if it has not been used. It will default to None and None will passed down.

So for exmple if you have old AWS provider and new Google provider, and you do not have delegate_to parameter in your operator, the gcs_to_s3 provider will call the GCSHook with "delegate_to" parameter set to None and this will crash.

This is far too invasive. Yes. It's ok to remove that feature, but it's not ok to break another provider's usage of this hook even if the feature has not been used.

Proposal:

I am ok with removing the delegate to parameter, but this change should be done differently.

• In all Google hooks using delegate_to, remove the delegate_to parameter but add **kwargs . This will stop the Hooks from crashing even if delegate_to=None parameter has been passed to it from an older version of Amazon or other providers.

• In those hooks convert the warning into error. Smth like:

if kwargs.get('delegate_to') is not None:
raise RuntimeError("The `delegate_to` parameter has been deprecated before and finally removed in this version of Google Provider. You MUST convert it to `impersonate_chain`).

I think this should be left here pretty much indefinitely.

[shahar1]

Done, including tests for the if statements in all GCP hooks

[eladkal]

cc @potiuk

[HammadASiddiqui]

Hi!
Can someone please guide me on this discussion

googleapis/google-api-python-client#2228

[shahar1]

related: #9461, #29088

• This PR removes the deprecated delegate_to parameter from GCP operators, hooks, sensors, transfers, and triggers, as well as from firebase hook.
• It also removes the parameter from Microsoft Azure, Presto, Trino, and gsuite transfers that interact with Google Cloud.
• The delegate_to param will still be available only in gsuite and marketing platform hooks, operators, sensors, and transfers that don't interact with Google Cloud (see reasoning in Unclear documentation for the delegate_to parameter #9461).

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

[eladkal]

Need also the following:

1. Add to all relevant provider.yaml with new breaking change version (Except google as there it's already set)
2. Add entry in changelog of each provider that explains the breaking change for the specific provider. This is simply an entry of "Removed delegate_to parameter from operators x, y, z"

[shahar1]

Need also the following:

1. Add to all relevant provider.yaml with new breaking change version (Except google as there it's already set)
2. Add entry in changelog of each provider that explains the breaking change for the specific provider. This is simply an entry of "Removed delegate_to parameter from operators x, y, z"

Done

[eladkal]

This line will be added automatically by the release manager during release.
What we want to describe via the PR is information about that the commit message itself does not cover.
For example: what operators involve? what action users needs to take.. basically additional information that may help user to bridge the breaking changes faster

[shahar1]

Done

[potiuk]

I have to think about it, but this has potential of breaking much more than just one provider, i "request for changes" for now until we discuss it. More comments are coming

[potiuk]

I think we cannot just remove the parameter here, because this will break an important property of the providers we have that they can be upgraded independently.

While I see why removing such deprecated parameter is needed at some point in time doing it this way has rather profound consequences.

For example if somene would like to keep old AWS provider but would like to upgrade a google provider to the new version, this will break silently for AWS provider (which will not be upgraded) - even if delegate_to parameter has NOT been used.

The problem is that old providers are already passing this parameter even if it has not been used. It will default to None and None will passed down.

So for exmple if you have old AWS provider and new Google provider, and you do not have delegate_to parameter in your operator, the gcs_to_s3 provider will call the GCSHook with "delegate_to" parameter set to None and this will crash.

This is far too invasive. Yes. It's ok to remove that feature, but it's not ok to break another provider's usage of this hook even if the feature has not been used.

Proposal:

I am ok with removing the delegate to parameter, but this change should be done differently.

• In all Google hooks using delegate_to, remove the delegate_to parameter but add **kwargs . This will stop the Hooks from crashing even if delegate_to=None parameter has been passed to it from an older version of Amazon or other providers.

• In those hooks convert the warning into error. Smth like:

if kwargs.get('delegate_to') is not None:
raise RuntimeError("The `delegate_to` parameter has been deprecated before and finally removed in this version of Google Provider. You MUST convert it to `impersonate_chain`).

I think this should be left here pretty much indefinitely.

[shahar1]

Done, including tests for the if statements in all GCP hooks

[eladkal]

cc @potiuk

[HammadASiddiqui]

Hi!
Can someone please guide me on this discussion

googleapis/google-api-python-client#2228

[shahar1]

related: #9461, #29088

• This PR removes the deprecated delegate_to parameter from GCP operators, hooks, sensors, transfers, and triggers, as well as from firebase hook.
• It also removes the parameter from Microsoft Azure, Presto, Trino, and gsuite transfers that interact with Google Cloud.
• The delegate_to param will still be available only in gsuite and marketing platform hooks, operators, sensors, and transfers that don't interact with Google Cloud (see reasoning in Unclear documentation for the delegate_to parameter #9461).

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

[eladkal]

Need also the following:

1. Add to all relevant provider.yaml with new breaking change version (Except google as there it's already set)
2. Add entry in changelog of each provider that explains the breaking change for the specific provider. This is simply an entry of "Removed delegate_to parameter from operators x, y, z"

[shahar1]

Need also the following:

1. Add to all relevant provider.yaml with new breaking change version (Except google as there it's already set)
2. Add entry in changelog of each provider that explains the breaking change for the specific provider. This is simply an entry of "Removed delegate_to parameter from operators x, y, z"

Done

[eladkal]

This line will be added automatically by the release manager during release.
What we want to describe via the PR is information about that the commit message itself does not cover.
For example: what operators involve? what action users needs to take.. basically additional information that may help user to bridge the breaking changes faster

[shahar1]

Done

[potiuk]

I have to think about it, but this has potential of breaking much more than just one provider, i "request for changes" for now until we discuss it. More comments are coming

[potiuk]

I think we cannot just remove the parameter here, because this will break an important property of the providers we have that they can be upgraded independently.

While I see why removing such deprecated parameter is needed at some point in time doing it this way has rather profound consequences.

For example if somene would like to keep old AWS provider but would like to upgrade a google provider to the new version, this will break silently for AWS provider (which will not be upgraded) - even if delegate_to parameter has NOT been used.

The problem is that old providers are already passing this parameter even if it has not been used. It will default to None and None will passed down.

So for exmple if you have old AWS provider and new Google provider, and you do not have delegate_to parameter in your operator, the gcs_to_s3 provider will call the GCSHook with "delegate_to" parameter set to None and this will crash.

This is far too invasive. Yes. It's ok to remove that feature, but it's not ok to break another provider's usage of this hook even if the feature has not been used.

Proposal:

I am ok with removing the delegate to parameter, but this change should be done differently.

• In all Google hooks using delegate_to, remove the delegate_to parameter but add **kwargs . This will stop the Hooks from crashing even if delegate_to=None parameter has been passed to it from an older version of Amazon or other providers.

• In those hooks convert the warning into error. Smth like:

if kwargs.get('delegate_to') is not None:
raise RuntimeError("The `delegate_to` parameter has been deprecated before and finally removed in this version of Google Provider. You MUST convert it to `impersonate_chain`).

I think this should be left here pretty much indefinitely.

[shahar1]

Done, including tests for the if statements in all GCP hooks

[eladkal]

cc @potiuk

[HammadASiddiqui]

Hi!
Can someone please guide me on this discussion

googleapis/google-api-python-client#2228