Added Hook for Amazon RDS - #20642

Merged
potiuk merged 1 commit into
apache:mainfrom
kazanzhy:add_aws_rds_hooks
Feb 25, 2022
Merged

Added Hook for Amazon RDS#20642
potiuk merged 1 commit into
apache:mainfrom
kazanzhy:add_aws_rds_hooks

Conversation

@kazanzhy

@kazanzhykazanzhy commented Jan 3, 2022•
edited
Loading

Copy link
Copy Markdown
Contributor

I had two different implementations of RDSHook.

dstandish
dstandish reviewed Jan 4, 2022
View reviewed changes
Comment threadairflow/providers/amazon/aws/hooks/rds.py Outdated
@kazanzhy
kazanzhyforce-pushed the add_aws_rds_hooks branch 2 times, most recently from aec5a1c to ae53b0dCompareJanuary 5, 2022 22:14
eladkal
eladkal reviewed Jan 6, 2022
View reviewed changes
Comment threadairflow/providers/amazon/aws/hooks/rds.py Outdated
@kazanzhy
kazanzhyforce-pushed the add_aws_rds_hooks branch 3 times, most recently from 03a38e2 to d093539CompareJanuary 10, 2022 23:21
@kazanzhy
kazanzhy marked this pull request as ready for review January 11, 2022 13:40
dstandish
dstandish requested changes Jan 11, 2022
View reviewed changes
Comment threadtests/providers/amazon/aws/hooks/test_rds.py Outdated
Comment threadtests/providers/amazon/aws/hooks/test_rds.py Outdated
Comment threadtests/providers/amazon/aws/hooks/test_rds.py Outdated
Comment threadsetup.py Outdated
Comment threadtests/providers/amazon/aws/hooks/test_rds.py Outdated
Comment threadtests/providers/amazon/aws/hooks/test_rds.py Outdated
dstandish
dstandish reviewed Jan 12, 2022
View reviewed changes

@dstandishdstandish left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me apart from the one file i have question about

Comment threaddocs/apache-airflow-providers-amazon/operators/rds.rst Outdated
@github-actions

github-actionsBot commented Jan 12, 2022

Copy link
Copy Markdown
Contributor

The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.

@github-actionsgithub-actionsBot added the full tests needed We need to run full set of tests for this PR to merge label Jan 12, 2022
@kazanzhy
kazanzhyforce-pushed the add_aws_rds_hooks branch 4 times, most recently from bdd5e00 to fb2567eCompareJanuary 17, 2022 14:48
@kazanzhykazanzhy mentioned this pull request Jan 17, 2022
Merged
eladkal
eladkal requested changes Jan 20, 2022
View reviewed changes
Comment threadairflow/providers/amazon/aws/hooks/rds.py Outdated
@kazanzhy
kazanzhyforce-pushed the add_aws_rds_hooks branch 2 times, most recently from 69157ae to c5451fdCompareJanuary 24, 2022 15:04
@kazanzhy
kazanzhyforce-pushed the add_aws_rds_hooks branch 2 times, most recently from 254c8c2 to 36a6a4fCompareJanuary 29, 2022 15:03
@kazanzhy
kazanzhyforce-pushed the add_aws_rds_hooks branch 3 times, most recently from 2308501 to 2432fe2CompareFebruary 18, 2022 18:40
@dstandish

dstandish commented Feb 18, 2022

Copy link
Copy Markdown
Contributor

@potiuk i'm a bit confused why we are seeing failures like this in this pr... https://github.com/apache/airflow/runs/5252964277?check_suite_focus=true#step:11:1533
anything pop out to you re why?

@kazanzhy

kazanzhy commented Feb 18, 2022•
edited
Loading

Copy link
Copy Markdown
ContributorAuthor

So there are three ways to install stubs for RDS.

  • boto3-stubs[rds] - ERRORS
  • boto3-stubs-lite[rds] - ERRORS
  • mypy-boto3-rds - OK

Seems boto3-stubs overload something in botocore.
I'll try to find the solution because it's more convenient to put all necessary libs inside boto3-stubs than list them.

@kazanzhy
kazanzhyforce-pushed the add_aws_rds_hooks branch 2 times, most recently from 2192a54 to 645365cCompareFebruary 20, 2022 15:15
@kazanzhykazanzhy reopened this Feb 20, 2022
@dstandish

dstandish commented Feb 23, 2022

Copy link
Copy Markdown
Contributor

So there are three ways to install stubs for RDS.
boto3-stubs[rds] - ERRORS
boto3-stubs-lite[rds] - ERRORS
mypy-boto3-rds - OK
Seems boto3-stubs overload something in botocore.
I'll try to find the solution because it's more convenient to put all necessary libs inside boto3-stubs than list them.

Nice! glad you figured it out.

@dstandish
dstandishforce-pushed the add_aws_rds_hooks branch 4 times, most recently from 168cb4c to f0ef165CompareFebruary 25, 2022 06:23
@potiuk

potiuk commented Feb 25, 2022

Copy link
Copy Markdown
Member

@kazanzhy@dstandish - I would really love if you could upvote and comment here: pypa/pip#10258 (comment).

The problem is that we have no way currently to determine what is the root cause of the problem. The problem is that pip is not able to solve dependencies in a reasonable time and runs backtracking for hours - but it does not really give us the possibilty to get the root cause message that could allow us to determine how we could help pip.

I proposed to add optional timeout on pip install execution producing a "conflict" message if the timeout is reached. That could give us at least a chance to understand what is the problem (someone released a new dependency that makes pip fail installing airflow from the scratch and we need to help pip to determine the right set of dependencies because it is unable to figure it out on its own).

@potiuk

potiuk commented Feb 25, 2022

Copy link
Copy Markdown
Member

@kazanzhy@dstandish I think I managed to workaround the pip resolver issue with #21824 - please rebase to latest main.

@dstandish

dstandish commented Feb 25, 2022

Copy link
Copy Markdown
Contributor

I would really love if you could upvote and comment here:

done

@kazanzhy@dstandish I think I managed to workaround the pip resolver issue with #21824 - please rebase to latest main.

thanks a bunch!

@dstandish

dstandish commented Feb 25, 2022

Copy link
Copy Markdown
Contributor

finally we can merge. thanks again to @potiuk for resolving the pip problem, and thank you @kazanzhy for your persistence and patience.

@potiuk
potiuk merged commit 0378659 into apache:mainFeb 25, 2022
@kazanzhy
kazanzhy deleted the add_aws_rds_hooks branch February 28, 2022 14:38
@jedcunninghamjedcunningham added the changelog:skip Changes that should be skipped from the changelog (CI, tests, etc..) label Feb 28, 2022
@potiukpotiuk mentioned this pull request Mar 7, 2022
Closed
84 tasks
@kazanzhykazanzhy mentioned this pull request Mar 9, 2022
Merged
@potiukpotiuk mentioned this pull request Mar 15, 2022
Closed
41 tasks
@kazanzhykazanzhy mentioned this pull request Aug 21, 2022
Merged
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dstandishdstandishdstandish approved these changes
@potiukpotiukpotiuk approved these changes
@eladkaleladkaleladkal approved these changes
@mik-lajmik-lajAwaiting requested review from mik-laj
@vikramkokavikramkokaAwaiting requested review from vikramkoka

Assignees

No one assigned

Labels

area:providerschangelog:skipChanges that should be skipped from the changelog (CI, tests, etc..)full tests neededWe need to run full set of tests for this PR to mergekind:documentationprovider:amazonAWS/Amazon - related issues

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@kazanzhy@potiuk@dstandish@eladkal@jedcunningham
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
Added Hook for Amazon RDS - #20642
Merged
potiuk merged 1 commit into
apache:mainfrom 
kazanzhy:add_aws_rds_hooks
Feb 25, 2022
Merged
Added Hook for Amazon RDS#20642
potiuk merged 1 commit into
apache:mainfrom 
kazanzhy:add_aws_rds_hooks
Conversation
@kazanzhy
@kazanzhykazanzhy
commented
Jan 3, 2022•

edited

Loading

Copy link


Copy Markdown

Contributor
I had two different implementations of RDSHook.
dstandish
dstandish
reviewed
Jan 4, 2022
View reviewed changes
Comment threadairflow/providers/amazon/aws/hooks/rds.py
Outdated

@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
2 times, most recently
from
aec5a1c to
ae53b0dCompareJanuary 5, 2022 22:14
eladkal
eladkal
reviewed
Jan 6, 2022
View reviewed changes
Comment threadairflow/providers/amazon/aws/hooks/rds.py
Outdated

@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
3 times, most recently
from
03a38e2 to
d093539CompareJanuary 10, 2022 23:21
@kazanzhy
kazanzhy
marked this pull request as ready for review
January 11, 2022 13:40
dstandish
dstandish
requested changes
Jan 11, 2022
View reviewed changes
Comment threadtests/providers/amazon/aws/hooks/test_rds.py
Outdated

Comment threadtests/providers/amazon/aws/hooks/test_rds.py
Outdated

Comment threadtests/providers/amazon/aws/hooks/test_rds.py
Outdated

Comment threadsetup.py
Outdated

Comment threadtests/providers/amazon/aws/hooks/test_rds.py
Outdated

Comment threadtests/providers/amazon/aws/hooks/test_rds.py
Outdated

dstandish
dstandish
reviewed
Jan 12, 2022
View reviewed changes
@dstandishdstandish
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me apart from the one file i have question about
Comment threaddocs/apache-airflow-providers-amazon/operators/rds.rst
Outdated

@github-actions
github-actionsBot
commented
Jan 12, 2022

Copy link


Copy Markdown

Contributor
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
@github-actionsgithub-actionsBot
added
the 
full tests needed
We need to run full set of tests for this PR to merge
label
Jan 12, 2022
@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
4 times, most recently
from
bdd5e00 to
fb2567eCompareJanuary 17, 2022 14:48
@kazanzhykazanzhy
mentioned this pull request
Jan 17, 2022
 Merged

eladkal
eladkal
requested changes
Jan 20, 2022
View reviewed changes
Comment threadairflow/providers/amazon/aws/hooks/rds.py
Outdated

@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
2 times, most recently
from
69157ae to
c5451fdCompareJanuary 24, 2022 15:04
@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
2 times, most recently
from
254c8c2 to
36a6a4fCompareJanuary 29, 2022 15:03
@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
3 times, most recently
from
2308501 to
2432fe2CompareFebruary 18, 2022 18:40
@dstandish
dstandish
commented
Feb 18, 2022

Copy link


Copy Markdown

Contributor
@potiuk i'm a bit confused why we are seeing failures like this in this pr... https://github.com/apache/airflow/runs/5252964277?check_suite_focus=true#step:11:1533

anything pop out to you re why?
@kazanzhy
kazanzhy
commented
Feb 18, 2022•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
So there are three ways to install stubs for RDS.
  • boto3-stubs[rds] - ERRORS
  • boto3-stubs-lite[rds] - ERRORS
  • mypy-boto3-rds - OK
Seems boto3-stubs overload something in botocore.

I'll try to find the solution because it's more convenient to put all necessary libs inside boto3-stubs than list them.
@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
2 times, most recently
from
2192a54 to
645365cCompareFebruary 20, 2022 15:15
@kazanzhykazanzhy
reopened this
Feb 20, 2022
@dstandish
dstandish
commented
Feb 23, 2022

Copy link


Copy Markdown

Contributor
So there are three ways to install stubs for RDS.

boto3-stubs[rds] - ERRORS

boto3-stubs-lite[rds] - ERRORS

mypy-boto3-rds - OK

Seems boto3-stubs overload something in botocore.

I'll try to find the solution because it's more convenient to put all necessary libs inside boto3-stubs than list them.
Nice! glad you figured it out.
@dstandish
dstandishforce-pushed
the
add_aws_rds_hooks
branch
4 times, most recently
from
168cb4c to
f0ef165CompareFebruary 25, 2022 06:23
@potiuk
potiuk
commented
Feb 25, 2022

Copy link


Copy Markdown

Member
@kazanzhy@dstandish - I would really love if you could upvote and comment here: pypa/pip#10258 (comment).
The problem is that we have no way currently to determine what is the root cause of the problem. The problem is that pip is not able to solve dependencies in a reasonable time and runs backtracking for hours - but it does not really give us the possibilty to get the root cause message that could allow us to determine how we could help pip.
I proposed to add optional timeout on pip install execution producing a "conflict" message if the timeout is reached. That could give us at least a chance to understand what is the problem (someone released a new dependency that makes pip fail installing airflow from the scratch and we need to help pip to determine the right set of dependencies because it is unable to figure it out on its own).
@potiuk
potiuk
commented
Feb 25, 2022

Copy link


Copy Markdown

Member
@kazanzhy@dstandish I think I managed to workaround the pip resolver issue with #21824 - please rebase to latest main.
@dstandish
dstandish
commented
Feb 25, 2022

Copy link


Copy Markdown

Contributor
I would really love if you could upvote and comment here:
done
@kazanzhy@dstandish I think I managed to workaround the pip resolver issue with #21824 - please rebase to latest main.
thanks a bunch!
@dstandish
dstandish
commented
Feb 25, 2022

Copy link


Copy Markdown

Contributor
finally we can merge. thanks again to @potiuk for resolving the pip problem, and thank you @kazanzhy for your persistence and patience.
@potiuk
potiuk
merged commit 0378659
into
apache:mainFeb 25, 2022
@kazanzhy
kazanzhy
deleted the

add_aws_rds_hooks

branch
February 28, 2022 14:38
@jedcunninghamjedcunningham
added
the 
changelog:skip
Changes that should be skipped from the changelog (CI, tests, etc..)
label
Feb 28, 2022
@potiukpotiuk
mentioned this pull request
Mar 7, 2022

Closed

84 tasks
@kazanzhykazanzhy
mentioned this pull request
Mar 9, 2022
 Merged

@potiukpotiuk
mentioned this pull request
Mar 15, 2022

Closed

41 tasks
@kazanzhykazanzhy
mentioned this pull request
Aug 21, 2022
 Merged

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@dstandishdstandishdstandish approved these changes
@potiukpotiukpotiuk approved these changes
@eladkaleladkaleladkal approved these changes
@mik-lajmik-lajAwaiting requested review from mik-laj
@vikramkokavikramkokaAwaiting requested review from vikramkoka

Assignees


No one assigned


Labels

area:providerschangelog:skipChanges that should be skipped from the changelog (CI, tests, etc..)full tests neededWe need to run full set of tests for this PR to mergekind:documentationprovider:amazonAWS/Amazon - related issues

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

5 participants

@kazanzhy@potiuk@dstandish@eladkal@jedcunningham

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Added Hook for Amazon RDS - #20642
Merged
potiuk merged 1 commit into
apache:mainfrom 
kazanzhy:add_aws_rds_hooks
Feb 25, 2022
Merged
Added Hook for Amazon RDS#20642
potiuk merged 1 commit into
apache:mainfrom 
kazanzhy:add_aws_rds_hooks
Conversation
@kazanzhy
@kazanzhykazanzhy
commented
Jan 3, 2022•

edited

Loading

Copy link


Copy Markdown

Contributor
I had two different implementations of RDSHook.
dstandish
dstandish
reviewed
Jan 4, 2022
View reviewed changes
Comment threadairflow/providers/amazon/aws/hooks/rds.py
Outdated

@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
2 times, most recently
from
aec5a1c to
ae53b0dCompareJanuary 5, 2022 22:14
eladkal
eladkal
reviewed
Jan 6, 2022
View reviewed changes
Comment threadairflow/providers/amazon/aws/hooks/rds.py
Outdated

@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
3 times, most recently
from
03a38e2 to
d093539CompareJanuary 10, 2022 23:21
@kazanzhy
kazanzhy
marked this pull request as ready for review
January 11, 2022 13:40
dstandish
dstandish
requested changes
Jan 11, 2022
View reviewed changes
Comment threadtests/providers/amazon/aws/hooks/test_rds.py
Outdated

Comment threadtests/providers/amazon/aws/hooks/test_rds.py
Outdated

Comment threadtests/providers/amazon/aws/hooks/test_rds.py
Outdated

Comment threadsetup.py
Outdated

Comment threadtests/providers/amazon/aws/hooks/test_rds.py
Outdated

Comment threadtests/providers/amazon/aws/hooks/test_rds.py
Outdated

dstandish
dstandish
reviewed
Jan 12, 2022
View reviewed changes
@dstandishdstandish
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me apart from the one file i have question about
Comment threaddocs/apache-airflow-providers-amazon/operators/rds.rst
Outdated

@github-actions
github-actionsBot
commented
Jan 12, 2022

Copy link


Copy Markdown

Contributor
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
@github-actionsgithub-actionsBot
added
the 
full tests needed
We need to run full set of tests for this PR to merge
label
Jan 12, 2022
@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
4 times, most recently
from
bdd5e00 to
fb2567eCompareJanuary 17, 2022 14:48
@kazanzhykazanzhy
mentioned this pull request
Jan 17, 2022
 Merged

eladkal
eladkal
requested changes
Jan 20, 2022
View reviewed changes
Comment threadairflow/providers/amazon/aws/hooks/rds.py
Outdated

@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
2 times, most recently
from
69157ae to
c5451fdCompareJanuary 24, 2022 15:04
@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
2 times, most recently
from
254c8c2 to
36a6a4fCompareJanuary 29, 2022 15:03
@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
3 times, most recently
from
2308501 to
2432fe2CompareFebruary 18, 2022 18:40
@dstandish
dstandish
commented
Feb 18, 2022

Copy link


Copy Markdown

Contributor
@potiuk i'm a bit confused why we are seeing failures like this in this pr... https://github.com/apache/airflow/runs/5252964277?check_suite_focus=true#step:11:1533

anything pop out to you re why?
@kazanzhy
kazanzhy
commented
Feb 18, 2022•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
So there are three ways to install stubs for RDS.
  • boto3-stubs[rds] - ERRORS
  • boto3-stubs-lite[rds] - ERRORS
  • mypy-boto3-rds - OK
Seems boto3-stubs overload something in botocore.

I'll try to find the solution because it's more convenient to put all necessary libs inside boto3-stubs than list them.
@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
2 times, most recently
from
2192a54 to
645365cCompareFebruary 20, 2022 15:15
@kazanzhykazanzhy
reopened this
Feb 20, 2022
@dstandish
dstandish
commented
Feb 23, 2022

Copy link


Copy Markdown

Contributor
So there are three ways to install stubs for RDS.

boto3-stubs[rds] - ERRORS

boto3-stubs-lite[rds] - ERRORS

mypy-boto3-rds - OK

Seems boto3-stubs overload something in botocore.

I'll try to find the solution because it's more convenient to put all necessary libs inside boto3-stubs than list them.
Nice! glad you figured it out.
@dstandish
dstandishforce-pushed
the
add_aws_rds_hooks
branch
4 times, most recently
from
168cb4c to
f0ef165CompareFebruary 25, 2022 06:23
@potiuk
potiuk
commented
Feb 25, 2022

Copy link


Copy Markdown

Member
@kazanzhy@dstandish - I would really love if you could upvote and comment here: pypa/pip#10258 (comment).
The problem is that we have no way currently to determine what is the root cause of the problem. The problem is that pip is not able to solve dependencies in a reasonable time and runs backtracking for hours - but it does not really give us the possibilty to get the root cause message that could allow us to determine how we could help pip.
I proposed to add optional timeout on pip install execution producing a "conflict" message if the timeout is reached. That could give us at least a chance to understand what is the problem (someone released a new dependency that makes pip fail installing airflow from the scratch and we need to help pip to determine the right set of dependencies because it is unable to figure it out on its own).
@potiuk
potiuk
commented
Feb 25, 2022

Copy link


Copy Markdown

Member
@kazanzhy@dstandish I think I managed to workaround the pip resolver issue with #21824 - please rebase to latest main.
@dstandish
dstandish
commented
Feb 25, 2022

Copy link


Copy Markdown

Contributor
I would really love if you could upvote and comment here:
done
@kazanzhy@dstandish I think I managed to workaround the pip resolver issue with #21824 - please rebase to latest main.
thanks a bunch!
@dstandish
dstandish
commented
Feb 25, 2022

Copy link


Copy Markdown

Contributor
finally we can merge. thanks again to @potiuk for resolving the pip problem, and thank you @kazanzhy for your persistence and patience.
@potiuk
potiuk
merged commit 0378659
into
apache:mainFeb 25, 2022
@kazanzhy
kazanzhy
deleted the

add_aws_rds_hooks

branch
February 28, 2022 14:38
@jedcunninghamjedcunningham
added
the 
changelog:skip
Changes that should be skipped from the changelog (CI, tests, etc..)
label
Feb 28, 2022
@potiukpotiuk
mentioned this pull request
Mar 7, 2022

Closed

84 tasks
@kazanzhykazanzhy
mentioned this pull request
Mar 9, 2022
 Merged

@potiukpotiuk
mentioned this pull request
Mar 15, 2022

Closed

41 tasks
@kazanzhykazanzhy
mentioned this pull request
Aug 21, 2022
 Merged

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@dstandishdstandishdstandish approved these changes
@potiukpotiukpotiuk approved these changes
@eladkaleladkaleladkal approved these changes
@mik-lajmik-lajAwaiting requested review from mik-laj
@vikramkokavikramkokaAwaiting requested review from vikramkoka

Assignees


No one assigned


Labels

area:providerschangelog:skipChanges that should be skipped from the changelog (CI, tests, etc..)full tests neededWe need to run full set of tests for this PR to mergekind:documentationprovider:amazonAWS/Amazon - related issues

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

5 participants

@kazanzhy@potiuk@dstandish@eladkal@jedcunningham

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Added Hook for Amazon RDS - #20642
Merged
potiuk merged 1 commit into
apache:mainfrom 
kazanzhy:add_aws_rds_hooks
Feb 25, 2022
Merged
Added Hook for Amazon RDS#20642
potiuk merged 1 commit into
apache:mainfrom 
kazanzhy:add_aws_rds_hooks
Conversation
@kazanzhy
@kazanzhykazanzhy
commented
Jan 3, 2022•

edited

Loading

Copy link


Copy Markdown

Contributor
I had two different implementations of RDSHook.
dstandish
dstandish
reviewed
Jan 4, 2022
View reviewed changes
Comment threadairflow/providers/amazon/aws/hooks/rds.py
Outdated

@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
2 times, most recently
from
aec5a1c to
ae53b0dCompareJanuary 5, 2022 22:14
eladkal
eladkal
reviewed
Jan 6, 2022
View reviewed changes
Comment threadairflow/providers/amazon/aws/hooks/rds.py
Outdated

@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
3 times, most recently
from
03a38e2 to
d093539CompareJanuary 10, 2022 23:21
@kazanzhy
kazanzhy
marked this pull request as ready for review
January 11, 2022 13:40
dstandish
dstandish
requested changes
Jan 11, 2022
View reviewed changes
Comment threadtests/providers/amazon/aws/hooks/test_rds.py
Outdated

Comment threadtests/providers/amazon/aws/hooks/test_rds.py
Outdated

Comment threadtests/providers/amazon/aws/hooks/test_rds.py
Outdated

Comment threadsetup.py
Outdated

Comment threadtests/providers/amazon/aws/hooks/test_rds.py
Outdated

Comment threadtests/providers/amazon/aws/hooks/test_rds.py
Outdated

dstandish
dstandish
reviewed
Jan 12, 2022
View reviewed changes
@dstandishdstandish
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me apart from the one file i have question about
Comment threaddocs/apache-airflow-providers-amazon/operators/rds.rst
Outdated

@github-actions
github-actionsBot
commented
Jan 12, 2022

Copy link


Copy Markdown

Contributor
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
@github-actionsgithub-actionsBot
added
the 
full tests needed
We need to run full set of tests for this PR to merge
label
Jan 12, 2022
@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
4 times, most recently
from
bdd5e00 to
fb2567eCompareJanuary 17, 2022 14:48
@kazanzhykazanzhy
mentioned this pull request
Jan 17, 2022
 Merged

eladkal
eladkal
requested changes
Jan 20, 2022
View reviewed changes
Comment threadairflow/providers/amazon/aws/hooks/rds.py
Outdated

@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
2 times, most recently
from
69157ae to
c5451fdCompareJanuary 24, 2022 15:04
@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
2 times, most recently
from
254c8c2 to
36a6a4fCompareJanuary 29, 2022 15:03
@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
3 times, most recently
from
2308501 to
2432fe2CompareFebruary 18, 2022 18:40
@dstandish
dstandish
commented
Feb 18, 2022

Copy link


Copy Markdown

Contributor
@potiuk i'm a bit confused why we are seeing failures like this in this pr... https://github.com/apache/airflow/runs/5252964277?check_suite_focus=true#step:11:1533

anything pop out to you re why?
@kazanzhy
kazanzhy
commented
Feb 18, 2022•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
So there are three ways to install stubs for RDS.
  • boto3-stubs[rds] - ERRORS
  • boto3-stubs-lite[rds] - ERRORS
  • mypy-boto3-rds - OK
Seems boto3-stubs overload something in botocore.

I'll try to find the solution because it's more convenient to put all necessary libs inside boto3-stubs than list them.
@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
2 times, most recently
from
2192a54 to
645365cCompareFebruary 20, 2022 15:15
@kazanzhykazanzhy
reopened this
Feb 20, 2022
@dstandish
dstandish
commented
Feb 23, 2022

Copy link


Copy Markdown

Contributor
So there are three ways to install stubs for RDS.

boto3-stubs[rds] - ERRORS

boto3-stubs-lite[rds] - ERRORS

mypy-boto3-rds - OK

Seems boto3-stubs overload something in botocore.

I'll try to find the solution because it's more convenient to put all necessary libs inside boto3-stubs than list them.
Nice! glad you figured it out.
@dstandish
dstandishforce-pushed
the
add_aws_rds_hooks
branch
4 times, most recently
from
168cb4c to
f0ef165CompareFebruary 25, 2022 06:23
@potiuk
potiuk
commented
Feb 25, 2022

Copy link


Copy Markdown

Member
@kazanzhy@dstandish - I would really love if you could upvote and comment here: pypa/pip#10258 (comment).
The problem is that we have no way currently to determine what is the root cause of the problem. The problem is that pip is not able to solve dependencies in a reasonable time and runs backtracking for hours - but it does not really give us the possibilty to get the root cause message that could allow us to determine how we could help pip.
I proposed to add optional timeout on pip install execution producing a "conflict" message if the timeout is reached. That could give us at least a chance to understand what is the problem (someone released a new dependency that makes pip fail installing airflow from the scratch and we need to help pip to determine the right set of dependencies because it is unable to figure it out on its own).
@potiuk
potiuk
commented
Feb 25, 2022

Copy link


Copy Markdown

Member
@kazanzhy@dstandish I think I managed to workaround the pip resolver issue with #21824 - please rebase to latest main.
@dstandish
dstandish
commented
Feb 25, 2022

Copy link


Copy Markdown

Contributor
I would really love if you could upvote and comment here:
done
@kazanzhy@dstandish I think I managed to workaround the pip resolver issue with #21824 - please rebase to latest main.
thanks a bunch!
@dstandish
dstandish
commented
Feb 25, 2022

Copy link


Copy Markdown

Contributor
finally we can merge. thanks again to @potiuk for resolving the pip problem, and thank you @kazanzhy for your persistence and patience.
@potiuk
potiuk
merged commit 0378659
into
apache:mainFeb 25, 2022
@kazanzhy
kazanzhy
deleted the

add_aws_rds_hooks

branch
February 28, 2022 14:38
@jedcunninghamjedcunningham
added
the 
changelog:skip
Changes that should be skipped from the changelog (CI, tests, etc..)
label
Feb 28, 2022
@potiukpotiuk
mentioned this pull request
Mar 7, 2022

Closed

84 tasks
@kazanzhykazanzhy
mentioned this pull request
Mar 9, 2022
 Merged

@potiukpotiuk
mentioned this pull request
Mar 15, 2022

Closed

41 tasks
@kazanzhykazanzhy
mentioned this pull request
Aug 21, 2022
 Merged

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@dstandishdstandishdstandish approved these changes
@potiukpotiukpotiuk approved these changes
@eladkaleladkaleladkal approved these changes
@mik-lajmik-lajAwaiting requested review from mik-laj
@vikramkokavikramkokaAwaiting requested review from vikramkoka

Assignees


No one assigned


Labels

area:providerschangelog:skipChanges that should be skipped from the changelog (CI, tests, etc..)full tests neededWe need to run full set of tests for this PR to mergekind:documentationprovider:amazonAWS/Amazon - related issues

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

5 participants

@kazanzhy@potiuk@dstandish@eladkal@jedcunningham

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
Added Hook for Amazon RDS - #20642
Merged
potiuk merged 1 commit into
apache:mainfrom 
kazanzhy:add_aws_rds_hooks
Feb 25, 2022
Merged
Added Hook for Amazon RDS#20642
potiuk merged 1 commit into
apache:mainfrom 
kazanzhy:add_aws_rds_hooks
Conversation
@kazanzhy
@kazanzhykazanzhy
commented
Jan 3, 2022•

edited

Loading

Copy link


Copy Markdown

Contributor
I had two different implementations of RDSHook.
dstandish
dstandish
reviewed
Jan 4, 2022
View reviewed changes
Comment threadairflow/providers/amazon/aws/hooks/rds.py
Outdated

@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
2 times, most recently
from
aec5a1c to
ae53b0dCompareJanuary 5, 2022 22:14
eladkal
eladkal
reviewed
Jan 6, 2022
View reviewed changes
Comment threadairflow/providers/amazon/aws/hooks/rds.py
Outdated

@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
3 times, most recently
from
03a38e2 to
d093539CompareJanuary 10, 2022 23:21
@kazanzhy
kazanzhy
marked this pull request as ready for review
January 11, 2022 13:40
dstandish
dstandish
requested changes
Jan 11, 2022
View reviewed changes
Comment threadtests/providers/amazon/aws/hooks/test_rds.py
Outdated

Comment threadtests/providers/amazon/aws/hooks/test_rds.py
Outdated

Comment threadtests/providers/amazon/aws/hooks/test_rds.py
Outdated

Comment threadsetup.py
Outdated

Comment threadtests/providers/amazon/aws/hooks/test_rds.py
Outdated

Comment threadtests/providers/amazon/aws/hooks/test_rds.py
Outdated

dstandish
dstandish
reviewed
Jan 12, 2022
View reviewed changes
@dstandishdstandish
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me apart from the one file i have question about
Comment threaddocs/apache-airflow-providers-amazon/operators/rds.rst
Outdated

@github-actions
github-actionsBot
commented
Jan 12, 2022

Copy link


Copy Markdown

Contributor
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
@github-actionsgithub-actionsBot
added
the 
full tests needed
We need to run full set of tests for this PR to merge
label
Jan 12, 2022
@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
4 times, most recently
from
bdd5e00 to
fb2567eCompareJanuary 17, 2022 14:48
@kazanzhykazanzhy
mentioned this pull request
Jan 17, 2022
 Merged

eladkal
eladkal
requested changes
Jan 20, 2022
View reviewed changes
Comment threadairflow/providers/amazon/aws/hooks/rds.py
Outdated

@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
2 times, most recently
from
69157ae to
c5451fdCompareJanuary 24, 2022 15:04
@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
2 times, most recently
from
254c8c2 to
36a6a4fCompareJanuary 29, 2022 15:03
@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
3 times, most recently
from
2308501 to
2432fe2CompareFebruary 18, 2022 18:40
@dstandish
dstandish
commented
Feb 18, 2022

Copy link


Copy Markdown

Contributor
@potiuk i'm a bit confused why we are seeing failures like this in this pr... https://github.com/apache/airflow/runs/5252964277?check_suite_focus=true#step:11:1533

anything pop out to you re why?
@kazanzhy
kazanzhy
commented
Feb 18, 2022•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
So there are three ways to install stubs for RDS.
  • boto3-stubs[rds] - ERRORS
  • boto3-stubs-lite[rds] - ERRORS
  • mypy-boto3-rds - OK
Seems boto3-stubs overload something in botocore.

I'll try to find the solution because it's more convenient to put all necessary libs inside boto3-stubs than list them.
@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
2 times, most recently
from
2192a54 to
645365cCompareFebruary 20, 2022 15:15
@kazanzhykazanzhy
reopened this
Feb 20, 2022
@dstandish
dstandish
commented
Feb 23, 2022

Copy link


Copy Markdown

Contributor
So there are three ways to install stubs for RDS.

boto3-stubs[rds] - ERRORS

boto3-stubs-lite[rds] - ERRORS

mypy-boto3-rds - OK

Seems boto3-stubs overload something in botocore.

I'll try to find the solution because it's more convenient to put all necessary libs inside boto3-stubs than list them.
Nice! glad you figured it out.
@dstandish
dstandishforce-pushed
the
add_aws_rds_hooks
branch
4 times, most recently
from
168cb4c to
f0ef165CompareFebruary 25, 2022 06:23
@potiuk
potiuk
commented
Feb 25, 2022

Copy link


Copy Markdown

Member
@kazanzhy@dstandish - I would really love if you could upvote and comment here: pypa/pip#10258 (comment).
The problem is that we have no way currently to determine what is the root cause of the problem. The problem is that pip is not able to solve dependencies in a reasonable time and runs backtracking for hours - but it does not really give us the possibilty to get the root cause message that could allow us to determine how we could help pip.
I proposed to add optional timeout on pip install execution producing a "conflict" message if the timeout is reached. That could give us at least a chance to understand what is the problem (someone released a new dependency that makes pip fail installing airflow from the scratch and we need to help pip to determine the right set of dependencies because it is unable to figure it out on its own).
@potiuk
potiuk
commented
Feb 25, 2022

Copy link


Copy Markdown

Member
@kazanzhy@dstandish I think I managed to workaround the pip resolver issue with #21824 - please rebase to latest main.
@dstandish
dstandish
commented
Feb 25, 2022

Copy link


Copy Markdown

Contributor
I would really love if you could upvote and comment here:
done
@kazanzhy@dstandish I think I managed to workaround the pip resolver issue with #21824 - please rebase to latest main.
thanks a bunch!
@dstandish
dstandish
commented
Feb 25, 2022

Copy link


Copy Markdown

Contributor
finally we can merge. thanks again to @potiuk for resolving the pip problem, and thank you @kazanzhy for your persistence and patience.
@potiuk
potiuk
merged commit 0378659
into
apache:mainFeb 25, 2022
@kazanzhy
kazanzhy
deleted the

add_aws_rds_hooks

branch
February 28, 2022 14:38
@jedcunninghamjedcunningham
added
the 
changelog:skip
Changes that should be skipped from the changelog (CI, tests, etc..)
label
Feb 28, 2022
@potiukpotiuk
mentioned this pull request
Mar 7, 2022

Closed

84 tasks
@kazanzhykazanzhy
mentioned this pull request
Mar 9, 2022
 Merged

@potiukpotiuk
mentioned this pull request
Mar 15, 2022

Closed

41 tasks
@kazanzhykazanzhy
mentioned this pull request
Aug 21, 2022
 Merged

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@dstandishdstandishdstandish approved these changes
@potiukpotiukpotiuk approved these changes
@eladkaleladkaleladkal approved these changes
@mik-lajmik-lajAwaiting requested review from mik-laj
@vikramkokavikramkokaAwaiting requested review from vikramkoka

Assignees


No one assigned


Labels

area:providerschangelog:skipChanges that should be skipped from the changelog (CI, tests, etc..)full tests neededWe need to run full set of tests for this PR to mergekind:documentationprovider:amazonAWS/Amazon - related issues

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

5 participants

@kazanzhy@potiuk@dstandish@eladkal@jedcunningham

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Added Hook for Amazon RDS - #20642
Merged
potiuk merged 1 commit into
apache:mainfrom 
kazanzhy:add_aws_rds_hooks
Feb 25, 2022
Merged
Added Hook for Amazon RDS#20642
potiuk merged 1 commit into
apache:mainfrom 
kazanzhy:add_aws_rds_hooks
Conversation
@kazanzhy
@kazanzhykazanzhy
commented
Jan 3, 2022•

edited

Loading

Copy link


Copy Markdown

Contributor
I had two different implementations of RDSHook.
dstandish
dstandish
reviewed
Jan 4, 2022
View reviewed changes
Comment threadairflow/providers/amazon/aws/hooks/rds.py
Outdated

@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
2 times, most recently
from
aec5a1c to
ae53b0dCompareJanuary 5, 2022 22:14
eladkal
eladkal
reviewed
Jan 6, 2022
View reviewed changes
Comment threadairflow/providers/amazon/aws/hooks/rds.py
Outdated

@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
3 times, most recently
from
03a38e2 to
d093539CompareJanuary 10, 2022 23:21
@kazanzhy
kazanzhy
marked this pull request as ready for review
January 11, 2022 13:40
dstandish
dstandish
requested changes
Jan 11, 2022
View reviewed changes
Comment threadtests/providers/amazon/aws/hooks/test_rds.py
Outdated

Comment threadtests/providers/amazon/aws/hooks/test_rds.py
Outdated

Comment threadtests/providers/amazon/aws/hooks/test_rds.py
Outdated

Comment threadsetup.py
Outdated

Comment threadtests/providers/amazon/aws/hooks/test_rds.py
Outdated

Comment threadtests/providers/amazon/aws/hooks/test_rds.py
Outdated

dstandish
dstandish
reviewed
Jan 12, 2022
View reviewed changes
@dstandishdstandish
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me apart from the one file i have question about
Comment threaddocs/apache-airflow-providers-amazon/operators/rds.rst
Outdated

@github-actions
github-actionsBot
commented
Jan 12, 2022

Copy link


Copy Markdown

Contributor
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
@github-actionsgithub-actionsBot
added
the 
full tests needed
We need to run full set of tests for this PR to merge
label
Jan 12, 2022
@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
4 times, most recently
from
bdd5e00 to
fb2567eCompareJanuary 17, 2022 14:48
@kazanzhykazanzhy
mentioned this pull request
Jan 17, 2022
 Merged

eladkal
eladkal
requested changes
Jan 20, 2022
View reviewed changes
Comment threadairflow/providers/amazon/aws/hooks/rds.py
Outdated

@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
2 times, most recently
from
69157ae to
c5451fdCompareJanuary 24, 2022 15:04
@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
2 times, most recently
from
254c8c2 to
36a6a4fCompareJanuary 29, 2022 15:03
@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
3 times, most recently
from
2308501 to
2432fe2CompareFebruary 18, 2022 18:40
@dstandish
dstandish
commented
Feb 18, 2022

Copy link


Copy Markdown

Contributor
@potiuk i'm a bit confused why we are seeing failures like this in this pr... https://github.com/apache/airflow/runs/5252964277?check_suite_focus=true#step:11:1533

anything pop out to you re why?
@kazanzhy
kazanzhy
commented
Feb 18, 2022•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
So there are three ways to install stubs for RDS.
  • boto3-stubs[rds] - ERRORS
  • boto3-stubs-lite[rds] - ERRORS
  • mypy-boto3-rds - OK
Seems boto3-stubs overload something in botocore.

I'll try to find the solution because it's more convenient to put all necessary libs inside boto3-stubs than list them.
@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
2 times, most recently
from
2192a54 to
645365cCompareFebruary 20, 2022 15:15
@kazanzhykazanzhy
reopened this
Feb 20, 2022
@dstandish
dstandish
commented
Feb 23, 2022

Copy link


Copy Markdown

Contributor
So there are three ways to install stubs for RDS.

boto3-stubs[rds] - ERRORS

boto3-stubs-lite[rds] - ERRORS

mypy-boto3-rds - OK

Seems boto3-stubs overload something in botocore.

I'll try to find the solution because it's more convenient to put all necessary libs inside boto3-stubs than list them.
Nice! glad you figured it out.
@dstandish
dstandishforce-pushed
the
add_aws_rds_hooks
branch
4 times, most recently
from
168cb4c to
f0ef165CompareFebruary 25, 2022 06:23
@potiuk
potiuk
commented
Feb 25, 2022

Copy link


Copy Markdown

Member
@kazanzhy@dstandish - I would really love if you could upvote and comment here: pypa/pip#10258 (comment).
The problem is that we have no way currently to determine what is the root cause of the problem. The problem is that pip is not able to solve dependencies in a reasonable time and runs backtracking for hours - but it does not really give us the possibilty to get the root cause message that could allow us to determine how we could help pip.
I proposed to add optional timeout on pip install execution producing a "conflict" message if the timeout is reached. That could give us at least a chance to understand what is the problem (someone released a new dependency that makes pip fail installing airflow from the scratch and we need to help pip to determine the right set of dependencies because it is unable to figure it out on its own).
@potiuk
potiuk
commented
Feb 25, 2022

Copy link


Copy Markdown

Member
@kazanzhy@dstandish I think I managed to workaround the pip resolver issue with #21824 - please rebase to latest main.
@dstandish
dstandish
commented
Feb 25, 2022

Copy link


Copy Markdown

Contributor
I would really love if you could upvote and comment here:
done
@kazanzhy@dstandish I think I managed to workaround the pip resolver issue with #21824 - please rebase to latest main.
thanks a bunch!
@dstandish
dstandish
commented
Feb 25, 2022

Copy link


Copy Markdown

Contributor
finally we can merge. thanks again to @potiuk for resolving the pip problem, and thank you @kazanzhy for your persistence and patience.
@potiuk
potiuk
merged commit 0378659
into
apache:mainFeb 25, 2022
@kazanzhy
kazanzhy
deleted the

add_aws_rds_hooks

branch
February 28, 2022 14:38
@jedcunninghamjedcunningham
added
the 
changelog:skip
Changes that should be skipped from the changelog (CI, tests, etc..)
label
Feb 28, 2022
@potiukpotiuk
mentioned this pull request
Mar 7, 2022

Closed

84 tasks
@kazanzhykazanzhy
mentioned this pull request
Mar 9, 2022
 Merged

@potiukpotiuk
mentioned this pull request
Mar 15, 2022

Closed

41 tasks
@kazanzhykazanzhy
mentioned this pull request
Aug 21, 2022
 Merged

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@dstandishdstandishdstandish approved these changes
@potiukpotiukpotiuk approved these changes
@eladkaleladkaleladkal approved these changes
@mik-lajmik-lajAwaiting requested review from mik-laj
@vikramkokavikramkokaAwaiting requested review from vikramkoka

Assignees


No one assigned


Labels

area:providerschangelog:skipChanges that should be skipped from the changelog (CI, tests, etc..)full tests neededWe need to run full set of tests for this PR to mergekind:documentationprovider:amazonAWS/Amazon - related issues

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

5 participants

@kazanzhy@potiuk@dstandish@eladkal@jedcunningham

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Added Hook for Amazon RDS - #20642
Merged
potiuk merged 1 commit into
apache:mainfrom 
kazanzhy:add_aws_rds_hooks
Feb 25, 2022
Merged
Added Hook for Amazon RDS#20642
potiuk merged 1 commit into
apache:mainfrom 
kazanzhy:add_aws_rds_hooks
Conversation
@kazanzhy
@kazanzhykazanzhy
commented
Jan 3, 2022•

edited

Loading

Copy link


Copy Markdown

Contributor
I had two different implementations of RDSHook.
dstandish
dstandish
reviewed
Jan 4, 2022
View reviewed changes
Comment threadairflow/providers/amazon/aws/hooks/rds.py
Outdated

@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
2 times, most recently
from
aec5a1c to
ae53b0dCompareJanuary 5, 2022 22:14
eladkal
eladkal
reviewed
Jan 6, 2022
View reviewed changes
Comment threadairflow/providers/amazon/aws/hooks/rds.py
Outdated

@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
3 times, most recently
from
03a38e2 to
d093539CompareJanuary 10, 2022 23:21
@kazanzhy
kazanzhy
marked this pull request as ready for review
January 11, 2022 13:40
dstandish
dstandish
requested changes
Jan 11, 2022
View reviewed changes
Comment threadtests/providers/amazon/aws/hooks/test_rds.py
Outdated

Comment threadtests/providers/amazon/aws/hooks/test_rds.py
Outdated

Comment threadtests/providers/amazon/aws/hooks/test_rds.py
Outdated

Comment threadsetup.py
Outdated

Comment threadtests/providers/amazon/aws/hooks/test_rds.py
Outdated

Comment threadtests/providers/amazon/aws/hooks/test_rds.py
Outdated

dstandish
dstandish
reviewed
Jan 12, 2022
View reviewed changes
@dstandishdstandish
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me apart from the one file i have question about
Comment threaddocs/apache-airflow-providers-amazon/operators/rds.rst
Outdated

@github-actions
github-actionsBot
commented
Jan 12, 2022

Copy link


Copy Markdown

Contributor
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
@github-actionsgithub-actionsBot
added
the 
full tests needed
We need to run full set of tests for this PR to merge
label
Jan 12, 2022
@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
4 times, most recently
from
bdd5e00 to
fb2567eCompareJanuary 17, 2022 14:48
@kazanzhykazanzhy
mentioned this pull request
Jan 17, 2022
 Merged

eladkal
eladkal
requested changes
Jan 20, 2022
View reviewed changes
Comment threadairflow/providers/amazon/aws/hooks/rds.py
Outdated

@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
2 times, most recently
from
69157ae to
c5451fdCompareJanuary 24, 2022 15:04
@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
2 times, most recently
from
254c8c2 to
36a6a4fCompareJanuary 29, 2022 15:03
@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
3 times, most recently
from
2308501 to
2432fe2CompareFebruary 18, 2022 18:40
@dstandish
dstandish
commented
Feb 18, 2022

Copy link


Copy Markdown

Contributor
@potiuk i'm a bit confused why we are seeing failures like this in this pr... https://github.com/apache/airflow/runs/5252964277?check_suite_focus=true#step:11:1533

anything pop out to you re why?
@kazanzhy
kazanzhy
commented
Feb 18, 2022•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
So there are three ways to install stubs for RDS.
  • boto3-stubs[rds] - ERRORS
  • boto3-stubs-lite[rds] - ERRORS
  • mypy-boto3-rds - OK
Seems boto3-stubs overload something in botocore.

I'll try to find the solution because it's more convenient to put all necessary libs inside boto3-stubs than list them.
@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
2 times, most recently
from
2192a54 to
645365cCompareFebruary 20, 2022 15:15
@kazanzhykazanzhy
reopened this
Feb 20, 2022
@dstandish
dstandish
commented
Feb 23, 2022

Copy link


Copy Markdown

Contributor
So there are three ways to install stubs for RDS.

boto3-stubs[rds] - ERRORS

boto3-stubs-lite[rds] - ERRORS

mypy-boto3-rds - OK

Seems boto3-stubs overload something in botocore.

I'll try to find the solution because it's more convenient to put all necessary libs inside boto3-stubs than list them.
Nice! glad you figured it out.
@dstandish
dstandishforce-pushed
the
add_aws_rds_hooks
branch
4 times, most recently
from
168cb4c to
f0ef165CompareFebruary 25, 2022 06:23
@potiuk
potiuk
commented
Feb 25, 2022

Copy link


Copy Markdown

Member
@kazanzhy@dstandish - I would really love if you could upvote and comment here: pypa/pip#10258 (comment).
The problem is that we have no way currently to determine what is the root cause of the problem. The problem is that pip is not able to solve dependencies in a reasonable time and runs backtracking for hours - but it does not really give us the possibilty to get the root cause message that could allow us to determine how we could help pip.
I proposed to add optional timeout on pip install execution producing a "conflict" message if the timeout is reached. That could give us at least a chance to understand what is the problem (someone released a new dependency that makes pip fail installing airflow from the scratch and we need to help pip to determine the right set of dependencies because it is unable to figure it out on its own).
@potiuk
potiuk
commented
Feb 25, 2022

Copy link


Copy Markdown

Member
@kazanzhy@dstandish I think I managed to workaround the pip resolver issue with #21824 - please rebase to latest main.
@dstandish
dstandish
commented
Feb 25, 2022

Copy link


Copy Markdown

Contributor
I would really love if you could upvote and comment here:
done
@kazanzhy@dstandish I think I managed to workaround the pip resolver issue with #21824 - please rebase to latest main.
thanks a bunch!
@dstandish
dstandish
commented
Feb 25, 2022

Copy link


Copy Markdown

Contributor
finally we can merge. thanks again to @potiuk for resolving the pip problem, and thank you @kazanzhy for your persistence and patience.
@potiuk
potiuk
merged commit 0378659
into
apache:mainFeb 25, 2022
@kazanzhy
kazanzhy
deleted the

add_aws_rds_hooks

branch
February 28, 2022 14:38
@jedcunninghamjedcunningham
added
the 
changelog:skip
Changes that should be skipped from the changelog (CI, tests, etc..)
label
Feb 28, 2022
@potiukpotiuk
mentioned this pull request
Mar 7, 2022

Closed

84 tasks
@kazanzhykazanzhy
mentioned this pull request
Mar 9, 2022
 Merged

@potiukpotiuk
mentioned this pull request
Mar 15, 2022

Closed

41 tasks
@kazanzhykazanzhy
mentioned this pull request
Aug 21, 2022
 Merged

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@dstandishdstandishdstandish approved these changes
@potiukpotiukpotiuk approved these changes
@eladkaleladkaleladkal approved these changes
@mik-lajmik-lajAwaiting requested review from mik-laj
@vikramkokavikramkokaAwaiting requested review from vikramkoka

Assignees


No one assigned


Labels

area:providerschangelog:skipChanges that should be skipped from the changelog (CI, tests, etc..)full tests neededWe need to run full set of tests for this PR to mergekind:documentationprovider:amazonAWS/Amazon - related issues

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

5 participants

@kazanzhy@potiuk@dstandish@eladkal@jedcunningham

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
Added Hook for Amazon RDS - #20642
Merged
potiuk merged 1 commit into
apache:mainfrom 
kazanzhy:add_aws_rds_hooks
Feb 25, 2022
Merged
Added Hook for Amazon RDS#20642
potiuk merged 1 commit into
apache:mainfrom 
kazanzhy:add_aws_rds_hooks
Conversation
@kazanzhy
@kazanzhykazanzhy
commented
Jan 3, 2022•

edited

Loading

Copy link


Copy Markdown

Contributor
I had two different implementations of RDSHook.
dstandish
dstandish
reviewed
Jan 4, 2022
View reviewed changes
Comment threadairflow/providers/amazon/aws/hooks/rds.py
Outdated

@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
2 times, most recently
from
aec5a1c to
ae53b0dCompareJanuary 5, 2022 22:14
eladkal
eladkal
reviewed
Jan 6, 2022
View reviewed changes
Comment threadairflow/providers/amazon/aws/hooks/rds.py
Outdated

@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
3 times, most recently
from
03a38e2 to
d093539CompareJanuary 10, 2022 23:21
@kazanzhy
kazanzhy
marked this pull request as ready for review
January 11, 2022 13:40
dstandish
dstandish
requested changes
Jan 11, 2022
View reviewed changes
Comment threadtests/providers/amazon/aws/hooks/test_rds.py
Outdated

Comment threadtests/providers/amazon/aws/hooks/test_rds.py
Outdated

Comment threadtests/providers/amazon/aws/hooks/test_rds.py
Outdated

Comment threadsetup.py
Outdated

Comment threadtests/providers/amazon/aws/hooks/test_rds.py
Outdated

Comment threadtests/providers/amazon/aws/hooks/test_rds.py
Outdated

dstandish
dstandish
reviewed
Jan 12, 2022
View reviewed changes
@dstandishdstandish
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me apart from the one file i have question about
Comment threaddocs/apache-airflow-providers-amazon/operators/rds.rst
Outdated

@github-actions
github-actionsBot
commented
Jan 12, 2022

Copy link


Copy Markdown

Contributor
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
@github-actionsgithub-actionsBot
added
the 
full tests needed
We need to run full set of tests for this PR to merge
label
Jan 12, 2022
@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
4 times, most recently
from
bdd5e00 to
fb2567eCompareJanuary 17, 2022 14:48
@kazanzhykazanzhy
mentioned this pull request
Jan 17, 2022
 Merged

eladkal
eladkal
requested changes
Jan 20, 2022
View reviewed changes
Comment threadairflow/providers/amazon/aws/hooks/rds.py
Outdated

@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
2 times, most recently
from
69157ae to
c5451fdCompareJanuary 24, 2022 15:04
@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
2 times, most recently
from
254c8c2 to
36a6a4fCompareJanuary 29, 2022 15:03
@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
3 times, most recently
from
2308501 to
2432fe2CompareFebruary 18, 2022 18:40
@dstandish
dstandish
commented
Feb 18, 2022

Copy link


Copy Markdown

Contributor
@potiuk i'm a bit confused why we are seeing failures like this in this pr... https://github.com/apache/airflow/runs/5252964277?check_suite_focus=true#step:11:1533

anything pop out to you re why?
@kazanzhy
kazanzhy
commented
Feb 18, 2022•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
So there are three ways to install stubs for RDS.
  • boto3-stubs[rds] - ERRORS
  • boto3-stubs-lite[rds] - ERRORS
  • mypy-boto3-rds - OK
Seems boto3-stubs overload something in botocore.

I'll try to find the solution because it's more convenient to put all necessary libs inside boto3-stubs than list them.
@kazanzhy
kazanzhyforce-pushed
the
add_aws_rds_hooks
branch
2 times, most recently
from
2192a54 to
645365cCompareFebruary 20, 2022 15:15
@kazanzhykazanzhy
reopened this
Feb 20, 2022
@dstandish
dstandish
commented
Feb 23, 2022

Copy link


Copy Markdown

Contributor
So there are three ways to install stubs for RDS.

boto3-stubs[rds] - ERRORS

boto3-stubs-lite[rds] - ERRORS

mypy-boto3-rds - OK

Seems boto3-stubs overload something in botocore.

I'll try to find the solution because it's more convenient to put all necessary libs inside boto3-stubs than list them.
Nice! glad you figured it out.
@dstandish
dstandishforce-pushed
the
add_aws_rds_hooks
branch
4 times, most recently
from
168cb4c to
f0ef165CompareFebruary 25, 2022 06:23
@potiuk
potiuk
commented
Feb 25, 2022

Copy link


Copy Markdown

Member
@kazanzhy@dstandish - I would really love if you could upvote and comment here: pypa/pip#10258 (comment).
The problem is that we have no way currently to determine what is the root cause of the problem. The problem is that pip is not able to solve dependencies in a reasonable time and runs backtracking for hours - but it does not really give us the possibilty to get the root cause message that could allow us to determine how we could help pip.
I proposed to add optional timeout on pip install execution producing a "conflict" message if the timeout is reached. That could give us at least a chance to understand what is the problem (someone released a new dependency that makes pip fail installing airflow from the scratch and we need to help pip to determine the right set of dependencies because it is unable to figure it out on its own).
@potiuk
potiuk
commented
Feb 25, 2022

Copy link


Copy Markdown

Member
@kazanzhy@dstandish I think I managed to workaround the pip resolver issue with #21824 - please rebase to latest main.
@dstandish
dstandish
commented
Feb 25, 2022

Copy link


Copy Markdown

Contributor
I would really love if you could upvote and comment here:
done
@kazanzhy@dstandish I think I managed to workaround the pip resolver issue with #21824 - please rebase to latest main.
thanks a bunch!
@dstandish
dstandish
commented
Feb 25, 2022

Copy link


Copy Markdown

Contributor
finally we can merge. thanks again to @potiuk for resolving the pip problem, and thank you @kazanzhy for your persistence and patience.
@potiuk
potiuk
merged commit 0378659
into
apache:mainFeb 25, 2022
@kazanzhy
kazanzhy
deleted the

add_aws_rds_hooks

branch
February 28, 2022 14:38
@jedcunninghamjedcunningham
added
the 
changelog:skip
Changes that should be skipped from the changelog (CI, tests, etc..)
label
Feb 28, 2022
@potiukpotiuk
mentioned this pull request
Mar 7, 2022

Closed

84 tasks
@kazanzhykazanzhy
mentioned this pull request
Mar 9, 2022
 Merged

@potiukpotiuk
mentioned this pull request
Mar 15, 2022

Closed

41 tasks
@kazanzhykazanzhy
mentioned this pull request
Aug 21, 2022
 Merged

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@dstandishdstandishdstandish approved these changes
@potiukpotiukpotiuk approved these changes
@eladkaleladkaleladkal approved these changes
@mik-lajmik-lajAwaiting requested review from mik-laj
@vikramkokavikramkokaAwaiting requested review from vikramkoka

Assignees


No one assigned


Labels

area:providerschangelog:skipChanges that should be skipped from the changelog (CI, tests, etc..)full tests neededWe need to run full set of tests for this PR to mergekind:documentationprovider:amazonAWS/Amazon - related issues

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

5 participants

@kazanzhy@potiuk@dstandish@eladkal@jedcunningham