Skip to content

ci prepare-build-cache - #22344

Merged
potiuk merged 1 commit into
apache:mainfrom
Bowrna:prepare-image-cache
Apr 1, 2022
Merged

ci prepare-build-cache#22344
potiuk merged 1 commit into
apache:mainfrom
Bowrna:prepare-image-cache

Conversation

@Bowrna

@BowrnaBowrna commented Mar 17, 2022

Copy link
Copy Markdown
Contributor

closes : #21102
related: #21102

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.

@Bowrna

Bowrna commented Mar 30, 2022

Copy link
Copy Markdown
ContributorAuthor

@potiuk
--cache:max should be added also in the CI image (because it is multi-segment image now)
#22492 (comment)

I will add this as part of this PR.

@Bowrna
Bowrnaforce-pushed the prepare-image-cache branch from 38ffc3e to 0f32a0aCompareMarch 30, 2022 06:52
@Bowrna

Bowrna commented Mar 30, 2022

Copy link
Copy Markdown
ContributorAuthor

@potiuk i want to know how we can pass the CI flag in python? In bash we set it as env var. Do we need to pass that as flag when we rewrite in python?

airflow/scripts/ci/libraries/_build_images.sh

Lines 454 to 463 in c58b5e5

if [[ ${CI}=="true" ]];then
EXTRA_DOCKER_PROD_BUILD_FLAGS+=(
"--build-arg""PIP_PROGRESS_BAR=off"
)
fi
if [[ -n"${AIRFLOW_CONSTRAINTS_LOCATION}" ]];then
extra_docker_ci_flags+=(
"--build-arg""AIRFLOW_CONSTRAINTS_LOCATION=${AIRFLOW_CONSTRAINTS_LOCATION}"
)
fi

@Bowrna
Bowrnaforce-pushed the prepare-image-cache branch 2 times, most recently from d553191 to 6cc513bCompareMarch 30, 2022 11:45
@Bowrna
Bowrna marked this pull request as ready for review March 30, 2022 11:45
@potiukpotiuk mentioned this pull request Mar 30, 2022
Merged
@Bowrna
Bowrnaforce-pushed the prepare-image-cache branch 2 times, most recently from 5830eee to 4a0260aCompareMarch 31, 2022 02:34
@Bowrna

Bowrna commented Mar 31, 2022

Copy link
Copy Markdown
ContributorAuthor

@potiuk I made changes specified in this #22618
Please verify it

@Bowrna

Bowrna commented Mar 31, 2022

Copy link
Copy Markdown
ContributorAuthor

@potiuk i want to know how we can pass the CI flag in python? In bash we set it as env var. Do we need to pass that as flag when we rewrite in python?

airflow/scripts/ci/libraries/_build_images.sh

Lines 454 to 463 in c58b5e5

if [[ ${CI}=="true" ]];then
EXTRA_DOCKER_PROD_BUILD_FLAGS+=(
"--build-arg""PIP_PROGRESS_BAR=off"
)
fi
if [[ -n"${AIRFLOW_CONSTRAINTS_LOCATION}" ]];then
extra_docker_ci_flags+=(
"--build-arg""AIRFLOW_CONSTRAINTS_LOCATION=${AIRFLOW_CONSTRAINTS_LOCATION}"
)
fi

@potiuk Please share your view on this when you get time.

@Bowrna
Bowrnaforce-pushed the prepare-image-cache branch from 4a0260a to 1a6ad6eCompareMarch 31, 2022 08:07
potiuk
potiuk reviewed Mar 31, 2022
View reviewed changes
Comment threaddev/breeze/src/airflow_breeze/breeze.py Outdated

@potiukpotiukMar 31, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
upgrade_to_newer_dependencies: bool,
upgrade_to_newer_dependencies: bool,

This should be Optional[str] actually :)

@potiukpotiukMar 31, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was the fix I made here -> #22597

@BowrnaBowrnaMar 31, 2022

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@potiuk Is it Optional[str] for prod image too?

@potiukpotiukMar 31, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep

@potiukpotiukMar 31, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added more fixes on the parameter in #22649 :). You will need to rebase and resolve conflicts :)

@Bowrna
Bowrnaforce-pushed the prepare-image-cache branch from 1a6ad6e to 3bc073bCompareMarch 31, 2022 11:22
@potiuk

potiuk commented Mar 31, 2022•
edited
Loading

Copy link
Copy Markdown
Member

@potiuk i want to know how we can pass the CI flag in python? In bash we set it as env var. Do we need to pass that as flag when we rewrite in python?

airflow/scripts/ci/libraries/_build_images.sh

Lines 454 to 463 in c58b5e5

if [[ ${CI}=="true" ]];then
EXTRA_DOCKER_PROD_BUILD_FLAGS+=(
"--build-arg""PIP_PROGRESS_BAR=off"
)
fi
if [[ -n"${AIRFLOW_CONSTRAINTS_LOCATION}" ]];then
extra_docker_ci_flags+=(
"--build-arg""AIRFLOW_CONSTRAINTS_LOCATION=${AIRFLOW_CONSTRAINTS_LOCATION}"
)
fi

@potiuk Please share your view on this when you get time.

Same as in Bash. Just add the PIP_PROGRESS_BAR==off if CI is set to true. There is no need to pass it further. But in order to do it nicely you can add --ci parameter with CI envvar via click.

@Bowrna
Bowrnaforce-pushed the prepare-image-cache branch from 3bc073b to 733bbe1CompareApril 1, 2022 03:51
@Bowrna

Bowrna commented Apr 1, 2022

Copy link
Copy Markdown
ContributorAuthor

@potiuk Added fix for flag upgrade_to_newer_dependencies :)

@Bowrna
Bowrnaforce-pushed the prepare-image-cache branch from 733bbe1 to bb10913CompareApril 1, 2022 06:23
@Bowrna

Bowrna commented Apr 1, 2022

Copy link
Copy Markdown
ContributorAuthor

@potiuk i want to know how we can pass the CI flag in python? In bash we set it as env var. Do we need to pass that as flag when we rewrite in python?

airflow/scripts/ci/libraries/_build_images.sh

Lines 454 to 463 in c58b5e5

if [[ ${CI}=="true" ]];then
EXTRA_DOCKER_PROD_BUILD_FLAGS+=(
"--build-arg""PIP_PROGRESS_BAR=off"
)
fi
if [[ -n"${AIRFLOW_CONSTRAINTS_LOCATION}" ]];then
extra_docker_ci_flags+=(
"--build-arg""AIRFLOW_CONSTRAINTS_LOCATION=${AIRFLOW_CONSTRAINTS_LOCATION}"
)
fi

@potiuk Please share your view on this when you get time.

Same as in Bash. Just add the PIP_PROGRESS_BAR==off if CI is set to true. There is no need to pass it further. But in order to do it nicely you can add --ci parameter with CI envvar via click.

@potiuk Added this support too

@potiuk
potiuk merged commit dc75f5d into apache:mainApr 1, 2022
joppevos
joppevos reviewed Apr 7, 2022
View reviewed changes
Comment threaddev/breeze/src/airflow_breeze/breeze.py
)
option_ci_flag = click.option(
'--ci',
help='Enabling this option will off the pip progress bar',

@joppevosjoppevosApr 7, 2022•
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume it should be "...will TURN off the pip..."? :)

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@potiukpotiukpotiuk approved these changes
@ashbashbAwaiting requested review from ashbashb is a code owner
@jedcunninghamjedcunninghamAwaiting requested review from jedcunninghamjedcunningham is a code owner
+1 more reviewer
@joppevosjoppevosjoppevos left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

area:dev-tools

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Breeze: Add 'prepare-image-cache' in Breeze

3 participants

@Bowrna@potiuk@joppevos
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all 
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
ci prepare-build-cache by Bowrna · Pull Request #22344 · apache/airflow · GitHub
Skip to content
ci prepare-build-cache - #22344
Merged
potiuk merged 1 commit into
apache:mainfrom 
Bowrna:prepare-image-cache
Apr 1, 2022
Merged
ci prepare-build-cache#22344
potiuk merged 1 commit into
apache:mainfrom 
Bowrna:prepare-image-cache
Conversation
@Bowrna
@BowrnaBowrna
commented
Mar 17, 2022

Copy link


Copy Markdown

Contributor
closes : #21102

related: #21102
^ Add meaningful description above
Read the Pull Request Guidelines for more information.

In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.

In case of a new dependency, check compliance with the ASF 3rd Party License Policy.

In case of backwards incompatible changes please leave a note in UPDATING.md.
@Bowrna
Bowrna
commented
Mar 30, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk
--cache:max should be added also in the CI image (because it is multi-segment image now)
#22492 (comment)
I will add this as part of this PR.
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
from
38ffc3e to
0f32a0aCompareMarch 30, 2022 06:52
@Bowrna
Bowrna
commented
Mar 30, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk i want to know how we can pass the CI flag in python? In bash we set it as env var. Do we need to pass that as flag when we rewrite in python?
airflow/scripts/ci/libraries/_build_images.sh

Lines 454 to 463
in
c58b5e5
if [[ ${CI}=="true" ]];then
 EXTRA_DOCKER_PROD_BUILD_FLAGS+=( 
"--build-arg""PIP_PROGRESS_BAR=off"
 ) 
fi
if [[ -n"${AIRFLOW_CONSTRAINTS_LOCATION}" ]];then
 extra_docker_ci_flags+=( 
"--build-arg""AIRFLOW_CONSTRAINTS_LOCATION=${AIRFLOW_CONSTRAINTS_LOCATION}"
 ) 
fi
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
2 times, most recently
from
d553191 to
6cc513bCompareMarch 30, 2022 11:45
@Bowrna
Bowrna
marked this pull request as ready for review
March 30, 2022 11:45
@potiukpotiuk
mentioned this pull request
Mar 30, 2022
 Merged

@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
2 times, most recently
from
5830eee to
4a0260aCompareMarch 31, 2022 02:34
@Bowrna
Bowrna
commented
Mar 31, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk I made changes specified in this #22618

Please verify it
@Bowrna
Bowrna
commented
Mar 31, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk i want to know how we can pass the CI flag in python? In bash we set it as env var. Do we need to pass that as flag when we rewrite in python?
airflow/scripts/ci/libraries/_build_images.sh

Lines 454 to 463
in
c58b5e5
if [[ ${CI}=="true" ]];then
 EXTRA_DOCKER_PROD_BUILD_FLAGS+=( 
"--build-arg""PIP_PROGRESS_BAR=off"
 ) 
fi
if [[ -n"${AIRFLOW_CONSTRAINTS_LOCATION}" ]];then
 extra_docker_ci_flags+=( 
"--build-arg""AIRFLOW_CONSTRAINTS_LOCATION=${AIRFLOW_CONSTRAINTS_LOCATION}"
 ) 
fi
@potiuk Please share your view on this when you get time.
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
from
4a0260a to
1a6ad6eCompareMarch 31, 2022 08:07
potiuk
potiuk
reviewed
Mar 31, 2022
View reviewed changes
Comment threaddev/breeze/src/airflow_breeze/breeze.py
Outdated

@potiukpotiukMar 31, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


Suggested change

upgrade_to_newer_dependencies: bool,
upgrade_to_newer_dependencies: bool,
This should be Optional[str] actually :)
@potiukpotiukMar 31, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was the fix I made here -> #22597
@BowrnaBowrnaMar 31, 2022

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@potiuk Is it Optional[str] for prod image too?
@potiukpotiukMar 31, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep
@potiukpotiukMar 31, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added more fixes on the parameter in #22649 :). You will need to rebase and resolve conflicts :)
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
from
1a6ad6e to
3bc073bCompareMarch 31, 2022 11:22
@potiuk
potiuk
commented
Mar 31, 2022•

edited

Loading

Copy link


Copy Markdown

Member
@potiuk i want to know how we can pass the CI flag in python? In bash we set it as env var. Do we need to pass that as flag when we rewrite in python?
airflow/scripts/ci/libraries/_build_images.sh

Lines 454 to 463
in
c58b5e5
if [[ ${CI}=="true" ]];then
 EXTRA_DOCKER_PROD_BUILD_FLAGS+=( 
"--build-arg""PIP_PROGRESS_BAR=off"
 ) 
fi
if [[ -n"${AIRFLOW_CONSTRAINTS_LOCATION}" ]];then
 extra_docker_ci_flags+=( 
"--build-arg""AIRFLOW_CONSTRAINTS_LOCATION=${AIRFLOW_CONSTRAINTS_LOCATION}"
 ) 
fi
@potiuk Please share your view on this when you get time.
Same as in Bash. Just add the PIP_PROGRESS_BAR==off if CI is set to true. There is no need to pass it further. But in order to do it nicely you can add --ci parameter with CI envvar via click.
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
from
3bc073b to
733bbe1CompareApril 1, 2022 03:51
@Bowrna
Bowrna
commented
Apr 1, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk Added fix for flag upgrade_to_newer_dependencies :)
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
from
733bbe1 to
bb10913CompareApril 1, 2022 06:23
@Bowrna
Bowrna
commented
Apr 1, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk i want to know how we can pass the CI flag in python? In bash we set it as env var. Do we need to pass that as flag when we rewrite in python?
airflow/scripts/ci/libraries/_build_images.sh

Lines 454 to 463
in
c58b5e5
if [[ ${CI}=="true" ]];then
 EXTRA_DOCKER_PROD_BUILD_FLAGS+=( 
"--build-arg""PIP_PROGRESS_BAR=off"
 ) 
fi
if [[ -n"${AIRFLOW_CONSTRAINTS_LOCATION}" ]];then
 extra_docker_ci_flags+=( 
"--build-arg""AIRFLOW_CONSTRAINTS_LOCATION=${AIRFLOW_CONSTRAINTS_LOCATION}"
 ) 
fi
@potiuk Please share your view on this when you get time.
Same as in Bash. Just add the PIP_PROGRESS_BAR==off if CI is set to true. There is no need to pass it further. But in order to do it nicely you can add --ci parameter with CI envvar via click.
@potiuk Added this support too
@potiuk
potiuk
merged commit dc75f5d
into
apache:mainApr 1, 2022
joppevos
joppevos
reviewed
Apr 7, 2022
View reviewed changes
Comment threaddev/breeze/src/airflow_breeze/breeze.py
)
option_ci_flag = click.option(
 '--ci',
 help='Enabling this option will off the pip progress bar',
@joppevosjoppevosApr 7, 2022•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume it should be "...will TURN off the pip..."? :)
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@potiukpotiukpotiuk approved these changes
@ashbashbAwaiting requested review from ashbashb is a code owner
@jedcunninghamjedcunninghamAwaiting requested review from jedcunninghamjedcunningham is a code owner

+1 more reviewer

@joppevosjoppevosjoppevos left review comments
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

area:dev-tools

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

Breeze: Add 'prepare-image-cache' in Breeze


3 participants

@Bowrna@potiuk@joppevos

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Force GitHub README to respect dark mode
(function() {
var style = document.createElement('style');
style.textContent = '
.markdown-body {
color-scheme: dark light;
}
.markdown-body pre { background: #161b22 !important; }
.markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }
.markdown-body table th, .markdown-body table td { border-color: #30363d !important; }
.markdown-body img { background: #0d1117; }
.markdown-body blockquote { border-left-color: #8b949e; }
.markdown-body hr { border-color: #30363d; }
';
document.head.appendChild(style);
})();
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
ci prepare-build-cache by Bowrna · Pull Request #22344 · apache/airflow · GitHub
Skip to content
ci prepare-build-cache - #22344
Merged
potiuk merged 1 commit into
apache:mainfrom 
Bowrna:prepare-image-cache
Apr 1, 2022
Merged
ci prepare-build-cache#22344
potiuk merged 1 commit into
apache:mainfrom 
Bowrna:prepare-image-cache
Conversation
@Bowrna
@BowrnaBowrna
commented
Mar 17, 2022

Copy link


Copy Markdown

Contributor
closes : #21102

related: #21102
^ Add meaningful description above
Read the Pull Request Guidelines for more information.

In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.

In case of a new dependency, check compliance with the ASF 3rd Party License Policy.

In case of backwards incompatible changes please leave a note in UPDATING.md.
@Bowrna
Bowrna
commented
Mar 30, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk
--cache:max should be added also in the CI image (because it is multi-segment image now)
#22492 (comment)
I will add this as part of this PR.
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
from
38ffc3e to
0f32a0aCompareMarch 30, 2022 06:52
@Bowrna
Bowrna
commented
Mar 30, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk i want to know how we can pass the CI flag in python? In bash we set it as env var. Do we need to pass that as flag when we rewrite in python?
airflow/scripts/ci/libraries/_build_images.sh

Lines 454 to 463
in
c58b5e5
if [[ ${CI}=="true" ]];then
 EXTRA_DOCKER_PROD_BUILD_FLAGS+=( 
"--build-arg""PIP_PROGRESS_BAR=off"
 ) 
fi
if [[ -n"${AIRFLOW_CONSTRAINTS_LOCATION}" ]];then
 extra_docker_ci_flags+=( 
"--build-arg""AIRFLOW_CONSTRAINTS_LOCATION=${AIRFLOW_CONSTRAINTS_LOCATION}"
 ) 
fi
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
2 times, most recently
from
d553191 to
6cc513bCompareMarch 30, 2022 11:45
@Bowrna
Bowrna
marked this pull request as ready for review
March 30, 2022 11:45
@potiukpotiuk
mentioned this pull request
Mar 30, 2022
 Merged

@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
2 times, most recently
from
5830eee to
4a0260aCompareMarch 31, 2022 02:34
@Bowrna
Bowrna
commented
Mar 31, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk I made changes specified in this #22618

Please verify it
@Bowrna
Bowrna
commented
Mar 31, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk i want to know how we can pass the CI flag in python? In bash we set it as env var. Do we need to pass that as flag when we rewrite in python?
airflow/scripts/ci/libraries/_build_images.sh

Lines 454 to 463
in
c58b5e5
if [[ ${CI}=="true" ]];then
 EXTRA_DOCKER_PROD_BUILD_FLAGS+=( 
"--build-arg""PIP_PROGRESS_BAR=off"
 ) 
fi
if [[ -n"${AIRFLOW_CONSTRAINTS_LOCATION}" ]];then
 extra_docker_ci_flags+=( 
"--build-arg""AIRFLOW_CONSTRAINTS_LOCATION=${AIRFLOW_CONSTRAINTS_LOCATION}"
 ) 
fi
@potiuk Please share your view on this when you get time.
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
from
4a0260a to
1a6ad6eCompareMarch 31, 2022 08:07
potiuk
potiuk
reviewed
Mar 31, 2022
View reviewed changes
Comment threaddev/breeze/src/airflow_breeze/breeze.py
Outdated

@potiukpotiukMar 31, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


Suggested change

upgrade_to_newer_dependencies: bool,
upgrade_to_newer_dependencies: bool,
This should be Optional[str] actually :)
@potiukpotiukMar 31, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was the fix I made here -> #22597
@BowrnaBowrnaMar 31, 2022

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@potiuk Is it Optional[str] for prod image too?
@potiukpotiukMar 31, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep
@potiukpotiukMar 31, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added more fixes on the parameter in #22649 :). You will need to rebase and resolve conflicts :)
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
from
1a6ad6e to
3bc073bCompareMarch 31, 2022 11:22
@potiuk
potiuk
commented
Mar 31, 2022•

edited

Loading

Copy link


Copy Markdown

Member
@potiuk i want to know how we can pass the CI flag in python? In bash we set it as env var. Do we need to pass that as flag when we rewrite in python?
airflow/scripts/ci/libraries/_build_images.sh

Lines 454 to 463
in
c58b5e5
if [[ ${CI}=="true" ]];then
 EXTRA_DOCKER_PROD_BUILD_FLAGS+=( 
"--build-arg""PIP_PROGRESS_BAR=off"
 ) 
fi
if [[ -n"${AIRFLOW_CONSTRAINTS_LOCATION}" ]];then
 extra_docker_ci_flags+=( 
"--build-arg""AIRFLOW_CONSTRAINTS_LOCATION=${AIRFLOW_CONSTRAINTS_LOCATION}"
 ) 
fi
@potiuk Please share your view on this when you get time.
Same as in Bash. Just add the PIP_PROGRESS_BAR==off if CI is set to true. There is no need to pass it further. But in order to do it nicely you can add --ci parameter with CI envvar via click.
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
from
3bc073b to
733bbe1CompareApril 1, 2022 03:51
@Bowrna
Bowrna
commented
Apr 1, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk Added fix for flag upgrade_to_newer_dependencies :)
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
from
733bbe1 to
bb10913CompareApril 1, 2022 06:23
@Bowrna
Bowrna
commented
Apr 1, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk i want to know how we can pass the CI flag in python? In bash we set it as env var. Do we need to pass that as flag when we rewrite in python?
airflow/scripts/ci/libraries/_build_images.sh

Lines 454 to 463
in
c58b5e5
if [[ ${CI}=="true" ]];then
 EXTRA_DOCKER_PROD_BUILD_FLAGS+=( 
"--build-arg""PIP_PROGRESS_BAR=off"
 ) 
fi
if [[ -n"${AIRFLOW_CONSTRAINTS_LOCATION}" ]];then
 extra_docker_ci_flags+=( 
"--build-arg""AIRFLOW_CONSTRAINTS_LOCATION=${AIRFLOW_CONSTRAINTS_LOCATION}"
 ) 
fi
@potiuk Please share your view on this when you get time.
Same as in Bash. Just add the PIP_PROGRESS_BAR==off if CI is set to true. There is no need to pass it further. But in order to do it nicely you can add --ci parameter with CI envvar via click.
@potiuk Added this support too
@potiuk
potiuk
merged commit dc75f5d
into
apache:mainApr 1, 2022
joppevos
joppevos
reviewed
Apr 7, 2022
View reviewed changes
Comment threaddev/breeze/src/airflow_breeze/breeze.py
)
option_ci_flag = click.option(
 '--ci',
 help='Enabling this option will off the pip progress bar',
@joppevosjoppevosApr 7, 2022•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume it should be "...will TURN off the pip..."? :)
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@potiukpotiukpotiuk approved these changes
@ashbashbAwaiting requested review from ashbashb is a code owner
@jedcunninghamjedcunninghamAwaiting requested review from jedcunninghamjedcunningham is a code owner

+1 more reviewer

@joppevosjoppevosjoppevos left review comments
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

area:dev-tools

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

Breeze: Add 'prepare-image-cache' in Breeze


3 participants

@Bowrna@potiuk@joppevos

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Highlight search terms from Google/DuckDuckGo/Bing referrer
(function() {
var ref = document.referrer;
var terms = [];
if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {
var url = new URL(ref);
var q = url.searchParams.get('q') || url.searchParams.get('p');
if (q) {
terms = q.split(/\s+/).filter(function(t) { return t.length > 2; });
}
}
if (terms.length === 0) return;
var style = document.createElement('style');
style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';
document.head.appendChild(style);
function highlight(node) {
if (node.nodeType === 3) { // text node
var text = node.textContent;
var found = false;
terms.forEach(function(term) {
var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi');
if (regex.test(text)) {
found = true;
var frag = document.createDocumentFragment();
var parts = text.split(regex);
parts.forEach(function(part, i) {
if (i % 2 === 0) {
frag.appendChild(document.createTextNode(part));
} else {
var span = document.createElement('span');
span.className = 'userscript-highlight';
span.textContent = part;
frag.appendChild(span);
}
});
node.parentNode.replaceChild(frag, node);
}
});
} else if (node.nodeType === 1 && node.childNodes) { // element
var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];
if (!skipTags.includes(node.tagName)) {
Array.from(node.childNodes).forEach(highlight);
}
}
}
highlight(document.body);
// Re-highlight on dynamic content
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1 || node.nodeType === 3) highlight(node);
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
ci prepare-build-cache by Bowrna · Pull Request #22344 · apache/airflow · GitHub
Skip to content
ci prepare-build-cache - #22344
Merged
potiuk merged 1 commit into
apache:mainfrom 
Bowrna:prepare-image-cache
Apr 1, 2022
Merged
ci prepare-build-cache#22344
potiuk merged 1 commit into
apache:mainfrom 
Bowrna:prepare-image-cache
Conversation
@Bowrna
@BowrnaBowrna
commented
Mar 17, 2022

Copy link


Copy Markdown

Contributor
closes : #21102

related: #21102
^ Add meaningful description above
Read the Pull Request Guidelines for more information.

In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.

In case of a new dependency, check compliance with the ASF 3rd Party License Policy.

In case of backwards incompatible changes please leave a note in UPDATING.md.
@Bowrna
Bowrna
commented
Mar 30, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk
--cache:max should be added also in the CI image (because it is multi-segment image now)
#22492 (comment)
I will add this as part of this PR.
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
from
38ffc3e to
0f32a0aCompareMarch 30, 2022 06:52
@Bowrna
Bowrna
commented
Mar 30, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk i want to know how we can pass the CI flag in python? In bash we set it as env var. Do we need to pass that as flag when we rewrite in python?
airflow/scripts/ci/libraries/_build_images.sh

Lines 454 to 463
in
c58b5e5
if [[ ${CI}=="true" ]];then
 EXTRA_DOCKER_PROD_BUILD_FLAGS+=( 
"--build-arg""PIP_PROGRESS_BAR=off"
 ) 
fi
if [[ -n"${AIRFLOW_CONSTRAINTS_LOCATION}" ]];then
 extra_docker_ci_flags+=( 
"--build-arg""AIRFLOW_CONSTRAINTS_LOCATION=${AIRFLOW_CONSTRAINTS_LOCATION}"
 ) 
fi
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
2 times, most recently
from
d553191 to
6cc513bCompareMarch 30, 2022 11:45
@Bowrna
Bowrna
marked this pull request as ready for review
March 30, 2022 11:45
@potiukpotiuk
mentioned this pull request
Mar 30, 2022
 Merged

@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
2 times, most recently
from
5830eee to
4a0260aCompareMarch 31, 2022 02:34
@Bowrna
Bowrna
commented
Mar 31, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk I made changes specified in this #22618

Please verify it
@Bowrna
Bowrna
commented
Mar 31, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk i want to know how we can pass the CI flag in python? In bash we set it as env var. Do we need to pass that as flag when we rewrite in python?
airflow/scripts/ci/libraries/_build_images.sh

Lines 454 to 463
in
c58b5e5
if [[ ${CI}=="true" ]];then
 EXTRA_DOCKER_PROD_BUILD_FLAGS+=( 
"--build-arg""PIP_PROGRESS_BAR=off"
 ) 
fi
if [[ -n"${AIRFLOW_CONSTRAINTS_LOCATION}" ]];then
 extra_docker_ci_flags+=( 
"--build-arg""AIRFLOW_CONSTRAINTS_LOCATION=${AIRFLOW_CONSTRAINTS_LOCATION}"
 ) 
fi
@potiuk Please share your view on this when you get time.
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
from
4a0260a to
1a6ad6eCompareMarch 31, 2022 08:07
potiuk
potiuk
reviewed
Mar 31, 2022
View reviewed changes
Comment threaddev/breeze/src/airflow_breeze/breeze.py
Outdated

@potiukpotiukMar 31, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


Suggested change

upgrade_to_newer_dependencies: bool,
upgrade_to_newer_dependencies: bool,
This should be Optional[str] actually :)
@potiukpotiukMar 31, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was the fix I made here -> #22597
@BowrnaBowrnaMar 31, 2022

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@potiuk Is it Optional[str] for prod image too?
@potiukpotiukMar 31, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep
@potiukpotiukMar 31, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added more fixes on the parameter in #22649 :). You will need to rebase and resolve conflicts :)
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
from
1a6ad6e to
3bc073bCompareMarch 31, 2022 11:22
@potiuk
potiuk
commented
Mar 31, 2022•

edited

Loading

Copy link


Copy Markdown

Member
@potiuk i want to know how we can pass the CI flag in python? In bash we set it as env var. Do we need to pass that as flag when we rewrite in python?
airflow/scripts/ci/libraries/_build_images.sh

Lines 454 to 463
in
c58b5e5
if [[ ${CI}=="true" ]];then
 EXTRA_DOCKER_PROD_BUILD_FLAGS+=( 
"--build-arg""PIP_PROGRESS_BAR=off"
 ) 
fi
if [[ -n"${AIRFLOW_CONSTRAINTS_LOCATION}" ]];then
 extra_docker_ci_flags+=( 
"--build-arg""AIRFLOW_CONSTRAINTS_LOCATION=${AIRFLOW_CONSTRAINTS_LOCATION}"
 ) 
fi
@potiuk Please share your view on this when you get time.
Same as in Bash. Just add the PIP_PROGRESS_BAR==off if CI is set to true. There is no need to pass it further. But in order to do it nicely you can add --ci parameter with CI envvar via click.
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
from
3bc073b to
733bbe1CompareApril 1, 2022 03:51
@Bowrna
Bowrna
commented
Apr 1, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk Added fix for flag upgrade_to_newer_dependencies :)
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
from
733bbe1 to
bb10913CompareApril 1, 2022 06:23
@Bowrna
Bowrna
commented
Apr 1, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk i want to know how we can pass the CI flag in python? In bash we set it as env var. Do we need to pass that as flag when we rewrite in python?
airflow/scripts/ci/libraries/_build_images.sh

Lines 454 to 463
in
c58b5e5
if [[ ${CI}=="true" ]];then
 EXTRA_DOCKER_PROD_BUILD_FLAGS+=( 
"--build-arg""PIP_PROGRESS_BAR=off"
 ) 
fi
if [[ -n"${AIRFLOW_CONSTRAINTS_LOCATION}" ]];then
 extra_docker_ci_flags+=( 
"--build-arg""AIRFLOW_CONSTRAINTS_LOCATION=${AIRFLOW_CONSTRAINTS_LOCATION}"
 ) 
fi
@potiuk Please share your view on this when you get time.
Same as in Bash. Just add the PIP_PROGRESS_BAR==off if CI is set to true. There is no need to pass it further. But in order to do it nicely you can add --ci parameter with CI envvar via click.
@potiuk Added this support too
@potiuk
potiuk
merged commit dc75f5d
into
apache:mainApr 1, 2022
joppevos
joppevos
reviewed
Apr 7, 2022
View reviewed changes
Comment threaddev/breeze/src/airflow_breeze/breeze.py
)
option_ci_flag = click.option(
 '--ci',
 help='Enabling this option will off the pip progress bar',
@joppevosjoppevosApr 7, 2022•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume it should be "...will TURN off the pip..."? :)
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@potiukpotiukpotiuk approved these changes
@ashbashbAwaiting requested review from ashbashb is a code owner
@jedcunninghamjedcunninghamAwaiting requested review from jedcunninghamjedcunningham is a code owner

+1 more reviewer

@joppevosjoppevosjoppevos left review comments
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

area:dev-tools

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

Breeze: Add 'prepare-image-cache' in Breeze


3 participants

@Bowrna@potiuk@joppevos

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Strip utm_, fbclid, gclid, etc. from all links on page
(function() {
var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',
'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',
'ref', 'ref_src', 'source', 'medium', 'campaign'];
function cleanUrl(url) {
try {
var u = new URL(url, window.location.origin);
var changed = false;
trackingParams.forEach(function(p) {
if (u.searchParams.has(p)) {
u.searchParams.delete(p);
changed = true;
}
});
return changed ? u.toString() : url;
} catch (e) {
return url;
}
}
function cleanLinks() {
document.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
cleanLinks();
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1) {
if (node.tagName === 'A') cleanLinks();
node.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '
ci prepare-build-cache by Bowrna · Pull Request #22344 · apache/airflow · GitHub
Skip to content
ci prepare-build-cache - #22344
Merged
potiuk merged 1 commit into
apache:mainfrom 
Bowrna:prepare-image-cache
Apr 1, 2022
Merged
ci prepare-build-cache#22344
potiuk merged 1 commit into
apache:mainfrom 
Bowrna:prepare-image-cache
Conversation
@Bowrna
@BowrnaBowrna
commented
Mar 17, 2022

Copy link


Copy Markdown

Contributor
closes : #21102

related: #21102
^ Add meaningful description above
Read the Pull Request Guidelines for more information.

In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.

In case of a new dependency, check compliance with the ASF 3rd Party License Policy.

In case of backwards incompatible changes please leave a note in UPDATING.md.
@Bowrna
Bowrna
commented
Mar 30, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk
--cache:max should be added also in the CI image (because it is multi-segment image now)
#22492 (comment)
I will add this as part of this PR.
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
from
38ffc3e to
0f32a0aCompareMarch 30, 2022 06:52
@Bowrna
Bowrna
commented
Mar 30, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk i want to know how we can pass the CI flag in python? In bash we set it as env var. Do we need to pass that as flag when we rewrite in python?
airflow/scripts/ci/libraries/_build_images.sh

Lines 454 to 463
in
c58b5e5
if [[ ${CI}=="true" ]];then
 EXTRA_DOCKER_PROD_BUILD_FLAGS+=( 
"--build-arg""PIP_PROGRESS_BAR=off"
 ) 
fi
if [[ -n"${AIRFLOW_CONSTRAINTS_LOCATION}" ]];then
 extra_docker_ci_flags+=( 
"--build-arg""AIRFLOW_CONSTRAINTS_LOCATION=${AIRFLOW_CONSTRAINTS_LOCATION}"
 ) 
fi
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
2 times, most recently
from
d553191 to
6cc513bCompareMarch 30, 2022 11:45
@Bowrna
Bowrna
marked this pull request as ready for review
March 30, 2022 11:45
@potiukpotiuk
mentioned this pull request
Mar 30, 2022
 Merged

@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
2 times, most recently
from
5830eee to
4a0260aCompareMarch 31, 2022 02:34
@Bowrna
Bowrna
commented
Mar 31, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk I made changes specified in this #22618

Please verify it
@Bowrna
Bowrna
commented
Mar 31, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk i want to know how we can pass the CI flag in python? In bash we set it as env var. Do we need to pass that as flag when we rewrite in python?
airflow/scripts/ci/libraries/_build_images.sh

Lines 454 to 463
in
c58b5e5
if [[ ${CI}=="true" ]];then
 EXTRA_DOCKER_PROD_BUILD_FLAGS+=( 
"--build-arg""PIP_PROGRESS_BAR=off"
 ) 
fi
if [[ -n"${AIRFLOW_CONSTRAINTS_LOCATION}" ]];then
 extra_docker_ci_flags+=( 
"--build-arg""AIRFLOW_CONSTRAINTS_LOCATION=${AIRFLOW_CONSTRAINTS_LOCATION}"
 ) 
fi
@potiuk Please share your view on this when you get time.
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
from
4a0260a to
1a6ad6eCompareMarch 31, 2022 08:07
potiuk
potiuk
reviewed
Mar 31, 2022
View reviewed changes
Comment threaddev/breeze/src/airflow_breeze/breeze.py
Outdated

@potiukpotiukMar 31, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


Suggested change

upgrade_to_newer_dependencies: bool,
upgrade_to_newer_dependencies: bool,
This should be Optional[str] actually :)
@potiukpotiukMar 31, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was the fix I made here -> #22597
@BowrnaBowrnaMar 31, 2022

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@potiuk Is it Optional[str] for prod image too?
@potiukpotiukMar 31, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep
@potiukpotiukMar 31, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added more fixes on the parameter in #22649 :). You will need to rebase and resolve conflicts :)
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
from
1a6ad6e to
3bc073bCompareMarch 31, 2022 11:22
@potiuk
potiuk
commented
Mar 31, 2022•

edited

Loading

Copy link


Copy Markdown

Member
@potiuk i want to know how we can pass the CI flag in python? In bash we set it as env var. Do we need to pass that as flag when we rewrite in python?
airflow/scripts/ci/libraries/_build_images.sh

Lines 454 to 463
in
c58b5e5
if [[ ${CI}=="true" ]];then
 EXTRA_DOCKER_PROD_BUILD_FLAGS+=( 
"--build-arg""PIP_PROGRESS_BAR=off"
 ) 
fi
if [[ -n"${AIRFLOW_CONSTRAINTS_LOCATION}" ]];then
 extra_docker_ci_flags+=( 
"--build-arg""AIRFLOW_CONSTRAINTS_LOCATION=${AIRFLOW_CONSTRAINTS_LOCATION}"
 ) 
fi
@potiuk Please share your view on this when you get time.
Same as in Bash. Just add the PIP_PROGRESS_BAR==off if CI is set to true. There is no need to pass it further. But in order to do it nicely you can add --ci parameter with CI envvar via click.
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
from
3bc073b to
733bbe1CompareApril 1, 2022 03:51
@Bowrna
Bowrna
commented
Apr 1, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk Added fix for flag upgrade_to_newer_dependencies :)
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
from
733bbe1 to
bb10913CompareApril 1, 2022 06:23
@Bowrna
Bowrna
commented
Apr 1, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk i want to know how we can pass the CI flag in python? In bash we set it as env var. Do we need to pass that as flag when we rewrite in python?
airflow/scripts/ci/libraries/_build_images.sh

Lines 454 to 463
in
c58b5e5
if [[ ${CI}=="true" ]];then
 EXTRA_DOCKER_PROD_BUILD_FLAGS+=( 
"--build-arg""PIP_PROGRESS_BAR=off"
 ) 
fi
if [[ -n"${AIRFLOW_CONSTRAINTS_LOCATION}" ]];then
 extra_docker_ci_flags+=( 
"--build-arg""AIRFLOW_CONSTRAINTS_LOCATION=${AIRFLOW_CONSTRAINTS_LOCATION}"
 ) 
fi
@potiuk Please share your view on this when you get time.
Same as in Bash. Just add the PIP_PROGRESS_BAR==off if CI is set to true. There is no need to pass it further. But in order to do it nicely you can add --ci parameter with CI envvar via click.
@potiuk Added this support too
@potiuk
potiuk
merged commit dc75f5d
into
apache:mainApr 1, 2022
joppevos
joppevos
reviewed
Apr 7, 2022
View reviewed changes
Comment threaddev/breeze/src/airflow_breeze/breeze.py
)
option_ci_flag = click.option(
 '--ci',
 help='Enabling this option will off the pip progress bar',
@joppevosjoppevosApr 7, 2022•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume it should be "...will TURN off the pip..."? :)
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@potiukpotiukpotiuk approved these changes
@ashbashbAwaiting requested review from ashbashb is a code owner
@jedcunninghamjedcunninghamAwaiting requested review from jedcunninghamjedcunningham is a code owner

+1 more reviewer

@joppevosjoppevosjoppevos left review comments
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

area:dev-tools

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

Breeze: Add 'prepare-image-cache' in Breeze


3 participants

@Bowrna@potiuk@joppevos

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Auto-enable theater mode on YouTube
(function() {
function tryTheater() {
var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]');
if (btn && !btn.classList.contains('activated')) {
btn.click();
}
}
// Try immediately
tryTheater();
// Try after navigation (SPA)
var lastUrl = location.href;
setInterval(function() {
if (location.href !== lastUrl) {
lastUrl = location.href;
setTimeout(tryTheater, 500);
}
}, 1000);
// Also try on player load
var observer = new MutationObserver(tryTheater);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
ci prepare-build-cache by Bowrna · Pull Request #22344 · apache/airflow · GitHub
Skip to content
ci prepare-build-cache - #22344
Merged
potiuk merged 1 commit into
apache:mainfrom 
Bowrna:prepare-image-cache
Apr 1, 2022
Merged
ci prepare-build-cache#22344
potiuk merged 1 commit into
apache:mainfrom 
Bowrna:prepare-image-cache
Conversation
@Bowrna
@BowrnaBowrna
commented
Mar 17, 2022

Copy link


Copy Markdown

Contributor
closes : #21102

related: #21102
^ Add meaningful description above
Read the Pull Request Guidelines for more information.

In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.

In case of a new dependency, check compliance with the ASF 3rd Party License Policy.

In case of backwards incompatible changes please leave a note in UPDATING.md.
@Bowrna
Bowrna
commented
Mar 30, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk
--cache:max should be added also in the CI image (because it is multi-segment image now)
#22492 (comment)
I will add this as part of this PR.
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
from
38ffc3e to
0f32a0aCompareMarch 30, 2022 06:52
@Bowrna
Bowrna
commented
Mar 30, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk i want to know how we can pass the CI flag in python? In bash we set it as env var. Do we need to pass that as flag when we rewrite in python?
airflow/scripts/ci/libraries/_build_images.sh

Lines 454 to 463
in
c58b5e5
if [[ ${CI}=="true" ]];then
 EXTRA_DOCKER_PROD_BUILD_FLAGS+=( 
"--build-arg""PIP_PROGRESS_BAR=off"
 ) 
fi
if [[ -n"${AIRFLOW_CONSTRAINTS_LOCATION}" ]];then
 extra_docker_ci_flags+=( 
"--build-arg""AIRFLOW_CONSTRAINTS_LOCATION=${AIRFLOW_CONSTRAINTS_LOCATION}"
 ) 
fi
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
2 times, most recently
from
d553191 to
6cc513bCompareMarch 30, 2022 11:45
@Bowrna
Bowrna
marked this pull request as ready for review
March 30, 2022 11:45
@potiukpotiuk
mentioned this pull request
Mar 30, 2022
 Merged

@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
2 times, most recently
from
5830eee to
4a0260aCompareMarch 31, 2022 02:34
@Bowrna
Bowrna
commented
Mar 31, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk I made changes specified in this #22618

Please verify it
@Bowrna
Bowrna
commented
Mar 31, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk i want to know how we can pass the CI flag in python? In bash we set it as env var. Do we need to pass that as flag when we rewrite in python?
airflow/scripts/ci/libraries/_build_images.sh

Lines 454 to 463
in
c58b5e5
if [[ ${CI}=="true" ]];then
 EXTRA_DOCKER_PROD_BUILD_FLAGS+=( 
"--build-arg""PIP_PROGRESS_BAR=off"
 ) 
fi
if [[ -n"${AIRFLOW_CONSTRAINTS_LOCATION}" ]];then
 extra_docker_ci_flags+=( 
"--build-arg""AIRFLOW_CONSTRAINTS_LOCATION=${AIRFLOW_CONSTRAINTS_LOCATION}"
 ) 
fi
@potiuk Please share your view on this when you get time.
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
from
4a0260a to
1a6ad6eCompareMarch 31, 2022 08:07
potiuk
potiuk
reviewed
Mar 31, 2022
View reviewed changes
Comment threaddev/breeze/src/airflow_breeze/breeze.py
Outdated

@potiukpotiukMar 31, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


Suggested change

upgrade_to_newer_dependencies: bool,
upgrade_to_newer_dependencies: bool,
This should be Optional[str] actually :)
@potiukpotiukMar 31, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was the fix I made here -> #22597
@BowrnaBowrnaMar 31, 2022

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@potiuk Is it Optional[str] for prod image too?
@potiukpotiukMar 31, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep
@potiukpotiukMar 31, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added more fixes on the parameter in #22649 :). You will need to rebase and resolve conflicts :)
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
from
1a6ad6e to
3bc073bCompareMarch 31, 2022 11:22
@potiuk
potiuk
commented
Mar 31, 2022•

edited

Loading

Copy link


Copy Markdown

Member
@potiuk i want to know how we can pass the CI flag in python? In bash we set it as env var. Do we need to pass that as flag when we rewrite in python?
airflow/scripts/ci/libraries/_build_images.sh

Lines 454 to 463
in
c58b5e5
if [[ ${CI}=="true" ]];then
 EXTRA_DOCKER_PROD_BUILD_FLAGS+=( 
"--build-arg""PIP_PROGRESS_BAR=off"
 ) 
fi
if [[ -n"${AIRFLOW_CONSTRAINTS_LOCATION}" ]];then
 extra_docker_ci_flags+=( 
"--build-arg""AIRFLOW_CONSTRAINTS_LOCATION=${AIRFLOW_CONSTRAINTS_LOCATION}"
 ) 
fi
@potiuk Please share your view on this when you get time.
Same as in Bash. Just add the PIP_PROGRESS_BAR==off if CI is set to true. There is no need to pass it further. But in order to do it nicely you can add --ci parameter with CI envvar via click.
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
from
3bc073b to
733bbe1CompareApril 1, 2022 03:51
@Bowrna
Bowrna
commented
Apr 1, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk Added fix for flag upgrade_to_newer_dependencies :)
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
from
733bbe1 to
bb10913CompareApril 1, 2022 06:23
@Bowrna
Bowrna
commented
Apr 1, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk i want to know how we can pass the CI flag in python? In bash we set it as env var. Do we need to pass that as flag when we rewrite in python?
airflow/scripts/ci/libraries/_build_images.sh

Lines 454 to 463
in
c58b5e5
if [[ ${CI}=="true" ]];then
 EXTRA_DOCKER_PROD_BUILD_FLAGS+=( 
"--build-arg""PIP_PROGRESS_BAR=off"
 ) 
fi
if [[ -n"${AIRFLOW_CONSTRAINTS_LOCATION}" ]];then
 extra_docker_ci_flags+=( 
"--build-arg""AIRFLOW_CONSTRAINTS_LOCATION=${AIRFLOW_CONSTRAINTS_LOCATION}"
 ) 
fi
@potiuk Please share your view on this when you get time.
Same as in Bash. Just add the PIP_PROGRESS_BAR==off if CI is set to true. There is no need to pass it further. But in order to do it nicely you can add --ci parameter with CI envvar via click.
@potiuk Added this support too
@potiuk
potiuk
merged commit dc75f5d
into
apache:mainApr 1, 2022
joppevos
joppevos
reviewed
Apr 7, 2022
View reviewed changes
Comment threaddev/breeze/src/airflow_breeze/breeze.py
)
option_ci_flag = click.option(
 '--ci',
 help='Enabling this option will off the pip progress bar',
@joppevosjoppevosApr 7, 2022•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume it should be "...will TURN off the pip..."? :)
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@potiukpotiukpotiuk approved these changes
@ashbashbAwaiting requested review from ashbashb is a code owner
@jedcunninghamjedcunninghamAwaiting requested review from jedcunninghamjedcunningham is a code owner

+1 more reviewer

@joppevosjoppevosjoppevos left review comments
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

area:dev-tools

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

Breeze: Add 'prepare-image-cache' in Breeze


3 participants

@Bowrna@potiuk@joppevos

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Remove or un-stick sticky/fixed headers that block content
(function() {
function unstick() {
document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) {
if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {
el.style.position = 'static';
el.style.top = 'auto';
el.style.zIndex = 'auto';
}
});
}
unstick();
var observer = new MutationObserver(unstick);
observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });
})();
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
ci prepare-build-cache by Bowrna · Pull Request #22344 · apache/airflow · GitHub
Skip to content
ci prepare-build-cache - #22344
Merged
potiuk merged 1 commit into
apache:mainfrom 
Bowrna:prepare-image-cache
Apr 1, 2022
Merged
ci prepare-build-cache#22344
potiuk merged 1 commit into
apache:mainfrom 
Bowrna:prepare-image-cache
Conversation
@Bowrna
@BowrnaBowrna
commented
Mar 17, 2022

Copy link


Copy Markdown

Contributor
closes : #21102

related: #21102
^ Add meaningful description above
Read the Pull Request Guidelines for more information.

In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.

In case of a new dependency, check compliance with the ASF 3rd Party License Policy.

In case of backwards incompatible changes please leave a note in UPDATING.md.
@Bowrna
Bowrna
commented
Mar 30, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk
--cache:max should be added also in the CI image (because it is multi-segment image now)
#22492 (comment)
I will add this as part of this PR.
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
from
38ffc3e to
0f32a0aCompareMarch 30, 2022 06:52
@Bowrna
Bowrna
commented
Mar 30, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk i want to know how we can pass the CI flag in python? In bash we set it as env var. Do we need to pass that as flag when we rewrite in python?
airflow/scripts/ci/libraries/_build_images.sh

Lines 454 to 463
in
c58b5e5
if [[ ${CI}=="true" ]];then
 EXTRA_DOCKER_PROD_BUILD_FLAGS+=( 
"--build-arg""PIP_PROGRESS_BAR=off"
 ) 
fi
if [[ -n"${AIRFLOW_CONSTRAINTS_LOCATION}" ]];then
 extra_docker_ci_flags+=( 
"--build-arg""AIRFLOW_CONSTRAINTS_LOCATION=${AIRFLOW_CONSTRAINTS_LOCATION}"
 ) 
fi
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
2 times, most recently
from
d553191 to
6cc513bCompareMarch 30, 2022 11:45
@Bowrna
Bowrna
marked this pull request as ready for review
March 30, 2022 11:45
@potiukpotiuk
mentioned this pull request
Mar 30, 2022
 Merged

@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
2 times, most recently
from
5830eee to
4a0260aCompareMarch 31, 2022 02:34
@Bowrna
Bowrna
commented
Mar 31, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk I made changes specified in this #22618

Please verify it
@Bowrna
Bowrna
commented
Mar 31, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk i want to know how we can pass the CI flag in python? In bash we set it as env var. Do we need to pass that as flag when we rewrite in python?
airflow/scripts/ci/libraries/_build_images.sh

Lines 454 to 463
in
c58b5e5
if [[ ${CI}=="true" ]];then
 EXTRA_DOCKER_PROD_BUILD_FLAGS+=( 
"--build-arg""PIP_PROGRESS_BAR=off"
 ) 
fi
if [[ -n"${AIRFLOW_CONSTRAINTS_LOCATION}" ]];then
 extra_docker_ci_flags+=( 
"--build-arg""AIRFLOW_CONSTRAINTS_LOCATION=${AIRFLOW_CONSTRAINTS_LOCATION}"
 ) 
fi
@potiuk Please share your view on this when you get time.
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
from
4a0260a to
1a6ad6eCompareMarch 31, 2022 08:07
potiuk
potiuk
reviewed
Mar 31, 2022
View reviewed changes
Comment threaddev/breeze/src/airflow_breeze/breeze.py
Outdated

@potiukpotiukMar 31, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


Suggested change

upgrade_to_newer_dependencies: bool,
upgrade_to_newer_dependencies: bool,
This should be Optional[str] actually :)
@potiukpotiukMar 31, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was the fix I made here -> #22597
@BowrnaBowrnaMar 31, 2022

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@potiuk Is it Optional[str] for prod image too?
@potiukpotiukMar 31, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep
@potiukpotiukMar 31, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added more fixes on the parameter in #22649 :). You will need to rebase and resolve conflicts :)
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
from
1a6ad6e to
3bc073bCompareMarch 31, 2022 11:22
@potiuk
potiuk
commented
Mar 31, 2022•

edited

Loading

Copy link


Copy Markdown

Member
@potiuk i want to know how we can pass the CI flag in python? In bash we set it as env var. Do we need to pass that as flag when we rewrite in python?
airflow/scripts/ci/libraries/_build_images.sh

Lines 454 to 463
in
c58b5e5
if [[ ${CI}=="true" ]];then
 EXTRA_DOCKER_PROD_BUILD_FLAGS+=( 
"--build-arg""PIP_PROGRESS_BAR=off"
 ) 
fi
if [[ -n"${AIRFLOW_CONSTRAINTS_LOCATION}" ]];then
 extra_docker_ci_flags+=( 
"--build-arg""AIRFLOW_CONSTRAINTS_LOCATION=${AIRFLOW_CONSTRAINTS_LOCATION}"
 ) 
fi
@potiuk Please share your view on this when you get time.
Same as in Bash. Just add the PIP_PROGRESS_BAR==off if CI is set to true. There is no need to pass it further. But in order to do it nicely you can add --ci parameter with CI envvar via click.
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
from
3bc073b to
733bbe1CompareApril 1, 2022 03:51
@Bowrna
Bowrna
commented
Apr 1, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk Added fix for flag upgrade_to_newer_dependencies :)
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
from
733bbe1 to
bb10913CompareApril 1, 2022 06:23
@Bowrna
Bowrna
commented
Apr 1, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk i want to know how we can pass the CI flag in python? In bash we set it as env var. Do we need to pass that as flag when we rewrite in python?
airflow/scripts/ci/libraries/_build_images.sh

Lines 454 to 463
in
c58b5e5
if [[ ${CI}=="true" ]];then
 EXTRA_DOCKER_PROD_BUILD_FLAGS+=( 
"--build-arg""PIP_PROGRESS_BAR=off"
 ) 
fi
if [[ -n"${AIRFLOW_CONSTRAINTS_LOCATION}" ]];then
 extra_docker_ci_flags+=( 
"--build-arg""AIRFLOW_CONSTRAINTS_LOCATION=${AIRFLOW_CONSTRAINTS_LOCATION}"
 ) 
fi
@potiuk Please share your view on this when you get time.
Same as in Bash. Just add the PIP_PROGRESS_BAR==off if CI is set to true. There is no need to pass it further. But in order to do it nicely you can add --ci parameter with CI envvar via click.
@potiuk Added this support too
@potiuk
potiuk
merged commit dc75f5d
into
apache:mainApr 1, 2022
joppevos
joppevos
reviewed
Apr 7, 2022
View reviewed changes
Comment threaddev/breeze/src/airflow_breeze/breeze.py
)
option_ci_flag = click.option(
 '--ci',
 help='Enabling this option will off the pip progress bar',
@joppevosjoppevosApr 7, 2022•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume it should be "...will TURN off the pip..."? :)
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@potiukpotiukpotiuk approved these changes
@ashbashbAwaiting requested review from ashbashb is a code owner
@jedcunninghamjedcunninghamAwaiting requested review from jedcunninghamjedcunningham is a code owner

+1 more reviewer

@joppevosjoppevosjoppevos left review comments
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

area:dev-tools

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

Breeze: Add 'prepare-image-cache' in Breeze


3 participants

@Bowrna@potiuk@joppevos

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Universal Dark Mode - works on any site
(function() {
var enabled = true;
function applyDarkMode() {
if (!enabled) return;
// Create style element if it doesn't exist
var style = document.getElementById('universal-dark-mode-style');
if (!style) {
style = document.createElement('style');
style.id = 'universal-dark-mode-style';
document.head.appendChild(style);
}
// Dark mode CSS - inverts colors but preserves images/video
style.textContent = '
/* Invert everything except media */
html {
filter: invert(1) hue-rotate(180deg) !important;
background: #1a1a2e !important;
}
/* Restore images, videos, iframes, canvas */
img, video, iframe, canvas, svg, picture, [style*="background-image"] {
filter: invert(1) hue-rotate(180deg) !important;
}
/* Preserve specific elements that should not be inverted */
.no-dark-mode, .no-dark-mode *,
[data-theme="light"], [data-theme="light"],
.ace_editor, .ace_editor *,
.CodeMirror, .CodeMirror *,
.monaco-editor, .monaco-editor *,
.markdown-body pre, .markdown-body pre *,
.highlight, .highlight *,
pre code, pre code * {
filter: none !important;
}
/* Fix common UI elements */
.modal, .popup, .dropdown-menu, .tooltip, .popover {
filter: invert(1) hue-rotate(180deg) !important;
background: #2d2d44 !important;
border-color: #444 !important;
}
/* Scrollbars */
::-webkit-scrollbar { background: #1a1a2e !important; }
::-webkit-scrollbar-thumb { background: #444 !important; }
::-webkit-scrollbar-thumb:hover { background: #555 !important; }
/* Selection */
::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
';
}
function removeDarkMode() {
var style = document.getElementById('universal-dark-mode-style');
if (style) style.remove();
}
// Toggle with Alt+Shift+D
document.addEventListener('keydown', function(e) {
if (e.altKey && e.shiftKey && e.key === 'D') {
e.preventDefault();
enabled = !enabled;
if (enabled) {
applyDarkMode();
console.log('[Universal Dark Mode] Enabled');
} else {
removeDarkMode();
console.log('[Universal Dark Mode] Disabled');
}
}
});
// Apply on load
applyDarkMode();
// Re-apply on dynamic content
var observer = new MutationObserver(function(mutations) {
if (enabled && !document.getElementById('universal-dark-mode-style')) {
applyDarkMode();
}
});
observer.observe(document.head, { childList: true });
console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');
})();
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();
ci prepare-build-cache by Bowrna · Pull Request #22344 · apache/airflow · GitHub
Skip to content
ci prepare-build-cache - #22344
Merged
potiuk merged 1 commit into
apache:mainfrom 
Bowrna:prepare-image-cache
Apr 1, 2022
Merged
ci prepare-build-cache#22344
potiuk merged 1 commit into
apache:mainfrom 
Bowrna:prepare-image-cache
Conversation
@Bowrna
@BowrnaBowrna
commented
Mar 17, 2022

Copy link


Copy Markdown

Contributor
closes : #21102

related: #21102
^ Add meaningful description above
Read the Pull Request Guidelines for more information.

In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.

In case of a new dependency, check compliance with the ASF 3rd Party License Policy.

In case of backwards incompatible changes please leave a note in UPDATING.md.
@Bowrna
Bowrna
commented
Mar 30, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk
--cache:max should be added also in the CI image (because it is multi-segment image now)
#22492 (comment)
I will add this as part of this PR.
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
from
38ffc3e to
0f32a0aCompareMarch 30, 2022 06:52
@Bowrna
Bowrna
commented
Mar 30, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk i want to know how we can pass the CI flag in python? In bash we set it as env var. Do we need to pass that as flag when we rewrite in python?
airflow/scripts/ci/libraries/_build_images.sh

Lines 454 to 463
in
c58b5e5
if [[ ${CI}=="true" ]];then
 EXTRA_DOCKER_PROD_BUILD_FLAGS+=( 
"--build-arg""PIP_PROGRESS_BAR=off"
 ) 
fi
if [[ -n"${AIRFLOW_CONSTRAINTS_LOCATION}" ]];then
 extra_docker_ci_flags+=( 
"--build-arg""AIRFLOW_CONSTRAINTS_LOCATION=${AIRFLOW_CONSTRAINTS_LOCATION}"
 ) 
fi
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
2 times, most recently
from
d553191 to
6cc513bCompareMarch 30, 2022 11:45
@Bowrna
Bowrna
marked this pull request as ready for review
March 30, 2022 11:45
@potiukpotiuk
mentioned this pull request
Mar 30, 2022
 Merged

@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
2 times, most recently
from
5830eee to
4a0260aCompareMarch 31, 2022 02:34
@Bowrna
Bowrna
commented
Mar 31, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk I made changes specified in this #22618

Please verify it
@Bowrna
Bowrna
commented
Mar 31, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk i want to know how we can pass the CI flag in python? In bash we set it as env var. Do we need to pass that as flag when we rewrite in python?
airflow/scripts/ci/libraries/_build_images.sh

Lines 454 to 463
in
c58b5e5
if [[ ${CI}=="true" ]];then
 EXTRA_DOCKER_PROD_BUILD_FLAGS+=( 
"--build-arg""PIP_PROGRESS_BAR=off"
 ) 
fi
if [[ -n"${AIRFLOW_CONSTRAINTS_LOCATION}" ]];then
 extra_docker_ci_flags+=( 
"--build-arg""AIRFLOW_CONSTRAINTS_LOCATION=${AIRFLOW_CONSTRAINTS_LOCATION}"
 ) 
fi
@potiuk Please share your view on this when you get time.
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
from
4a0260a to
1a6ad6eCompareMarch 31, 2022 08:07
potiuk
potiuk
reviewed
Mar 31, 2022
View reviewed changes
Comment threaddev/breeze/src/airflow_breeze/breeze.py
Outdated

@potiukpotiukMar 31, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


Suggested change

upgrade_to_newer_dependencies: bool,
upgrade_to_newer_dependencies: bool,
This should be Optional[str] actually :)
@potiukpotiukMar 31, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was the fix I made here -> #22597
@BowrnaBowrnaMar 31, 2022

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@potiuk Is it Optional[str] for prod image too?
@potiukpotiukMar 31, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep
@potiukpotiukMar 31, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added more fixes on the parameter in #22649 :). You will need to rebase and resolve conflicts :)
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
from
1a6ad6e to
3bc073bCompareMarch 31, 2022 11:22
@potiuk
potiuk
commented
Mar 31, 2022•

edited

Loading

Copy link


Copy Markdown

Member
@potiuk i want to know how we can pass the CI flag in python? In bash we set it as env var. Do we need to pass that as flag when we rewrite in python?
airflow/scripts/ci/libraries/_build_images.sh

Lines 454 to 463
in
c58b5e5
if [[ ${CI}=="true" ]];then
 EXTRA_DOCKER_PROD_BUILD_FLAGS+=( 
"--build-arg""PIP_PROGRESS_BAR=off"
 ) 
fi
if [[ -n"${AIRFLOW_CONSTRAINTS_LOCATION}" ]];then
 extra_docker_ci_flags+=( 
"--build-arg""AIRFLOW_CONSTRAINTS_LOCATION=${AIRFLOW_CONSTRAINTS_LOCATION}"
 ) 
fi
@potiuk Please share your view on this when you get time.
Same as in Bash. Just add the PIP_PROGRESS_BAR==off if CI is set to true. There is no need to pass it further. But in order to do it nicely you can add --ci parameter with CI envvar via click.
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
from
3bc073b to
733bbe1CompareApril 1, 2022 03:51
@Bowrna
Bowrna
commented
Apr 1, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk Added fix for flag upgrade_to_newer_dependencies :)
@Bowrna
Bowrnaforce-pushed
the
prepare-image-cache
branch
from
733bbe1 to
bb10913CompareApril 1, 2022 06:23
@Bowrna
Bowrna
commented
Apr 1, 2022

Copy link


Copy Markdown

ContributorAuthor
@potiuk i want to know how we can pass the CI flag in python? In bash we set it as env var. Do we need to pass that as flag when we rewrite in python?
airflow/scripts/ci/libraries/_build_images.sh

Lines 454 to 463
in
c58b5e5
if [[ ${CI}=="true" ]];then
 EXTRA_DOCKER_PROD_BUILD_FLAGS+=( 
"--build-arg""PIP_PROGRESS_BAR=off"
 ) 
fi
if [[ -n"${AIRFLOW_CONSTRAINTS_LOCATION}" ]];then
 extra_docker_ci_flags+=( 
"--build-arg""AIRFLOW_CONSTRAINTS_LOCATION=${AIRFLOW_CONSTRAINTS_LOCATION}"
 ) 
fi
@potiuk Please share your view on this when you get time.
Same as in Bash. Just add the PIP_PROGRESS_BAR==off if CI is set to true. There is no need to pass it further. But in order to do it nicely you can add --ci parameter with CI envvar via click.
@potiuk Added this support too
@potiuk
potiuk
merged commit dc75f5d
into
apache:mainApr 1, 2022
joppevos
joppevos
reviewed
Apr 7, 2022
View reviewed changes
Comment threaddev/breeze/src/airflow_breeze/breeze.py
)
option_ci_flag = click.option(
 '--ci',
 help='Enabling this option will off the pip progress bar',
@joppevosjoppevosApr 7, 2022•

edited

Loading

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume it should be "...will TURN off the pip..."? :)
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@potiukpotiukpotiuk approved these changes
@ashbashbAwaiting requested review from ashbashb is a code owner
@jedcunninghamjedcunninghamAwaiting requested review from jedcunninghamjedcunningham is a code owner

+1 more reviewer

@joppevosjoppevosjoppevos left review comments
Reviewers whose approvals may not affect merge requirements

Assignees


No one assigned


Labels

area:dev-tools

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

Breeze: Add 'prepare-image-cache' in Breeze


3 participants

@Bowrna@potiuk@joppevos