fix typo in dag_schedule_dataset_alias_reference migration file - #43314

Merged
Lee-W merged 1 commit into
apache:v2-10-testfrom
astronomer:fix-2-10-migration-typo
Oct 26, 2024
Merged

fix typo in dag_schedule_dataset_alias_reference migration file#43314
Lee-W merged 1 commit into
apache:v2-10-testfrom
astronomer:fix-2-10-migration-typo

Conversation

@Lee-W

@Lee-WLee-W commented Oct 23, 2024

Copy link
Copy Markdown
Member

backport https://github.com/apache/airflow/pull/43245/files#diff-465bfca34e4030692c541248495b5abfe4deea606db126241dade041746fbd60

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@boring-cyborgboring-cyborgBot added the area:db-migrations PRs with DB migration label Oct 23, 2024
This was referenced Oct 23, 2024
Merged
Closed
@shahar1

shahar1 commented Oct 23, 2024•
edited
Loading

Copy link
Copy Markdown
Contributor

Static checks fail for some reason

@Lee-W
Lee-Wforce-pushed the fix-2-10-migration-typo branch from bd3f3b1 to 4cbf70dCompareOctober 23, 2024 14:40
@Lee-W
Lee-W requested a review from potiuk as a code ownerOctober 23, 2024 14:40
kaxil
kaxil requested changes Oct 23, 2024
View reviewed changes

@kaxilkaxil left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm not so sure about just renaming constraints. What about folks who are already on this version?

@Lee-W

Lee-W commented Oct 24, 2024

Copy link
Copy Markdown
MemberAuthor

Hmm not so sure about just renaming constraints. What about folks who are already on this version?

These reflect how the actual model is defined. As no feature is expected to be added to this version, I think these columns won't change in the future either.

@kaxil

kaxil commented Oct 24, 2024•
edited
Loading

Copy link
Copy Markdown
Member

Hmm not so sure about just renaming constraints. What about folks who are already on this version?

These reflect how the actual model is defined. As no feature is expected to be added to this version, I think these columns won't change in the future either.

If you download Airflow 2.8.0, run airflow db migrate, upgrade to 2.10.2 and run airflow db migrate again with Postgres or MySQL backend, the constraints are created with the old names. Example:

image

If we merge and ship this change, we will have some users with dsdar_dag_fkey as name of constraint and some on dsdar_dataset_fkey. In future, if we need to access a constraint in another migration via name that won't work for some users.

This is true for folks upgrading from let's say Airflow 2.8.0 to 2.10.2 --> for those folks tables are created from "migration" . It is only for new Airflow 2.10.0 user where all tables are just created from ORM.

image

@potiuk

potiuk commented Oct 24, 2024

Copy link
Copy Markdown
Member

Yep. Agree with @kaxil. That should be a separate migration. Also the original cahnge in main should be reverted and re-applied after that because people will go eventually 2.x -> 2.11 -> 3.x and things will break for them.

@Lee-W
Lee-Wforce-pushed the fix-2-10-migration-typo branch from 4cbf70d to e181cb0CompareOctober 25, 2024 05:00
@Lee-W

Lee-W commented Oct 25, 2024

Copy link
Copy Markdown
MemberAuthor

Got it. makes sense. I will create a fix pr to the main as well

@Lee-W
Lee-Wforce-pushed the fix-2-10-migration-typo branch from e181cb0 to 836d590CompareOctober 25, 2024 05:35
@Lee-WLee-W mentioned this pull request Oct 25, 2024
Merged
@Lee-W
Lee-Wforce-pushed the fix-2-10-migration-typo branch from 836d590 to 915c398CompareOctober 25, 2024 05:38
@Lee-W
Lee-W requested a review from kaxilOctober 25, 2024 08:32
kaxil
kaxil reviewed Oct 25, 2024
View reviewed changes
Comment threadairflow/migrations/versions/0152_2_10_3_fix_dag_schedule_dataset_alias_reference_naming.py Outdated
kaxil
kaxil reviewed Oct 25, 2024
View reviewed changes
Comment threadairflow/models/dataset.py
@Lee-W
Lee-Wforce-pushed the fix-2-10-migration-typo branch from 915c398 to 45fc2e8CompareOctober 25, 2024 15:53
kaxil
kaxil reviewed Oct 25, 2024
View reviewed changes
Comment threadairflow/migrations/versions/0152_2_10_3_fix_dag_schedule_dataset_alias_reference_naming.py Outdated
kaxil
kaxil approved these changes Oct 25, 2024
View reviewed changes

@kaxilkaxil left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added 1 comment for downgrade but lgtm otherwise

https://github.com/apache/airflow/pull/43314/files#r1816982650

@Lee-W
Lee-Wforce-pushed the fix-2-10-migration-typo branch from 45fc2e8 to 062f65eCompareOctober 25, 2024 16:16
@potiukpotiuk added this to the Airflow 2.10.3 milestone Oct 25, 2024
@potiuk

potiuk commented Oct 25, 2024

Copy link
Copy Markdown
Member

Also I guess it should be back-ported ?

@Lee-W
Lee-Wforce-pushed the fix-2-10-migration-typo branch from 062f65e to a31600dCompareOctober 26, 2024 03:26
@Lee-W

Lee-W commented Oct 26, 2024

Copy link
Copy Markdown
MemberAuthor

Also I guess it should be back-ported ?

This is for 2.10.x, but I guess what you mean is for main and yes, the PR is here #43373

@Lee-W
Lee-W merged commit 9a868e0 into apache:v2-10-testOct 26, 2024
@Lee-W
Lee-W deleted the fix-2-10-migration-typo branch October 26, 2024 15:05
@Lee-WLee-W self-assigned this Oct 28, 2024
utkarsharma2 pushed a commit that referenced this pull request Oct 28, 2024
@Lee-W@utkarsharma2
rename ill-named constraints in dag_schedule_dataset_alias_reference …
6cad40e
…table #43314
(cherry picked from commit 9a868e0)
@utkarsharma2utkarsharma2 added the changelog:skip Changes that should be skipped from the changelog (CI, tests, etc..) label Oct 28, 2024
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@potiukpotiukpotiuk approved these changes
@bbovenzibbovenzibbovenzi approved these changes
@kaxilkaxilkaxil approved these changes
@hussein-awalahussein-awalahussein-awala approved these changes
@shahar1shahar1shahar1 approved these changes
@XD-DENGXD-DENGAwaiting requested review from XD-DENGXD-DENG is a code owner
@ashbashbAwaiting requested review from ashbashb is a code owner

Assignees

@Lee-WLee-W

Labels

area:db-migrationsPRs with DB migrationchangelog:skipChanges that should be skipped from the changelog (CI, tests, etc..)

Projects

No open projects
AIP-74 Introducing Data Assets
Status: Done

Milestone

Airflow 2.10.3

Development

Successfully merging this pull request may close these issues.

7 participants

@Lee-W@shahar1@kaxil@potiuk@bbovenzi@hussein-awala@utkarsharma2
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
fix typo in dag_schedule_dataset_alias_reference migration file - #43314
Merged
Lee-W merged 1 commit into
apache:v2-10-testfrom 
astronomer:fix-2-10-migration-typo
Oct 26, 2024
Merged
fix typo in dag_schedule_dataset_alias_reference migration file#43314
Lee-W merged 1 commit into
apache:v2-10-testfrom 
astronomer:fix-2-10-migration-typo
Conversation
@Lee-W
@Lee-WLee-W
commented
Oct 23, 2024

Copy link


Copy Markdown

Member
backport https://github.com/apache/airflow/pull/43245/files#diff-465bfca34e4030692c541248495b5abfe4deea606db126241dade041746fbd60
^ Add meaningful description above

Read the Pull Request Guidelines for more information.

In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.

In case of a new dependency, check compliance with the ASF 3rd Party License Policy.

In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.
@boring-cyborgboring-cyborgBot
added
the 
area:db-migrations
PRs with DB migration
label
Oct 23, 2024

This was referenced Oct 23, 2024
 Merged


Closed

@shahar1
shahar1
commented
Oct 23, 2024•

edited

Loading

Copy link


Copy Markdown

Contributor
Static checks fail for some reason
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
bd3f3b1 to
4cbf70dCompareOctober 23, 2024 14:40
@Lee-W
Lee-W
requested a review
from potiuk
as a code ownerOctober 23, 2024 14:40
kaxil
kaxil
requested changes
Oct 23, 2024
View reviewed changes
@kaxilkaxil
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm not so sure about just renaming constraints. What about folks who are already on this version?
@Lee-W
Lee-W
commented
Oct 24, 2024

Copy link


Copy Markdown

MemberAuthor
Hmm not so sure about just renaming constraints. What about folks who are already on this version?
These reflect how the actual model is defined. As no feature is expected to be added to this version, I think these columns won't change in the future either.
@kaxil
kaxil
commented
Oct 24, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Hmm not so sure about just renaming constraints. What about folks who are already on this version?
These reflect how the actual model is defined. As no feature is expected to be added to this version, I think these columns won't change in the future either.
If you download Airflow 2.8.0, run airflow db migrate, upgrade to 2.10.2 and run airflow db migrate again with Postgres or MySQL backend, the constraints are created with the old names. Example:
image
If we merge and ship this change, we will have some users with dsdar_dag_fkey as name of constraint and some on dsdar_dataset_fkey. In future, if we need to access a constraint in another migration via name that won't work for some users.
This is true for folks upgrading from let's say Airflow 2.8.0 to 2.10.2 --> for those folks tables are created from "migration" . It is only for new Airflow 2.10.0 user where all tables are just created from ORM.
image
@potiuk
potiuk
commented
Oct 24, 2024

Copy link


Copy Markdown

Member
Yep. Agree with @kaxil. That should be a separate migration. Also the original cahnge in main should be reverted and re-applied after that because people will go eventually 2.x -> 2.11 -> 3.x and things will break for them.
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
4cbf70d to
e181cb0CompareOctober 25, 2024 05:00
@Lee-W
Lee-W
commented
Oct 25, 2024

Copy link


Copy Markdown

MemberAuthor
Got it. makes sense. I will create a fix pr to the main as well
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
e181cb0 to
836d590CompareOctober 25, 2024 05:35
@Lee-WLee-W
mentioned this pull request
Oct 25, 2024
 Merged

@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
836d590 to
915c398CompareOctober 25, 2024 05:38
@Lee-W
Lee-W
requested a review
from kaxilOctober 25, 2024 08:32
kaxil
kaxil
reviewed
Oct 25, 2024
View reviewed changes
Comment threadairflow/migrations/versions/0152_2_10_3_fix_dag_schedule_dataset_alias_reference_naming.py
Outdated

kaxil
kaxil
reviewed
Oct 25, 2024
View reviewed changes
Comment threadairflow/models/dataset.py
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
915c398 to
45fc2e8CompareOctober 25, 2024 15:53
kaxil
kaxil
reviewed
Oct 25, 2024
View reviewed changes
Comment threadairflow/migrations/versions/0152_2_10_3_fix_dag_schedule_dataset_alias_reference_naming.py
Outdated

kaxil
kaxil
approved these changes
Oct 25, 2024
View reviewed changes
@kaxilkaxil
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added 1 comment for downgrade but lgtm otherwise
https://github.com/apache/airflow/pull/43314/files#r1816982650
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
45fc2e8 to
062f65eCompareOctober 25, 2024 16:16
@potiukpotiuk
added this to the Airflow 2.10.3 milestone
Oct 25, 2024
@potiuk
potiuk
commented
Oct 25, 2024

Copy link


Copy Markdown

Member
Also I guess it should be back-ported ?
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
062f65e to
a31600dCompareOctober 26, 2024 03:26
@Lee-W
Lee-W
commented
Oct 26, 2024

Copy link


Copy Markdown

MemberAuthor
Also I guess it should be back-ported ?
This is for 2.10.x, but I guess what you mean is for main and yes, the PR is here #43373
@Lee-W
Lee-W
merged commit 9a868e0
into
apache:v2-10-testOct 26, 2024
@Lee-W
Lee-W
deleted the

fix-2-10-migration-typo

branch
October 26, 2024 15:05
@Lee-WLee-W
self-assigned this
Oct 28, 2024
utkarsharma2 pushed a commit
that referenced
this pull request
Oct 28, 2024
@Lee-W@utkarsharma2
rename ill-named constraints in dag_schedule_dataset_alias_reference …
6cad40e
…table #43314
(cherry picked from commit 9a868e0)
@utkarsharma2utkarsharma2
added
the 
changelog:skip
Changes that should be skipped from the changelog (CI, tests, etc..)
label
Oct 28, 2024
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@potiukpotiukpotiuk approved these changes
@bbovenzibbovenzibbovenzi approved these changes
@kaxilkaxilkaxil approved these changes
@hussein-awalahussein-awalahussein-awala approved these changes
@shahar1shahar1shahar1 approved these changes
@XD-DENGXD-DENGAwaiting requested review from XD-DENGXD-DENG is a code owner
@ashbashbAwaiting requested review from ashbashb is a code owner

Assignees

@Lee-WLee-W

Labels

area:db-migrationsPRs with DB migrationchangelog:skipChanges that should be skipped from the changelog (CI, tests, etc..)

Projects

No open projects
AIP-74 Introducing Data Assets

Status: Done


Milestone

Airflow 2.10.3

Development

Successfully merging this pull request may close these issues.

7 participants

@Lee-W@shahar1@kaxil@potiuk@bbovenzi@hussein-awala@utkarsharma2

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
fix typo in dag_schedule_dataset_alias_reference migration file - #43314
Merged
Lee-W merged 1 commit into
apache:v2-10-testfrom 
astronomer:fix-2-10-migration-typo
Oct 26, 2024
Merged
fix typo in dag_schedule_dataset_alias_reference migration file#43314
Lee-W merged 1 commit into
apache:v2-10-testfrom 
astronomer:fix-2-10-migration-typo
Conversation
@Lee-W
@Lee-WLee-W
commented
Oct 23, 2024

Copy link


Copy Markdown

Member
backport https://github.com/apache/airflow/pull/43245/files#diff-465bfca34e4030692c541248495b5abfe4deea606db126241dade041746fbd60
^ Add meaningful description above

Read the Pull Request Guidelines for more information.

In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.

In case of a new dependency, check compliance with the ASF 3rd Party License Policy.

In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.
@boring-cyborgboring-cyborgBot
added
the 
area:db-migrations
PRs with DB migration
label
Oct 23, 2024

This was referenced Oct 23, 2024
 Merged


Closed

@shahar1
shahar1
commented
Oct 23, 2024•

edited

Loading

Copy link


Copy Markdown

Contributor
Static checks fail for some reason
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
bd3f3b1 to
4cbf70dCompareOctober 23, 2024 14:40
@Lee-W
Lee-W
requested a review
from potiuk
as a code ownerOctober 23, 2024 14:40
kaxil
kaxil
requested changes
Oct 23, 2024
View reviewed changes
@kaxilkaxil
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm not so sure about just renaming constraints. What about folks who are already on this version?
@Lee-W
Lee-W
commented
Oct 24, 2024

Copy link


Copy Markdown

MemberAuthor
Hmm not so sure about just renaming constraints. What about folks who are already on this version?
These reflect how the actual model is defined. As no feature is expected to be added to this version, I think these columns won't change in the future either.
@kaxil
kaxil
commented
Oct 24, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Hmm not so sure about just renaming constraints. What about folks who are already on this version?
These reflect how the actual model is defined. As no feature is expected to be added to this version, I think these columns won't change in the future either.
If you download Airflow 2.8.0, run airflow db migrate, upgrade to 2.10.2 and run airflow db migrate again with Postgres or MySQL backend, the constraints are created with the old names. Example:
image
If we merge and ship this change, we will have some users with dsdar_dag_fkey as name of constraint and some on dsdar_dataset_fkey. In future, if we need to access a constraint in another migration via name that won't work for some users.
This is true for folks upgrading from let's say Airflow 2.8.0 to 2.10.2 --> for those folks tables are created from "migration" . It is only for new Airflow 2.10.0 user where all tables are just created from ORM.
image
@potiuk
potiuk
commented
Oct 24, 2024

Copy link


Copy Markdown

Member
Yep. Agree with @kaxil. That should be a separate migration. Also the original cahnge in main should be reverted and re-applied after that because people will go eventually 2.x -> 2.11 -> 3.x and things will break for them.
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
4cbf70d to
e181cb0CompareOctober 25, 2024 05:00
@Lee-W
Lee-W
commented
Oct 25, 2024

Copy link


Copy Markdown

MemberAuthor
Got it. makes sense. I will create a fix pr to the main as well
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
e181cb0 to
836d590CompareOctober 25, 2024 05:35
@Lee-WLee-W
mentioned this pull request
Oct 25, 2024
 Merged

@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
836d590 to
915c398CompareOctober 25, 2024 05:38
@Lee-W
Lee-W
requested a review
from kaxilOctober 25, 2024 08:32
kaxil
kaxil
reviewed
Oct 25, 2024
View reviewed changes
Comment threadairflow/migrations/versions/0152_2_10_3_fix_dag_schedule_dataset_alias_reference_naming.py
Outdated

kaxil
kaxil
reviewed
Oct 25, 2024
View reviewed changes
Comment threadairflow/models/dataset.py
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
915c398 to
45fc2e8CompareOctober 25, 2024 15:53
kaxil
kaxil
reviewed
Oct 25, 2024
View reviewed changes
Comment threadairflow/migrations/versions/0152_2_10_3_fix_dag_schedule_dataset_alias_reference_naming.py
Outdated

kaxil
kaxil
approved these changes
Oct 25, 2024
View reviewed changes
@kaxilkaxil
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added 1 comment for downgrade but lgtm otherwise
https://github.com/apache/airflow/pull/43314/files#r1816982650
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
45fc2e8 to
062f65eCompareOctober 25, 2024 16:16
@potiukpotiuk
added this to the Airflow 2.10.3 milestone
Oct 25, 2024
@potiuk
potiuk
commented
Oct 25, 2024

Copy link


Copy Markdown

Member
Also I guess it should be back-ported ?
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
062f65e to
a31600dCompareOctober 26, 2024 03:26
@Lee-W
Lee-W
commented
Oct 26, 2024

Copy link


Copy Markdown

MemberAuthor
Also I guess it should be back-ported ?
This is for 2.10.x, but I guess what you mean is for main and yes, the PR is here #43373
@Lee-W
Lee-W
merged commit 9a868e0
into
apache:v2-10-testOct 26, 2024
@Lee-W
Lee-W
deleted the

fix-2-10-migration-typo

branch
October 26, 2024 15:05
@Lee-WLee-W
self-assigned this
Oct 28, 2024
utkarsharma2 pushed a commit
that referenced
this pull request
Oct 28, 2024
@Lee-W@utkarsharma2
rename ill-named constraints in dag_schedule_dataset_alias_reference …
6cad40e
…table #43314
(cherry picked from commit 9a868e0)
@utkarsharma2utkarsharma2
added
the 
changelog:skip
Changes that should be skipped from the changelog (CI, tests, etc..)
label
Oct 28, 2024
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@potiukpotiukpotiuk approved these changes
@bbovenzibbovenzibbovenzi approved these changes
@kaxilkaxilkaxil approved these changes
@hussein-awalahussein-awalahussein-awala approved these changes
@shahar1shahar1shahar1 approved these changes
@XD-DENGXD-DENGAwaiting requested review from XD-DENGXD-DENG is a code owner
@ashbashbAwaiting requested review from ashbashb is a code owner

Assignees

@Lee-WLee-W

Labels

area:db-migrationsPRs with DB migrationchangelog:skipChanges that should be skipped from the changelog (CI, tests, etc..)

Projects

No open projects
AIP-74 Introducing Data Assets

Status: Done


Milestone

Airflow 2.10.3

Development

Successfully merging this pull request may close these issues.

7 participants

@Lee-W@shahar1@kaxil@potiuk@bbovenzi@hussein-awala@utkarsharma2

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
fix typo in dag_schedule_dataset_alias_reference migration file - #43314
Merged
Lee-W merged 1 commit into
apache:v2-10-testfrom 
astronomer:fix-2-10-migration-typo
Oct 26, 2024
Merged
fix typo in dag_schedule_dataset_alias_reference migration file#43314
Lee-W merged 1 commit into
apache:v2-10-testfrom 
astronomer:fix-2-10-migration-typo
Conversation
@Lee-W
@Lee-WLee-W
commented
Oct 23, 2024

Copy link


Copy Markdown

Member
backport https://github.com/apache/airflow/pull/43245/files#diff-465bfca34e4030692c541248495b5abfe4deea606db126241dade041746fbd60
^ Add meaningful description above

Read the Pull Request Guidelines for more information.

In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.

In case of a new dependency, check compliance with the ASF 3rd Party License Policy.

In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.
@boring-cyborgboring-cyborgBot
added
the 
area:db-migrations
PRs with DB migration
label
Oct 23, 2024

This was referenced Oct 23, 2024
 Merged


Closed

@shahar1
shahar1
commented
Oct 23, 2024•

edited

Loading

Copy link


Copy Markdown

Contributor
Static checks fail for some reason
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
bd3f3b1 to
4cbf70dCompareOctober 23, 2024 14:40
@Lee-W
Lee-W
requested a review
from potiuk
as a code ownerOctober 23, 2024 14:40
kaxil
kaxil
requested changes
Oct 23, 2024
View reviewed changes
@kaxilkaxil
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm not so sure about just renaming constraints. What about folks who are already on this version?
@Lee-W
Lee-W
commented
Oct 24, 2024

Copy link


Copy Markdown

MemberAuthor
Hmm not so sure about just renaming constraints. What about folks who are already on this version?
These reflect how the actual model is defined. As no feature is expected to be added to this version, I think these columns won't change in the future either.
@kaxil
kaxil
commented
Oct 24, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Hmm not so sure about just renaming constraints. What about folks who are already on this version?
These reflect how the actual model is defined. As no feature is expected to be added to this version, I think these columns won't change in the future either.
If you download Airflow 2.8.0, run airflow db migrate, upgrade to 2.10.2 and run airflow db migrate again with Postgres or MySQL backend, the constraints are created with the old names. Example:
image
If we merge and ship this change, we will have some users with dsdar_dag_fkey as name of constraint and some on dsdar_dataset_fkey. In future, if we need to access a constraint in another migration via name that won't work for some users.
This is true for folks upgrading from let's say Airflow 2.8.0 to 2.10.2 --> for those folks tables are created from "migration" . It is only for new Airflow 2.10.0 user where all tables are just created from ORM.
image
@potiuk
potiuk
commented
Oct 24, 2024

Copy link


Copy Markdown

Member
Yep. Agree with @kaxil. That should be a separate migration. Also the original cahnge in main should be reverted and re-applied after that because people will go eventually 2.x -> 2.11 -> 3.x and things will break for them.
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
4cbf70d to
e181cb0CompareOctober 25, 2024 05:00
@Lee-W
Lee-W
commented
Oct 25, 2024

Copy link


Copy Markdown

MemberAuthor
Got it. makes sense. I will create a fix pr to the main as well
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
e181cb0 to
836d590CompareOctober 25, 2024 05:35
@Lee-WLee-W
mentioned this pull request
Oct 25, 2024
 Merged

@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
836d590 to
915c398CompareOctober 25, 2024 05:38
@Lee-W
Lee-W
requested a review
from kaxilOctober 25, 2024 08:32
kaxil
kaxil
reviewed
Oct 25, 2024
View reviewed changes
Comment threadairflow/migrations/versions/0152_2_10_3_fix_dag_schedule_dataset_alias_reference_naming.py
Outdated

kaxil
kaxil
reviewed
Oct 25, 2024
View reviewed changes
Comment threadairflow/models/dataset.py
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
915c398 to
45fc2e8CompareOctober 25, 2024 15:53
kaxil
kaxil
reviewed
Oct 25, 2024
View reviewed changes
Comment threadairflow/migrations/versions/0152_2_10_3_fix_dag_schedule_dataset_alias_reference_naming.py
Outdated

kaxil
kaxil
approved these changes
Oct 25, 2024
View reviewed changes
@kaxilkaxil
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added 1 comment for downgrade but lgtm otherwise
https://github.com/apache/airflow/pull/43314/files#r1816982650
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
45fc2e8 to
062f65eCompareOctober 25, 2024 16:16
@potiukpotiuk
added this to the Airflow 2.10.3 milestone
Oct 25, 2024
@potiuk
potiuk
commented
Oct 25, 2024

Copy link


Copy Markdown

Member
Also I guess it should be back-ported ?
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
062f65e to
a31600dCompareOctober 26, 2024 03:26
@Lee-W
Lee-W
commented
Oct 26, 2024

Copy link


Copy Markdown

MemberAuthor
Also I guess it should be back-ported ?
This is for 2.10.x, but I guess what you mean is for main and yes, the PR is here #43373
@Lee-W
Lee-W
merged commit 9a868e0
into
apache:v2-10-testOct 26, 2024
@Lee-W
Lee-W
deleted the

fix-2-10-migration-typo

branch
October 26, 2024 15:05
@Lee-WLee-W
self-assigned this
Oct 28, 2024
utkarsharma2 pushed a commit
that referenced
this pull request
Oct 28, 2024
@Lee-W@utkarsharma2
rename ill-named constraints in dag_schedule_dataset_alias_reference …
6cad40e
…table #43314
(cherry picked from commit 9a868e0)
@utkarsharma2utkarsharma2
added
the 
changelog:skip
Changes that should be skipped from the changelog (CI, tests, etc..)
label
Oct 28, 2024
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@potiukpotiukpotiuk approved these changes
@bbovenzibbovenzibbovenzi approved these changes
@kaxilkaxilkaxil approved these changes
@hussein-awalahussein-awalahussein-awala approved these changes
@shahar1shahar1shahar1 approved these changes
@XD-DENGXD-DENGAwaiting requested review from XD-DENGXD-DENG is a code owner
@ashbashbAwaiting requested review from ashbashb is a code owner

Assignees

@Lee-WLee-W

Labels

area:db-migrationsPRs with DB migrationchangelog:skipChanges that should be skipped from the changelog (CI, tests, etc..)

Projects

No open projects
AIP-74 Introducing Data Assets

Status: Done


Milestone

Airflow 2.10.3

Development

Successfully merging this pull request may close these issues.

7 participants

@Lee-W@shahar1@kaxil@potiuk@bbovenzi@hussein-awala@utkarsharma2

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
fix typo in dag_schedule_dataset_alias_reference migration file - #43314
Merged
Lee-W merged 1 commit into
apache:v2-10-testfrom 
astronomer:fix-2-10-migration-typo
Oct 26, 2024
Merged
fix typo in dag_schedule_dataset_alias_reference migration file#43314
Lee-W merged 1 commit into
apache:v2-10-testfrom 
astronomer:fix-2-10-migration-typo
Conversation
@Lee-W
@Lee-WLee-W
commented
Oct 23, 2024

Copy link


Copy Markdown

Member
backport https://github.com/apache/airflow/pull/43245/files#diff-465bfca34e4030692c541248495b5abfe4deea606db126241dade041746fbd60
^ Add meaningful description above

Read the Pull Request Guidelines for more information.

In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.

In case of a new dependency, check compliance with the ASF 3rd Party License Policy.

In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.
@boring-cyborgboring-cyborgBot
added
the 
area:db-migrations
PRs with DB migration
label
Oct 23, 2024

This was referenced Oct 23, 2024
 Merged


Closed

@shahar1
shahar1
commented
Oct 23, 2024•

edited

Loading

Copy link


Copy Markdown

Contributor
Static checks fail for some reason
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
bd3f3b1 to
4cbf70dCompareOctober 23, 2024 14:40
@Lee-W
Lee-W
requested a review
from potiuk
as a code ownerOctober 23, 2024 14:40
kaxil
kaxil
requested changes
Oct 23, 2024
View reviewed changes
@kaxilkaxil
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm not so sure about just renaming constraints. What about folks who are already on this version?
@Lee-W
Lee-W
commented
Oct 24, 2024

Copy link


Copy Markdown

MemberAuthor
Hmm not so sure about just renaming constraints. What about folks who are already on this version?
These reflect how the actual model is defined. As no feature is expected to be added to this version, I think these columns won't change in the future either.
@kaxil
kaxil
commented
Oct 24, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Hmm not so sure about just renaming constraints. What about folks who are already on this version?
These reflect how the actual model is defined. As no feature is expected to be added to this version, I think these columns won't change in the future either.
If you download Airflow 2.8.0, run airflow db migrate, upgrade to 2.10.2 and run airflow db migrate again with Postgres or MySQL backend, the constraints are created with the old names. Example:
image
If we merge and ship this change, we will have some users with dsdar_dag_fkey as name of constraint and some on dsdar_dataset_fkey. In future, if we need to access a constraint in another migration via name that won't work for some users.
This is true for folks upgrading from let's say Airflow 2.8.0 to 2.10.2 --> for those folks tables are created from "migration" . It is only for new Airflow 2.10.0 user where all tables are just created from ORM.
image
@potiuk
potiuk
commented
Oct 24, 2024

Copy link


Copy Markdown

Member
Yep. Agree with @kaxil. That should be a separate migration. Also the original cahnge in main should be reverted and re-applied after that because people will go eventually 2.x -> 2.11 -> 3.x and things will break for them.
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
4cbf70d to
e181cb0CompareOctober 25, 2024 05:00
@Lee-W
Lee-W
commented
Oct 25, 2024

Copy link


Copy Markdown

MemberAuthor
Got it. makes sense. I will create a fix pr to the main as well
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
e181cb0 to
836d590CompareOctober 25, 2024 05:35
@Lee-WLee-W
mentioned this pull request
Oct 25, 2024
 Merged

@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
836d590 to
915c398CompareOctober 25, 2024 05:38
@Lee-W
Lee-W
requested a review
from kaxilOctober 25, 2024 08:32
kaxil
kaxil
reviewed
Oct 25, 2024
View reviewed changes
Comment threadairflow/migrations/versions/0152_2_10_3_fix_dag_schedule_dataset_alias_reference_naming.py
Outdated

kaxil
kaxil
reviewed
Oct 25, 2024
View reviewed changes
Comment threadairflow/models/dataset.py
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
915c398 to
45fc2e8CompareOctober 25, 2024 15:53
kaxil
kaxil
reviewed
Oct 25, 2024
View reviewed changes
Comment threadairflow/migrations/versions/0152_2_10_3_fix_dag_schedule_dataset_alias_reference_naming.py
Outdated

kaxil
kaxil
approved these changes
Oct 25, 2024
View reviewed changes
@kaxilkaxil
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added 1 comment for downgrade but lgtm otherwise
https://github.com/apache/airflow/pull/43314/files#r1816982650
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
45fc2e8 to
062f65eCompareOctober 25, 2024 16:16
@potiukpotiuk
added this to the Airflow 2.10.3 milestone
Oct 25, 2024
@potiuk
potiuk
commented
Oct 25, 2024

Copy link


Copy Markdown

Member
Also I guess it should be back-ported ?
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
062f65e to
a31600dCompareOctober 26, 2024 03:26
@Lee-W
Lee-W
commented
Oct 26, 2024

Copy link


Copy Markdown

MemberAuthor
Also I guess it should be back-ported ?
This is for 2.10.x, but I guess what you mean is for main and yes, the PR is here #43373
@Lee-W
Lee-W
merged commit 9a868e0
into
apache:v2-10-testOct 26, 2024
@Lee-W
Lee-W
deleted the

fix-2-10-migration-typo

branch
October 26, 2024 15:05
@Lee-WLee-W
self-assigned this
Oct 28, 2024
utkarsharma2 pushed a commit
that referenced
this pull request
Oct 28, 2024
@Lee-W@utkarsharma2
rename ill-named constraints in dag_schedule_dataset_alias_reference …
6cad40e
…table #43314
(cherry picked from commit 9a868e0)
@utkarsharma2utkarsharma2
added
the 
changelog:skip
Changes that should be skipped from the changelog (CI, tests, etc..)
label
Oct 28, 2024
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@potiukpotiukpotiuk approved these changes
@bbovenzibbovenzibbovenzi approved these changes
@kaxilkaxilkaxil approved these changes
@hussein-awalahussein-awalahussein-awala approved these changes
@shahar1shahar1shahar1 approved these changes
@XD-DENGXD-DENGAwaiting requested review from XD-DENGXD-DENG is a code owner
@ashbashbAwaiting requested review from ashbashb is a code owner

Assignees

@Lee-WLee-W

Labels

area:db-migrationsPRs with DB migrationchangelog:skipChanges that should be skipped from the changelog (CI, tests, etc..)

Projects

No open projects
AIP-74 Introducing Data Assets

Status: Done


Milestone

Airflow 2.10.3

Development

Successfully merging this pull request may close these issues.

7 participants

@Lee-W@shahar1@kaxil@potiuk@bbovenzi@hussein-awala@utkarsharma2

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
fix typo in dag_schedule_dataset_alias_reference migration file - #43314
Merged
Lee-W merged 1 commit into
apache:v2-10-testfrom 
astronomer:fix-2-10-migration-typo
Oct 26, 2024
Merged
fix typo in dag_schedule_dataset_alias_reference migration file#43314
Lee-W merged 1 commit into
apache:v2-10-testfrom 
astronomer:fix-2-10-migration-typo
Conversation
@Lee-W
@Lee-WLee-W
commented
Oct 23, 2024

Copy link


Copy Markdown

Member
backport https://github.com/apache/airflow/pull/43245/files#diff-465bfca34e4030692c541248495b5abfe4deea606db126241dade041746fbd60
^ Add meaningful description above

Read the Pull Request Guidelines for more information.

In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.

In case of a new dependency, check compliance with the ASF 3rd Party License Policy.

In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.
@boring-cyborgboring-cyborgBot
added
the 
area:db-migrations
PRs with DB migration
label
Oct 23, 2024

This was referenced Oct 23, 2024
 Merged


Closed

@shahar1
shahar1
commented
Oct 23, 2024•

edited

Loading

Copy link


Copy Markdown

Contributor
Static checks fail for some reason
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
bd3f3b1 to
4cbf70dCompareOctober 23, 2024 14:40
@Lee-W
Lee-W
requested a review
from potiuk
as a code ownerOctober 23, 2024 14:40
kaxil
kaxil
requested changes
Oct 23, 2024
View reviewed changes
@kaxilkaxil
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm not so sure about just renaming constraints. What about folks who are already on this version?
@Lee-W
Lee-W
commented
Oct 24, 2024

Copy link


Copy Markdown

MemberAuthor
Hmm not so sure about just renaming constraints. What about folks who are already on this version?
These reflect how the actual model is defined. As no feature is expected to be added to this version, I think these columns won't change in the future either.
@kaxil
kaxil
commented
Oct 24, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Hmm not so sure about just renaming constraints. What about folks who are already on this version?
These reflect how the actual model is defined. As no feature is expected to be added to this version, I think these columns won't change in the future either.
If you download Airflow 2.8.0, run airflow db migrate, upgrade to 2.10.2 and run airflow db migrate again with Postgres or MySQL backend, the constraints are created with the old names. Example:
image
If we merge and ship this change, we will have some users with dsdar_dag_fkey as name of constraint and some on dsdar_dataset_fkey. In future, if we need to access a constraint in another migration via name that won't work for some users.
This is true for folks upgrading from let's say Airflow 2.8.0 to 2.10.2 --> for those folks tables are created from "migration" . It is only for new Airflow 2.10.0 user where all tables are just created from ORM.
image
@potiuk
potiuk
commented
Oct 24, 2024

Copy link


Copy Markdown

Member
Yep. Agree with @kaxil. That should be a separate migration. Also the original cahnge in main should be reverted and re-applied after that because people will go eventually 2.x -> 2.11 -> 3.x and things will break for them.
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
4cbf70d to
e181cb0CompareOctober 25, 2024 05:00
@Lee-W
Lee-W
commented
Oct 25, 2024

Copy link


Copy Markdown

MemberAuthor
Got it. makes sense. I will create a fix pr to the main as well
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
e181cb0 to
836d590CompareOctober 25, 2024 05:35
@Lee-WLee-W
mentioned this pull request
Oct 25, 2024
 Merged

@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
836d590 to
915c398CompareOctober 25, 2024 05:38
@Lee-W
Lee-W
requested a review
from kaxilOctober 25, 2024 08:32
kaxil
kaxil
reviewed
Oct 25, 2024
View reviewed changes
Comment threadairflow/migrations/versions/0152_2_10_3_fix_dag_schedule_dataset_alias_reference_naming.py
Outdated

kaxil
kaxil
reviewed
Oct 25, 2024
View reviewed changes
Comment threadairflow/models/dataset.py
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
915c398 to
45fc2e8CompareOctober 25, 2024 15:53
kaxil
kaxil
reviewed
Oct 25, 2024
View reviewed changes
Comment threadairflow/migrations/versions/0152_2_10_3_fix_dag_schedule_dataset_alias_reference_naming.py
Outdated

kaxil
kaxil
approved these changes
Oct 25, 2024
View reviewed changes
@kaxilkaxil
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added 1 comment for downgrade but lgtm otherwise
https://github.com/apache/airflow/pull/43314/files#r1816982650
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
45fc2e8 to
062f65eCompareOctober 25, 2024 16:16
@potiukpotiuk
added this to the Airflow 2.10.3 milestone
Oct 25, 2024
@potiuk
potiuk
commented
Oct 25, 2024

Copy link


Copy Markdown

Member
Also I guess it should be back-ported ?
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
062f65e to
a31600dCompareOctober 26, 2024 03:26
@Lee-W
Lee-W
commented
Oct 26, 2024

Copy link


Copy Markdown

MemberAuthor
Also I guess it should be back-ported ?
This is for 2.10.x, but I guess what you mean is for main and yes, the PR is here #43373
@Lee-W
Lee-W
merged commit 9a868e0
into
apache:v2-10-testOct 26, 2024
@Lee-W
Lee-W
deleted the

fix-2-10-migration-typo

branch
October 26, 2024 15:05
@Lee-WLee-W
self-assigned this
Oct 28, 2024
utkarsharma2 pushed a commit
that referenced
this pull request
Oct 28, 2024
@Lee-W@utkarsharma2
rename ill-named constraints in dag_schedule_dataset_alias_reference …
6cad40e
…table #43314
(cherry picked from commit 9a868e0)
@utkarsharma2utkarsharma2
added
the 
changelog:skip
Changes that should be skipped from the changelog (CI, tests, etc..)
label
Oct 28, 2024
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@potiukpotiukpotiuk approved these changes
@bbovenzibbovenzibbovenzi approved these changes
@kaxilkaxilkaxil approved these changes
@hussein-awalahussein-awalahussein-awala approved these changes
@shahar1shahar1shahar1 approved these changes
@XD-DENGXD-DENGAwaiting requested review from XD-DENGXD-DENG is a code owner
@ashbashbAwaiting requested review from ashbashb is a code owner

Assignees

@Lee-WLee-W

Labels

area:db-migrationsPRs with DB migrationchangelog:skipChanges that should be skipped from the changelog (CI, tests, etc..)

Projects

No open projects
AIP-74 Introducing Data Assets

Status: Done


Milestone

Airflow 2.10.3

Development

Successfully merging this pull request may close these issues.

7 participants

@Lee-W@shahar1@kaxil@potiuk@bbovenzi@hussein-awala@utkarsharma2

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
fix typo in dag_schedule_dataset_alias_reference migration file - #43314
Merged
Lee-W merged 1 commit into
apache:v2-10-testfrom 
astronomer:fix-2-10-migration-typo
Oct 26, 2024
Merged
fix typo in dag_schedule_dataset_alias_reference migration file#43314
Lee-W merged 1 commit into
apache:v2-10-testfrom 
astronomer:fix-2-10-migration-typo
Conversation
@Lee-W
@Lee-WLee-W
commented
Oct 23, 2024

Copy link


Copy Markdown

Member
backport https://github.com/apache/airflow/pull/43245/files#diff-465bfca34e4030692c541248495b5abfe4deea606db126241dade041746fbd60
^ Add meaningful description above

Read the Pull Request Guidelines for more information.

In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.

In case of a new dependency, check compliance with the ASF 3rd Party License Policy.

In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.
@boring-cyborgboring-cyborgBot
added
the 
area:db-migrations
PRs with DB migration
label
Oct 23, 2024

This was referenced Oct 23, 2024
 Merged


Closed

@shahar1
shahar1
commented
Oct 23, 2024•

edited

Loading

Copy link


Copy Markdown

Contributor
Static checks fail for some reason
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
bd3f3b1 to
4cbf70dCompareOctober 23, 2024 14:40
@Lee-W
Lee-W
requested a review
from potiuk
as a code ownerOctober 23, 2024 14:40
kaxil
kaxil
requested changes
Oct 23, 2024
View reviewed changes
@kaxilkaxil
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm not so sure about just renaming constraints. What about folks who are already on this version?
@Lee-W
Lee-W
commented
Oct 24, 2024

Copy link


Copy Markdown

MemberAuthor
Hmm not so sure about just renaming constraints. What about folks who are already on this version?
These reflect how the actual model is defined. As no feature is expected to be added to this version, I think these columns won't change in the future either.
@kaxil
kaxil
commented
Oct 24, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Hmm not so sure about just renaming constraints. What about folks who are already on this version?
These reflect how the actual model is defined. As no feature is expected to be added to this version, I think these columns won't change in the future either.
If you download Airflow 2.8.0, run airflow db migrate, upgrade to 2.10.2 and run airflow db migrate again with Postgres or MySQL backend, the constraints are created with the old names. Example:
image
If we merge and ship this change, we will have some users with dsdar_dag_fkey as name of constraint and some on dsdar_dataset_fkey. In future, if we need to access a constraint in another migration via name that won't work for some users.
This is true for folks upgrading from let's say Airflow 2.8.0 to 2.10.2 --> for those folks tables are created from "migration" . It is only for new Airflow 2.10.0 user where all tables are just created from ORM.
image
@potiuk
potiuk
commented
Oct 24, 2024

Copy link


Copy Markdown

Member
Yep. Agree with @kaxil. That should be a separate migration. Also the original cahnge in main should be reverted and re-applied after that because people will go eventually 2.x -> 2.11 -> 3.x and things will break for them.
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
4cbf70d to
e181cb0CompareOctober 25, 2024 05:00
@Lee-W
Lee-W
commented
Oct 25, 2024

Copy link


Copy Markdown

MemberAuthor
Got it. makes sense. I will create a fix pr to the main as well
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
e181cb0 to
836d590CompareOctober 25, 2024 05:35
@Lee-WLee-W
mentioned this pull request
Oct 25, 2024
 Merged

@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
836d590 to
915c398CompareOctober 25, 2024 05:38
@Lee-W
Lee-W
requested a review
from kaxilOctober 25, 2024 08:32
kaxil
kaxil
reviewed
Oct 25, 2024
View reviewed changes
Comment threadairflow/migrations/versions/0152_2_10_3_fix_dag_schedule_dataset_alias_reference_naming.py
Outdated

kaxil
kaxil
reviewed
Oct 25, 2024
View reviewed changes
Comment threadairflow/models/dataset.py
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
915c398 to
45fc2e8CompareOctober 25, 2024 15:53
kaxil
kaxil
reviewed
Oct 25, 2024
View reviewed changes
Comment threadairflow/migrations/versions/0152_2_10_3_fix_dag_schedule_dataset_alias_reference_naming.py
Outdated

kaxil
kaxil
approved these changes
Oct 25, 2024
View reviewed changes
@kaxilkaxil
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added 1 comment for downgrade but lgtm otherwise
https://github.com/apache/airflow/pull/43314/files#r1816982650
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
45fc2e8 to
062f65eCompareOctober 25, 2024 16:16
@potiukpotiuk
added this to the Airflow 2.10.3 milestone
Oct 25, 2024
@potiuk
potiuk
commented
Oct 25, 2024

Copy link


Copy Markdown

Member
Also I guess it should be back-ported ?
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
062f65e to
a31600dCompareOctober 26, 2024 03:26
@Lee-W
Lee-W
commented
Oct 26, 2024

Copy link


Copy Markdown

MemberAuthor
Also I guess it should be back-ported ?
This is for 2.10.x, but I guess what you mean is for main and yes, the PR is here #43373
@Lee-W
Lee-W
merged commit 9a868e0
into
apache:v2-10-testOct 26, 2024
@Lee-W
Lee-W
deleted the

fix-2-10-migration-typo

branch
October 26, 2024 15:05
@Lee-WLee-W
self-assigned this
Oct 28, 2024
utkarsharma2 pushed a commit
that referenced
this pull request
Oct 28, 2024
@Lee-W@utkarsharma2
rename ill-named constraints in dag_schedule_dataset_alias_reference …
6cad40e
…table #43314
(cherry picked from commit 9a868e0)
@utkarsharma2utkarsharma2
added
the 
changelog:skip
Changes that should be skipped from the changelog (CI, tests, etc..)
label
Oct 28, 2024
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@potiukpotiukpotiuk approved these changes
@bbovenzibbovenzibbovenzi approved these changes
@kaxilkaxilkaxil approved these changes
@hussein-awalahussein-awalahussein-awala approved these changes
@shahar1shahar1shahar1 approved these changes
@XD-DENGXD-DENGAwaiting requested review from XD-DENGXD-DENG is a code owner
@ashbashbAwaiting requested review from ashbashb is a code owner

Assignees

@Lee-WLee-W

Labels

area:db-migrationsPRs with DB migrationchangelog:skipChanges that should be skipped from the changelog (CI, tests, etc..)

Projects

No open projects
AIP-74 Introducing Data Assets

Status: Done


Milestone

Airflow 2.10.3

Development

Successfully merging this pull request may close these issues.

7 participants

@Lee-W@shahar1@kaxil@potiuk@bbovenzi@hussein-awala@utkarsharma2

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
fix typo in dag_schedule_dataset_alias_reference migration file - #43314
Merged
Lee-W merged 1 commit into
apache:v2-10-testfrom 
astronomer:fix-2-10-migration-typo
Oct 26, 2024
Merged
fix typo in dag_schedule_dataset_alias_reference migration file#43314
Lee-W merged 1 commit into
apache:v2-10-testfrom 
astronomer:fix-2-10-migration-typo
Conversation
@Lee-W
@Lee-WLee-W
commented
Oct 23, 2024

Copy link


Copy Markdown

Member
backport https://github.com/apache/airflow/pull/43245/files#diff-465bfca34e4030692c541248495b5abfe4deea606db126241dade041746fbd60
^ Add meaningful description above

Read the Pull Request Guidelines for more information.

In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.

In case of a new dependency, check compliance with the ASF 3rd Party License Policy.

In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.
@boring-cyborgboring-cyborgBot
added
the 
area:db-migrations
PRs with DB migration
label
Oct 23, 2024

This was referenced Oct 23, 2024
 Merged


Closed

@shahar1
shahar1
commented
Oct 23, 2024•

edited

Loading

Copy link


Copy Markdown

Contributor
Static checks fail for some reason
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
bd3f3b1 to
4cbf70dCompareOctober 23, 2024 14:40
@Lee-W
Lee-W
requested a review
from potiuk
as a code ownerOctober 23, 2024 14:40
kaxil
kaxil
requested changes
Oct 23, 2024
View reviewed changes
@kaxilkaxil
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm not so sure about just renaming constraints. What about folks who are already on this version?
@Lee-W
Lee-W
commented
Oct 24, 2024

Copy link


Copy Markdown

MemberAuthor
Hmm not so sure about just renaming constraints. What about folks who are already on this version?
These reflect how the actual model is defined. As no feature is expected to be added to this version, I think these columns won't change in the future either.
@kaxil
kaxil
commented
Oct 24, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Hmm not so sure about just renaming constraints. What about folks who are already on this version?
These reflect how the actual model is defined. As no feature is expected to be added to this version, I think these columns won't change in the future either.
If you download Airflow 2.8.0, run airflow db migrate, upgrade to 2.10.2 and run airflow db migrate again with Postgres or MySQL backend, the constraints are created with the old names. Example:
image
If we merge and ship this change, we will have some users with dsdar_dag_fkey as name of constraint and some on dsdar_dataset_fkey. In future, if we need to access a constraint in another migration via name that won't work for some users.
This is true for folks upgrading from let's say Airflow 2.8.0 to 2.10.2 --> for those folks tables are created from "migration" . It is only for new Airflow 2.10.0 user where all tables are just created from ORM.
image
@potiuk
potiuk
commented
Oct 24, 2024

Copy link


Copy Markdown

Member
Yep. Agree with @kaxil. That should be a separate migration. Also the original cahnge in main should be reverted and re-applied after that because people will go eventually 2.x -> 2.11 -> 3.x and things will break for them.
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
4cbf70d to
e181cb0CompareOctober 25, 2024 05:00
@Lee-W
Lee-W
commented
Oct 25, 2024

Copy link


Copy Markdown

MemberAuthor
Got it. makes sense. I will create a fix pr to the main as well
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
e181cb0 to
836d590CompareOctober 25, 2024 05:35
@Lee-WLee-W
mentioned this pull request
Oct 25, 2024
 Merged

@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
836d590 to
915c398CompareOctober 25, 2024 05:38
@Lee-W
Lee-W
requested a review
from kaxilOctober 25, 2024 08:32
kaxil
kaxil
reviewed
Oct 25, 2024
View reviewed changes
Comment threadairflow/migrations/versions/0152_2_10_3_fix_dag_schedule_dataset_alias_reference_naming.py
Outdated

kaxil
kaxil
reviewed
Oct 25, 2024
View reviewed changes
Comment threadairflow/models/dataset.py
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
915c398 to
45fc2e8CompareOctober 25, 2024 15:53
kaxil
kaxil
reviewed
Oct 25, 2024
View reviewed changes
Comment threadairflow/migrations/versions/0152_2_10_3_fix_dag_schedule_dataset_alias_reference_naming.py
Outdated

kaxil
kaxil
approved these changes
Oct 25, 2024
View reviewed changes
@kaxilkaxil
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added 1 comment for downgrade but lgtm otherwise
https://github.com/apache/airflow/pull/43314/files#r1816982650
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
45fc2e8 to
062f65eCompareOctober 25, 2024 16:16
@potiukpotiuk
added this to the Airflow 2.10.3 milestone
Oct 25, 2024
@potiuk
potiuk
commented
Oct 25, 2024

Copy link


Copy Markdown

Member
Also I guess it should be back-ported ?
@Lee-W
Lee-Wforce-pushed
the
fix-2-10-migration-typo
branch
from
062f65e to
a31600dCompareOctober 26, 2024 03:26
@Lee-W
Lee-W
commented
Oct 26, 2024

Copy link


Copy Markdown

MemberAuthor
Also I guess it should be back-ported ?
This is for 2.10.x, but I guess what you mean is for main and yes, the PR is here #43373
@Lee-W
Lee-W
merged commit 9a868e0
into
apache:v2-10-testOct 26, 2024
@Lee-W
Lee-W
deleted the

fix-2-10-migration-typo

branch
October 26, 2024 15:05
@Lee-WLee-W
self-assigned this
Oct 28, 2024
utkarsharma2 pushed a commit
that referenced
this pull request
Oct 28, 2024
@Lee-W@utkarsharma2
rename ill-named constraints in dag_schedule_dataset_alias_reference …
6cad40e
…table #43314
(cherry picked from commit 9a868e0)
@utkarsharma2utkarsharma2
added
the 
changelog:skip
Changes that should be skipped from the changelog (CI, tests, etc..)
label
Oct 28, 2024
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@potiukpotiukpotiuk approved these changes
@bbovenzibbovenzibbovenzi approved these changes
@kaxilkaxilkaxil approved these changes
@hussein-awalahussein-awalahussein-awala approved these changes
@shahar1shahar1shahar1 approved these changes
@XD-DENGXD-DENGAwaiting requested review from XD-DENGXD-DENG is a code owner
@ashbashbAwaiting requested review from ashbashb is a code owner

Assignees

@Lee-WLee-W

Labels

area:db-migrationsPRs with DB migrationchangelog:skipChanges that should be skipped from the changelog (CI, tests, etc..)

Projects

No open projects
AIP-74 Introducing Data Assets

Status: Done


Milestone

Airflow 2.10.3

Development

Successfully merging this pull request may close these issues.

7 participants

@Lee-W@shahar1@kaxil@potiuk@bbovenzi@hussein-awala@utkarsharma2