Modernize DAG-related URL routes and rename "tree" to "grid" - #20730

Merged
uranusjr merged 6 commits into
apache:mainfrom
IKholopov:ikholopov/url_single_path_dag_page
Feb 10, 2022
Merged

Modernize DAG-related URL routes and rename "tree" to "grid"#20730
uranusjr merged 6 commits into
apache:mainfrom
IKholopov:ikholopov/url_single_path_dag_page

Conversation

@IKholopov

Copy link
Copy Markdown
Contributor

related: #19944

  • Rename "tree" view to "grid" view
  • Update URL routes and add redirects from old path:
    • /tree -> /dags/<dag_id>/grid
    • /graph -> /dags/<dag_id>/graph
    • /landing_times -> /dags/<dag_id>/landing_times
    • /duration -> /dags/<dag_id>/duration
    • /tries -> /dags/<dag_id>/tries
    • /calendar -> /dags/<dag_id>/calendar
    • /gantt -> /dags/<dag_id>/gantt
    • /code -> /dags/<dag_id>/code
    • /dag_details -> /dags/<dag_id>/details
  • New redirect - /dags/<dag_id> -> /dags/<dag_id>/grid

Once a new single DAG page is ready, the subroutes can be dropped and transformed into redirects to single /dags/<dag_id> page (with proper query params). As alternative, only single view could've been renamed (/tree -> /dags/<dag_id>), but I thought that it would be better to keep routes self-consistent mid-flight (in case if single-page view won't make it into 2.3.0).

DAGs page before: Screenshot 2022-01-06 at 14-14-22 tutorial - Tree - Airflow
DAGs page after: Screenshot 2022-01-06 at 14-02-14 tutorial - Tree - Airflow


@boring-cyborgboring-cyborgBot added area:UI Related to UI/UX. For Frontend Developers. area:webserver Webserver related Issues labels Jan 6, 2022
@boring-cyborg

Copy link
Copy Markdown

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst)
Here are some useful points:

  • Pay attention to the quality of your code (flake8, mypy and type annotations). Our pre-commits will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
  • Consider using Breeze environment for testing locally, it’s a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

Comment threadairflow/www/templates/airflow/dag.html Outdated

@bbovenzibbovenzi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good! Just some quick comments:

  • I agree with the use of legacy_calendar for old links instead of dag_calendar for new links.
  • We should also update the reference from tree to grid here
  • There are a lot more places where we need to change references of "tree" to "grid" in the code and documentation. That should happen shortly after this PR.

IKholopovand others added 2 commits January 30, 2022 20:49
…" view
- Rename tree view to grid view
- Updated test cases for new paths
- Update URL routes and add redirects
@IKholopov
IKholopovforce-pushed the ikholopov/url_single_path_dag_page branch from 50dbc04 to 3ae4b3cCompareJanuary 30, 2022 23:54
@IKholopov

Copy link
Copy Markdown
ContributorAuthor
  1. Renamed. This did add an additional backwards compatibility complexity. There is a number of places in code, where it is assumed that dag_id is passed exclusively over query parameter. To keep those working (especially the ones that are controlled by configuration, like default_view. To keep those working, both 'tree' and 'legacy_tree' views had to be kept/introduced. I'll address those places and make them work nicely with new routing schema shortly in separate PR.
  2. Done
  3. Yes, I'll tackle that in subsequent PR.

uranusjr
uranusjr previously requested changes Jan 31, 2022
Comment threadairflow/www/decorators.py Outdated
<meta name="dag_stats_url" content="{{ url_for('Airflow.dag_stats') }}">
<meta name="task_stats_url" content="{{ url_for('Airflow.task_stats') }}">
<meta name="tree_url" content="{{ url_for('Airflow.tree') }}">
<meta name="tree_url" content="{{ url_for('Airflow.legacy_tree') }}">

@bbovenzibbovenziJan 31, 2022

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to use the legacy urls here? Is it because of how it gets dag_id? I feel like it would be best to update those if it isn't too much effort.

@IKholopovIKholopovJan 31, 2022

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. There are multiple places in the JS code where the dag_id is specified as a query parameter. As I've mentioned before, I would much prefer to address it in a separate PR as I imagine that it is not obvious what would be the most correct way to pass Python-generated URL to JS code with view_arg. (And I'd like to keep changes compact, rather than having a full and hard to debug rewrite in a single PR).

Are we doing to insert some templated string like '<:dag_id>'? If yes, how can we avoid breakage if the view_arg is renamed/new view_args added for a path? Or should we avoid passing dynamic URLs over the meta tags all together and come up with something else?

Plus, right now with legacy_ and tree views we will have a clear indication for places that will need to be updated (which I've started working on already in subsequent commit).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok thats fine. I don't want to expand the scope of this PR too much.

@uranusjr

Copy link
Copy Markdown
Member

Need to fix the static checks, but logic-wise lgtm.

@eladkal

Copy link
Copy Markdown
Contributor

What this means for people who have
dag_default_view = tree in airflow.cfg?

# Default DAG view. Valid values are: ``tree``, ``graph``, ``duration``, ``gantt``, ``landing_times``
dag_default_view = tree

Comment threadairflow/www/views.py Outdated
"""Redirect from url param."""
return redirect(url_for('Airflow.landing_times', **request.args))

@expose('/dags/<string:dag_id>/landing_times')

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While we're changing this, could we use a hyphen instead? Hyphens are a more common, modern practice and we've used them in recent additions (e.g. /rendered-k8s, /rendered-templates).

Suggested change
@expose('/dags/<string:dag_id>/landing_times')
@expose('/dags/<string:dag_id>/landing-times')

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

@IKholopov

Copy link
Copy Markdown
ContributorAuthor

Fixed static checks (sorry, forgot about them:) and addressed @ryanahamilton comment.

@eladkal DAG's default_view property used to dynamically construct URL in one of two ways:
a) {{ url_for('Airflow.' + dag.default_view, dag_id=...)}} - will direct to new views right away (except for tree, in this case it will redirect to grid).
b) {{ url_for('Airflow.' + dag.default_view)}} - changed to {{ url_for('Airflow.legacy_' + dag.default_view)}}, will turn into a link redirecting to new versions of views.
This configuration is one of the next places in the code to cleanup and migrate to new naming (grid). We will have to keep backwards compatibility for tree though.

@bbovenzibbovenzi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@github-actionsgithub-actionsBot added the okay to merge It's ok to merge this PR as it does not require more tests label Feb 10, 2022
@github-actions

Copy link
Copy Markdown
Contributor

The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest main or amend the last commit of the PR, and push it with --force-with-lease.

@uranusjruranusjr changed the title Webserver - Change URL routes for DAG page and rename "tree" to "grid"Modernize DAG-related URL routes and rename "tree" to "grid"Feb 10, 2022
@uranusjr
uranusjr merged commit f217bec into apache:mainFeb 10, 2022
@boring-cyborg

Copy link
Copy Markdown

Awesome work, congrats on your first merged pull request!

ashb added a commit that referenced this pull request Feb 10, 2022
ferruzzi pushed a commit to ferruzzi/airflow that referenced this pull request Feb 11, 2022
@jedcunninghamjedcunningham added the type:improvement Changelog: Improvements label Feb 28, 2022
@bbovenzibbovenzi mentioned this pull request Mar 9, 2022
@bbovenzibbovenzi mentioned this pull request Feb 3, 2023
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:UIRelated to UI/UX. For Frontend Developers.area:webserverWebserver related Issuesokay to mergeIt's ok to merge this PR as it does not require more teststype:improvementChangelog: Improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@IKholopov@uranusjr@eladkal@ryanahamilton@bbovenzi@jedcunningham
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Modernize DAG-related URL routes and rename "tree" to "grid" - #20730

Merged
uranusjr merged 6 commits into
apache:mainfrom
IKholopov:ikholopov/url_single_path_dag_page
Feb 10, 2022
Merged

Modernize DAG-related URL routes and rename "tree" to "grid"#20730
uranusjr merged 6 commits into
apache:mainfrom
IKholopov:ikholopov/url_single_path_dag_page

Conversation

@IKholopov

Copy link
Copy Markdown
Contributor

related: #19944

  • Rename "tree" view to "grid" view
  • Update URL routes and add redirects from old path:
    • /tree -> /dags/<dag_id>/grid
    • /graph -> /dags/<dag_id>/graph
    • /landing_times -> /dags/<dag_id>/landing_times
    • /duration -> /dags/<dag_id>/duration
    • /tries -> /dags/<dag_id>/tries
    • /calendar -> /dags/<dag_id>/calendar
    • /gantt -> /dags/<dag_id>/gantt
    • /code -> /dags/<dag_id>/code
    • /dag_details -> /dags/<dag_id>/details
  • New redirect - /dags/<dag_id> -> /dags/<dag_id>/grid

Once a new single DAG page is ready, the subroutes can be dropped and transformed into redirects to single /dags/<dag_id> page (with proper query params). As alternative, only single view could've been renamed (/tree -> /dags/<dag_id>), but I thought that it would be better to keep routes self-consistent mid-flight (in case if single-page view won't make it into 2.3.0).

DAGs page before: Screenshot 2022-01-06 at 14-14-22 tutorial - Tree - Airflow
DAGs page after: Screenshot 2022-01-06 at 14-02-14 tutorial - Tree - Airflow


@boring-cyborgboring-cyborgBot added area:UI Related to UI/UX. For Frontend Developers. area:webserver Webserver related Issues labels Jan 6, 2022
@boring-cyborg

Copy link
Copy Markdown

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst)
Here are some useful points:

  • Pay attention to the quality of your code (flake8, mypy and type annotations). Our pre-commits will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
  • Consider using Breeze environment for testing locally, it’s a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

Comment threadairflow/www/templates/airflow/dag.html Outdated

@bbovenzibbovenzi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good! Just some quick comments:

  • I agree with the use of legacy_calendar for old links instead of dag_calendar for new links.
  • We should also update the reference from tree to grid here
  • There are a lot more places where we need to change references of "tree" to "grid" in the code and documentation. That should happen shortly after this PR.

IKholopovand others added 2 commits January 30, 2022 20:49
…" view
- Rename tree view to grid view
- Updated test cases for new paths
- Update URL routes and add redirects
@IKholopov
IKholopovforce-pushed the ikholopov/url_single_path_dag_page branch from 50dbc04 to 3ae4b3cCompareJanuary 30, 2022 23:54
@IKholopov

Copy link
Copy Markdown
ContributorAuthor
  1. Renamed. This did add an additional backwards compatibility complexity. There is a number of places in code, where it is assumed that dag_id is passed exclusively over query parameter. To keep those working (especially the ones that are controlled by configuration, like default_view. To keep those working, both 'tree' and 'legacy_tree' views had to be kept/introduced. I'll address those places and make them work nicely with new routing schema shortly in separate PR.
  2. Done
  3. Yes, I'll tackle that in subsequent PR.

uranusjr
uranusjr previously requested changes Jan 31, 2022
Comment threadairflow/www/decorators.py Outdated
<meta name="dag_stats_url" content="{{ url_for('Airflow.dag_stats') }}">
<meta name="task_stats_url" content="{{ url_for('Airflow.task_stats') }}">
<meta name="tree_url" content="{{ url_for('Airflow.tree') }}">
<meta name="tree_url" content="{{ url_for('Airflow.legacy_tree') }}">

@bbovenzibbovenziJan 31, 2022

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to use the legacy urls here? Is it because of how it gets dag_id? I feel like it would be best to update those if it isn't too much effort.

@IKholopovIKholopovJan 31, 2022

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. There are multiple places in the JS code where the dag_id is specified as a query parameter. As I've mentioned before, I would much prefer to address it in a separate PR as I imagine that it is not obvious what would be the most correct way to pass Python-generated URL to JS code with view_arg. (And I'd like to keep changes compact, rather than having a full and hard to debug rewrite in a single PR).

Are we doing to insert some templated string like '<:dag_id>'? If yes, how can we avoid breakage if the view_arg is renamed/new view_args added for a path? Or should we avoid passing dynamic URLs over the meta tags all together and come up with something else?

Plus, right now with legacy_ and tree views we will have a clear indication for places that will need to be updated (which I've started working on already in subsequent commit).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok thats fine. I don't want to expand the scope of this PR too much.

@uranusjr

Copy link
Copy Markdown
Member

Need to fix the static checks, but logic-wise lgtm.

@eladkal

Copy link
Copy Markdown
Contributor

What this means for people who have
dag_default_view = tree in airflow.cfg?

# Default DAG view. Valid values are: ``tree``, ``graph``, ``duration``, ``gantt``, ``landing_times``
dag_default_view = tree

Comment threadairflow/www/views.py Outdated
"""Redirect from url param."""
return redirect(url_for('Airflow.landing_times', **request.args))

@expose('/dags/<string:dag_id>/landing_times')

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While we're changing this, could we use a hyphen instead? Hyphens are a more common, modern practice and we've used them in recent additions (e.g. /rendered-k8s, /rendered-templates).

Suggested change
@expose('/dags/<string:dag_id>/landing_times')
@expose('/dags/<string:dag_id>/landing-times')

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

@IKholopov

Copy link
Copy Markdown
ContributorAuthor

Fixed static checks (sorry, forgot about them:) and addressed @ryanahamilton comment.

@eladkal DAG's default_view property used to dynamically construct URL in one of two ways:
a) {{ url_for('Airflow.' + dag.default_view, dag_id=...)}} - will direct to new views right away (except for tree, in this case it will redirect to grid).
b) {{ url_for('Airflow.' + dag.default_view)}} - changed to {{ url_for('Airflow.legacy_' + dag.default_view)}}, will turn into a link redirecting to new versions of views.
This configuration is one of the next places in the code to cleanup and migrate to new naming (grid). We will have to keep backwards compatibility for tree though.

@bbovenzibbovenzi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@github-actionsgithub-actionsBot added the okay to merge It's ok to merge this PR as it does not require more tests label Feb 10, 2022
@github-actions

Copy link
Copy Markdown
Contributor

The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest main or amend the last commit of the PR, and push it with --force-with-lease.

@uranusjruranusjr changed the title Webserver - Change URL routes for DAG page and rename "tree" to "grid"Modernize DAG-related URL routes and rename "tree" to "grid"Feb 10, 2022
@uranusjr
uranusjr merged commit f217bec into apache:mainFeb 10, 2022
@boring-cyborg

Copy link
Copy Markdown

Awesome work, congrats on your first merged pull request!

ashb added a commit that referenced this pull request Feb 10, 2022
ferruzzi pushed a commit to ferruzzi/airflow that referenced this pull request Feb 11, 2022
@jedcunninghamjedcunningham added the type:improvement Changelog: Improvements label Feb 28, 2022
@bbovenzibbovenzi mentioned this pull request Mar 9, 2022
@bbovenzibbovenzi mentioned this pull request Feb 3, 2023
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:UIRelated to UI/UX. For Frontend Developers.area:webserverWebserver related Issuesokay to mergeIt's ok to merge this PR as it does not require more teststype:improvementChangelog: Improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@IKholopov@uranusjr@eladkal@ryanahamilton@bbovenzi@jedcunningham
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Modernize DAG-related URL routes and rename "tree" to "grid" - #20730

Merged
uranusjr merged 6 commits into
apache:mainfrom
IKholopov:ikholopov/url_single_path_dag_page
Feb 10, 2022
Merged

Modernize DAG-related URL routes and rename "tree" to "grid"#20730
uranusjr merged 6 commits into
apache:mainfrom
IKholopov:ikholopov/url_single_path_dag_page

Conversation

@IKholopov

Copy link
Copy Markdown
Contributor

related: #19944

  • Rename "tree" view to "grid" view
  • Update URL routes and add redirects from old path:
    • /tree -> /dags/<dag_id>/grid
    • /graph -> /dags/<dag_id>/graph
    • /landing_times -> /dags/<dag_id>/landing_times
    • /duration -> /dags/<dag_id>/duration
    • /tries -> /dags/<dag_id>/tries
    • /calendar -> /dags/<dag_id>/calendar
    • /gantt -> /dags/<dag_id>/gantt
    • /code -> /dags/<dag_id>/code
    • /dag_details -> /dags/<dag_id>/details
  • New redirect - /dags/<dag_id> -> /dags/<dag_id>/grid

Once a new single DAG page is ready, the subroutes can be dropped and transformed into redirects to single /dags/<dag_id> page (with proper query params). As alternative, only single view could've been renamed (/tree -> /dags/<dag_id>), but I thought that it would be better to keep routes self-consistent mid-flight (in case if single-page view won't make it into 2.3.0).

DAGs page before: Screenshot 2022-01-06 at 14-14-22 tutorial - Tree - Airflow
DAGs page after: Screenshot 2022-01-06 at 14-02-14 tutorial - Tree - Airflow


@boring-cyborgboring-cyborgBot added area:UI Related to UI/UX. For Frontend Developers. area:webserver Webserver related Issues labels Jan 6, 2022
@boring-cyborg

Copy link
Copy Markdown

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst)
Here are some useful points:

  • Pay attention to the quality of your code (flake8, mypy and type annotations). Our pre-commits will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
  • Consider using Breeze environment for testing locally, it’s a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

Comment threadairflow/www/templates/airflow/dag.html Outdated

@bbovenzibbovenzi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good! Just some quick comments:

  • I agree with the use of legacy_calendar for old links instead of dag_calendar for new links.
  • We should also update the reference from tree to grid here
  • There are a lot more places where we need to change references of "tree" to "grid" in the code and documentation. That should happen shortly after this PR.

IKholopovand others added 2 commits January 30, 2022 20:49
…" view
- Rename tree view to grid view
- Updated test cases for new paths
- Update URL routes and add redirects
@IKholopov
IKholopovforce-pushed the ikholopov/url_single_path_dag_page branch from 50dbc04 to 3ae4b3cCompareJanuary 30, 2022 23:54
@IKholopov

Copy link
Copy Markdown
ContributorAuthor
  1. Renamed. This did add an additional backwards compatibility complexity. There is a number of places in code, where it is assumed that dag_id is passed exclusively over query parameter. To keep those working (especially the ones that are controlled by configuration, like default_view. To keep those working, both 'tree' and 'legacy_tree' views had to be kept/introduced. I'll address those places and make them work nicely with new routing schema shortly in separate PR.
  2. Done
  3. Yes, I'll tackle that in subsequent PR.

uranusjr
uranusjr previously requested changes Jan 31, 2022
Comment threadairflow/www/decorators.py Outdated
<meta name="dag_stats_url" content="{{ url_for('Airflow.dag_stats') }}">
<meta name="task_stats_url" content="{{ url_for('Airflow.task_stats') }}">
<meta name="tree_url" content="{{ url_for('Airflow.tree') }}">
<meta name="tree_url" content="{{ url_for('Airflow.legacy_tree') }}">

@bbovenzibbovenziJan 31, 2022

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to use the legacy urls here? Is it because of how it gets dag_id? I feel like it would be best to update those if it isn't too much effort.

@IKholopovIKholopovJan 31, 2022

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. There are multiple places in the JS code where the dag_id is specified as a query parameter. As I've mentioned before, I would much prefer to address it in a separate PR as I imagine that it is not obvious what would be the most correct way to pass Python-generated URL to JS code with view_arg. (And I'd like to keep changes compact, rather than having a full and hard to debug rewrite in a single PR).

Are we doing to insert some templated string like '<:dag_id>'? If yes, how can we avoid breakage if the view_arg is renamed/new view_args added for a path? Or should we avoid passing dynamic URLs over the meta tags all together and come up with something else?

Plus, right now with legacy_ and tree views we will have a clear indication for places that will need to be updated (which I've started working on already in subsequent commit).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok thats fine. I don't want to expand the scope of this PR too much.

@uranusjr

Copy link
Copy Markdown
Member

Need to fix the static checks, but logic-wise lgtm.

@eladkal

Copy link
Copy Markdown
Contributor

What this means for people who have
dag_default_view = tree in airflow.cfg?

# Default DAG view. Valid values are: ``tree``, ``graph``, ``duration``, ``gantt``, ``landing_times``
dag_default_view = tree

Comment threadairflow/www/views.py Outdated
"""Redirect from url param."""
return redirect(url_for('Airflow.landing_times', **request.args))

@expose('/dags/<string:dag_id>/landing_times')

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While we're changing this, could we use a hyphen instead? Hyphens are a more common, modern practice and we've used them in recent additions (e.g. /rendered-k8s, /rendered-templates).

Suggested change
@expose('/dags/<string:dag_id>/landing_times')
@expose('/dags/<string:dag_id>/landing-times')

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

@IKholopov

Copy link
Copy Markdown
ContributorAuthor

Fixed static checks (sorry, forgot about them:) and addressed @ryanahamilton comment.

@eladkal DAG's default_view property used to dynamically construct URL in one of two ways:
a) {{ url_for('Airflow.' + dag.default_view, dag_id=...)}} - will direct to new views right away (except for tree, in this case it will redirect to grid).
b) {{ url_for('Airflow.' + dag.default_view)}} - changed to {{ url_for('Airflow.legacy_' + dag.default_view)}}, will turn into a link redirecting to new versions of views.
This configuration is one of the next places in the code to cleanup and migrate to new naming (grid). We will have to keep backwards compatibility for tree though.

@bbovenzibbovenzi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@github-actionsgithub-actionsBot added the okay to merge It's ok to merge this PR as it does not require more tests label Feb 10, 2022
@github-actions

Copy link
Copy Markdown
Contributor

The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest main or amend the last commit of the PR, and push it with --force-with-lease.

@uranusjruranusjr changed the title Webserver - Change URL routes for DAG page and rename "tree" to "grid"Modernize DAG-related URL routes and rename "tree" to "grid"Feb 10, 2022
@uranusjr
uranusjr merged commit f217bec into apache:mainFeb 10, 2022
@boring-cyborg

Copy link
Copy Markdown

Awesome work, congrats on your first merged pull request!

ashb added a commit that referenced this pull request Feb 10, 2022
ferruzzi pushed a commit to ferruzzi/airflow that referenced this pull request Feb 11, 2022
@jedcunninghamjedcunningham added the type:improvement Changelog: Improvements label Feb 28, 2022
@bbovenzibbovenzi mentioned this pull request Mar 9, 2022
@bbovenzibbovenzi mentioned this pull request Feb 3, 2023
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:UIRelated to UI/UX. For Frontend Developers.area:webserverWebserver related Issuesokay to mergeIt's ok to merge this PR as it does not require more teststype:improvementChangelog: Improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@IKholopov@uranusjr@eladkal@ryanahamilton@bbovenzi@jedcunningham
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Modernize DAG-related URL routes and rename "tree" to "grid" - #20730

Merged
uranusjr merged 6 commits into
apache:mainfrom
IKholopov:ikholopov/url_single_path_dag_page
Feb 10, 2022
Merged

Modernize DAG-related URL routes and rename "tree" to "grid"#20730
uranusjr merged 6 commits into
apache:mainfrom
IKholopov:ikholopov/url_single_path_dag_page

Conversation

@IKholopov

Copy link
Copy Markdown
Contributor

related: #19944

  • Rename "tree" view to "grid" view
  • Update URL routes and add redirects from old path:
    • /tree -> /dags/<dag_id>/grid
    • /graph -> /dags/<dag_id>/graph
    • /landing_times -> /dags/<dag_id>/landing_times
    • /duration -> /dags/<dag_id>/duration
    • /tries -> /dags/<dag_id>/tries
    • /calendar -> /dags/<dag_id>/calendar
    • /gantt -> /dags/<dag_id>/gantt
    • /code -> /dags/<dag_id>/code
    • /dag_details -> /dags/<dag_id>/details
  • New redirect - /dags/<dag_id> -> /dags/<dag_id>/grid

Once a new single DAG page is ready, the subroutes can be dropped and transformed into redirects to single /dags/<dag_id> page (with proper query params). As alternative, only single view could've been renamed (/tree -> /dags/<dag_id>), but I thought that it would be better to keep routes self-consistent mid-flight (in case if single-page view won't make it into 2.3.0).

DAGs page before: Screenshot 2022-01-06 at 14-14-22 tutorial - Tree - Airflow
DAGs page after: Screenshot 2022-01-06 at 14-02-14 tutorial - Tree - Airflow


@boring-cyborgboring-cyborgBot added area:UI Related to UI/UX. For Frontend Developers. area:webserver Webserver related Issues labels Jan 6, 2022
@boring-cyborg

Copy link
Copy Markdown

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst)
Here are some useful points:

  • Pay attention to the quality of your code (flake8, mypy and type annotations). Our pre-commits will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
  • Consider using Breeze environment for testing locally, it’s a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

Comment threadairflow/www/templates/airflow/dag.html Outdated

@bbovenzibbovenzi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good! Just some quick comments:

  • I agree with the use of legacy_calendar for old links instead of dag_calendar for new links.
  • We should also update the reference from tree to grid here
  • There are a lot more places where we need to change references of "tree" to "grid" in the code and documentation. That should happen shortly after this PR.

IKholopovand others added 2 commits January 30, 2022 20:49
…" view
- Rename tree view to grid view
- Updated test cases for new paths
- Update URL routes and add redirects
@IKholopov
IKholopovforce-pushed the ikholopov/url_single_path_dag_page branch from 50dbc04 to 3ae4b3cCompareJanuary 30, 2022 23:54
@IKholopov

Copy link
Copy Markdown
ContributorAuthor
  1. Renamed. This did add an additional backwards compatibility complexity. There is a number of places in code, where it is assumed that dag_id is passed exclusively over query parameter. To keep those working (especially the ones that are controlled by configuration, like default_view. To keep those working, both 'tree' and 'legacy_tree' views had to be kept/introduced. I'll address those places and make them work nicely with new routing schema shortly in separate PR.
  2. Done
  3. Yes, I'll tackle that in subsequent PR.

uranusjr
uranusjr previously requested changes Jan 31, 2022
Comment threadairflow/www/decorators.py Outdated
<meta name="dag_stats_url" content="{{ url_for('Airflow.dag_stats') }}">
<meta name="task_stats_url" content="{{ url_for('Airflow.task_stats') }}">
<meta name="tree_url" content="{{ url_for('Airflow.tree') }}">
<meta name="tree_url" content="{{ url_for('Airflow.legacy_tree') }}">

@bbovenzibbovenziJan 31, 2022

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to use the legacy urls here? Is it because of how it gets dag_id? I feel like it would be best to update those if it isn't too much effort.

@IKholopovIKholopovJan 31, 2022

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. There are multiple places in the JS code where the dag_id is specified as a query parameter. As I've mentioned before, I would much prefer to address it in a separate PR as I imagine that it is not obvious what would be the most correct way to pass Python-generated URL to JS code with view_arg. (And I'd like to keep changes compact, rather than having a full and hard to debug rewrite in a single PR).

Are we doing to insert some templated string like '<:dag_id>'? If yes, how can we avoid breakage if the view_arg is renamed/new view_args added for a path? Or should we avoid passing dynamic URLs over the meta tags all together and come up with something else?

Plus, right now with legacy_ and tree views we will have a clear indication for places that will need to be updated (which I've started working on already in subsequent commit).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok thats fine. I don't want to expand the scope of this PR too much.

@uranusjr

Copy link
Copy Markdown
Member

Need to fix the static checks, but logic-wise lgtm.

@eladkal

Copy link
Copy Markdown
Contributor

What this means for people who have
dag_default_view = tree in airflow.cfg?

# Default DAG view. Valid values are: ``tree``, ``graph``, ``duration``, ``gantt``, ``landing_times``
dag_default_view = tree

Comment threadairflow/www/views.py Outdated
"""Redirect from url param."""
return redirect(url_for('Airflow.landing_times', **request.args))

@expose('/dags/<string:dag_id>/landing_times')

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While we're changing this, could we use a hyphen instead? Hyphens are a more common, modern practice and we've used them in recent additions (e.g. /rendered-k8s, /rendered-templates).

Suggested change
@expose('/dags/<string:dag_id>/landing_times')
@expose('/dags/<string:dag_id>/landing-times')

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

@IKholopov

Copy link
Copy Markdown
ContributorAuthor

Fixed static checks (sorry, forgot about them:) and addressed @ryanahamilton comment.

@eladkal DAG's default_view property used to dynamically construct URL in one of two ways:
a) {{ url_for('Airflow.' + dag.default_view, dag_id=...)}} - will direct to new views right away (except for tree, in this case it will redirect to grid).
b) {{ url_for('Airflow.' + dag.default_view)}} - changed to {{ url_for('Airflow.legacy_' + dag.default_view)}}, will turn into a link redirecting to new versions of views.
This configuration is one of the next places in the code to cleanup and migrate to new naming (grid). We will have to keep backwards compatibility for tree though.

@bbovenzibbovenzi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@github-actionsgithub-actionsBot added the okay to merge It's ok to merge this PR as it does not require more tests label Feb 10, 2022
@github-actions

Copy link
Copy Markdown
Contributor

The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest main or amend the last commit of the PR, and push it with --force-with-lease.

@uranusjruranusjr changed the title Webserver - Change URL routes for DAG page and rename "tree" to "grid"Modernize DAG-related URL routes and rename "tree" to "grid"Feb 10, 2022
@uranusjr
uranusjr merged commit f217bec into apache:mainFeb 10, 2022
@boring-cyborg

Copy link
Copy Markdown

Awesome work, congrats on your first merged pull request!

ashb added a commit that referenced this pull request Feb 10, 2022
ferruzzi pushed a commit to ferruzzi/airflow that referenced this pull request Feb 11, 2022
@jedcunninghamjedcunningham added the type:improvement Changelog: Improvements label Feb 28, 2022
@bbovenzibbovenzi mentioned this pull request Mar 9, 2022
@bbovenzibbovenzi mentioned this pull request Feb 3, 2023
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:UIRelated to UI/UX. For Frontend Developers.area:webserverWebserver related Issuesokay to mergeIt's ok to merge this PR as it does not require more teststype:improvementChangelog: Improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@IKholopov@uranusjr@eladkal@ryanahamilton@bbovenzi@jedcunningham
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Modernize DAG-related URL routes and rename "tree" to "grid" - #20730

Merged
uranusjr merged 6 commits into
apache:mainfrom
IKholopov:ikholopov/url_single_path_dag_page
Feb 10, 2022
Merged

Modernize DAG-related URL routes and rename "tree" to "grid"#20730
uranusjr merged 6 commits into
apache:mainfrom
IKholopov:ikholopov/url_single_path_dag_page

Conversation

@IKholopov

Copy link
Copy Markdown
Contributor

related: #19944

  • Rename "tree" view to "grid" view
  • Update URL routes and add redirects from old path:
    • /tree -> /dags/<dag_id>/grid
    • /graph -> /dags/<dag_id>/graph
    • /landing_times -> /dags/<dag_id>/landing_times
    • /duration -> /dags/<dag_id>/duration
    • /tries -> /dags/<dag_id>/tries
    • /calendar -> /dags/<dag_id>/calendar
    • /gantt -> /dags/<dag_id>/gantt
    • /code -> /dags/<dag_id>/code
    • /dag_details -> /dags/<dag_id>/details
  • New redirect - /dags/<dag_id> -> /dags/<dag_id>/grid

Once a new single DAG page is ready, the subroutes can be dropped and transformed into redirects to single /dags/<dag_id> page (with proper query params). As alternative, only single view could've been renamed (/tree -> /dags/<dag_id>), but I thought that it would be better to keep routes self-consistent mid-flight (in case if single-page view won't make it into 2.3.0).

DAGs page before: Screenshot 2022-01-06 at 14-14-22 tutorial - Tree - Airflow
DAGs page after: Screenshot 2022-01-06 at 14-02-14 tutorial - Tree - Airflow


@boring-cyborgboring-cyborgBot added area:UI Related to UI/UX. For Frontend Developers. area:webserver Webserver related Issues labels Jan 6, 2022
@boring-cyborg

Copy link
Copy Markdown

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst)
Here are some useful points:

  • Pay attention to the quality of your code (flake8, mypy and type annotations). Our pre-commits will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
  • Consider using Breeze environment for testing locally, it’s a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

Comment threadairflow/www/templates/airflow/dag.html Outdated

@bbovenzibbovenzi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good! Just some quick comments:

  • I agree with the use of legacy_calendar for old links instead of dag_calendar for new links.
  • We should also update the reference from tree to grid here
  • There are a lot more places where we need to change references of "tree" to "grid" in the code and documentation. That should happen shortly after this PR.

IKholopovand others added 2 commits January 30, 2022 20:49
…" view
- Rename tree view to grid view
- Updated test cases for new paths
- Update URL routes and add redirects
@IKholopov
IKholopovforce-pushed the ikholopov/url_single_path_dag_page branch from 50dbc04 to 3ae4b3cCompareJanuary 30, 2022 23:54
@IKholopov

Copy link
Copy Markdown
ContributorAuthor
  1. Renamed. This did add an additional backwards compatibility complexity. There is a number of places in code, where it is assumed that dag_id is passed exclusively over query parameter. To keep those working (especially the ones that are controlled by configuration, like default_view. To keep those working, both 'tree' and 'legacy_tree' views had to be kept/introduced. I'll address those places and make them work nicely with new routing schema shortly in separate PR.
  2. Done
  3. Yes, I'll tackle that in subsequent PR.

uranusjr
uranusjr previously requested changes Jan 31, 2022
Comment threadairflow/www/decorators.py Outdated
<meta name="dag_stats_url" content="{{ url_for('Airflow.dag_stats') }}">
<meta name="task_stats_url" content="{{ url_for('Airflow.task_stats') }}">
<meta name="tree_url" content="{{ url_for('Airflow.tree') }}">
<meta name="tree_url" content="{{ url_for('Airflow.legacy_tree') }}">

@bbovenzibbovenziJan 31, 2022

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to use the legacy urls here? Is it because of how it gets dag_id? I feel like it would be best to update those if it isn't too much effort.

@IKholopovIKholopovJan 31, 2022

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. There are multiple places in the JS code where the dag_id is specified as a query parameter. As I've mentioned before, I would much prefer to address it in a separate PR as I imagine that it is not obvious what would be the most correct way to pass Python-generated URL to JS code with view_arg. (And I'd like to keep changes compact, rather than having a full and hard to debug rewrite in a single PR).

Are we doing to insert some templated string like '<:dag_id>'? If yes, how can we avoid breakage if the view_arg is renamed/new view_args added for a path? Or should we avoid passing dynamic URLs over the meta tags all together and come up with something else?

Plus, right now with legacy_ and tree views we will have a clear indication for places that will need to be updated (which I've started working on already in subsequent commit).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok thats fine. I don't want to expand the scope of this PR too much.

@uranusjr

Copy link
Copy Markdown
Member

Need to fix the static checks, but logic-wise lgtm.

@eladkal

Copy link
Copy Markdown
Contributor

What this means for people who have
dag_default_view = tree in airflow.cfg?

# Default DAG view. Valid values are: ``tree``, ``graph``, ``duration``, ``gantt``, ``landing_times``
dag_default_view = tree

Comment threadairflow/www/views.py Outdated
"""Redirect from url param."""
return redirect(url_for('Airflow.landing_times', **request.args))

@expose('/dags/<string:dag_id>/landing_times')

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While we're changing this, could we use a hyphen instead? Hyphens are a more common, modern practice and we've used them in recent additions (e.g. /rendered-k8s, /rendered-templates).

Suggested change
@expose('/dags/<string:dag_id>/landing_times')
@expose('/dags/<string:dag_id>/landing-times')

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

@IKholopov

Copy link
Copy Markdown
ContributorAuthor

Fixed static checks (sorry, forgot about them:) and addressed @ryanahamilton comment.

@eladkal DAG's default_view property used to dynamically construct URL in one of two ways:
a) {{ url_for('Airflow.' + dag.default_view, dag_id=...)}} - will direct to new views right away (except for tree, in this case it will redirect to grid).
b) {{ url_for('Airflow.' + dag.default_view)}} - changed to {{ url_for('Airflow.legacy_' + dag.default_view)}}, will turn into a link redirecting to new versions of views.
This configuration is one of the next places in the code to cleanup and migrate to new naming (grid). We will have to keep backwards compatibility for tree though.

@bbovenzibbovenzi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@github-actionsgithub-actionsBot added the okay to merge It's ok to merge this PR as it does not require more tests label Feb 10, 2022
@github-actions

Copy link
Copy Markdown
Contributor

The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest main or amend the last commit of the PR, and push it with --force-with-lease.

@uranusjruranusjr changed the title Webserver - Change URL routes for DAG page and rename "tree" to "grid"Modernize DAG-related URL routes and rename "tree" to "grid"Feb 10, 2022
@uranusjr
uranusjr merged commit f217bec into apache:mainFeb 10, 2022
@boring-cyborg

Copy link
Copy Markdown

Awesome work, congrats on your first merged pull request!

ashb added a commit that referenced this pull request Feb 10, 2022
ferruzzi pushed a commit to ferruzzi/airflow that referenced this pull request Feb 11, 2022
@jedcunninghamjedcunningham added the type:improvement Changelog: Improvements label Feb 28, 2022
@bbovenzibbovenzi mentioned this pull request Mar 9, 2022
@bbovenzibbovenzi mentioned this pull request Feb 3, 2023
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:UIRelated to UI/UX. For Frontend Developers.area:webserverWebserver related Issuesokay to mergeIt's ok to merge this PR as it does not require more teststype:improvementChangelog: Improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@IKholopov@uranusjr@eladkal@ryanahamilton@bbovenzi@jedcunningham
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Modernize DAG-related URL routes and rename "tree" to "grid" - #20730

Merged
uranusjr merged 6 commits into
apache:mainfrom
IKholopov:ikholopov/url_single_path_dag_page
Feb 10, 2022
Merged

Modernize DAG-related URL routes and rename "tree" to "grid"#20730
uranusjr merged 6 commits into
apache:mainfrom
IKholopov:ikholopov/url_single_path_dag_page

Conversation

@IKholopov

Copy link
Copy Markdown
Contributor

related: #19944

  • Rename "tree" view to "grid" view
  • Update URL routes and add redirects from old path:
    • /tree -> /dags/<dag_id>/grid
    • /graph -> /dags/<dag_id>/graph
    • /landing_times -> /dags/<dag_id>/landing_times
    • /duration -> /dags/<dag_id>/duration
    • /tries -> /dags/<dag_id>/tries
    • /calendar -> /dags/<dag_id>/calendar
    • /gantt -> /dags/<dag_id>/gantt
    • /code -> /dags/<dag_id>/code
    • /dag_details -> /dags/<dag_id>/details
  • New redirect - /dags/<dag_id> -> /dags/<dag_id>/grid

Once a new single DAG page is ready, the subroutes can be dropped and transformed into redirects to single /dags/<dag_id> page (with proper query params). As alternative, only single view could've been renamed (/tree -> /dags/<dag_id>), but I thought that it would be better to keep routes self-consistent mid-flight (in case if single-page view won't make it into 2.3.0).

DAGs page before: Screenshot 2022-01-06 at 14-14-22 tutorial - Tree - Airflow
DAGs page after: Screenshot 2022-01-06 at 14-02-14 tutorial - Tree - Airflow


@boring-cyborgboring-cyborgBot added area:UI Related to UI/UX. For Frontend Developers. area:webserver Webserver related Issues labels Jan 6, 2022
@boring-cyborg

Copy link
Copy Markdown

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst)
Here are some useful points:

  • Pay attention to the quality of your code (flake8, mypy and type annotations). Our pre-commits will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
  • Consider using Breeze environment for testing locally, it’s a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

Comment threadairflow/www/templates/airflow/dag.html Outdated

@bbovenzibbovenzi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good! Just some quick comments:

  • I agree with the use of legacy_calendar for old links instead of dag_calendar for new links.
  • We should also update the reference from tree to grid here
  • There are a lot more places where we need to change references of "tree" to "grid" in the code and documentation. That should happen shortly after this PR.

IKholopovand others added 2 commits January 30, 2022 20:49
…" view
- Rename tree view to grid view
- Updated test cases for new paths
- Update URL routes and add redirects
@IKholopov
IKholopovforce-pushed the ikholopov/url_single_path_dag_page branch from 50dbc04 to 3ae4b3cCompareJanuary 30, 2022 23:54
@IKholopov

Copy link
Copy Markdown
ContributorAuthor
  1. Renamed. This did add an additional backwards compatibility complexity. There is a number of places in code, where it is assumed that dag_id is passed exclusively over query parameter. To keep those working (especially the ones that are controlled by configuration, like default_view. To keep those working, both 'tree' and 'legacy_tree' views had to be kept/introduced. I'll address those places and make them work nicely with new routing schema shortly in separate PR.
  2. Done
  3. Yes, I'll tackle that in subsequent PR.

uranusjr
uranusjr previously requested changes Jan 31, 2022
Comment threadairflow/www/decorators.py Outdated
<meta name="dag_stats_url" content="{{ url_for('Airflow.dag_stats') }}">
<meta name="task_stats_url" content="{{ url_for('Airflow.task_stats') }}">
<meta name="tree_url" content="{{ url_for('Airflow.tree') }}">
<meta name="tree_url" content="{{ url_for('Airflow.legacy_tree') }}">

@bbovenzibbovenziJan 31, 2022

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to use the legacy urls here? Is it because of how it gets dag_id? I feel like it would be best to update those if it isn't too much effort.

@IKholopovIKholopovJan 31, 2022

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. There are multiple places in the JS code where the dag_id is specified as a query parameter. As I've mentioned before, I would much prefer to address it in a separate PR as I imagine that it is not obvious what would be the most correct way to pass Python-generated URL to JS code with view_arg. (And I'd like to keep changes compact, rather than having a full and hard to debug rewrite in a single PR).

Are we doing to insert some templated string like '<:dag_id>'? If yes, how can we avoid breakage if the view_arg is renamed/new view_args added for a path? Or should we avoid passing dynamic URLs over the meta tags all together and come up with something else?

Plus, right now with legacy_ and tree views we will have a clear indication for places that will need to be updated (which I've started working on already in subsequent commit).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok thats fine. I don't want to expand the scope of this PR too much.

@uranusjr

Copy link
Copy Markdown
Member

Need to fix the static checks, but logic-wise lgtm.

@eladkal

Copy link
Copy Markdown
Contributor

What this means for people who have
dag_default_view = tree in airflow.cfg?

# Default DAG view. Valid values are: ``tree``, ``graph``, ``duration``, ``gantt``, ``landing_times``
dag_default_view = tree

Comment threadairflow/www/views.py Outdated
"""Redirect from url param."""
return redirect(url_for('Airflow.landing_times', **request.args))

@expose('/dags/<string:dag_id>/landing_times')

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While we're changing this, could we use a hyphen instead? Hyphens are a more common, modern practice and we've used them in recent additions (e.g. /rendered-k8s, /rendered-templates).

Suggested change
@expose('/dags/<string:dag_id>/landing_times')
@expose('/dags/<string:dag_id>/landing-times')

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

@IKholopov

Copy link
Copy Markdown
ContributorAuthor

Fixed static checks (sorry, forgot about them:) and addressed @ryanahamilton comment.

@eladkal DAG's default_view property used to dynamically construct URL in one of two ways:
a) {{ url_for('Airflow.' + dag.default_view, dag_id=...)}} - will direct to new views right away (except for tree, in this case it will redirect to grid).
b) {{ url_for('Airflow.' + dag.default_view)}} - changed to {{ url_for('Airflow.legacy_' + dag.default_view)}}, will turn into a link redirecting to new versions of views.
This configuration is one of the next places in the code to cleanup and migrate to new naming (grid). We will have to keep backwards compatibility for tree though.

@bbovenzibbovenzi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@github-actionsgithub-actionsBot added the okay to merge It's ok to merge this PR as it does not require more tests label Feb 10, 2022
@github-actions

Copy link
Copy Markdown
Contributor

The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest main or amend the last commit of the PR, and push it with --force-with-lease.

@uranusjruranusjr changed the title Webserver - Change URL routes for DAG page and rename "tree" to "grid"Modernize DAG-related URL routes and rename "tree" to "grid"Feb 10, 2022
@uranusjr
uranusjr merged commit f217bec into apache:mainFeb 10, 2022
@boring-cyborg

Copy link
Copy Markdown

Awesome work, congrats on your first merged pull request!

ashb added a commit that referenced this pull request Feb 10, 2022
ferruzzi pushed a commit to ferruzzi/airflow that referenced this pull request Feb 11, 2022
@jedcunninghamjedcunningham added the type:improvement Changelog: Improvements label Feb 28, 2022
@bbovenzibbovenzi mentioned this pull request Mar 9, 2022
@bbovenzibbovenzi mentioned this pull request Feb 3, 2023
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:UIRelated to UI/UX. For Frontend Developers.area:webserverWebserver related Issuesokay to mergeIt's ok to merge this PR as it does not require more teststype:improvementChangelog: Improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@IKholopov@uranusjr@eladkal@ryanahamilton@bbovenzi@jedcunningham
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Modernize DAG-related URL routes and rename "tree" to "grid" - #20730

Merged
uranusjr merged 6 commits into
apache:mainfrom
IKholopov:ikholopov/url_single_path_dag_page
Feb 10, 2022
Merged

Modernize DAG-related URL routes and rename "tree" to "grid"#20730
uranusjr merged 6 commits into
apache:mainfrom
IKholopov:ikholopov/url_single_path_dag_page

Conversation

@IKholopov

Copy link
Copy Markdown
Contributor

related: #19944

  • Rename "tree" view to "grid" view
  • Update URL routes and add redirects from old path:
    • /tree -> /dags/<dag_id>/grid
    • /graph -> /dags/<dag_id>/graph
    • /landing_times -> /dags/<dag_id>/landing_times
    • /duration -> /dags/<dag_id>/duration
    • /tries -> /dags/<dag_id>/tries
    • /calendar -> /dags/<dag_id>/calendar
    • /gantt -> /dags/<dag_id>/gantt
    • /code -> /dags/<dag_id>/code
    • /dag_details -> /dags/<dag_id>/details
  • New redirect - /dags/<dag_id> -> /dags/<dag_id>/grid

Once a new single DAG page is ready, the subroutes can be dropped and transformed into redirects to single /dags/<dag_id> page (with proper query params). As alternative, only single view could've been renamed (/tree -> /dags/<dag_id>), but I thought that it would be better to keep routes self-consistent mid-flight (in case if single-page view won't make it into 2.3.0).

DAGs page before: Screenshot 2022-01-06 at 14-14-22 tutorial - Tree - Airflow
DAGs page after: Screenshot 2022-01-06 at 14-02-14 tutorial - Tree - Airflow


@boring-cyborgboring-cyborgBot added area:UI Related to UI/UX. For Frontend Developers. area:webserver Webserver related Issues labels Jan 6, 2022
@boring-cyborg

Copy link
Copy Markdown

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst)
Here are some useful points:

  • Pay attention to the quality of your code (flake8, mypy and type annotations). Our pre-commits will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
  • Consider using Breeze environment for testing locally, it’s a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

Comment threadairflow/www/templates/airflow/dag.html Outdated

@bbovenzibbovenzi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good! Just some quick comments:

  • I agree with the use of legacy_calendar for old links instead of dag_calendar for new links.
  • We should also update the reference from tree to grid here
  • There are a lot more places where we need to change references of "tree" to "grid" in the code and documentation. That should happen shortly after this PR.

IKholopovand others added 2 commits January 30, 2022 20:49
…" view
- Rename tree view to grid view
- Updated test cases for new paths
- Update URL routes and add redirects
@IKholopov
IKholopovforce-pushed the ikholopov/url_single_path_dag_page branch from 50dbc04 to 3ae4b3cCompareJanuary 30, 2022 23:54
@IKholopov

Copy link
Copy Markdown
ContributorAuthor
  1. Renamed. This did add an additional backwards compatibility complexity. There is a number of places in code, where it is assumed that dag_id is passed exclusively over query parameter. To keep those working (especially the ones that are controlled by configuration, like default_view. To keep those working, both 'tree' and 'legacy_tree' views had to be kept/introduced. I'll address those places and make them work nicely with new routing schema shortly in separate PR.
  2. Done
  3. Yes, I'll tackle that in subsequent PR.

uranusjr
uranusjr previously requested changes Jan 31, 2022
Comment threadairflow/www/decorators.py Outdated
<meta name="dag_stats_url" content="{{ url_for('Airflow.dag_stats') }}">
<meta name="task_stats_url" content="{{ url_for('Airflow.task_stats') }}">
<meta name="tree_url" content="{{ url_for('Airflow.tree') }}">
<meta name="tree_url" content="{{ url_for('Airflow.legacy_tree') }}">

@bbovenzibbovenziJan 31, 2022

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to use the legacy urls here? Is it because of how it gets dag_id? I feel like it would be best to update those if it isn't too much effort.

@IKholopovIKholopovJan 31, 2022

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. There are multiple places in the JS code where the dag_id is specified as a query parameter. As I've mentioned before, I would much prefer to address it in a separate PR as I imagine that it is not obvious what would be the most correct way to pass Python-generated URL to JS code with view_arg. (And I'd like to keep changes compact, rather than having a full and hard to debug rewrite in a single PR).

Are we doing to insert some templated string like '<:dag_id>'? If yes, how can we avoid breakage if the view_arg is renamed/new view_args added for a path? Or should we avoid passing dynamic URLs over the meta tags all together and come up with something else?

Plus, right now with legacy_ and tree views we will have a clear indication for places that will need to be updated (which I've started working on already in subsequent commit).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok thats fine. I don't want to expand the scope of this PR too much.

@uranusjr

Copy link
Copy Markdown
Member

Need to fix the static checks, but logic-wise lgtm.

@eladkal

Copy link
Copy Markdown
Contributor

What this means for people who have
dag_default_view = tree in airflow.cfg?

# Default DAG view. Valid values are: ``tree``, ``graph``, ``duration``, ``gantt``, ``landing_times``
dag_default_view = tree

Comment threadairflow/www/views.py Outdated
"""Redirect from url param."""
return redirect(url_for('Airflow.landing_times', **request.args))

@expose('/dags/<string:dag_id>/landing_times')

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While we're changing this, could we use a hyphen instead? Hyphens are a more common, modern practice and we've used them in recent additions (e.g. /rendered-k8s, /rendered-templates).

Suggested change
@expose('/dags/<string:dag_id>/landing_times')
@expose('/dags/<string:dag_id>/landing-times')

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

@IKholopov

Copy link
Copy Markdown
ContributorAuthor

Fixed static checks (sorry, forgot about them:) and addressed @ryanahamilton comment.

@eladkal DAG's default_view property used to dynamically construct URL in one of two ways:
a) {{ url_for('Airflow.' + dag.default_view, dag_id=...)}} - will direct to new views right away (except for tree, in this case it will redirect to grid).
b) {{ url_for('Airflow.' + dag.default_view)}} - changed to {{ url_for('Airflow.legacy_' + dag.default_view)}}, will turn into a link redirecting to new versions of views.
This configuration is one of the next places in the code to cleanup and migrate to new naming (grid). We will have to keep backwards compatibility for tree though.

@bbovenzibbovenzi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@github-actionsgithub-actionsBot added the okay to merge It's ok to merge this PR as it does not require more tests label Feb 10, 2022
@github-actions

Copy link
Copy Markdown
Contributor

The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest main or amend the last commit of the PR, and push it with --force-with-lease.

@uranusjruranusjr changed the title Webserver - Change URL routes for DAG page and rename "tree" to "grid"Modernize DAG-related URL routes and rename "tree" to "grid"Feb 10, 2022
@uranusjr
uranusjr merged commit f217bec into apache:mainFeb 10, 2022
@boring-cyborg

Copy link
Copy Markdown

Awesome work, congrats on your first merged pull request!

ashb added a commit that referenced this pull request Feb 10, 2022
ferruzzi pushed a commit to ferruzzi/airflow that referenced this pull request Feb 11, 2022
@jedcunninghamjedcunningham added the type:improvement Changelog: Improvements label Feb 28, 2022
@bbovenzibbovenzi mentioned this pull request Mar 9, 2022
@bbovenzibbovenzi mentioned this pull request Feb 3, 2023
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:UIRelated to UI/UX. For Frontend Developers.area:webserverWebserver related Issuesokay to mergeIt's ok to merge this PR as it does not require more teststype:improvementChangelog: Improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@IKholopov@uranusjr@eladkal@ryanahamilton@bbovenzi@jedcunningham
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Modernize DAG-related URL routes and rename "tree" to "grid" - #20730

Merged
uranusjr merged 6 commits into
apache:mainfrom
IKholopov:ikholopov/url_single_path_dag_page
Feb 10, 2022
Merged

Modernize DAG-related URL routes and rename "tree" to "grid"#20730
uranusjr merged 6 commits into
apache:mainfrom
IKholopov:ikholopov/url_single_path_dag_page

Conversation

@IKholopov

Copy link
Copy Markdown
Contributor

related: #19944

  • Rename "tree" view to "grid" view
  • Update URL routes and add redirects from old path:
    • /tree -> /dags/<dag_id>/grid
    • /graph -> /dags/<dag_id>/graph
    • /landing_times -> /dags/<dag_id>/landing_times
    • /duration -> /dags/<dag_id>/duration
    • /tries -> /dags/<dag_id>/tries
    • /calendar -> /dags/<dag_id>/calendar
    • /gantt -> /dags/<dag_id>/gantt
    • /code -> /dags/<dag_id>/code
    • /dag_details -> /dags/<dag_id>/details
  • New redirect - /dags/<dag_id> -> /dags/<dag_id>/grid

Once a new single DAG page is ready, the subroutes can be dropped and transformed into redirects to single /dags/<dag_id> page (with proper query params). As alternative, only single view could've been renamed (/tree -> /dags/<dag_id>), but I thought that it would be better to keep routes self-consistent mid-flight (in case if single-page view won't make it into 2.3.0).

DAGs page before: Screenshot 2022-01-06 at 14-14-22 tutorial - Tree - Airflow
DAGs page after: Screenshot 2022-01-06 at 14-02-14 tutorial - Tree - Airflow


@boring-cyborgboring-cyborgBot added area:UI Related to UI/UX. For Frontend Developers. area:webserver Webserver related Issues labels Jan 6, 2022
@boring-cyborg

Copy link
Copy Markdown

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst)
Here are some useful points:

  • Pay attention to the quality of your code (flake8, mypy and type annotations). Our pre-commits will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
  • Consider using Breeze environment for testing locally, it’s a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

Comment threadairflow/www/templates/airflow/dag.html Outdated

@bbovenzibbovenzi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good! Just some quick comments:

  • I agree with the use of legacy_calendar for old links instead of dag_calendar for new links.
  • We should also update the reference from tree to grid here
  • There are a lot more places where we need to change references of "tree" to "grid" in the code and documentation. That should happen shortly after this PR.

IKholopovand others added 2 commits January 30, 2022 20:49
…" view
- Rename tree view to grid view
- Updated test cases for new paths
- Update URL routes and add redirects
@IKholopov
IKholopovforce-pushed the ikholopov/url_single_path_dag_page branch from 50dbc04 to 3ae4b3cCompareJanuary 30, 2022 23:54
@IKholopov

Copy link
Copy Markdown
ContributorAuthor
  1. Renamed. This did add an additional backwards compatibility complexity. There is a number of places in code, where it is assumed that dag_id is passed exclusively over query parameter. To keep those working (especially the ones that are controlled by configuration, like default_view. To keep those working, both 'tree' and 'legacy_tree' views had to be kept/introduced. I'll address those places and make them work nicely with new routing schema shortly in separate PR.
  2. Done
  3. Yes, I'll tackle that in subsequent PR.

uranusjr
uranusjr previously requested changes Jan 31, 2022
Comment threadairflow/www/decorators.py Outdated
<meta name="dag_stats_url" content="{{ url_for('Airflow.dag_stats') }}">
<meta name="task_stats_url" content="{{ url_for('Airflow.task_stats') }}">
<meta name="tree_url" content="{{ url_for('Airflow.tree') }}">
<meta name="tree_url" content="{{ url_for('Airflow.legacy_tree') }}">

@bbovenzibbovenziJan 31, 2022

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to use the legacy urls here? Is it because of how it gets dag_id? I feel like it would be best to update those if it isn't too much effort.

@IKholopovIKholopovJan 31, 2022

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. There are multiple places in the JS code where the dag_id is specified as a query parameter. As I've mentioned before, I would much prefer to address it in a separate PR as I imagine that it is not obvious what would be the most correct way to pass Python-generated URL to JS code with view_arg. (And I'd like to keep changes compact, rather than having a full and hard to debug rewrite in a single PR).

Are we doing to insert some templated string like '<:dag_id>'? If yes, how can we avoid breakage if the view_arg is renamed/new view_args added for a path? Or should we avoid passing dynamic URLs over the meta tags all together and come up with something else?

Plus, right now with legacy_ and tree views we will have a clear indication for places that will need to be updated (which I've started working on already in subsequent commit).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok thats fine. I don't want to expand the scope of this PR too much.

@uranusjr

Copy link
Copy Markdown
Member

Need to fix the static checks, but logic-wise lgtm.

@eladkal

Copy link
Copy Markdown
Contributor

What this means for people who have
dag_default_view = tree in airflow.cfg?

# Default DAG view. Valid values are: ``tree``, ``graph``, ``duration``, ``gantt``, ``landing_times``
dag_default_view = tree

Comment threadairflow/www/views.py Outdated
"""Redirect from url param."""
return redirect(url_for('Airflow.landing_times', **request.args))

@expose('/dags/<string:dag_id>/landing_times')

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While we're changing this, could we use a hyphen instead? Hyphens are a more common, modern practice and we've used them in recent additions (e.g. /rendered-k8s, /rendered-templates).

Suggested change
@expose('/dags/<string:dag_id>/landing_times')
@expose('/dags/<string:dag_id>/landing-times')

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

@IKholopov

Copy link
Copy Markdown
ContributorAuthor

Fixed static checks (sorry, forgot about them:) and addressed @ryanahamilton comment.

@eladkal DAG's default_view property used to dynamically construct URL in one of two ways:
a) {{ url_for('Airflow.' + dag.default_view, dag_id=...)}} - will direct to new views right away (except for tree, in this case it will redirect to grid).
b) {{ url_for('Airflow.' + dag.default_view)}} - changed to {{ url_for('Airflow.legacy_' + dag.default_view)}}, will turn into a link redirecting to new versions of views.
This configuration is one of the next places in the code to cleanup and migrate to new naming (grid). We will have to keep backwards compatibility for tree though.

@bbovenzibbovenzi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@github-actionsgithub-actionsBot added the okay to merge It's ok to merge this PR as it does not require more tests label Feb 10, 2022
@github-actions

Copy link
Copy Markdown
Contributor

The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest main or amend the last commit of the PR, and push it with --force-with-lease.

@uranusjruranusjr changed the title Webserver - Change URL routes for DAG page and rename "tree" to "grid"Modernize DAG-related URL routes and rename "tree" to "grid"Feb 10, 2022
@uranusjr
uranusjr merged commit f217bec into apache:mainFeb 10, 2022
@boring-cyborg

Copy link
Copy Markdown

Awesome work, congrats on your first merged pull request!

ashb added a commit that referenced this pull request Feb 10, 2022
ferruzzi pushed a commit to ferruzzi/airflow that referenced this pull request Feb 11, 2022
@jedcunninghamjedcunningham added the type:improvement Changelog: Improvements label Feb 28, 2022
@bbovenzibbovenzi mentioned this pull request Mar 9, 2022
@bbovenzibbovenzi mentioned this pull request Feb 3, 2023
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:UIRelated to UI/UX. For Frontend Developers.area:webserverWebserver related Issuesokay to mergeIt's ok to merge this PR as it does not require more teststype:improvementChangelog: Improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@IKholopov@uranusjr@eladkal@ryanahamilton@bbovenzi@jedcunningham