allows users to write dag_id and task_id in their national characters, added display name for dag / task

[xgao1023-old]

---

This is picking up the work from #28183. But do not silently slugify the IDs

[boring-cyborg]

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst)
Here are some useful points:

• Pay attention to the quality of your code (ruff, mypy and type annotations). Our pre-commits will help you with that.
• In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
• Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
• Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
• Be sure to read the Airflow Coding style.
  Apache Airflow is a community-driven project and together we are making it better 🚀.
  In case of doubts contact the developers at:
  Mailing List: dev@airflow.apache.org
  Slack: https://s.apache.org/airflow-slack

[uranusjr]

Please set up pre-commit locally and fix the static check errors. See guide here: https://github.com/apache/airflow/blob/main/STATIC_CODE_CHECKS.rst#pre-commit-hooks

[bbovenzi]

Could you add an example dag so that we can test it out?

[xgao1023-old]

Could you add an example dag so that we can test it out?

Added

[uranusjr]

Do we also want a unit test? (Open question; there’s not much to be tested tbh, most things are just declarative.)

[xgao1023-old]

Seems that have display_name a read-only property breaking some ORM code

https://github.com/apache/airflow/actions/runs/5687937364/job/15417962420?pr=32520

[uranusjr]

Looks like display_name is already taken by multiple providers. We should probably come up with something else… Maybe task_display_name?

[xgao1023-old]

Looks like display_name is already taken by multiple providers. We should probably come up with something else… Maybe task_display_name?

Yeah probably. And dag_display_name in DAG?

[uranusjr]

Sounds right to me

[jscheffl]

I really like this approach, LGTM!

[jscheffl]

I did a deployment and faced a couple of "glitches". Can you check and rework on these? Otherwise as expressed in the comments I really like the feature and would like to have this merged ASAP!

1. I see many screens, w.g. Grid view not picking up the cool display names. I asuspect a bit of rework in the UI is needed. Low hanging fruit would be the DAG page header - if you could add using display name there would be great.
2. Starting the system with breeze raised an import error with two examples. Must be fixed, otherwise I assume also a lot of DAGs "in the wild" will fail in there
3. Two example DAGs are showing import errors when starting the system, probably the same root cause: "error DAG Import Errors (2)"

Error trace for 2)

ERROR [airflow.models.dagbag.DagBag] Failed to import: /opt/airflow/airflow/example_dags/example_dynamic_task_mapping.py
Traceback (most recent call last):
File "/opt/airflow/airflow/models/dagbag.py", line 345, in parse
loader.exec_module(new_module)
File "<frozen importlib._bootstrap_external>", line 843, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/opt/airflow/airflow/example_dags/example_dynamic_task_mapping.py", line 37, in <module>
added_values = add_one.expand(x=[1, 2, 3])
File "/opt/airflow/airflow/decorators/base.py", line 410, in expand
return self._expand(DictOfListsExpandInput(map_kwargs), strict=False)
File "/opt/airflow/airflow/decorators/base.py", line 478, in _expand
operator = _MappedOperator(
TypeError: __init__() missing 1 required keyword-only argument: 'task_display_name'
ERROR [airflow.models.dagbag.DagBag] Failed to import: /opt/airflow/airflow/example_dags/example_params_trigger_ui.py
Traceback (most recent call last):
File "/opt/airflow/airflow/models/dagbag.py", line 345, in parse
loader.exec_module(new_module)
File "<frozen importlib._bootstrap_external>", line 843, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/opt/airflow/airflow/example_dags/example_params_trigger_ui.py", line 103, in <module>
english_greetings = generate_english_greeting.expand(name=names)
File "/opt/airflow/airflow/decorators/base.py", line 410, in expand
return self._expand(DictOfListsExpandInput(map_kwargs), strict=False)
File "/opt/airflow/airflow/decorators/base.py", line 478, in _expand
operator = _MappedOperator(
TypeError: __init__() missing 1 required keyword-only argument: 'task_display_name'
Database migrating done!

Error trace for 3)

error DAG Import Errors (2) expand_less
expand_moreBroken DAG: [/opt/airflow/airflow/example_dags/example_dynamic_task_mapping.py] Traceback (most recent call last):
File "/opt/airflow/airflow/decorators/base.py", line 410, in expand
return self._expand(DictOfListsExpandInput(map_kwargs), strict=False)
File "/opt/airflow/airflow/decorators/base.py", line 478, in _expand
operator = _MappedOperator(
TypeError: __init__() missing 1 required keyword-only argument: 'task_display_name'
expand_moreBroken DAG: [/opt/airflow/airflow/example_dags/example_params_trigger_ui.py] Traceback (most recent call last):
File "/opt/airflow/airflow/decorators/base.py", line 410, in expand
return self._expand(DictOfListsExpandInput(map_kwargs), strict=False)
File "/opt/airflow/airflow/decorators/base.py", line 478, in _expand
operator = _MappedOperator(
TypeError: __init__() missing 1 required keyword-only argument: 'task_display_name'
DAGs

[uranusjr]

I tried to add a commit to address the error mentioned above, but there are multiple linting errors here and a merge conflict that must be fixed first.

[pierrejeambrun]

I think you also want to update the react part (Grid, Cluster Activity, Dataset) to use this new display_name instead of the current task_id, dag_id

[xgao1023-old]

I'm resuming working on this - fix the issues and also change UI part.

[xgao1023-old]

I tried to add a commit to address the error mentioned above, but there are multiple linting errors here and a merge conflict that must be fixed first.

I resolved the conflicts and rebased the branch

[xgao1023-old]

I think you also want to update the react part (Grid, Cluster Activity, Dataset) to use this new display_name instead of the current task_id, dag_id

I will try on this but I’m surely not a frontend expert.

[pierrejeambrun]

I will try on this but I’m surely not a frontend expert.

No worries, try your best. If you feel like this is too much, I think we can do that in follow up PRs, step by step.

cc: @bbovenzi

[jscheffl]

I really like this PR and am looking forward getting this merged. I feel like there are probably another 100 places where the DAG would benefit from displaying the "display name" and not the ID. Might be able to have this updated in the UI incrementally as well to get this merged.

Can you fix-up the DB migration (I hope I understood the alembic tooling right, with the commented fixes it was working for a local test - I hope you don't need to fully re-generate

[xgao1023-old]

I tried to add a commit to address the error mentioned above, but there are multiple linting errors here and a merge conflict that must be fixed first.

@uranusjr
I was trying to fix all errors but I stuck on one thing:

In general the fields on BaseOperator cannot be mapped. But it would be very useful if the display name of each mapped tasks instance can be different according to the expand argument.

Can you give me some hints on how to achieve this?

[jscheffl]

I'd really love to have this in - 2.9.0 release cut is across the corner - possible to get the conflicts resolved, the final bug fixed and have this merged?

[uranusjr]

Hopefully I can spend some time on this maybe next week.

[jscheffl]

As linked in #38446 - I tried to spend time on the weekend to continue working on this PR... let's see if we can get this reviewed and completed...

[ephraimbuddy]

Nit

[ephraimbuddy]

Suggested change

	else:
	returnself.node_id
	returnself.node_id

[jscheffl]

Closing this PR as completed with #38446 - thanks for the contribution on this branch which was the base for the efforts to make it in (hopefully, if all runs fine) Airflow 2.9.0!