Do not fail scheduled tasks when serialized Dag is briefly missing - #72243

Closed
Vamsi-klu wants to merge 5 commits into
apache:mainfrom
Vamsi-klu:fix/62050-skip-missing-serialized-dag-queue
Closed

Do not fail scheduled tasks when serialized Dag is briefly missing#72243
Vamsi-klu wants to merge 5 commits into
apache:mainfrom
Vamsi-klu:fix/62050-skip-missing-serialized-dag-queue

Conversation

@Vamsi-klu

Copy link
Copy Markdown
Contributor

What is the change?

_task_concurrency_allows_execution no longer bulk-UPDATEs every SCHEDULED task instance to FAILED when get_dag_for_run returns None. It logs the same error as _create_dag_runs and returns False so this tick skips, and the next tick retries.

Why did I do it?

closes: #62050

The miss sits inside a concurrency check. A missing serialized row is not a concurrency answer. On HA schedulers, a parse or version hole then failed the whole warehouse load; retry worked because the next parse had a row. The create path already continues (test_scheduler_create_dag_runs_does_not_raise_error_when_no_serdag). Queue never got the same treatment. #58259 / #56422 made misses rarer; they did not remove the UPDATE.

How did I do it?

Deleted the session.execute(update(TI)...FAILED) block. return False was already there. Session stays uncommitted. I did not add a miss counter, did not fail only the one TI, and did not change _create_dag_runs. Permanently missing Dags stay SCHEDULED for the existing stale/import-error cleanup.

This path only runs when dag_model.has_task_concurrency_limits is True (max_active_tis_per_dag / max_active_tis_per_dagrun). Tests set max_active_tis_per_dag so the helper is actually entered.

What's the impact?

A transient serialized_dag hole no longer fails every SCHEDULED TI for that Dag (including every mapped index, backfill slice, and asset-triggered run that hits this helper). HA schedulers all skip instead of racing to stamp FAILED. Deleted Dags can sit in SCHEDULED until other cleanup; that is intentional.

What's the test plan?

New tests next to the create-path skip test:

  • test_executable_task_instances_skip_when_serialized_dag_missing: two tasks with concurrency limits, mock get_dag_for_run to None, queued list empty, both TIs still SCHEDULED
  • test_executable_task_instances_queue_when_serialized_dag_present: control, t1/t2 still queue

I restored the UPDATE and re-ran the skip test: TIs became FAILED (the first TI's UPDATE failed all SCHEDULED TIs including t2).

uv run --project airflow-core pytest \
airflow-core/tests/unit/jobs/test_scheduler_job.py \
-k 'skip_when_serialized_dag_missing or queue_when_serialized_dag_present or no_serdag'

3 passed. Ruff and airflow-core mypy passed via prek.


Was generative AI tooling used to co-author this PR?
  • Yes — Grok 4.6

Generated-by: Grok 4.6 following the guidelines


Drafted-by: Grok 4.6 (no human review before posting)

A concurrency check is not the place to mark every SCHEDULED task
FAILED. A parse blip then takes down the whole warehouse load;
the create path already skips and retries next tick.
@boring-cyborgboring-cyborgBot added the area:Scheduler including HA (high availability) scheduler label Aug 29, 2026
@Vamsi-klu
Vamsi-klu marked this pull request as ready for review August 29, 2026 03:32
Vamsi-kluand others added 3 commits August 29, 2026 03:57
test_queued_task_instances_fails_with_missing_dag still expected
FAILED and was outside the original -k selector, so CI would fail.
Align it with skip-not-fail.
A TLS reset during parallel provider version lookups failed the
constraints job on an otherwise green scheduler change.
Co-authored-by: Cursor <cursoragent@cursor.com>
@potiuk

Copy link
Copy Markdown
Member

Have you thought about side-effect of it? What are they? Do ypu (not your LLM) understand what your are doing here?have How it can happen that serialized dag is missing? Is it maybye a sign that something else is wrong - and you are just masking a problem?

Note - I have years of experience in Airflow but if I were to touch this code - I would think100 times and talk to someone over slack explaining how I reproduced .

Did you actually experience and reproduce it on running Airlfow instance? Do you have some proof of that?

Looking at the patterns of your contribution - you contribute like a shotgun - wherever your LLM thinks there is an issue, but you do not have deeper undersrtanding.

i am provisionally closing that - the "critical" parts of the code shoudl not be contributed to significantly without either experience or significant proof that you have run it locally, reproduced the issue and solved it and that you thought and reasoned about the consequences.

Thare are other areas where you can contribute smaller things in Airflow - if all you do is put your LLLm on it.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:Schedulerincluding HA (high availability) scheduler

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Scheduler bulk-fails all scheduled tasks when serialized DAG is transiently missing

2 participants

@Vamsi-klu@potiuk
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Do not fail scheduled tasks when serialized Dag is briefly missing - #72243

Closed
Vamsi-klu wants to merge 5 commits into
apache:mainfrom
Vamsi-klu:fix/62050-skip-missing-serialized-dag-queue
Closed

Do not fail scheduled tasks when serialized Dag is briefly missing#72243
Vamsi-klu wants to merge 5 commits into
apache:mainfrom
Vamsi-klu:fix/62050-skip-missing-serialized-dag-queue

Conversation

@Vamsi-klu

Copy link
Copy Markdown
Contributor

What is the change?

_task_concurrency_allows_execution no longer bulk-UPDATEs every SCHEDULED task instance to FAILED when get_dag_for_run returns None. It logs the same error as _create_dag_runs and returns False so this tick skips, and the next tick retries.

Why did I do it?

closes: #62050

The miss sits inside a concurrency check. A missing serialized row is not a concurrency answer. On HA schedulers, a parse or version hole then failed the whole warehouse load; retry worked because the next parse had a row. The create path already continues (test_scheduler_create_dag_runs_does_not_raise_error_when_no_serdag). Queue never got the same treatment. #58259 / #56422 made misses rarer; they did not remove the UPDATE.

How did I do it?

Deleted the session.execute(update(TI)...FAILED) block. return False was already there. Session stays uncommitted. I did not add a miss counter, did not fail only the one TI, and did not change _create_dag_runs. Permanently missing Dags stay SCHEDULED for the existing stale/import-error cleanup.

This path only runs when dag_model.has_task_concurrency_limits is True (max_active_tis_per_dag / max_active_tis_per_dagrun). Tests set max_active_tis_per_dag so the helper is actually entered.

What's the impact?

A transient serialized_dag hole no longer fails every SCHEDULED TI for that Dag (including every mapped index, backfill slice, and asset-triggered run that hits this helper). HA schedulers all skip instead of racing to stamp FAILED. Deleted Dags can sit in SCHEDULED until other cleanup; that is intentional.

What's the test plan?

New tests next to the create-path skip test:

  • test_executable_task_instances_skip_when_serialized_dag_missing: two tasks with concurrency limits, mock get_dag_for_run to None, queued list empty, both TIs still SCHEDULED
  • test_executable_task_instances_queue_when_serialized_dag_present: control, t1/t2 still queue

I restored the UPDATE and re-ran the skip test: TIs became FAILED (the first TI's UPDATE failed all SCHEDULED TIs including t2).

uv run --project airflow-core pytest \
airflow-core/tests/unit/jobs/test_scheduler_job.py \
-k 'skip_when_serialized_dag_missing or queue_when_serialized_dag_present or no_serdag'

3 passed. Ruff and airflow-core mypy passed via prek.


Was generative AI tooling used to co-author this PR?
  • Yes — Grok 4.6

Generated-by: Grok 4.6 following the guidelines


Drafted-by: Grok 4.6 (no human review before posting)

A concurrency check is not the place to mark every SCHEDULED task
FAILED. A parse blip then takes down the whole warehouse load;
the create path already skips and retries next tick.
@boring-cyborgboring-cyborgBot added the area:Scheduler including HA (high availability) scheduler label Aug 29, 2026
@Vamsi-klu
Vamsi-klu marked this pull request as ready for review August 29, 2026 03:32
Vamsi-kluand others added 3 commits August 29, 2026 03:57
test_queued_task_instances_fails_with_missing_dag still expected
FAILED and was outside the original -k selector, so CI would fail.
Align it with skip-not-fail.
A TLS reset during parallel provider version lookups failed the
constraints job on an otherwise green scheduler change.
Co-authored-by: Cursor <cursoragent@cursor.com>
@potiuk

Copy link
Copy Markdown
Member

Have you thought about side-effect of it? What are they? Do ypu (not your LLM) understand what your are doing here?have How it can happen that serialized dag is missing? Is it maybye a sign that something else is wrong - and you are just masking a problem?

Note - I have years of experience in Airflow but if I were to touch this code - I would think100 times and talk to someone over slack explaining how I reproduced .

Did you actually experience and reproduce it on running Airlfow instance? Do you have some proof of that?

Looking at the patterns of your contribution - you contribute like a shotgun - wherever your LLM thinks there is an issue, but you do not have deeper undersrtanding.

i am provisionally closing that - the "critical" parts of the code shoudl not be contributed to significantly without either experience or significant proof that you have run it locally, reproduced the issue and solved it and that you thought and reasoned about the consequences.

Thare are other areas where you can contribute smaller things in Airflow - if all you do is put your LLLm on it.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:Schedulerincluding HA (high availability) scheduler

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Scheduler bulk-fails all scheduled tasks when serialized DAG is transiently missing

2 participants

@Vamsi-klu@potiuk
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Do not fail scheduled tasks when serialized Dag is briefly missing - #72243

Closed
Vamsi-klu wants to merge 5 commits into
apache:mainfrom
Vamsi-klu:fix/62050-skip-missing-serialized-dag-queue
Closed

Do not fail scheduled tasks when serialized Dag is briefly missing#72243
Vamsi-klu wants to merge 5 commits into
apache:mainfrom
Vamsi-klu:fix/62050-skip-missing-serialized-dag-queue

Conversation

@Vamsi-klu

Copy link
Copy Markdown
Contributor

What is the change?

_task_concurrency_allows_execution no longer bulk-UPDATEs every SCHEDULED task instance to FAILED when get_dag_for_run returns None. It logs the same error as _create_dag_runs and returns False so this tick skips, and the next tick retries.

Why did I do it?

closes: #62050

The miss sits inside a concurrency check. A missing serialized row is not a concurrency answer. On HA schedulers, a parse or version hole then failed the whole warehouse load; retry worked because the next parse had a row. The create path already continues (test_scheduler_create_dag_runs_does_not_raise_error_when_no_serdag). Queue never got the same treatment. #58259 / #56422 made misses rarer; they did not remove the UPDATE.

How did I do it?

Deleted the session.execute(update(TI)...FAILED) block. return False was already there. Session stays uncommitted. I did not add a miss counter, did not fail only the one TI, and did not change _create_dag_runs. Permanently missing Dags stay SCHEDULED for the existing stale/import-error cleanup.

This path only runs when dag_model.has_task_concurrency_limits is True (max_active_tis_per_dag / max_active_tis_per_dagrun). Tests set max_active_tis_per_dag so the helper is actually entered.

What's the impact?

A transient serialized_dag hole no longer fails every SCHEDULED TI for that Dag (including every mapped index, backfill slice, and asset-triggered run that hits this helper). HA schedulers all skip instead of racing to stamp FAILED. Deleted Dags can sit in SCHEDULED until other cleanup; that is intentional.

What's the test plan?

New tests next to the create-path skip test:

  • test_executable_task_instances_skip_when_serialized_dag_missing: two tasks with concurrency limits, mock get_dag_for_run to None, queued list empty, both TIs still SCHEDULED
  • test_executable_task_instances_queue_when_serialized_dag_present: control, t1/t2 still queue

I restored the UPDATE and re-ran the skip test: TIs became FAILED (the first TI's UPDATE failed all SCHEDULED TIs including t2).

uv run --project airflow-core pytest \
airflow-core/tests/unit/jobs/test_scheduler_job.py \
-k 'skip_when_serialized_dag_missing or queue_when_serialized_dag_present or no_serdag'

3 passed. Ruff and airflow-core mypy passed via prek.


Was generative AI tooling used to co-author this PR?
  • Yes — Grok 4.6

Generated-by: Grok 4.6 following the guidelines


Drafted-by: Grok 4.6 (no human review before posting)

A concurrency check is not the place to mark every SCHEDULED task
FAILED. A parse blip then takes down the whole warehouse load;
the create path already skips and retries next tick.
@boring-cyborgboring-cyborgBot added the area:Scheduler including HA (high availability) scheduler label Aug 29, 2026
@Vamsi-klu
Vamsi-klu marked this pull request as ready for review August 29, 2026 03:32
Vamsi-kluand others added 3 commits August 29, 2026 03:57
test_queued_task_instances_fails_with_missing_dag still expected
FAILED and was outside the original -k selector, so CI would fail.
Align it with skip-not-fail.
A TLS reset during parallel provider version lookups failed the
constraints job on an otherwise green scheduler change.
Co-authored-by: Cursor <cursoragent@cursor.com>
@potiuk

Copy link
Copy Markdown
Member

Have you thought about side-effect of it? What are they? Do ypu (not your LLM) understand what your are doing here?have How it can happen that serialized dag is missing? Is it maybye a sign that something else is wrong - and you are just masking a problem?

Note - I have years of experience in Airflow but if I were to touch this code - I would think100 times and talk to someone over slack explaining how I reproduced .

Did you actually experience and reproduce it on running Airlfow instance? Do you have some proof of that?

Looking at the patterns of your contribution - you contribute like a shotgun - wherever your LLM thinks there is an issue, but you do not have deeper undersrtanding.

i am provisionally closing that - the "critical" parts of the code shoudl not be contributed to significantly without either experience or significant proof that you have run it locally, reproduced the issue and solved it and that you thought and reasoned about the consequences.

Thare are other areas where you can contribute smaller things in Airflow - if all you do is put your LLLm on it.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:Schedulerincluding HA (high availability) scheduler

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Scheduler bulk-fails all scheduled tasks when serialized DAG is transiently missing

2 participants

@Vamsi-klu@potiuk
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Do not fail scheduled tasks when serialized Dag is briefly missing - #72243

Closed
Vamsi-klu wants to merge 5 commits into
apache:mainfrom
Vamsi-klu:fix/62050-skip-missing-serialized-dag-queue
Closed

Do not fail scheduled tasks when serialized Dag is briefly missing#72243
Vamsi-klu wants to merge 5 commits into
apache:mainfrom
Vamsi-klu:fix/62050-skip-missing-serialized-dag-queue

Conversation

@Vamsi-klu

Copy link
Copy Markdown
Contributor

What is the change?

_task_concurrency_allows_execution no longer bulk-UPDATEs every SCHEDULED task instance to FAILED when get_dag_for_run returns None. It logs the same error as _create_dag_runs and returns False so this tick skips, and the next tick retries.

Why did I do it?

closes: #62050

The miss sits inside a concurrency check. A missing serialized row is not a concurrency answer. On HA schedulers, a parse or version hole then failed the whole warehouse load; retry worked because the next parse had a row. The create path already continues (test_scheduler_create_dag_runs_does_not_raise_error_when_no_serdag). Queue never got the same treatment. #58259 / #56422 made misses rarer; they did not remove the UPDATE.

How did I do it?

Deleted the session.execute(update(TI)...FAILED) block. return False was already there. Session stays uncommitted. I did not add a miss counter, did not fail only the one TI, and did not change _create_dag_runs. Permanently missing Dags stay SCHEDULED for the existing stale/import-error cleanup.

This path only runs when dag_model.has_task_concurrency_limits is True (max_active_tis_per_dag / max_active_tis_per_dagrun). Tests set max_active_tis_per_dag so the helper is actually entered.

What's the impact?

A transient serialized_dag hole no longer fails every SCHEDULED TI for that Dag (including every mapped index, backfill slice, and asset-triggered run that hits this helper). HA schedulers all skip instead of racing to stamp FAILED. Deleted Dags can sit in SCHEDULED until other cleanup; that is intentional.

What's the test plan?

New tests next to the create-path skip test:

  • test_executable_task_instances_skip_when_serialized_dag_missing: two tasks with concurrency limits, mock get_dag_for_run to None, queued list empty, both TIs still SCHEDULED
  • test_executable_task_instances_queue_when_serialized_dag_present: control, t1/t2 still queue

I restored the UPDATE and re-ran the skip test: TIs became FAILED (the first TI's UPDATE failed all SCHEDULED TIs including t2).

uv run --project airflow-core pytest \
airflow-core/tests/unit/jobs/test_scheduler_job.py \
-k 'skip_when_serialized_dag_missing or queue_when_serialized_dag_present or no_serdag'

3 passed. Ruff and airflow-core mypy passed via prek.


Was generative AI tooling used to co-author this PR?
  • Yes — Grok 4.6

Generated-by: Grok 4.6 following the guidelines


Drafted-by: Grok 4.6 (no human review before posting)

A concurrency check is not the place to mark every SCHEDULED task
FAILED. A parse blip then takes down the whole warehouse load;
the create path already skips and retries next tick.
@boring-cyborgboring-cyborgBot added the area:Scheduler including HA (high availability) scheduler label Aug 29, 2026
@Vamsi-klu
Vamsi-klu marked this pull request as ready for review August 29, 2026 03:32
Vamsi-kluand others added 3 commits August 29, 2026 03:57
test_queued_task_instances_fails_with_missing_dag still expected
FAILED and was outside the original -k selector, so CI would fail.
Align it with skip-not-fail.
A TLS reset during parallel provider version lookups failed the
constraints job on an otherwise green scheduler change.
Co-authored-by: Cursor <cursoragent@cursor.com>
@potiuk

Copy link
Copy Markdown
Member

Have you thought about side-effect of it? What are they? Do ypu (not your LLM) understand what your are doing here?have How it can happen that serialized dag is missing? Is it maybye a sign that something else is wrong - and you are just masking a problem?

Note - I have years of experience in Airflow but if I were to touch this code - I would think100 times and talk to someone over slack explaining how I reproduced .

Did you actually experience and reproduce it on running Airlfow instance? Do you have some proof of that?

Looking at the patterns of your contribution - you contribute like a shotgun - wherever your LLM thinks there is an issue, but you do not have deeper undersrtanding.

i am provisionally closing that - the "critical" parts of the code shoudl not be contributed to significantly without either experience or significant proof that you have run it locally, reproduced the issue and solved it and that you thought and reasoned about the consequences.

Thare are other areas where you can contribute smaller things in Airflow - if all you do is put your LLLm on it.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:Schedulerincluding HA (high availability) scheduler

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Scheduler bulk-fails all scheduled tasks when serialized DAG is transiently missing

2 participants

@Vamsi-klu@potiuk
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Do not fail scheduled tasks when serialized Dag is briefly missing - #72243

Closed
Vamsi-klu wants to merge 5 commits into
apache:mainfrom
Vamsi-klu:fix/62050-skip-missing-serialized-dag-queue
Closed

Do not fail scheduled tasks when serialized Dag is briefly missing#72243
Vamsi-klu wants to merge 5 commits into
apache:mainfrom
Vamsi-klu:fix/62050-skip-missing-serialized-dag-queue

Conversation

@Vamsi-klu

Copy link
Copy Markdown
Contributor

What is the change?

_task_concurrency_allows_execution no longer bulk-UPDATEs every SCHEDULED task instance to FAILED when get_dag_for_run returns None. It logs the same error as _create_dag_runs and returns False so this tick skips, and the next tick retries.

Why did I do it?

closes: #62050

The miss sits inside a concurrency check. A missing serialized row is not a concurrency answer. On HA schedulers, a parse or version hole then failed the whole warehouse load; retry worked because the next parse had a row. The create path already continues (test_scheduler_create_dag_runs_does_not_raise_error_when_no_serdag). Queue never got the same treatment. #58259 / #56422 made misses rarer; they did not remove the UPDATE.

How did I do it?

Deleted the session.execute(update(TI)...FAILED) block. return False was already there. Session stays uncommitted. I did not add a miss counter, did not fail only the one TI, and did not change _create_dag_runs. Permanently missing Dags stay SCHEDULED for the existing stale/import-error cleanup.

This path only runs when dag_model.has_task_concurrency_limits is True (max_active_tis_per_dag / max_active_tis_per_dagrun). Tests set max_active_tis_per_dag so the helper is actually entered.

What's the impact?

A transient serialized_dag hole no longer fails every SCHEDULED TI for that Dag (including every mapped index, backfill slice, and asset-triggered run that hits this helper). HA schedulers all skip instead of racing to stamp FAILED. Deleted Dags can sit in SCHEDULED until other cleanup; that is intentional.

What's the test plan?

New tests next to the create-path skip test:

  • test_executable_task_instances_skip_when_serialized_dag_missing: two tasks with concurrency limits, mock get_dag_for_run to None, queued list empty, both TIs still SCHEDULED
  • test_executable_task_instances_queue_when_serialized_dag_present: control, t1/t2 still queue

I restored the UPDATE and re-ran the skip test: TIs became FAILED (the first TI's UPDATE failed all SCHEDULED TIs including t2).

uv run --project airflow-core pytest \
airflow-core/tests/unit/jobs/test_scheduler_job.py \
-k 'skip_when_serialized_dag_missing or queue_when_serialized_dag_present or no_serdag'

3 passed. Ruff and airflow-core mypy passed via prek.


Was generative AI tooling used to co-author this PR?
  • Yes — Grok 4.6

Generated-by: Grok 4.6 following the guidelines


Drafted-by: Grok 4.6 (no human review before posting)

A concurrency check is not the place to mark every SCHEDULED task
FAILED. A parse blip then takes down the whole warehouse load;
the create path already skips and retries next tick.
@boring-cyborgboring-cyborgBot added the area:Scheduler including HA (high availability) scheduler label Aug 29, 2026
@Vamsi-klu
Vamsi-klu marked this pull request as ready for review August 29, 2026 03:32
Vamsi-kluand others added 3 commits August 29, 2026 03:57
test_queued_task_instances_fails_with_missing_dag still expected
FAILED and was outside the original -k selector, so CI would fail.
Align it with skip-not-fail.
A TLS reset during parallel provider version lookups failed the
constraints job on an otherwise green scheduler change.
Co-authored-by: Cursor <cursoragent@cursor.com>
@potiuk

Copy link
Copy Markdown
Member

Have you thought about side-effect of it? What are they? Do ypu (not your LLM) understand what your are doing here?have How it can happen that serialized dag is missing? Is it maybye a sign that something else is wrong - and you are just masking a problem?

Note - I have years of experience in Airflow but if I were to touch this code - I would think100 times and talk to someone over slack explaining how I reproduced .

Did you actually experience and reproduce it on running Airlfow instance? Do you have some proof of that?

Looking at the patterns of your contribution - you contribute like a shotgun - wherever your LLM thinks there is an issue, but you do not have deeper undersrtanding.

i am provisionally closing that - the "critical" parts of the code shoudl not be contributed to significantly without either experience or significant proof that you have run it locally, reproduced the issue and solved it and that you thought and reasoned about the consequences.

Thare are other areas where you can contribute smaller things in Airflow - if all you do is put your LLLm on it.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:Schedulerincluding HA (high availability) scheduler

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Scheduler bulk-fails all scheduled tasks when serialized DAG is transiently missing

2 participants

@Vamsi-klu@potiuk
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Do not fail scheduled tasks when serialized Dag is briefly missing - #72243

Closed
Vamsi-klu wants to merge 5 commits into
apache:mainfrom
Vamsi-klu:fix/62050-skip-missing-serialized-dag-queue
Closed

Do not fail scheduled tasks when serialized Dag is briefly missing#72243
Vamsi-klu wants to merge 5 commits into
apache:mainfrom
Vamsi-klu:fix/62050-skip-missing-serialized-dag-queue

Conversation

@Vamsi-klu

Copy link
Copy Markdown
Contributor

What is the change?

_task_concurrency_allows_execution no longer bulk-UPDATEs every SCHEDULED task instance to FAILED when get_dag_for_run returns None. It logs the same error as _create_dag_runs and returns False so this tick skips, and the next tick retries.

Why did I do it?

closes: #62050

The miss sits inside a concurrency check. A missing serialized row is not a concurrency answer. On HA schedulers, a parse or version hole then failed the whole warehouse load; retry worked because the next parse had a row. The create path already continues (test_scheduler_create_dag_runs_does_not_raise_error_when_no_serdag). Queue never got the same treatment. #58259 / #56422 made misses rarer; they did not remove the UPDATE.

How did I do it?

Deleted the session.execute(update(TI)...FAILED) block. return False was already there. Session stays uncommitted. I did not add a miss counter, did not fail only the one TI, and did not change _create_dag_runs. Permanently missing Dags stay SCHEDULED for the existing stale/import-error cleanup.

This path only runs when dag_model.has_task_concurrency_limits is True (max_active_tis_per_dag / max_active_tis_per_dagrun). Tests set max_active_tis_per_dag so the helper is actually entered.

What's the impact?

A transient serialized_dag hole no longer fails every SCHEDULED TI for that Dag (including every mapped index, backfill slice, and asset-triggered run that hits this helper). HA schedulers all skip instead of racing to stamp FAILED. Deleted Dags can sit in SCHEDULED until other cleanup; that is intentional.

What's the test plan?

New tests next to the create-path skip test:

  • test_executable_task_instances_skip_when_serialized_dag_missing: two tasks with concurrency limits, mock get_dag_for_run to None, queued list empty, both TIs still SCHEDULED
  • test_executable_task_instances_queue_when_serialized_dag_present: control, t1/t2 still queue

I restored the UPDATE and re-ran the skip test: TIs became FAILED (the first TI's UPDATE failed all SCHEDULED TIs including t2).

uv run --project airflow-core pytest \
airflow-core/tests/unit/jobs/test_scheduler_job.py \
-k 'skip_when_serialized_dag_missing or queue_when_serialized_dag_present or no_serdag'

3 passed. Ruff and airflow-core mypy passed via prek.


Was generative AI tooling used to co-author this PR?
  • Yes — Grok 4.6

Generated-by: Grok 4.6 following the guidelines


Drafted-by: Grok 4.6 (no human review before posting)

A concurrency check is not the place to mark every SCHEDULED task
FAILED. A parse blip then takes down the whole warehouse load;
the create path already skips and retries next tick.
@boring-cyborgboring-cyborgBot added the area:Scheduler including HA (high availability) scheduler label Aug 29, 2026
@Vamsi-klu
Vamsi-klu marked this pull request as ready for review August 29, 2026 03:32
Vamsi-kluand others added 3 commits August 29, 2026 03:57
test_queued_task_instances_fails_with_missing_dag still expected
FAILED and was outside the original -k selector, so CI would fail.
Align it with skip-not-fail.
A TLS reset during parallel provider version lookups failed the
constraints job on an otherwise green scheduler change.
Co-authored-by: Cursor <cursoragent@cursor.com>
@potiuk

Copy link
Copy Markdown
Member

Have you thought about side-effect of it? What are they? Do ypu (not your LLM) understand what your are doing here?have How it can happen that serialized dag is missing? Is it maybye a sign that something else is wrong - and you are just masking a problem?

Note - I have years of experience in Airflow but if I were to touch this code - I would think100 times and talk to someone over slack explaining how I reproduced .

Did you actually experience and reproduce it on running Airlfow instance? Do you have some proof of that?

Looking at the patterns of your contribution - you contribute like a shotgun - wherever your LLM thinks there is an issue, but you do not have deeper undersrtanding.

i am provisionally closing that - the "critical" parts of the code shoudl not be contributed to significantly without either experience or significant proof that you have run it locally, reproduced the issue and solved it and that you thought and reasoned about the consequences.

Thare are other areas where you can contribute smaller things in Airflow - if all you do is put your LLLm on it.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:Schedulerincluding HA (high availability) scheduler

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Scheduler bulk-fails all scheduled tasks when serialized DAG is transiently missing

2 participants

@Vamsi-klu@potiuk
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Do not fail scheduled tasks when serialized Dag is briefly missing - #72243

Closed
Vamsi-klu wants to merge 5 commits into
apache:mainfrom
Vamsi-klu:fix/62050-skip-missing-serialized-dag-queue
Closed

Do not fail scheduled tasks when serialized Dag is briefly missing#72243
Vamsi-klu wants to merge 5 commits into
apache:mainfrom
Vamsi-klu:fix/62050-skip-missing-serialized-dag-queue

Conversation

@Vamsi-klu

Copy link
Copy Markdown
Contributor

What is the change?

_task_concurrency_allows_execution no longer bulk-UPDATEs every SCHEDULED task instance to FAILED when get_dag_for_run returns None. It logs the same error as _create_dag_runs and returns False so this tick skips, and the next tick retries.

Why did I do it?

closes: #62050

The miss sits inside a concurrency check. A missing serialized row is not a concurrency answer. On HA schedulers, a parse or version hole then failed the whole warehouse load; retry worked because the next parse had a row. The create path already continues (test_scheduler_create_dag_runs_does_not_raise_error_when_no_serdag). Queue never got the same treatment. #58259 / #56422 made misses rarer; they did not remove the UPDATE.

How did I do it?

Deleted the session.execute(update(TI)...FAILED) block. return False was already there. Session stays uncommitted. I did not add a miss counter, did not fail only the one TI, and did not change _create_dag_runs. Permanently missing Dags stay SCHEDULED for the existing stale/import-error cleanup.

This path only runs when dag_model.has_task_concurrency_limits is True (max_active_tis_per_dag / max_active_tis_per_dagrun). Tests set max_active_tis_per_dag so the helper is actually entered.

What's the impact?

A transient serialized_dag hole no longer fails every SCHEDULED TI for that Dag (including every mapped index, backfill slice, and asset-triggered run that hits this helper). HA schedulers all skip instead of racing to stamp FAILED. Deleted Dags can sit in SCHEDULED until other cleanup; that is intentional.

What's the test plan?

New tests next to the create-path skip test:

  • test_executable_task_instances_skip_when_serialized_dag_missing: two tasks with concurrency limits, mock get_dag_for_run to None, queued list empty, both TIs still SCHEDULED
  • test_executable_task_instances_queue_when_serialized_dag_present: control, t1/t2 still queue

I restored the UPDATE and re-ran the skip test: TIs became FAILED (the first TI's UPDATE failed all SCHEDULED TIs including t2).

uv run --project airflow-core pytest \
airflow-core/tests/unit/jobs/test_scheduler_job.py \
-k 'skip_when_serialized_dag_missing or queue_when_serialized_dag_present or no_serdag'

3 passed. Ruff and airflow-core mypy passed via prek.


Was generative AI tooling used to co-author this PR?
  • Yes — Grok 4.6

Generated-by: Grok 4.6 following the guidelines


Drafted-by: Grok 4.6 (no human review before posting)

A concurrency check is not the place to mark every SCHEDULED task
FAILED. A parse blip then takes down the whole warehouse load;
the create path already skips and retries next tick.
@boring-cyborgboring-cyborgBot added the area:Scheduler including HA (high availability) scheduler label Aug 29, 2026
@Vamsi-klu
Vamsi-klu marked this pull request as ready for review August 29, 2026 03:32
Vamsi-kluand others added 3 commits August 29, 2026 03:57
test_queued_task_instances_fails_with_missing_dag still expected
FAILED and was outside the original -k selector, so CI would fail.
Align it with skip-not-fail.
A TLS reset during parallel provider version lookups failed the
constraints job on an otherwise green scheduler change.
Co-authored-by: Cursor <cursoragent@cursor.com>
@potiuk

Copy link
Copy Markdown
Member

Have you thought about side-effect of it? What are they? Do ypu (not your LLM) understand what your are doing here?have How it can happen that serialized dag is missing? Is it maybye a sign that something else is wrong - and you are just masking a problem?

Note - I have years of experience in Airflow but if I were to touch this code - I would think100 times and talk to someone over slack explaining how I reproduced .

Did you actually experience and reproduce it on running Airlfow instance? Do you have some proof of that?

Looking at the patterns of your contribution - you contribute like a shotgun - wherever your LLM thinks there is an issue, but you do not have deeper undersrtanding.

i am provisionally closing that - the "critical" parts of the code shoudl not be contributed to significantly without either experience or significant proof that you have run it locally, reproduced the issue and solved it and that you thought and reasoned about the consequences.

Thare are other areas where you can contribute smaller things in Airflow - if all you do is put your LLLm on it.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:Schedulerincluding HA (high availability) scheduler

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Scheduler bulk-fails all scheduled tasks when serialized DAG is transiently missing

2 participants

@Vamsi-klu@potiuk
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Do not fail scheduled tasks when serialized Dag is briefly missing - #72243

Closed
Vamsi-klu wants to merge 5 commits into
apache:mainfrom
Vamsi-klu:fix/62050-skip-missing-serialized-dag-queue
Closed

Do not fail scheduled tasks when serialized Dag is briefly missing#72243
Vamsi-klu wants to merge 5 commits into
apache:mainfrom
Vamsi-klu:fix/62050-skip-missing-serialized-dag-queue

Conversation

@Vamsi-klu

Copy link
Copy Markdown
Contributor

What is the change?

_task_concurrency_allows_execution no longer bulk-UPDATEs every SCHEDULED task instance to FAILED when get_dag_for_run returns None. It logs the same error as _create_dag_runs and returns False so this tick skips, and the next tick retries.

Why did I do it?

closes: #62050

The miss sits inside a concurrency check. A missing serialized row is not a concurrency answer. On HA schedulers, a parse or version hole then failed the whole warehouse load; retry worked because the next parse had a row. The create path already continues (test_scheduler_create_dag_runs_does_not_raise_error_when_no_serdag). Queue never got the same treatment. #58259 / #56422 made misses rarer; they did not remove the UPDATE.

How did I do it?

Deleted the session.execute(update(TI)...FAILED) block. return False was already there. Session stays uncommitted. I did not add a miss counter, did not fail only the one TI, and did not change _create_dag_runs. Permanently missing Dags stay SCHEDULED for the existing stale/import-error cleanup.

This path only runs when dag_model.has_task_concurrency_limits is True (max_active_tis_per_dag / max_active_tis_per_dagrun). Tests set max_active_tis_per_dag so the helper is actually entered.

What's the impact?

A transient serialized_dag hole no longer fails every SCHEDULED TI for that Dag (including every mapped index, backfill slice, and asset-triggered run that hits this helper). HA schedulers all skip instead of racing to stamp FAILED. Deleted Dags can sit in SCHEDULED until other cleanup; that is intentional.

What's the test plan?

New tests next to the create-path skip test:

  • test_executable_task_instances_skip_when_serialized_dag_missing: two tasks with concurrency limits, mock get_dag_for_run to None, queued list empty, both TIs still SCHEDULED
  • test_executable_task_instances_queue_when_serialized_dag_present: control, t1/t2 still queue

I restored the UPDATE and re-ran the skip test: TIs became FAILED (the first TI's UPDATE failed all SCHEDULED TIs including t2).

uv run --project airflow-core pytest \
airflow-core/tests/unit/jobs/test_scheduler_job.py \
-k 'skip_when_serialized_dag_missing or queue_when_serialized_dag_present or no_serdag'

3 passed. Ruff and airflow-core mypy passed via prek.


Was generative AI tooling used to co-author this PR?
  • Yes — Grok 4.6

Generated-by: Grok 4.6 following the guidelines


Drafted-by: Grok 4.6 (no human review before posting)

A concurrency check is not the place to mark every SCHEDULED task
FAILED. A parse blip then takes down the whole warehouse load;
the create path already skips and retries next tick.
@boring-cyborgboring-cyborgBot added the area:Scheduler including HA (high availability) scheduler label Aug 29, 2026
@Vamsi-klu
Vamsi-klu marked this pull request as ready for review August 29, 2026 03:32
Vamsi-kluand others added 3 commits August 29, 2026 03:57
test_queued_task_instances_fails_with_missing_dag still expected
FAILED and was outside the original -k selector, so CI would fail.
Align it with skip-not-fail.
A TLS reset during parallel provider version lookups failed the
constraints job on an otherwise green scheduler change.
Co-authored-by: Cursor <cursoragent@cursor.com>
@potiuk

Copy link
Copy Markdown
Member

Have you thought about side-effect of it? What are they? Do ypu (not your LLM) understand what your are doing here?have How it can happen that serialized dag is missing? Is it maybye a sign that something else is wrong - and you are just masking a problem?

Note - I have years of experience in Airflow but if I were to touch this code - I would think100 times and talk to someone over slack explaining how I reproduced .

Did you actually experience and reproduce it on running Airlfow instance? Do you have some proof of that?

Looking at the patterns of your contribution - you contribute like a shotgun - wherever your LLM thinks there is an issue, but you do not have deeper undersrtanding.

i am provisionally closing that - the "critical" parts of the code shoudl not be contributed to significantly without either experience or significant proof that you have run it locally, reproduced the issue and solved it and that you thought and reasoned about the consequences.

Thare are other areas where you can contribute smaller things in Airflow - if all you do is put your LLLm on it.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:Schedulerincluding HA (high availability) scheduler

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Scheduler bulk-fails all scheduled tasks when serialized DAG is transiently missing

2 participants

@Vamsi-klu@potiuk