connector-airtable: bound reconcile chain + signal sync completion - #128

Closed
KrisBraun wants to merge 2 commits into
mainfrom
airtable/fix-reconcile-runaway
Closed

connector-airtable: bound reconcile chain + signal sync completion#128
KrisBraun wants to merge 2 commits into
mainfrom
airtable/fix-reconcile-runaway

Conversation

@KrisBraun

Copy link
Copy Markdown
Contributor

Summary

  • reconcileComments self-reschedules on every invocation with no chain-depth guard. In production a single Airtable instance ran the chain fast enough to starve the shared run-production queue for hours, blocking sync recovery for 4 unrelated users. We've stopped the immediate flood by setting twist_instance.suspended_at on that instance; the same flood will recur the moment any Airtable instance is unsuspended without this fix.
  • The connector also never called integrations.channelSyncCompleted, so the connection card's "Syncing" indicator stayed stuck after the initial backfill completed.

What changes

  • syncBatch now calls integrations.channelSyncCompleted(baseId) when the final table+page is exhausted, matching the canonical google-contacts pattern (connectors/google-contacts/src/google-contacts.ts:147). The runtime call is idempotent.
  • reconcileComments takes an optional chainCount argument (back-compat for already-scheduled callbacks: undefined → fresh chain) and stores the previous fire time in reconcile_last_at_<baseId>. Healthy ~30-minute chains reset the depth to 0; chains arriving faster than RECONCILE_INTERVAL_MS - 60s accumulate the depth and bail with a console.warn once it crosses RECONCILE_FAST_CHAIN_CAP (1000). The cap is intentionally a defense-in-depth safety net — a connector with thousands of comments to backfill will still chain that long, since reconcile pages within a single execution and reschedules at 30-minute intervals.
  • reconcileComments also calls channelSyncCompleted on its steady-state path so legacy instances whose backfill ran on the pre-completion version of syncBatch still clear the syncing indicator on the next reconcile tick.
  • stopSync clears the new reconcile_last_at_<baseId> key.

Out of scope

Per the brief, platform-side fixes (per-twist queue isolation, Tasks.send rate limiting, auto-suspend on burst rate, dead-letter queues) are being investigated separately. This PR is the immediate-cause resolution for the Airtable connector itself.

Other connectors

I read this connector specifically and didn't audit others, but the brief asked to flag any similar self-chain patterns I noticed in passing — none jumped out from the imports/files I had to read. A pass over connectors that schedule their own follow-up work via runTask (gmail, google-calendar, slack, etc.) is worth doing as a separate sweep but is out of scope here.

Test plan

  • Spot-check that a fresh sync of an Airtable base completes (initial backfill via syncBatch chain): connection card shows "Syncing" then clears once the final page is processed.
  • Spot-check that an empty / already-synced base returns from reconcileComments after one call without queueing another faster than the 30-minute interval; reconcile_last_at_<baseId> is set; chainCount ratchets only under fast self-chaining.
  • Confirm channelSyncCompleted is called when syncBatch exhausts all tables (the syncing indicator clears in the Flutter app).
  • Confirm channelSyncCompleted is called on the first reconcile tick of a legacy instance whose initial backfill ran before this change (idempotent on the runtime side, so safe to call again).
  • In a forced fast-chain scenario (manually invoke reconcileComments 1001 times in quick succession), the 1001st invocation logs Airtable reconcile: fast-chain cap reached, bailing out and stops the chain.

Rollout

After this PR merges, a plotday/core PR will need to bump the submodule pointer; plot deploy then rolls out the new connector version. The suspended Airtable instance (019dbbb2-abe1-7faf-a89e-5379b655295b) should not be unsuspended until that deploy lands.

🤖 Generated with Claude Code

KrisBraunand others added 2 commits April 29, 2026 15:30
reconcileComments self-reschedules on every invocation with no chain-
depth guard. In production, a single instance ran the chain fast enough
to starve the shared task queue for hours, blocking sync recovery for
unrelated users. The connector also never called channelSyncCompleted,
so the connection card's "Syncing" indicator stayed stuck after the
initial backfill.
- syncBatch now calls integrations.channelSyncCompleted when the final
table+page is exhausted, matching the canonical google-contacts
pattern. Idempotent on the runtime side.
- reconcileComments takes an optional chainCount (back-compat for
already-scheduled callbacks) and tracks the previous fire timestamp
in store. Healthy 30-min chains reset the depth; faster-than-interval
chains accumulate it and bail at 1000. Logs a warn so a future runaway
surfaces in the in-app twist log.
- reconcileComments also calls channelSyncCompleted on its steady-state
path so legacy instances whose backfill ran on the pre-completion
version of syncBatch still clear the syncing indicator.
- stopSync clears the new reconcile_last_at_<baseId> key.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@KrisBraun
KrisBraun deleted the airtable/fix-reconcile-runaway branch April 29, 2026 21:06
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@KrisBraun
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

connector-airtable: bound reconcile chain + signal sync completion - #128

Closed
KrisBraun wants to merge 2 commits into
mainfrom
airtable/fix-reconcile-runaway
Closed

connector-airtable: bound reconcile chain + signal sync completion#128
KrisBraun wants to merge 2 commits into
mainfrom
airtable/fix-reconcile-runaway

Conversation

@KrisBraun

Copy link
Copy Markdown
Contributor

Summary

  • reconcileComments self-reschedules on every invocation with no chain-depth guard. In production a single Airtable instance ran the chain fast enough to starve the shared run-production queue for hours, blocking sync recovery for 4 unrelated users. We've stopped the immediate flood by setting twist_instance.suspended_at on that instance; the same flood will recur the moment any Airtable instance is unsuspended without this fix.
  • The connector also never called integrations.channelSyncCompleted, so the connection card's "Syncing" indicator stayed stuck after the initial backfill completed.

What changes

  • syncBatch now calls integrations.channelSyncCompleted(baseId) when the final table+page is exhausted, matching the canonical google-contacts pattern (connectors/google-contacts/src/google-contacts.ts:147). The runtime call is idempotent.
  • reconcileComments takes an optional chainCount argument (back-compat for already-scheduled callbacks: undefined → fresh chain) and stores the previous fire time in reconcile_last_at_<baseId>. Healthy ~30-minute chains reset the depth to 0; chains arriving faster than RECONCILE_INTERVAL_MS - 60s accumulate the depth and bail with a console.warn once it crosses RECONCILE_FAST_CHAIN_CAP (1000). The cap is intentionally a defense-in-depth safety net — a connector with thousands of comments to backfill will still chain that long, since reconcile pages within a single execution and reschedules at 30-minute intervals.
  • reconcileComments also calls channelSyncCompleted on its steady-state path so legacy instances whose backfill ran on the pre-completion version of syncBatch still clear the syncing indicator on the next reconcile tick.
  • stopSync clears the new reconcile_last_at_<baseId> key.

Out of scope

Per the brief, platform-side fixes (per-twist queue isolation, Tasks.send rate limiting, auto-suspend on burst rate, dead-letter queues) are being investigated separately. This PR is the immediate-cause resolution for the Airtable connector itself.

Other connectors

I read this connector specifically and didn't audit others, but the brief asked to flag any similar self-chain patterns I noticed in passing — none jumped out from the imports/files I had to read. A pass over connectors that schedule their own follow-up work via runTask (gmail, google-calendar, slack, etc.) is worth doing as a separate sweep but is out of scope here.

Test plan

  • Spot-check that a fresh sync of an Airtable base completes (initial backfill via syncBatch chain): connection card shows "Syncing" then clears once the final page is processed.
  • Spot-check that an empty / already-synced base returns from reconcileComments after one call without queueing another faster than the 30-minute interval; reconcile_last_at_<baseId> is set; chainCount ratchets only under fast self-chaining.
  • Confirm channelSyncCompleted is called when syncBatch exhausts all tables (the syncing indicator clears in the Flutter app).
  • Confirm channelSyncCompleted is called on the first reconcile tick of a legacy instance whose initial backfill ran before this change (idempotent on the runtime side, so safe to call again).
  • In a forced fast-chain scenario (manually invoke reconcileComments 1001 times in quick succession), the 1001st invocation logs Airtable reconcile: fast-chain cap reached, bailing out and stops the chain.

Rollout

After this PR merges, a plotday/core PR will need to bump the submodule pointer; plot deploy then rolls out the new connector version. The suspended Airtable instance (019dbbb2-abe1-7faf-a89e-5379b655295b) should not be unsuspended until that deploy lands.

🤖 Generated with Claude Code

KrisBraunand others added 2 commits April 29, 2026 15:30
reconcileComments self-reschedules on every invocation with no chain-
depth guard. In production, a single instance ran the chain fast enough
to starve the shared task queue for hours, blocking sync recovery for
unrelated users. The connector also never called channelSyncCompleted,
so the connection card's "Syncing" indicator stayed stuck after the
initial backfill.
- syncBatch now calls integrations.channelSyncCompleted when the final
table+page is exhausted, matching the canonical google-contacts
pattern. Idempotent on the runtime side.
- reconcileComments takes an optional chainCount (back-compat for
already-scheduled callbacks) and tracks the previous fire timestamp
in store. Healthy 30-min chains reset the depth; faster-than-interval
chains accumulate it and bail at 1000. Logs a warn so a future runaway
surfaces in the in-app twist log.
- reconcileComments also calls channelSyncCompleted on its steady-state
path so legacy instances whose backfill ran on the pre-completion
version of syncBatch still clear the syncing indicator.
- stopSync clears the new reconcile_last_at_<baseId> key.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@KrisBraun
KrisBraun deleted the airtable/fix-reconcile-runaway branch April 29, 2026 21:06
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@KrisBraun
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

connector-airtable: bound reconcile chain + signal sync completion - #128

Closed
KrisBraun wants to merge 2 commits into
mainfrom
airtable/fix-reconcile-runaway
Closed

connector-airtable: bound reconcile chain + signal sync completion#128
KrisBraun wants to merge 2 commits into
mainfrom
airtable/fix-reconcile-runaway

Conversation

@KrisBraun

Copy link
Copy Markdown
Contributor

Summary

  • reconcileComments self-reschedules on every invocation with no chain-depth guard. In production a single Airtable instance ran the chain fast enough to starve the shared run-production queue for hours, blocking sync recovery for 4 unrelated users. We've stopped the immediate flood by setting twist_instance.suspended_at on that instance; the same flood will recur the moment any Airtable instance is unsuspended without this fix.
  • The connector also never called integrations.channelSyncCompleted, so the connection card's "Syncing" indicator stayed stuck after the initial backfill completed.

What changes

  • syncBatch now calls integrations.channelSyncCompleted(baseId) when the final table+page is exhausted, matching the canonical google-contacts pattern (connectors/google-contacts/src/google-contacts.ts:147). The runtime call is idempotent.
  • reconcileComments takes an optional chainCount argument (back-compat for already-scheduled callbacks: undefined → fresh chain) and stores the previous fire time in reconcile_last_at_<baseId>. Healthy ~30-minute chains reset the depth to 0; chains arriving faster than RECONCILE_INTERVAL_MS - 60s accumulate the depth and bail with a console.warn once it crosses RECONCILE_FAST_CHAIN_CAP (1000). The cap is intentionally a defense-in-depth safety net — a connector with thousands of comments to backfill will still chain that long, since reconcile pages within a single execution and reschedules at 30-minute intervals.
  • reconcileComments also calls channelSyncCompleted on its steady-state path so legacy instances whose backfill ran on the pre-completion version of syncBatch still clear the syncing indicator on the next reconcile tick.
  • stopSync clears the new reconcile_last_at_<baseId> key.

Out of scope

Per the brief, platform-side fixes (per-twist queue isolation, Tasks.send rate limiting, auto-suspend on burst rate, dead-letter queues) are being investigated separately. This PR is the immediate-cause resolution for the Airtable connector itself.

Other connectors

I read this connector specifically and didn't audit others, but the brief asked to flag any similar self-chain patterns I noticed in passing — none jumped out from the imports/files I had to read. A pass over connectors that schedule their own follow-up work via runTask (gmail, google-calendar, slack, etc.) is worth doing as a separate sweep but is out of scope here.

Test plan

  • Spot-check that a fresh sync of an Airtable base completes (initial backfill via syncBatch chain): connection card shows "Syncing" then clears once the final page is processed.
  • Spot-check that an empty / already-synced base returns from reconcileComments after one call without queueing another faster than the 30-minute interval; reconcile_last_at_<baseId> is set; chainCount ratchets only under fast self-chaining.
  • Confirm channelSyncCompleted is called when syncBatch exhausts all tables (the syncing indicator clears in the Flutter app).
  • Confirm channelSyncCompleted is called on the first reconcile tick of a legacy instance whose initial backfill ran before this change (idempotent on the runtime side, so safe to call again).
  • In a forced fast-chain scenario (manually invoke reconcileComments 1001 times in quick succession), the 1001st invocation logs Airtable reconcile: fast-chain cap reached, bailing out and stops the chain.

Rollout

After this PR merges, a plotday/core PR will need to bump the submodule pointer; plot deploy then rolls out the new connector version. The suspended Airtable instance (019dbbb2-abe1-7faf-a89e-5379b655295b) should not be unsuspended until that deploy lands.

🤖 Generated with Claude Code

KrisBraunand others added 2 commits April 29, 2026 15:30
reconcileComments self-reschedules on every invocation with no chain-
depth guard. In production, a single instance ran the chain fast enough
to starve the shared task queue for hours, blocking sync recovery for
unrelated users. The connector also never called channelSyncCompleted,
so the connection card's "Syncing" indicator stayed stuck after the
initial backfill.
- syncBatch now calls integrations.channelSyncCompleted when the final
table+page is exhausted, matching the canonical google-contacts
pattern. Idempotent on the runtime side.
- reconcileComments takes an optional chainCount (back-compat for
already-scheduled callbacks) and tracks the previous fire timestamp
in store. Healthy 30-min chains reset the depth; faster-than-interval
chains accumulate it and bail at 1000. Logs a warn so a future runaway
surfaces in the in-app twist log.
- reconcileComments also calls channelSyncCompleted on its steady-state
path so legacy instances whose backfill ran on the pre-completion
version of syncBatch still clear the syncing indicator.
- stopSync clears the new reconcile_last_at_<baseId> key.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@KrisBraun
KrisBraun deleted the airtable/fix-reconcile-runaway branch April 29, 2026 21:06
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@KrisBraun
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

connector-airtable: bound reconcile chain + signal sync completion - #128

Closed
KrisBraun wants to merge 2 commits into
mainfrom
airtable/fix-reconcile-runaway
Closed

connector-airtable: bound reconcile chain + signal sync completion#128
KrisBraun wants to merge 2 commits into
mainfrom
airtable/fix-reconcile-runaway

Conversation

@KrisBraun

Copy link
Copy Markdown
Contributor

Summary

  • reconcileComments self-reschedules on every invocation with no chain-depth guard. In production a single Airtable instance ran the chain fast enough to starve the shared run-production queue for hours, blocking sync recovery for 4 unrelated users. We've stopped the immediate flood by setting twist_instance.suspended_at on that instance; the same flood will recur the moment any Airtable instance is unsuspended without this fix.
  • The connector also never called integrations.channelSyncCompleted, so the connection card's "Syncing" indicator stayed stuck after the initial backfill completed.

What changes

  • syncBatch now calls integrations.channelSyncCompleted(baseId) when the final table+page is exhausted, matching the canonical google-contacts pattern (connectors/google-contacts/src/google-contacts.ts:147). The runtime call is idempotent.
  • reconcileComments takes an optional chainCount argument (back-compat for already-scheduled callbacks: undefined → fresh chain) and stores the previous fire time in reconcile_last_at_<baseId>. Healthy ~30-minute chains reset the depth to 0; chains arriving faster than RECONCILE_INTERVAL_MS - 60s accumulate the depth and bail with a console.warn once it crosses RECONCILE_FAST_CHAIN_CAP (1000). The cap is intentionally a defense-in-depth safety net — a connector with thousands of comments to backfill will still chain that long, since reconcile pages within a single execution and reschedules at 30-minute intervals.
  • reconcileComments also calls channelSyncCompleted on its steady-state path so legacy instances whose backfill ran on the pre-completion version of syncBatch still clear the syncing indicator on the next reconcile tick.
  • stopSync clears the new reconcile_last_at_<baseId> key.

Out of scope

Per the brief, platform-side fixes (per-twist queue isolation, Tasks.send rate limiting, auto-suspend on burst rate, dead-letter queues) are being investigated separately. This PR is the immediate-cause resolution for the Airtable connector itself.

Other connectors

I read this connector specifically and didn't audit others, but the brief asked to flag any similar self-chain patterns I noticed in passing — none jumped out from the imports/files I had to read. A pass over connectors that schedule their own follow-up work via runTask (gmail, google-calendar, slack, etc.) is worth doing as a separate sweep but is out of scope here.

Test plan

  • Spot-check that a fresh sync of an Airtable base completes (initial backfill via syncBatch chain): connection card shows "Syncing" then clears once the final page is processed.
  • Spot-check that an empty / already-synced base returns from reconcileComments after one call without queueing another faster than the 30-minute interval; reconcile_last_at_<baseId> is set; chainCount ratchets only under fast self-chaining.
  • Confirm channelSyncCompleted is called when syncBatch exhausts all tables (the syncing indicator clears in the Flutter app).
  • Confirm channelSyncCompleted is called on the first reconcile tick of a legacy instance whose initial backfill ran before this change (idempotent on the runtime side, so safe to call again).
  • In a forced fast-chain scenario (manually invoke reconcileComments 1001 times in quick succession), the 1001st invocation logs Airtable reconcile: fast-chain cap reached, bailing out and stops the chain.

Rollout

After this PR merges, a plotday/core PR will need to bump the submodule pointer; plot deploy then rolls out the new connector version. The suspended Airtable instance (019dbbb2-abe1-7faf-a89e-5379b655295b) should not be unsuspended until that deploy lands.

🤖 Generated with Claude Code

KrisBraunand others added 2 commits April 29, 2026 15:30
reconcileComments self-reschedules on every invocation with no chain-
depth guard. In production, a single instance ran the chain fast enough
to starve the shared task queue for hours, blocking sync recovery for
unrelated users. The connector also never called channelSyncCompleted,
so the connection card's "Syncing" indicator stayed stuck after the
initial backfill.
- syncBatch now calls integrations.channelSyncCompleted when the final
table+page is exhausted, matching the canonical google-contacts
pattern. Idempotent on the runtime side.
- reconcileComments takes an optional chainCount (back-compat for
already-scheduled callbacks) and tracks the previous fire timestamp
in store. Healthy 30-min chains reset the depth; faster-than-interval
chains accumulate it and bail at 1000. Logs a warn so a future runaway
surfaces in the in-app twist log.
- reconcileComments also calls channelSyncCompleted on its steady-state
path so legacy instances whose backfill ran on the pre-completion
version of syncBatch still clear the syncing indicator.
- stopSync clears the new reconcile_last_at_<baseId> key.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@KrisBraun
KrisBraun deleted the airtable/fix-reconcile-runaway branch April 29, 2026 21:06
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@KrisBraun
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

connector-airtable: bound reconcile chain + signal sync completion - #128

Closed
KrisBraun wants to merge 2 commits into
mainfrom
airtable/fix-reconcile-runaway
Closed

connector-airtable: bound reconcile chain + signal sync completion#128
KrisBraun wants to merge 2 commits into
mainfrom
airtable/fix-reconcile-runaway

Conversation

@KrisBraun

Copy link
Copy Markdown
Contributor

Summary

  • reconcileComments self-reschedules on every invocation with no chain-depth guard. In production a single Airtable instance ran the chain fast enough to starve the shared run-production queue for hours, blocking sync recovery for 4 unrelated users. We've stopped the immediate flood by setting twist_instance.suspended_at on that instance; the same flood will recur the moment any Airtable instance is unsuspended without this fix.
  • The connector also never called integrations.channelSyncCompleted, so the connection card's "Syncing" indicator stayed stuck after the initial backfill completed.

What changes

  • syncBatch now calls integrations.channelSyncCompleted(baseId) when the final table+page is exhausted, matching the canonical google-contacts pattern (connectors/google-contacts/src/google-contacts.ts:147). The runtime call is idempotent.
  • reconcileComments takes an optional chainCount argument (back-compat for already-scheduled callbacks: undefined → fresh chain) and stores the previous fire time in reconcile_last_at_<baseId>. Healthy ~30-minute chains reset the depth to 0; chains arriving faster than RECONCILE_INTERVAL_MS - 60s accumulate the depth and bail with a console.warn once it crosses RECONCILE_FAST_CHAIN_CAP (1000). The cap is intentionally a defense-in-depth safety net — a connector with thousands of comments to backfill will still chain that long, since reconcile pages within a single execution and reschedules at 30-minute intervals.
  • reconcileComments also calls channelSyncCompleted on its steady-state path so legacy instances whose backfill ran on the pre-completion version of syncBatch still clear the syncing indicator on the next reconcile tick.
  • stopSync clears the new reconcile_last_at_<baseId> key.

Out of scope

Per the brief, platform-side fixes (per-twist queue isolation, Tasks.send rate limiting, auto-suspend on burst rate, dead-letter queues) are being investigated separately. This PR is the immediate-cause resolution for the Airtable connector itself.

Other connectors

I read this connector specifically and didn't audit others, but the brief asked to flag any similar self-chain patterns I noticed in passing — none jumped out from the imports/files I had to read. A pass over connectors that schedule their own follow-up work via runTask (gmail, google-calendar, slack, etc.) is worth doing as a separate sweep but is out of scope here.

Test plan

  • Spot-check that a fresh sync of an Airtable base completes (initial backfill via syncBatch chain): connection card shows "Syncing" then clears once the final page is processed.
  • Spot-check that an empty / already-synced base returns from reconcileComments after one call without queueing another faster than the 30-minute interval; reconcile_last_at_<baseId> is set; chainCount ratchets only under fast self-chaining.
  • Confirm channelSyncCompleted is called when syncBatch exhausts all tables (the syncing indicator clears in the Flutter app).
  • Confirm channelSyncCompleted is called on the first reconcile tick of a legacy instance whose initial backfill ran before this change (idempotent on the runtime side, so safe to call again).
  • In a forced fast-chain scenario (manually invoke reconcileComments 1001 times in quick succession), the 1001st invocation logs Airtable reconcile: fast-chain cap reached, bailing out and stops the chain.

Rollout

After this PR merges, a plotday/core PR will need to bump the submodule pointer; plot deploy then rolls out the new connector version. The suspended Airtable instance (019dbbb2-abe1-7faf-a89e-5379b655295b) should not be unsuspended until that deploy lands.

🤖 Generated with Claude Code

KrisBraunand others added 2 commits April 29, 2026 15:30
reconcileComments self-reschedules on every invocation with no chain-
depth guard. In production, a single instance ran the chain fast enough
to starve the shared task queue for hours, blocking sync recovery for
unrelated users. The connector also never called channelSyncCompleted,
so the connection card's "Syncing" indicator stayed stuck after the
initial backfill.
- syncBatch now calls integrations.channelSyncCompleted when the final
table+page is exhausted, matching the canonical google-contacts
pattern. Idempotent on the runtime side.
- reconcileComments takes an optional chainCount (back-compat for
already-scheduled callbacks) and tracks the previous fire timestamp
in store. Healthy 30-min chains reset the depth; faster-than-interval
chains accumulate it and bail at 1000. Logs a warn so a future runaway
surfaces in the in-app twist log.
- reconcileComments also calls channelSyncCompleted on its steady-state
path so legacy instances whose backfill ran on the pre-completion
version of syncBatch still clear the syncing indicator.
- stopSync clears the new reconcile_last_at_<baseId> key.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@KrisBraun
KrisBraun deleted the airtable/fix-reconcile-runaway branch April 29, 2026 21:06
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@KrisBraun
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

connector-airtable: bound reconcile chain + signal sync completion - #128

Closed
KrisBraun wants to merge 2 commits into
mainfrom
airtable/fix-reconcile-runaway
Closed

connector-airtable: bound reconcile chain + signal sync completion#128
KrisBraun wants to merge 2 commits into
mainfrom
airtable/fix-reconcile-runaway

Conversation

@KrisBraun

Copy link
Copy Markdown
Contributor

Summary

  • reconcileComments self-reschedules on every invocation with no chain-depth guard. In production a single Airtable instance ran the chain fast enough to starve the shared run-production queue for hours, blocking sync recovery for 4 unrelated users. We've stopped the immediate flood by setting twist_instance.suspended_at on that instance; the same flood will recur the moment any Airtable instance is unsuspended without this fix.
  • The connector also never called integrations.channelSyncCompleted, so the connection card's "Syncing" indicator stayed stuck after the initial backfill completed.

What changes

  • syncBatch now calls integrations.channelSyncCompleted(baseId) when the final table+page is exhausted, matching the canonical google-contacts pattern (connectors/google-contacts/src/google-contacts.ts:147). The runtime call is idempotent.
  • reconcileComments takes an optional chainCount argument (back-compat for already-scheduled callbacks: undefined → fresh chain) and stores the previous fire time in reconcile_last_at_<baseId>. Healthy ~30-minute chains reset the depth to 0; chains arriving faster than RECONCILE_INTERVAL_MS - 60s accumulate the depth and bail with a console.warn once it crosses RECONCILE_FAST_CHAIN_CAP (1000). The cap is intentionally a defense-in-depth safety net — a connector with thousands of comments to backfill will still chain that long, since reconcile pages within a single execution and reschedules at 30-minute intervals.
  • reconcileComments also calls channelSyncCompleted on its steady-state path so legacy instances whose backfill ran on the pre-completion version of syncBatch still clear the syncing indicator on the next reconcile tick.
  • stopSync clears the new reconcile_last_at_<baseId> key.

Out of scope

Per the brief, platform-side fixes (per-twist queue isolation, Tasks.send rate limiting, auto-suspend on burst rate, dead-letter queues) are being investigated separately. This PR is the immediate-cause resolution for the Airtable connector itself.

Other connectors

I read this connector specifically and didn't audit others, but the brief asked to flag any similar self-chain patterns I noticed in passing — none jumped out from the imports/files I had to read. A pass over connectors that schedule their own follow-up work via runTask (gmail, google-calendar, slack, etc.) is worth doing as a separate sweep but is out of scope here.

Test plan

  • Spot-check that a fresh sync of an Airtable base completes (initial backfill via syncBatch chain): connection card shows "Syncing" then clears once the final page is processed.
  • Spot-check that an empty / already-synced base returns from reconcileComments after one call without queueing another faster than the 30-minute interval; reconcile_last_at_<baseId> is set; chainCount ratchets only under fast self-chaining.
  • Confirm channelSyncCompleted is called when syncBatch exhausts all tables (the syncing indicator clears in the Flutter app).
  • Confirm channelSyncCompleted is called on the first reconcile tick of a legacy instance whose initial backfill ran before this change (idempotent on the runtime side, so safe to call again).
  • In a forced fast-chain scenario (manually invoke reconcileComments 1001 times in quick succession), the 1001st invocation logs Airtable reconcile: fast-chain cap reached, bailing out and stops the chain.

Rollout

After this PR merges, a plotday/core PR will need to bump the submodule pointer; plot deploy then rolls out the new connector version. The suspended Airtable instance (019dbbb2-abe1-7faf-a89e-5379b655295b) should not be unsuspended until that deploy lands.

🤖 Generated with Claude Code

KrisBraunand others added 2 commits April 29, 2026 15:30
reconcileComments self-reschedules on every invocation with no chain-
depth guard. In production, a single instance ran the chain fast enough
to starve the shared task queue for hours, blocking sync recovery for
unrelated users. The connector also never called channelSyncCompleted,
so the connection card's "Syncing" indicator stayed stuck after the
initial backfill.
- syncBatch now calls integrations.channelSyncCompleted when the final
table+page is exhausted, matching the canonical google-contacts
pattern. Idempotent on the runtime side.
- reconcileComments takes an optional chainCount (back-compat for
already-scheduled callbacks) and tracks the previous fire timestamp
in store. Healthy 30-min chains reset the depth; faster-than-interval
chains accumulate it and bail at 1000. Logs a warn so a future runaway
surfaces in the in-app twist log.
- reconcileComments also calls channelSyncCompleted on its steady-state
path so legacy instances whose backfill ran on the pre-completion
version of syncBatch still clear the syncing indicator.
- stopSync clears the new reconcile_last_at_<baseId> key.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@KrisBraun
KrisBraun deleted the airtable/fix-reconcile-runaway branch April 29, 2026 21:06
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@KrisBraun
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

connector-airtable: bound reconcile chain + signal sync completion - #128

Closed
KrisBraun wants to merge 2 commits into
mainfrom
airtable/fix-reconcile-runaway
Closed

connector-airtable: bound reconcile chain + signal sync completion#128
KrisBraun wants to merge 2 commits into
mainfrom
airtable/fix-reconcile-runaway

Conversation

@KrisBraun

Copy link
Copy Markdown
Contributor

Summary

  • reconcileComments self-reschedules on every invocation with no chain-depth guard. In production a single Airtable instance ran the chain fast enough to starve the shared run-production queue for hours, blocking sync recovery for 4 unrelated users. We've stopped the immediate flood by setting twist_instance.suspended_at on that instance; the same flood will recur the moment any Airtable instance is unsuspended without this fix.
  • The connector also never called integrations.channelSyncCompleted, so the connection card's "Syncing" indicator stayed stuck after the initial backfill completed.

What changes

  • syncBatch now calls integrations.channelSyncCompleted(baseId) when the final table+page is exhausted, matching the canonical google-contacts pattern (connectors/google-contacts/src/google-contacts.ts:147). The runtime call is idempotent.
  • reconcileComments takes an optional chainCount argument (back-compat for already-scheduled callbacks: undefined → fresh chain) and stores the previous fire time in reconcile_last_at_<baseId>. Healthy ~30-minute chains reset the depth to 0; chains arriving faster than RECONCILE_INTERVAL_MS - 60s accumulate the depth and bail with a console.warn once it crosses RECONCILE_FAST_CHAIN_CAP (1000). The cap is intentionally a defense-in-depth safety net — a connector with thousands of comments to backfill will still chain that long, since reconcile pages within a single execution and reschedules at 30-minute intervals.
  • reconcileComments also calls channelSyncCompleted on its steady-state path so legacy instances whose backfill ran on the pre-completion version of syncBatch still clear the syncing indicator on the next reconcile tick.
  • stopSync clears the new reconcile_last_at_<baseId> key.

Out of scope

Per the brief, platform-side fixes (per-twist queue isolation, Tasks.send rate limiting, auto-suspend on burst rate, dead-letter queues) are being investigated separately. This PR is the immediate-cause resolution for the Airtable connector itself.

Other connectors

I read this connector specifically and didn't audit others, but the brief asked to flag any similar self-chain patterns I noticed in passing — none jumped out from the imports/files I had to read. A pass over connectors that schedule their own follow-up work via runTask (gmail, google-calendar, slack, etc.) is worth doing as a separate sweep but is out of scope here.

Test plan

  • Spot-check that a fresh sync of an Airtable base completes (initial backfill via syncBatch chain): connection card shows "Syncing" then clears once the final page is processed.
  • Spot-check that an empty / already-synced base returns from reconcileComments after one call without queueing another faster than the 30-minute interval; reconcile_last_at_<baseId> is set; chainCount ratchets only under fast self-chaining.
  • Confirm channelSyncCompleted is called when syncBatch exhausts all tables (the syncing indicator clears in the Flutter app).
  • Confirm channelSyncCompleted is called on the first reconcile tick of a legacy instance whose initial backfill ran before this change (idempotent on the runtime side, so safe to call again).
  • In a forced fast-chain scenario (manually invoke reconcileComments 1001 times in quick succession), the 1001st invocation logs Airtable reconcile: fast-chain cap reached, bailing out and stops the chain.

Rollout

After this PR merges, a plotday/core PR will need to bump the submodule pointer; plot deploy then rolls out the new connector version. The suspended Airtable instance (019dbbb2-abe1-7faf-a89e-5379b655295b) should not be unsuspended until that deploy lands.

🤖 Generated with Claude Code

KrisBraunand others added 2 commits April 29, 2026 15:30
reconcileComments self-reschedules on every invocation with no chain-
depth guard. In production, a single instance ran the chain fast enough
to starve the shared task queue for hours, blocking sync recovery for
unrelated users. The connector also never called channelSyncCompleted,
so the connection card's "Syncing" indicator stayed stuck after the
initial backfill.
- syncBatch now calls integrations.channelSyncCompleted when the final
table+page is exhausted, matching the canonical google-contacts
pattern. Idempotent on the runtime side.
- reconcileComments takes an optional chainCount (back-compat for
already-scheduled callbacks) and tracks the previous fire timestamp
in store. Healthy 30-min chains reset the depth; faster-than-interval
chains accumulate it and bail at 1000. Logs a warn so a future runaway
surfaces in the in-app twist log.
- reconcileComments also calls channelSyncCompleted on its steady-state
path so legacy instances whose backfill ran on the pre-completion
version of syncBatch still clear the syncing indicator.
- stopSync clears the new reconcile_last_at_<baseId> key.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@KrisBraun
KrisBraun deleted the airtable/fix-reconcile-runaway branch April 29, 2026 21:06
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@KrisBraun
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

connector-airtable: bound reconcile chain + signal sync completion - #128

Closed
KrisBraun wants to merge 2 commits into
mainfrom
airtable/fix-reconcile-runaway
Closed

connector-airtable: bound reconcile chain + signal sync completion#128
KrisBraun wants to merge 2 commits into
mainfrom
airtable/fix-reconcile-runaway

Conversation

@KrisBraun

Copy link
Copy Markdown
Contributor

Summary

  • reconcileComments self-reschedules on every invocation with no chain-depth guard. In production a single Airtable instance ran the chain fast enough to starve the shared run-production queue for hours, blocking sync recovery for 4 unrelated users. We've stopped the immediate flood by setting twist_instance.suspended_at on that instance; the same flood will recur the moment any Airtable instance is unsuspended without this fix.
  • The connector also never called integrations.channelSyncCompleted, so the connection card's "Syncing" indicator stayed stuck after the initial backfill completed.

What changes

  • syncBatch now calls integrations.channelSyncCompleted(baseId) when the final table+page is exhausted, matching the canonical google-contacts pattern (connectors/google-contacts/src/google-contacts.ts:147). The runtime call is idempotent.
  • reconcileComments takes an optional chainCount argument (back-compat for already-scheduled callbacks: undefined → fresh chain) and stores the previous fire time in reconcile_last_at_<baseId>. Healthy ~30-minute chains reset the depth to 0; chains arriving faster than RECONCILE_INTERVAL_MS - 60s accumulate the depth and bail with a console.warn once it crosses RECONCILE_FAST_CHAIN_CAP (1000). The cap is intentionally a defense-in-depth safety net — a connector with thousands of comments to backfill will still chain that long, since reconcile pages within a single execution and reschedules at 30-minute intervals.
  • reconcileComments also calls channelSyncCompleted on its steady-state path so legacy instances whose backfill ran on the pre-completion version of syncBatch still clear the syncing indicator on the next reconcile tick.
  • stopSync clears the new reconcile_last_at_<baseId> key.

Out of scope

Per the brief, platform-side fixes (per-twist queue isolation, Tasks.send rate limiting, auto-suspend on burst rate, dead-letter queues) are being investigated separately. This PR is the immediate-cause resolution for the Airtable connector itself.

Other connectors

I read this connector specifically and didn't audit others, but the brief asked to flag any similar self-chain patterns I noticed in passing — none jumped out from the imports/files I had to read. A pass over connectors that schedule their own follow-up work via runTask (gmail, google-calendar, slack, etc.) is worth doing as a separate sweep but is out of scope here.

Test plan

  • Spot-check that a fresh sync of an Airtable base completes (initial backfill via syncBatch chain): connection card shows "Syncing" then clears once the final page is processed.
  • Spot-check that an empty / already-synced base returns from reconcileComments after one call without queueing another faster than the 30-minute interval; reconcile_last_at_<baseId> is set; chainCount ratchets only under fast self-chaining.
  • Confirm channelSyncCompleted is called when syncBatch exhausts all tables (the syncing indicator clears in the Flutter app).
  • Confirm channelSyncCompleted is called on the first reconcile tick of a legacy instance whose initial backfill ran before this change (idempotent on the runtime side, so safe to call again).
  • In a forced fast-chain scenario (manually invoke reconcileComments 1001 times in quick succession), the 1001st invocation logs Airtable reconcile: fast-chain cap reached, bailing out and stops the chain.

Rollout

After this PR merges, a plotday/core PR will need to bump the submodule pointer; plot deploy then rolls out the new connector version. The suspended Airtable instance (019dbbb2-abe1-7faf-a89e-5379b655295b) should not be unsuspended until that deploy lands.

🤖 Generated with Claude Code

KrisBraunand others added 2 commits April 29, 2026 15:30
reconcileComments self-reschedules on every invocation with no chain-
depth guard. In production, a single instance ran the chain fast enough
to starve the shared task queue for hours, blocking sync recovery for
unrelated users. The connector also never called channelSyncCompleted,
so the connection card's "Syncing" indicator stayed stuck after the
initial backfill.
- syncBatch now calls integrations.channelSyncCompleted when the final
table+page is exhausted, matching the canonical google-contacts
pattern. Idempotent on the runtime side.
- reconcileComments takes an optional chainCount (back-compat for
already-scheduled callbacks) and tracks the previous fire timestamp
in store. Healthy 30-min chains reset the depth; faster-than-interval
chains accumulate it and bail at 1000. Logs a warn so a future runaway
surfaces in the in-app twist log.
- reconcileComments also calls channelSyncCompleted on its steady-state
path so legacy instances whose backfill ran on the pre-completion
version of syncBatch still clear the syncing indicator.
- stopSync clears the new reconcile_last_at_<baseId> key.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@KrisBraun
KrisBraun deleted the airtable/fix-reconcile-runaway branch April 29, 2026 21:06
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@KrisBraun