connector-airtable: webhook-first sync, drop standard-plan comments

[KrisBraun]

Summary

Replaces the 30-minute "walk every record in every detected task table" reconcile loop with a webhook-first architecture that uses Airtable's delivery + cursor model as the truth source, plus explicit health tracking and auto-recovery for the failure modes the old polling masked.

The 30-min reconcile was masking three real reliability problems:

• Webhooks expire 7 days after creation/refresh; the connector never refreshed them, so they silently died and the reconcile became the only thing keeping data flowing.
• Missed deliveries (transient 5xx, network blip) advanced no cursor and were never retried, dropping changes until the next reconcile re-read every record.
• Schema changes (renamed status field, new options) weren't subscribed to, so cached task_tables_${baseId} went stale until the next channel re-enable.

Comment sync is dropped entirely. Airtable's standard webhooks have no comment data type — that's only available via the Enterprise-only Change Events API — and adding a comment doesn't update Last Modified Time either, so there's no efficient filterByFormula workaround. Removing it eliminates the dominant per-reconcile cost (one listComments call per record).

What's new

• Webhook expiration tracking — setupWebhook persists expirationTime and schedules a renewWebhook ~24h before expiry. On /refresh it persists the new expiration and re-schedules; on 404 it recreates from scratch; on auth failure it bails (user reconnect required); on transient errors it retries in 1h.
• Schema-change subscription — dataTypes is now ["tableData", "tableFields"]; recordChangeScope dropped so the subscription is base-wide. applyPayload re-runs scoreTable when field events arrive, keeping the task_tables map fresh without webhook recreation.
• Self-rescheduling payload drain — processWebhookPayloads self-reschedules when it exits the 10-batch guard with mightHaveMore: true; bumps last_activity_at_${baseId} on ingestion.
• 6h safety poll — new pollWebhookPayloads runs every 6h per enabled channel: catches missed deliveries via cursor, keeps the webhook alive (the /payloads endpoint resets Airtable's 7-day inactivity timer), and triggers recoverWebhook on a 404, missing webhook id, or 14+ days of total silence.
• Auto-recovery — recoverWebhook clears webhook state, bumps last_activity_at to "now" (so a failed recovery doesn't re-fire next poll), then queues setupWebhook + detectAndSync(initialSync=false) to re-emit current record state after a confirmed-or-suspected gap.
• Comment sync removed — drops data.recordComments:read/:write scopes, handleReplies, onNoteCreated/onNoteUpdated, mention translation helpers, and the listComments/createComment/updateComment client methods.

Migration

In-flight reconcileComments tasks from currently-deployed connectors keep dispatching after upgrade — the method is preserved as a one-shot shim that runs:

1. Phase 2 backfill: schedules renewWebhook for channels missing webhook_expires_at (would otherwise silently die at the 7-day mark)
2. Phase 4 backfill: bootstraps pollWebhookPayloads, gated by a poll_initialized flag also set in onChannelEnabled to prevent duplicate poll chains

Then exits without rescheduling. After ~1 hour post-deploy, all in-flight reconcile tasks have fired once and the shim can be deleted in a follow-up.

Existing comment-keyed Plot notes (key="comment-<airtableId>") remain in users' threads as orphan local notes — no data deletion, just no further sync to/from Airtable.

Cost reduction

Old: every base, every 30 minutes, ~2N requests for N records (full table walk + per-record listComments).

New:

• Real-time record updates via webhook delivery
• 6-hourly safety poll: ~1 request unless deliveries were missed
• 14-day stale recovery for quiet bases: ~N requests per 14 days (one detectAndSync)

For a 1k-record base: ~700× cost reduction.

Test plan

• Connect a fresh Airtable base, verify webhook is created with dataTypes: ["tableData", "tableFields"] (check Airtable's webhook list endpoint or stored webhook id)
• Make a record edit in Airtable, verify Plot thread updates within seconds
• Rename a status field's options, verify the cached task_tables_${baseId} reflects the new option names without channel re-enable
• Disable channel, verify all *_${baseId} storage keys are cleared
• Re-enable channel, verify fresh webhook + initial detectAndSync runs
• (Time-dependent) Verify renewWebhook runs ~6 days after channel enable and pushes expiration out
• (Time-dependent) Verify pollWebhookPayloads runs every 6h and processes any pending payloads
• Manually delete the webhook in Airtable; on next safety poll, verify connector recreates it and triggers detectAndSync
• Verify pre-deploy in-flight reconcileComments tasks fire once after upgrade and don't reschedule (check via storage keys + log absence after 30 min)

🤖 Generated with Claude Code

[KrisBraun]

Summary

Replaces the 30-minute "walk every record in every detected task table" reconcile loop with a webhook-first architecture that uses Airtable's delivery + cursor model as the truth source, plus explicit health tracking and auto-recovery for the failure modes the old polling masked.

The 30-min reconcile was masking three real reliability problems:

• Webhooks expire 7 days after creation/refresh; the connector never refreshed them, so they silently died and the reconcile became the only thing keeping data flowing.
• Missed deliveries (transient 5xx, network blip) advanced no cursor and were never retried, dropping changes until the next reconcile re-read every record.
• Schema changes (renamed status field, new options) weren't subscribed to, so cached task_tables_${baseId} went stale until the next channel re-enable.

Comment sync is dropped entirely. Airtable's standard webhooks have no comment data type — that's only available via the Enterprise-only Change Events API — and adding a comment doesn't update Last Modified Time either, so there's no efficient filterByFormula workaround. Removing it eliminates the dominant per-reconcile cost (one listComments call per record).

What's new

• Webhook expiration tracking — setupWebhook persists expirationTime and schedules a renewWebhook ~24h before expiry. On /refresh it persists the new expiration and re-schedules; on 404 it recreates from scratch; on auth failure it bails (user reconnect required); on transient errors it retries in 1h.
• Schema-change subscription — dataTypes is now ["tableData", "tableFields"]; recordChangeScope dropped so the subscription is base-wide. applyPayload re-runs scoreTable when field events arrive, keeping the task_tables map fresh without webhook recreation.
• Self-rescheduling payload drain — processWebhookPayloads self-reschedules when it exits the 10-batch guard with mightHaveMore: true; bumps last_activity_at_${baseId} on ingestion.
• 6h safety poll — new pollWebhookPayloads runs every 6h per enabled channel: catches missed deliveries via cursor, keeps the webhook alive (the /payloads endpoint resets Airtable's 7-day inactivity timer), and triggers recoverWebhook on a 404, missing webhook id, or 14+ days of total silence.
• Auto-recovery — recoverWebhook clears webhook state, bumps last_activity_at to "now" (so a failed recovery doesn't re-fire next poll), then queues setupWebhook + detectAndSync(initialSync=false) to re-emit current record state after a confirmed-or-suspected gap.
• Comment sync removed — drops data.recordComments:read/:write scopes, handleReplies, onNoteCreated/onNoteUpdated, mention translation helpers, and the listComments/createComment/updateComment client methods.

Migration

In-flight reconcileComments tasks from currently-deployed connectors keep dispatching after upgrade — the method is preserved as a one-shot shim that runs:

1. Phase 2 backfill: schedules renewWebhook for channels missing webhook_expires_at (would otherwise silently die at the 7-day mark)
2. Phase 4 backfill: bootstraps pollWebhookPayloads, gated by a poll_initialized flag also set in onChannelEnabled to prevent duplicate poll chains

Then exits without rescheduling. After ~1 hour post-deploy, all in-flight reconcile tasks have fired once and the shim can be deleted in a follow-up.

Existing comment-keyed Plot notes (key="comment-<airtableId>") remain in users' threads as orphan local notes — no data deletion, just no further sync to/from Airtable.

Cost reduction

Old: every base, every 30 minutes, ~2N requests for N records (full table walk + per-record listComments).

New:

• Real-time record updates via webhook delivery
• 6-hourly safety poll: ~1 request unless deliveries were missed
• 14-day stale recovery for quiet bases: ~N requests per 14 days (one detectAndSync)

For a 1k-record base: ~700× cost reduction.

Test plan

• Connect a fresh Airtable base, verify webhook is created with dataTypes: ["tableData", "tableFields"] (check Airtable's webhook list endpoint or stored webhook id)
• Make a record edit in Airtable, verify Plot thread updates within seconds
• Rename a status field's options, verify the cached task_tables_${baseId} reflects the new option names without channel re-enable
• Disable channel, verify all *_${baseId} storage keys are cleared
• Re-enable channel, verify fresh webhook + initial detectAndSync runs
• (Time-dependent) Verify renewWebhook runs ~6 days after channel enable and pushes expiration out
• (Time-dependent) Verify pollWebhookPayloads runs every 6h and processes any pending payloads
• Manually delete the webhook in Airtable; on next safety poll, verify connector recreates it and triggers detectAndSync
• Verify pre-deploy in-flight reconcileComments tasks fire once after upgrade and don't reschedule (check via storage keys + log absence after 30 min)

🤖 Generated with Claude Code

[KrisBraun]

Summary

Replaces the 30-minute "walk every record in every detected task table" reconcile loop with a webhook-first architecture that uses Airtable's delivery + cursor model as the truth source, plus explicit health tracking and auto-recovery for the failure modes the old polling masked.

The 30-min reconcile was masking three real reliability problems:

• Webhooks expire 7 days after creation/refresh; the connector never refreshed them, so they silently died and the reconcile became the only thing keeping data flowing.
• Missed deliveries (transient 5xx, network blip) advanced no cursor and were never retried, dropping changes until the next reconcile re-read every record.
• Schema changes (renamed status field, new options) weren't subscribed to, so cached task_tables_${baseId} went stale until the next channel re-enable.

Comment sync is dropped entirely. Airtable's standard webhooks have no comment data type — that's only available via the Enterprise-only Change Events API — and adding a comment doesn't update Last Modified Time either, so there's no efficient filterByFormula workaround. Removing it eliminates the dominant per-reconcile cost (one listComments call per record).

What's new

• Webhook expiration tracking — setupWebhook persists expirationTime and schedules a renewWebhook ~24h before expiry. On /refresh it persists the new expiration and re-schedules; on 404 it recreates from scratch; on auth failure it bails (user reconnect required); on transient errors it retries in 1h.
• Schema-change subscription — dataTypes is now ["tableData", "tableFields"]; recordChangeScope dropped so the subscription is base-wide. applyPayload re-runs scoreTable when field events arrive, keeping the task_tables map fresh without webhook recreation.
• Self-rescheduling payload drain — processWebhookPayloads self-reschedules when it exits the 10-batch guard with mightHaveMore: true; bumps last_activity_at_${baseId} on ingestion.
• 6h safety poll — new pollWebhookPayloads runs every 6h per enabled channel: catches missed deliveries via cursor, keeps the webhook alive (the /payloads endpoint resets Airtable's 7-day inactivity timer), and triggers recoverWebhook on a 404, missing webhook id, or 14+ days of total silence.
• Auto-recovery — recoverWebhook clears webhook state, bumps last_activity_at to "now" (so a failed recovery doesn't re-fire next poll), then queues setupWebhook + detectAndSync(initialSync=false) to re-emit current record state after a confirmed-or-suspected gap.
• Comment sync removed — drops data.recordComments:read/:write scopes, handleReplies, onNoteCreated/onNoteUpdated, mention translation helpers, and the listComments/createComment/updateComment client methods.

Migration

In-flight reconcileComments tasks from currently-deployed connectors keep dispatching after upgrade — the method is preserved as a one-shot shim that runs:

1. Phase 2 backfill: schedules renewWebhook for channels missing webhook_expires_at (would otherwise silently die at the 7-day mark)
2. Phase 4 backfill: bootstraps pollWebhookPayloads, gated by a poll_initialized flag also set in onChannelEnabled to prevent duplicate poll chains

Then exits without rescheduling. After ~1 hour post-deploy, all in-flight reconcile tasks have fired once and the shim can be deleted in a follow-up.

Existing comment-keyed Plot notes (key="comment-<airtableId>") remain in users' threads as orphan local notes — no data deletion, just no further sync to/from Airtable.

Cost reduction

Old: every base, every 30 minutes, ~2N requests for N records (full table walk + per-record listComments).

New:

• Real-time record updates via webhook delivery
• 6-hourly safety poll: ~1 request unless deliveries were missed
• 14-day stale recovery for quiet bases: ~N requests per 14 days (one detectAndSync)

For a 1k-record base: ~700× cost reduction.

Test plan

• Connect a fresh Airtable base, verify webhook is created with dataTypes: ["tableData", "tableFields"] (check Airtable's webhook list endpoint or stored webhook id)
• Make a record edit in Airtable, verify Plot thread updates within seconds
• Rename a status field's options, verify the cached task_tables_${baseId} reflects the new option names without channel re-enable
• Disable channel, verify all *_${baseId} storage keys are cleared
• Re-enable channel, verify fresh webhook + initial detectAndSync runs
• (Time-dependent) Verify renewWebhook runs ~6 days after channel enable and pushes expiration out
• (Time-dependent) Verify pollWebhookPayloads runs every 6h and processes any pending payloads
• Manually delete the webhook in Airtable; on next safety poll, verify connector recreates it and triggers detectAndSync
• Verify pre-deploy in-flight reconcileComments tasks fire once after upgrade and don't reschedule (check via storage keys + log absence after 30 min)

🤖 Generated with Claude Code

[KrisBraun]

Summary

Replaces the 30-minute "walk every record in every detected task table" reconcile loop with a webhook-first architecture that uses Airtable's delivery + cursor model as the truth source, plus explicit health tracking and auto-recovery for the failure modes the old polling masked.

The 30-min reconcile was masking three real reliability problems:

• Webhooks expire 7 days after creation/refresh; the connector never refreshed them, so they silently died and the reconcile became the only thing keeping data flowing.
• Missed deliveries (transient 5xx, network blip) advanced no cursor and were never retried, dropping changes until the next reconcile re-read every record.
• Schema changes (renamed status field, new options) weren't subscribed to, so cached task_tables_${baseId} went stale until the next channel re-enable.

Comment sync is dropped entirely. Airtable's standard webhooks have no comment data type — that's only available via the Enterprise-only Change Events API — and adding a comment doesn't update Last Modified Time either, so there's no efficient filterByFormula workaround. Removing it eliminates the dominant per-reconcile cost (one listComments call per record).

What's new

• Webhook expiration tracking — setupWebhook persists expirationTime and schedules a renewWebhook ~24h before expiry. On /refresh it persists the new expiration and re-schedules; on 404 it recreates from scratch; on auth failure it bails (user reconnect required); on transient errors it retries in 1h.
• Schema-change subscription — dataTypes is now ["tableData", "tableFields"]; recordChangeScope dropped so the subscription is base-wide. applyPayload re-runs scoreTable when field events arrive, keeping the task_tables map fresh without webhook recreation.
• Self-rescheduling payload drain — processWebhookPayloads self-reschedules when it exits the 10-batch guard with mightHaveMore: true; bumps last_activity_at_${baseId} on ingestion.
• 6h safety poll — new pollWebhookPayloads runs every 6h per enabled channel: catches missed deliveries via cursor, keeps the webhook alive (the /payloads endpoint resets Airtable's 7-day inactivity timer), and triggers recoverWebhook on a 404, missing webhook id, or 14+ days of total silence.
• Auto-recovery — recoverWebhook clears webhook state, bumps last_activity_at to "now" (so a failed recovery doesn't re-fire next poll), then queues setupWebhook + detectAndSync(initialSync=false) to re-emit current record state after a confirmed-or-suspected gap.
• Comment sync removed — drops data.recordComments:read/:write scopes, handleReplies, onNoteCreated/onNoteUpdated, mention translation helpers, and the listComments/createComment/updateComment client methods.

Migration

In-flight reconcileComments tasks from currently-deployed connectors keep dispatching after upgrade — the method is preserved as a one-shot shim that runs:

1. Phase 2 backfill: schedules renewWebhook for channels missing webhook_expires_at (would otherwise silently die at the 7-day mark)
2. Phase 4 backfill: bootstraps pollWebhookPayloads, gated by a poll_initialized flag also set in onChannelEnabled to prevent duplicate poll chains

Then exits without rescheduling. After ~1 hour post-deploy, all in-flight reconcile tasks have fired once and the shim can be deleted in a follow-up.

Existing comment-keyed Plot notes (key="comment-<airtableId>") remain in users' threads as orphan local notes — no data deletion, just no further sync to/from Airtable.

Cost reduction

Old: every base, every 30 minutes, ~2N requests for N records (full table walk + per-record listComments).

New:

• Real-time record updates via webhook delivery
• 6-hourly safety poll: ~1 request unless deliveries were missed
• 14-day stale recovery for quiet bases: ~N requests per 14 days (one detectAndSync)

For a 1k-record base: ~700× cost reduction.

Test plan

• Connect a fresh Airtable base, verify webhook is created with dataTypes: ["tableData", "tableFields"] (check Airtable's webhook list endpoint or stored webhook id)
• Make a record edit in Airtable, verify Plot thread updates within seconds
• Rename a status field's options, verify the cached task_tables_${baseId} reflects the new option names without channel re-enable
• Disable channel, verify all *_${baseId} storage keys are cleared
• Re-enable channel, verify fresh webhook + initial detectAndSync runs
• (Time-dependent) Verify renewWebhook runs ~6 days after channel enable and pushes expiration out
• (Time-dependent) Verify pollWebhookPayloads runs every 6h and processes any pending payloads
• Manually delete the webhook in Airtable; on next safety poll, verify connector recreates it and triggers detectAndSync
• Verify pre-deploy in-flight reconcileComments tasks fire once after upgrade and don't reschedule (check via storage keys + log absence after 30 min)

🤖 Generated with Claude Code

[KrisBraun]

Summary

Replaces the 30-minute "walk every record in every detected task table" reconcile loop with a webhook-first architecture that uses Airtable's delivery + cursor model as the truth source, plus explicit health tracking and auto-recovery for the failure modes the old polling masked.

The 30-min reconcile was masking three real reliability problems:

• Webhooks expire 7 days after creation/refresh; the connector never refreshed them, so they silently died and the reconcile became the only thing keeping data flowing.
• Missed deliveries (transient 5xx, network blip) advanced no cursor and were never retried, dropping changes until the next reconcile re-read every record.
• Schema changes (renamed status field, new options) weren't subscribed to, so cached task_tables_${baseId} went stale until the next channel re-enable.

Comment sync is dropped entirely. Airtable's standard webhooks have no comment data type — that's only available via the Enterprise-only Change Events API — and adding a comment doesn't update Last Modified Time either, so there's no efficient filterByFormula workaround. Removing it eliminates the dominant per-reconcile cost (one listComments call per record).

What's new

• Webhook expiration tracking — setupWebhook persists expirationTime and schedules a renewWebhook ~24h before expiry. On /refresh it persists the new expiration and re-schedules; on 404 it recreates from scratch; on auth failure it bails (user reconnect required); on transient errors it retries in 1h.
• Schema-change subscription — dataTypes is now ["tableData", "tableFields"]; recordChangeScope dropped so the subscription is base-wide. applyPayload re-runs scoreTable when field events arrive, keeping the task_tables map fresh without webhook recreation.
• Self-rescheduling payload drain — processWebhookPayloads self-reschedules when it exits the 10-batch guard with mightHaveMore: true; bumps last_activity_at_${baseId} on ingestion.
• 6h safety poll — new pollWebhookPayloads runs every 6h per enabled channel: catches missed deliveries via cursor, keeps the webhook alive (the /payloads endpoint resets Airtable's 7-day inactivity timer), and triggers recoverWebhook on a 404, missing webhook id, or 14+ days of total silence.
• Auto-recovery — recoverWebhook clears webhook state, bumps last_activity_at to "now" (so a failed recovery doesn't re-fire next poll), then queues setupWebhook + detectAndSync(initialSync=false) to re-emit current record state after a confirmed-or-suspected gap.
• Comment sync removed — drops data.recordComments:read/:write scopes, handleReplies, onNoteCreated/onNoteUpdated, mention translation helpers, and the listComments/createComment/updateComment client methods.

Migration

In-flight reconcileComments tasks from currently-deployed connectors keep dispatching after upgrade — the method is preserved as a one-shot shim that runs:

1. Phase 2 backfill: schedules renewWebhook for channels missing webhook_expires_at (would otherwise silently die at the 7-day mark)
2. Phase 4 backfill: bootstraps pollWebhookPayloads, gated by a poll_initialized flag also set in onChannelEnabled to prevent duplicate poll chains

Then exits without rescheduling. After ~1 hour post-deploy, all in-flight reconcile tasks have fired once and the shim can be deleted in a follow-up.

Existing comment-keyed Plot notes (key="comment-<airtableId>") remain in users' threads as orphan local notes — no data deletion, just no further sync to/from Airtable.

Cost reduction

Old: every base, every 30 minutes, ~2N requests for N records (full table walk + per-record listComments).

New:

• Real-time record updates via webhook delivery
• 6-hourly safety poll: ~1 request unless deliveries were missed
• 14-day stale recovery for quiet bases: ~N requests per 14 days (one detectAndSync)

For a 1k-record base: ~700× cost reduction.

Test plan

• Connect a fresh Airtable base, verify webhook is created with dataTypes: ["tableData", "tableFields"] (check Airtable's webhook list endpoint or stored webhook id)
• Make a record edit in Airtable, verify Plot thread updates within seconds
• Rename a status field's options, verify the cached task_tables_${baseId} reflects the new option names without channel re-enable
• Disable channel, verify all *_${baseId} storage keys are cleared
• Re-enable channel, verify fresh webhook + initial detectAndSync runs
• (Time-dependent) Verify renewWebhook runs ~6 days after channel enable and pushes expiration out
• (Time-dependent) Verify pollWebhookPayloads runs every 6h and processes any pending payloads
• Manually delete the webhook in Airtable; on next safety poll, verify connector recreates it and triggers detectAndSync
• Verify pre-deploy in-flight reconcileComments tasks fire once after upgrade and don't reschedule (check via storage keys + log absence after 30 min)

🤖 Generated with Claude Code

[KrisBraun]

Summary

Replaces the 30-minute "walk every record in every detected task table" reconcile loop with a webhook-first architecture that uses Airtable's delivery + cursor model as the truth source, plus explicit health tracking and auto-recovery for the failure modes the old polling masked.

The 30-min reconcile was masking three real reliability problems:

• Webhooks expire 7 days after creation/refresh; the connector never refreshed them, so they silently died and the reconcile became the only thing keeping data flowing.
• Missed deliveries (transient 5xx, network blip) advanced no cursor and were never retried, dropping changes until the next reconcile re-read every record.
• Schema changes (renamed status field, new options) weren't subscribed to, so cached task_tables_${baseId} went stale until the next channel re-enable.

Comment sync is dropped entirely. Airtable's standard webhooks have no comment data type — that's only available via the Enterprise-only Change Events API — and adding a comment doesn't update Last Modified Time either, so there's no efficient filterByFormula workaround. Removing it eliminates the dominant per-reconcile cost (one listComments call per record).

What's new

• Webhook expiration tracking — setupWebhook persists expirationTime and schedules a renewWebhook ~24h before expiry. On /refresh it persists the new expiration and re-schedules; on 404 it recreates from scratch; on auth failure it bails (user reconnect required); on transient errors it retries in 1h.
• Schema-change subscription — dataTypes is now ["tableData", "tableFields"]; recordChangeScope dropped so the subscription is base-wide. applyPayload re-runs scoreTable when field events arrive, keeping the task_tables map fresh without webhook recreation.
• Self-rescheduling payload drain — processWebhookPayloads self-reschedules when it exits the 10-batch guard with mightHaveMore: true; bumps last_activity_at_${baseId} on ingestion.
• 6h safety poll — new pollWebhookPayloads runs every 6h per enabled channel: catches missed deliveries via cursor, keeps the webhook alive (the /payloads endpoint resets Airtable's 7-day inactivity timer), and triggers recoverWebhook on a 404, missing webhook id, or 14+ days of total silence.
• Auto-recovery — recoverWebhook clears webhook state, bumps last_activity_at to "now" (so a failed recovery doesn't re-fire next poll), then queues setupWebhook + detectAndSync(initialSync=false) to re-emit current record state after a confirmed-or-suspected gap.
• Comment sync removed — drops data.recordComments:read/:write scopes, handleReplies, onNoteCreated/onNoteUpdated, mention translation helpers, and the listComments/createComment/updateComment client methods.

Migration

In-flight reconcileComments tasks from currently-deployed connectors keep dispatching after upgrade — the method is preserved as a one-shot shim that runs:

1. Phase 2 backfill: schedules renewWebhook for channels missing webhook_expires_at (would otherwise silently die at the 7-day mark)
2. Phase 4 backfill: bootstraps pollWebhookPayloads, gated by a poll_initialized flag also set in onChannelEnabled to prevent duplicate poll chains

Then exits without rescheduling. After ~1 hour post-deploy, all in-flight reconcile tasks have fired once and the shim can be deleted in a follow-up.

Existing comment-keyed Plot notes (key="comment-<airtableId>") remain in users' threads as orphan local notes — no data deletion, just no further sync to/from Airtable.

Cost reduction

Old: every base, every 30 minutes, ~2N requests for N records (full table walk + per-record listComments).

New:

• Real-time record updates via webhook delivery
• 6-hourly safety poll: ~1 request unless deliveries were missed
• 14-day stale recovery for quiet bases: ~N requests per 14 days (one detectAndSync)

For a 1k-record base: ~700× cost reduction.

Test plan

• Connect a fresh Airtable base, verify webhook is created with dataTypes: ["tableData", "tableFields"] (check Airtable's webhook list endpoint or stored webhook id)
• Make a record edit in Airtable, verify Plot thread updates within seconds
• Rename a status field's options, verify the cached task_tables_${baseId} reflects the new option names without channel re-enable
• Disable channel, verify all *_${baseId} storage keys are cleared
• Re-enable channel, verify fresh webhook + initial detectAndSync runs
• (Time-dependent) Verify renewWebhook runs ~6 days after channel enable and pushes expiration out
• (Time-dependent) Verify pollWebhookPayloads runs every 6h and processes any pending payloads
• Manually delete the webhook in Airtable; on next safety poll, verify connector recreates it and triggers detectAndSync
• Verify pre-deploy in-flight reconcileComments tasks fire once after upgrade and don't reschedule (check via storage keys + log absence after 30 min)

🤖 Generated with Claude Code

[KrisBraun]

Summary

Replaces the 30-minute "walk every record in every detected task table" reconcile loop with a webhook-first architecture that uses Airtable's delivery + cursor model as the truth source, plus explicit health tracking and auto-recovery for the failure modes the old polling masked.

The 30-min reconcile was masking three real reliability problems:

• Webhooks expire 7 days after creation/refresh; the connector never refreshed them, so they silently died and the reconcile became the only thing keeping data flowing.
• Missed deliveries (transient 5xx, network blip) advanced no cursor and were never retried, dropping changes until the next reconcile re-read every record.
• Schema changes (renamed status field, new options) weren't subscribed to, so cached task_tables_${baseId} went stale until the next channel re-enable.

Comment sync is dropped entirely. Airtable's standard webhooks have no comment data type — that's only available via the Enterprise-only Change Events API — and adding a comment doesn't update Last Modified Time either, so there's no efficient filterByFormula workaround. Removing it eliminates the dominant per-reconcile cost (one listComments call per record).

What's new

• Webhook expiration tracking — setupWebhook persists expirationTime and schedules a renewWebhook ~24h before expiry. On /refresh it persists the new expiration and re-schedules; on 404 it recreates from scratch; on auth failure it bails (user reconnect required); on transient errors it retries in 1h.
• Schema-change subscription — dataTypes is now ["tableData", "tableFields"]; recordChangeScope dropped so the subscription is base-wide. applyPayload re-runs scoreTable when field events arrive, keeping the task_tables map fresh without webhook recreation.
• Self-rescheduling payload drain — processWebhookPayloads self-reschedules when it exits the 10-batch guard with mightHaveMore: true; bumps last_activity_at_${baseId} on ingestion.
• 6h safety poll — new pollWebhookPayloads runs every 6h per enabled channel: catches missed deliveries via cursor, keeps the webhook alive (the /payloads endpoint resets Airtable's 7-day inactivity timer), and triggers recoverWebhook on a 404, missing webhook id, or 14+ days of total silence.
• Auto-recovery — recoverWebhook clears webhook state, bumps last_activity_at to "now" (so a failed recovery doesn't re-fire next poll), then queues setupWebhook + detectAndSync(initialSync=false) to re-emit current record state after a confirmed-or-suspected gap.
• Comment sync removed — drops data.recordComments:read/:write scopes, handleReplies, onNoteCreated/onNoteUpdated, mention translation helpers, and the listComments/createComment/updateComment client methods.

Migration

In-flight reconcileComments tasks from currently-deployed connectors keep dispatching after upgrade — the method is preserved as a one-shot shim that runs:

1. Phase 2 backfill: schedules renewWebhook for channels missing webhook_expires_at (would otherwise silently die at the 7-day mark)
2. Phase 4 backfill: bootstraps pollWebhookPayloads, gated by a poll_initialized flag also set in onChannelEnabled to prevent duplicate poll chains

Then exits without rescheduling. After ~1 hour post-deploy, all in-flight reconcile tasks have fired once and the shim can be deleted in a follow-up.

Existing comment-keyed Plot notes (key="comment-<airtableId>") remain in users' threads as orphan local notes — no data deletion, just no further sync to/from Airtable.

Cost reduction

Old: every base, every 30 minutes, ~2N requests for N records (full table walk + per-record listComments).

New:

• Real-time record updates via webhook delivery
• 6-hourly safety poll: ~1 request unless deliveries were missed
• 14-day stale recovery for quiet bases: ~N requests per 14 days (one detectAndSync)

For a 1k-record base: ~700× cost reduction.

Test plan

• Connect a fresh Airtable base, verify webhook is created with dataTypes: ["tableData", "tableFields"] (check Airtable's webhook list endpoint or stored webhook id)
• Make a record edit in Airtable, verify Plot thread updates within seconds
• Rename a status field's options, verify the cached task_tables_${baseId} reflects the new option names without channel re-enable
• Disable channel, verify all *_${baseId} storage keys are cleared
• Re-enable channel, verify fresh webhook + initial detectAndSync runs
• (Time-dependent) Verify renewWebhook runs ~6 days after channel enable and pushes expiration out
• (Time-dependent) Verify pollWebhookPayloads runs every 6h and processes any pending payloads
• Manually delete the webhook in Airtable; on next safety poll, verify connector recreates it and triggers detectAndSync
• Verify pre-deploy in-flight reconcileComments tasks fire once after upgrade and don't reschedule (check via storage keys + log absence after 30 min)

🤖 Generated with Claude Code

[KrisBraun]

Summary

Replaces the 30-minute "walk every record in every detected task table" reconcile loop with a webhook-first architecture that uses Airtable's delivery + cursor model as the truth source, plus explicit health tracking and auto-recovery for the failure modes the old polling masked.

The 30-min reconcile was masking three real reliability problems:

• Webhooks expire 7 days after creation/refresh; the connector never refreshed them, so they silently died and the reconcile became the only thing keeping data flowing.
• Missed deliveries (transient 5xx, network blip) advanced no cursor and were never retried, dropping changes until the next reconcile re-read every record.
• Schema changes (renamed status field, new options) weren't subscribed to, so cached task_tables_${baseId} went stale until the next channel re-enable.

Comment sync is dropped entirely. Airtable's standard webhooks have no comment data type — that's only available via the Enterprise-only Change Events API — and adding a comment doesn't update Last Modified Time either, so there's no efficient filterByFormula workaround. Removing it eliminates the dominant per-reconcile cost (one listComments call per record).

What's new

• Webhook expiration tracking — setupWebhook persists expirationTime and schedules a renewWebhook ~24h before expiry. On /refresh it persists the new expiration and re-schedules; on 404 it recreates from scratch; on auth failure it bails (user reconnect required); on transient errors it retries in 1h.
• Schema-change subscription — dataTypes is now ["tableData", "tableFields"]; recordChangeScope dropped so the subscription is base-wide. applyPayload re-runs scoreTable when field events arrive, keeping the task_tables map fresh without webhook recreation.
• Self-rescheduling payload drain — processWebhookPayloads self-reschedules when it exits the 10-batch guard with mightHaveMore: true; bumps last_activity_at_${baseId} on ingestion.
• 6h safety poll — new pollWebhookPayloads runs every 6h per enabled channel: catches missed deliveries via cursor, keeps the webhook alive (the /payloads endpoint resets Airtable's 7-day inactivity timer), and triggers recoverWebhook on a 404, missing webhook id, or 14+ days of total silence.
• Auto-recovery — recoverWebhook clears webhook state, bumps last_activity_at to "now" (so a failed recovery doesn't re-fire next poll), then queues setupWebhook + detectAndSync(initialSync=false) to re-emit current record state after a confirmed-or-suspected gap.
• Comment sync removed — drops data.recordComments:read/:write scopes, handleReplies, onNoteCreated/onNoteUpdated, mention translation helpers, and the listComments/createComment/updateComment client methods.

Migration

In-flight reconcileComments tasks from currently-deployed connectors keep dispatching after upgrade — the method is preserved as a one-shot shim that runs:

1. Phase 2 backfill: schedules renewWebhook for channels missing webhook_expires_at (would otherwise silently die at the 7-day mark)
2. Phase 4 backfill: bootstraps pollWebhookPayloads, gated by a poll_initialized flag also set in onChannelEnabled to prevent duplicate poll chains

Then exits without rescheduling. After ~1 hour post-deploy, all in-flight reconcile tasks have fired once and the shim can be deleted in a follow-up.

Existing comment-keyed Plot notes (key="comment-<airtableId>") remain in users' threads as orphan local notes — no data deletion, just no further sync to/from Airtable.

Cost reduction

Old: every base, every 30 minutes, ~2N requests for N records (full table walk + per-record listComments).

New:

• Real-time record updates via webhook delivery
• 6-hourly safety poll: ~1 request unless deliveries were missed
• 14-day stale recovery for quiet bases: ~N requests per 14 days (one detectAndSync)

For a 1k-record base: ~700× cost reduction.

Test plan

• Connect a fresh Airtable base, verify webhook is created with dataTypes: ["tableData", "tableFields"] (check Airtable's webhook list endpoint or stored webhook id)
• Make a record edit in Airtable, verify Plot thread updates within seconds
• Rename a status field's options, verify the cached task_tables_${baseId} reflects the new option names without channel re-enable
• Disable channel, verify all *_${baseId} storage keys are cleared
• Re-enable channel, verify fresh webhook + initial detectAndSync runs
• (Time-dependent) Verify renewWebhook runs ~6 days after channel enable and pushes expiration out
• (Time-dependent) Verify pollWebhookPayloads runs every 6h and processes any pending payloads
• Manually delete the webhook in Airtable; on next safety poll, verify connector recreates it and triggers detectAndSync
• Verify pre-deploy in-flight reconcileComments tasks fire once after upgrade and don't reschedule (check via storage keys + log absence after 30 min)

🤖 Generated with Claude Code