connector-gmail: mailbox-wide watch + per-thread channel routing - #133

Merged
KrisBraun merged 1 commit into
mainfrom
gmail/mailbox-wide-watch
Apr 30, 2026
Merged

connector-gmail: mailbox-wide watch + per-thread channel routing#133
KrisBraun merged 1 commit into
mainfrom
gmail/mailbox-wide-watch

Conversation

@KrisBraun

Copy link
Copy Markdown
Contributor

Summary

  • Switch the Gmail connector to a single mailbox-wide `users.watch` per twist instance and route each changed thread to whichever enabled channel(s) it actually belongs to.
  • Fixes incoming replies on existing email threads being missed in Plot when the new reply isn't itself starred or auto-marked Important.
  • Adds an `upgrade()` migration that probes known system labels and resumes any in-flight backfills under the new callback path.

Why

Gmail's per-label `users.watch` and `users.history.list?labelId=` only fire/return for messages currently carrying that label. New replies in a thread don't inherit the thread's labels, so a reply that wasn't itself starred (or auto-marked Important by Gmail's heuristics) was missed entirely — no Pub/Sub notification, and no history record under the label-scoped filter. Threads silently stalled on their last sync.

The fix: register one mailbox-wide watch (no `labelIds`), call `history.list` without a `labelId` filter, and have the connector pick a channel per thread by inspecting the actual labels carried by its messages.

What changed

`connectors/gmail/src/gmail-api.ts`

  • `setupWatch(topicName, labelId?)` — `labelId` is now optional. When omitted, Gmail watches the whole mailbox.
  • New `syncGmailMailboxIncremental(api, historyId)` — calls `history.list` with no label, returns either `{ expired: true }` or every changed thread.

`connectors/gmail/src/gmail.ts`

  • One mailbox watch + Pub/Sub topic per twist instance. `ensureMailboxWebhook` (idempotent) runs whenever a channel is enabled; `teardownMailboxWebhook` runs when the last channel is disabled.
  • Two sync paths: `initialSyncBatch(channelId, batch)` for label-scoped first-time backfill, `incrementalSyncBatch()` for mailbox-wide changes.
  • New `pickChannelForThread` precedence: custom user labels (alphabetical) → STARRED → IMPORTANT → INBOX → SENT → DRAFT. Threads that don't match any enabled channel are skipped.
  • `onGmailWebhook(request, _channelId?)` — single handler. Ignores `channelId` for new mailbox webhooks; the optional arg is preserved so already-deployed per-channel Pub/Sub callbacks dispatch cleanly.
  • `upgrade()` migrates existing instances: stops the old Gmail watch, probes known system labels for old per-channel state, populates `enabled_channels`, resumes in-flight backfills, deletes old per-channel topics, and sets up the new mailbox webhook.
  • Backwards-compatible shims preserve the original signatures of `syncBatch`, `setupChannelWebhook`, and `renewWatch` so any already-queued callbacks finish gracefully.

Migration caveat

Connectors can't enumerate stored keys (`Tool.list()` exists but is private to built-in tools, not exposed on `Connector`). The `upgrade()` migration handles system-label channels (STARRED, IMPORTANT, INBOX, SENT, DRAFT) cleanly. Users with a custom Gmail label enabled (e.g. `Label_14`) will need to disable+re-enable that channel after deploy.

Test plan

  • Deploy the connector and verify `upgrade()` runs against an existing twist instance with STARRED + IMPORTANT enabled
  • Confirm new mailbox-wide `mailbox_webhook` Pub/Sub topic is created and the old per-channel topics are cleaned up
  • Send a new email to a Gmail thread that already has a starred message; verify the new reply (which is itself NOT starred) appears in Plot within seconds
  • Star a not-previously-starred message and confirm the corresponding thread starts syncing
  • Disable the last enabled channel and confirm the mailbox watch is torn down (`mailbox_webhook` cleared, Pub/Sub topic deleted)
  • Re-enable a channel and confirm the watch is recreated, plus a fresh per-channel initial backfill runs
  • Existing in-flight per-channel pageToken backfill resumes under `initialSyncBatch` after upgrade

🤖 Generated with Claude Code

Gmail's per-label `users.watch` and `users.history.list` filters only
return events for messages currently carrying the given label. New
replies in an existing thread don't inherit the thread's labels, so a
reply that wasn't itself starred or auto-marked Important was missed
entirely — both at the watch (no Pub/Sub notification) and at history
(no entry returned). Threads stayed stuck on their last sync.
Switch to a single mailbox-wide Gmail watch per twist instance, and
route each changed thread to whichever enabled channel(s) its messages
actually belong to. New keys: mailbox_webhook, mailbox_renewal_task,
incremental_state, enabled_channels, initial_state_<channelId>.
`upgrade()` migrates existing instances by probing known system labels
(STARRED/IMPORTANT/INBOX/SENT/DRAFT) and resuming any in-flight
backfills under the new callback. Custom-label channels need a
re-toggle after deploy — the runtime doesn't expose key listing to
connectors, so we can't discover their IDs.
Backwards-compatible shims preserve the original signatures of
syncBatch, setupChannelWebhook, renewWatch, and onGmailWebhook so
already-deployed callbacks finish gracefully.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@KrisBraun
KrisBraun merged commit 87ac389 into mainApr 30, 2026
1 check passed
@KrisBraun
KrisBraun deleted the gmail/mailbox-wide-watch branch April 30, 2026 19:25
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@KrisBraun
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

connector-gmail: mailbox-wide watch + per-thread channel routing - #133

Merged
KrisBraun merged 1 commit into
mainfrom
gmail/mailbox-wide-watch
Apr 30, 2026
Merged

connector-gmail: mailbox-wide watch + per-thread channel routing#133
KrisBraun merged 1 commit into
mainfrom
gmail/mailbox-wide-watch

Conversation

@KrisBraun

Copy link
Copy Markdown
Contributor

Summary

  • Switch the Gmail connector to a single mailbox-wide `users.watch` per twist instance and route each changed thread to whichever enabled channel(s) it actually belongs to.
  • Fixes incoming replies on existing email threads being missed in Plot when the new reply isn't itself starred or auto-marked Important.
  • Adds an `upgrade()` migration that probes known system labels and resumes any in-flight backfills under the new callback path.

Why

Gmail's per-label `users.watch` and `users.history.list?labelId=` only fire/return for messages currently carrying that label. New replies in a thread don't inherit the thread's labels, so a reply that wasn't itself starred (or auto-marked Important by Gmail's heuristics) was missed entirely — no Pub/Sub notification, and no history record under the label-scoped filter. Threads silently stalled on their last sync.

The fix: register one mailbox-wide watch (no `labelIds`), call `history.list` without a `labelId` filter, and have the connector pick a channel per thread by inspecting the actual labels carried by its messages.

What changed

`connectors/gmail/src/gmail-api.ts`

  • `setupWatch(topicName, labelId?)` — `labelId` is now optional. When omitted, Gmail watches the whole mailbox.
  • New `syncGmailMailboxIncremental(api, historyId)` — calls `history.list` with no label, returns either `{ expired: true }` or every changed thread.

`connectors/gmail/src/gmail.ts`

  • One mailbox watch + Pub/Sub topic per twist instance. `ensureMailboxWebhook` (idempotent) runs whenever a channel is enabled; `teardownMailboxWebhook` runs when the last channel is disabled.
  • Two sync paths: `initialSyncBatch(channelId, batch)` for label-scoped first-time backfill, `incrementalSyncBatch()` for mailbox-wide changes.
  • New `pickChannelForThread` precedence: custom user labels (alphabetical) → STARRED → IMPORTANT → INBOX → SENT → DRAFT. Threads that don't match any enabled channel are skipped.
  • `onGmailWebhook(request, _channelId?)` — single handler. Ignores `channelId` for new mailbox webhooks; the optional arg is preserved so already-deployed per-channel Pub/Sub callbacks dispatch cleanly.
  • `upgrade()` migrates existing instances: stops the old Gmail watch, probes known system labels for old per-channel state, populates `enabled_channels`, resumes in-flight backfills, deletes old per-channel topics, and sets up the new mailbox webhook.
  • Backwards-compatible shims preserve the original signatures of `syncBatch`, `setupChannelWebhook`, and `renewWatch` so any already-queued callbacks finish gracefully.

Migration caveat

Connectors can't enumerate stored keys (`Tool.list()` exists but is private to built-in tools, not exposed on `Connector`). The `upgrade()` migration handles system-label channels (STARRED, IMPORTANT, INBOX, SENT, DRAFT) cleanly. Users with a custom Gmail label enabled (e.g. `Label_14`) will need to disable+re-enable that channel after deploy.

Test plan

  • Deploy the connector and verify `upgrade()` runs against an existing twist instance with STARRED + IMPORTANT enabled
  • Confirm new mailbox-wide `mailbox_webhook` Pub/Sub topic is created and the old per-channel topics are cleaned up
  • Send a new email to a Gmail thread that already has a starred message; verify the new reply (which is itself NOT starred) appears in Plot within seconds
  • Star a not-previously-starred message and confirm the corresponding thread starts syncing
  • Disable the last enabled channel and confirm the mailbox watch is torn down (`mailbox_webhook` cleared, Pub/Sub topic deleted)
  • Re-enable a channel and confirm the watch is recreated, plus a fresh per-channel initial backfill runs
  • Existing in-flight per-channel pageToken backfill resumes under `initialSyncBatch` after upgrade

🤖 Generated with Claude Code

Gmail's per-label `users.watch` and `users.history.list` filters only
return events for messages currently carrying the given label. New
replies in an existing thread don't inherit the thread's labels, so a
reply that wasn't itself starred or auto-marked Important was missed
entirely — both at the watch (no Pub/Sub notification) and at history
(no entry returned). Threads stayed stuck on their last sync.
Switch to a single mailbox-wide Gmail watch per twist instance, and
route each changed thread to whichever enabled channel(s) its messages
actually belong to. New keys: mailbox_webhook, mailbox_renewal_task,
incremental_state, enabled_channels, initial_state_<channelId>.
`upgrade()` migrates existing instances by probing known system labels
(STARRED/IMPORTANT/INBOX/SENT/DRAFT) and resuming any in-flight
backfills under the new callback. Custom-label channels need a
re-toggle after deploy — the runtime doesn't expose key listing to
connectors, so we can't discover their IDs.
Backwards-compatible shims preserve the original signatures of
syncBatch, setupChannelWebhook, renewWatch, and onGmailWebhook so
already-deployed callbacks finish gracefully.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@KrisBraun
KrisBraun merged commit 87ac389 into mainApr 30, 2026
1 check passed
@KrisBraun
KrisBraun deleted the gmail/mailbox-wide-watch branch April 30, 2026 19:25
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@KrisBraun
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

connector-gmail: mailbox-wide watch + per-thread channel routing - #133

Merged
KrisBraun merged 1 commit into
mainfrom
gmail/mailbox-wide-watch
Apr 30, 2026
Merged

connector-gmail: mailbox-wide watch + per-thread channel routing#133
KrisBraun merged 1 commit into
mainfrom
gmail/mailbox-wide-watch

Conversation

@KrisBraun

Copy link
Copy Markdown
Contributor

Summary

  • Switch the Gmail connector to a single mailbox-wide `users.watch` per twist instance and route each changed thread to whichever enabled channel(s) it actually belongs to.
  • Fixes incoming replies on existing email threads being missed in Plot when the new reply isn't itself starred or auto-marked Important.
  • Adds an `upgrade()` migration that probes known system labels and resumes any in-flight backfills under the new callback path.

Why

Gmail's per-label `users.watch` and `users.history.list?labelId=` only fire/return for messages currently carrying that label. New replies in a thread don't inherit the thread's labels, so a reply that wasn't itself starred (or auto-marked Important by Gmail's heuristics) was missed entirely — no Pub/Sub notification, and no history record under the label-scoped filter. Threads silently stalled on their last sync.

The fix: register one mailbox-wide watch (no `labelIds`), call `history.list` without a `labelId` filter, and have the connector pick a channel per thread by inspecting the actual labels carried by its messages.

What changed

`connectors/gmail/src/gmail-api.ts`

  • `setupWatch(topicName, labelId?)` — `labelId` is now optional. When omitted, Gmail watches the whole mailbox.
  • New `syncGmailMailboxIncremental(api, historyId)` — calls `history.list` with no label, returns either `{ expired: true }` or every changed thread.

`connectors/gmail/src/gmail.ts`

  • One mailbox watch + Pub/Sub topic per twist instance. `ensureMailboxWebhook` (idempotent) runs whenever a channel is enabled; `teardownMailboxWebhook` runs when the last channel is disabled.
  • Two sync paths: `initialSyncBatch(channelId, batch)` for label-scoped first-time backfill, `incrementalSyncBatch()` for mailbox-wide changes.
  • New `pickChannelForThread` precedence: custom user labels (alphabetical) → STARRED → IMPORTANT → INBOX → SENT → DRAFT. Threads that don't match any enabled channel are skipped.
  • `onGmailWebhook(request, _channelId?)` — single handler. Ignores `channelId` for new mailbox webhooks; the optional arg is preserved so already-deployed per-channel Pub/Sub callbacks dispatch cleanly.
  • `upgrade()` migrates existing instances: stops the old Gmail watch, probes known system labels for old per-channel state, populates `enabled_channels`, resumes in-flight backfills, deletes old per-channel topics, and sets up the new mailbox webhook.
  • Backwards-compatible shims preserve the original signatures of `syncBatch`, `setupChannelWebhook`, and `renewWatch` so any already-queued callbacks finish gracefully.

Migration caveat

Connectors can't enumerate stored keys (`Tool.list()` exists but is private to built-in tools, not exposed on `Connector`). The `upgrade()` migration handles system-label channels (STARRED, IMPORTANT, INBOX, SENT, DRAFT) cleanly. Users with a custom Gmail label enabled (e.g. `Label_14`) will need to disable+re-enable that channel after deploy.

Test plan

  • Deploy the connector and verify `upgrade()` runs against an existing twist instance with STARRED + IMPORTANT enabled
  • Confirm new mailbox-wide `mailbox_webhook` Pub/Sub topic is created and the old per-channel topics are cleaned up
  • Send a new email to a Gmail thread that already has a starred message; verify the new reply (which is itself NOT starred) appears in Plot within seconds
  • Star a not-previously-starred message and confirm the corresponding thread starts syncing
  • Disable the last enabled channel and confirm the mailbox watch is torn down (`mailbox_webhook` cleared, Pub/Sub topic deleted)
  • Re-enable a channel and confirm the watch is recreated, plus a fresh per-channel initial backfill runs
  • Existing in-flight per-channel pageToken backfill resumes under `initialSyncBatch` after upgrade

🤖 Generated with Claude Code

Gmail's per-label `users.watch` and `users.history.list` filters only
return events for messages currently carrying the given label. New
replies in an existing thread don't inherit the thread's labels, so a
reply that wasn't itself starred or auto-marked Important was missed
entirely — both at the watch (no Pub/Sub notification) and at history
(no entry returned). Threads stayed stuck on their last sync.
Switch to a single mailbox-wide Gmail watch per twist instance, and
route each changed thread to whichever enabled channel(s) its messages
actually belong to. New keys: mailbox_webhook, mailbox_renewal_task,
incremental_state, enabled_channels, initial_state_<channelId>.
`upgrade()` migrates existing instances by probing known system labels
(STARRED/IMPORTANT/INBOX/SENT/DRAFT) and resuming any in-flight
backfills under the new callback. Custom-label channels need a
re-toggle after deploy — the runtime doesn't expose key listing to
connectors, so we can't discover their IDs.
Backwards-compatible shims preserve the original signatures of
syncBatch, setupChannelWebhook, renewWatch, and onGmailWebhook so
already-deployed callbacks finish gracefully.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@KrisBraun
KrisBraun merged commit 87ac389 into mainApr 30, 2026
1 check passed
@KrisBraun
KrisBraun deleted the gmail/mailbox-wide-watch branch April 30, 2026 19:25
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@KrisBraun
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

connector-gmail: mailbox-wide watch + per-thread channel routing - #133

Merged
KrisBraun merged 1 commit into
mainfrom
gmail/mailbox-wide-watch
Apr 30, 2026
Merged

connector-gmail: mailbox-wide watch + per-thread channel routing#133
KrisBraun merged 1 commit into
mainfrom
gmail/mailbox-wide-watch

Conversation

@KrisBraun

Copy link
Copy Markdown
Contributor

Summary

  • Switch the Gmail connector to a single mailbox-wide `users.watch` per twist instance and route each changed thread to whichever enabled channel(s) it actually belongs to.
  • Fixes incoming replies on existing email threads being missed in Plot when the new reply isn't itself starred or auto-marked Important.
  • Adds an `upgrade()` migration that probes known system labels and resumes any in-flight backfills under the new callback path.

Why

Gmail's per-label `users.watch` and `users.history.list?labelId=` only fire/return for messages currently carrying that label. New replies in a thread don't inherit the thread's labels, so a reply that wasn't itself starred (or auto-marked Important by Gmail's heuristics) was missed entirely — no Pub/Sub notification, and no history record under the label-scoped filter. Threads silently stalled on their last sync.

The fix: register one mailbox-wide watch (no `labelIds`), call `history.list` without a `labelId` filter, and have the connector pick a channel per thread by inspecting the actual labels carried by its messages.

What changed

`connectors/gmail/src/gmail-api.ts`

  • `setupWatch(topicName, labelId?)` — `labelId` is now optional. When omitted, Gmail watches the whole mailbox.
  • New `syncGmailMailboxIncremental(api, historyId)` — calls `history.list` with no label, returns either `{ expired: true }` or every changed thread.

`connectors/gmail/src/gmail.ts`

  • One mailbox watch + Pub/Sub topic per twist instance. `ensureMailboxWebhook` (idempotent) runs whenever a channel is enabled; `teardownMailboxWebhook` runs when the last channel is disabled.
  • Two sync paths: `initialSyncBatch(channelId, batch)` for label-scoped first-time backfill, `incrementalSyncBatch()` for mailbox-wide changes.
  • New `pickChannelForThread` precedence: custom user labels (alphabetical) → STARRED → IMPORTANT → INBOX → SENT → DRAFT. Threads that don't match any enabled channel are skipped.
  • `onGmailWebhook(request, _channelId?)` — single handler. Ignores `channelId` for new mailbox webhooks; the optional arg is preserved so already-deployed per-channel Pub/Sub callbacks dispatch cleanly.
  • `upgrade()` migrates existing instances: stops the old Gmail watch, probes known system labels for old per-channel state, populates `enabled_channels`, resumes in-flight backfills, deletes old per-channel topics, and sets up the new mailbox webhook.
  • Backwards-compatible shims preserve the original signatures of `syncBatch`, `setupChannelWebhook`, and `renewWatch` so any already-queued callbacks finish gracefully.

Migration caveat

Connectors can't enumerate stored keys (`Tool.list()` exists but is private to built-in tools, not exposed on `Connector`). The `upgrade()` migration handles system-label channels (STARRED, IMPORTANT, INBOX, SENT, DRAFT) cleanly. Users with a custom Gmail label enabled (e.g. `Label_14`) will need to disable+re-enable that channel after deploy.

Test plan

  • Deploy the connector and verify `upgrade()` runs against an existing twist instance with STARRED + IMPORTANT enabled
  • Confirm new mailbox-wide `mailbox_webhook` Pub/Sub topic is created and the old per-channel topics are cleaned up
  • Send a new email to a Gmail thread that already has a starred message; verify the new reply (which is itself NOT starred) appears in Plot within seconds
  • Star a not-previously-starred message and confirm the corresponding thread starts syncing
  • Disable the last enabled channel and confirm the mailbox watch is torn down (`mailbox_webhook` cleared, Pub/Sub topic deleted)
  • Re-enable a channel and confirm the watch is recreated, plus a fresh per-channel initial backfill runs
  • Existing in-flight per-channel pageToken backfill resumes under `initialSyncBatch` after upgrade

🤖 Generated with Claude Code

Gmail's per-label `users.watch` and `users.history.list` filters only
return events for messages currently carrying the given label. New
replies in an existing thread don't inherit the thread's labels, so a
reply that wasn't itself starred or auto-marked Important was missed
entirely — both at the watch (no Pub/Sub notification) and at history
(no entry returned). Threads stayed stuck on their last sync.
Switch to a single mailbox-wide Gmail watch per twist instance, and
route each changed thread to whichever enabled channel(s) its messages
actually belong to. New keys: mailbox_webhook, mailbox_renewal_task,
incremental_state, enabled_channels, initial_state_<channelId>.
`upgrade()` migrates existing instances by probing known system labels
(STARRED/IMPORTANT/INBOX/SENT/DRAFT) and resuming any in-flight
backfills under the new callback. Custom-label channels need a
re-toggle after deploy — the runtime doesn't expose key listing to
connectors, so we can't discover their IDs.
Backwards-compatible shims preserve the original signatures of
syncBatch, setupChannelWebhook, renewWatch, and onGmailWebhook so
already-deployed callbacks finish gracefully.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@KrisBraun
KrisBraun merged commit 87ac389 into mainApr 30, 2026
1 check passed
@KrisBraun
KrisBraun deleted the gmail/mailbox-wide-watch branch April 30, 2026 19:25
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@KrisBraun
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

connector-gmail: mailbox-wide watch + per-thread channel routing - #133

Merged
KrisBraun merged 1 commit into
mainfrom
gmail/mailbox-wide-watch
Apr 30, 2026
Merged

connector-gmail: mailbox-wide watch + per-thread channel routing#133
KrisBraun merged 1 commit into
mainfrom
gmail/mailbox-wide-watch

Conversation

@KrisBraun

Copy link
Copy Markdown
Contributor

Summary

  • Switch the Gmail connector to a single mailbox-wide `users.watch` per twist instance and route each changed thread to whichever enabled channel(s) it actually belongs to.
  • Fixes incoming replies on existing email threads being missed in Plot when the new reply isn't itself starred or auto-marked Important.
  • Adds an `upgrade()` migration that probes known system labels and resumes any in-flight backfills under the new callback path.

Why

Gmail's per-label `users.watch` and `users.history.list?labelId=` only fire/return for messages currently carrying that label. New replies in a thread don't inherit the thread's labels, so a reply that wasn't itself starred (or auto-marked Important by Gmail's heuristics) was missed entirely — no Pub/Sub notification, and no history record under the label-scoped filter. Threads silently stalled on their last sync.

The fix: register one mailbox-wide watch (no `labelIds`), call `history.list` without a `labelId` filter, and have the connector pick a channel per thread by inspecting the actual labels carried by its messages.

What changed

`connectors/gmail/src/gmail-api.ts`

  • `setupWatch(topicName, labelId?)` — `labelId` is now optional. When omitted, Gmail watches the whole mailbox.
  • New `syncGmailMailboxIncremental(api, historyId)` — calls `history.list` with no label, returns either `{ expired: true }` or every changed thread.

`connectors/gmail/src/gmail.ts`

  • One mailbox watch + Pub/Sub topic per twist instance. `ensureMailboxWebhook` (idempotent) runs whenever a channel is enabled; `teardownMailboxWebhook` runs when the last channel is disabled.
  • Two sync paths: `initialSyncBatch(channelId, batch)` for label-scoped first-time backfill, `incrementalSyncBatch()` for mailbox-wide changes.
  • New `pickChannelForThread` precedence: custom user labels (alphabetical) → STARRED → IMPORTANT → INBOX → SENT → DRAFT. Threads that don't match any enabled channel are skipped.
  • `onGmailWebhook(request, _channelId?)` — single handler. Ignores `channelId` for new mailbox webhooks; the optional arg is preserved so already-deployed per-channel Pub/Sub callbacks dispatch cleanly.
  • `upgrade()` migrates existing instances: stops the old Gmail watch, probes known system labels for old per-channel state, populates `enabled_channels`, resumes in-flight backfills, deletes old per-channel topics, and sets up the new mailbox webhook.
  • Backwards-compatible shims preserve the original signatures of `syncBatch`, `setupChannelWebhook`, and `renewWatch` so any already-queued callbacks finish gracefully.

Migration caveat

Connectors can't enumerate stored keys (`Tool.list()` exists but is private to built-in tools, not exposed on `Connector`). The `upgrade()` migration handles system-label channels (STARRED, IMPORTANT, INBOX, SENT, DRAFT) cleanly. Users with a custom Gmail label enabled (e.g. `Label_14`) will need to disable+re-enable that channel after deploy.

Test plan

  • Deploy the connector and verify `upgrade()` runs against an existing twist instance with STARRED + IMPORTANT enabled
  • Confirm new mailbox-wide `mailbox_webhook` Pub/Sub topic is created and the old per-channel topics are cleaned up
  • Send a new email to a Gmail thread that already has a starred message; verify the new reply (which is itself NOT starred) appears in Plot within seconds
  • Star a not-previously-starred message and confirm the corresponding thread starts syncing
  • Disable the last enabled channel and confirm the mailbox watch is torn down (`mailbox_webhook` cleared, Pub/Sub topic deleted)
  • Re-enable a channel and confirm the watch is recreated, plus a fresh per-channel initial backfill runs
  • Existing in-flight per-channel pageToken backfill resumes under `initialSyncBatch` after upgrade

🤖 Generated with Claude Code

Gmail's per-label `users.watch` and `users.history.list` filters only
return events for messages currently carrying the given label. New
replies in an existing thread don't inherit the thread's labels, so a
reply that wasn't itself starred or auto-marked Important was missed
entirely — both at the watch (no Pub/Sub notification) and at history
(no entry returned). Threads stayed stuck on their last sync.
Switch to a single mailbox-wide Gmail watch per twist instance, and
route each changed thread to whichever enabled channel(s) its messages
actually belong to. New keys: mailbox_webhook, mailbox_renewal_task,
incremental_state, enabled_channels, initial_state_<channelId>.
`upgrade()` migrates existing instances by probing known system labels
(STARRED/IMPORTANT/INBOX/SENT/DRAFT) and resuming any in-flight
backfills under the new callback. Custom-label channels need a
re-toggle after deploy — the runtime doesn't expose key listing to
connectors, so we can't discover their IDs.
Backwards-compatible shims preserve the original signatures of
syncBatch, setupChannelWebhook, renewWatch, and onGmailWebhook so
already-deployed callbacks finish gracefully.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@KrisBraun
KrisBraun merged commit 87ac389 into mainApr 30, 2026
1 check passed
@KrisBraun
KrisBraun deleted the gmail/mailbox-wide-watch branch April 30, 2026 19:25
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@KrisBraun
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

connector-gmail: mailbox-wide watch + per-thread channel routing - #133

Merged
KrisBraun merged 1 commit into
mainfrom
gmail/mailbox-wide-watch
Apr 30, 2026
Merged

connector-gmail: mailbox-wide watch + per-thread channel routing#133
KrisBraun merged 1 commit into
mainfrom
gmail/mailbox-wide-watch

Conversation

@KrisBraun

Copy link
Copy Markdown
Contributor

Summary

  • Switch the Gmail connector to a single mailbox-wide `users.watch` per twist instance and route each changed thread to whichever enabled channel(s) it actually belongs to.
  • Fixes incoming replies on existing email threads being missed in Plot when the new reply isn't itself starred or auto-marked Important.
  • Adds an `upgrade()` migration that probes known system labels and resumes any in-flight backfills under the new callback path.

Why

Gmail's per-label `users.watch` and `users.history.list?labelId=` only fire/return for messages currently carrying that label. New replies in a thread don't inherit the thread's labels, so a reply that wasn't itself starred (or auto-marked Important by Gmail's heuristics) was missed entirely — no Pub/Sub notification, and no history record under the label-scoped filter. Threads silently stalled on their last sync.

The fix: register one mailbox-wide watch (no `labelIds`), call `history.list` without a `labelId` filter, and have the connector pick a channel per thread by inspecting the actual labels carried by its messages.

What changed

`connectors/gmail/src/gmail-api.ts`

  • `setupWatch(topicName, labelId?)` — `labelId` is now optional. When omitted, Gmail watches the whole mailbox.
  • New `syncGmailMailboxIncremental(api, historyId)` — calls `history.list` with no label, returns either `{ expired: true }` or every changed thread.

`connectors/gmail/src/gmail.ts`

  • One mailbox watch + Pub/Sub topic per twist instance. `ensureMailboxWebhook` (idempotent) runs whenever a channel is enabled; `teardownMailboxWebhook` runs when the last channel is disabled.
  • Two sync paths: `initialSyncBatch(channelId, batch)` for label-scoped first-time backfill, `incrementalSyncBatch()` for mailbox-wide changes.
  • New `pickChannelForThread` precedence: custom user labels (alphabetical) → STARRED → IMPORTANT → INBOX → SENT → DRAFT. Threads that don't match any enabled channel are skipped.
  • `onGmailWebhook(request, _channelId?)` — single handler. Ignores `channelId` for new mailbox webhooks; the optional arg is preserved so already-deployed per-channel Pub/Sub callbacks dispatch cleanly.
  • `upgrade()` migrates existing instances: stops the old Gmail watch, probes known system labels for old per-channel state, populates `enabled_channels`, resumes in-flight backfills, deletes old per-channel topics, and sets up the new mailbox webhook.
  • Backwards-compatible shims preserve the original signatures of `syncBatch`, `setupChannelWebhook`, and `renewWatch` so any already-queued callbacks finish gracefully.

Migration caveat

Connectors can't enumerate stored keys (`Tool.list()` exists but is private to built-in tools, not exposed on `Connector`). The `upgrade()` migration handles system-label channels (STARRED, IMPORTANT, INBOX, SENT, DRAFT) cleanly. Users with a custom Gmail label enabled (e.g. `Label_14`) will need to disable+re-enable that channel after deploy.

Test plan

  • Deploy the connector and verify `upgrade()` runs against an existing twist instance with STARRED + IMPORTANT enabled
  • Confirm new mailbox-wide `mailbox_webhook` Pub/Sub topic is created and the old per-channel topics are cleaned up
  • Send a new email to a Gmail thread that already has a starred message; verify the new reply (which is itself NOT starred) appears in Plot within seconds
  • Star a not-previously-starred message and confirm the corresponding thread starts syncing
  • Disable the last enabled channel and confirm the mailbox watch is torn down (`mailbox_webhook` cleared, Pub/Sub topic deleted)
  • Re-enable a channel and confirm the watch is recreated, plus a fresh per-channel initial backfill runs
  • Existing in-flight per-channel pageToken backfill resumes under `initialSyncBatch` after upgrade

🤖 Generated with Claude Code

Gmail's per-label `users.watch` and `users.history.list` filters only
return events for messages currently carrying the given label. New
replies in an existing thread don't inherit the thread's labels, so a
reply that wasn't itself starred or auto-marked Important was missed
entirely — both at the watch (no Pub/Sub notification) and at history
(no entry returned). Threads stayed stuck on their last sync.
Switch to a single mailbox-wide Gmail watch per twist instance, and
route each changed thread to whichever enabled channel(s) its messages
actually belong to. New keys: mailbox_webhook, mailbox_renewal_task,
incremental_state, enabled_channels, initial_state_<channelId>.
`upgrade()` migrates existing instances by probing known system labels
(STARRED/IMPORTANT/INBOX/SENT/DRAFT) and resuming any in-flight
backfills under the new callback. Custom-label channels need a
re-toggle after deploy — the runtime doesn't expose key listing to
connectors, so we can't discover their IDs.
Backwards-compatible shims preserve the original signatures of
syncBatch, setupChannelWebhook, renewWatch, and onGmailWebhook so
already-deployed callbacks finish gracefully.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@KrisBraun
KrisBraun merged commit 87ac389 into mainApr 30, 2026
1 check passed
@KrisBraun
KrisBraun deleted the gmail/mailbox-wide-watch branch April 30, 2026 19:25
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@KrisBraun
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

connector-gmail: mailbox-wide watch + per-thread channel routing - #133

Merged
KrisBraun merged 1 commit into
mainfrom
gmail/mailbox-wide-watch
Apr 30, 2026
Merged

connector-gmail: mailbox-wide watch + per-thread channel routing#133
KrisBraun merged 1 commit into
mainfrom
gmail/mailbox-wide-watch

Conversation

@KrisBraun

Copy link
Copy Markdown
Contributor

Summary

  • Switch the Gmail connector to a single mailbox-wide `users.watch` per twist instance and route each changed thread to whichever enabled channel(s) it actually belongs to.
  • Fixes incoming replies on existing email threads being missed in Plot when the new reply isn't itself starred or auto-marked Important.
  • Adds an `upgrade()` migration that probes known system labels and resumes any in-flight backfills under the new callback path.

Why

Gmail's per-label `users.watch` and `users.history.list?labelId=` only fire/return for messages currently carrying that label. New replies in a thread don't inherit the thread's labels, so a reply that wasn't itself starred (or auto-marked Important by Gmail's heuristics) was missed entirely — no Pub/Sub notification, and no history record under the label-scoped filter. Threads silently stalled on their last sync.

The fix: register one mailbox-wide watch (no `labelIds`), call `history.list` without a `labelId` filter, and have the connector pick a channel per thread by inspecting the actual labels carried by its messages.

What changed

`connectors/gmail/src/gmail-api.ts`

  • `setupWatch(topicName, labelId?)` — `labelId` is now optional. When omitted, Gmail watches the whole mailbox.
  • New `syncGmailMailboxIncremental(api, historyId)` — calls `history.list` with no label, returns either `{ expired: true }` or every changed thread.

`connectors/gmail/src/gmail.ts`

  • One mailbox watch + Pub/Sub topic per twist instance. `ensureMailboxWebhook` (idempotent) runs whenever a channel is enabled; `teardownMailboxWebhook` runs when the last channel is disabled.
  • Two sync paths: `initialSyncBatch(channelId, batch)` for label-scoped first-time backfill, `incrementalSyncBatch()` for mailbox-wide changes.
  • New `pickChannelForThread` precedence: custom user labels (alphabetical) → STARRED → IMPORTANT → INBOX → SENT → DRAFT. Threads that don't match any enabled channel are skipped.
  • `onGmailWebhook(request, _channelId?)` — single handler. Ignores `channelId` for new mailbox webhooks; the optional arg is preserved so already-deployed per-channel Pub/Sub callbacks dispatch cleanly.
  • `upgrade()` migrates existing instances: stops the old Gmail watch, probes known system labels for old per-channel state, populates `enabled_channels`, resumes in-flight backfills, deletes old per-channel topics, and sets up the new mailbox webhook.
  • Backwards-compatible shims preserve the original signatures of `syncBatch`, `setupChannelWebhook`, and `renewWatch` so any already-queued callbacks finish gracefully.

Migration caveat

Connectors can't enumerate stored keys (`Tool.list()` exists but is private to built-in tools, not exposed on `Connector`). The `upgrade()` migration handles system-label channels (STARRED, IMPORTANT, INBOX, SENT, DRAFT) cleanly. Users with a custom Gmail label enabled (e.g. `Label_14`) will need to disable+re-enable that channel after deploy.

Test plan

  • Deploy the connector and verify `upgrade()` runs against an existing twist instance with STARRED + IMPORTANT enabled
  • Confirm new mailbox-wide `mailbox_webhook` Pub/Sub topic is created and the old per-channel topics are cleaned up
  • Send a new email to a Gmail thread that already has a starred message; verify the new reply (which is itself NOT starred) appears in Plot within seconds
  • Star a not-previously-starred message and confirm the corresponding thread starts syncing
  • Disable the last enabled channel and confirm the mailbox watch is torn down (`mailbox_webhook` cleared, Pub/Sub topic deleted)
  • Re-enable a channel and confirm the watch is recreated, plus a fresh per-channel initial backfill runs
  • Existing in-flight per-channel pageToken backfill resumes under `initialSyncBatch` after upgrade

🤖 Generated with Claude Code

Gmail's per-label `users.watch` and `users.history.list` filters only
return events for messages currently carrying the given label. New
replies in an existing thread don't inherit the thread's labels, so a
reply that wasn't itself starred or auto-marked Important was missed
entirely — both at the watch (no Pub/Sub notification) and at history
(no entry returned). Threads stayed stuck on their last sync.
Switch to a single mailbox-wide Gmail watch per twist instance, and
route each changed thread to whichever enabled channel(s) its messages
actually belong to. New keys: mailbox_webhook, mailbox_renewal_task,
incremental_state, enabled_channels, initial_state_<channelId>.
`upgrade()` migrates existing instances by probing known system labels
(STARRED/IMPORTANT/INBOX/SENT/DRAFT) and resuming any in-flight
backfills under the new callback. Custom-label channels need a
re-toggle after deploy — the runtime doesn't expose key listing to
connectors, so we can't discover their IDs.
Backwards-compatible shims preserve the original signatures of
syncBatch, setupChannelWebhook, renewWatch, and onGmailWebhook so
already-deployed callbacks finish gracefully.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@KrisBraun
KrisBraun merged commit 87ac389 into mainApr 30, 2026
1 check passed
@KrisBraun
KrisBraun deleted the gmail/mailbox-wide-watch branch April 30, 2026 19:25
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@KrisBraun
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

connector-gmail: mailbox-wide watch + per-thread channel routing - #133

Merged
KrisBraun merged 1 commit into
mainfrom
gmail/mailbox-wide-watch
Apr 30, 2026
Merged

connector-gmail: mailbox-wide watch + per-thread channel routing#133
KrisBraun merged 1 commit into
mainfrom
gmail/mailbox-wide-watch

Conversation

@KrisBraun

Copy link
Copy Markdown
Contributor

Summary

  • Switch the Gmail connector to a single mailbox-wide `users.watch` per twist instance and route each changed thread to whichever enabled channel(s) it actually belongs to.
  • Fixes incoming replies on existing email threads being missed in Plot when the new reply isn't itself starred or auto-marked Important.
  • Adds an `upgrade()` migration that probes known system labels and resumes any in-flight backfills under the new callback path.

Why

Gmail's per-label `users.watch` and `users.history.list?labelId=` only fire/return for messages currently carrying that label. New replies in a thread don't inherit the thread's labels, so a reply that wasn't itself starred (or auto-marked Important by Gmail's heuristics) was missed entirely — no Pub/Sub notification, and no history record under the label-scoped filter. Threads silently stalled on their last sync.

The fix: register one mailbox-wide watch (no `labelIds`), call `history.list` without a `labelId` filter, and have the connector pick a channel per thread by inspecting the actual labels carried by its messages.

What changed

`connectors/gmail/src/gmail-api.ts`

  • `setupWatch(topicName, labelId?)` — `labelId` is now optional. When omitted, Gmail watches the whole mailbox.
  • New `syncGmailMailboxIncremental(api, historyId)` — calls `history.list` with no label, returns either `{ expired: true }` or every changed thread.

`connectors/gmail/src/gmail.ts`

  • One mailbox watch + Pub/Sub topic per twist instance. `ensureMailboxWebhook` (idempotent) runs whenever a channel is enabled; `teardownMailboxWebhook` runs when the last channel is disabled.
  • Two sync paths: `initialSyncBatch(channelId, batch)` for label-scoped first-time backfill, `incrementalSyncBatch()` for mailbox-wide changes.
  • New `pickChannelForThread` precedence: custom user labels (alphabetical) → STARRED → IMPORTANT → INBOX → SENT → DRAFT. Threads that don't match any enabled channel are skipped.
  • `onGmailWebhook(request, _channelId?)` — single handler. Ignores `channelId` for new mailbox webhooks; the optional arg is preserved so already-deployed per-channel Pub/Sub callbacks dispatch cleanly.
  • `upgrade()` migrates existing instances: stops the old Gmail watch, probes known system labels for old per-channel state, populates `enabled_channels`, resumes in-flight backfills, deletes old per-channel topics, and sets up the new mailbox webhook.
  • Backwards-compatible shims preserve the original signatures of `syncBatch`, `setupChannelWebhook`, and `renewWatch` so any already-queued callbacks finish gracefully.

Migration caveat

Connectors can't enumerate stored keys (`Tool.list()` exists but is private to built-in tools, not exposed on `Connector`). The `upgrade()` migration handles system-label channels (STARRED, IMPORTANT, INBOX, SENT, DRAFT) cleanly. Users with a custom Gmail label enabled (e.g. `Label_14`) will need to disable+re-enable that channel after deploy.

Test plan

  • Deploy the connector and verify `upgrade()` runs against an existing twist instance with STARRED + IMPORTANT enabled
  • Confirm new mailbox-wide `mailbox_webhook` Pub/Sub topic is created and the old per-channel topics are cleaned up
  • Send a new email to a Gmail thread that already has a starred message; verify the new reply (which is itself NOT starred) appears in Plot within seconds
  • Star a not-previously-starred message and confirm the corresponding thread starts syncing
  • Disable the last enabled channel and confirm the mailbox watch is torn down (`mailbox_webhook` cleared, Pub/Sub topic deleted)
  • Re-enable a channel and confirm the watch is recreated, plus a fresh per-channel initial backfill runs
  • Existing in-flight per-channel pageToken backfill resumes under `initialSyncBatch` after upgrade

🤖 Generated with Claude Code

Gmail's per-label `users.watch` and `users.history.list` filters only
return events for messages currently carrying the given label. New
replies in an existing thread don't inherit the thread's labels, so a
reply that wasn't itself starred or auto-marked Important was missed
entirely — both at the watch (no Pub/Sub notification) and at history
(no entry returned). Threads stayed stuck on their last sync.
Switch to a single mailbox-wide Gmail watch per twist instance, and
route each changed thread to whichever enabled channel(s) its messages
actually belong to. New keys: mailbox_webhook, mailbox_renewal_task,
incremental_state, enabled_channels, initial_state_<channelId>.
`upgrade()` migrates existing instances by probing known system labels
(STARRED/IMPORTANT/INBOX/SENT/DRAFT) and resuming any in-flight
backfills under the new callback. Custom-label channels need a
re-toggle after deploy — the runtime doesn't expose key listing to
connectors, so we can't discover their IDs.
Backwards-compatible shims preserve the original signatures of
syncBatch, setupChannelWebhook, renewWatch, and onGmailWebhook so
already-deployed callbacks finish gracefully.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@KrisBraun
KrisBraun merged commit 87ac389 into mainApr 30, 2026
1 check passed
@KrisBraun
KrisBraun deleted the gmail/mailbox-wide-watch branch April 30, 2026 19:25
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@KrisBraun