Skip to content

v0.6.99: tables filter operators, copilot chat persistence consolidation, together AI, Baseten, Ollama Cloud, Linq integrations, kb doc ownership bindings - #4836

Merged
waleedlatif1 merged 13 commits into
mainfrom
staging
Jun 2, 2026

Conversation

@waleedlatif1

@waleedlatif1waleedlatif1 commented Jun 1, 2026

Copy link
Copy Markdown
Collaborator

waleedlatif1and others added 5 commits May 31, 2026 11:22
…, remove JSONB dual-write (#4826)
Stop writing/reading the legacy copilot_chats.messages JSONB column now that
reads are cut over to copilot_messages. Make appendCopilotChatMessages the
primary write (throws on failure instead of swallowing), repoint peripheral
readers (workspace VFS, chat cleanup, data drains, fork, superuser import) to
copilot_messages, and persist the assistant turn inside finalizeAssistantTurn's
transaction so it commits atomically with the stream-marker clear. The column
itself is dropped in a follow-up migration after this bakes.
…, not-in, empty) (#4827)
Add does-not-contain ($ncontains), starts-with ($startsWith), ends-with
($endsWith), not-in-array ($nin, previously executed server-side but unexposed
in the UI), and is-empty/is-not-empty ($empty) filter operators end-to-end —
SQL builder, condition types, query-builder converters/constants, the filter
UI, the Table tools/block descriptions, and docs.
Also fix correctness bugs in the filter builder surfaced by the wider operator
set:
- Same-column AND rules (e.g. age > 18 AND age < 65, or name startsWith 'A'
AND name endsWith 'Z') silently overwrote each other because the AND group
was keyed by column name. They now merge into one operator object, which
also makes Filter -> rules -> Filter round-trip losslessly for multi-operator
columns.
- $nin values were not split into an array like $in, and textual-match values
like "123" were numeric-coerced (breaking the ILIKE path).
- A non-boolean $empty operand from the raw API silently inverted the check; it
now coerces 'true'/'false' strings and otherwise returns a 400.
…nscripts (#4829)
Opening a Mothership task could take many seconds because a single persisted
assistant message in copilot_messages.content can reach hundreds of MB, almost
entirely inside contentBlocks[].toolCall.result.output (e.g. a get_workflow_logs
or run_workflow result). The DB query is ~2ms; the cost is detoasting that
payload, shipping it to the browser, and parsing it.
These outputs are dead weight on the Sim side: they are never rendered (the
thread shows only tool name/title/status) and never replayed to the model (the
upstream copilot service owns conversation memory). So drop result.output before
it is persisted, keeping result.success/error plus the tool metadata.
- add stripToolResultOutput() in persisted-message.ts
- apply it in messages-store toRow (covers every write path) and in
loadCopilotChatMessages (existing rows render fast on read)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…viders (#4830)
* feat(providers): add Together AI, Baseten, and Ollama Cloud model providers
* fix(providers): guard Ollama streaming fast-path with hasActiveTools
Match Together/Baseten/Fireworks: when tools are supplied but all are
filtered out (usageControl 'none'), take the single streaming call instead
of an extra non-streaming round-trip.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(providers): filter non-chat model types from Together model list
* refactor(providers): dedupe Ollama Cloud upstream schema
ollamaCloudUpstreamResponseSchema was byte-for-byte identical to
ollamaUpstreamResponseSchema (both /api/tags endpoints return the same
{ models: [{ name }] } shape). Drop the duplicate and reuse the shared schema.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
@cursor

cursorBot commented Jun 1, 2026

Copy link
Copy Markdown

PR Summary

High Risk
Knowledge-base authorization and upload binding changes are security-critical; incorrect rollout could block legitimate files or leave gaps until backfill completes.

Overview
This release tightens knowledge-base file security by authorizing reads and deletes from trusted workspace_files ownership bindings plus an active document on the exact storageKey, instead of inferring access from document.fileUrl. KB uploads now require workspaceId, write/admin permission, and record ownership on presigned, batch, multipart, and direct paths; failed binding after multipart complete rolls back the object.

Copilot chat updates, stop, fork, import, and mothership flows route transcript persistence through messages-store (replacing JSONB dual-write) with transactional updates where needed.

Tables gain richer string filters ($ncontains, $startsWith, $endsWith, $empty) in docs and UI; CSV append dispatches workflow triggers after commit; run UI tracks active dispatches.

Integrations: new Linq tool (34 ops, docs, icons, upload route); Together, Baseten, and Ollama Cloud model listing APIs and BYOK/landing wiring.

Permissions align deletes for workflows, folders, MCP servers, and schedules with write (not admin-only where changed). Vitest bumps to ^4.1.0 in realtime.

Reviewed by Cursor Bugbot for commit 3ccb3a3. Bugbot is set up for automated code reviews on this repo. Configure here.

@vercel

vercelBot commented Jun 1, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
ProjectDeploymentActionsUpdated (UTC)
docsSkippedSkippedJun 2, 2026 1:12am

Request Review

…fter tx commit (#4503)
Two bugs in the CSV-import path:
- addTableColumnsWithTx rebuilt the schema with only `columns`, dropping
`workflowGroups` (and any other top-level schema fields). Importing CSV
into a table that has workflow groups erased the group config. Spread
`table.schema` first so siblings survive.
- batchInsertRowsWithTx fired fireTableTrigger and scheduleRunsForRows from
inside the caller's transaction. Both read through the global db
connection, so they could run before the inserts committed and see no
rows. Extracted the dispatch into dispatchAfterBatchInsert; non-tx
wrapper fires it after `db.transaction(...)` resolves, and the CSV
import route does the same after its tx.
* improvement(kbs): ownership association
* address comments
* more comments
@icecrasher321icecrasher321 changed the title v0.6.99: tables filter operators, copilot chat persistence consolidation, together AI, Baseten, Ollama Cloud, Linq integrations v0.6.99: tables filter operators, copilot chat persistence consolidation, together AI, Baseten, Ollama Cloud, Linq integrations, kb doc ownership bindingsJun 1, 2026
…ory (GHSA-5xrq-8626-4rwp) (#4837)
* fix(deps): upgrade vitest to ^4.1.0 to patch critical Vitest UI advisory (GHSA-5xrq-8626-4rwp)
- Bump vitest and @vitest/coverage-v8 to ^4.1.0 across all workspaces (only patched release for the critical 'Vitest UI server arbitrary file read/execute' advisory; no 3.x backport exists)
- Widen @sim/testing peer range to ^3.0.0 || ^4.0.0
- Migrate constructor mocks to class expressions: vitest 4 uses Reflect.construct for mocks invoked with new, and arrow/function implementations are not constructable (function expressions also get reverted to arrows by biome's useArrowFunction)
- Remove deprecated test.poolOptions from apps/sim/vitest.config.ts (options are now top-level in vitest 4)
* fix(deps): exclude vulnerable vitest 4.0.x from @sim/testing peer range
Tighten the v4 arm of the peer range to >=4.1.0 <5.0.0 so the peer
requirement cannot be satisfied by the unpatched 4.0.x builds that
GHSA-5xrq-8626-4rwp affects.
* fix(testing): make vitest 4 constructor mocks type-check cleanly
- logging-session & mcp-oauth mocks: a class passed to mockImplementation has
a construct signature that isn't assignable to its (...args) => any parameter,
failing tsc. Use named function declarations instead (constructable via
Reflect.construct, assignable to mockImplementation, and not rewritten to
arrows by biome's useArrowFunction).
- database.mock.ts: vitest 4's generic vi.fn typings no longer break the
self-referential cycle on the transaction callback's tx param; loosen tx and
annotate the callback's return type to resolve the implicit-any errors.
* test(isolated-vm): de-flake queue-capacity scheduler tests
The 'queue is full' and 'per-owner queued limit' tests relied on
'await sleep(1)' to assume the first request had reached the queue before
submitting the overflow request. The first request only enqueues after an
async spawn-failure chain (acquireWorker -> spawn exit -> resolve null ->
enqueue), which isn't guaranteed within 1ms under CI load — the overflow
request then found an empty queue and hit the 200ms queue-wait timeout
instead of the capacity rejection.
Replace the wall-clock barrier with a deterministic, event-driven one: hold
the single global concurrency slot (IVM_MAX_CONCURRENT=1) with an active
worker and await an explicit 'dispatched' signal (fired when the worker
receives its execute message, after the scheduler counts it active). The
follow-up requests then deterministically hit the synchronous enqueue path.
Also drops the queue-wait timeout from 200ms to 50ms, so the tests run faster.
…#4839)
* feat(tables): add PostHog events for table-workflow run/stop gestures
* fix(tables): pass run source from action-bar play/refresh dispatchers
* fix(tables): enforce plan limits in mothership user_table tool
* improvement(tables): truncate over-limit CSV imports to the plan cap instead of rejecting
* fix(tables): log rollback failures and surface a clear reason on failed CSV import
Comment threadapps/sim/app/api/files/presigned/route.ts
…s parity for delete (#4840)
* fix(tool-input): keep block-tool params selected across store replace
* fix tests and extract kb ownership helper
…gating for cell runs (#4838)
* fix(tables): reliable stop-all, accurate "X running", and rate/usage gating for cell runs
Stop-all:
- Make the cancellation guard status-based (not executionId-scoped) so a
`cancelled` tombstone stamped while a cell is still a dispatcher pre-stamp
(null executionId) keeps the cell dead — fixes function-execute cells that
resurrected after Stop all. Consolidated into shared isExecCancelled /
isExecCancelledAfter predicates in deps.ts, reused by the in-memory guard,
the SQL guard, the dispatcher tombstone filter, the worker, and resume.
- Add an explicit pre-execution cancellation read so a cell that dequeues
after Stop all (e.g. from the trigger.dev queue) never runs.
- Resume worker aborts a cancelled paused/awaiting cell before resuming;
cancelWorkflowGroupRuns marks paused executions cancelling.
"X running":
- Emit a dispatch SSE at dispatch start so auto-fired/capped runs surface
immediately; show the control whenever a dispatch is active.
Checkbox dependency:
- Treat boolean `false` as an unmet dependency so unchecking never reruns
dependents — only checking does. deriveExecClearsForDataPatch no longer
re-arms a downstream group whose deps are unmet after the patch.
Rate / usage gating:
- Route table cell execution through preprocessExecution (billing actor =
workspace billed account, usage limit, per-plan timeout), keeping draft.
- Rate limit: pace & retry per cell (async counter) so rows aren't skipped.
- Usage limit: halt the dispatch without marking cells and emit a
usageLimitReached event; the client shows an Upgrade prompt that routes to
subscription settings.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(tables): dedupe usage-limit event + release rate-limited cells on cancel
Addresses PR review:
- Usage limit: only the cell that transitions the dispatch active→complete
(via completeDispatchIfActive) emits usageLimitReached, so concurrent cells
don't fire up to 20 identical "upgrade" toasts.
- Rate-limit retry: re-check the cancelled tombstone after each sleep so a
Stop All mid-wait releases the concurrency slot promptly (signal never fires
on the trigger.dev backend).
* fix(tables): jitter rate-limit retry backoff to avoid thundering herd
Passing the bucket's shared resetAt as retryAfterMs made backoffWithJitter
return a fixed clamped value (no jitter, attempt ignored), so all concurrent
cells retried in lockstep. Pass null to get jittered exponential backoff.
* fix(tables): unstick cells + resync counter on usage-limit halt
Addresses PR review:
- Clear each blocked cell's pre-stamp on a 402 so it reverts to un-run instead
of being stuck "Queued" (no error/cancelled badge); covers auto-fire cells
with no owning dispatch.
- Client re-syncs run-state counts and refetches rows on usageLimitReached so
the stale "X running" / Stop-all control clears and queued cells drop.
- Make usageLimitReached.dispatchId optional; client only touches the dispatch
overlay when present.
* fix(tables): don't emit stale dispatching event after a mid-window halt
If a cell halts the dispatch mid-window (usage limit), re-read the dispatch
status after the batch and bail instead of emitting a per-window 'dispatching'
event that would arrive after the client dropped the dispatch and re-add it
(flickering 'X running' back).
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@cursorcursorBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 3ccb3a3. Configure here.

Comment threadapps/sim/app/api/files/upload/route.ts
Comment threadapps/sim/app/api/knowledge/utils.ts Outdated
@waleedlatif1
waleedlatif1 merged commit 12ada0c into mainJun 2, 2026
30 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@waleedlatif1@icecrasher321@TheodoreSpeaks