Skip to content

Vector Search follow-up - #5367

Draft
janniklasrose wants to merge 10 commits into
mainfrom
janniklasrose/vector-search-followup
Draft

Vector Search follow-up#5367
janniklasrose wants to merge 10 commits into
mainfrom
janniklasrose/vector-search-followup

Conversation

@janniklasrose

@janniklasrosejanniklasrose commented May 28, 2026

Copy link
Copy Markdown
Contributor

Changes

  • Add an acceptance test covering vector search endpoint recreation on rename (recreate/name).
  • Have the testserver echo columns_to_sync/columns_to_index on VS index reads, and surface the index response's frozen endpoint_id so the fake mirrors the real API surface.

Note on endpoint UUID handling

An earlier commit switched the index resource to read endpoint_id off the VectorIndex response instead of looking the endpoint up by name. That was reverted: endpoint_id is frozen at index-creation time, so it can't detect a same-name endpoint being deleted and recreated out of band. The CLI keeps the live GetEndpointByEndpointName lookup feeding the state-tracked endpoint_uuid, which is what drives orphan/recreate drift detection (see #5123). endpoint_id remains a REMOTE-only informational field.

Drop the separate VectorSearchEndpoints.GetEndpointByEndpointName call
from DoRead/DoCreate and read endpoint_id directly off the VectorIndex
response. This removes the race noted in #5123 where the index's
endpoint and the lookup-by-name could disagree if the endpoint was
deleted/recreated between the two calls.
Won't compile until the SDK gains EndpointId on VectorIndex.
The SDK exposes a dedicated permission level type for vector search
endpoints; switch VectorSearchEndpoint.Permissions to use it via the
PermissionT[L] generic, matching the typing already in place for jobs,
apps, model serving, etc. Regenerated jsonschema.json and added the
annotations placeholder for the new VectorSearchEndpointPermission.
Production RemapState already round-trips both fields from the GET
response, but the testserver was dropping them. Bring the fake in line
with the real backend so acceptance tests see the same shape on read.
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented May 28, 2026

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 1f71abd

Run: 29915800629

Env🟨​KNOWN🔄​flaky💚​RECOVERED🙈​SKIP✅​pass🙈​skipTime
💚​aws linux44314105714:54
🔄​aws windows14431510559:49
💚​azure linux44314105612:31
💚​azure windows4431610548:54
🟨​gcp linux314313105816:38
🟨​gcp windows314315105611:26
9 interesting tests: 4 SKIP, 3 KNOWN, 1 RECOVERED, 1 flaky
Test Nameaws linuxaws windowsazure linuxazure windowsgcp linuxgcp windows
💚​TestAccept💚​R💚​R💚​R💚​R💚​R💚​R
🙈​TestAccept/bundle/invariant/no_drift🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/ssh/connection🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🔄​TestSyncIncrementalSyncFileToPythonNotebook✅​p🔄​f✅​p✅​p✅​p✅​p
🟨​TestFetchRepositoryInfoAPI_FromRepo💚​R💚​R💚​R💚​R🟨​K🟨​K
🟨​TestFetchRepositoryInfoAPI_FromRepo/root💚​R💚​R💚​R💚​R🟨​K🟨​K
🟨​TestFetchRepositoryInfoAPI_FromRepo/subdir💚​R💚​R💚​R💚​R🟨​K🟨​K
Top 25 slowest tests (at least 2 minutes):
durationenvtestname
5:21aws linuxTestImportDirDoesNotOverwrite
3:13aws windowsTestAccept
3:10aws windowsTestAccept/bundle/deploy/mlops-stacks/DATABRICKS_BUNDLE_ENGINE=terraform
3:08azure windowsTestAccept
3:07gcp windowsTestFilerWorkspaceFilesExtensionsReadDir
3:06gcp windowsTestAccept
3:06aws linuxTestAccept/bundle/deploy/mlops-stacks/DATABRICKS_BUNDLE_ENGINE=terraform
3:02azure linuxTestAccept/bundle/deploy/mlops-stacks/DATABRICKS_BUNDLE_ENGINE=terraform
2:55gcp windowsTestAccept/bundle/deploy/mlops-stacks/DATABRICKS_BUNDLE_ENGINE=terraform
2:47aws windowsTestImportDirWithOverwriteFlag
2:45aws windowsTestFilerRecursiveDelete/workspace_files_extensions
2:29aws windowsTestFilerRecursiveDelete/workspace_files
2:29aws windowsTestFilerWorkspaceFilesExtensionsRead
2:28aws windowsTestFilerWorkspaceFilesExtensionsReadDir
2:24azure windowsTestAccept/bundle/deploy/mlops-stacks/DATABRICKS_BUNDLE_ENGINE=direct
2:23gcp linuxTestFilerWorkspaceFilesExtensionsRead
2:22gcp linuxTestFilerWorkspaceFilesExtensionsReadDir
2:21azure windowsTestAccept/bundle/deploy/mlops-stacks/DATABRICKS_BUNDLE_ENGINE=terraform
2:13gcp linuxTestWorkspaceFilesExtensionsNotebooksAreNotReadAsFiles
2:12azure windowsTestFilerWorkspaceFilesExtensionsRead
2:12gcp windowsTestFilerReadWrite/workspace_files
2:09aws linuxTestFilerWorkspaceFilesExtensionsReadDir
2:05aws windowsTestAccept/bundle/deploy/mlops-stacks/DATABRICKS_BUNDLE_ENGINE=direct
2:01gcp linuxTestImportDirWithOverwriteFlag
2:00azure windowsTestExportDir

@janniklasrosejanniklasrose changed the title Vector Search follow-up[Blocked] Vector Search follow-upJun 4, 2026
@janniklasrosejanniklasrose changed the title [Blocked] Vector Search follow-upVector Search follow-upJun 18, 2026
…search-followup
# Conflicts:
#	bundle/internal/schema/annotations.yml
#	bundle/schema/jsonschema.json
#	bundle/schema/jsonschema_for_docs.json
@janniklasrose
janniklasrose marked this pull request as ready for review July 2, 2026 11:50
@github-actions

github-actionsBot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Approval status: pending

/acceptance/bundle/ - needs approval

8 files changed
Suggested: @pietern
Also eligible: @denik, @shreyas-goenka, @andrewnester, @anton-107, @lennartkats-db

General files (require maintainer)

Files: libs/testserver/fake_workspace.go, libs/testserver/vector_search_indexes.go
Based on git history:

  • @pietern -- recent work in libs/testserver/

Any maintainer (@andrewnester, @anton-107, @denik, @pietern, @shreyas-goenka, @simonfaltum, @renaudhartert-db) can approve all areas.
See OWNERS for ownership rules.

Removes the overlap now carried by vector-search-permission-type,
vector-search-doc-link, and vector-search-columns-testserver so this
branch keeps only the endpoint-UUID recreate work.
Co-authored-by: Isaac
@janniklasrose
janniklasrose marked this pull request as draft July 22, 2026 11:27
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@janniklasrose@eng-dev-ecosystem-bot