test(rag): guard RC9 — lexical text path must not fabricate a cosine similarity - #245
Merged
Conversation
…similarity Verified RC9 is fully handled at the SQL layer: match_document_chunks_text sets similarity=0 (documented inline) with the lexical signal in a hybrid_score capped at 0.5 (below the 0.64 moderate gate); match_document_lookup_chunks_text and match_document_table_facts_text return only text_rank, with no similarity column to fabricate. Adds a source-text regression guard so the chunks-text similarity=0 + capped-hybrid_score invariant cannot silently drift back (the neutralized 20260702170000 migration + known hybrid-RPC drift make schema.sql the canonical definition this guards). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
RC9 verify (safe, offline, no live spend). RC9 was the concern that the text-only retrieval fallback fabricated a
similarity(0.56 + text_rank*0.39) that downstream gates read as a real cosine, letting a pure keyword hit clear the>=0.64"moderate/strong" evidence bar.Audit result — already comprehensively fixed in
supabase/schema.sql:match_document_chunks_textsetssimilarity = 0(with an inline comment naming the exact concern) and carries the lexical signal in ahybrid_scorecapped atleast(0.5, …)— strictly below the 0.64 moderate gate — plus a separatelexical_score.match_document_lookup_chunks_textandmatch_document_table_facts_textreturn onlytext_rank— nosimilarity/hybrid_scorecolumn to fabricate (thesimilarity()in table-facts is the pg_trgm function, not an output).hybrid_score ?? similarity ?? 0, and result mapping coalescessimilarityto a number, so a lexical-only row contributes at most the cappedhybrid_scoreand never a synthetic cosine.This PR adds a regression guard (source-text assertion in
tests/supabase-schema.test.ts) so thesimilarity = 0+ capped-hybrid_scoreinvariant can't silently drift back.schema.sqlis the canonical definition (the20260702170000migration is a documented no-op; live/schema reconciliation is a separate tracked item).Test-only change:
supabase-schema.test.ts31/31 pass, prettier + typecheck clean.🤖 Generated with Claude Code