Skip to content

Fix atomic reindex repair metadata after PR 86#89

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
BigSimmo merged 1 commit into from
Jun 28, 2026
Merged

Fix atomic reindex repair metadata after PR 86 #89

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions tests/worker-visual-capture.test.ts
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -41,6 +41,7 @@ describe("worker visual capture hardening", () => {
expect(workerSource).toContain("await deleteStaleIndexGenerationRows(args.documentId, args.indexGenerationId)");
expect(workerSource).toContain("async function deleteStaleIndexGenerationRows");
expect(workerSource).toContain("`${imagePrefix}/${indexGenerationId}/image-${index + 1}${ext}`");
expect(workerSource).toContain('indexing_v3_agent_repair_reason: "core_index_committed"');
expect(workerSource).toContain("indexing_v3_agent_repair_reason: null");
});

Expand Down
4 changes: 4 additions & 0 deletions worker/main.ts
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -1542,6 +1542,10 @@ async function processJob(job: JobRow) {
index_quality_issues: initialQuality.issues,
index_quality_metrics: initialQuality.metrics,
optional_index_write_issues: optionalIndexWriteIssues,
indexing_v3_agent_status: "pending",
indexing_v3_agent_last_error: coreAgentMessage,
indexing_v3_agent_repair_reason: "core_index_committed",
indexing_v3_agent_updated_at: indexedAt,
Comment on lines +1545 to +1548

@chatgpt-codex-connectorchatgpt-codex-connectorBotJun 28, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reset stale v3 retry state for fresh generations

When atomically reindexing a document whose existing metadata has an exhausted indexing_v3_agent_attempt_count/indexing_v3_agent_max_attempts pair or a future indexing_v3_agent_next_run_at, this new pending marker still inherits those stale fields from job.documents.metadata; claim_indexing_v3_agent_jobs filters on attempt_count < max_attempts and next_run_at <= now, so if the worker stops after the core-generation commit, the fresh generation remains unclaimable despite being marked pending. Clear/reset the retry and schedule fields alongside the fresh pending status.

Useful? React with 👍 / 👎.

Comment on lines +1545 to +1548

@chatgpt-codex-connectorchatgpt-codex-connectorBotJun 28, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Delay v3 agent visibility until metadata rewrites finish

When the cron claims this document after commitDocumentIndexGeneration returns but before processJob reaches the later whole-metadata updateDocument (especially while WORKER_INLINE_ENRICHMENT=true is still enriching), claim_indexing_v3_agent_jobs can set the v3 agent lock/status to processing; the local worker then writes its precomputed finalMetadata and erases that lock/attempt state, allowing the same document to be claimed again while local and edge enrichment race over the same artifacts. Avoid making the row claimable until the local worker is done rewriting metadata, or preserve any intervening agent claim.

Useful? React with 👍 / 👎.

embedding_model: env.OPENAI_EMBEDDING_MODEL,
...metrics,
};
Expand Down