Skip to content

docs(product): define live pilot entry criteria - #3589

Merged
ootakazuhiko merged 2 commits into
mainfrom
feat/3586-live-pilot-criteria
Jul 1, 2026
Merged

ootakazuhiko merged 2 commits into
mainfrom
feat/3586-live-pilot-criteria

Conversation

@ootakazuhiko

Copy link
Copy Markdown
Collaborator

Summary

  • Add docs/product/LIVE-PILOT-ENTRY-CRITERIA.md as the SSOT for live/external pilot entry decisions before fixture-backed Evidence Sprint material can support live claims.
  • Define entry states, minimum consent/data-handling/retention criteria, required evidence artifacts, required measurement fields, and supported/unsupported claim boundaries.
  • Link the criteria from README/docs navigation, Evidence Sprint measurement and pilot reports, external pilot onboarding/runbook/report, release assets, and the launch kit.

Acceptance

  • Live-pilot entry criteria are documented with explicit supported/unsupported claim boundaries.
  • Required evidence artifacts and measurement fields are listed.
  • External repository, data-retention, and consent assumptions are explicit.
  • Fixture-backed Web API/event-driven pilots and the ACP-097 report remain marked report-only/dry-run until the criteria are satisfied.

Verification

  • pnpm -s run check:doc-consistency
  • pnpm -s run docs:lint (0 warnings after README wording adjustment)
  • pnpm install --frozen-lockfile (local worktree dependency install before schema/type verification)
  • node scripts/ci/validate-json.mjs
  • pnpm -s run check:schemas
  • pnpm -s run types:check
  • pnpm -s run verify:lite (passed; existing lint findings remain warn-only under baseline)
  • git diff --check && git diff --cached --check

Context Pack conflict

  • none. This is a product documentation and navigation update. spec/context-pack/boundary-map.json has no direct docs/product, docs/guides, README.md, or docs/README.md slice mapping affected by this change, and no Context Pack schema or boundary semantics are modified.

Rollback

  • Revert this PR to remove the live-pilot entry criteria document and restore the prior documentation links. No runtime, schema, CI policy, or enforcement behavior is changed.

Closes #3586

Copilot AI review requested due to automatic review settings July 1, 2026 21:23
@github-actions github-actions Bot added the risk:low Expected low implementation risk label Jul 1, 2026
@github-actions

github-actions Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Generate Artifacts Preview

Generated at: 2026-07-01T21:35:28.470Z

  • tests/api/generated: clean
  • artifacts/codex: clean
  • artifacts/spec: clean

@github-actions

github-actions Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

KvOnce Trace Validation

  • OTLP: ✅ Success (Issues: 0)
  • NDJSON: ✅ Success (Issues: 0)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f922718550

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread docs/guides/external-pilot-onboarding.md

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a new product SSOT document that defines live/external pilot entry criteria (consent, data handling/retention, required artifacts, required measurement fields, and supported/unsupported claim boundaries) and wires it into the existing Evidence Sprint and pilot documentation so fixture-backed/dry-run materials remain explicitly report-only until the criteria are met.

Changes:

  • Add docs/product/LIVE-PILOT-ENTRY-CRITERIA.md as the SSOT for promoting fixture-backed/dry-run pilot material to live external collection and publishable claims.
  • Update README/docs navigation and multiple Evidence Sprint / pilot docs to reference the entry criteria as a prerequisite for any live external claims.
  • Align launch kit and measurement protocol wording with the new gate (explicitly separating report-only readiness from claim-ready evidence).

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
README.md Adds the live-pilot entry criteria into the “first-time evaluation” evidence route and product evidence index.
docs/README.md Links the new entry criteria from the docs index and recommended product-evidence reading route (EN/JA).
docs/product/PILOT-RUNBOOK-2026Q3.md Updates runbook intro to require consulting the live-pilot entry criteria before collection.
docs/product/PILOT-REPORT-2026Q3-01.md Adds a prerequisite reference to the entry criteria before live collection/claims; tightens follow-up guidance.
docs/product/LIVE-PILOT-ENTRY-CRITERIA.md New SSOT defining entry states, minimum criteria, required artifacts, required measurement fields, and claim boundaries (EN/JA).
docs/product/LAUNCH-KIT.md Adds the criteria to the launch-kit flow and clarifies what it enables vs. what it does not.
docs/product/EVIDENCE-SPRINT-WEB-API-PILOT-2026-07-01.md Reinforces “report-only” scope and adds explicit gating to the new entry criteria for any future live claims.
docs/product/EVIDENCE-SPRINT-RELEASE-ASSETS-2026-07-01.md Adds the criteria as a referenced asset and updates follow-up wording to point to the SSOT.
docs/product/EVIDENCE-SPRINT-EVENT-DRIVEN-PILOT-2026-07-01.md Mirrors the Web API pilot update: explicit gate to entry criteria before any live/event external claims.
docs/product/DOGFOODING-PILOT-MEASUREMENT-PROTOCOL.md Adds explicit requirement to apply entry criteria before converting fixture/dry-run observations into live external claims (EN/JA).
docs/guides/external-pilot-onboarding.md Updates onboarding to consult the entry criteria before starting live collection.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/product/LIVE-PILOT-ENTRY-CRITERIA.md Outdated
Comment thread docs/product/LIVE-PILOT-ENTRY-CRITERIA.md Outdated
Comment thread docs/product/PILOT-REPORT-2026Q3-01.md Outdated
@ootakazuhiko
ootakazuhiko force-pushed the feat/3586-live-pilot-criteria branch from 6d56a4e to f20cfe4 Compare July 1, 2026 21:34
@ootakazuhiko
ootakazuhiko merged commit eeee14f into main Jul 1, 2026
29 checks passed
@ootakazuhiko
ootakazuhiko deleted the feat/3586-live-pilot-criteria branch July 1, 2026 21:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:low Expected low implementation risk

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Evidence Sprint follow-up] Define live-pilot entry criteria for external claims

2 participants