Skip to content

Repository files navigation

Canary402

Canary402 is a service-contract inspector and mystery shopper for x402-paid services:

Before an agent integrates with or pays a service, Canary402 checks the contract and verifies the delivery.

It inspects public service specifications, generates value-free repair templates, validates the live x402 challenge, optionally makes one strictly budgeted downstream payment through the Obol remote signer, evaluates the delivered result with OpenRouter, and retains an evidence-backed report.

The original concept is preserved in idea.md.

Live Agent

The public product is the Canary402 Agent at:

POST https://andrei-obol-agent.dvlabs.dev/v1/chat/completions

The previous shared-origin path, POST /services/canary402-agent/v1/chat/completions, remains a working compatibility alias.

The Agent is a Hermes/OpenRouter service sold for 0.001 USDC per request on Base mainnet. Its deterministic Go audit API remains deployed inside the Obol cluster and is not sold, advertised, or routed publicly. Calling the Agent therefore incurs one Canary402 charge, not a second internal API charge.

Canary402 is registered as ERC-8004 Agent ID 59094 on Base mainnet, matching its x402 payment network. The same signer also retains the legacy Agent ID 8104 on Base Sepolia. Both registration entries point to the same public Agent and are owned by the Canary402 wallet.

Public discovery documents are available from the dedicated Agent origin:

  • GET https://andrei-obol-agent.dvlabs.dev/openapi.json
  • GET https://andrei-obol-agent.dvlabs.dev/.well-known/x402
  • GET https://andrei-obol-agent.dvlabs.dev/.well-known/agent-registration.json

x402scan crawled the migrated dedicated origin and registered its single current paid resource (1/1), deprecating the earlier shared-origin resource. The listing command warned only that the generated OpenAPI lacks info.contact.email; that field is optional for indexing but enables ownership/contact customization.

The public registration document is active and advertises only the Base Agent identity and endpoint. The legacy Sepolia registration remains owned by the same wallet and retained in Obol's durable AgentIdentity state for historical continuity, but is not advertised by this Base-paid offer.

The old /services/canary402 HTTP offer and its public health/report routes have intentionally been removed. Reports are still retained internally, and the Agent returns the report ID and an evidence-based interpretation in its response.

Buy an Agent request

The endpoint accepts OpenAI-style chat-completions JSON behind an x402 v2 gate. The buyer wallet needs canonical USDC on Base mainnet; the inbound payment itself does not require the buyer to fund a separate Canary402 wallet.

Keep all private keys outside source control:

npm install @x402/fetch @x402/evm viem
import{x402Client,wrapFetchWithPayment}from"@x402/fetch";import{registerExactEvmScheme}from"@x402/evm/exact/client";import{privateKeyToAccount}from"viem/accounts";constsigner=privateKeyToAccount(process.env.EVM_PRIVATE_KEY);constclient=newx402Client();registerExactEvmScheme(client,{ signer });constfetchWithPayment=wrapFetchWithPayment(fetch,client);constresponse=awaitfetchWithPayment("https://andrei-obol-agent.dvlabs.dev/v1/chat/completions",{method: "POST",headers: {"Content-Type": "application/json"},body: JSON.stringify({model: "openrouter/auto",stream: false,messages: [{role: "user",content: "Review the specification and probe https://seller.example.com/weather. Generate repair templates, but do not make a downstream payment.",}],}),},);console.log(response.status,awaitresponse.json());

An unpaid request should return 402 Payment Required. Its challenge should advertise:

  • network eip155:8453 (Base mainnet)
  • canonical Base USDC
  • amount 1000 atomic units (0.001 USDC)
  • the exact HTTPS Agent URL as its resource

The Agent defaults to no downstream payment. Paying to talk to Canary402 does not authorize Canary402 to pay the inspected service.

Ask Canary402 for an audit

A normal request can combine these duties:

  • Specification review inspects bounded copies of /openapi.json, /.well-known/agent-registration.json, and /skill.md, then compares them with the requested operation and live 402 metadata.
  • Repair generation proposes value-free OpenAPI and Bazaar fragments. These are starting points for seller review, not inferred business truth.
  • Probe-only verification validates reachability and the x402 challenge without signing or spending anything downstream.
  • Paid verification makes one budget-capped downstream payment, retries the target, and evaluates the delivered response.

Example prompts:

Review the specification and probe https://seller.example.com/services/weather.
Generate repair templates, but do not make a downstream payment.
Make a real paid audit of https://seller.example.com/service.
Use Base mainnet and do not spend more than 20000 atomic USDC units.

For a real downstream payment, the user must explicitly supply both:

  • payment_network: base or base-sepolia
  • max_payment_atomic: the maximum atomic token amount Canary402 may authorize

For six-decimal USDC, 1000 atomic units is 0.001 USDC. Canary402 also enforces its deployment-wide cap, currently 20000 atomic units (0.02 USDC).

Report meaning

The Agent summarizes the API's report ID, verdict, score, coverage, specification status, findings, repair artifacts, individual checks, and limitations. Reports may contain these checks:

  • service_discovery
  • request_contract
  • reachability
  • x402_challenge
  • payment_budget
  • paid_delivery
  • task_outcome
VerdictMeaning
PROBE_PASSThe payment challenge is valid, but no downstream payment was made
PROBE_PASS_WITH_WARNINGSThe probe worked, but service-contract metadata needs repair
PASSChallenge, paid delivery, and requested outcome passed
PASS_WITH_WARNINGSPaid delivery worked, but some evaluation was unavailable or uncertain
FAILAt least one required check failed

score measures checks that ran. coverage_percent shows how much of the full paid flow was exercised, so a probe-only result must be read with its lower coverage.

Stored reports do not contain purchased response bodies or fetched discovery-document bodies. They retain status, latency, content type, captured byte count, SHA-256 digests, payment terms, settlement evidence when present, and a short evaluator reason. Caller-supplied JSON property names can appear in generated repair fragments, so do not use repair generation for confidential request shapes.

Proven settlement history

Before the current Base-mainnet-only Agent offer was selected, a paid Agent call settled 0.001 Base Sepolia test USDC in transaction 0x8dd8…f91b.

Canary402 has also made a downstream payment of 0.02 Base mainnet USDC while auditing another hackathon service; transaction 0x9eb2…431c confirms the transfer.

These are point-in-time tests, not guarantees about either service.

Run locally

Prerequisites are an Obol Stack v0.14.0-rc0 cluster, Docker, Go 1.25+, and a working LiteLLM/OpenRouter configuration. Install or upgrade the Stack with:

OBOL_RELEASE=v0.14.0-rc0 bash <(curl -s https://stack.obol.org)# For this existing k3d stack, free its current host ports before forced init.
k3d cluster stop obol-stack-peaceful-starling
obol stack init --force --backend k3d --yes
obol stack up

For a new stack with no existing configuration, use obol stack init && obol stack up instead.

make test
make deploy
make smoke

make deploy builds and imports the local image, applies deploy/k8s.yaml, and keeps reports on the canary402-reports PVC. make smoke port-forwards the internal API and verifies its health without publishing it.

Publish or reconcile only the Agent offer:

make sell-agent

This command:

  1. creates or updates the Hermes Agent from agent/objective.md;
  2. applies the Agent's narrow cross-namespace egress policy;
  3. reconciles the x402 price to 0.001 USDC on Base mainnet;
  4. activates the offer against the existing Base Agent ID without minting another identity;
  5. binds the permanent hostname to v0.14's native dedicated-offer route and persists the resume ledger; and
  6. verifies the local offer.

Forks using another hostname should set CANARY_AGENT_HOSTNAME before running make sell-agent.

Direct development requests

The Go API is accessible only inside the cluster or through an operator-created port-forward:

obol kubectl -n llm port-forward service/canary402 18080:8080

Then run a no-spend audit directly:

curl --noproxy '*' -sS http://127.0.0.1:18080/audit \
-H 'Content-Type: application/json' \
--data '{ "url": "https://seller.example.com/services/weather/current", "method": "POST", "body": {"city": "Istanbul"}, "expectation": "The response contains a numeric temperature for Istanbul.", "spec_review": true, "generate_repairs": true, "pay": false }'

A complete request is also available in examples/audit-spec-review.json.

Safety model

Canary402 fails closed:

  • Only public HTTPS targets on port 443 are accepted.
  • Loopback, private, link-local, metadata, benchmark, documentation, carrier-grade NAT, IPv4-mapped IPv6, and standard NAT64 translation ranges are blocked.
  • DNS is checked again at connection time to reduce rebinding risk.
  • Redirects are never followed, especially after payment authorization is attached.
  • Request bodies are JSON and capped at 64 KiB; target responses are capped at 1 MiB.
  • Response bodies and fetched specification bodies are not persisted.
  • Discovery fetches only three fixed same-origin paths, does not follow redirects, and caps each document at 256 KiB.
  • Query strings and credentials are removed from stored report URLs.
  • Client authentication, cookies, arbitrary headers, and payment headers are not forwarded.
  • Downstream payment is opt-in and bounded by both the request cap and CANARY_MAX_PAYMENT_ATOMIC.
  • Target content is treated as untrusted evidence, including in the semantic-evaluation prompt.
  • Objective protocol checks remain separate from the subjective model verdict.

The downstream client supports x402 v2 exact/EIP-3009 with canonical USDC on Base (eip155:8453) and Base Sepolia (eip155:84532). Permit2, arbitrary ERC-20 assets, redirects, asynchronous jobs, custom caller headers, and SSE targets are deliberately unsupported.

Verification

make test
make vet
make status
make logs

The test suite covers specification discovery, path-template matching, deterministic repair generation, challenge comparison, caps, remote signing, paid retry, settlement evidence, semantic evaluation, persistent reports, IPv4/mapped-IPv6 equivalence, NAT64 blocks, and the invariant that inferred schemas contain no example values.

Configuration

Internal API runtime:

VariableDefaultMeaning
PORT8080HTTP listen port
CANARY_REPORT_DIR/data/reportsPersistent report directory
CANARY_MAX_PAYMENT_ATOMIC20000Absolute downstream per-audit cap
CANARY_ALLOWED_NETWORKSbase-sepolia,baseAllowed downstream networks
CANARY_MAX_CONCURRENT4Concurrent audit limit
CANARY_TARGET_TIMEOUT_SECONDS20Per-target timeout
CANARY_ALLOW_HTTPfalseDevelopment-only HTTP target support
CANARY_ALLOW_PRIVATE_TARGETSfalseTest-only private target support; never enable publicly
LITELLM_BASE_URLin-cluster LiteLLMOpenAI-compatible evaluator URL
LITELLM_MASTER_KEYemptyLiteLLM bearer token; missing means semantic evaluation is disabled
CANARY_MODELopenrouter/autoSemantic evaluator model
REMOTE_SIGNER_URLin-cluster signerObol typed-data signer URL
REMOTE_SIGNER_TOKENemptyOptional signer bearer token

Agent publication:

VariableDefaultMeaning
CANARY_AGENT_MODELopenrouter/autoHermes Agent model
CANARY_AGENT_NETWORKbaseInbound x402 payment network
CANARY_AGENT_IDENTITY_NETWORKbaseERC-8004 identity lookup chain; normally match the payment network
CANARY_AGENT_PRICE0.001Per-request USDC price
CANARY_AGENT_HOSTNAMEandrei-obol-agent.dvlabs.devDedicated public Agent origin

LiteLLM authentication comes from the existing llm/litellm-secrets Kubernetes secret. Payment signing uses the existing remote signer; its private key is never mounted into Canary402. The signer address is intentionally cached for the process lifetime, so a coordinated signer-key rotation requires restarting Canary402.

Secrets and source control

Local Obol/OpenRouter and tunnel credentials belong only in .env, which Git and Docker ignore. Never commit wallet private keys, remote-signer credentials, LLM_API_KEY, or the secret-bearing tunnel command.

git status --short --ignored
git check-ignore -v .env

Obol and registration notes

Obol v0.14.0-rc0 natively publishes the dedicated hostname, preserves the external HTTPS scheme in x402 challenges, exposes /.well-known/x402, and creates namespace-qualified cross-namespace grants. Canary402 therefore no longer carries the v0.13 tunnel-route or ReferenceGrant workarounds.

The primary identity is Base Agent ID 59094. Its mint transaction and agentURI correction succeeded. The corrected on-chain URI is https://andrei-obol-agent.dvlabs.dev/.well-known/agent-registration.json.

Legacy Base Sepolia Agent ID 8104 remains owned by the same wallet; its mint transaction also succeeded. Registration is idempotent while x402/AgentIdentity contains the correct per-chain IDs. Do not delete or overwrite that durable record, and pass the origin—not a service path—to obol sell register --origin.

ERC-8004 identity, an active x402 ServiceOffer, and x402scan discovery are separate states. After the v0.14 dedicated-origin migration, obol sell register x402scan idempotently registered 1/1 current resources and deprecated the old shared-origin resource.

About

Evidence-backed reliability audits for x402-paid agent services

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
GitHub - pinebit/canary402: Evidence-backed reliability audits for x402-paid agent services · GitHub
Skip to content

Repository files navigation

Canary402

Canary402 is a service-contract inspector and mystery shopper for x402-paid services:

Before an agent integrates with or pays a service, Canary402 checks the contract and verifies the delivery.

It inspects public service specifications, generates value-free repair templates, validates the live x402 challenge, optionally makes one strictly budgeted downstream payment through the Obol remote signer, evaluates the delivered result with OpenRouter, and retains an evidence-backed report.

The original concept is preserved in idea.md.

Live Agent

The public product is the Canary402 Agent at:

POST https://andrei-obol-agent.dvlabs.dev/v1/chat/completions

The previous shared-origin path, POST /services/canary402-agent/v1/chat/completions, remains a working compatibility alias.

The Agent is a Hermes/OpenRouter service sold for 0.001 USDC per request on Base mainnet. Its deterministic Go audit API remains deployed inside the Obol cluster and is not sold, advertised, or routed publicly. Calling the Agent therefore incurs one Canary402 charge, not a second internal API charge.

Canary402 is registered as ERC-8004 Agent ID 59094 on Base mainnet, matching its x402 payment network. The same signer also retains the legacy Agent ID 8104 on Base Sepolia. Both registration entries point to the same public Agent and are owned by the Canary402 wallet.

Public discovery documents are available from the dedicated Agent origin:

  • GET https://andrei-obol-agent.dvlabs.dev/openapi.json
  • GET https://andrei-obol-agent.dvlabs.dev/.well-known/x402
  • GET https://andrei-obol-agent.dvlabs.dev/.well-known/agent-registration.json

x402scan crawled the migrated dedicated origin and registered its single current paid resource (1/1), deprecating the earlier shared-origin resource. The listing command warned only that the generated OpenAPI lacks info.contact.email; that field is optional for indexing but enables ownership/contact customization.

The public registration document is active and advertises only the Base Agent identity and endpoint. The legacy Sepolia registration remains owned by the same wallet and retained in Obol's durable AgentIdentity state for historical continuity, but is not advertised by this Base-paid offer.

The old /services/canary402 HTTP offer and its public health/report routes have intentionally been removed. Reports are still retained internally, and the Agent returns the report ID and an evidence-based interpretation in its response.

Buy an Agent request

The endpoint accepts OpenAI-style chat-completions JSON behind an x402 v2 gate. The buyer wallet needs canonical USDC on Base mainnet; the inbound payment itself does not require the buyer to fund a separate Canary402 wallet.

Keep all private keys outside source control:

npm install @x402/fetch @x402/evm viem
import{x402Client,wrapFetchWithPayment}from"@x402/fetch";import{registerExactEvmScheme}from"@x402/evm/exact/client";import{privateKeyToAccount}from"viem/accounts";constsigner=privateKeyToAccount(process.env.EVM_PRIVATE_KEY);constclient=newx402Client();registerExactEvmScheme(client,{ signer });constfetchWithPayment=wrapFetchWithPayment(fetch,client);constresponse=awaitfetchWithPayment("https://andrei-obol-agent.dvlabs.dev/v1/chat/completions",{method: "POST",headers: {"Content-Type": "application/json"},body: JSON.stringify({model: "openrouter/auto",stream: false,messages: [{role: "user",content: "Review the specification and probe https://seller.example.com/weather. Generate repair templates, but do not make a downstream payment.",}],}),},);console.log(response.status,awaitresponse.json());

An unpaid request should return 402 Payment Required. Its challenge should advertise:

  • network eip155:8453 (Base mainnet)
  • canonical Base USDC
  • amount 1000 atomic units (0.001 USDC)
  • the exact HTTPS Agent URL as its resource

The Agent defaults to no downstream payment. Paying to talk to Canary402 does not authorize Canary402 to pay the inspected service.

Ask Canary402 for an audit

A normal request can combine these duties:

  • Specification review inspects bounded copies of /openapi.json, /.well-known/agent-registration.json, and /skill.md, then compares them with the requested operation and live 402 metadata.
  • Repair generation proposes value-free OpenAPI and Bazaar fragments. These are starting points for seller review, not inferred business truth.
  • Probe-only verification validates reachability and the x402 challenge without signing or spending anything downstream.
  • Paid verification makes one budget-capped downstream payment, retries the target, and evaluates the delivered response.

Example prompts:

Review the specification and probe https://seller.example.com/services/weather.
Generate repair templates, but do not make a downstream payment.
Make a real paid audit of https://seller.example.com/service.
Use Base mainnet and do not spend more than 20000 atomic USDC units.

For a real downstream payment, the user must explicitly supply both:

  • payment_network: base or base-sepolia
  • max_payment_atomic: the maximum atomic token amount Canary402 may authorize

For six-decimal USDC, 1000 atomic units is 0.001 USDC. Canary402 also enforces its deployment-wide cap, currently 20000 atomic units (0.02 USDC).

Report meaning

The Agent summarizes the API's report ID, verdict, score, coverage, specification status, findings, repair artifacts, individual checks, and limitations. Reports may contain these checks:

  • service_discovery
  • request_contract
  • reachability
  • x402_challenge
  • payment_budget
  • paid_delivery
  • task_outcome
VerdictMeaning
PROBE_PASSThe payment challenge is valid, but no downstream payment was made
PROBE_PASS_WITH_WARNINGSThe probe worked, but service-contract metadata needs repair
PASSChallenge, paid delivery, and requested outcome passed
PASS_WITH_WARNINGSPaid delivery worked, but some evaluation was unavailable or uncertain
FAILAt least one required check failed

score measures checks that ran. coverage_percent shows how much of the full paid flow was exercised, so a probe-only result must be read with its lower coverage.

Stored reports do not contain purchased response bodies or fetched discovery-document bodies. They retain status, latency, content type, captured byte count, SHA-256 digests, payment terms, settlement evidence when present, and a short evaluator reason. Caller-supplied JSON property names can appear in generated repair fragments, so do not use repair generation for confidential request shapes.

Proven settlement history

Before the current Base-mainnet-only Agent offer was selected, a paid Agent call settled 0.001 Base Sepolia test USDC in transaction 0x8dd8…f91b.

Canary402 has also made a downstream payment of 0.02 Base mainnet USDC while auditing another hackathon service; transaction 0x9eb2…431c confirms the transfer.

These are point-in-time tests, not guarantees about either service.

Run locally

Prerequisites are an Obol Stack v0.14.0-rc0 cluster, Docker, Go 1.25+, and a working LiteLLM/OpenRouter configuration. Install or upgrade the Stack with:

OBOL_RELEASE=v0.14.0-rc0 bash <(curl -s https://stack.obol.org)# For this existing k3d stack, free its current host ports before forced init.
k3d cluster stop obol-stack-peaceful-starling
obol stack init --force --backend k3d --yes
obol stack up

For a new stack with no existing configuration, use obol stack init && obol stack up instead.

make test
make deploy
make smoke

make deploy builds and imports the local image, applies deploy/k8s.yaml, and keeps reports on the canary402-reports PVC. make smoke port-forwards the internal API and verifies its health without publishing it.

Publish or reconcile only the Agent offer:

make sell-agent

This command:

  1. creates or updates the Hermes Agent from agent/objective.md;
  2. applies the Agent's narrow cross-namespace egress policy;
  3. reconciles the x402 price to 0.001 USDC on Base mainnet;
  4. activates the offer against the existing Base Agent ID without minting another identity;
  5. binds the permanent hostname to v0.14's native dedicated-offer route and persists the resume ledger; and
  6. verifies the local offer.

Forks using another hostname should set CANARY_AGENT_HOSTNAME before running make sell-agent.

Direct development requests

The Go API is accessible only inside the cluster or through an operator-created port-forward:

obol kubectl -n llm port-forward service/canary402 18080:8080

Then run a no-spend audit directly:

curl --noproxy '*' -sS http://127.0.0.1:18080/audit \
-H 'Content-Type: application/json' \
--data '{ "url": "https://seller.example.com/services/weather/current", "method": "POST", "body": {"city": "Istanbul"}, "expectation": "The response contains a numeric temperature for Istanbul.", "spec_review": true, "generate_repairs": true, "pay": false }'

A complete request is also available in examples/audit-spec-review.json.

Safety model

Canary402 fails closed:

  • Only public HTTPS targets on port 443 are accepted.
  • Loopback, private, link-local, metadata, benchmark, documentation, carrier-grade NAT, IPv4-mapped IPv6, and standard NAT64 translation ranges are blocked.
  • DNS is checked again at connection time to reduce rebinding risk.
  • Redirects are never followed, especially after payment authorization is attached.
  • Request bodies are JSON and capped at 64 KiB; target responses are capped at 1 MiB.
  • Response bodies and fetched specification bodies are not persisted.
  • Discovery fetches only three fixed same-origin paths, does not follow redirects, and caps each document at 256 KiB.
  • Query strings and credentials are removed from stored report URLs.
  • Client authentication, cookies, arbitrary headers, and payment headers are not forwarded.
  • Downstream payment is opt-in and bounded by both the request cap and CANARY_MAX_PAYMENT_ATOMIC.
  • Target content is treated as untrusted evidence, including in the semantic-evaluation prompt.
  • Objective protocol checks remain separate from the subjective model verdict.

The downstream client supports x402 v2 exact/EIP-3009 with canonical USDC on Base (eip155:8453) and Base Sepolia (eip155:84532). Permit2, arbitrary ERC-20 assets, redirects, asynchronous jobs, custom caller headers, and SSE targets are deliberately unsupported.

Verification

make test
make vet
make status
make logs

The test suite covers specification discovery, path-template matching, deterministic repair generation, challenge comparison, caps, remote signing, paid retry, settlement evidence, semantic evaluation, persistent reports, IPv4/mapped-IPv6 equivalence, NAT64 blocks, and the invariant that inferred schemas contain no example values.

Configuration

Internal API runtime:

VariableDefaultMeaning
PORT8080HTTP listen port
CANARY_REPORT_DIR/data/reportsPersistent report directory
CANARY_MAX_PAYMENT_ATOMIC20000Absolute downstream per-audit cap
CANARY_ALLOWED_NETWORKSbase-sepolia,baseAllowed downstream networks
CANARY_MAX_CONCURRENT4Concurrent audit limit
CANARY_TARGET_TIMEOUT_SECONDS20Per-target timeout
CANARY_ALLOW_HTTPfalseDevelopment-only HTTP target support
CANARY_ALLOW_PRIVATE_TARGETSfalseTest-only private target support; never enable publicly
LITELLM_BASE_URLin-cluster LiteLLMOpenAI-compatible evaluator URL
LITELLM_MASTER_KEYemptyLiteLLM bearer token; missing means semantic evaluation is disabled
CANARY_MODELopenrouter/autoSemantic evaluator model
REMOTE_SIGNER_URLin-cluster signerObol typed-data signer URL
REMOTE_SIGNER_TOKENemptyOptional signer bearer token

Agent publication:

VariableDefaultMeaning
CANARY_AGENT_MODELopenrouter/autoHermes Agent model
CANARY_AGENT_NETWORKbaseInbound x402 payment network
CANARY_AGENT_IDENTITY_NETWORKbaseERC-8004 identity lookup chain; normally match the payment network
CANARY_AGENT_PRICE0.001Per-request USDC price
CANARY_AGENT_HOSTNAMEandrei-obol-agent.dvlabs.devDedicated public Agent origin

LiteLLM authentication comes from the existing llm/litellm-secrets Kubernetes secret. Payment signing uses the existing remote signer; its private key is never mounted into Canary402. The signer address is intentionally cached for the process lifetime, so a coordinated signer-key rotation requires restarting Canary402.

Secrets and source control

Local Obol/OpenRouter and tunnel credentials belong only in .env, which Git and Docker ignore. Never commit wallet private keys, remote-signer credentials, LLM_API_KEY, or the secret-bearing tunnel command.

git status --short --ignored
git check-ignore -v .env

Obol and registration notes

Obol v0.14.0-rc0 natively publishes the dedicated hostname, preserves the external HTTPS scheme in x402 challenges, exposes /.well-known/x402, and creates namespace-qualified cross-namespace grants. Canary402 therefore no longer carries the v0.13 tunnel-route or ReferenceGrant workarounds.

The primary identity is Base Agent ID 59094. Its mint transaction and agentURI correction succeeded. The corrected on-chain URI is https://andrei-obol-agent.dvlabs.dev/.well-known/agent-registration.json.

Legacy Base Sepolia Agent ID 8104 remains owned by the same wallet; its mint transaction also succeeded. Registration is idempotent while x402/AgentIdentity contains the correct per-chain IDs. Do not delete or overwrite that durable record, and pass the origin—not a service path—to obol sell register --origin.

ERC-8004 identity, an active x402 ServiceOffer, and x402scan discovery are separate states. After the v0.14 dedicated-origin migration, obol sell register x402scan idempotently registered 1/1 current resources and deprecated the old shared-origin resource.

About

Evidence-backed reliability audits for x402-paid agent services

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { // Force GitHub README to respect dark mode (function() { var style = document.createElement('style'); style.textContent = ' .markdown-body { color-scheme: dark light; } .markdown-body pre { background: #161b22 !important; } .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; } .markdown-body table th, .markdown-body table td { border-color: #30363d !important; } .markdown-body img { background: #0d1117; } .markdown-body blockquote { border-left-color: #8b949e; } .markdown-body hr { border-color: #30363d; } '; document.head.appendChild(style); })(); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' GitHub - pinebit/canary402: Evidence-backed reliability audits for x402-paid agent services · GitHub
Skip to content

Repository files navigation

Canary402

Canary402 is a service-contract inspector and mystery shopper for x402-paid services:

Before an agent integrates with or pays a service, Canary402 checks the contract and verifies the delivery.

It inspects public service specifications, generates value-free repair templates, validates the live x402 challenge, optionally makes one strictly budgeted downstream payment through the Obol remote signer, evaluates the delivered result with OpenRouter, and retains an evidence-backed report.

The original concept is preserved in idea.md.

Live Agent

The public product is the Canary402 Agent at:

POST https://andrei-obol-agent.dvlabs.dev/v1/chat/completions

The previous shared-origin path, POST /services/canary402-agent/v1/chat/completions, remains a working compatibility alias.

The Agent is a Hermes/OpenRouter service sold for 0.001 USDC per request on Base mainnet. Its deterministic Go audit API remains deployed inside the Obol cluster and is not sold, advertised, or routed publicly. Calling the Agent therefore incurs one Canary402 charge, not a second internal API charge.

Canary402 is registered as ERC-8004 Agent ID 59094 on Base mainnet, matching its x402 payment network. The same signer also retains the legacy Agent ID 8104 on Base Sepolia. Both registration entries point to the same public Agent and are owned by the Canary402 wallet.

Public discovery documents are available from the dedicated Agent origin:

  • GET https://andrei-obol-agent.dvlabs.dev/openapi.json
  • GET https://andrei-obol-agent.dvlabs.dev/.well-known/x402
  • GET https://andrei-obol-agent.dvlabs.dev/.well-known/agent-registration.json

x402scan crawled the migrated dedicated origin and registered its single current paid resource (1/1), deprecating the earlier shared-origin resource. The listing command warned only that the generated OpenAPI lacks info.contact.email; that field is optional for indexing but enables ownership/contact customization.

The public registration document is active and advertises only the Base Agent identity and endpoint. The legacy Sepolia registration remains owned by the same wallet and retained in Obol's durable AgentIdentity state for historical continuity, but is not advertised by this Base-paid offer.

The old /services/canary402 HTTP offer and its public health/report routes have intentionally been removed. Reports are still retained internally, and the Agent returns the report ID and an evidence-based interpretation in its response.

Buy an Agent request

The endpoint accepts OpenAI-style chat-completions JSON behind an x402 v2 gate. The buyer wallet needs canonical USDC on Base mainnet; the inbound payment itself does not require the buyer to fund a separate Canary402 wallet.

Keep all private keys outside source control:

npm install @x402/fetch @x402/evm viem
import{x402Client,wrapFetchWithPayment}from"@x402/fetch";import{registerExactEvmScheme}from"@x402/evm/exact/client";import{privateKeyToAccount}from"viem/accounts";constsigner=privateKeyToAccount(process.env.EVM_PRIVATE_KEY);constclient=newx402Client();registerExactEvmScheme(client,{ signer });constfetchWithPayment=wrapFetchWithPayment(fetch,client);constresponse=awaitfetchWithPayment("https://andrei-obol-agent.dvlabs.dev/v1/chat/completions",{method: "POST",headers: {"Content-Type": "application/json"},body: JSON.stringify({model: "openrouter/auto",stream: false,messages: [{role: "user",content: "Review the specification and probe https://seller.example.com/weather. Generate repair templates, but do not make a downstream payment.",}],}),},);console.log(response.status,awaitresponse.json());

An unpaid request should return 402 Payment Required. Its challenge should advertise:

  • network eip155:8453 (Base mainnet)
  • canonical Base USDC
  • amount 1000 atomic units (0.001 USDC)
  • the exact HTTPS Agent URL as its resource

The Agent defaults to no downstream payment. Paying to talk to Canary402 does not authorize Canary402 to pay the inspected service.

Ask Canary402 for an audit

A normal request can combine these duties:

  • Specification review inspects bounded copies of /openapi.json, /.well-known/agent-registration.json, and /skill.md, then compares them with the requested operation and live 402 metadata.
  • Repair generation proposes value-free OpenAPI and Bazaar fragments. These are starting points for seller review, not inferred business truth.
  • Probe-only verification validates reachability and the x402 challenge without signing or spending anything downstream.
  • Paid verification makes one budget-capped downstream payment, retries the target, and evaluates the delivered response.

Example prompts:

Review the specification and probe https://seller.example.com/services/weather.
Generate repair templates, but do not make a downstream payment.
Make a real paid audit of https://seller.example.com/service.
Use Base mainnet and do not spend more than 20000 atomic USDC units.

For a real downstream payment, the user must explicitly supply both:

  • payment_network: base or base-sepolia
  • max_payment_atomic: the maximum atomic token amount Canary402 may authorize

For six-decimal USDC, 1000 atomic units is 0.001 USDC. Canary402 also enforces its deployment-wide cap, currently 20000 atomic units (0.02 USDC).

Report meaning

The Agent summarizes the API's report ID, verdict, score, coverage, specification status, findings, repair artifacts, individual checks, and limitations. Reports may contain these checks:

  • service_discovery
  • request_contract
  • reachability
  • x402_challenge
  • payment_budget
  • paid_delivery
  • task_outcome
VerdictMeaning
PROBE_PASSThe payment challenge is valid, but no downstream payment was made
PROBE_PASS_WITH_WARNINGSThe probe worked, but service-contract metadata needs repair
PASSChallenge, paid delivery, and requested outcome passed
PASS_WITH_WARNINGSPaid delivery worked, but some evaluation was unavailable or uncertain
FAILAt least one required check failed

score measures checks that ran. coverage_percent shows how much of the full paid flow was exercised, so a probe-only result must be read with its lower coverage.

Stored reports do not contain purchased response bodies or fetched discovery-document bodies. They retain status, latency, content type, captured byte count, SHA-256 digests, payment terms, settlement evidence when present, and a short evaluator reason. Caller-supplied JSON property names can appear in generated repair fragments, so do not use repair generation for confidential request shapes.

Proven settlement history

Before the current Base-mainnet-only Agent offer was selected, a paid Agent call settled 0.001 Base Sepolia test USDC in transaction 0x8dd8…f91b.

Canary402 has also made a downstream payment of 0.02 Base mainnet USDC while auditing another hackathon service; transaction 0x9eb2…431c confirms the transfer.

These are point-in-time tests, not guarantees about either service.

Run locally

Prerequisites are an Obol Stack v0.14.0-rc0 cluster, Docker, Go 1.25+, and a working LiteLLM/OpenRouter configuration. Install or upgrade the Stack with:

OBOL_RELEASE=v0.14.0-rc0 bash <(curl -s https://stack.obol.org)# For this existing k3d stack, free its current host ports before forced init.
k3d cluster stop obol-stack-peaceful-starling
obol stack init --force --backend k3d --yes
obol stack up

For a new stack with no existing configuration, use obol stack init && obol stack up instead.

make test
make deploy
make smoke

make deploy builds and imports the local image, applies deploy/k8s.yaml, and keeps reports on the canary402-reports PVC. make smoke port-forwards the internal API and verifies its health without publishing it.

Publish or reconcile only the Agent offer:

make sell-agent

This command:

  1. creates or updates the Hermes Agent from agent/objective.md;
  2. applies the Agent's narrow cross-namespace egress policy;
  3. reconciles the x402 price to 0.001 USDC on Base mainnet;
  4. activates the offer against the existing Base Agent ID without minting another identity;
  5. binds the permanent hostname to v0.14's native dedicated-offer route and persists the resume ledger; and
  6. verifies the local offer.

Forks using another hostname should set CANARY_AGENT_HOSTNAME before running make sell-agent.

Direct development requests

The Go API is accessible only inside the cluster or through an operator-created port-forward:

obol kubectl -n llm port-forward service/canary402 18080:8080

Then run a no-spend audit directly:

curl --noproxy '*' -sS http://127.0.0.1:18080/audit \
-H 'Content-Type: application/json' \
--data '{ "url": "https://seller.example.com/services/weather/current", "method": "POST", "body": {"city": "Istanbul"}, "expectation": "The response contains a numeric temperature for Istanbul.", "spec_review": true, "generate_repairs": true, "pay": false }'

A complete request is also available in examples/audit-spec-review.json.

Safety model

Canary402 fails closed:

  • Only public HTTPS targets on port 443 are accepted.
  • Loopback, private, link-local, metadata, benchmark, documentation, carrier-grade NAT, IPv4-mapped IPv6, and standard NAT64 translation ranges are blocked.
  • DNS is checked again at connection time to reduce rebinding risk.
  • Redirects are never followed, especially after payment authorization is attached.
  • Request bodies are JSON and capped at 64 KiB; target responses are capped at 1 MiB.
  • Response bodies and fetched specification bodies are not persisted.
  • Discovery fetches only three fixed same-origin paths, does not follow redirects, and caps each document at 256 KiB.
  • Query strings and credentials are removed from stored report URLs.
  • Client authentication, cookies, arbitrary headers, and payment headers are not forwarded.
  • Downstream payment is opt-in and bounded by both the request cap and CANARY_MAX_PAYMENT_ATOMIC.
  • Target content is treated as untrusted evidence, including in the semantic-evaluation prompt.
  • Objective protocol checks remain separate from the subjective model verdict.

The downstream client supports x402 v2 exact/EIP-3009 with canonical USDC on Base (eip155:8453) and Base Sepolia (eip155:84532). Permit2, arbitrary ERC-20 assets, redirects, asynchronous jobs, custom caller headers, and SSE targets are deliberately unsupported.

Verification

make test
make vet
make status
make logs

The test suite covers specification discovery, path-template matching, deterministic repair generation, challenge comparison, caps, remote signing, paid retry, settlement evidence, semantic evaluation, persistent reports, IPv4/mapped-IPv6 equivalence, NAT64 blocks, and the invariant that inferred schemas contain no example values.

Configuration

Internal API runtime:

VariableDefaultMeaning
PORT8080HTTP listen port
CANARY_REPORT_DIR/data/reportsPersistent report directory
CANARY_MAX_PAYMENT_ATOMIC20000Absolute downstream per-audit cap
CANARY_ALLOWED_NETWORKSbase-sepolia,baseAllowed downstream networks
CANARY_MAX_CONCURRENT4Concurrent audit limit
CANARY_TARGET_TIMEOUT_SECONDS20Per-target timeout
CANARY_ALLOW_HTTPfalseDevelopment-only HTTP target support
CANARY_ALLOW_PRIVATE_TARGETSfalseTest-only private target support; never enable publicly
LITELLM_BASE_URLin-cluster LiteLLMOpenAI-compatible evaluator URL
LITELLM_MASTER_KEYemptyLiteLLM bearer token; missing means semantic evaluation is disabled
CANARY_MODELopenrouter/autoSemantic evaluator model
REMOTE_SIGNER_URLin-cluster signerObol typed-data signer URL
REMOTE_SIGNER_TOKENemptyOptional signer bearer token

Agent publication:

VariableDefaultMeaning
CANARY_AGENT_MODELopenrouter/autoHermes Agent model
CANARY_AGENT_NETWORKbaseInbound x402 payment network
CANARY_AGENT_IDENTITY_NETWORKbaseERC-8004 identity lookup chain; normally match the payment network
CANARY_AGENT_PRICE0.001Per-request USDC price
CANARY_AGENT_HOSTNAMEandrei-obol-agent.dvlabs.devDedicated public Agent origin

LiteLLM authentication comes from the existing llm/litellm-secrets Kubernetes secret. Payment signing uses the existing remote signer; its private key is never mounted into Canary402. The signer address is intentionally cached for the process lifetime, so a coordinated signer-key rotation requires restarting Canary402.

Secrets and source control

Local Obol/OpenRouter and tunnel credentials belong only in .env, which Git and Docker ignore. Never commit wallet private keys, remote-signer credentials, LLM_API_KEY, or the secret-bearing tunnel command.

git status --short --ignored
git check-ignore -v .env

Obol and registration notes

Obol v0.14.0-rc0 natively publishes the dedicated hostname, preserves the external HTTPS scheme in x402 challenges, exposes /.well-known/x402, and creates namespace-qualified cross-namespace grants. Canary402 therefore no longer carries the v0.13 tunnel-route or ReferenceGrant workarounds.

The primary identity is Base Agent ID 59094. Its mint transaction and agentURI correction succeeded. The corrected on-chain URI is https://andrei-obol-agent.dvlabs.dev/.well-known/agent-registration.json.

Legacy Base Sepolia Agent ID 8104 remains owned by the same wallet; its mint transaction also succeeded. Registration is idempotent while x402/AgentIdentity contains the correct per-chain IDs. Do not delete or overwrite that durable record, and pass the origin—not a service path—to obol sell register --origin.

ERC-8004 identity, an active x402 ServiceOffer, and x402scan discovery are separate states. After the v0.14 dedicated-origin migration, obol sell register x402scan idempotently registered 1/1 current resources and deprecated the old shared-origin resource.

About

Evidence-backed reliability audits for x402-paid agent services

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { // Highlight search terms from Google/DuckDuckGo/Bing referrer (function() { var ref = document.referrer; var terms = []; if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) { var url = new URL(ref); var q = url.searchParams.get('q') || url.searchParams.get('p'); if (q) { terms = q.split(/\s+/).filter(function(t) { return t.length > 2; }); } } if (terms.length === 0) return; var style = document.createElement('style'); style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }'; document.head.appendChild(style); function highlight(node) { if (node.nodeType === 3) { // text node var text = node.textContent; var found = false; terms.forEach(function(term) { var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi'); if (regex.test(text)) { found = true; var frag = document.createDocumentFragment(); var parts = text.split(regex); parts.forEach(function(part, i) { if (i % 2 === 0) { frag.appendChild(document.createTextNode(part)); } else { var span = document.createElement('span'); span.className = 'userscript-highlight'; span.textContent = part; frag.appendChild(span); } }); node.parentNode.replaceChild(frag, node); } }); } else if (node.nodeType === 1 && node.childNodes) { // element var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT']; if (!skipTags.includes(node.tagName)) { Array.from(node.childNodes).forEach(highlight); } } } highlight(document.body); // Re-highlight on dynamic content var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1 || node.nodeType === 3) highlight(node); }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' GitHub - pinebit/canary402: Evidence-backed reliability audits for x402-paid agent services · GitHub
Skip to content

Repository files navigation

Canary402

Canary402 is a service-contract inspector and mystery shopper for x402-paid services:

Before an agent integrates with or pays a service, Canary402 checks the contract and verifies the delivery.

It inspects public service specifications, generates value-free repair templates, validates the live x402 challenge, optionally makes one strictly budgeted downstream payment through the Obol remote signer, evaluates the delivered result with OpenRouter, and retains an evidence-backed report.

The original concept is preserved in idea.md.

Live Agent

The public product is the Canary402 Agent at:

POST https://andrei-obol-agent.dvlabs.dev/v1/chat/completions

The previous shared-origin path, POST /services/canary402-agent/v1/chat/completions, remains a working compatibility alias.

The Agent is a Hermes/OpenRouter service sold for 0.001 USDC per request on Base mainnet. Its deterministic Go audit API remains deployed inside the Obol cluster and is not sold, advertised, or routed publicly. Calling the Agent therefore incurs one Canary402 charge, not a second internal API charge.

Canary402 is registered as ERC-8004 Agent ID 59094 on Base mainnet, matching its x402 payment network. The same signer also retains the legacy Agent ID 8104 on Base Sepolia. Both registration entries point to the same public Agent and are owned by the Canary402 wallet.

Public discovery documents are available from the dedicated Agent origin:

  • GET https://andrei-obol-agent.dvlabs.dev/openapi.json
  • GET https://andrei-obol-agent.dvlabs.dev/.well-known/x402
  • GET https://andrei-obol-agent.dvlabs.dev/.well-known/agent-registration.json

x402scan crawled the migrated dedicated origin and registered its single current paid resource (1/1), deprecating the earlier shared-origin resource. The listing command warned only that the generated OpenAPI lacks info.contact.email; that field is optional for indexing but enables ownership/contact customization.

The public registration document is active and advertises only the Base Agent identity and endpoint. The legacy Sepolia registration remains owned by the same wallet and retained in Obol's durable AgentIdentity state for historical continuity, but is not advertised by this Base-paid offer.

The old /services/canary402 HTTP offer and its public health/report routes have intentionally been removed. Reports are still retained internally, and the Agent returns the report ID and an evidence-based interpretation in its response.

Buy an Agent request

The endpoint accepts OpenAI-style chat-completions JSON behind an x402 v2 gate. The buyer wallet needs canonical USDC on Base mainnet; the inbound payment itself does not require the buyer to fund a separate Canary402 wallet.

Keep all private keys outside source control:

npm install @x402/fetch @x402/evm viem
import{x402Client,wrapFetchWithPayment}from"@x402/fetch";import{registerExactEvmScheme}from"@x402/evm/exact/client";import{privateKeyToAccount}from"viem/accounts";constsigner=privateKeyToAccount(process.env.EVM_PRIVATE_KEY);constclient=newx402Client();registerExactEvmScheme(client,{ signer });constfetchWithPayment=wrapFetchWithPayment(fetch,client);constresponse=awaitfetchWithPayment("https://andrei-obol-agent.dvlabs.dev/v1/chat/completions",{method: "POST",headers: {"Content-Type": "application/json"},body: JSON.stringify({model: "openrouter/auto",stream: false,messages: [{role: "user",content: "Review the specification and probe https://seller.example.com/weather. Generate repair templates, but do not make a downstream payment.",}],}),},);console.log(response.status,awaitresponse.json());

An unpaid request should return 402 Payment Required. Its challenge should advertise:

  • network eip155:8453 (Base mainnet)
  • canonical Base USDC
  • amount 1000 atomic units (0.001 USDC)
  • the exact HTTPS Agent URL as its resource

The Agent defaults to no downstream payment. Paying to talk to Canary402 does not authorize Canary402 to pay the inspected service.

Ask Canary402 for an audit

A normal request can combine these duties:

  • Specification review inspects bounded copies of /openapi.json, /.well-known/agent-registration.json, and /skill.md, then compares them with the requested operation and live 402 metadata.
  • Repair generation proposes value-free OpenAPI and Bazaar fragments. These are starting points for seller review, not inferred business truth.
  • Probe-only verification validates reachability and the x402 challenge without signing or spending anything downstream.
  • Paid verification makes one budget-capped downstream payment, retries the target, and evaluates the delivered response.

Example prompts:

Review the specification and probe https://seller.example.com/services/weather.
Generate repair templates, but do not make a downstream payment.
Make a real paid audit of https://seller.example.com/service.
Use Base mainnet and do not spend more than 20000 atomic USDC units.

For a real downstream payment, the user must explicitly supply both:

  • payment_network: base or base-sepolia
  • max_payment_atomic: the maximum atomic token amount Canary402 may authorize

For six-decimal USDC, 1000 atomic units is 0.001 USDC. Canary402 also enforces its deployment-wide cap, currently 20000 atomic units (0.02 USDC).

Report meaning

The Agent summarizes the API's report ID, verdict, score, coverage, specification status, findings, repair artifacts, individual checks, and limitations. Reports may contain these checks:

  • service_discovery
  • request_contract
  • reachability
  • x402_challenge
  • payment_budget
  • paid_delivery
  • task_outcome
VerdictMeaning
PROBE_PASSThe payment challenge is valid, but no downstream payment was made
PROBE_PASS_WITH_WARNINGSThe probe worked, but service-contract metadata needs repair
PASSChallenge, paid delivery, and requested outcome passed
PASS_WITH_WARNINGSPaid delivery worked, but some evaluation was unavailable or uncertain
FAILAt least one required check failed

score measures checks that ran. coverage_percent shows how much of the full paid flow was exercised, so a probe-only result must be read with its lower coverage.

Stored reports do not contain purchased response bodies or fetched discovery-document bodies. They retain status, latency, content type, captured byte count, SHA-256 digests, payment terms, settlement evidence when present, and a short evaluator reason. Caller-supplied JSON property names can appear in generated repair fragments, so do not use repair generation for confidential request shapes.

Proven settlement history

Before the current Base-mainnet-only Agent offer was selected, a paid Agent call settled 0.001 Base Sepolia test USDC in transaction 0x8dd8…f91b.

Canary402 has also made a downstream payment of 0.02 Base mainnet USDC while auditing another hackathon service; transaction 0x9eb2…431c confirms the transfer.

These are point-in-time tests, not guarantees about either service.

Run locally

Prerequisites are an Obol Stack v0.14.0-rc0 cluster, Docker, Go 1.25+, and a working LiteLLM/OpenRouter configuration. Install or upgrade the Stack with:

OBOL_RELEASE=v0.14.0-rc0 bash <(curl -s https://stack.obol.org)# For this existing k3d stack, free its current host ports before forced init.
k3d cluster stop obol-stack-peaceful-starling
obol stack init --force --backend k3d --yes
obol stack up

For a new stack with no existing configuration, use obol stack init && obol stack up instead.

make test
make deploy
make smoke

make deploy builds and imports the local image, applies deploy/k8s.yaml, and keeps reports on the canary402-reports PVC. make smoke port-forwards the internal API and verifies its health without publishing it.

Publish or reconcile only the Agent offer:

make sell-agent

This command:

  1. creates or updates the Hermes Agent from agent/objective.md;
  2. applies the Agent's narrow cross-namespace egress policy;
  3. reconciles the x402 price to 0.001 USDC on Base mainnet;
  4. activates the offer against the existing Base Agent ID without minting another identity;
  5. binds the permanent hostname to v0.14's native dedicated-offer route and persists the resume ledger; and
  6. verifies the local offer.

Forks using another hostname should set CANARY_AGENT_HOSTNAME before running make sell-agent.

Direct development requests

The Go API is accessible only inside the cluster or through an operator-created port-forward:

obol kubectl -n llm port-forward service/canary402 18080:8080

Then run a no-spend audit directly:

curl --noproxy '*' -sS http://127.0.0.1:18080/audit \
-H 'Content-Type: application/json' \
--data '{ "url": "https://seller.example.com/services/weather/current", "method": "POST", "body": {"city": "Istanbul"}, "expectation": "The response contains a numeric temperature for Istanbul.", "spec_review": true, "generate_repairs": true, "pay": false }'

A complete request is also available in examples/audit-spec-review.json.

Safety model

Canary402 fails closed:

  • Only public HTTPS targets on port 443 are accepted.
  • Loopback, private, link-local, metadata, benchmark, documentation, carrier-grade NAT, IPv4-mapped IPv6, and standard NAT64 translation ranges are blocked.
  • DNS is checked again at connection time to reduce rebinding risk.
  • Redirects are never followed, especially after payment authorization is attached.
  • Request bodies are JSON and capped at 64 KiB; target responses are capped at 1 MiB.
  • Response bodies and fetched specification bodies are not persisted.
  • Discovery fetches only three fixed same-origin paths, does not follow redirects, and caps each document at 256 KiB.
  • Query strings and credentials are removed from stored report URLs.
  • Client authentication, cookies, arbitrary headers, and payment headers are not forwarded.
  • Downstream payment is opt-in and bounded by both the request cap and CANARY_MAX_PAYMENT_ATOMIC.
  • Target content is treated as untrusted evidence, including in the semantic-evaluation prompt.
  • Objective protocol checks remain separate from the subjective model verdict.

The downstream client supports x402 v2 exact/EIP-3009 with canonical USDC on Base (eip155:8453) and Base Sepolia (eip155:84532). Permit2, arbitrary ERC-20 assets, redirects, asynchronous jobs, custom caller headers, and SSE targets are deliberately unsupported.

Verification

make test
make vet
make status
make logs

The test suite covers specification discovery, path-template matching, deterministic repair generation, challenge comparison, caps, remote signing, paid retry, settlement evidence, semantic evaluation, persistent reports, IPv4/mapped-IPv6 equivalence, NAT64 blocks, and the invariant that inferred schemas contain no example values.

Configuration

Internal API runtime:

VariableDefaultMeaning
PORT8080HTTP listen port
CANARY_REPORT_DIR/data/reportsPersistent report directory
CANARY_MAX_PAYMENT_ATOMIC20000Absolute downstream per-audit cap
CANARY_ALLOWED_NETWORKSbase-sepolia,baseAllowed downstream networks
CANARY_MAX_CONCURRENT4Concurrent audit limit
CANARY_TARGET_TIMEOUT_SECONDS20Per-target timeout
CANARY_ALLOW_HTTPfalseDevelopment-only HTTP target support
CANARY_ALLOW_PRIVATE_TARGETSfalseTest-only private target support; never enable publicly
LITELLM_BASE_URLin-cluster LiteLLMOpenAI-compatible evaluator URL
LITELLM_MASTER_KEYemptyLiteLLM bearer token; missing means semantic evaluation is disabled
CANARY_MODELopenrouter/autoSemantic evaluator model
REMOTE_SIGNER_URLin-cluster signerObol typed-data signer URL
REMOTE_SIGNER_TOKENemptyOptional signer bearer token

Agent publication:

VariableDefaultMeaning
CANARY_AGENT_MODELopenrouter/autoHermes Agent model
CANARY_AGENT_NETWORKbaseInbound x402 payment network
CANARY_AGENT_IDENTITY_NETWORKbaseERC-8004 identity lookup chain; normally match the payment network
CANARY_AGENT_PRICE0.001Per-request USDC price
CANARY_AGENT_HOSTNAMEandrei-obol-agent.dvlabs.devDedicated public Agent origin

LiteLLM authentication comes from the existing llm/litellm-secrets Kubernetes secret. Payment signing uses the existing remote signer; its private key is never mounted into Canary402. The signer address is intentionally cached for the process lifetime, so a coordinated signer-key rotation requires restarting Canary402.

Secrets and source control

Local Obol/OpenRouter and tunnel credentials belong only in .env, which Git and Docker ignore. Never commit wallet private keys, remote-signer credentials, LLM_API_KEY, or the secret-bearing tunnel command.

git status --short --ignored
git check-ignore -v .env

Obol and registration notes

Obol v0.14.0-rc0 natively publishes the dedicated hostname, preserves the external HTTPS scheme in x402 challenges, exposes /.well-known/x402, and creates namespace-qualified cross-namespace grants. Canary402 therefore no longer carries the v0.13 tunnel-route or ReferenceGrant workarounds.

The primary identity is Base Agent ID 59094. Its mint transaction and agentURI correction succeeded. The corrected on-chain URI is https://andrei-obol-agent.dvlabs.dev/.well-known/agent-registration.json.

Legacy Base Sepolia Agent ID 8104 remains owned by the same wallet; its mint transaction also succeeded. Registration is idempotent while x402/AgentIdentity contains the correct per-chain IDs. Do not delete or overwrite that durable record, and pass the origin—not a service path—to obol sell register --origin.

ERC-8004 identity, an active x402 ServiceOffer, and x402scan discovery are separate states. After the v0.14 dedicated-origin migration, obol sell register x402scan idempotently registered 1/1 current resources and deprecated the old shared-origin resource.

About

Evidence-backed reliability audits for x402-paid agent services

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { // Strip utm_, fbclid, gclid, etc. from all links on page (function() { var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid', 'ref', 'ref_src', 'source', 'medium', 'campaign']; function cleanUrl(url) { try { var u = new URL(url, window.location.origin); var changed = false; trackingParams.forEach(function(p) { if (u.searchParams.has(p)) { u.searchParams.delete(p); changed = true; } }); return changed ? u.toString() : url; } catch (e) { return url; } } function cleanLinks() { document.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } cleanLinks(); var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1) { if (node.tagName === 'A') cleanLinks(); node.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + ' GitHub - pinebit/canary402: Evidence-backed reliability audits for x402-paid agent services · GitHub
Skip to content

Repository files navigation

Canary402

Canary402 is a service-contract inspector and mystery shopper for x402-paid services:

Before an agent integrates with or pays a service, Canary402 checks the contract and verifies the delivery.

It inspects public service specifications, generates value-free repair templates, validates the live x402 challenge, optionally makes one strictly budgeted downstream payment through the Obol remote signer, evaluates the delivered result with OpenRouter, and retains an evidence-backed report.

The original concept is preserved in idea.md.

Live Agent

The public product is the Canary402 Agent at:

POST https://andrei-obol-agent.dvlabs.dev/v1/chat/completions

The previous shared-origin path, POST /services/canary402-agent/v1/chat/completions, remains a working compatibility alias.

The Agent is a Hermes/OpenRouter service sold for 0.001 USDC per request on Base mainnet. Its deterministic Go audit API remains deployed inside the Obol cluster and is not sold, advertised, or routed publicly. Calling the Agent therefore incurs one Canary402 charge, not a second internal API charge.

Canary402 is registered as ERC-8004 Agent ID 59094 on Base mainnet, matching its x402 payment network. The same signer also retains the legacy Agent ID 8104 on Base Sepolia. Both registration entries point to the same public Agent and are owned by the Canary402 wallet.

Public discovery documents are available from the dedicated Agent origin:

  • GET https://andrei-obol-agent.dvlabs.dev/openapi.json
  • GET https://andrei-obol-agent.dvlabs.dev/.well-known/x402
  • GET https://andrei-obol-agent.dvlabs.dev/.well-known/agent-registration.json

x402scan crawled the migrated dedicated origin and registered its single current paid resource (1/1), deprecating the earlier shared-origin resource. The listing command warned only that the generated OpenAPI lacks info.contact.email; that field is optional for indexing but enables ownership/contact customization.

The public registration document is active and advertises only the Base Agent identity and endpoint. The legacy Sepolia registration remains owned by the same wallet and retained in Obol's durable AgentIdentity state for historical continuity, but is not advertised by this Base-paid offer.

The old /services/canary402 HTTP offer and its public health/report routes have intentionally been removed. Reports are still retained internally, and the Agent returns the report ID and an evidence-based interpretation in its response.

Buy an Agent request

The endpoint accepts OpenAI-style chat-completions JSON behind an x402 v2 gate. The buyer wallet needs canonical USDC on Base mainnet; the inbound payment itself does not require the buyer to fund a separate Canary402 wallet.

Keep all private keys outside source control:

npm install @x402/fetch @x402/evm viem
import{x402Client,wrapFetchWithPayment}from"@x402/fetch";import{registerExactEvmScheme}from"@x402/evm/exact/client";import{privateKeyToAccount}from"viem/accounts";constsigner=privateKeyToAccount(process.env.EVM_PRIVATE_KEY);constclient=newx402Client();registerExactEvmScheme(client,{ signer });constfetchWithPayment=wrapFetchWithPayment(fetch,client);constresponse=awaitfetchWithPayment("https://andrei-obol-agent.dvlabs.dev/v1/chat/completions",{method: "POST",headers: {"Content-Type": "application/json"},body: JSON.stringify({model: "openrouter/auto",stream: false,messages: [{role: "user",content: "Review the specification and probe https://seller.example.com/weather. Generate repair templates, but do not make a downstream payment.",}],}),},);console.log(response.status,awaitresponse.json());

An unpaid request should return 402 Payment Required. Its challenge should advertise:

  • network eip155:8453 (Base mainnet)
  • canonical Base USDC
  • amount 1000 atomic units (0.001 USDC)
  • the exact HTTPS Agent URL as its resource

The Agent defaults to no downstream payment. Paying to talk to Canary402 does not authorize Canary402 to pay the inspected service.

Ask Canary402 for an audit

A normal request can combine these duties:

  • Specification review inspects bounded copies of /openapi.json, /.well-known/agent-registration.json, and /skill.md, then compares them with the requested operation and live 402 metadata.
  • Repair generation proposes value-free OpenAPI and Bazaar fragments. These are starting points for seller review, not inferred business truth.
  • Probe-only verification validates reachability and the x402 challenge without signing or spending anything downstream.
  • Paid verification makes one budget-capped downstream payment, retries the target, and evaluates the delivered response.

Example prompts:

Review the specification and probe https://seller.example.com/services/weather.
Generate repair templates, but do not make a downstream payment.
Make a real paid audit of https://seller.example.com/service.
Use Base mainnet and do not spend more than 20000 atomic USDC units.

For a real downstream payment, the user must explicitly supply both:

  • payment_network: base or base-sepolia
  • max_payment_atomic: the maximum atomic token amount Canary402 may authorize

For six-decimal USDC, 1000 atomic units is 0.001 USDC. Canary402 also enforces its deployment-wide cap, currently 20000 atomic units (0.02 USDC).

Report meaning

The Agent summarizes the API's report ID, verdict, score, coverage, specification status, findings, repair artifacts, individual checks, and limitations. Reports may contain these checks:

  • service_discovery
  • request_contract
  • reachability
  • x402_challenge
  • payment_budget
  • paid_delivery
  • task_outcome
VerdictMeaning
PROBE_PASSThe payment challenge is valid, but no downstream payment was made
PROBE_PASS_WITH_WARNINGSThe probe worked, but service-contract metadata needs repair
PASSChallenge, paid delivery, and requested outcome passed
PASS_WITH_WARNINGSPaid delivery worked, but some evaluation was unavailable or uncertain
FAILAt least one required check failed

score measures checks that ran. coverage_percent shows how much of the full paid flow was exercised, so a probe-only result must be read with its lower coverage.

Stored reports do not contain purchased response bodies or fetched discovery-document bodies. They retain status, latency, content type, captured byte count, SHA-256 digests, payment terms, settlement evidence when present, and a short evaluator reason. Caller-supplied JSON property names can appear in generated repair fragments, so do not use repair generation for confidential request shapes.

Proven settlement history

Before the current Base-mainnet-only Agent offer was selected, a paid Agent call settled 0.001 Base Sepolia test USDC in transaction 0x8dd8…f91b.

Canary402 has also made a downstream payment of 0.02 Base mainnet USDC while auditing another hackathon service; transaction 0x9eb2…431c confirms the transfer.

These are point-in-time tests, not guarantees about either service.

Run locally

Prerequisites are an Obol Stack v0.14.0-rc0 cluster, Docker, Go 1.25+, and a working LiteLLM/OpenRouter configuration. Install or upgrade the Stack with:

OBOL_RELEASE=v0.14.0-rc0 bash <(curl -s https://stack.obol.org)# For this existing k3d stack, free its current host ports before forced init.
k3d cluster stop obol-stack-peaceful-starling
obol stack init --force --backend k3d --yes
obol stack up

For a new stack with no existing configuration, use obol stack init && obol stack up instead.

make test
make deploy
make smoke

make deploy builds and imports the local image, applies deploy/k8s.yaml, and keeps reports on the canary402-reports PVC. make smoke port-forwards the internal API and verifies its health without publishing it.

Publish or reconcile only the Agent offer:

make sell-agent

This command:

  1. creates or updates the Hermes Agent from agent/objective.md;
  2. applies the Agent's narrow cross-namespace egress policy;
  3. reconciles the x402 price to 0.001 USDC on Base mainnet;
  4. activates the offer against the existing Base Agent ID without minting another identity;
  5. binds the permanent hostname to v0.14's native dedicated-offer route and persists the resume ledger; and
  6. verifies the local offer.

Forks using another hostname should set CANARY_AGENT_HOSTNAME before running make sell-agent.

Direct development requests

The Go API is accessible only inside the cluster or through an operator-created port-forward:

obol kubectl -n llm port-forward service/canary402 18080:8080

Then run a no-spend audit directly:

curl --noproxy '*' -sS http://127.0.0.1:18080/audit \
-H 'Content-Type: application/json' \
--data '{ "url": "https://seller.example.com/services/weather/current", "method": "POST", "body": {"city": "Istanbul"}, "expectation": "The response contains a numeric temperature for Istanbul.", "spec_review": true, "generate_repairs": true, "pay": false }'

A complete request is also available in examples/audit-spec-review.json.

Safety model

Canary402 fails closed:

  • Only public HTTPS targets on port 443 are accepted.
  • Loopback, private, link-local, metadata, benchmark, documentation, carrier-grade NAT, IPv4-mapped IPv6, and standard NAT64 translation ranges are blocked.
  • DNS is checked again at connection time to reduce rebinding risk.
  • Redirects are never followed, especially after payment authorization is attached.
  • Request bodies are JSON and capped at 64 KiB; target responses are capped at 1 MiB.
  • Response bodies and fetched specification bodies are not persisted.
  • Discovery fetches only three fixed same-origin paths, does not follow redirects, and caps each document at 256 KiB.
  • Query strings and credentials are removed from stored report URLs.
  • Client authentication, cookies, arbitrary headers, and payment headers are not forwarded.
  • Downstream payment is opt-in and bounded by both the request cap and CANARY_MAX_PAYMENT_ATOMIC.
  • Target content is treated as untrusted evidence, including in the semantic-evaluation prompt.
  • Objective protocol checks remain separate from the subjective model verdict.

The downstream client supports x402 v2 exact/EIP-3009 with canonical USDC on Base (eip155:8453) and Base Sepolia (eip155:84532). Permit2, arbitrary ERC-20 assets, redirects, asynchronous jobs, custom caller headers, and SSE targets are deliberately unsupported.

Verification

make test
make vet
make status
make logs

The test suite covers specification discovery, path-template matching, deterministic repair generation, challenge comparison, caps, remote signing, paid retry, settlement evidence, semantic evaluation, persistent reports, IPv4/mapped-IPv6 equivalence, NAT64 blocks, and the invariant that inferred schemas contain no example values.

Configuration

Internal API runtime:

VariableDefaultMeaning
PORT8080HTTP listen port
CANARY_REPORT_DIR/data/reportsPersistent report directory
CANARY_MAX_PAYMENT_ATOMIC20000Absolute downstream per-audit cap
CANARY_ALLOWED_NETWORKSbase-sepolia,baseAllowed downstream networks
CANARY_MAX_CONCURRENT4Concurrent audit limit
CANARY_TARGET_TIMEOUT_SECONDS20Per-target timeout
CANARY_ALLOW_HTTPfalseDevelopment-only HTTP target support
CANARY_ALLOW_PRIVATE_TARGETSfalseTest-only private target support; never enable publicly
LITELLM_BASE_URLin-cluster LiteLLMOpenAI-compatible evaluator URL
LITELLM_MASTER_KEYemptyLiteLLM bearer token; missing means semantic evaluation is disabled
CANARY_MODELopenrouter/autoSemantic evaluator model
REMOTE_SIGNER_URLin-cluster signerObol typed-data signer URL
REMOTE_SIGNER_TOKENemptyOptional signer bearer token

Agent publication:

VariableDefaultMeaning
CANARY_AGENT_MODELopenrouter/autoHermes Agent model
CANARY_AGENT_NETWORKbaseInbound x402 payment network
CANARY_AGENT_IDENTITY_NETWORKbaseERC-8004 identity lookup chain; normally match the payment network
CANARY_AGENT_PRICE0.001Per-request USDC price
CANARY_AGENT_HOSTNAMEandrei-obol-agent.dvlabs.devDedicated public Agent origin

LiteLLM authentication comes from the existing llm/litellm-secrets Kubernetes secret. Payment signing uses the existing remote signer; its private key is never mounted into Canary402. The signer address is intentionally cached for the process lifetime, so a coordinated signer-key rotation requires restarting Canary402.

Secrets and source control

Local Obol/OpenRouter and tunnel credentials belong only in .env, which Git and Docker ignore. Never commit wallet private keys, remote-signer credentials, LLM_API_KEY, or the secret-bearing tunnel command.

git status --short --ignored
git check-ignore -v .env

Obol and registration notes

Obol v0.14.0-rc0 natively publishes the dedicated hostname, preserves the external HTTPS scheme in x402 challenges, exposes /.well-known/x402, and creates namespace-qualified cross-namespace grants. Canary402 therefore no longer carries the v0.13 tunnel-route or ReferenceGrant workarounds.

The primary identity is Base Agent ID 59094. Its mint transaction and agentURI correction succeeded. The corrected on-chain URI is https://andrei-obol-agent.dvlabs.dev/.well-known/agent-registration.json.

Legacy Base Sepolia Agent ID 8104 remains owned by the same wallet; its mint transaction also succeeded. Registration is idempotent while x402/AgentIdentity contains the correct per-chain IDs. Do not delete or overwrite that durable record, and pass the origin—not a service path—to obol sell register --origin.

ERC-8004 identity, an active x402 ServiceOffer, and x402scan discovery are separate states. After the v0.14 dedicated-origin migration, obol sell register x402scan idempotently registered 1/1 current resources and deprecated the old shared-origin resource.

About

Evidence-backed reliability audits for x402-paid agent services

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { // Auto-enable theater mode on YouTube (function() { function tryTheater() { var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]'); if (btn && !btn.classList.contains('activated')) { btn.click(); } } // Try immediately tryTheater(); // Try after navigation (SPA) var lastUrl = location.href; setInterval(function() { if (location.href !== lastUrl) { lastUrl = location.href; setTimeout(tryTheater, 500); } }, 1000); // Also try on player load var observer = new MutationObserver(tryTheater); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' GitHub - pinebit/canary402: Evidence-backed reliability audits for x402-paid agent services · GitHub
Skip to content

Repository files navigation

Canary402

Canary402 is a service-contract inspector and mystery shopper for x402-paid services:

Before an agent integrates with or pays a service, Canary402 checks the contract and verifies the delivery.

It inspects public service specifications, generates value-free repair templates, validates the live x402 challenge, optionally makes one strictly budgeted downstream payment through the Obol remote signer, evaluates the delivered result with OpenRouter, and retains an evidence-backed report.

The original concept is preserved in idea.md.

Live Agent

The public product is the Canary402 Agent at:

POST https://andrei-obol-agent.dvlabs.dev/v1/chat/completions

The previous shared-origin path, POST /services/canary402-agent/v1/chat/completions, remains a working compatibility alias.

The Agent is a Hermes/OpenRouter service sold for 0.001 USDC per request on Base mainnet. Its deterministic Go audit API remains deployed inside the Obol cluster and is not sold, advertised, or routed publicly. Calling the Agent therefore incurs one Canary402 charge, not a second internal API charge.

Canary402 is registered as ERC-8004 Agent ID 59094 on Base mainnet, matching its x402 payment network. The same signer also retains the legacy Agent ID 8104 on Base Sepolia. Both registration entries point to the same public Agent and are owned by the Canary402 wallet.

Public discovery documents are available from the dedicated Agent origin:

  • GET https://andrei-obol-agent.dvlabs.dev/openapi.json
  • GET https://andrei-obol-agent.dvlabs.dev/.well-known/x402
  • GET https://andrei-obol-agent.dvlabs.dev/.well-known/agent-registration.json

x402scan crawled the migrated dedicated origin and registered its single current paid resource (1/1), deprecating the earlier shared-origin resource. The listing command warned only that the generated OpenAPI lacks info.contact.email; that field is optional for indexing but enables ownership/contact customization.

The public registration document is active and advertises only the Base Agent identity and endpoint. The legacy Sepolia registration remains owned by the same wallet and retained in Obol's durable AgentIdentity state for historical continuity, but is not advertised by this Base-paid offer.

The old /services/canary402 HTTP offer and its public health/report routes have intentionally been removed. Reports are still retained internally, and the Agent returns the report ID and an evidence-based interpretation in its response.

Buy an Agent request

The endpoint accepts OpenAI-style chat-completions JSON behind an x402 v2 gate. The buyer wallet needs canonical USDC on Base mainnet; the inbound payment itself does not require the buyer to fund a separate Canary402 wallet.

Keep all private keys outside source control:

npm install @x402/fetch @x402/evm viem
import{x402Client,wrapFetchWithPayment}from"@x402/fetch";import{registerExactEvmScheme}from"@x402/evm/exact/client";import{privateKeyToAccount}from"viem/accounts";constsigner=privateKeyToAccount(process.env.EVM_PRIVATE_KEY);constclient=newx402Client();registerExactEvmScheme(client,{ signer });constfetchWithPayment=wrapFetchWithPayment(fetch,client);constresponse=awaitfetchWithPayment("https://andrei-obol-agent.dvlabs.dev/v1/chat/completions",{method: "POST",headers: {"Content-Type": "application/json"},body: JSON.stringify({model: "openrouter/auto",stream: false,messages: [{role: "user",content: "Review the specification and probe https://seller.example.com/weather. Generate repair templates, but do not make a downstream payment.",}],}),},);console.log(response.status,awaitresponse.json());

An unpaid request should return 402 Payment Required. Its challenge should advertise:

  • network eip155:8453 (Base mainnet)
  • canonical Base USDC
  • amount 1000 atomic units (0.001 USDC)
  • the exact HTTPS Agent URL as its resource

The Agent defaults to no downstream payment. Paying to talk to Canary402 does not authorize Canary402 to pay the inspected service.

Ask Canary402 for an audit

A normal request can combine these duties:

  • Specification review inspects bounded copies of /openapi.json, /.well-known/agent-registration.json, and /skill.md, then compares them with the requested operation and live 402 metadata.
  • Repair generation proposes value-free OpenAPI and Bazaar fragments. These are starting points for seller review, not inferred business truth.
  • Probe-only verification validates reachability and the x402 challenge without signing or spending anything downstream.
  • Paid verification makes one budget-capped downstream payment, retries the target, and evaluates the delivered response.

Example prompts:

Review the specification and probe https://seller.example.com/services/weather.
Generate repair templates, but do not make a downstream payment.
Make a real paid audit of https://seller.example.com/service.
Use Base mainnet and do not spend more than 20000 atomic USDC units.

For a real downstream payment, the user must explicitly supply both:

  • payment_network: base or base-sepolia
  • max_payment_atomic: the maximum atomic token amount Canary402 may authorize

For six-decimal USDC, 1000 atomic units is 0.001 USDC. Canary402 also enforces its deployment-wide cap, currently 20000 atomic units (0.02 USDC).

Report meaning

The Agent summarizes the API's report ID, verdict, score, coverage, specification status, findings, repair artifacts, individual checks, and limitations. Reports may contain these checks:

  • service_discovery
  • request_contract
  • reachability
  • x402_challenge
  • payment_budget
  • paid_delivery
  • task_outcome
VerdictMeaning
PROBE_PASSThe payment challenge is valid, but no downstream payment was made
PROBE_PASS_WITH_WARNINGSThe probe worked, but service-contract metadata needs repair
PASSChallenge, paid delivery, and requested outcome passed
PASS_WITH_WARNINGSPaid delivery worked, but some evaluation was unavailable or uncertain
FAILAt least one required check failed

score measures checks that ran. coverage_percent shows how much of the full paid flow was exercised, so a probe-only result must be read with its lower coverage.

Stored reports do not contain purchased response bodies or fetched discovery-document bodies. They retain status, latency, content type, captured byte count, SHA-256 digests, payment terms, settlement evidence when present, and a short evaluator reason. Caller-supplied JSON property names can appear in generated repair fragments, so do not use repair generation for confidential request shapes.

Proven settlement history

Before the current Base-mainnet-only Agent offer was selected, a paid Agent call settled 0.001 Base Sepolia test USDC in transaction 0x8dd8…f91b.

Canary402 has also made a downstream payment of 0.02 Base mainnet USDC while auditing another hackathon service; transaction 0x9eb2…431c confirms the transfer.

These are point-in-time tests, not guarantees about either service.

Run locally

Prerequisites are an Obol Stack v0.14.0-rc0 cluster, Docker, Go 1.25+, and a working LiteLLM/OpenRouter configuration. Install or upgrade the Stack with:

OBOL_RELEASE=v0.14.0-rc0 bash <(curl -s https://stack.obol.org)# For this existing k3d stack, free its current host ports before forced init.
k3d cluster stop obol-stack-peaceful-starling
obol stack init --force --backend k3d --yes
obol stack up

For a new stack with no existing configuration, use obol stack init && obol stack up instead.

make test
make deploy
make smoke

make deploy builds and imports the local image, applies deploy/k8s.yaml, and keeps reports on the canary402-reports PVC. make smoke port-forwards the internal API and verifies its health without publishing it.

Publish or reconcile only the Agent offer:

make sell-agent

This command:

  1. creates or updates the Hermes Agent from agent/objective.md;
  2. applies the Agent's narrow cross-namespace egress policy;
  3. reconciles the x402 price to 0.001 USDC on Base mainnet;
  4. activates the offer against the existing Base Agent ID without minting another identity;
  5. binds the permanent hostname to v0.14's native dedicated-offer route and persists the resume ledger; and
  6. verifies the local offer.

Forks using another hostname should set CANARY_AGENT_HOSTNAME before running make sell-agent.

Direct development requests

The Go API is accessible only inside the cluster or through an operator-created port-forward:

obol kubectl -n llm port-forward service/canary402 18080:8080

Then run a no-spend audit directly:

curl --noproxy '*' -sS http://127.0.0.1:18080/audit \
-H 'Content-Type: application/json' \
--data '{ "url": "https://seller.example.com/services/weather/current", "method": "POST", "body": {"city": "Istanbul"}, "expectation": "The response contains a numeric temperature for Istanbul.", "spec_review": true, "generate_repairs": true, "pay": false }'

A complete request is also available in examples/audit-spec-review.json.

Safety model

Canary402 fails closed:

  • Only public HTTPS targets on port 443 are accepted.
  • Loopback, private, link-local, metadata, benchmark, documentation, carrier-grade NAT, IPv4-mapped IPv6, and standard NAT64 translation ranges are blocked.
  • DNS is checked again at connection time to reduce rebinding risk.
  • Redirects are never followed, especially after payment authorization is attached.
  • Request bodies are JSON and capped at 64 KiB; target responses are capped at 1 MiB.
  • Response bodies and fetched specification bodies are not persisted.
  • Discovery fetches only three fixed same-origin paths, does not follow redirects, and caps each document at 256 KiB.
  • Query strings and credentials are removed from stored report URLs.
  • Client authentication, cookies, arbitrary headers, and payment headers are not forwarded.
  • Downstream payment is opt-in and bounded by both the request cap and CANARY_MAX_PAYMENT_ATOMIC.
  • Target content is treated as untrusted evidence, including in the semantic-evaluation prompt.
  • Objective protocol checks remain separate from the subjective model verdict.

The downstream client supports x402 v2 exact/EIP-3009 with canonical USDC on Base (eip155:8453) and Base Sepolia (eip155:84532). Permit2, arbitrary ERC-20 assets, redirects, asynchronous jobs, custom caller headers, and SSE targets are deliberately unsupported.

Verification

make test
make vet
make status
make logs

The test suite covers specification discovery, path-template matching, deterministic repair generation, challenge comparison, caps, remote signing, paid retry, settlement evidence, semantic evaluation, persistent reports, IPv4/mapped-IPv6 equivalence, NAT64 blocks, and the invariant that inferred schemas contain no example values.

Configuration

Internal API runtime:

VariableDefaultMeaning
PORT8080HTTP listen port
CANARY_REPORT_DIR/data/reportsPersistent report directory
CANARY_MAX_PAYMENT_ATOMIC20000Absolute downstream per-audit cap
CANARY_ALLOWED_NETWORKSbase-sepolia,baseAllowed downstream networks
CANARY_MAX_CONCURRENT4Concurrent audit limit
CANARY_TARGET_TIMEOUT_SECONDS20Per-target timeout
CANARY_ALLOW_HTTPfalseDevelopment-only HTTP target support
CANARY_ALLOW_PRIVATE_TARGETSfalseTest-only private target support; never enable publicly
LITELLM_BASE_URLin-cluster LiteLLMOpenAI-compatible evaluator URL
LITELLM_MASTER_KEYemptyLiteLLM bearer token; missing means semantic evaluation is disabled
CANARY_MODELopenrouter/autoSemantic evaluator model
REMOTE_SIGNER_URLin-cluster signerObol typed-data signer URL
REMOTE_SIGNER_TOKENemptyOptional signer bearer token

Agent publication:

VariableDefaultMeaning
CANARY_AGENT_MODELopenrouter/autoHermes Agent model
CANARY_AGENT_NETWORKbaseInbound x402 payment network
CANARY_AGENT_IDENTITY_NETWORKbaseERC-8004 identity lookup chain; normally match the payment network
CANARY_AGENT_PRICE0.001Per-request USDC price
CANARY_AGENT_HOSTNAMEandrei-obol-agent.dvlabs.devDedicated public Agent origin

LiteLLM authentication comes from the existing llm/litellm-secrets Kubernetes secret. Payment signing uses the existing remote signer; its private key is never mounted into Canary402. The signer address is intentionally cached for the process lifetime, so a coordinated signer-key rotation requires restarting Canary402.

Secrets and source control

Local Obol/OpenRouter and tunnel credentials belong only in .env, which Git and Docker ignore. Never commit wallet private keys, remote-signer credentials, LLM_API_KEY, or the secret-bearing tunnel command.

git status --short --ignored
git check-ignore -v .env

Obol and registration notes

Obol v0.14.0-rc0 natively publishes the dedicated hostname, preserves the external HTTPS scheme in x402 challenges, exposes /.well-known/x402, and creates namespace-qualified cross-namespace grants. Canary402 therefore no longer carries the v0.13 tunnel-route or ReferenceGrant workarounds.

The primary identity is Base Agent ID 59094. Its mint transaction and agentURI correction succeeded. The corrected on-chain URI is https://andrei-obol-agent.dvlabs.dev/.well-known/agent-registration.json.

Legacy Base Sepolia Agent ID 8104 remains owned by the same wallet; its mint transaction also succeeded. Registration is idempotent while x402/AgentIdentity contains the correct per-chain IDs. Do not delete or overwrite that durable record, and pass the origin—not a service path—to obol sell register --origin.

ERC-8004 identity, an active x402 ServiceOffer, and x402scan discovery are separate states. After the v0.14 dedicated-origin migration, obol sell register x402scan idempotently registered 1/1 current resources and deprecated the old shared-origin resource.

About

Evidence-backed reliability audits for x402-paid agent services

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { // Remove or un-stick sticky/fixed headers that block content (function() { function unstick() { document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) { if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') { el.style.position = 'static'; el.style.top = 'auto'; el.style.zIndex = 'auto'; } }); } unstick(); var observer = new MutationObserver(unstick); observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] }); })(); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); })(); GitHub - pinebit/canary402: Evidence-backed reliability audits for x402-paid agent services · GitHub
Skip to content

Repository files navigation

Canary402

Canary402 is a service-contract inspector and mystery shopper for x402-paid services:

Before an agent integrates with or pays a service, Canary402 checks the contract and verifies the delivery.

It inspects public service specifications, generates value-free repair templates, validates the live x402 challenge, optionally makes one strictly budgeted downstream payment through the Obol remote signer, evaluates the delivered result with OpenRouter, and retains an evidence-backed report.

The original concept is preserved in idea.md.

Live Agent

The public product is the Canary402 Agent at:

POST https://andrei-obol-agent.dvlabs.dev/v1/chat/completions

The previous shared-origin path, POST /services/canary402-agent/v1/chat/completions, remains a working compatibility alias.

The Agent is a Hermes/OpenRouter service sold for 0.001 USDC per request on Base mainnet. Its deterministic Go audit API remains deployed inside the Obol cluster and is not sold, advertised, or routed publicly. Calling the Agent therefore incurs one Canary402 charge, not a second internal API charge.

Canary402 is registered as ERC-8004 Agent ID 59094 on Base mainnet, matching its x402 payment network. The same signer also retains the legacy Agent ID 8104 on Base Sepolia. Both registration entries point to the same public Agent and are owned by the Canary402 wallet.

Public discovery documents are available from the dedicated Agent origin:

  • GET https://andrei-obol-agent.dvlabs.dev/openapi.json
  • GET https://andrei-obol-agent.dvlabs.dev/.well-known/x402
  • GET https://andrei-obol-agent.dvlabs.dev/.well-known/agent-registration.json

x402scan crawled the migrated dedicated origin and registered its single current paid resource (1/1), deprecating the earlier shared-origin resource. The listing command warned only that the generated OpenAPI lacks info.contact.email; that field is optional for indexing but enables ownership/contact customization.

The public registration document is active and advertises only the Base Agent identity and endpoint. The legacy Sepolia registration remains owned by the same wallet and retained in Obol's durable AgentIdentity state for historical continuity, but is not advertised by this Base-paid offer.

The old /services/canary402 HTTP offer and its public health/report routes have intentionally been removed. Reports are still retained internally, and the Agent returns the report ID and an evidence-based interpretation in its response.

Buy an Agent request

The endpoint accepts OpenAI-style chat-completions JSON behind an x402 v2 gate. The buyer wallet needs canonical USDC on Base mainnet; the inbound payment itself does not require the buyer to fund a separate Canary402 wallet.

Keep all private keys outside source control:

npm install @x402/fetch @x402/evm viem
import{x402Client,wrapFetchWithPayment}from"@x402/fetch";import{registerExactEvmScheme}from"@x402/evm/exact/client";import{privateKeyToAccount}from"viem/accounts";constsigner=privateKeyToAccount(process.env.EVM_PRIVATE_KEY);constclient=newx402Client();registerExactEvmScheme(client,{ signer });constfetchWithPayment=wrapFetchWithPayment(fetch,client);constresponse=awaitfetchWithPayment("https://andrei-obol-agent.dvlabs.dev/v1/chat/completions",{method: "POST",headers: {"Content-Type": "application/json"},body: JSON.stringify({model: "openrouter/auto",stream: false,messages: [{role: "user",content: "Review the specification and probe https://seller.example.com/weather. Generate repair templates, but do not make a downstream payment.",}],}),},);console.log(response.status,awaitresponse.json());

An unpaid request should return 402 Payment Required. Its challenge should advertise:

  • network eip155:8453 (Base mainnet)
  • canonical Base USDC
  • amount 1000 atomic units (0.001 USDC)
  • the exact HTTPS Agent URL as its resource

The Agent defaults to no downstream payment. Paying to talk to Canary402 does not authorize Canary402 to pay the inspected service.

Ask Canary402 for an audit

A normal request can combine these duties:

  • Specification review inspects bounded copies of /openapi.json, /.well-known/agent-registration.json, and /skill.md, then compares them with the requested operation and live 402 metadata.
  • Repair generation proposes value-free OpenAPI and Bazaar fragments. These are starting points for seller review, not inferred business truth.
  • Probe-only verification validates reachability and the x402 challenge without signing or spending anything downstream.
  • Paid verification makes one budget-capped downstream payment, retries the target, and evaluates the delivered response.

Example prompts:

Review the specification and probe https://seller.example.com/services/weather.
Generate repair templates, but do not make a downstream payment.
Make a real paid audit of https://seller.example.com/service.
Use Base mainnet and do not spend more than 20000 atomic USDC units.

For a real downstream payment, the user must explicitly supply both:

  • payment_network: base or base-sepolia
  • max_payment_atomic: the maximum atomic token amount Canary402 may authorize

For six-decimal USDC, 1000 atomic units is 0.001 USDC. Canary402 also enforces its deployment-wide cap, currently 20000 atomic units (0.02 USDC).

Report meaning

The Agent summarizes the API's report ID, verdict, score, coverage, specification status, findings, repair artifacts, individual checks, and limitations. Reports may contain these checks:

  • service_discovery
  • request_contract
  • reachability
  • x402_challenge
  • payment_budget
  • paid_delivery
  • task_outcome
VerdictMeaning
PROBE_PASSThe payment challenge is valid, but no downstream payment was made
PROBE_PASS_WITH_WARNINGSThe probe worked, but service-contract metadata needs repair
PASSChallenge, paid delivery, and requested outcome passed
PASS_WITH_WARNINGSPaid delivery worked, but some evaluation was unavailable or uncertain
FAILAt least one required check failed

score measures checks that ran. coverage_percent shows how much of the full paid flow was exercised, so a probe-only result must be read with its lower coverage.

Stored reports do not contain purchased response bodies or fetched discovery-document bodies. They retain status, latency, content type, captured byte count, SHA-256 digests, payment terms, settlement evidence when present, and a short evaluator reason. Caller-supplied JSON property names can appear in generated repair fragments, so do not use repair generation for confidential request shapes.

Proven settlement history

Before the current Base-mainnet-only Agent offer was selected, a paid Agent call settled 0.001 Base Sepolia test USDC in transaction 0x8dd8…f91b.

Canary402 has also made a downstream payment of 0.02 Base mainnet USDC while auditing another hackathon service; transaction 0x9eb2…431c confirms the transfer.

These are point-in-time tests, not guarantees about either service.

Run locally

Prerequisites are an Obol Stack v0.14.0-rc0 cluster, Docker, Go 1.25+, and a working LiteLLM/OpenRouter configuration. Install or upgrade the Stack with:

OBOL_RELEASE=v0.14.0-rc0 bash <(curl -s https://stack.obol.org)# For this existing k3d stack, free its current host ports before forced init.
k3d cluster stop obol-stack-peaceful-starling
obol stack init --force --backend k3d --yes
obol stack up

For a new stack with no existing configuration, use obol stack init && obol stack up instead.

make test
make deploy
make smoke

make deploy builds and imports the local image, applies deploy/k8s.yaml, and keeps reports on the canary402-reports PVC. make smoke port-forwards the internal API and verifies its health without publishing it.

Publish or reconcile only the Agent offer:

make sell-agent

This command:

  1. creates or updates the Hermes Agent from agent/objective.md;
  2. applies the Agent's narrow cross-namespace egress policy;
  3. reconciles the x402 price to 0.001 USDC on Base mainnet;
  4. activates the offer against the existing Base Agent ID without minting another identity;
  5. binds the permanent hostname to v0.14's native dedicated-offer route and persists the resume ledger; and
  6. verifies the local offer.

Forks using another hostname should set CANARY_AGENT_HOSTNAME before running make sell-agent.

Direct development requests

The Go API is accessible only inside the cluster or through an operator-created port-forward:

obol kubectl -n llm port-forward service/canary402 18080:8080

Then run a no-spend audit directly:

curl --noproxy '*' -sS http://127.0.0.1:18080/audit \
-H 'Content-Type: application/json' \
--data '{ "url": "https://seller.example.com/services/weather/current", "method": "POST", "body": {"city": "Istanbul"}, "expectation": "The response contains a numeric temperature for Istanbul.", "spec_review": true, "generate_repairs": true, "pay": false }'

A complete request is also available in examples/audit-spec-review.json.

Safety model

Canary402 fails closed:

  • Only public HTTPS targets on port 443 are accepted.
  • Loopback, private, link-local, metadata, benchmark, documentation, carrier-grade NAT, IPv4-mapped IPv6, and standard NAT64 translation ranges are blocked.
  • DNS is checked again at connection time to reduce rebinding risk.
  • Redirects are never followed, especially after payment authorization is attached.
  • Request bodies are JSON and capped at 64 KiB; target responses are capped at 1 MiB.
  • Response bodies and fetched specification bodies are not persisted.
  • Discovery fetches only three fixed same-origin paths, does not follow redirects, and caps each document at 256 KiB.
  • Query strings and credentials are removed from stored report URLs.
  • Client authentication, cookies, arbitrary headers, and payment headers are not forwarded.
  • Downstream payment is opt-in and bounded by both the request cap and CANARY_MAX_PAYMENT_ATOMIC.
  • Target content is treated as untrusted evidence, including in the semantic-evaluation prompt.
  • Objective protocol checks remain separate from the subjective model verdict.

The downstream client supports x402 v2 exact/EIP-3009 with canonical USDC on Base (eip155:8453) and Base Sepolia (eip155:84532). Permit2, arbitrary ERC-20 assets, redirects, asynchronous jobs, custom caller headers, and SSE targets are deliberately unsupported.

Verification

make test
make vet
make status
make logs

The test suite covers specification discovery, path-template matching, deterministic repair generation, challenge comparison, caps, remote signing, paid retry, settlement evidence, semantic evaluation, persistent reports, IPv4/mapped-IPv6 equivalence, NAT64 blocks, and the invariant that inferred schemas contain no example values.

Configuration

Internal API runtime:

VariableDefaultMeaning
PORT8080HTTP listen port
CANARY_REPORT_DIR/data/reportsPersistent report directory
CANARY_MAX_PAYMENT_ATOMIC20000Absolute downstream per-audit cap
CANARY_ALLOWED_NETWORKSbase-sepolia,baseAllowed downstream networks
CANARY_MAX_CONCURRENT4Concurrent audit limit
CANARY_TARGET_TIMEOUT_SECONDS20Per-target timeout
CANARY_ALLOW_HTTPfalseDevelopment-only HTTP target support
CANARY_ALLOW_PRIVATE_TARGETSfalseTest-only private target support; never enable publicly
LITELLM_BASE_URLin-cluster LiteLLMOpenAI-compatible evaluator URL
LITELLM_MASTER_KEYemptyLiteLLM bearer token; missing means semantic evaluation is disabled
CANARY_MODELopenrouter/autoSemantic evaluator model
REMOTE_SIGNER_URLin-cluster signerObol typed-data signer URL
REMOTE_SIGNER_TOKENemptyOptional signer bearer token

Agent publication:

VariableDefaultMeaning
CANARY_AGENT_MODELopenrouter/autoHermes Agent model
CANARY_AGENT_NETWORKbaseInbound x402 payment network
CANARY_AGENT_IDENTITY_NETWORKbaseERC-8004 identity lookup chain; normally match the payment network
CANARY_AGENT_PRICE0.001Per-request USDC price
CANARY_AGENT_HOSTNAMEandrei-obol-agent.dvlabs.devDedicated public Agent origin

LiteLLM authentication comes from the existing llm/litellm-secrets Kubernetes secret. Payment signing uses the existing remote signer; its private key is never mounted into Canary402. The signer address is intentionally cached for the process lifetime, so a coordinated signer-key rotation requires restarting Canary402.

Secrets and source control

Local Obol/OpenRouter and tunnel credentials belong only in .env, which Git and Docker ignore. Never commit wallet private keys, remote-signer credentials, LLM_API_KEY, or the secret-bearing tunnel command.

git status --short --ignored
git check-ignore -v .env

Obol and registration notes

Obol v0.14.0-rc0 natively publishes the dedicated hostname, preserves the external HTTPS scheme in x402 challenges, exposes /.well-known/x402, and creates namespace-qualified cross-namespace grants. Canary402 therefore no longer carries the v0.13 tunnel-route or ReferenceGrant workarounds.

The primary identity is Base Agent ID 59094. Its mint transaction and agentURI correction succeeded. The corrected on-chain URI is https://andrei-obol-agent.dvlabs.dev/.well-known/agent-registration.json.

Legacy Base Sepolia Agent ID 8104 remains owned by the same wallet; its mint transaction also succeeded. Registration is idempotent while x402/AgentIdentity contains the correct per-chain IDs. Do not delete or overwrite that durable record, and pass the origin—not a service path—to obol sell register --origin.

ERC-8004 identity, an active x402 ServiceOffer, and x402scan discovery are separate states. After the v0.14 dedicated-origin migration, obol sell register x402scan idempotently registered 1/1 current resources and deprecated the old shared-origin resource.

About

Evidence-backed reliability audits for x402-paid agent services

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages