Skip to content

hermes log forwarding adapter (LLP 0118-0125) - #339

Merged
philcunliffe merged 12 commits into
masterfrom
integration/hermes-log-forwarding
Jul 21, 2026
Merged

hermes log forwarding adapter (LLP 0118-0125)#339
philcunliffe merged 12 commits into
masterfrom
integration/hermes-log-forwarding

Conversation

@philcunliffe

Copy link
Copy Markdown
Contributor

Implements the @hypaware/hermes log-forwarding adapter designed in the LLP 0118-0125 cluster (spec 0118, design 0122, plan 0123).

Tasks (all verified-merged, --no-ff)

  • T1 engines bump to >=22.12 + read-only state.db reader (node:sqlite)
  • T2 projector to AiGatewayProjectedExchange (deterministic ids, usage-policy skip, session-end part)
  • T3 backfill provider (--since windowing)
  • T4 poll source with per-session watermarks
  • T5 plugin wiring (manifest/config/activate, workspace registration)
  • T6 hermetic smoke (hermes_backfill_roundtrip)

Neutral pipeline: design-first cluster merged to master (#331), seeded integration/hermes-log-forwarding, implemented by the neutral wave-loop across 6 dependency-ordered tasks. Held for review.

@ref LLP 0122 [implements]

Change-Set: hermes-log-forwarding

neutral-loopand others added 12 commits July 20, 2026 22:19
…reader
Repo-wide `engines` floor moves to >=22.12 and CI pins a setup-node matrix
of Node 22 and 24 (LLP 0125), so node:sqlite is guaranteed present for
every later hermes task's tests.
Adds the read-only state.db reader for the @hypaware/hermes adapter
(LLP 0119, LLP 0122#sqlite): node:sqlite DatabaseSync opened
{ readOnly: true }, a one-line activation probe that turns a missing
builtin into a clear HermesStateDbError refusal instead of a crash, a
bounded SQLITE_BUSY/SQLITE_LOCKED retry, and queries for sessions,
messages-by-session, and the changed-session aggregate that the poll
model (T4) will re-project from (LLP 0122#watermark). Row and watermark
shapes live in src/types.d.ts.
Tests build a small state.db fixture with node:sqlite mirroring the
hermes schema (sessions + messages incl. tool calls, reasoning, NULL
cwd) and cover readonly-open enforcement, the activation probe, bounded
busy retry (unit + through the reader), changed-session detection, and
the missing-file probe.
Task-Id: T1
Add the hermes projector (src/projector.js) that turns full
HermesStateDb sessions into AiGatewayProjectedExchange items via
projectedExchangeItem: deterministic message_id/part_id minted from
(session id, message id, part index), session_id namespaced
hermes-<id>, provider normalized from billing/base_url fields, LLP
0035 usage mapping, tool-call and reasoning part expansion, and
deriveRepoFromCwd repo enrichment for interactive sessions.
Channel sessions (telegram, discord, slack, whatsapp, signal, email)
are stamped with the canonical policy scope path
~/.hermes/channels/<source>, with the real daemon cwd preserved in
attributes (LLP 0124). The shared usage-policy resolver is consulted
against the effective scope before any row is built, so ignored
sessions are skipped up front (LLP 0050). A synthetic session-end
part is appended once ended_at is set, with a stable id derived from
(session id, session_end), carrying end_reason, final totals, and
costs (LLP 0122#session-end-part).
Add src/git_repo.js (deriveRepoFromCwd, redactRemoteUserinfo),
mirroring the claude/codex adapters' git-shell-out convention: remote
+ toplevel only, never HEAD, since a derived sha would be
anachronistic for historical sessions.
Tests: golden projection of the T1 fixture (open + ended sessions),
id determinism across re-runs, ignored-cwd session skipped before any
row is built, channel session stamped and governed by a marked
channel scope, NULL-cwd interactive session recorded unconditionally,
session-end part present exactly once for an ended session and absent
for an open one, plus unit coverage for deriveRepoFromCwd and
provider normalization.
Task-Id: T2
Add the @hypaware/hermes backfill provider (src/backfill.js):
ctx.backfills.register contribution, --since/--until windowing via the
shared resolveWindow/filterByWindow (windowing selects sessions by
started_at, not individual messages, so a session's identity always
comes from its full row set), one ai_gateway.projected_exchange item
per session via projectHermesSession (T2), and provenance carrying the
state.db path. A missing state.db degrades to a clean no-op (spec R9)
rather than an error, matching the claude/codex backfills.
Tests cover window filtering (since/until/no-window), an empty store
and a missing store as no-ops, provenance shape, the delegated
usage-policy skip, and rerun determinism.
Task-Id: T3
Add startHermesSource: idle mode when ~/.hermes/state.db is missing
(spec R9), a per-tick poll loop that finds changed sessions against a
persisted watermark (LLP 0122#watermark), re-projects each changed
session whole through projectHermesSession, and appends through the
shared ai_gateway.projected_exchange materializer so its pre-write
part_id dedupe turns a whole-session re-projection into "append only
the new tail". The watermark ({ max_message_id, ended_at } per
session) persists as a sidecar JSON in the plugin's kernel-managed
state dir. status()/reload()/stop() round out the StartedSource
lifecycle, with a hermes.poll span and structured logs per spec R7.
Tests cover the missing-store idle path, watermark advance and
persistence, dedupe-reliant re-projection writing only the new tail,
an ended_at transition with no new messages still landing the
session-end part, and a clean idempotent stop().
Task-Id: T4
Adds hypaware.plugin.json for @hypaware/hermes (requires.plugins maps
@hypaware/ai-gateway as a hard dependency, contributes the [hermes]
config_section and the hermes source, no datasets of its own since
rows land in ai_gateway_messages via the shared materializer).
src/config.js validates the [hermes] section (enabled, state_db,
poll_interval), reusing DURATION_RE from T4's source.js. src/index.js
activate() registers the config section, the hermes poll source, and
the hermes backfill provider, and starts the poll source unless
enabled=false (a static kill switch that only gates ongoing-poll
auto-start, not backfill or the source's own idle-on-missing-state.db
behavior).
Registers @hypaware/hermes in V1_BUNDLED_PLUGIN_ALLOWLIST beside
claude/codex so it default-activates like the other adapters, and
bumps cli_bundled_plugins_activated's expected plugins_skipped count
from 5 to 6 to account for the newly bundled-but-unconfigured plugin.
Adds manifest and config validation tests under test/plugins/.
Task-Id: T5
Add hermes_backfill_roundtrip.js: boots @hypaware/ai-gateway and
@hypaware/hermes against a fixture state.db under a tmp HYP_HOME (the
live poll source disabled so hyp backfill hermes is the sole writer),
then asserts the LLP 0123 T6 contract end to end - included-session
rows land in ai_gateway_messages with client_name/conversation_source
'hermes', a .hypignore-scoped session is dropped at the capture seam
(spec R3), a second backfill run writes zero new rows on the same
deterministic ids (spec R2), and the internal telemetry proving the
hermes-specific path ran (backfill.provider_finish/backfill.write
spans tagged provider: 'hermes', the hermes.backfill.scan_complete
log, and the component: 'hermes' usage_policy_drop log) is present.
Task-Id: T6
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@philcunliffe

Copy link
Copy Markdown
ContributorAuthor

Neutral review - @hypaware/hermes log-forwarding adapter (T1-T6)

Verdict: clean. No actionable correctness, determinism, spec-adherence, or style defect. The change set is purely additive at the plugin seam and faithfully realizes LLP 0118 / 0119 / 0120 / 0121 / 0122 / 0124 / 0125. Nothing pushed.

Reviewed head 57278bc in an isolated worktree.

What I verified

  • Determinism (spec R2).mintHermesMessageId(sessionId, messageId, partIndex) -> hermes-<s>-<m>-<p> and mintHermesSessionEndId -> hermes-<s>-session_end are pure functions of hermes's own stable integer keys; with all-digit segments and a fixed 3-dash shape they cannot alias, and session_end never collides with a numeric part. Whole-session re-projection every time keeps message_index/previous_message_id/part_id independent of observation order. The idempotency guarantee ultimately rests on the shared aiGatewayBackfillMaterializer pre-write part_id dedupe (ai-gateway/src/dataset.js), which scans all partitions of ai_gateway_messages (scanExistingPartIds -> discoverCachePartitions({datasets:[...]})), so poll-written (hermes segment) and backfill-written rows dedupe against each other. Empirically: the smoke's second hyp backfill hermes writes rows_written === 0.
  • Projection. Provider normalization (base_url host map, fallback to billing_provider, never hermes), tool-call/reasoning/text part expansion with one-carrier usage on the last part (@ref LLP 0035#one-carrier), and the synthetic session-end part emitted exactly once when ended_at is set and absent while open (verified in hermes-projector.test.js).
  • Watermark/source (R7/R9). Missing-store idle mode (logged once, re-probed each tick, status() reports it), per-session {max_message_id, ended_at} persisted atomically, changed-sessions-only re-projection, ended_at NULL->set re-trigger with no new messages, stop() clears the timer and closes the db, hermes.poll span with component: 'hermes'.
  • Usage-policy skip (R3, LLP 0050). Resolved over the effective scope (real cwd, or the ~/.hermes/channels/<source> path for channel sessions) before any row is built; the smoke drops a real .hypignore session via the production resolver (not a test stub).
  • Channel scope (R10, LLP 0124).cwd stamped ~/.hermes/channels/<source>, real daemon cwd preserved in attributes.hermes.real_cwd.
  • sqlite reader (R5, LLP 0119/0122#sqlite).DatabaseSync { readOnly: true }, activation probe -> typed sqlite_unavailable refusal instead of a crash, bounded SQLITE_BUSY/LOCKED retry -> typed sqlite_busy that degrades status.
  • Wiring (LLP 0121). Manifest requires @hypaware/ai-gateway, contributes config_sections+sources, no datasets; registered in V1_BUNDLED_PLUGIN_ALLOWLIST beside claude/codex; the only non-hermes edits are the bundled list, the cli_bundled_plugins_activated skip count (5->6), the CI node 22/24 matrix, and the >=22.12 engines bump. Nothing touches ai-gateway, the cache schema, settlement, or any sink.
  • Smoke (T6). Asserts rows land with client_name='hermes'/provider='openai', zero new rows on rerun, the ignored-cwd session absent, and the internal telemetry (backfill.provider_finish/backfill.write/hermes.backfill.scan_complete/plugin.hermes.usage_policy_drop). Stable DEV_RUN_ID/smoke_name/smoke_step; discovered by filename.
  • Style (CLAUDE.md). No semicolons, no em dashes, JSDoc-only types, no inline import('...') types, root-anchored .js type-import specifiers. All @ref anchors resolve (checked 0030#decision, 0032#remote-redaction/#repo-commit-nodes, 0035#one-carrier, 0120#row-semantics, 0122 anchors, 0123#tasks).

Local results

  • node --test test/plugins/hermes-*.test.js: 62/62 pass.
  • npm test: 2405 pass, 7 fail - all 7 are the pre-existing leave-command.test.js failures unrelated to this PR.
  • npm run smoke -- hermes_backfill_roundtrip: ok.
  • npm run typecheck: clean.

Non-blocking notes for the maintainer (no change made)

  1. Session token totals are invisible to the canonical usage report (design tradeoff, not a code defect). Hermes's messages table carries only a single token_count per row, so per-message usage projects as { total_tokens } only - never the LLP 0035 input_tokens/output_tokens/cache_*/reasoning_tokens split. That split exists only at session level and rides the synthetic session-end part, which is deliberately role: 'system' (projector.js sessionEndMessage). The flagship usage report SQL sums ... WHERE role='assistant' AND attributes.usage IS NOT NULL (see hypaware-ai-usage-report/SKILL.md), so hermes sessions report 0 input/output/cache/reasoning tokens there, open or ended. LLP 0122#session-end-part reasons about this choice explicitly (role='system' to avoid double-counting a SUM over assistant rows), and the code matches that accepted design - so this is a design-level call, not a code bug, and I did not alter it. Worth deciding whether the flagship "hermes alongside Claude/Codex" motivation (LLP 0118) is met when the standard report shows zero token cost for hermes; a follow-up could either land the session totals on the last assistant row (safe: hermes assistant rows carry no input_tokens, so no double count) or teach the usage report to read the end part.
  2. reload() with a changed state_db path can bleed a stale watermark (low, edge).source.js reload sets runner.watermark = {} when the path changes, but the next tryOpen immediately re-reads readHermesWatermark(runner.stateDir) - and the watermark sidecar is keyed by stateDir, not by db path. Since hermes session ids are store-scoped integers starting at 1, the previous db's marks would suppress re-projection of the new db's low-id sessions. This only bites the profile-switching path that LLP 0122 open-question [codex] Remove OpenTelemetry npm dependencies #2 already defers as future work, so it is a latent limitation rather than a regression.

@philcunliffe
philcunliffe marked this pull request as ready for review July 21, 2026 00:55
@philcunliffephilcunliffe added the neutral:approved neutral reviewed this and holds it for a maintainer merge (own or adopted PR; LLP 0025/0030) label Jul 21, 2026
@philcunliffe
philcunliffe merged commit cb2a344 into masterJul 21, 2026
8 checks passed
@philcunliffe
philcunliffe deleted the integration/hermes-log-forwarding branch July 21, 2026 01:54
philcunliffe added a commit that referenced this pull request Jul 21, 2026
The hermes log-forwarding change set (LLP 0118-0125) shipped in PR #339, but
the design-first flip to Active (LLP 0016 Shipped-is-Active) did not ride the
implementation PR, so neutral implementable kept re-surfacing the design as
owed-implementation. This lifecycle move marks it shipped.
Co-authored-by: neutral-loop <neutral-loop@localhost>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

neutral:approvedneutral reviewed this and holds it for a maintainer merge (own or adopted PR; LLP 0025/0030)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@philcunliffe