Skip to content

feat: Perfetto 风格的本地缓存链接(local_cache_key) - #65

Open
JavaZeroo wants to merge 3 commits into
masterfrom
cc/peaceful-edison-jbanws
Open

feat: Perfetto 风格的本地缓存链接(local_cache_key)#65
JavaZeroo wants to merge 3 commits into
masterfrom
cc/peaceful-edison-jbanws

Conversation

@JavaZeroo

Copy link
Copy Markdown
Owner

背景

ui.perfetto.dev 把打开过的 trace 存在浏览器里、用一个 UUID 索引它,并把 UUID 写进 URL:

https://ui.perfetto.dev/#!/viewer?local_cache_key=00000000-0000-0000-1343-6d0de2416d47

刷新、收藏、开第二个标签页都能回到同一份 trace,不用重新上传。这个 PR 把本地这一半搬过来给日志分析用。跨机分享(Perfetto 靠上传 GCS 换短链)不做 —— 那会让日志离开浏览器,和项目"数据不上传"的定位冲突。

做了什么

每次分析(文件内容 + 解析出的 metrics + 产生它们的解析配置)快照进 IndexedDB,地址栏自动变成 #!/viewer?local_cache_key=<uuid>

  • 刷新 / 收藏 / 第二个标签页打开该链接 → 同一份日志和图表直接回来
  • 不带 hash 打开 → 恢复最近一次分析(原来"刷新后文件还在"的行为不变)
  • 链接在别的机器打开 → 提示"该链接对应的缓存不在此浏览器中",不会白屏
  • 侧边栏新增「缓存」入口:新建、打开、重命名、复制链接、删除、清空

顺带一个收益:旧逻辑对 >5MB 的文件会丢掉 content 只留 metricsData(标记 needsReupload),缓存快照把上限提到 64MB,大日志现在也能整份存下来。

改动

文件作用
src/utils/idb.js共享的 IndexedDB open/upgrade。fileStorage 和缓存同库,版本必须统一,否则老的 open(v1) 会 VersionError
src/utils/sessionCache.js快照存储 + 轻量摘要索引(列缓存时不反序列化日志正文);LRU 淘汰:30 天 / 20 条 / 192MB;配额超限时先清后重试一次
src/utils/cacheUrl.jskey 的解析、校验、生成,以及 history.replaceState/pushState
src/components/SessionsModal.jsx缓存列表 UI
src/App.jsx挂载时按 URL key → 最近快照 → 旧 working set 的顺序恢复;防抖 600ms 写入;解析中不写(避免快照到空 metrics)

第二个 commit 是本地体验的三处修补:

  1. 切换分析会丢最后一次编辑(真 bug)。写入是防抖的,而打开另一份缓存会让 effect 清掉定时器 —— 改完解析配置立刻切走,这次改动就没了。现在切换前先 flush。
  2. 缓存列表几乎永远只有一条。之前想要空白画布只能点「清空文件」,而那会连快照一起删掉。新增「新建分析(保留当前缓存)」;「清空文件」维持原来的销毁语义不变。
  3. 地址栏自己变了需要解释:首次生成 key 时给一次性提示;列表里的名字可点击重命名(默认的 train.log +2 攒几条以后认不出来)。

设计取舍

  • 用 hash 而不是 query string:纯客户端路由,GitHub Pages / Vercel / vite preview 都不用改 rewrite 规则。
  • 两个 storesessions 存完整 payload,sessionIndex 只存摘要,所以列缓存时不会把几十 MB 日志正文读进内存。
  • 必须淘汰:一直涨到浏览器把整个 origin 的存储清掉,比自己按 LRU 忘掉更糟。
  • 多标签页写同一个 key 是后写覆盖先写,没做跨标签页加锁(Perfetto 也有同类问题),觉得需要再单开一个改动。

验证

  • npm run lint 干净(只剩既有的 react-refresh warning)
  • npm run test:24 文件 / 147 用例全过。新增覆盖 URL helper、fake-indexeddb 下的存储层(含 LRU 淘汰、超大快照降级、重命名)、以及 App 级流程(生成 key → 刷新恢复 → 未知 key 告警 → 新建分析后两份快照并存)
  • 另外用 Chromium 跑了 build 产物真实链路:上传 → 地址栏生成 key → 刷新恢复图表 → 第二标签页同链接恢复 → 新建分析 → 重命名 → 切回第一份 → 刷新仍停在第一份且第二份还在缓存里

新增一个 devDependency:fake-indexeddb(仅测试用,jsdom 没有 IndexedDB)。


Generated by Claude Code

ui.perfetto.dev keeps every opened trace in browser storage under a UUID
and puts that UUID in the URL, so a reload or a bookmark reopens the same
trace without re-uploading it. Do the same for log analyses.
Each analysis (files + parsed metrics + the parsing config that produced
them) is snapshotted to IndexedDB under a cache key, and the key is
mirrored in the hash as `#!/viewer?local_cache_key=<uuid>`. Reload, a
bookmark, a second tab or the new cache list all reopen it; the bare URL
still restores the most recent analysis, so nothing regresses for users
who never touch a link. Links are local to the browser by design — the
modal and the miss-toast say so — and nothing is uploaded anywhere.
- idb.js: one shared DB open/upgrade, since fileStorage and the session
cache live in the same database and must agree on its version
- sessionCache.js: snapshot store + lightweight summary index (listing
the cache never deserializes log text), LRU eviction by age (30d),
count (20) and bytes (192MB), and quota-exceeded recovery
- cacheUrl.js: key parse/build/validate + history rewriting
- SessionsModal.jsx: cached analyses with open / copy link / delete
- Restore order on mount: URL key → most recent snapshot → legacy
working set (adopted into a cache entry on its first save)
Tests cover the URL helpers, the store against fake-indexeddb, and the
App-level flow (mint a key, restore from it, warn on an unknown link).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0167bNHXjLvWKocE6AXUJz3A
Three gaps in the cache as first shipped, all of them local-experience:
- Switching analyses cancelled the 600ms debounced write, so an edit made
just before opening another snapshot was silently dropped. The pending
payload is now flushed before the switch.
- The only route to an empty canvas was "clear all", which deletes the
current snapshot — so the cache could never hold more than one entry and
the list had nothing to list. Added "New analysis", which keeps the
current snapshot and mints a fresh key on the next upload.
- The URL changing on its own needed explaining: a one-time toast after
the first key is minted, plus inline rename in the list (renameSession
existed but nothing called it) so a history of analyses stays readable.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0167bNHXjLvWKocE6AXUJz3A
@github-actions

github-actionsBot commented Aug 29, 2026

Copy link
Copy Markdown
PR Preview Action v1.8.1

QR code for preview link

🚀 View preview at
https://log.javazero.top/pr-preview/pr-65/

Built to branch gh-pages at 2026-08-29 10:19 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

@chatgpt-codex-connectorchatgpt-codex-connectorBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit:fabae606cb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment threadsrc/App.jsx
Comment on lines +414 to +416
} else {
console.warn('Failed to cache analysis', err);
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Fall back when cache writes fail

When indexedDB exists but openDB()/the cache write rejects for a non-quota reason, such as a blocked/private IndexedDB or a version error, this branch only logs and the effect never reaches the legacy saveFilesToStorage serialization below. The previous persistence layer fell back to localStorage for exactly these contexts, so small logs in those browsers now silently stop persisting across refreshes; set the cache path disabled and save through the legacy path, or otherwise call the fallback here.

Useful? React with 👍 / 👎.

Comment threadsrc/App.jsx
sessionIdRef.current = null;
setCacheKey(null);
clearCacheKey();
deleteCachedSession(staleId).catch(() => { /* best effort */ });

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Clear the legacy working set when files are removed

For upgraded browsers that still have the pre-cache files/all IndexedDB working-set record, this empty-files path deletes only the current session. After the user clicks the FilesPanel clear-all action, the URL is cleared and there are no sessions left, so the next bare reload falls through to loadWorkingSet() and resurrects the supposedly deleted old logs; clear fileStorage in this branch as well.

Useful? React with 👍 / 👎.

Measured in Chromium against the previous layout, which held the raw logs
and every parsed point inside one session record:
20MB / 10 files: first save 718ms, re-save on any state change 752ms
200MB / 10 files: first save 1503ms, re-save 1532ms
...and the raw logs were silently dropped at 64MB
Every checkbox toggle, range edit or metric rename rewrote the whole
snapshot, so the cost scaled with everything cached rather than with what
changed. The 64MB per-session ceiling also degraded all-or-nothing: one
byte over and every file lost its content, which 30 x 2MB already tripped
because parsed points counted toward it.
Now content and metrics live in their own records, one per file, written
only when they actually change; the session record holds names, enabled
flags, configs and references. Same machine, same scenarios:
20MB / 10 files: first save 177ms, toggle 3ms, restore 73ms
60MB / 30 files: first save 505ms, toggle 4ms, restore 190ms
200MB / 10 files: first save 2728ms, toggle 5ms, restore 1295ms
...with all raw logs kept
Also:
- Points pack into Float64Array pairs rather than {x, y} objects: 1M
points write in 106ms instead of 779ms, read in 191ms instead of 931ms.
- Content is stored as UTF-8 bytes taken from the text already in memory,
not as the original File. A File is a reference to the file on disk, so
the next training run overwriting train.log would leave the cache
serving content that no longer matches the metrics parsed from it.
- The byte ceiling is now min(2GB, half the origin's quota) instead of a
flat 192MB, and storage.persist() is requested so the browser is less
likely to evict us.
- Out of space: evict other analyses first, then keep the parsed series
and drop the raw logs, so charts still render.
The lazy-encode rule has its own test: encoding content before deciding
whether it needs writing costs exactly as much as writing it, and put
seconds back into every toggle once already.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0167bNHXjLvWKocE6AXUJz3A
@JavaZerooClaude

Copy link
Copy Markdown
OwnerAuthor

补一个 commit(4fb1e1a):按"经常上传大量日志文件"的场景实测后,重做了存储布局。评审时请以这个 commit 后的代码为准,前两个 commit 的存储方案在这个量级下站不住。

原方案的问题(Chromium 实测)

原来一次分析的所有内容(原文 + 所有解析点)塞在一条 session 记录里:

场景首次保存任意状态变化后重写原文是否保留
20MB / 10 文件718ms752ms
60MB / 30 文件586ms202ms
200MB / 10 文件1503ms1532ms

两个问题:

  1. 勾选框一点就重写整份快照。成本随"缓存了多少"增长,而不是随"改了什么"增长。勾一下文件、改个坐标范围、重命名指标,都要重写几百 MB。
  2. 64MB 上限是全有全无的降级。超一个字节,所有文件的原文一起丢掉(全部标记需要重新上传)。而且解析点也算进这个额度(40 字节/点),所以 30×2MB 就已经越线了 —— 上表第二行的 202ms 正是原文被丢掉后的假象。

改法

内容和解析结果各自独占记录,按文件分开,只在真正变化时才写;session 记录只留文件名、启用状态、配置和引用。

场景首次保存勾选框重写恢复原文
20MB / 10 文件177ms3ms73ms全部保留
60MB / 30 文件505ms4ms190ms全部保留
200MB / 10 文件2728ms5ms1295ms全部保留

交互期的写入变成与数据量无关的常数。首次保存仍随数据量增长,但那一次本来就和 FileReader 读取 + worker 解析同一量级,而且是防抖异步的。

其他几处:

  • 解析点打包成 Float64Array 对,不再存 {x, y} 对象数组:100 万点写入 779ms → 106ms,读取 931ms → 191ms。点本身就是纯数字(见 logParser.worker.js),无损。
  • 原文存内存里那份文本的 UTF-8 字节,而不是原始 File 对象。存 File 几乎零成本,但它是磁盘文件的引用 —— 下一次训练把 train.log 覆盖掉,缓存读回来的就和当初解析出的 metrics 对不上了,甚至直接 NotReadableError。这个便宜不能占。
  • 容量上限从固定 192MB 改成 min(2GB, 配额的一半),并 best-effort 申请 navigator.storage.persist()
  • 空间不够时的降级:先淘汰其他分析;仍不够才丢原文、保留解析结果(图表照常渲染,文件标记为需重新上传),不再是一刀切。

一个容易踩回去的点

"先编码再判断要不要写"和"直接写"成本一样 —— 我自己就先踩了一次,勾选框重写从 4ms 回到 1763ms。所以 planWrites 里先判断复用、后编码,并且专门有一个测试盯着它(spy TextEncoder.prototype.encode,断言未被调用)。

测试 155 个全过;另外用 Chromium 对 build 产物跑了完整链路,确认刷新后原文确实回来了(不出现"需要重新上传"标记)。


Generated by Claude Code

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@JavaZeroo@claude