feat: add jsonwrite and jsonparse builtins

[ddehilster]

Serializing the knowledge base is where a KB analyzer actually spends its time. Measured on date-time, its output.nlp — thirteen lines that call JsonKB() and SaveKB() — was 43.9% of total runtime, all of it inside the 692-line interpreted KBFuncs.nlp.

I tried the cheap fix first: memoizing the indent helper in NLP++ bought 4.6%. The cost isn't one hot helper, it's interpreted execution itself (~73µs per emitted line). C++ is the only lever.

jsonwrite(file, concept)	serialize a concept tree to JSON
jsonparse(json_str, parent)	parse JSON into concepts under parent

jsonwrite — a drop-in for JsonKB

Same argument order, byte-identical output, so KBFuncs.nlp's JsonKB becomes a one-line wrapper and an analyzer gets the speedup without touching its output pass:

JsonKB(L("file"),L("con")) {
jsonwrite(L("file"),L("con"));
}

date-time at ×30 input, three runs each:

	JsonKB (NLP++)	jsonwrite (C++)
output pass	1.44–1.61s	0.41–0.55s	~3.1×
total run	1.62–1.81s	0.59–0.81s	~2.6×

Byte-identical verified at 10,537 bytes (single) and 309,457 bytes (×30).

Two quirks reproduced deliberately, both commented at their definitions:

• SpacesStr(n) emits max(1,n) indents, not n — so SpacesStr(0) is two spaces.
• SeparateConCount's early exit is a no-op statement, so its backward digit scan never stops: a1b2 splits as base a1 / count 12.

Fixing either would silently change existing analyzers' output. They should be fixed together with their NLP++ originals, as their own change.

jsonparse — the inverse

Follows the mapping python/json2kbb.py documents. Doing it in the engine removes the Python dependency, which matters for the npm and pypi distributions — a python pass shells out to an interpreter neither bundles. It also removes that script's stale intermediate: it won't overwrite an existing .kbb, so editing the .json silently does nothing until you delete it.

Pairs with 3.8.0's readfile:

jsonparse(readfile(G("$apppath") + "/data/config.json"), G("kb"))

Handles nested objects/arrays, all string escapes including \uXXXX with surrogate-pair combining, int/float discrimination, and true/false/null. Requires a top-level object and rejects trailing garbage, so a truncated document is reported rather than half-loaded.

The round-trip fixture earned its keep

It caught a real bug: jsonwrite synthesizes "id" from a counted concept's name suffix (item1 → "id":"1"), and jsonparse was storing that as a genuine attribute — so a second write emitted it twice. jsonparse now drops "id" inside array elements, where the name already carries it. The fixture asserts the count stays at 2, not 4.

Scope

Analyzers carry their own copy of KBFuncs.nlp (13 in the analyzers repo, 21 bundled in the extension), and those copies already differ from the shared template. So rewiring the template gives newly created analyzers the speedup automatically; an existing analyzer gets it when its copy is refreshed. This is not a silent global upgrade, and I corrected an earlier claim in this description that implied it was.

Note on sequencing

The KBFuncs.nlp rewiring is a separate PR in visualtext-files. It must not ship before this engine change is released, or analyzers on an older engine would call a builtin that doesn't exist.

🤖 Generated with Claude Code