Skip to content

Document dGPredictor KEGG carry-over defect in the NAR 2026 Methods drafts - #286

Open
freiburgermsu wants to merge 2 commits into
ModelSEED:devfrom
freiburgermsu:methods-dgpredictor-kegg-carryover
Open

Document dGPredictor KEGG carry-over defect in the NAR 2026 Methods drafts#286
freiburgermsu wants to merge 2 commits into
ModelSEED:devfrom
freiburgermsu:methods-dgpredictor-kegg-carryover

Conversation

@freiburgermsu

Copy link
Copy Markdown
Member

Summary

Expands the multi-source thermodynamics Methods draft (Papers/NAR_Update_2026/drafts/methods_multi_source_thermodynamics.md) and its inlined copy in MANUSCRIPT.md with the discovery and correction of a systematic identifier carry-over defect in the original (KEGG-mediated) dGPredictor integration.

The finding being documented

  • Of the 27,715 reactions carrying a stored dGPredictor record, 17,271 (62%) hold an estimate computed for a KEGG reaction id that is not among the reaction's aliases (the reaction records and Unique_ModelSEED_Reaction_Aliases.txt agree on all 17,271 cases).
  • Root cause: a stale-variable bug in the 2023 staging notebook — the KEGG-alias extraction loop never reset its holding variable between reactions, so every KEGG-less reaction silently inherited the id of the nearest preceding KEGG-bearing reaction (e.g. rxn00019, a nitroalkane oxidase, carries RuBisCO's R00024 energy of −8.63 ± 0.04 kcal/mol; rxn13478–rxn13481 all inherited R09245 from rxn13477).
  • A database-wide fingerprint test explains 100.0% of the 17,784 contaminated staged entries by carry-over, with zero exceptions.
  • The contaminated subset shows null correlation with eQuilibrator and Group Contribution (r = −0.001 / −0.006) versus r = 0.61 / 0.74 for the correctly-mapped subset — explaining the method's previously puzzling low agreement with other sources.
  • Because Update_Reaction_dGPredictor_Energies.py never validates staged KEGG ids against aliases, every Rerun_Thermodynamics.sh pass re-propagated the contamination, and the promotion policy's lowest-uncertainty rule favored the deceptively tight copied errors: 7,466 contaminated values were promoted into canonical deltag/deltagerr/reversibility.

What this PR changes

Documentation only — no data or script changes:

  • New Methods subsection (defect finding → forensic confirmation → why it persisted → correction path), styled to mirror the existing GC Convention-A subsection.
  • The retrained dGPredictor-ModelSEED source is noted as structurally immune (keyed directly by ModelSEED rxn id, no KEGG detour) and unaffected.
  • [TBD] markers on the correction pieces not yet landed: regenerated staging JSONs (corrected extraction script lives in the dGPredictor repo), an alias-validation guard in the ingest script, and the re-promotion pass repairing the 7,466 canonical triples. The draft's "Open loose ends" list tracks these, plus a suggested regression test pinning known carry-over cases.
  • Results [NUMBERS PENDING] stub now flags that the head-to-head ΔrG′ comparison must be computed from post-repair records.

🤖 Generated with Claude Code

https://claude.ai/code/session_01XoJyv1ukSyPfME8WaHewJh

freiburgermsuand others added 2 commits August 20, 2026 20:00
Expand the multi-source thermodynamics Methods draft (and its inlined
copy in MANUSCRIPT.md) with the audit finding that 17,271 of 27,715
stored dGPredictor records (62%) carry an estimate for a KEGG reaction
id absent from the reaction's aliases, caused by a stale-variable bug
in the 2023 staging notebook: the alias-extraction loop never reset its
holding variable, so KEGG-less reactions inherited the previous
reaction's id. A database-wide fingerprint test explains 100.0% of
contaminated entries by carry-over (zero exceptions); the contaminated
subset shows null correlation with eQuilibrator/GC (r ~ 0.00 vs
0.61/0.74 for correct mappings), and 7,466 contaminated values were
promoted into canonical deltag/deltagerr/reversibility.
The new subsection documents the defect, forensic confirmation, why
reruns re-propagated it (no alias validation in
Update_Reaction_dGPredictor_Energies.py), and the correction path,
with [TBD] markers on the pieces not yet landed (regenerated staging
JSONs, alias-validation guard, re-promotion pass). The retrained
dGPredictor-ModelSEED source is noted as structurally immune (keyed
directly by rxn id). Results stub now flags that the head-to-head
comparison must use post-repair records.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XoJyv1ukSyPfME8WaHewJh
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@freiburgermsu