Skip to content
View johnzfitch's full-sized avatar

Block or report johnzfitch

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
johnzfitch/README.md

Header

definitelynot.aiInternet UniverseUC Berkeley MathematicsEmail

SF Bay Area • Git Page • All icons from iconics


OpenAI Codex — The Ghost in the Machine

Important

A pre-main()(⁠#[ctor::ctor]) constructor was silently stripping LD_ / DYLD_ from Codex tool subprocesses, forcing 11–300× slowdowns through CUDA/MKL fallbacks across every supported OS. OpenAI's specialized debugging team investigated for weeks and walked away without a root cause — the diagnostics couldn't see code that ran before they loaded. I traced the regression to a single commit, built the reproduction harness, and shipped the upstream fix in rust-v0.80.0 with attribution. It was the main blocker to Codex spawning and controlling effective subagents.

Proof: Issue #8945 | PR #8951 | Release notes (rust-v0.80.0)

Full Investigation Details

The Ghost

In October 2025, OpenAI assembled a specialized debugging team to investigate mysterious slowdowns affecting Codex. After a week of intensive investigation: nothing.

The name fits. pre_main_hardening() executed before main() — before profilers attached, before logging initialized, before any instrumentation could see it run. It stripped LD_LIBRARY_PATH and DYLD_LIBRARY_PATH from the process environment, then control returned to main() and the constructor was gone. Users could see the variables set in their shell. Inside codex exec, they were empty.


The Hunt

Within 3 days of their announcement, I had the introducing commit (PR #4521) and a working hypothesis. I sent both to @tibo_openai.

Identification is not proof. The next 2 months were repro harnesses, benchmarks across CUDA / Conda / MKL / HPC stacks, and cross-referencing 15+ scattered user reports until the pattern was undeniable.

Timeline

DateEvent
Sept 30, 2025PR #4521 merges, enabling pre_main_hardening() in release builds
Oct 1, 2025rust-v0.43.0 ships (first affected release)
Oct 6, 2025First “painfully slow” regression reports
Oct 1–29, 2025Spike in env/PATH inheritance issues across platforms
Oct 29, 2025Emergency PATH fix lands (did not catch root cause)
Late Oct 2025OpenAI’s specialized team investigates, declares there is no root cause, identifies issue as user behavior change
Jan 9, 2026My fix merged, credited in release notes

Evidence Collected

PlatformIssuesFailure Mode
macOS#6012, #5679, #5339, #6243, #6218DYLD_* stripping breaking dynamic linking
Linux/WSL2#4843, #3891, #6200, #5837, #6263LD_LIBRARY_PATH stripping → silent CUDA/MKL degradation

Compiled evidence packages:

 Platform-specific failure modes
Reproduction steps with quantifiable performance regressions (11–300×) and benchmarks
 Pattern analysis
Cross-referenced 15+ scattered user reports over 3 months, traced process environment inheritance through fork/exec boundaries

Comprehensive Technical Analysis
Investigation Methodology


Why Conventional Debugging Failed

The bug was designed to be invisible:

Pre-main execution
Used #[ctor::ctor] to run before main(), before any logging or instrumentation
Silent stripping
No warnings, no errors — just missing environment variables
Distributed symptoms
Appeared as unrelated issues across different platforms and configurations
User attribution
Everyone assumed they misconfigured something (shell looked fine)
Wrong search space
Team was debugging post-main application code

[!NOTE] Standard debugging tools cannot see pre-main execution. Profilers start at main(). Log hooks are not initialized yet. The code executes, modifies the environment, and vanishes.


The Impact

OpenAI confirmed and merged the fix within 24 hours, explicitly crediting the investigation in v0.80.0 release notes:

"Codex CLI subprocesses again inherit env vars like LD_LIBRARY_PATH/DYLD_LIBRARY_PATH to avoid runtime issues. As explained in #8945, failure to pass along these environment variables to subprocesses that expect them (notably GPU-related ones), was causing 10×+ performance regressions! Special thanks to @johnzfitch for the detailed investigation and write-up in #8945."

Restored:

GPU accelerationInternal ML/AI dev teams
CUDA/PyTorchML researchers
MKL/NumPyScientific computing users
Conda environmentsCross-platform compatibility
Enterprise driversDatabase connectivity

When the tools are blind, the system lies. Everyone else has stopped looking.


Recent Work

claude-cowork-linux⭐410
The Linux port of Claude Desktop's Cowork mode. Bubblewrap sandbox in place of a VM; the ASAR is unpacked from the host before any sandboxed code runs. Highest-adoption project in the portfolio.
llmx
Local-first codebase indexer. BM25 + mdbr-leaf-ir neural embeddings (Burn) fused via Reciprocal Rank Fusion; deterministic chunking; runs in-browser via WebGPU/WASM. Live at llm.cat.
dota
Post-quantum secrets manager. v7 TC-HKEM hybrid (ML-KEM-768 + X25519); Argon2id master key; AES-256-GCM encrypted JSON vault. Terminal UI.
claude-wiki⭐20
Comprehensive Markdown documentation mirror for Anthropic's Claude, featuring 2000+ articles on APIs, SDKs, agents, and integrations.
pyghidra-lite⭐34
Token-efficient MCP server for Ghidra, enabling analysis of ELF, Mach-O, and PE binaries with Swift, Objective-C, and Hermes support.
raley-bot
Automated grocery shopping assistant leveraging web API for intelligent product selection, price tracking, and coupon clipping via CLI and MCP.
indepacer
A Python CLI tool for querying PACER, downloading federal court dockets and documents, and managing case data efficiently.
claude-warden⭐60
Security hooks for Claude Code: blocks SSRF probes, caps subagent spawn budgets, compresses MCP outputs, and exports every tool call to OTEL traces.

Selected Work

claude-cowork-linux⭐410
Run the official Claude Desktop app's Cowork mode natively on Linux. Bubblewrap sandbox in place of a VM; the ASAR is unpacked from the host before any sandboxed code runs.
specHO
LLM watermark detection via phonetic/semantic analysis (The Echo Rule) — live demo at definitelynot.ai
codex-patcher
Automated Rust code patching tool leveraging tree-sitter for syntax-aware modifications and reliable LLM-generated updates.
htmx-docs
Curated HTMX documentation in Markdown, including API references, Big Sky repos, and relevant RFCs, organized for easy access.
filearchy
Filearchy is a Wayland file manager forked from cosmic-files, enhancing workflows with custom MIME icons, extended archive support, and terminal integration.
nautilus-plus
Enhanced Nautilus file manager with sub-millisecond search, large animated thumbnail support, and crash prevention features.
indepacer
CLI tool for querying PACER, enabling case searches, docket downloads, and document retrieval from federal court records.

Self-hosting bare metal infrastructure (NixOS) with post-quantum cryptography, authoritative DNS, and containerized services.


Live Demos

Cosmic Code Cleaner @ definitelynot.ai
LLM paste sanitizer with vectorhit algorithm — fix curly quotes, invisible Unicode, confusable punctuation, dedent blocks
LLMX Ingestor @ llm.cat
WebAssembly codebase indexer — private, deterministic chunking and BM25 search for large folders
LINTENIUM FIELD @ internetuniverse.org
Terminal-based ARG experience — interactive mystery with audio visualizations
Observatory @ look.definitelynot.ai
WebGPU deepfake detection running 4 ML models in browser

Featured

dota — Post-Quantum Secrets Manager

Defense of the Artifacts: A secrets manager engineered for cryptographic longevity. While current encryption remains secure, "harvest now, decrypt later" attacks mean secrets stored today may be vulnerable to quantum computers within their lifetime. dota addresses this with hybrid post-quantum encryption that provides security against both classical and quantum adversaries.

LayerImplementationWhy It Matters
Key EncapsulationML-KEM-768 + X25519 hybridNIST-standardized lattice crypto with classical fallback — if either is broken, the other protects
Key DerivationArgon2id (memory-hard)Resists GPU/ASIC brute-force; tunable time/memory parameters
StorageSQLCipher (AES-256-CBC)Encrypted at rest with authenticated pages; survives partial file corruption
Hardware AuthHMAC-SHA1 challenge-responseYubiKey/SoloKey required for unlock — no master password alone can decrypt

The TUI (Ratatui) provides vim-style navigation, fuzzy search across entries, secure clipboard integration with auto-clear, and TOTP generation for 2FA codes.

Stack: Rust • pqcrypto (ML-KEM) • x25519-dalek • argon2 • SQLCipher • Ratatui


llmx — Codebase Indexer for Local Agents

Live Demo:llm.cat (WebAssembly — runs entirely in browser, no upload)

Local-first codebase indexing with real neural embeddings (mdbr-leaf-ir) running via WebGPU. No server, no API calls, no data leaving your machine. Hybrid search combines BM25 keyword ranking with vector similarity using RRF for best-of-both-worlds retrieval.

llmx index ~/projects/myapp # Build trigram + BM25 index
llmx search "authentication middleware" --limit 20
llmx export --format md --max-tokens 8000 # Context-window-aware export
llmx serve --port 8080 # Local HTTP API for agents
CapabilityImplementation
Neural Embeddingsmdbr-leaf-ir vectors with WebGPU acceleration — ~50ms inference, same quality as server-side
Hybrid SearchBM25 + vector similarity fused via RRF — handles exact matches and semantic similarity
Smart ChunkingDeterministic by file type: functions, headings, JSON keys — same input always yields identical chunks
Semantic ExportsHierarchical outline format (llm.md) with function names and heading breadcrumbs for selective retrieval
Proof
7,147 files(Apple HIG corpus)31 MB index()1,625 tokens(99.98% savings)
Stack
Rust • tantivy • tree-sitter • WASM • WebGPU

claude-warden — Security Hooks for Claude Code

A defense-in-depth hook system for Claude Code that addresses token efficiency, security boundaries, and observability. Born from months of production use identifying failure modes in LLM coding agents.

The Problem: Claude Code's default behavior can burn tokens on verbose command output, leak internal network topology via SSRF, spawn unbounded subagents, and produce unobservable execution traces.

HookThreat ModelMitigation
quiet-overridesToken exhaustion from npm install, cargo build, git logInjects -q/--silent/--quiet flags; caps output at configurable byte limit
ssrf-protectionAgent fetching http://169.254.169.254 (cloud metadata) or internal servicesBlocks RFC1918/link-local ranges; allowlist for legitimate internal APIs
mcp-compressionMCP tool outputs flooding context windowgzip + base64 for large payloads; configurable threshold
subagent-budgetRecursive agent spawning exhausting API quotaPer-session spawn limits; depth tracking; cost estimation
otel-tracingBlack-box execution; no audit trailExports spans to Grafana/Loki with tool calls, durations, token counts
# Example: warden blocks verbose npm and injects quiet flag
$ claude "install dependencies"# [warden] Intercepted: npm install → npm install --silent# [warden] Output capped at 4096 bytes (was 847KB)

Stack: Shell • jq • OpenTelemetry • Prometheus • Grafana/Loki


AI / ML / Agent Tooling

  • claude-wiki ⭐20 — Comprehensive Anthropic documentation wiki — 749+ docs across 24 categories
  • observatory — WebGPU deepfake detection with 4 ML models — live: look.definitelynot.ai
  • specHO — LLM watermark detection via phonetic/semantic analysis — live: definitelynot.ai
  • burn-plugin — Claude Code plugin for the Burn deep learning framework
  • raley-bot — Automated grocery assistant with F5 bot detection evasion, unit pricing across bizarre measurements, automatic coupon clipping, and MCP server for Claude Desktop

Infrastructure

Primary server: Dedicated bare-metal NixOS host (details available on request)

SecurityPost-quantum SSH • Rosenpass VPN • nftables firewall
DNSUnbound resolver with DNSSEC • ad/tracker blocking
ServicesFreshRSS • Caddy (HTTPS/HTTP/3) • cPanel/WHM • Podman containers
NetworkLocal 10Gbps • Authoritative BIND9 with RFC 2136 ACME

Philosophy

Pinned Loading

  1. claude-wardenclaude-wardenPublic

    Security hooks and monitoring for Claude Code — quiet overrides, SSRF protection, MCP compression, OTEL tracing

    Shell 60 2

  2. pyghidra-litepyghidra-litePublic

    Lightweight MCP server for Ghidra-based reverse engineering with iOS, Linux, and game file support

    Python 34 2

  3. claude-cowork-linuxclaude-cowork-linuxPublic

    Run Claude Desktop’s Cowork mode natively on Linux — no macOS or VM required

    JavaScript 411 82

  4. dotadotaPublic

    Defense of the Artifacts - Post-quantum secure secrets manager with TUI

    Rust 2

  5. claude-wikiclaude-wikiPublic

    Official Claude Documentation - Converted to 2000+ Markdown Files - All documentation sourced from first party sources- 24 categories - refreshed daily, multi-agent orchestration

    HTML 20 4

  6. human-interface-markdownhuman-interface-markdownPublic

    Apple Human Interface Guidelines archive (1980-2014) - 35 documents optimized for LLM consumption and human exploration. Spanning Lisa, Mac, NeXT, Newton, Aqua, and iOS eras.

    9 1