docval

AI Cost Tracking

• 🤖 LLM usage: $0.6002 (8 commits)
• 👤 Human dev: ~$824 (8.2h @ $100/h, 30min dedup)

Generated on 2026-07-06 using openrouter/qwen/qwen3-coder-next

---

Validate and refactor Markdown documentation against source code — detect outdated, orphaned, duplicate, and invalid docs using heuristics + optional LLM.

How it works

docs/ ──→ chunk by heading ──→ heuristic checks ──→ cross-ref with code ──→ (optional) LLM ──→ report/fix

Three validation layers, each progressively deeper:

1. Heuristic validator (fast, free) — empty sections, broken internal links, TODO/FIXME markers, duplicate detection via difflib, stale version references, archive path detection, explicit deprecation markers
2. Cross-reference validator (fast, free) — checks that backtick-quoted symbols (ClassName, function_name), import paths in code blocks, and CLI commands actually exist in the project source
3. LLM validator (optional, paid) — semantic validation via litellm for chunks that heuristics couldn't resolve with high confidence

Installation

pip install docval

With LLM support:

pip install docval[llm]

From source:

git clone https://github.com/wronai/docval.git
cd docval
pip install -e ".[dev]"

CLI Usage

Scan and report issues

docval scan docs/
docval scan docs/ --project /path/to/repo -v
docval scan docs/ -o report.md
docval scan docs/ -o report.json

Fix documentation (dry-run by default)

docval fix docs/ # preview changes
docval fix docs/ --no-dry-run # apply fixes
docval fix docs/ --no-dry-run --llm # with LLM validation

Generate a patch file

docval patch docs/ -o fixes.txt
docval patch docs/ --llm --model gpt-4o -o fixes.txt

View documentation statistics

docval stats docs/

LLM validation

export OPENAI_API_KEY=sk-...
docval scan docs/ --llm --model gpt-4o-mini
docval scan docs/ --llm --model anthropic/claude-sonnet-4-20250514
docval scan docs/ --llm --model groq/llama-3.3-70b-versatile

Any model supported by litellm works.

Python API

frompathlibimportPathfromdocval.pipelineimportscanfromdocval.reportersimportConsoleReporter, MarkdownReporter# Run validationresult=scan(
docs_dir=Path("docs/"),
project_root=Path("."),
use_llm=False,
)
# Print to consoleConsoleReporter(verbose=True).report(result)
# Write markdown reportMarkdownReporter().report(result, Path("validation-report.md"))

Using individual validators

fromdocval.chunkerimportchunk_directoryfromdocval.contextimportbuild_contextfromdocval.validatorsimportHeuristicValidator, CrossRefValidator# Chunk docsdoc_files=chunk_directory(Path("docs/"))
# Build project contextctx=build_context(Path("."))
# Run heuristicsheuristic=HeuristicValidator(ctx=ctx)
heuristic.validate(doc_files)
# Cross-reference checkcrossref=CrossRefValidator(ctx=ctx)
crossref.validate(doc_files)
# Inspect resultsforfindoc_files:
forchunkinf.chunks:
ifchunk.issues:
print(f"{f.relative_path}:{chunk.line_start} [{chunk.status.value}] {chunk.heading}")
forissueinchunk.issues:
print(f" {issue.severity.value}: {issue.message}")

What it detects

Check	Layer	Example
Empty sections	Heuristic	Heading with no body text
Broken internal links	Heuristic	[guide](./deleted-file.md)
Deprecated markers	Heuristic	DEPRECATED, OBSOLETE, DO NOT USE
Archive path	Heuristic	Files in docs/archive/ directories
Stale versions	Heuristic	References to v1.x when project is v3.x
Duplicates	Heuristic	>80% similar content across files
TODO/FIXME	Heuristic	Unfinished documentation markers
Orphaned code refs	CrossRef	`NonExistentClass` in backticks
Broken imports	CrossRef	from mypackage.deleted import X in code blocks
Semantic accuracy	LLM	Content that doesn't match actual project behavior

Architecture

src/docval/
├── cli.py # Click CLI: scan, fix, patch, stats
├── pipeline.py # Orchestrates: discover → chunk → validate → report
├── models.py # Data models: DocChunk, DocFile, ValidationResult
├── chunker.py # MD → heading-based semantic chunks
├── context.py # Build project context (AST, git, .toon files)
├── validators/
│ ├── heuristic.py # Rule-based checks (free, fast)
│ ├── crossref.py # Code ↔ docs cross-reference
│ └── llm_validator.py # Semantic validation via litellm
├── actions/
│ └── executor.py # Apply fixes: delete, archive, patch
└── reporters/
├── console.py # Rich CLI output
├── markdown_report.py # .md report
└── json_report.py # .json for CI/CD

Integration with .toon files

docval understands .toon.yaml files from the code2llm ecosystem. When present, it extracts module names, class names, and exported functions for cross-referencing, giving more accurate orphaned-reference detection.

License

Licensed under Apache-2.0.

Status

Last updated by taskill at 2026-04-25 13:37 UTC

Metric	Value
HEAD	4fba32f
Coverage	—
Failing tests	—
Commits in last cycle	6

Add markdown output for documentation generation (docs feature). Commit also added inclusion of commit messages in the markdown output.