vallm

A complete toolkit for validating LLM-generated code.

AI Cost Tracking

🤖 LLM usage: $2.1286 (159 commits)
👤 Human dev: ~$5263 (52.6h @ $100/h, 30min dedup)

Generated on 2026-07-29 using openrouter/deep/deep-v4-pro

vallm validates code proposals through a four-tier pipeline — from millisecond syntax checks to LLM-as-judge semantic review — before a single line ships.

Features

Multi-language AST parsing via tree-sitter (165+ languages)
Syntax validation with ast.parse (Python) and tree-sitter error detection
Import resolution checking for Python, JavaScript/TypeScript, Go, Rust, Java, C/C++
Complexity metrics via radon (Python) and lizard (16 languages)
Security scanning with language-specific patterns and optional bandit integration
LLM-as-judge semantic review via Ollama, litellm, or direct HTTP
Code graph analysis — import/call graph diffing for structural regression detection
AST similarity scoring with normalized fingerprinting
Pluggy-based plugin system for custom validators
Rich CLI with JSON/text output formats
MCP integration — Model Context Protocol server for LLM tool calling

Supported Languages

Language	Syntax	Imports	Complexity	Security
Python	✅ AST + tree-sitter	✅ Full resolution (22 methods)	✅ radon + lizard	✅ bandit + patterns
JavaScript	✅ tree-sitter	✅ Node.js builtins	✅ lizard	✅ XSS, eval patterns
TypeScript	✅ tree-sitter	✅ Node.js builtins	✅ lizard	✅ XSS, eval patterns
Go	✅ tree-sitter	✅ stdlib + modules	✅ lizard	✅ SQL injection, exec
Rust	✅ tree-sitter	✅ crates	✅ lizard	✅ unsafe, unwrap
Java	✅ tree-sitter	✅ stdlib packages	✅ lizard	✅ Runtime.exec, SQL
C/C++	✅ tree-sitter	✅ std headers	✅ lizard	✅ buffer overflow, system
Ruby	✅ tree-sitter	⚠️ Limited	✅ lizard	⚠️ Limited
PHP	✅ tree-sitter	⚠️ Limited	✅ lizard	⚠️ Limited
Swift	✅ tree-sitter	⚠️ Limited	✅ lizard	⚠️ Limited
Kotlin	✅ tree-sitter	⚠️ Limited	✅ lizard	⚠️ Limited
Scala	✅ tree-sitter	⚠️ Limited	✅ lizard	⚠️ Limited

Installation

pip install vallm

With optional dependencies:

pip install vallm[all] # Everything
pip install vallm[llm] # Ollama + litellm for semantic review
pip install vallm[security] # bandit integration
pip install vallm[semantic] # CodeBERTScore
pip install vallm[graph] # NetworkX graph analysis

Quick Start

Validate Entire Project

# Install with LLM support
pip install vallm[llm]
# Setup Ollama (for semantic review)
ollama pull qwen2.5-coder:7b
ollama serve
# Validate entire project recursively
vallm batch . --recursive --semantic --model qwen2.5-coder:7b
# Fast validation for quick feedback (skip imports and complexity)
vallm batch . --recursive --no-imports --no-complexity
# Generate validation report in TOON format
vallm batch . --recursive --output toon > ./project/validation.toon

Project Structure & Files

Core Source Code:

src/vallm/ - Main package directory
- cli.py - Simplified CLI entry point (9 lines)
- config.py - Configuration management
- scoring.py - Validation pipeline and scoring
- hookspecs.py - Plugin system specifications
- core/ - Core functionality
  - languages.py - Language detection and support
  - proposal.py - Code proposal model
  - ast_compare.py - AST similarity analysis
  - graph_builder.py - Code graph construction
  - graph_diff.py - Graph diffing
- validators/ - Validation modules
  - syntax.py - Syntax validation
  - complexity.py - Complexity analysis
  - security.py - Security scanning
  - semantic.py - LLM semantic review
  - imports/ - Import validation modules
- sandbox/ - Code execution sandbox
- cli/ - Modular CLI components
  - command_handlers.py - CLI command implementations
  - output_formatters.py - Output formatting
  - settings_builders.py - Settings configuration
  - batch_processor.py - Batch processing logic

Examples & Documentation:

examples/ - Comprehensive examples directory
- 01_basic_validation/ - Basic validation pipeline
- 02_ast_comparison/ - AST similarity scoring
- 03_security_check/ - Security pattern detection
- 04_graph_analysis/ - Import/call graph analysis
- 05_llm_semantic_review/ - LLM semantic review
- 06_multilang_validation/ - Multi-language validation
- 07_multi_language/ - Comprehensive multi-language demo
- 08_code2llm_integration/ - Code2LLM integration
- 09_code2logic_integration/ - Code2Logic integration
- 10_mcp_ollama_demo/ - MCP + Ollama demo
- 11_claude_code_autonomous/ - Autonomous refactoring
- 12_ollama_simple_demo/ - Simplified Ollama demo
docs/ - Documentation
- README.md - Documentation overview
- TESTING.md - Testing guidelines

Configuration Files:

pyproject.toml - Project configuration and dependencies
.gitignore - Git ignore patterns (includes .git/ exclusion)
LICENSE - Apache 2.0 license
CONTRIBUTING.md - Contribution guidelines
CHANGELOG.md - Version history
TODO.md - Development roadmap

Scripts & Tools:

project.sh - Project analysis script
scripts/ - Utility scripts
- bump_version.py - Version management
- test_docker_installation.sh - Docker testing

Testing:

tests/ - Comprehensive test suite
- test_cli_e2e.py - End-to-end CLI tests
- test_syntax.py - Syntax validation tests
- test_imports.py - Import validation tests
- test_complexity.py - Complexity analysis tests
- test_security.py - Security scanning tests
- test_semantic_validation.py - Semantic validation tests
- conftest.py - Test configuration

CI/CD & GitHub:

.github/workflows/ - GitHub Actions workflows
- ci.yml - Continuous integration
- comprehensive-tests.yml - Full test suite
- publish.yml - Package publishing
Dockerfile.test - Testing container

Project Analysis:

project/ - Analysis outputs and visualizations
- validation.toon - Validation results (TOON format)
- project.toon - Project analysis
- flow.toon - Workflow visualization
- calls.toon - Call graph analysis
- README.md - Project analysis overview

Python API

fromvallmimportProposal, validate, VallmSettingscode="""def fibonacci(n: int) -> list[int]: if n <= 0: return [] fib = [0, 1] for i in range(2, n): fib.append(fib[i-1] + fib[i-2]) return fib"""proposal=Proposal(code=code, language="python")
result=validate(proposal)
print(f"Verdict: {result.verdict.value}") # pass / review / failprint(f"Score: {result.weighted_score:.2f}")

CLI Commands Reference

# Batch validation (best for entire projects)
vallm batch . --recursive --semantic --model qwen2.5-coder:7b
vallm batch src/ --recursive --include "*.py,*.js" --exclude "*/test/*"
vallm batch . --recursive --format json --fail-fast
vallm batch . --recursive --verbose --show-issues # Detailed per-file results# Output formats for batch results
vallm batch . --recursive --format json # Machine-readable JSON
vallm batch . --recursive --format yaml # YAML format
vallm batch . --recursive --format toon # Compact TOON format
vallm batch . --recursive --format text # Plain text# Single file validation
vallm validate --file mycode.py --semantic --model qwen2.5-coder:7b
vallm validate --file app.js --security
vallm validate --file mycode.py --format json # JSON output# Quick syntax check only
vallm check mycode.py
vallm check src/main.go
# Configuration and info
vallm info

Real-World Usage Examples

1. Fast Project Validation (Recommended for CI/CD)

# Quick syntax check - excludes .git/ and other system files automatically
vallm batch . --recursive --no-imports --no-complexity
# Output: Excluded 30246 files by .gitignore, Validating 169 files...# ✓ 83 files passed, ✗ 115 files failed (mostly non-code files)

2. Generate Validation Report

# Save TOON format report to project directory
vallm batch . --recursive --output toon > ./project/validation.toon
# Save JSON report for CI/CD integration
vallm batch . --recursive --output json > ./project/validation.json
# Save detailed text report with security checks
vallm batch . --recursive --security --output text > ./project/validation-report.txt

3. Selective File Validation

# Validate only Python and JavaScript files
vallm batch . --recursive --include "*.py,*.js" --exclude "*/test/*"# Validate specific directory with custom patterns
vallm batch src/ --recursive --include "*.py" --exclude "*/__pycache__/*"# Validate with custom gitignore override
vallm batch . --recursive --no-gitignore --exclude "*.log,tmp/*"

4. Full Pipeline with LLM Review

# Complete validation with semantic analysis
vallm batch . --recursive --semantic --model qwen2.5-coder:7b --security
# Export full results with per-file details
vallm batch . --recursive --semantic --model qwen2.5-coder:7b --output json > full-validation.json

5. Development Workflow Integration

# Pre-commit validation (fast)
vallm batch . --recursive --no-imports --no-complexity --fail-fast
# Feature branch validation (medium)
vallm batch src/ --recursive --no-complexity --show-issues
# Release validation (full)
vallm batch . --recursive --semantic --model qwen2.5-coder:7b --security --verbose

Fast Validation Options

When validating large projects (100+ files), use these options to speed up validation:

# Fastest - syntax only (skip imports and complexity)
vallm batch . --recursive --no-imports --no-complexity
# Fast - skip import validation (often the slowest)
vallm batch . --recursive --no-imports
# Parallel processing for multi-core speedup# Note: --parallel option was removed in v0.1.16 due to module conflicts# Use --no-imports --no-complexity for better performance# Combine for maximum speed
vallm batch . --recursive --no-imports --no-complexity
# Quick syntax check only (single files)
vallm check src/proxym/config.py

Option	Speed Impact	Description
`--no-imports`	High	Skip import resolution (slowest validator)
`--no-complexity`	Medium	Skip complexity analysis (radon/lizard)
`--security`	Low	Add security checks (fast pattern matching)
`--semantic`	Very High	LLM semantic review (requires Ollama/OpenAI)

Performance Benchmarks:

Fast mode: --no-imports --no-complexity - ~100 files/second
Normal mode: Default settings - ~20 files/second
Full mode: With --semantic - ~2 files/second

Recommendation for CI/CD:

# Fast validation for quick feedback (PR checks)
vallm batch src/ --recursive --no-imports --no-complexity --fail-fast
# Full validation before merge (quality gate)
vallm batch src/ --recursive --security
# Release validation with LLM review
vallm batch . --recursive --semantic --model qwen2.5-coder:7b

Generate Validation Summary File

# JSON summary for entire project (with per-file details and issues)
vallm batch . --recursive --output json > validation-summary.json
# YAML summary for src/ directory (with per-file details and issues)
vallm batch src/ --recursive --output yaml > validation-summary.yaml
# TOON format (compact, human-readable) with per-file details
vallm batch . --recursive --output toon > validation-summary.toon
# Text format with security checks
vallm batch . --recursive --output text --security > validation-report.txt
# Full validation with semantic review - save to file
vallm batch . --recursive --semantic --model qwen2.5-coder:7b --output json > full-validation.json
# Tee output to both console and file
vallm batch . --recursive --output json | tee validation-summary.json
# Save to project directory for analysis integration
vallm batch . --recursive --output toon > ./project/validation.toon

Sample Output Files:

project/validation.toon - Real validation results (TOON format)
project/validation.json - JSON format (generate with --output json)
project/validation-report.txt - Text format (generate with --output text)

Output Structure (JSON/YAML/TOON formats now include per-file details):

{
"summary": {
"total_files": 146,
"passed": 145,
"failed": 1
},
"files": [
{
"path": "src/proxym/config.py",
"language": "python",
"verdict": "fail",
"score": 0.45,
"issues_count": 3,
"issues": [
{
"validator": "syntax",
"severity": "error",
"message": "Invalid syntax at line 42",
"line": 42,
"column": 15
},
{
"validator": "imports",
"severity": "error", "message": "Module 'requests' not found",
"line": 5,
"column": 0
}
]
}
],
"failed_files": [
{"path": "src/proxym/config.py", "error": "Validation fail"}
]
}

---
summary:
total_files: 146passed: 145failed: 1files:
- path: src/proxym/config.pylanguage: pythonverdict: failscore: 0.45issues_count: 3issues:
- validator: syntaxseverity: errormessage: "Invalid syntax at line 42"line: 42column: 15
- validator: importsseverity: errormessage: "Module 'requests' not found"line: 5

# vallm batch | 146f | 145✓ 1✗
SUMMARY:
total: 146
passed: 145
failed: 1
FILES:
[python]
✗ src/proxym/config.py
verdict: fail
score: 0.45
issues: 2
[error] syntax: Invalid syntax at line 42@42
[error] imports: Module 'requests' not found@5
✓ src/proxym/ctl.py
verdict: pass
score: 0.92
issues: 0
FAILED:
✗ src/proxym/config.py: Validation fail

Batch Command Options

Option	Short	Description
`--recursive`	`-r`	Recurse into subdirectories
`--include`		File patterns to include (e.g., ".py,.js")
`--exclude`		File patterns to exclude
`--use-gitignore`		Respect .gitignore patterns (default: true)
`--format`	`-f`	Output format: `rich`, `json`, `yaml`, `toon`, `text`
`--fail-fast`	`-x`	Stop on first failure
`--semantic`		Enable LLM-as-judge semantic review
`--security`		Enable security checks
`--model`	`-m`	LLM model for semantic review
`--verbose`	`-v`	Show detailed validation results for each file
`--show-issues`	`-i`	Show issues for failed files

With Ollama (LLM-as-judge)

# 1. Install and start Ollama
ollama pull qwen2.5-coder:7b
# 2. Run with semantic review
vallm validate --file mycode.py --semantic

fromvallmimportProposal, validate, VallmSettingssettings=VallmSettings(
enable_semantic=True,
llm_provider="ollama",
llm_model="qwen2.5-coder:7b",
)
proposal=Proposal(
code=new_code,
language="python",
reference_code=existing_code, # optional: compare against reference
)
result=validate(proposal, settings)

Validation Pipeline

Tier	Speed	Validators	What it catches
1	ms	syntax, imports	Parse errors, missing modules
2	seconds	complexity, security	High CC, dangerous patterns
3	seconds	semantic (LLM)	Logic errors, poor practices
4	minutes	regression (tests)	Behavioral regressions

The pipeline fails fast — Tier 1 errors stop execution immediately.

Configuration

Via environment variables (VALLM_*), vallm.toml, or pyproject.toml [tool.vallm]:

# vallm.tomlpass_threshold = 0.8review_threshold = 0.5max_cyclomatic_complexity = 15enable_semantic = truellm_provider = "ollama"llm_model = "qwen2.5-coder:7b"

Quality Pipeline (pyqual)

vallm uses pyqual — a declarative quality gate system — to ensure code meets all quality standards before shipping.

Quality Gates

Gate	Metric	Threshold	Current
Cyclomatic Complexity	cc	≤ 15	3.4 ✅
Vallm Pass Rate	vallm_pass	≥ 90%	97.7% ✅
Test Coverage	coverage	≥ 55%	63.9% ✅

Pipeline Stages

# pyqual.yamlpipeline:
name: quality-loop-with-llxmetrics:
cc_max: 15vallm_pass_min: 90coverage_min: 55stages:
- setup # Dependency check
- analyze # code2llm analysis
- validate # vallm batch validation
- lint # ruff linting
- test # pytest with coverage
- prefact # Prefactoring (optional)
- fix # Auto-fix with LLX (optional)
- verify # Post-fix validation
- push # Auto-commit & push
- publish # Build & publish

Running the Pipeline

# Full pipeline with quality gates
pyqual run
# Check current metrics
pyqual status
# View pipeline logs
pyqual logs
# Validate pyqual.yaml config
pyqual validate

Publishing to PyPI

# Set credentials and publish
TWINE_USERNAME=__token__ TWINE_PASSWORD=pypi-xxx make publish
# Or publish to TestPyPI
TWINE_USERNAME=__token__ TWINE_PASSWORD=pypi-xxx make publish-test

Without credentials, the publish stage gracefully skips with a warning.

MCP Integration

vallm provides Model Context Protocol (MCP) server integration, exposing validation tools as MCP endpoints for LLM tool calling.

Starting the MCP Server

# Start the MCP server from project root
python3 mcp_server.py
# Or start the packaged module directly
python3 -m mcp.server.self_server

Claude Desktop Configuration

Add to your claude_desktop_config.json:

{
"mcpServers": {
"vallm": {
"command": "python3",
"args": ["/path/to/vallm/mcp_server.py"],
"env": {
"PYTHONPATH": "/path/to/vallm/src"
}
}
}
}

Available MCP Tools

Tool	Description	Parameters
`validate_syntax`	Multi-language syntax checking	`code`, `language`, `filename`
`validate_imports`	Import resolution validation	`code`, `language`, `filename`
`validate_security`	Security issue detection	`code`, `language`, `filename`
`validate_code`	Full pipeline validation	`code`, `language`, `filename`, `reference_code`, `enable_*` flags

Example Tool Calls

{
"method": "tools/call",
"params": {
"name": "validate_security",
"arguments": {
"code": "eval('1+1')",
"language": "python"
}
}
}

{
"method": "tools/call", "params": {
"name": "validate_code",
"arguments": {
"code": "def test(): pass",
"language": "python",
"enable_syntax": true,
"enable_security": true,
"enable_complexity": false
}
}
}

Testing MCP Integration

# Test all MCP tools
python3 test_mcp.py
# Quick validation tests
python3 mcp/tests/quick_test.py
# Test individual tools
PYTHONPATH=src python3 -c "from mcp.server._tools_vallm import validate_syntax; print(validate_syntax('print(\"hello\")', 'python')['verdict'])"# Run the complete Docker e2e flow (host build + container-side runner)
bash mcp/tests/run_e2e.sh
# Run the same e2e flow via single-service docker-compose
bash mcp/tests/run_e2e.sh --compose
# Run examples
python3 examples/mcp_demo.py

Response Format

All MCP tools return a consistent JSON response:

{
"success": true,
"validator": "security",
"score": 0.3,
"weight": 1.5,
"confidence": 0.9,
"verdict": "fail",
"issues": [
{
"message": "Use of eval() detected",
"severity": "warning",
"line": 1,
"column": 0,
"rule": "security.eval"
}
],
"details": {}
}

Plugin System

Write custom validators using pluggy:

fromvallm.hookspecsimporthookimplfromvallm.scoringimportValidationResultclassMyValidator:
tier=2name="custom"weight=1.0@hookimpldefvalidate_proposal(self, proposal, context):
# Your validation logicreturnValidationResult(validator=self.name, score=1.0, weight=self.weight)

Register via pyproject.toml:

[project.entry-points."vallm.validators"]
custom = "mypackage.validators:MyValidator"

Multi-Language Support

vallm supports 30+ programming languages via tree-sitter parsers:

Auto-Detection

fromvallmimportdetect_language, Language# Auto-detect from file pathlang=detect_language("main.rs") # → Language.RUSTprint(lang.display_name) # "Rust"print(lang.is_compiled) # True

CLI with Auto-Detection

# Language auto-detected from file extension
vallm validate --file script.py # → Python
vallm check main.go # → Go 
vallm validate --file lib.rs # → Rust# Batch validation with mixed languages
vallm batch src/ --recursive --include "*.py,*.js,*.ts,*.go,*.rs"

Supported Languages

Language	Category	Complexity	Syntax
Python	Scripting	✓ radon + lizard	✓ ast + tree-sitter
JavaScript	Web/Scripting	✓ lizard	✓ tree-sitter
TypeScript	Web/Scripting	✓ lizard	✓ tree-sitter
Go	Compiled	✓ lizard	✓ tree-sitter
Rust	Compiled	✓ lizard	✓ tree-sitter
Java	Compiled	✓ lizard	✓ tree-sitter
C/C++	Compiled	✓ lizard	✓ tree-sitter
Ruby	Scripting	✓ lizard	✓ tree-sitter
PHP	Web	✓ lizard	✓ tree-sitter
Swift	Compiled	✓ lizard	✓ tree-sitter
+ 20 more via tree-sitter		✓ tree-sitter	✓ tree-sitter

See examples/07_multi_language/ for a comprehensive demo.

Examples

Each example lives in its own folder with main.py and README.md. Run all at once:

cd examples && ./run.sh

Example Details & Links

Example	What it demonstrates	Files	Description
`01_basic_validation/`	Default pipeline — good, bad, and complex code	`main.py`, `README.md`	Basic validation with syntax, imports, complexity, and security checks
`02_ast_comparison/`	AST similarity scoring, tree-sitter multi-language parsing	`main.py`, `README.md`	Compare code similarity using AST fingerprinting
`03_security_check/`	Security pattern detection (eval, exec, hardcoded secrets)	`main.py`, `README.md`	Detect security vulnerabilities and anti-patterns
`04_graph_analysis/`	Import/call graph building and structural diffing	`main.py`, `README.md`	Build and analyze code dependency graphs
`05_llm_semantic_review/`	Ollama Qwen 2.5 Coder 7B LLM-as-judge review	`main.py`, `README.md`	Semantic code review using LLM
`06_multilang_validation/`	JavaScript and C validation via tree-sitter	`main.py`, `README.md`	Multi-language validation examples
`07_multi_language/`	Comprehensive multi-language support — 8+ languages with auto-detection	`main.py`, `README.md`	Complete multi-language validation demo
`08_code2llm_integration/`	Project analysis integration with code2llm	`main.py`, `README.md`	Integration with code2llm analysis tools
`09_code2logic_integration/`	Call graph analysis with code2logic	`main.py`, `README.md`	Advanced call graph analysis
`10_mcp_ollama_demo/`	MCP (Model Context Protocol) demo with Ollama	`main.py`, `README.md`	Model Context Protocol integration
`11_claude_code_autonomous/`	Autonomous refactoring with Claude Code	`claude_autonomous_demo.py`, `README.md`	AI-powered autonomous code refactoring
`12_ollama_simple_demo/`	Simplified Ollama integration example	`ollama_simple_demo.py`, `README.md`	Basic Ollama LLM integration

Running Examples

# Run all examplescd examples && ./run.sh
# Run specific example
python examples/01_basic_validation/main.py
# Run with validation
vallm validate --file examples/01_basic_validation/main.py --verbose
# Batch validate all examples
vallm batch examples/ --recursive --include "*.py" --verbose

Architecture

src/vallm/
├── cli/ # 🆕 Modular CLI package
│ ├── __init__.py # Command registration and app export
│ ├── command_handlers.py # CLI command implementations
│ ├── output_formatters.py # Output formatting utilities
│ ├── settings_builders.py # Settings configuration logic
│ └── batch_processor.py # Batch processing logic
├── cli.py # 🆕 Simplified main entry point (9L)
├── config.py # pydantic-settings (VALLM_* env vars)
├── hookspecs.py # pluggy hook specifications
├── scoring.py # Weighted scoring + verdict engine (CC=18 validate function)
├── core/
│ ├── languages.py # Language enum, auto-detection, 30+ languages
│ ├── proposal.py # Proposal model
│ ├── ast_compare.py # tree-sitter + Python AST similarity
│ ├── graph_builder.py # Import/call graph construction
│ └── graph_diff.py # Before/after graph comparison
├── validators/
│ ├── syntax.py # Tier 1: ast.parse + tree-sitter (multi-lang)
│ ├── imports/ # 🆕 Modular import validators
│ │ ├── base.py # 🆕 Enhanced base class with shared validate()
│ │ ├── factory.py # Validator factory
│ │ ├── python_imports.py
│ │ ├── go_imports.py # 🆕 Uses shared validation logic
│ │ ├── rust_imports.py # 🆕 Uses shared validation logic
│ │ └── java_imports.py # 🆕 Uses shared validation logic
│ ├── complexity.py # Tier 2: radon (Python) + lizard (16+ langs)
│ ├── security.py # Tier 2: patterns + bandit
│ └── semantic.py # Tier 3: LLM-as-judge
└── sandbox/
└── runner.py # subprocess / Docker execution

🆕 Code Health Improvements

Recent Refactoring Achievements:

✅ CLI Modularization - Split 850L god module into focused packages:

cli/command_handlers.py - Command implementations
cli/output_formatters.py - Output formatting logic
cli/settings_builders.py - Settings configuration
cli/batch_processor.py - Batch processing logic
cli/__init__.py - Command registration and app export

✅ Import Validator Cleanup - Removed 653L legacy module:

Enhanced BaseImportValidator with shared validation logic
Eliminated duplicate validate() methods across language validators
Improved maintainability through template method pattern

✅ Code Deduplication - Removed 469 lines of duplicated code:

Shared validation runners for examples (154 lines saved)
Centralized analysis data saving (66 lines saved)
Common demo utilities (60 lines saved)
LLM response parsing utilities (40 lines saved)
Import validator logic consolidation (40 lines saved)
Additional utility function consolidation (109 lines saved)

Updated Code Metrics:

Metric	Before	After	Improvement
God Modules (>500L)	2	0	✅ 100% eliminated
Max Cyclomatic Complexity	42	~18	✅ 57% reduction
Code Duplication	504 lines	35 lines	✅ 93% eliminated
CLI Module Size	850 lines	9 lines	✅ 99% reduction

Remaining Critical Functions:

Function	Location	CC	Status
`validate`	`scoring.py:122`	18	🟡 Acceptable
`_check_lizard`	`complexity.py`	12	🟡 Acceptable
`_parse_response`	`semantic.py`	12	🟡 Acceptable

Roadmap

v0.2 — Completeness ✅ MAJOR PROGRESS

✅ CLI modularization - Split 850L god module into focused packages
✅ Import validator cleanup - Removed 653L legacy module
✅ Code deduplication - Eliminated 469 lines of duplicate code
✅ God module elimination - 100% reduction in god modules
✅ Complexity reduction - 57% reduction in max cyclomatic complexity
Wire pluggy plugin manager (entry_point-based validator discovery)
Add LogicalErrorValidator (pyflakes) and LintValidator (ruff)
TOML config loading (vallm.toml, [tool.vallm])
Pre-commit hook integration
GitHub Actions CI/CD

v0.3 — Depth

AST edit distance via apted/zss
CodeBERTScore embedding similarity
NetworkX cycle detection and centrality in graph analysis
RegressionValidator (Tier 4) with pytest-json-report
TypeCheckValidator (mypy/pyright)
Extract output formatters

v0.4 — Intelligence

--fix auto-repair mode (LLM-based retry loop)
hypothesis/crosshair property-based test generation
E2B cloud sandbox backend
Streaming LLM output

See TODO.md for the full task breakdown.

Testing

Running Tests

# Run all tests
pytest
# Run specific test categories
pytest tests/test_syntax.py
pytest tests/test_imports.py
pytest tests/test_complexity.py
pytest tests/test_security.py
pytest tests/test_semantic_validation.py
# Run CLI end-to-end tests
pytest tests/test_cli_e2e.py -v
# Run with coverage
pytest --cov=vallm --cov-report=html
# Run performance tests
pytest tests/test_performance.py -v

Test Files Reference

tests/ - Complete test suite
- conftest.py - Test configuration and fixtures
- test_cli_e2e.py - End-to-end CLI integration tests
- test_syntax.py - Syntax validation tests
- test_imports.py - Import resolution tests
- test_complexity.py - Complexity analysis tests
- test_security.py - Security scanning tests
- test_semantic_validation.py - LLM semantic review tests
- test_languages.py - Language detection tests
- test_ast_compare.py - AST similarity tests
- test_graph.py - Graph analysis tests
- test_plugins.py - Plugin system tests
- test_performance.py - Performance benchmarks
- test_installation.py - Installation tests
- test_gitignore.py - Git ignore pattern tests
- test_pipeline.py - End-to-end pipeline tests

Test Coverage

Current test coverage: 85% across all modules.

✅ Syntax validation: 95% coverage
✅ Import resolution: 87% coverage
✅ Complexity analysis: 82% coverage
✅ Security scanning: 79% coverage
✅ Semantic validation: 71% coverage
✅ CLI commands: 89% coverage

License

Licensed under Apache-2.0.

Author

Tom Sapletta

Additional Resources

Documentation & Guides

docs/README.md - Extended documentation
docs/TESTING.md - Testing guidelines and best practices
CONTRIBUTING.md - Contribution guidelines
CHANGELOG.md - Version history and release notes
TODO.md - Development roadmap and planned features

Package Configuration

pyproject.toml - Project metadata, dependencies, and build configuration
.gitignore - Git ignore patterns (includes .git/ exclusion)
Dockerfile.test - Testing container configuration
Makefile - Build and development automation

CI/CD & Automation

.github/workflows/ - GitHub Actions workflows
- ci.yml - Continuous integration pipeline
- comprehensive-tests.yml - Full test suite
- publish.yml - Package publishing to PyPI
.pre-commit-config.yaml - Pre-commit hooks configuration
.pre-commit-hooks.yaml - Custom pre-commit hooks

Project Analysis & Metrics

project/ - Analysis outputs and visualizations
- validation.toon - Latest validation results
- project.toon - Project structure analysis
- flow.toon - Workflow visualization
- calls.toon - Call graph analysis
- compact_flow.mmd - Mermaid flow diagram
- README.md - Project analysis overview
- analysis.yaml - Detailed analysis data
- dashboard.html - Interactive dashboard

Development Tools

scripts/ - Development and maintenance scripts
- bump_version.py - Automated version management
- test_docker_installation.sh - Docker environment testing
project.sh - Project analysis and validation script

Examples & Demos

examples/ - 12 comprehensive examples with README files
examples/run.sh - Run all examples script
examples/README.md - Examples overview and guide

Source Code Organization

src/vallm/ - Main package source code
src/vallm/__init__.py - Package initialization
src/vallm/__main__.py - Module execution support
src/vallm/py.typed - Type checking marker

Status

Last updated by taskill at 2026-04-25 13:48 UTC

Metric	Value
HEAD	`4ac3a46`
Coverage	—
Failing tests	—
Commits in last cycle	50

Mostly documentation and refactoring work: multiple README/docs updates, added markdown output (with tests), new examples, a version bump, and several refactors extracting and simplifying high-complexity code paths.

Name		Name	Last commit message	Last commit date
Latest commit History 167 Commits
.github/workflows		.github/workflows
.pyqual		.pyqual
.taskill		.taskill
.vallm		.vallm
TODO		TODO
backend/routers/tickets		backend/routers/tickets
code2llm_output		code2llm_output
docs		docs
examples		examples
frontend		frontend
mcp		mcp
project		project
scripts		scripts
src/vallm		src/vallm
tests		tests
.env.example		.env.example
.gitignore		.gitignore
.nojekyll		.nojekyll
.pre-commit-config.yaml		.pre-commit-config.yaml
.pre-commit-hooks.yaml		.pre-commit-hooks.yaml
.vallmignore		.vallmignore
CHANGELOG.md		CHANGELOG.md
CONTRIBUTING.md		CONTRIBUTING.md
Dockerfile.test		Dockerfile.test
LICENSE		LICENSE
MCP_INTEGRATION_SUMMARY.md		MCP_INTEGRATION_SUMMARY.md
Makefile		Makefile
README.md		README.md
SUMD.md		SUMD.md
SUMR.md		SUMR.md
TICKET		TICKET
TODO.md		TODO.md
Taskfile.yml		Taskfile.yml
VERSION		VERSION
app.doql.css		app.doql.css
app.doql.less		app.doql.less
goal.yaml		goal.yaml
index.html		index.html
local.dev.txt		local.dev.txt
mcp_server.py		mcp_server.py
planfile.yaml		planfile.yaml
prefact.yaml		prefact.yaml
project.sh		project.sh
pyproject.toml		pyproject.toml
pyqual.yaml		pyqual.yaml
redsl.yaml		redsl.yaml
redsl_change_1776004347.txt		redsl_change_1776004347.txt
redsl_change_1776004821.txt		redsl_change_1776004821.txt
redsl_refactor_plan.md		redsl_refactor_plan.md
redsl_refactor_plan.toon.yaml		redsl_refactor_plan.toon.yaml
redsl_refactor_report.md		redsl_refactor_report.md
redsl_refactor_report.toon.yaml		redsl_refactor_report.toon.yaml
sumd.json		sumd.json
tox.ini		tox.ini
tree.sh		tree.sh
uv.lock		uv.lock

Uh oh!

Folders and files

Latest commit

History

Repository files navigation

vallm

AI Cost Tracking

Features

Supported Languages

Installation

Quick Start

Validate Entire Project

Project Structure & Files

Python API

CLI Commands Reference

Real-World Usage Examples

1. Fast Project Validation (Recommended for CI/CD)

2. Generate Validation Report

3. Selective File Validation

4. Full Pipeline with LLM Review

5. Development Workflow Integration

Fast Validation Options

Generate Validation Summary File

Batch Command Options

With Ollama (LLM-as-judge)

Validation Pipeline

Configuration

Quality Pipeline (pyqual)

Quality Gates

Pipeline Stages

Running the Pipeline

Publishing to PyPI

MCP Integration

Starting the MCP Server

Claude Desktop Configuration

Available MCP Tools

Example Tool Calls

Testing MCP Integration

Response Format

Plugin System

Multi-Language Support

Auto-Detection

CLI with Auto-Detection

Supported Languages

Examples

Example Details & Links

Running Examples

Architecture

🆕 Code Health Improvements

Roadmap

Testing

Running Tests

Test Files Reference

Test Coverage

License

Author

Additional Resources

Documentation & Guides

Package Configuration

CI/CD & Automation

Project Analysis & Metrics

Development Tools

Examples & Demos

Source Code Organization

Status

About

Topics

Resources

Contributing

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages