JWTSecAPI is a reproducible research framework for testing whether JWT library APIs enforce cryptographic security constraints. It connects the complete workflow described in the paper:

1. retrieve evidence from JWT and cryptographic documents and generate security constraints;
2. construct library-specific paired tests: a compliant function (Fc) and a violation function (Fv);
3. validate and deterministically refine generated code;
4. execute the pair in isolated, language-specific Docker sandboxes;
5. classify whether the target API accepts Fc and rejects Fv.

The repository contains both the implementation and curated intermediate artifacts. This distinction is deliberate: artifact mode can be reproduced without sending data to an LLM, while regeneration mode exercises the RAG and LLM stages from source documents.

The original metadata for the 62 JWT libraries analyzed in the paper is provided under resource/library_metadata/analyzed_libraries_original_info/. The reviewer-facing CSV/JSONL export includes each library's identifier, language, package manager/name, repository or documentation URL, stars/forks from the source dataset, declared JWT/JOSE support service, runtime/container metadata, install commands, adapter status, and source dataset row index. The deeper source catalog remains available as resource/library_metadata/libraries_dataset.json, and the manual security oracle is kept separately under resource/library_metadata/manual_results/final_import/.

standards/papers -> RAG constraint generation -> Rule-list
|
library catalog + manual oracle ---------------+-> test plan + prompt contexts
|
v
Fc/Fv generation
|
validate -> refine
|
v
Docker sandbox execution
|
v
result analysis

The public jwtsapi command is a configuration-driven orchestration layer. The underlying implementations remain independently callable in:

• src/kb_preprocess: document preprocessing, layered retrieval, rule generation, and Rule-list normalization;
• src/test_generation: library/rule test planning and shared fixtures;
• src/llm_test_generator: prompt contexts, Fc/Fv generation, validation, refinement, per-language executors, sandbox aggregation, and analysis.

Requirements:

• Python 3.10 or newer;
• Docker, for isolated execution;
• network access only when building images or regenerating LLM artifacts.

Install the local package:

python -m pip install -e .

Build the Python sandbox once:

docker build -t jwtsapi-python:3.12 -f docker/python/Dockerfile .

For the reviewer-oriented full static generation and multilingual sandbox workflows, use:

python scripts/reproduce.py static
python scripts/reproduce.py full --build-images

See docs/LATEST_DOCKER_VALIDATION.md for Docker availability checks, resumable full-run behavior, the validated image list, and the latest complete execution-audit baseline.

See docs/ARTIFACT_EVALUATION.md for exact result semantics and docs/PAPER_ALIGNMENT.md for the paper-to-code mapping and currently disclosed coverage gaps.

The static command rebuilds the full refined Fc/Fv artifact from the curated reviewed templates and validates the generated pair schema. The full command executes the complete Docker sandbox workflow.

python scripts/reproduce.py static uses the published canonical constraints and curated reviewed Fc/Fv templates, then reruns full static generation and validation.

configs/pipeline.regenerate.example.json enables both live stages:

• constraints.mode = "rag" rebuilds the knowledge base and calls an OpenAI-compatible chat-completions endpoint;
• generation.mode = "live" regenerates library-specific Fc/Fv pairs.

To use it, place legally redistributable standards/papers in documents/, set the API key only in the environment, review the model/base URL, and run:

export LLM_API_KEY="..."
jwtsapi run --config configs/pipeline.regenerate.example.json

PowerShell:

$env:LLM_API_KEY="..."
jwtsapi run --config configs/pipeline.regenerate.example.json

The framework never requires an API key in a tracked file. Source documents are excluded from Git by default because their redistribution terms may differ.

For each library/constraint pair, Fc is the function using the compliant security-relevant parameter and should execute successfully. Fv changes that parameter to violate the constraint and should be rejected by a library that enforces the policy. Treating the pair together prevents ordinary API misuse, dependency failures, or malformed fixtures from being mislabeled as security enforcement.

The generated JSONL schema preserves the library, rule, operation, fixture bindings, expected outcome, function role, imports, and code body. Validation runs before and after deterministic refinement. Execution feedback can also be supplied to llm_test_generator.cli feedback-refine-cases for LLM-assisted repair of compiler/runtime failures.

The manifest runner dispatches Python, Java, JavaScript, C#, PHP, Ruby, Go, Kotlin, Lua, C, and C++ jobs to their existing executors. Dockerfiles live under docker/<language>/; C/C++ use the native library configuration in resource/llm_test_generation/native_docker_libraries.json. These containers isolate test dependencies and processes for research experiments; they are not a hardened boundary for deliberately malicious code.

Build only the images needed by the jobs you select.

The lower-level commands remain useful for focused experiments:

python -m kb_preprocess.cli --help
python -m test_generation.cli --help
python -m llm_test_generator.cli --help

They expose individual steps such as batch-generate-rules, build-language-contexts, generate-cases, refine-generated-cases, feedback-refine-cases, and run-sandbox.

Generated run directories, raw LLM responses, source documents, local environments, package caches, and secrets are ignored. Before publishing a fork, inspect git status, verify the dataset and document licenses, and run a secret scanner over the full Git history—not only the working tree.

python -m pip install -e ".[dev]"
pytest
python -m compileall -q src/jwtsapi src/kb_preprocess src/library_metadata src/test_generation src/llm_test_generator

The maintained RAG implementation is src/kb_preprocess/. Earlier notebook-export prototypes were removed from the public tree because they were not executable Python and contained obsolete embedded credentials.

The PEM files under resource/test_generation/fixtures/ are intentionally public test keys. Never use them outside the test harness.

See CONTRIBUTING.md, SECURITY.md, and CITATION.cff. The code is released under the MIT License; third-party datasets, papers, standards, generated artifacts, and dependencies retain their own terms.