DetectZoo

DetectZoo is a research-oriented Python toolkit that provides implementations of AI-generated content detectors across multiple modalities, including text, images, and audio.

The goal of DetectZoo is to make detection methods easy to use, reproducible, and extensible, enabling researchers and practitioners to benchmark and deploy AI-generated content detectors with minimal effort.

DetectZoo aggregates detection approaches into a single, unified API, allowing users to load and apply detectors with just a few lines of code.

---

Installation

For the sake of anonymity, we put the package on TestPyPI and you can install it with the following command:

Note: This is a temporary solution and we will release the package on PyPI after the paper is accepted.

pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ detectzoo-anon

or install from source:

git clone https://anonymous.4open.science/r/DetectZoo-1BEC/
cd detectzoo
pip install -e .

Optional extra for contributors (pytest, pytest-cov, ruff):

pip install -e ".[dev]"

The base install already includes dependencies for text, image, and audio detectors.

---

Quick Start

Detect AI-generated text

fromdetectzooimportload_detectordetector=load_detector("fast_detectgpt")
text="Large language models are transforming many fields."result=detector.predict(text)
print(result)
# DetectionResult(score=1.2345, label='ai', confidence=0.8012)print(result.score, result.label)

Detect AI-generated images

fromdetectzooimportload_detectordetector=load_detector("aeroblade")
result=detector.predict("image.png")
print(result.label) # "ai" or "human"

Detect synthetic audio

fromdetectzooimportload_detectordetector=load_detector("rawnet2")
result=detector.predict("speech.wav")
print(result.score, result.label)

List all available detectors

fromdetectzooimportlist_detectorsprint(list_detectors()) # all detectorsprint(list_detectors("text")) # text-onlyprint(list_detectors("image")) # image-onlyprint(list_detectors("audio")) # audio-only

---

Supported Detectors

DetectZoo ships detectors for text, images, and audio. Each uses the same interface: detector.predict(input) → DetectionResult.

See METHODS_AND_MODELS.md for detailed tables of supported detectors, including registry names, implementation classes, and method summaries. To programmatically list available detector names in code, use list_detectors() or specify a type: list_detectors("text" | "image" | "audio").

---

Core Components

DetectionResult

Every predict() call returns a DetectionResult dataclass:

@dataclassclassDetectionResult:
score: float# Higher = more likely AI-generatedlabel: str# "ai" or "human"confidence: float# Confidence in the label (0–1)metadata: dict# Detector-specific extra info

The metadata dictionary varies by detector and may include values like avg_log_likelihood, mean_curvature, ppl_observer, hf_lf_ratio, etc.

---

Benchmarking

DetectZoo includes a built-in evaluation pipeline for comparing detectors on labelled datasets.

Built-in datasets

DetectZoo ships with loaders for popular detection benchmarks. Data is downloaded and cached automatically on first use — no manual setup needed.

See METHODS_AND_MODELS.md — Built-in datasets for a complete table of built-in datasets, with class names, descriptions, sources, and load_dataset registry keys.

fromdetectzoo.datasetsimportCHEATDataset# Auto-downloads from GitHub on first call, cached in .detectzoo_data/cheat/dataset=CHEATDataset()
dataset=CHEATDataset(categories=["generation"]) # only first-pass ChatGPT abstracts# Or point to a local copydataset=CHEATDataset(path="data/cheat/")
foritemindataset:
print(item.label, item.data[:80])

All datasets cache downloaded files under a .detectzoo_data/ directory (configurable via cache_dir) so subsequent loads are instant.

Using the evaluator

fromdetectzooimportload_detectorfromdetectzoo.datasetsimportBaseDataset, HC3Datasetfromdetectzoo.benchmarksimportBenchmarkEvaluator# Built-in benchmark datasetdataset=HC3Dataset(subsets=["finance"])
# Or load a dataset from two directoriesdataset=BaseDataset.from_directory("data/real/", "data/fake/")
# Or from a CSV (text modality)dataset=BaseDataset.from_csv("data/texts.csv", text_column="text", label_column="label")
# Evaluate detectorsevaluator=BenchmarkEvaluator(dataset)
evaluator.run_and_print([
load_detector("log_likelihood"),
load_detector("entropy"),
load_detector("fast_detectgpt"),
])

This prints a comparison table with accuracy, precision, recall, F1, and AUROC.

Metrics

The compute_metrics utility computes standard binary-classification metrics:

fromdetectzoo.utilsimportcompute_metricsmetrics=compute_metrics(
labels=[0, 0, 1, 1],
scores=[0.1, 0.3, 0.8, 0.9],
threshold=0.5,
)
# {'accuracy': 1.0, 'precision': 1.0, 'recall': 1.0, 'f1': 1.0, 'tpr': 1.0, 'fpr': 0.0, 'roc_auc': 1.0, 'pr_auc': 1.0, 'avg_precision': 1.0}

Design Philosophy

DetectZoo is built around three principles.

1. Reproducibility

Many detection methods are difficult to reproduce due to missing implementation details. DetectZoo provides clean and standardized implementations of published detectors with references to the original papers.

2. Accessibility

Users should not need to reimplement detectors. DetectZoo provides simple imports and unified interfaces. Loading any detector is a single function call.

3. Extensibility

Adding a new detector takes a single file. Subclass BaseDetector, implement predict, and register with a decorator:

fromdetectzoo.detectorsimportBaseDetectorfromdetectzoo.core.registryimportregister_detector@register_detector("my_detector")classMyDetector(BaseDetector):
modality="text"# or "image" or "audio"def__init__(self, threshold=0.5, device="cpu", **kwargs):
super().__init__(threshold=threshold, device=device, **kwargs)
defpredict(self, input_data):
# Your detection logic herescore=0.0returnself._make_result(score)

The detector is then immediately available via load_detector("my_detector"). See examples/custom_detector.py for a complete runnable example.

---

Examples

The examples/ directory contains runnable scripts grouped by modality. Most replication scripts download public benchmark data, run detectors with BenchmarkEvaluator, and write metrics under experiments/.

Getting started

Script	Description
custom_detector.py	Create, register, and use a toy custom text detector (word_length).

Replication scripts

Script	Description
text/ood_replicate.py	Replicate OOD paper baselines on the labeled RAID test split (default 1000 samples).
text/gecscore_replicate.py	Replicate GECScore baselines on released normal_data JSON files (per source × generator model).
text/imbd_replicate.py	Replicate ImBD baselines on released rewrite/paraphrase JSON (human original vs AI rewritten).
text/text_fluoroscopy_replicate.py	Replicate Text-Fluoroscopy baselines on processed JSON files from the authors' repo.
image/image_replicate.py	Run image detectors on built-in datasets (self_synthesis, aigcdetect, cnn_detection, genimage, univfd_diffusion) and save benchmark JSON.
audio/audio_replicate.py	Run audio detectors on built-in benchmarks (asvspoof2019, for, in_the_wild, deepfake_eval_2024) with balanced sampling.

Run from the project root:

# Quick start — no downloads
python examples/custom_detector.py
# Text replication (OOD on RAID)
python examples/text/ood_replicate.py --device cuda --max-samples 100
# Image replication
python examples/image/image_replicate.py \
--dataset self_synthesis \
--partitions AttGAN BEGAN \
--detectors cnnspot patchcraft univfd
# Audio replication
python examples/audio/audio_replicate.py --dataset in_the_wild --detectors rawnet2 aasist

---

Contributing

We welcome community contributions. You can contribute by:

• Adding new detectors (see the extensibility section above)
• Improving existing implementations
• Adding benchmark datasets
• Improving documentation
• Reporting issues and suggesting features