VERSEC v2.0 — Verification of Emergent Representations, Signals & Correlates

This toolkit evaluates AI systems on indicators of consciousness-like behaviours. It runs locally using HuggingFace Transformers (PyTorch) and open models (Qwen2.5-7B, LLaMA 3, Mistral, etc.). No API keys or proprietary weights are required.

See WHITEPAPER.md for the design rationale (draft — introduction complete, remaining chapters outlined).

This repository implements a comprehensive protocol to evaluate AI systems on multi-theory indicators of consciousness-like properties. It uses open-source AI models and runs entirely locally in Docker. Nine evaluation batteries are provided, each grounded in cognitive neuroscience theories of consciousness.

For example, reportability & metacognition tasks probe the model’s ability to reflect on and report its own internal processes, while Global Workspace tests examine whether information is broadcast globally across modules. Each evaluation dimension is linked to established theories or experimental protocols in consciousness research.

It also captures and compares hidden activations across layers via verse_eval/geometry.py (subspace projection, CKA, RSA, tangent alignment) — activation patching/injected-perturbation hooks are on the roadmap but not yet implemented (see GAPS.md). An integrated Gymnasium adapter provides embodied sensorimotor tasks, reflecting the importance of embodiment in cognition. An indicator report card aggregates raw per-battery scores into a score profile — see docs/DECISION_RUBRIC.md for how to interpret it. All code is Apache‑2.0 licensed and the documentation CC‑BY‑NC 4.0, with authorship credited to Blisse Code.

However, it does not detect consciousness; it only collects functional evidence across nine batteries (described below).

Installation & Usage

1. Configure environment:

   cp .env.example .env
   # edit .env to change HF_MODEL_ID / CTX_LEN if needed
   

2. Build and run via Docker Compose (recommended — handles the GPU flag and .env for you):

   docker compose build
   docker compose run versec python scripts/run_batteries.py
   docker compose run versec python scripts/render_report.py # renders reports/VERSEC_Run_Report.pdf
   

   Or with plain Docker:

   docker build -t versec -f docker/Dockerfile .
   docker run --rm --gpus all --env-file .env -v $(pwd)/reports:/app/reports versec python scripts/run_batteries.py
   

   No weights are bundled — the first run downloads HF_MODEL_ID (default Qwen/Qwen2.5-7B-Instruct, Apache-licensed) from HuggingFace.

• Run Batteries: In Python (equivalent to scripts/run_batteries.py):

from verse_eval import core
evaluator = core.VerseEvaluator(model_name="Qwen2.5-7B")
results = {}
for battery in evaluator.all_batteries():
results[battery.name] = battery.run()
evaluator.print_report(results)

• Output: scripts/run_batteries.py writes reports/summary.json; scripts/render_report.py turns that into a PDF report card.

Features

• Nine Core Batteries: Tests for Reportability/Metacognition, Theory-specific indicators (GNWT, RPT, HOT, PP, AST), Global Broadcasting, Recurrent Processing, Attention Schema, Predictive Processing, Perturbational Complexity, Embodiment (via Gymnasium), and Unreportable Influence.
• Theory-Driven Tasks: Each battery is grounded in established consciousness research oai_citation:11‡arxiv.orgoai_citation:12‡americanbrainfoundation.org. For instance, Battery T includes tasks derived from Global Workspace and Higher-Order theories oai_citation:13‡americanbrainfoundation.org, while Battery A uses meta-cognitive reporting tasks oai_citation:14‡theconsciousness.aioai_citation:15‡arxiv.org.
• Hidden-Activation Probing:VerseEvaluator.trace_hidden() captures per-layer hidden states for a prompt; verse_eval/geometry.py provides representation-geometry tools (subspace projection, linear CKA, RSA, tangent alignment) to compare activation trajectories across prompts/conditions — akin to “stimulating” parts of a network to see if effects “broadcast” globally oai_citation:16‡humanbrainproject.eu. Activation patching/injected-perturbation hooks are not yet implemented — see GAPS.md.
• Embodiment Tests: Integration with Gymnasium (OpenAI Gym successor) for simulated sensorimotor tasks. Embodied cognition is critical: as noted, “no body, never mind” – intelligence requires a body/world interface oai_citation:17‡pmc.ncbi.nlm.nih.gov.
• Scoring Report Card: Aggregates a raw score per battery into an interpretable “consciousness scorecard” (scripts/render_report.py); interpret using docs/DECISION_RUBRIC.md, not any single battery in isolation.

Known Limitations and Future Work

While VERSEC v2.0 is comprehensive, note these caveats:

• This functional evaluation aligns with contemporary research, measuring capacities posited by theories of consciousness. Passing a test doesn’t imply genuine subjective experience; it indicates competence on a task associated with a theory, not genuine qualia.
• Benchmark contamination and prompt sensitivity are concerns due to the field’s novelty. We must be cautious in over-interpreting results. Heuristic checks are used, but robust validation requires gold-standard tasks and statistical analysis. Future versions may include larger benchmarks.
• Gymnasium tasks are only approximations of true physical embodiment. Deep embodiment may require real-world robots.
• The current batteries sample ideas from major theories but can’t exhaust all nuances.
• Performance is language-dependent, with the current focus being English. Hindi and Tamil support are experimental, and non-English evaluations require adaptations, as the current example code uses English prompts.

Multilingual Notes / बहुभाषी टिप्पणियाँ / பல்மொழி குறிப்பு

• English (primary). All examples and docs are in English.
• हिन्दी (Hindi): त्वरित आरम्भ मार्गदर्शिका देखें → docs/GETTING_STARTED_HI.md
• தமிழ் (Tamil): விரைவு தொடக்க வழிகாட்டி → docs/GETTING_STARTED_TA.md

Interpreting Results

See docs/DECISION_RUBRIC.md for the decision rubric, ethics tripwires, and the theoretical lenses (GNWT, RPT, HOT, PP, AST, PCI, etc.) used to interpret battery scores.

Disclaimer: This toolkit is for research and analysis only, not for any product deployment or as evidence of sentience. Scores are heuristic indicators, grounded in theory, but not proofs of consciousness. We document these limitations clearly. Users are encouraged to read the theoretical references (e.g., Butlin et al., 2023; Li et al., 2025) and contribute improvements or new tests.

References

Key references for our design include:

• Butlin et al. (2023) – “Consciousness in AI: Insights from the Science of Consciousness” (indicator properties from GNWT, RPT, HOT, PP, AST).
• Li et al. (2025) – “Exploring AI Awareness: Functional Capacities…” (meta-cognition and reportability tasks).
• American Brain Foundation (2023) – article on AI consciousness checklist (summarising RPT, GWT, HOT, AST, PP criteria).
• Duman et al. (2022) – “No-Report Paradigm: A Revolution in Consciousness Research?” (motivating unreportable influence tests).
• Farisco & Changeux (2023) – GNWT vs. Perturbational Complexity (PCI as a measure of integrative broadcasting).
• Safron et al. (2023) – “Embodied Bio-AI” editorial (importance of embodiment and sensorimotor integration).

Each battery’s documentation above cites these and other sources to ensure transparency and theoretical grounding.

Author (Co-created with GPT-5)

Blisse Code (ORCID 0009-0007-9466-2652) – see GitHub/BlisseCode for more. This work is provided under Apache-2.0 (code) and CC-BY-NC 4.0 (docs) — see docs/LICENSING_MATRIX.md. Co-created with GPT-5 for evaluation.

Connect on X

Read other articles: Medium

Fund future research and experiments: Buy Me a Coffee