CulTrace

Codebase for CulTrace: Tracing Internal Cultural Reasoning in Large Language Models

CulTrace is built on LatentQA (Pan, Chen & Steinhardt, 2024) and branched from it at commit a2dcb6f. See Relationship to LatentQA.

---

Repository layout

lit/ LatentQA, inherited. Decoder training and activation plumbing.
Six files are modified; see NOTICE and `git diff a2dcb6f -- lit/`.
cultrace/ New in CulTrace. The layer-wise decoding pipeline.
scripts/ Runnable entry points, grouped by stage.
data/ CulTrace data + the LatentQA decoder training sets.
docs/ LatentQA's original README, kept for provenance.

cultrace/

Module	What it does
cultrace_main.py	The layer sweep. Decodes a target model's activations at every layer, swapping one per-layer adapter at a time so the whole sweep fits in 23 GB.
eval.py	Decoder perplexity on the LatentQA eval set, layer by layer.
feature_extraction.py	Factual-recall probe on the Patchscopes relations (LatentQA paper Sec 5.1).
hub_utils.py	Resolves decoder adapters to their Hugging Face Hub repo.

All three entry points load decoders from the Hub — one repo per target model, one read{L}/ subfolder per layer. Nothing needs downloading by hand:

--decoder_owner copenlu # repo derived from --target_model_name
--decoder_repo_id <owner>/<repo># a specific layer-sharded repo
--decoder_model_name <id|path># a single adapter, Hub id or local dir

---

Setup

git clone https://github.com/copenlu/CulTrace
cd CulTrace
pip install -r requirements.txt

Every script reads its defaults from scripts/common.sh and can be overridden from the environment — nothing is hardcoded to a particular machine:

Variable	Default	Meaning
PYTHON / TORCHRUN	python / torchrun	Interpreters to use
VLLM_PYTHON	$PYTHON	Separate env for the vLLM data conversion
DATA_DIR	./data/external	Where the third-party corpora live
HF_TOKEN	~/.cache/huggingface/token	Needed for gated models (Llama-3, Gemma-3)

DATA_DIR=/scratch/corpora ./scripts/cultrace/cultrace_culture.sh

---

Data

CulTrace redistributes no third-party corpora. Fetch them from their original sources into $DATA_DIR:

./scripts/data/download_data.sh # all targets
./scripts/data/download_data.sh blend # or one at a time

The script documents each dataset's origin, license, and any derivation steps it can't automate. NOTICE records the licensing.

---

Usage

1. Train per-layer decoders

One LoRA adapter per (read layer, write layer) pair, on top of the target model. Train the layers you need:

./scripts/train/train_lit.sh 15 0 # read layer 15, write layer 0
NUM_GPUS=4 ./scripts/train/train_lit.sh 15 0

Or skip training entirely and use the released adapters from the Hub — the sweep and eval entry points load them by default.

2. Sweep layers

python -m cultrace.cultrace_main \
--dataset blend \
--decoder_repo_id copenlu/CulTrace-latentqa-decoder-llama-3-8b-instruct \
--min_layer_to_read 0 --max_layer_to_read 32 \
--start_num 0 --num_countries 5

Or via the script, sharded across GPUs by country batch:

START_NUM=0 ./scripts/cultrace/cultrace_culture.sh &
START_NUM=5 ./scripts/cultrace/cultrace_culture.sh &
START_NUM=10 ./scripts/cultrace/cultrace_culture.sh &wait

Results land in ./<dataset>_scope/<model>_<seed>/ as one CSV per country, named by --file_prefix (tracing_, context_, recovery_, …).

3. Evaluate

Decoder perplexity per layer, and the factual-recall probe:

./scripts/eval/eval_cultrace.sh
./scripts/eval/extract_cultrace.sh

---

Results

Decoding sweeps run to several GB of per-country, per-layer CSVs and are not in git (see .gitignore). They are released separately, and land under <dataset>_scope/.

The analysis code that turns those sweeps into the paper's figures — accuracy by culture, top-5 confusions, reasoning-depth curves, semantic similarity, and the recovery controls — is not part of this release.

---

Relationship to LatentQA

CulTrace is a derivative work of LatentQA, Apache-2.0, branched at a2dcb6f.

• Everything under lit/ originates in LatentQA. Six files are modified, to support target models beyond Llama-3 (Gemma-3, Qwen3, Ministral), per-layer decoder training, and system-prompt masking. NOTICE lists them; git diff a2dcb6f -- lit/ shows the exact changes.
• Everything under cultrace/ and scripts/ is new.

If you use the decoder training machinery, please cite LatentQA:

@article{pan2024latentqa,
title = {LatentQA: Teaching LLMs to Decode Activations Into Natural Language},
author = {Pan, Alexander and Chen, Lijie and Steinhardt, Jacob},
journal = {arXiv preprint arXiv:2412.08686},
year = {2024}
}

License

Two licenses, mapped file-by-file in NOTICE:

• lit/ and the other files inherited from LatentQA: Apache-2.0 (LICENSE.Apache-2.0)
• CulTrace's own code — cultrace/, scripts/, the CulTrace data files: MIT (LICENSE)

Third-party datasets keep their own licenses and are not redistributed here; data/candle_qa_full.json is a derived work of CANDLE and carries CC-BY-4.0.