C2Rust — Fine-Tuned Qwen3.5-27B for C-to-Rust Translation

This repository contains the paper's execution-based benchmark and evaluation harness for moxin-org/C2Rust, a full-parameter BF16 fine-tune of Qwen/Qwen3.5-27B specialized for C-to-Rust program translation.

The model is trained with a three-stage curriculum: Rust-focused continued pretraining, debugging-aware supervised fine-tuning on Verus data, and task-specific supervised fine-tuning on paired C/Rust programs. Under the fixed five-seed evaluation protocol described below, the resulting 27B model reaches 87.30% success rate, compared with 72.30% for the untuned base model.

Paper

Fine-Tuning Qwen3-27B for C-to-Rust Code Translation: A Three-Stage Curriculum of Pretraining, Debugging-Aware SFT, and Task-Specific SFT

Pu Zhao, Changdi Yang, Yixiao Chen, Yi Gao, Yifan Cao, Haochen Zeng, and Yanzhi Wang.
Northeastern University · EmbodyX Inc · Aibao LLC

The paper title uses “Qwen3-27B”; the released source checkpoint documented in Section 3 is Qwen/Qwen3.5-27B.

Research highlights

Result	Value
C2Rust Success Rate (five-seed mean)	87.30%
Gain over Qwen3.5-27B base	+15.00 percentage points
SWE-bench Verified pass@1	70.6
Model size	27B dense parameters

• The fine-tuned 27B model outperforms Qwen3.5-Plus (77.20%), MiniMax-M2.5 (83.90%), and GLM-5 (84.40%) on the same C2Rust evaluation.
• It remains below GLM-5.2 (89.90%) and Claude Code-4.6 (90.01%).
• On SWE-bench Verified, it scores 70.6 versus 72.4 for the untuned base, indicating a modest specialization cost in general software-engineering capability.

Three-stage training curriculum

Stage	Objective	Training data	Configuration
1. Rust continued pretraining	Strengthen Rust syntax, idioms, completion, repair, and standard-library knowledge	1,673,289 examples from seven Rust-focused sources	Full-parameter BF16, 1 epoch, LR 1e-6
2. Debugging-aware SFT	Learn to consume structured verifier feedback and make targeted repairs	microsoft/Verus_Training_Data	Full-parameter BF16, 2 epochs, LR 2e-7
3. C2Rust task SFT	Learn direct C-to-Rust semantic translation	C2Rust-Moxin functions/ and programs/ pairs	Full-parameter BF16, 2 epochs, LR 2e-7

All stages use a 16,384-token sequence length and DeepSpeed ZeRO Stage 3 on eight NVIDIA B300 GPUs. Training is text-only: the Qwen3.5 vision encoder remains in the checkpoint but is not used by the curriculum or evaluation.

Evaluation artifact

The benchmark contains 200 C programs: 92 take command-line arguments and 108 read standard input. Each generated Rust translation is compiled and run against the reference C program's own test inputs. A problem passes only when the Rust program reproduces every expected output within a six-attempt translation and repair budget.

Decoding is fixed for every model (temperature 0.6, top-p 0.95, top-k 20, maximum output length 1,536). Because generation is stochastic, the reported score is the arithmetic mean of five seeds; scripts/aggregate.py also reports the standard deviation and a problem-level bootstrap confidence interval.

The paper evaluates with SACTOR's verification-driven methodology. The default configurations shipped in this repository run its interface-preserving, unidiomatic translation stage, which may retain unsafe Rust; success therefore measures executable behavioral agreement, not memory safety or formal semantic equivalence.

---

How one problem is evaluated

raw_data/<mode>/<problem>.c the reference C program
│
│ run_eval.py tells the prompt builder whether input arrives via argv or stdin
│ (SACTOR_IO_MODE), so the model is told the I/O convention instead of guessing
▼
sactor translate --unidiomatic-only ──► Rust translation (single stage, unsafe Rust)
│
│ compile; on failure the engine retries with the compiler error as feedback,
│ up to max_translation_attempts (6 in the shipped configs)
▼
test_tasks/<mode>/<problem>.c.json one test command per problem
│
│ run the Rust binary on the inputs in generated_tests/, diff against the
│ expected output captured from the reference C program
▼
status.json ──► {"returncode": 0, "status": "success", "reason": "success", ...}

--unidiomatic-only means one translation stage: preserve the C interface, produce compiling Rust that behaves identically. The engine's second ("idiomatic") stage is not part of this benchmark.

Repository layout

CodeNet/ the dataset, self-contained (see CodeNet/README.md)
raw_data/{argv,scanf}/ 200 reference C programs (92 argv + 108 scanf)
generated_tests/{argv,scanf}/ input / expected-output cases
test_tasks/{argv,scanf}/ the test command per problem
manifest.json problem list, metric, decoding settings
scripts/run_eval.py the runner: translate all 200 in parallel, write per-problem status
scripts/aggregate.py pass@1 mean ± std + bootstrap CI across seeds
scripts/launch_model.sh serve a local checkpoint with sglang
configs/generic_prompt.toml harness + decoding config, generic translation prompt
configs/native_prompt.toml same, SFT-native prompt (for instruction-tuned checkpoints)
configs/api_openrouter.toml same, against an API endpoint instead of a local server
run_5seed.sh driver: 5 seeds against one model, then aggregate
fix_paths.sh one-time fixup after cloning (see step 1)
engine/ the SACTOR translation engine, vendored as source

---

Setup (once per machine)

1. Re-point the tests

Every CodeNet/test_tasks/*.json embeds an absolute path to its generated_tests counterpart. They ship as /__DATASET__/... placeholders, so rewrite them to your checkout:

bash fix_paths.sh # prints: re-pointed 200 test_tasks to <your path>

Skip this and every problem fails with Invalid test command.

2. Build the engine

engine/ ships as source only (no venv, no compiled artifacts — same policy as model weights):

cd engine
uv sync # creates engine/.venv from uv.lock
./update_rust_ast_parser.sh # builds the rust_ast_parser extension module
cargo build --release # rust_ast_parser + sactor_proc_macroscd ..

The Rust toolchain version is pinned in engine/rust-toolchain.toml.

3. Check the tools SACTOR requires on PATH

The engine calls check_all_requirements() at import and raises OSError: Missing requirements unless all four resolve — crown, rustup, c2rust, and gcc or clang:

fortin crown rustup c2rust gcc;doprintf'%-8s %s\n'"$t""$(command -v $t||echo MISSING)";done

run_eval.py prepends $SACTOR_HOME/.venv/bin, ~/.local/bin, $SACTOR_HOME/crown/target/release and ~/.cargo/bin to PATH, so a symlink in ~/.local/bin is enough for anything built elsewhere.

4. Point at the engine

export SACTOR_HOME=$PWD/engine # default; set it elsewhere to use another install

SACTOR_HOME is both where the sactor CLI is found ($SACTOR_HOME/.venv/bin/sactor) and the working directory used for every translate call.

See SETUP.md for the serving stack, the environment variables launch_model.sh sets and why, and the failure modes that look like model quality but are not.

---

Running

Step 1 — serve the model

Local checkpoint:

export SERVE_VENV=/path/to/your/sglang-venv
./scripts/launch_model.sh /path/to/checkpoint 0,1 30878 2

Arguments are MODEL_PATH GPUS [PORT] [TP] [CHAT_TEMPLATE]. Loading a large checkpoint takes a while; wait for health 200 before starting an eval:

curl -s -o /dev/null -w "%{http_code}\n" localhost:30878/health # want 200

The port must match api_base in the config you evaluate with (the shipped configs use 30878). Note the ptxas= field in the launcher's banner — see SETUP.md §3 for why it matters.

API endpoint instead: edit configs/api_openrouter.toml to set the model id, then export OPENROUTER_API_KEY=.... No server, no GPU.

Step 2 — smoke-test 2 problems first

Do this every time you move to a new machine. It separates environment failures from model failures, which otherwise look identical in a 200-problem log:

python3 scripts/run_eval.py configs/native_prompt.toml results/_smoke \
--modes argv --limit 2 --workers 1

You want reason=success. Anything else — read results/_smoke/argv/*/translate.log and fix the environment before launching the full set. Then rm -rf results/_smoke.

Step 3 — the full run, 5 seeds

PORT=30878 CFG=configs/native_prompt.toml TAG=mymodel ./run_5seed.sh

The driver waits for /health, runs 5 seeds sequentially, writes one log per seed under results/_driverlogs/, and aggregates at the end. For an API-hosted model there is no server to wait for:

SKIP_HEALTH=1 CFG=configs/api_openrouter.toml TAG=mymodel_api ./run_5seed.sh

Or call the runner directly, if you want control over the loop:

forsin 1 2 3 4 5;do
python3 scripts/run_eval.py configs/native_prompt.toml results/eval_mymodel_s$s \
--root ./CodeNet --modes argv,scanf --workers 4
done

--workers 4 is the default for good reason — see Operational rules.

Step 4 — aggregate

python3 scripts/aggregate.py results eval_mymodel

Output format (numbers below are illustrative, not a result):

eval_mymodel: pass@1 = NN.N +- N.N (per-seed ['NN.N', 'NN.N', 'NN.N', 'NN.N', 'NN.N']; n=200, 5 seeds)
95% CI (bootstrap by problem): [NN.N, NN.N]

It reads results/<prefix>_s*/ and intersects the problem sets across seeds, so a partially finished seed lowers n rather than silently skewing the mean. Check that n=200.

---

Reading the output

results/
_driverlogs/eval_mymodel_s1.log driver log: one line per finished problem
eval_mymodel_s1/
summary.json totals + reason histogram + every per-problem record
argv/codenet_argv_001.c/
status.json the verdict for this problem
translate.log full stdout/stderr of the sactor run ← start here
translated_code_unidiomatic/
combined.rs the Rust the model produced
functions/, clippy_stat.json
llm_stat_unidiomatic.json attempts / token counts
unidiomatic_failure_info.json per-function status report
logs/*.jsonl prompt / response trace
config.json the config as resolved for this run
scanf/codenet_scanf_001.c/ ...

unidiomatic_failure_info.json is not a failure marker — it is written for successful problems too (with "status": "success" inside). Judge pass/fail by status.json only.

status.json is six fields:

{"mode": "argv", "file": "codenet_argv_001.c", "returncode": 0,
"status": "success", "reason": "success", "time_sec": 30.7}

reason classifies what happened, which is where the useful signal lives:

reason	what it means
success	compiled and reproduced the reference output
max_attempts_unidiomatic	never produced a passing translation within max_translation_attempts — the normal way a weaker model fails
max_attempts_idiomatic	same, in the idiomatic stage (not used by the shipped configs)
timeout	the harness killed the whole process tree after --timeout
crash	engine traceback — most often the model server was unreachable
no_test_task	Invalid test command — fix_paths.sh was not run
struct_not_found, circular_deps	static-analysis dead ends on that C source
other	unclassified; read translate.log

A quick histogram of a finished seed:

python3 -c "import json;print(json.load(open('results/eval_mymodel_s1/summary.json'))['reasons'])"

Re-running

Runs resume: a problem that already has a status.json is skipped, so re-running the same command only fills in gaps. To redo work, delete it first:

rm -rf results/eval_mymodel_s1/argv/codenet_argv_042.c # one problem
rm -rf results/eval_mymodel_s1 # a whole seed

To re-run a specific subset, list mode/file entries one per line and pass --only:

printf'argv/codenet_argv_042.c\nscanf/codenet_scanf_007.c\n'> /tmp/redo.txt
python3 scripts/run_eval.py configs/native_prompt.toml results/eval_mymodel_s1 --only /tmp/redo.txt

Environment variables

var	used by	meaning
SACTOR_HOME	run_eval.py	engine install holding .venv/bin/sactor; also the cwd for each translate call. Default <repo>/engine
SERVE_VENV	launch_model.sh	sglang serving venv (required)
CUDA_DIR	launch_model.sh	toolkit to borrow ptxas from; default /usr/local/cuda-13.0, used only if present. Set empty to skip
CFG, TAG	run_5seed.sh	config file and results-name prefix (both required)
PORT, SEEDS, WORKERS	run_5seed.sh	defaults 30878, 1 2 3 4 5, 4
SKIP_HEALTH=1	run_5seed.sh	skip the server health wait (API-hosted models)
BENCH_DIR	run_5seed.sh	benchmark root; defaults to the script's own directory
OPENROUTER_API_KEY	API config	read as os.environ/OPENROUTER_API_KEY

run_eval.py --help lists its own flags (--root, --sactor-home, --workers, --timeout, --modes, --only, --limit).

Operational rules

• Keep total concurrent workers ≤ ~15 across every job on the machine; watch cat /proc/loadavg. High parallelism spawns many rustc processes and can orphan looping test binaries → CPU overload → false timeouts that silently corrupt results. run_eval.py caps rustc with CARGO_BUILD_JOBS=1 and kills the whole process group on timeout, but still watch the load.
• Always average 5 seeds. One run is worth ±2–3 points; a single run is not a result.
• Never compare across settings.test_pass_threshold, CoT, thinking mode and max_tokens each move the number by several points. The shipped configs pin them (threshold 1, CoT off, thinking off, max_tokens=1536); change one and you can only compare within your own change.
• A config does not name a model.generic_prompt.toml and native_prompt.toml differ only in prompt mode; what you are measuring is decided by whichever checkpoint is served on the port. Keep the config fixed and swap the server.
• Prompt mode matters.sft_native_prompt=false is the generic prompt; true is the SFT-native prompt that instruction-tuned checkpoints expect. Using the wrong one is worth many points.
• run_eval.py sets CARGO_NET_OFFLINE=true, since compute nodes often have no network.

Troubleshooting

symptom	cause	fix
every problem reason=no_test_task	fix_paths.sh never ran	run it
OSError: Missing requirements	crown / rustup / c2rust / gcc not all on PATH	setup step 3
FileNotFoundError: 'sactor'	SACTOR_HOME wrong, or engine/.venv not built	setup steps 2 and 4
every problem reason=crash, log says Connection error	nothing serving on the port in the config's api_base	start the server, check /health
server starts, then dies on the first request	Triton's bundled ptxas does not support your GPU	use launch_model.sh; check its ptxas= banner and SETUP.md §3
many reason=timeout	CPU overload from too many workers	lower --workers, check /proc/loadavg
n= below 200 in the aggregate	a seed did not finish	re-run that seed; it resumes
all reason=max_attempts_unidiomatic with garbage in combined.rs	prompt mode mismatch, or a checkpoint that degenerates at this temperature	try the other config; check translated_code_unidiomatic/combined.rs

Credits & licenses

• Translation engine — engine/ is SACTOR by Tianyang Zhou et al. (paper), Apache-2.0, vendored as source with local modifications for this benchmark (the argv/stdin I/O-prompt handling, feedback and timeout knobs). Upstream license retained at engine/LICENSE; see NOTICE.
• Dataset — derived in part from IBM Project CodeNet (CDLA-Permissive-2.0). See CodeNet/README.md for provenance and terms.
• This repository — Apache-2.0, see LICENSE.

Citation

@article{zhao2026c2rust,
title = {Fine-Tuning Qwen3-27B for C-to-Rust Code Translation: A Three-Stage Curriculum of Pretraining, Debugging-Aware SFT, and Task-Specific SFT},
author = {Zhao, Pu and Yang, Changdi and Chen, Yixiao and Gao, Yi and Cao, Yifan and Zeng, Haochen and Wang, Yanzhi},
journal = {arXiv preprint arXiv:2608.13681},
year = {2026},
doi = {10.48550/arXiv.2608.13681}
}