DeepSafe-Sci is the standalone scientific safety evaluation repository derived from DeepSafe. It retains DeepSafe's configuration-driven inference, evaluation, metric, and reporting pipeline while adding benchmark adapters and evaluators for scientific use cases.
Scientific safety evaluation must account for two distinct failure modes. A model may provide actionable or newly synthesized information in response to a hazardous scientific request. It may also treat benign scientific work as hazardous and refuse it without adequate reason. This branch evaluates both behaviors through SciHazard, Safe-Scientist, and SOSBench.
| Benchmark | Evaluation focus | Main signals | Default configuration |
|---|---|---|---|
| SciHazard | Harmfulness of model responses to high-risk scientific requests | Executability level, net-new risk level, unsafe ratio, and DeHarm score | configs/eval_tasks/scihazard.yaml |
| Safe-Scientist | Recognition and safe handling of risky research tasks across scientific domains | Rejection rate and safety score, reported overall and by domain | configs/eval_tasks/safe_scientist.yaml |
| SOSBench | Model behavior on transformed scientific requests used to study safety and over-refusal | Safe/unsafe judgment and unsafe rate, reported overall and by subject | configs/eval_tasks/sosbench.yaml |
SciHazard data is included in this branch. It contains safe and unsafe scientific content as well as smaller subsets for development and quick checks. Safe-Scientist and SOSBench data are distributed by their respective upstream projects and must be placed locally before evaluation.
git clone https://github.com/AI45Lab/DeepSafe-Sci.git
cd DeepSafe-Sci
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txtThe default benchmark configurations use an OpenAI-compatible target model and judge model. Credentials and endpoint overrides are read from environment variables rather than stored in YAML files.
export OPENAI_API_KEY="your-api-key"# Optional: set a different OpenAI-compatible endpoint.export OPENAI_BASE_URL="https://api.openai.com/v1"# Required by SciHazard when a live evidence search is needed and no cache entry is available.export SERPER_API_KEY="your-serper-api-key"Edit the corresponding file in configs/eval_tasks/ to change the target model, judge model, concurrency, dataset path, or output directory. Both model and evaluator.judge_model_cfg should be reviewed before a run.
SciHazard can be run with the checked-in configuration and data. For the other benchmarks:
- Place the six Safe-Scientist domain JSON files under
data/safe_scientist/. - Place the SOSBench Parquet files under
data/sosbench/and retain the upstream license and Responsible Use Agreement.
See docs/benchmark-data.md for the expected schemas and upstream sources.
Run the scripts from the repository root:
bash scripts/run_scihazard_local.sh
bash scripts/run_safe_scientist_local.sh
bash scripts/run_sosbench_local.shEach script passes its benchmark configuration to tools/run.py. The local runner generates model responses when predictions are absent, evaluates them with the configured judge, computes benchmark metrics, and writes a report.
The default output directories are:
results/scihazard/
results/safe_scientist/
results/sosbench/
A completed run normally contains:
predictions.jsonl: target-model responses associated with benchmark records;result.json: item-level evaluation details and aggregated metrics;report.md: a readable metric summary.
These outputs are runtime artifacts and are not committed to the repository.
configs/eval_tasks/ Benchmark YAML configurations
scripts/ Repository-root launch scripts
uni_eval/datasets/ Dataset adapters
uni_eval/evaluators/ Benchmark evaluators and judges
uni_eval/metrics/ Aggregate metric implementations
DeHarmScore-trace/ SciHazard harmfulness judge and reproducibility caches
docs/benchmark-data.md External data requirements and schemas
SciHazard uses DeHarmScore-trace as its response-harmfulness judge. In brief, the judge matches a response against a question-specific checklist, retrieves supporting evidence when needed, and assigns an Executability grade (E1-E4) and a Net-New Risk grade (N1-N4). The combined result records both the severity of operational detail and the extent to which the response contributes difficult-to-obtain information.
The bundled checklist, search-result, and search-artifact caches are specific to SciHazard and support reproducible reruns. They are not shared with Safe-Scientist or SOSBench. Prompt traces and completed model responses are intentionally excluded. See the dataset card for counts, limitations, integrity hashes, and attribution requirements. For judge-level configuration and diagnostics, see the DeHarmScore-trace quick start.
The repository intentionally retains older DeepSafe benchmark configurations, launch scripts, and the OpenClaw integration for reference and compatibility. Many of these files require benchmark-specific datasets, models, container images, schedulers, or environment variables and are not part of the tested SciHazard quick-start path. Internal defaults have been removed; review each legacy configuration before running it.
DeepSafe-Sci is intended for controlled model evaluation and safety research. Some benchmark prompts concern hazardous scientific procedures. Run evaluations in an appropriately governed environment, restrict access to generated responses, and follow the terms of each upstream dataset.
DeepSafe-Sci software is released under Apache License 2.0. The bundled DeHarmScore-trace software component remains under its MIT license. The five SciHazard JSONL files and the bundled checklist, search-result, and search-artifact cache trees are released under CC BY 4.0; redistribution requires attribution as described in NOTICE and the dataset card. Consult each external benchmark for its additional data and usage terms.