ACToR

ACToR is an Adaptive Critical Token-aware Retrieval framework designed for repository-level code generation tasks.

🎯 Overview

---

📁 Project Structure

ACToR/
├── README.md # Project documentation
├── classifiers/ # Critical Token classifiers ├── weights/ # Position-aware weights
├── datasets/ # Benchmarks and training datasets
├── repositories/ # Code repositories for training and testing
└── src/ # Source code directory
├── pipeline.py
├── critoken.py
├── train.py
├── data/ # Data processing pipeline
│ ├── __init__.py
│ ├── repo.py
│ ├── task.py
│ └── process/ # Data processing modules
│ ├── __init__.py
│ ├── data.py
│ ├── window.py
│ ├── vector.py
│ ├── search.py
│ ├── prompt.py
│ └── utils.py
└── server/ # Model server integration
├── __init__.py
├── classifier.py
└── llm.py

---

🛠️ Quick Start

1. Prerequisites & Installation

Requirements:

• Python 3.12.11 (recommended)

Installation:

pip install -r requirements.txt

2. Repository Setup

Organize code repositories in the repositories/ directory following this structure:

repositories/
├── codereval/
│ └── python/ # CoderEval Python repositories
├── repoexec/
│ └── test-app/ # RepoExec test repositories
└── repost/
└── train/ # RepoST training repositories

Setup Instructions:

1. CoderEval: Clone CoderEval Python repositories to repositories/codereval/python/
2. RepoExec: Clone RepoExec test repositories to repositories/repoexec/test-app/
3. RepoST: Clone RepoST training repositories to repositories/repost/train/ (for training reproduction)

Note: More Details are available in src/data/process/utils.py

⚡ Usage

1. Repo Context Prep

python src/pipeline.py repo --benchmark 'repoexec'# alternative 'codereval-python', 'repost_train'

2. (Optional) Training Data Prep

python src/pipeline.py train --task_type 'data' --model_name 'model_name'# e.g. codellama-7b-hf

3. (Optional) Classifier Training

python src/pipeline.py train --task_type 'classifier' --model_name 'model_name'# e.g. codellama-7b-hf

4. Run Adaptive Critical Token-aware Retrieval augmented generation.

python src/pipeline.py task token --model_name 'model_name' --benchmark 'repoexec'