Skip to content

feat: select few-shot examples by dataset ids - #1381

Open
foxboy66 wants to merge 1 commit into
huggingface:mainfrom
foxboy66:codex/lighteval-634-fewshot-ids
Open

foxboy66 wants to merge 1 commit into
huggingface:mainfrom
foxboy66:codex/lighteval-634-fewshot-ids

Conversation

@foxboy66

@foxboy66 foxboy66 commented Sep 8, 2026

Copy link
Copy Markdown

Closes #634.

Adds few_shots_id_column and few_shots_id_list, filters few-shot documents by matching IDs, validates paired configuration, and adds a focused test.

Validation: git diff --check and Python syntax compilation passed. Targeted pytest could not collect on this Windows environment because datasets is unavailable; upstream README notes Windows is not currently supported.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FT] Build in a way to specify specific IDs/Lines in Dataset to use as few-shot examples in the same split

1 participant