Skip to content

Improving the "contributor's guide" and simplifying the benchmark's schema - #21

Merged
xuafeng merged 14 commits into
sys-intelligence:mainfrom
bastoica:main
Dec 1, 2025
Merged

Improving the "contributor's guide" and simplifying the benchmark's schema#21
xuafeng merged 14 commits into
sys-intelligence:mainfrom
bastoica:main

Conversation

@bastoica

@bastoicabastoica commented Nov 20, 2025

Copy link
Copy Markdown
Collaborator

Description

This PR improves the step-by-step "contributor's guide" by simplifying the text, explaining what to change, enhance and how to add a new artifact (with code snippets), as well as simplifying the benchmark's schema.

Changes

  • A simplified README.md with more details about what to modify and extend when adding a new artifact to ArtEvalBench
  • A new WHY.md which contains general background about the artifact evaluation (AE) process
  • A more descriptive benchmark schema

Testing

Tested locally on a single artifact.

Checklist

  • Tests pass locally
  • Code follows project style guidelines
  • Documentation updated (if needed)

@bastoicabastoica self-assigned this Nov 20, 2025
@bastoicabastoica added documentation Improvements or additions to documentation enhancement New feature or request labels Nov 20, 2025
This was linked to issues Nov 20, 2025
Improve the step-by-step guidelines vor ArtEvalBench by showing a walkthrough example #17
Closed
Add a simpler artifact to ArtEvalBench as an example #18
Closed
@bastoica
bastoica marked this pull request as ready for review November 20, 2025 21:32
tareknaser
tareknaser reviewed Nov 21, 2025
View reviewed changes

@tareknasertareknaser left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the nice work! I just have a few minor comments and nitpicks

Comment threadbenchmarks/arteval_bench/WHY.md
Comment threadbenchmarks/arteval_bench/README.md Outdated
Comment threadbenchmarks/arteval_bench/README.md Outdated
Comment threadbenchmarks/arteval_bench/README.md Outdated
Comment threadbenchmarks/arteval_bench/README.md
Comment threadbenchmarks/arteval_bench/WHY.md Outdated
Comment threadbenchmarks/arteval_bench/WHY.md Outdated
Comment threadbenchmarks/arteval_bench/WHY.md Outdated
Comment threadbenchmarks/arteval_bench/WHY.md Outdated
@bastoica
bastoica marked this pull request as draft November 25, 2025 03:40
@tareknasertareknaser mentioned this pull request Nov 25, 2025
Open
@ygxioygxio mentioned this pull request Nov 27, 2025
Open
3 tasks
xuafeng
xuafeng approved these changes Dec 1, 2025
View reviewed changes

@xuafengxuafeng left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes look good, especially the WHY.md part. Let me merge it.

@xuafeng
xuafeng requested review from tareknaser and removed request for tareknaserDecember 1, 2025 18:08
@bastoica
bastoica marked this pull request as ready for review December 1, 2025 18:24
@xuafeng
xuafeng merged commit 13c5fa1 into sys-intelligence:mainDec 1, 2025
2 checks passed
@bastoicabastoica mentioned this pull request Dec 6, 2025
Merged
@tareknasertareknaser mentioned this pull request Dec 12, 2025
Merged
Couen pushed a commit to Couen/system-intelligence-benchmark that referenced this pull request Jan 22, 2026
@xuafeng
Merge pull requestsys-intelligence#21from bastoica/main
9c089bb
Improving the "contributor's guide" and simplifying the benchmark's schema
tareknaser pushed a commit that referenced this pull request Feb 5, 2026
@xuafeng
Merge pull request#21from bastoica/main
586ca1c
Improving the "contributor's guide" and simplifying the benchmark's schema
Couen pushed a commit to Couen/system-intelligence-benchmark that referenced this pull request Feb 25, 2026
@xuafeng
Merge pull requestsys-intelligence#21from bastoica/main
49fed57
Improving the "contributor's guide" and simplifying the benchmark's schema
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@tareknasertareknasertareknaser left review comments
@xuafengxuafengxuafeng approved these changes

Assignees

@bastoicabastoica

Labels

documentationImprovements or additions to documentationenhancementNew feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Add a simpler artifact to ArtEvalBench as an example Improve the step-by-step guidelines vor ArtEvalBench by showing a walkthrough example

3 participants

@bastoica@xuafeng@tareknaser