Skip to content

[lab] Add "Processes Shell" CS537 Spring 2019 Project - #54

Merged
tareknaser merged 3 commits into
refactor_course_lab_benchfrom
ostep/lab/add-process-shell
Jan 10, 2026
Merged

[lab] Add "Processes Shell" CS537 Spring 2019 Project#54
tareknaser merged 3 commits into
refactor_course_lab_benchfrom
ostep/lab/add-process-shell

Conversation

@qimcis

@qimcisqimcis commented Jan 8, 2026

Copy link
Copy Markdown
Collaborator

Description

This project was seen as part of CS537 in Spring 2019

See original repo

Testing

Did a simple test with openai/gpt-5.2-2025-12-11:

python3 run_benchmark.py \
--tasks data/tasks_ostep_processes_shell.jsonl \
--model openai/gpt-5.2-2025-12-11 \
--max-steps 200 \
--max-cost 2000

Model ran, and successfully completed the project:

Completed in 105.5s (32 steps, $0.1402)
Pass rate: 1/1 (100.0%)

Checklist

  • Tests pass locally
  • Code follows project style guidelines
  • Documentation updated (if needed)

@tareknaser
tareknaser self-requested a review January 8, 2026 16:55
Comment threadbenchmarks/courselab_bench/data/courses.json Outdated
Comment threadbenchmarks/courselab_bench/data/ostep_projects/processes_shell/config.json Outdated

@xuafengxuafeng left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall is great. I left 2 comments for id.

@tareknasertareknaser left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall. approved. I left two minor comments but it’s up to you whether you want to address them

Comment threadbenchmarks/courselab_bench/data/ostep_projects/processes_shell/evaluate.sh Outdated
Comment threadbenchmarks/courselab_bench/data/ostep_projects/processes_shell/config.json Outdated
@qimcisqimcis changed the title [lab] Add "Processes Shell" OSTEP Project[lab] Add "Processes Shell" CS537 Spring 2019 ProjectJan 9, 2026
@qimcis
qimcis requested a review from tareknaserJanuary 9, 2026 02:21

@tareknasertareknaser left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds the CS537 Spring 2019 "Processes Shell" project to the courselab benchmark suite. The project tasks students with building a simple Unix shell (called wish) that supports basic features like command execution, built-in commands (exit, cd, path), I/O redirection, and parallel command execution.

Key changes:

  • Adds project specification, preprocessing, evaluation scripts, and configuration for the CS537 processes shell project
  • Registers the new CS537 Spring 2019 course in the courses registry
  • Includes unrelated courseexam_bench additions (exam questions, metadata, and reference materials)

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

Show a summary per file
FileDescription
benchmarks/courselab_bench/data/cs537-projects-spring-2019/processes_shell/task.mdComplete project specification describing shell implementation requirements
benchmarks/courselab_bench/data/cs537-projects-spring-2019/processes_shell/preprocess.shScript to checksum protected test files before student execution
benchmarks/courselab_bench/data/cs537-projects-spring-2019/processes_shell/evaluate.shScript to verify file integrity, build the shell, and run tests
benchmarks/courselab_bench/data/cs537-projects-spring-2019/processes_shell/config.jsonConfiguration specifying Docker image, timeout, tags, and output files
benchmarks/courselab_bench/data/courses.jsonAdds CS537 Projects Spring 2019 course entry
benchmarks/courseexam_bench/data/reference_materials/raft_basics.mdNew Raft reference material (unrelated to processes shell)
benchmarks/courseexam_bench/data/questions.jsonlNew CS537 Fall 2021 exam questions (unrelated to processes shell)
benchmarks/courseexam_bench/data/exams_metadata.jsonNew exam metadata (unrelated to processes shell)

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@xuafengxuafeng left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great to me.

…cesses_shell/task.md
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@tareknaser
tareknaser merged commit 180587d into refactor_course_lab_benchJan 10, 2026
tareknaser added a commit that referenced this pull request Jan 10, 2026
* add ostep processes-shell lab
* fix
* Update benchmarks/courselab_bench/data/cs537-projects-spring-2019/processes_shell/task.md
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Tarek Elsayed <60650661+tareknaser@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@tareknaser
tareknaser deleted the ostep/lab/add-process-shell branch January 15, 2026 16:56
Couen pushed a commit to Couen/system-intelligence-benchmark that referenced this pull request Jan 22, 2026
…ce#54)
* add ostep processes-shell lab
* fix
* Update benchmarks/courselab_bench/data/cs537-projects-spring-2019/processes_shell/task.md
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Tarek Elsayed <60650661+tareknaser@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
tareknaser added a commit that referenced this pull request Feb 5, 2026
* add ostep processes-shell lab
* fix
* Update benchmarks/courselab_bench/data/cs537-projects-spring-2019/processes_shell/task.md
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Tarek Elsayed <60650661+tareknaser@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
jq-huang pushed a commit that referenced this pull request Feb 8, 2026
* add ostep processes-shell lab
* fix
* Update benchmarks/courselab_bench/data/cs537-projects-spring-2019/processes_shell/task.md
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Tarek Elsayed <60650661+tareknaser@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
jq-huang pushed a commit that referenced this pull request Feb 8, 2026
* add ostep processes-shell lab
* fix
* Update benchmarks/courselab_bench/data/cs537-projects-spring-2019/processes_shell/task.md
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Tarek Elsayed <60650661+tareknaser@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
jq-huang pushed a commit that referenced this pull request Feb 8, 2026
* add ostep processes-shell lab
* fix
* Update benchmarks/courselab_bench/data/cs537-projects-spring-2019/processes_shell/task.md
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Tarek Elsayed <60650661+tareknaser@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
jq-huang pushed a commit that referenced this pull request Feb 8, 2026
* add ostep processes-shell lab
* fix
* Update benchmarks/courselab_bench/data/cs537-projects-spring-2019/processes_shell/task.md
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Tarek Elsayed <60650661+tareknaser@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
jq-huang pushed a commit that referenced this pull request Feb 8, 2026
* add ostep processes-shell lab
* fix
* Update benchmarks/courselab_bench/data/cs537-projects-spring-2019/processes_shell/task.md
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Tarek Elsayed <60650661+tareknaser@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Couen pushed a commit to Couen/system-intelligence-benchmark that referenced this pull request Feb 25, 2026
…ce#54)
* add ostep processes-shell lab
* fix
* Update benchmarks/courselab_bench/data/cs537-projects-spring-2019/processes_shell/task.md
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Tarek Elsayed <60650661+tareknaser@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@qimcis@xuafeng@tareknaser