Testing

Testing

Beez uses Google Test and CTest. Tests are organized in layers; pick the lowest layer that can express the behavior, then add higher layers when components interact or user-visible behavior matters.

Running tests

make test# Release build tests
make test BUILD_TYPE=Debug # Debug build tests

Output is tee'd to report/test/test-report.txt.

Run a single test binary directly:

cd build/build/Release
./bin/beez_tests --gtest_filter=Registry.*
ctest --output-on-failure -R beez_integration

Test layers

Layer	Location	When to add
Unit	tests/unit/	Pure logic, parsers, options, cache math
Integration	tests/integration/	Orchestrator + plugins, CLI as subprocess
System	tests/system/scenarios/	End-to-end with fixture build.lua projects
Performance	tests/performance/	Throughput or timing regressions
Fuzz	tests/fuzz/	DSL parser robustness

Unit tests

• Executable: beez_tests
• Mirror src/ folders: tests/unit/core/, tests/unit/cli/, tests/unit/plugins/lua/, etc.
• Cover positive and negative cases (invalid input, missing fields, unknown names).

Integration tests

• Executable: beez_integration_tests
• Uses real beez binary via BEEZ_EXECUTABLE compile definition
• Examples: Lua + shell pipeline, step cache pipeline, CLI completion install

System tests

• Fixture project per scenario under tests/system/fixtures/<name>/
• Scenario .cpp files under tests/system/scenarios/
• Run Beez against fixture build.lua and assert exit codes and output

Adding a system fixture:

tests/system/fixtures/my-scenario/
build.lua

Then add a scenario test that points at that fixture.

Performance tests

• Labeled with CTest property performance
• Long timeout (600s) for benchmark stability
• Not part of the default quick loop unless you filter by label

Fuzzer

Target: fuzz_lua_dsl (LLVM libFuzzer).

make fuzzer-smoke # 30s smoke (default FUZZER_TIME)
make fuzzer-corpus # 60s corpus expansion
FUZZER_TIME=120 make fuzzer-smoke

Corpus rules:

• Add seeds only as descriptive .lua files under tests/fuzz/corpus/lua_dsl/
• Do not commit hash-named artifacts from fuzzer runs
• Dictionary: tests/fuzz/lua_dsl.dict
• Required when lua_dsl.cpp or DSL syntax changes

TDD expectation

Write a failing test first, implement the minimum to pass, then refactor. See Feature Development Workflow.

Registering new tests

Every new .cpp test file must be listed in the appropriate CMakeLists.txt:

• tests/unit/CMakeLists.txt for unit tests
• tests/integration/CMakeLists.txt for integration tests
• tests/system/CMakeLists.txt for system tests

Forgetting registration is a common review failure: the file compiles nowhere and CI never runs it.

Coverage: new production code should include tests so the project stays at ≥ 85% line coverage on src/. Run make coverage before opening a PR.

Related pages

• Code Quality - coverage and sanitizer test runs
• Feature Development Workflow - Red/Green/Refactor cycle
• Repository Layout - fixture and fuzz paths