Phase 5 · Test: the composition proves the signed promises

Produce the composition's own tests/ and a report: stories, probes, scenarios, and the check against the signed record. The bricks' own suites prove the bricks, not your assembly (lab, 2026-10-17: a build passed 31/31 hidden scenarios yet shipped no test of its own). Back: dodil://erp/build.

Steps

1. Boot test: tests/test_composition.py (from modules/test_composition.py) boots the composition on SQLite and calls a signed-in read on every brick.
2. Acceptance: one test per story id, named for it (test_hr_01_freelancer_intake…), asserting its Given/When/Then through the doors (route or MCP tool), never by writing a table.
3. Access: per role, the right rows; another person's row is 404, a missing permission 403.
4. Generated probes and scenarios from erp_platform.scenarios: every role × use case × both doors, idempotent replay, mass assignment. Run each brick's catalogue, and your own scenarios/catalogue.py built on YOUR compose.yaml (Catalogue("acme", compose="compose.yaml", roles="roles.yaml")) holding the cross-brick stories SC-<MODULE>-nn.
5. Read back the customer's configuration through the bricks' use cases: the signer, the cadence, the timezone they chose.
6. Check against the sign-off record: regenerate the process view from the configured test copy (show_playbook_bpmn) and run erp-compose signoff verify --module <module> --bpmn <it>. It must say conforms; a mismatch exits non-zero and names each added, removed or changed task, flow, gateway and timer by its stable id. Every scenario_ids entry has a passing test and still walks (scenario_paths), and playbook_deviations lists exactly the accepted deviations.
7. The bench checks the same, independently (conforms_to_signed, part of PASS when a signoff/<module>.json exists): it boots your composition in process, runs your configuration setup (config/<module>.calls.yaml, a list of {use_case, args}, or seed.py with seed(call)), re-exports the signed playbook and compares hash, paths and deviations. Keep the setup in one of those two shapes so the check can replay it. Event-log conformance (replaying a run's events on the signed BPMN) is not built yet: say in the report it was not measured.

Example

.venv/bin/python -m pytest -q --tb=short tests
.venv/bin/erp-scenarios run brick-hr/tests/scenarios/catalogue.py          # in process, seconds
.venv/bin/erp-scenarios run scenarios/catalogue.py --junit out/scenarios.xml
# live, after phase 6-8, against the dev deploy only (refused unless the bucket is stamped demo):
.venv/bin/erp-scenarios run scenarios/catalogue.py --live --base-url "$DEV_URL" --bucket acme-dev

The bar for "validated together" is the crm-demo composition (2026-10-17): security probes 486/486, stories 28/28, live.

Done when

• Every story id has a named, passing test; every role has an access test.
• Probes and scenarios are green in process; the report names what was not measured.
• erp-compose signoff verify says conforms on the regenerated process view.

Next: dodil://erp/phase/deploy