Runner Smoke Tests

Purpose

Define the minimal reproducible smoke-test matrix for the YAML runner validate and run commands, including exact commands and expected artefacts.

Audience

• Maintainers preparing release evidence
• Engineers triaging runner regressions
• Reviewers confirming gate QG-5 and QG-6

Test scope

This smoke suite is intentionally small. It proves:

• CLI argument handling for validate and run
• Config resolution and runner pre-flight path
• End-to-end artefact writing for one full run

This smoke suite does not replace unit tests or full package checks.

Preconditions

Run from repository root:

# Navigate to your local DSAMbayes checkout
cd /path/to/DSAMbayes
mkdir -p .Rlib .cache
export R_LIBS_USER="$PWD/.Rlib"
export XDG_CACHE_HOME="$PWD/.cache"

Expected outcome: commands resolve local package/library paths and use repo-scoped Stan cache.

Install DSAMbayes locally if needed:

R_LIBS_USER="$PWD/.Rlib" R -q -e 'install.packages(".", repos = NULL, type = "source")'

Expected outcome: library(DSAMbayes) succeeds in the same shell session.

Smoke-test matrix

Canonical commands

SMK-VAL-01

R_LIBS_USER="$PWD/.Rlib" \
  Rscript scripts/dsambayes.R validate \
    --config config/blm_synthetic_mcmc.yaml \
    --run-dir results/smoke_validate_blm

Expected outcome: validation completes without Stan fitting and prints Status: ok.

SMK-VAL-02

R_LIBS_USER="$PWD/.Rlib" \
  Rscript scripts/dsambayes.R validate \
    --config config/hierarchical_re_synthetic_mcmc.yaml \
    --run-dir results/smoke_validate_hier_re

Expected outcome: validation completes for hierarchical RE config and prints Status: ok.

SMK-VAL-03

R_LIBS_USER="$PWD/.Rlib" \
  Rscript scripts/dsambayes.R validate \
    --config config/pooled_synthetic_mcmc.yaml \
    --run-dir results/smoke_validate_pooled

Expected outcome: validation completes for pooled config and prints Status: ok.

SMK-RUN-01

R_LIBS_USER="$PWD/.Rlib" \
  Rscript scripts/dsambayes.R run \
    --config config/blm_synthetic_mcmc.yaml \
    --run-dir results/smoke_run_blm

Expected outcome: full pipeline completes and prints Run complete with a resolved run directory.

Expected artefacts

Validate artefacts (SMK-VAL-01 to SMK-VAL-03)

For each validate run directory, these files are required:

• 00_run_metadata/config.original.yaml
• 00_run_metadata/config.resolved.yaml
• 00_run_metadata/session_info.txt

Failure rule: missing any required file is a smoke-test failure.

Run artefacts (SMK-RUN-01)

Required core artefacts:

• 00_run_metadata/config.resolved.yaml
• 20_model_fit/model.rds
• 30_post_run/posterior_summary.csv
• 40_diagnostics/diagnostics_report.csv

Recommended additional checks for stronger confidence:

• 30_post_run/fitted.csv
• 30_post_run/observed.csv
• 40_diagnostics/diagnostics_summary.txt

Failure rule: missing any required core artefact is a smoke-test failure.

Verification helper commands

Check validate artefacts quickly:

for d in results/smoke_validate_blm results/smoke_validate_hier_re results/smoke_validate_pooled; do
  test -f "$d/00_run_metadata/config.original.yaml" || echo "MISSING: $d config.original.yaml"
  test -f "$d/00_run_metadata/config.resolved.yaml" || echo "MISSING: $d config.resolved.yaml"
  test -f "$d/00_run_metadata/session_info.txt" || echo "MISSING: $d session_info.txt"
done

Expected outcome: no MISSING: lines.

Check core run artefacts quickly:

d="results/smoke_run_blm"
test -f "$d/00_run_metadata/config.resolved.yaml" || echo "MISSING: config.resolved.yaml"
test -f "$d/20_model_fit/model.rds" || echo "MISSING: model.rds"
test -f "$d/30_post_run/posterior_summary.csv" || echo "MISSING: posterior_summary.csv"
test -f "$d/40_diagnostics/diagnostics_report.csv" || echo "MISSING: diagnostics_report.csv"

Expected outcome: no MISSING: lines.

Failure triage

1. If validate fails, run the same command again with a clean run directory path and inspect CLI error output.
2. If run fails before fitting, inspect 00_run_metadata/config.resolved.yaml to confirm resolved values.
3. If run fails during fitting, verify local Stan toolchain and cache path from Install and Setup.
4. If artefacts are missing after success exit code, inspect outputs.* flags in the resolved config.

Evidence capture

For release evidence, capture:

1. Full terminal logs and exit codes for SMK-VAL-01 to SMK-RUN-01.
2. Directory listings for each smoke run directory.
3. The required artefacts listed above.

Store and review evidence with:

• Quality Gates
• Release Evidence Pack
• Release Sign-off Template