Testing and Validation

Purpose

Define the canonical testing and validation workflow for DSAMbayes v1.2.2, from local pre-merge checks through release-quality gates.

Audience

  • Engineers running local checks before merge
  • Maintainers preparing release candidates
  • Reviewers validating release evidence

Validation layers

Layer Objective Primary command(s) Output proof
Lint Catch style and static issues early Rscript scripts/check.R --lint Exit code 0, no lint failures
Style Enforce formatting compliance on changed files Rscript scripts/check.R --style Exit code 0, no reformat-required files
Unit tests Catch behavioural regressions in package logic R -q -e 'testthat::test_dir("tests/testthat")' Exit code 0, no test failures
Package check Validate package-level install and check behaviour R -q -e 'rcmdcheck::rcmdcheck(...)' No ERROR; no unresolved WARNING
Runner validate Validate config and data contracts without fitting Rscript scripts/dsambayes.R validate ... Exit code 0, metadata artefacts
Runner run Validate end-to-end runner execution and artefacts Rscript scripts/dsambayes.R run ... Exit code 0, core run artefacts
Docs build Validate pkgdown documentation buildability Rscript -e 'pkgdown::build_site_github_pages(...)' Exit code 0, successful site build
CI workflows Cross-platform and publish-path verification GitHub Actions workflow runs Green workflow status on candidate commit

Environment setup

Run all commands from repository root:

# Navigate to your local DSAMbayes checkout
cd /path/to/DSAMbayes
mkdir -p .Rlib .cache
export R_LIBS_USER="$PWD/.Rlib"
export XDG_CACHE_HOME="$PWD/.cache"

Expected outcome: checks run in a repo-scoped environment with reproducible library and cache paths.

Local validation workflows

Developer fast path (pre-merge)

Run the consolidated local gate:

R_LIBS_USER="$PWD/.Rlib" Rscript scripts/check.R --all

Expected outcome: lint, style, tests, and coverage complete successfully.

Implementation note: scripts/check.R --all is a convenience gate. It does not replace rcmdcheck, runner smoke tests, or docs build.

Release-candidate full path

Run mandatory gates in this exact order:

R_LIBS_USER="$PWD/.Rlib" Rscript scripts/check.R --lint
R_LIBS_USER="$PWD/.Rlib" Rscript scripts/check.R --style
R_LIBS_USER="$PWD/.Rlib" R -q -e 'testthat::test_dir("tests/testthat")'
R_LIBS_USER="$PWD/.Rlib" R -q -e 'rcmdcheck::rcmdcheck(args = c("--no-manual","--compact-vignettes=gs+qpdf"))'
R_LIBS_USER="$PWD/.Rlib" Rscript scripts/dsambayes.R validate --config config/blm_synthetic_mcmc.yaml --run-dir results/quality_gate_validate
R_LIBS_USER="$PWD/.Rlib" Rscript scripts/dsambayes.R run --config config/blm_synthetic_mcmc.yaml --run-dir results/quality_gate_run
R_LIBS_USER="$PWD/.Rlib" Rscript -e 'pkgdown::build_site_github_pages(new_process = FALSE, install = FALSE)'

Expected outcome: all gates complete with exit code 0, with no unresolved release blockers.

Runner smoke-test expectations

Minimum release smoke expectations:

  1. validate command succeeds and writes metadata artefacts.
  2. run command succeeds and writes model, posterior summary, and diagnostics artefacts.
  3. Required runner artefact paths exist under results/quality_gate_validate/ and results/quality_gate_run/.

For matrix and exact artefact paths, use Runner Smoke Tests.

CI validation expectations

Candidate commit should have:

  1. Passing .github/workflows/R-CMD-check.yaml.
  2. Passing .github/workflows/pkgdown.yaml build.
  3. Deploy-step eligibility confirmed for non-PR release flow.

For workflow semantics and expected proof, use CI Workflows.

Evidence capture requirements

Before sign-off, capture:

  1. Full command logs and exit codes for all mandatory gates.
  2. Runner smoke artefacts from validate and run directories.
  3. CI run URLs and statuses for both workflows.
  4. Candidate commit hash and top changelog section.

Use Release Evidence Pack as the authoritative bundle contract.

Failure handling

  1. Any gate failure is a release blocker until resolved.
  2. Re-run the full failed gate after remediation.
  3. If rcmdcheck emits NOTE, record reviewer rationale explicitly.
  4. If runner artefacts are missing, inspect resolved config and outputs.* flags.