# Benchmark Harness Usage

This directory stores reproducible benchmark notes and reports for Mercury tools.
The mhash harness lives at `scripts/benchmark-mhash.ps1` and writes generated
artifacts under `target\mhash-benchmark\results` by default.

## Default Digest Benchmark

```powershell
pwsh -NoProfile -ExecutionPolicy Bypass -File .\scripts\benchmark-mhash.ps1
```

The default matrix keeps runtime reasonable: 1MiB, 64MiB, and 256MiB inputs;
MD5, SHA-1, SHA-256, and SHA-512; 1 warmup and 3 measured runs. It compares
Mercury `mhash` auto/read modes against available non-admin external tools.

## Fast Smoke Check

```powershell
pwsh -NoProfile -ExecutionPolicy Bypass -File .\scripts\benchmark-mhash-smoke.ps1 -SkipBuild
```

The smoke check runs a 1KiB SHA-256 Mercury-only matrix, validates required
telemetry fields, checks JSONL export counts, and verifies `-SizeOnly` emits
artifact telemetry without process records.

## Useful Modes

```powershell
pwsh -NoProfile -ExecutionPolicy Bypass -File .\scripts\benchmark-mhash.ps1 -SizeOnly -SkipBuild
```

Use `-SizeOnly` when you only need binary artifact telemetry. It skips fixture
creation and process benchmarks, but still writes raw JSON, artifact JSON/CSV,
empty summary files, and a Markdown report.

To inspect a prebuilt size-focused binary, point the harness at it:

```powershell
cargo build --profile release-size -p mercury-mhash --bin mhash
pwsh -NoProfile -ExecutionPolicy Bypass -File .\scripts\benchmark-mhash.ps1 -SizeOnly -SkipBuild -DigestPath .\target\release-size\mhash.exe
```

```powershell
pwsh -NoProfile -ExecutionPolicy Bypass -File .\scripts\benchmark-mhash.ps1 -Extended -Hyperfine
```

Use `-Extended` for a deeper matrix. Unless explicitly overridden, it raises
the run count to 5, warmups to 2, adds a 1GiB fixture, and benchmarks Mercury
`sum`, `jsonl`, and `json` output formats. Use `-Hyperfine` to add optional
`hyperfine` timing exports when `hyperfine` is available on `PATH`.

```powershell
pwsh -NoProfile -ExecutionPolicy Bypass -File .\scripts\benchmark-mhash.ps1 -Sizes 1KiB -Algorithms sha256 -DigestFormats sum,jsonl -Repeat 1 -Warmup 0 -NoExternal -SkipBuild
```

Use explicit small matrices for local harness checks or quick regression
triage.

## Emitted Telemetry

Each run now records wall time, CPU time, CPU as percent of one core and of the
machine, peak working set, mhash output format, binary path, binary size, and a
short stdout/stderr sample. Summaries include mean wall time, min wall time,
mean CPU milliseconds, throughput, peak working set, CPU percent of one core,
and binary size.

The raw report includes reproducibility metadata: invocation, script path,
workspace root, OS and process architecture, PowerShell version, logical
processor count, git branch/commit/status entries, fixture SHA-256 hashes,
tool paths/versions, artifact sizes, artifact SHA-256 hashes, and optional
hyperfine suites.

Generated files include:

- `*.raw.json`: full environment, records, summary, artifact, and hyperfine data.
- `*.records.jsonl`: one process measurement per line.
- `*.summary.json` and `*.summary.jsonl`: grouped summary rows.
- `*.summary.csv`: spreadsheet-friendly grouped summary rows.
- `*.artifacts.json` and `*.artifacts.csv`: binary size and identity metadata.
- `*.hyperfine.json` and `*.hyperfine.csv`: aggregate optional hyperfine output.
- `*.md`: human-readable report.

## Measurement Caveats

Windows does not expose a safe non-admin cache-drop primitive, so results are
warm-cache CLI timings. Very small fixtures mostly measure process startup and
output overhead. Peak working set comes from process telemetry and can miss very
short-lived allocation spikes. Hyperfine is opt-in and complements the built-in
wall/CPU/memory telemetry; it does not replace the harness process metrics.