chore(release): prepare public source release
This commit is contained in:
MercuryToolbox Release
@@ -0,0 +1,90 @@
|
||||
# Benchmark Harness Usage
|
||||
|
||||
This directory stores reproducible benchmark notes and reports for Mercury tools.
|
||||
The mhash harness lives at `scripts/benchmark-mhash.ps1` and writes generated
|
||||
artifacts under `target\mhash-benchmark\results` by default.
|
||||
|
||||
## Default Digest Benchmark
|
||||
|
||||
```powershell
|
||||
pwsh -NoProfile -ExecutionPolicy Bypass -File .\scripts\benchmark-mhash.ps1
|
||||
```
|
||||
|
||||
The default matrix keeps runtime reasonable: 1MiB, 64MiB, and 256MiB inputs;
|
||||
MD5, SHA-1, SHA-256, and SHA-512; 1 warmup and 3 measured runs. It compares
|
||||
Mercury `mhash` auto/read modes against available non-admin external tools.
|
||||
|
||||
## Fast Smoke Check
|
||||
|
||||
```powershell
|
||||
pwsh -NoProfile -ExecutionPolicy Bypass -File .\scripts\benchmark-mhash-smoke.ps1 -SkipBuild
|
||||
```
|
||||
|
||||
The smoke check runs a 1KiB SHA-256 Mercury-only matrix, validates required
|
||||
telemetry fields, checks JSONL export counts, and verifies `-SizeOnly` emits
|
||||
artifact telemetry without process records.
|
||||
|
||||
## Useful Modes
|
||||
|
||||
```powershell
|
||||
pwsh -NoProfile -ExecutionPolicy Bypass -File .\scripts\benchmark-mhash.ps1 -SizeOnly -SkipBuild
|
||||
```
|
||||
|
||||
Use `-SizeOnly` when you only need binary artifact telemetry. It skips fixture
|
||||
creation and process benchmarks, but still writes raw JSON, artifact JSON/CSV,
|
||||
empty summary files, and a Markdown report.
|
||||
|
||||
To inspect a prebuilt size-focused binary, point the harness at it:
|
||||
|
||||
```powershell
|
||||
cargo build --profile release-size -p mercury-mhash --bin mhash
|
||||
pwsh -NoProfile -ExecutionPolicy Bypass -File .\scripts\benchmark-mhash.ps1 -SizeOnly -SkipBuild -DigestPath .\target\release-size\mhash.exe
|
||||
```
|
||||
|
||||
```powershell
|
||||
pwsh -NoProfile -ExecutionPolicy Bypass -File .\scripts\benchmark-mhash.ps1 -Extended -Hyperfine
|
||||
```
|
||||
|
||||
Use `-Extended` for a deeper matrix. Unless explicitly overridden, it raises
|
||||
the run count to 5, warmups to 2, adds a 1GiB fixture, and benchmarks Mercury
|
||||
`sum`, `jsonl`, and `json` output formats. Use `-Hyperfine` to add optional
|
||||
`hyperfine` timing exports when `hyperfine` is available on `PATH`.
|
||||
|
||||
```powershell
|
||||
pwsh -NoProfile -ExecutionPolicy Bypass -File .\scripts\benchmark-mhash.ps1 -Sizes 1KiB -Algorithms sha256 -DigestFormats sum,jsonl -Repeat 1 -Warmup 0 -NoExternal -SkipBuild
|
||||
```
|
||||
|
||||
Use explicit small matrices for local harness checks or quick regression
|
||||
triage.
|
||||
|
||||
## Emitted Telemetry
|
||||
|
||||
Each run now records wall time, CPU time, CPU as percent of one core and of the
|
||||
machine, peak working set, mhash output format, binary path, binary size, and a
|
||||
short stdout/stderr sample. Summaries include mean wall time, min wall time,
|
||||
mean CPU milliseconds, throughput, peak working set, CPU percent of one core,
|
||||
and binary size.
|
||||
|
||||
The raw report includes reproducibility metadata: invocation, script path,
|
||||
workspace root, OS and process architecture, PowerShell version, logical
|
||||
processor count, git branch/commit/status entries, fixture SHA-256 hashes,
|
||||
tool paths/versions, artifact sizes, artifact SHA-256 hashes, and optional
|
||||
hyperfine suites.
|
||||
|
||||
Generated files include:
|
||||
|
||||
- `*.raw.json`: full environment, records, summary, artifact, and hyperfine data.
|
||||
- `*.records.jsonl`: one process measurement per line.
|
||||
- `*.summary.json` and `*.summary.jsonl`: grouped summary rows.
|
||||
- `*.summary.csv`: spreadsheet-friendly grouped summary rows.
|
||||
- `*.artifacts.json` and `*.artifacts.csv`: binary size and identity metadata.
|
||||
- `*.hyperfine.json` and `*.hyperfine.csv`: aggregate optional hyperfine output.
|
||||
- `*.md`: human-readable report.
|
||||
|
||||
## Measurement Caveats
|
||||
|
||||
Windows does not expose a safe non-admin cache-drop primitive, so results are
|
||||
warm-cache CLI timings. Very small fixtures mostly measure process startup and
|
||||
output overhead. Peak working set comes from process telemetry and can miss very
|
||||
short-lived allocation spikes. Hyperfine is opt-in and complements the built-in
|
||||
wall/CPU/memory telemetry; it does not replace the harness process metrics.
|
||||
@@ -0,0 +1,77 @@
|
||||
# Digest Benchmark - 2026-04-25
|
||||
|
||||
This benchmark compares Mercury `mhash` against common Windows and CLI hash tools on the same machine.
|
||||
|
||||
## Method
|
||||
|
||||
- Harness: `scripts/benchmark-mhash.ps1`
|
||||
- Scenario: warm-cache CLI hashing; Windows does not provide a safe non-admin cache-drop primitive, so cold-cache disk results are intentionally excluded.
|
||||
- Inputs: deterministic 1MiB, 64MiB, and 256MiB files under `target\mhash-benchmark\data`.
|
||||
- Algorithms: `md5`, `sha1`, `sha256`, `sha512`.
|
||||
- Repeats: 1 warmup plus 3 measured runs per tool/algorithm/size.
|
||||
- Metrics: wall-clock time, throughput, process CPU time as percent of one core, and peak working set.
|
||||
- Machine: `DESKTOP-EKBHO4B`, Windows `10.0.26200`, 32 logical processors.
|
||||
|
||||
## Tools
|
||||
|
||||
- Mercury `mhash 0.1.0`
|
||||
- PowerShell `Get-FileHash` via PowerShell `7.6.1`
|
||||
- Windows `certutil`
|
||||
- uutils coreutils `0.8.0` commands: `md5sum`, `sha1sum`, `sha256sum`, `sha512sum`
|
||||
- OpenSSL `3.6.1`
|
||||
|
||||
## Key Finding
|
||||
|
||||
The original Mercury `mhash` default `auto` mode memory-mapped large ordinary hash jobs. On this machine that increased peak working set to roughly the file size and was slower than streaming reads. The default was changed so streaming-friendly hash sets use one-pass buffered reads, while whole-file paths remain available for explicit `--io mmap` and for algorithms that benefit from whole-file/internal parallel processing.
|
||||
|
||||
## 256MiB Before/After
|
||||
|
||||
| Algorithm | Before auto ms | After auto ms | Speedup | Before peak MiB | After peak MiB | Peak drop MiB |
|
||||
|---|---:|---:|---:|---:|---:|---:|
|
||||
| md5 | 427.06 | 362.24 | 1.18x | 261.54 | 10.30 | 251.24 |
|
||||
| sha1 | 201.81 | 163.24 | 1.24x | 255.72 | 10.55 | 245.17 |
|
||||
| sha256 | 226.63 | 156.31 | 1.45x | 261.05 | 10.29 | 250.76 |
|
||||
| sha512 | 341.78 | 273.64 | 1.25x | 264.60 | 10.57 | 254.03 |
|
||||
|
||||
## 256MiB Post-Optimization Comparison
|
||||
|
||||
| Algorithm | Tool | Mean ms | MiB/s | Peak WS MiB | CPU % of one core |
|
||||
|---|---|---:|---:|---:|---:|
|
||||
| md5 | openssl:dgst | 353.02 | 725.17 | 12.17 | 100.5 |
|
||||
| md5 | mercury-mhash:auto | 362.24 | 706.71 | 10.30 | 94.9 |
|
||||
| md5 | mercury-mhash:read | 364.38 | 702.57 | 10.30 | 95.8 |
|
||||
| md5 | coreutils:md5sum | 377.49 | 678.17 | 9.44 | 96.6 |
|
||||
| md5 | windows:certutil | 521.46 | 490.93 | 14.71 | 91.9 |
|
||||
| md5 | powershell:Get-FileHash | 742.67 | 344.70 | 65.20 | 113.7 |
|
||||
| sha1 | mercury-mhash:read | 155.19 | 1649.56 | 10.54 | 90.6 |
|
||||
| sha1 | coreutils:sha1sum | 160.77 | 1592.36 | 9.45 | 90.7 |
|
||||
| sha1 | mercury-mhash:auto | 163.24 | 1568.23 | 10.55 | 98.9 |
|
||||
| sha1 | openssl:dgst | 196.64 | 1301.90 | 10.98 | 98.0 |
|
||||
| sha1 | windows:certutil | 398.24 | 642.82 | 14.73 | 99.4 |
|
||||
| sha1 | powershell:Get-FileHash | 641.07 | 399.33 | 66.26 | 114.5 |
|
||||
| sha256 | mercury-mhash:read | 155.78 | 1643.35 | 10.29 | 93.6 |
|
||||
| sha256 | mercury-mhash:auto | 156.31 | 1637.75 | 10.29 | 90.0 |
|
||||
| sha256 | coreutils:sha256sum | 170.32 | 1503.08 | 10.36 | 97.9 |
|
||||
| sha256 | openssl:dgst | 206.06 | 1242.36 | 10.99 | 91.0 |
|
||||
| sha256 | windows:certutil | 270.40 | 946.74 | 14.75 | 92.5 |
|
||||
| sha256 | powershell:Get-FileHash | 519.56 | 492.72 | 65.04 | 122.3 |
|
||||
| sha512 | mercury-mhash:auto | 273.64 | 935.52 | 10.57 | 99.0 |
|
||||
| sha512 | mercury-mhash:read | 273.88 | 934.70 | 10.31 | 93.2 |
|
||||
| sha512 | openssl:dgst | 277.36 | 922.98 | 10.99 | 99.5 |
|
||||
| sha512 | coreutils:sha512sum | 299.48 | 854.80 | 9.45 | 93.9 |
|
||||
| sha512 | windows:certutil | 910.10 | 281.29 | 14.73 | 99.6 |
|
||||
| sha512 | powershell:Get-FileHash | 1168.26 | 219.13 | 65.48 | 106.6 |
|
||||
|
||||
## Reproduce
|
||||
|
||||
```powershell
|
||||
just mhash-benchmark
|
||||
```
|
||||
|
||||
or:
|
||||
|
||||
```powershell
|
||||
pwsh -NoProfile -ExecutionPolicy Bypass -File .\scripts\benchmark-mhash.ps1
|
||||
```
|
||||
|
||||
The harness writes raw JSON, summary JSON, CSV, and Markdown reports under `target\mhash-benchmark\results`.
|
||||
Reference in New Issue
Block a user