forked from Crockan/MercuryToolbox
chore(release): prepare public source release
This commit is contained in:
@@ -0,0 +1,172 @@
|
||||
# Mercury Toolbox V2 Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Rebrand the repository as Mercury Toolbox, add four new high-leverage CLI commands, and ship a first-class Windows installer that puts the full toolbox on PATH.
|
||||
|
||||
**Architecture:** Keep the toolbox as a Rust workspace of independent binaries with a shared CLI contract in `crates/common`. Add the new commands as focused crates with heuristic parsers and stable JSON payloads, and implement installation through PowerShell scripts that build and copy the binaries into a dedicated Mercury Toolbox bin directory.
|
||||
|
||||
**Tech Stack:** Rust stable, `lexopt`-style manual CLI parsing, shared `common` crate, `regex`, `serde_json`, PowerShell install scripts, `assert_cmd`, `predicates`, `tempfile`, and the existing Jade verification scripts.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Branding And Install Surface
|
||||
|
||||
**Files:**
|
||||
- Create: `docs/superpowers/specs/2026-04-21-mercury-toolbox-v2-design.md`
|
||||
- Create: `docs/superpowers/plans/2026-04-21-mercury-toolbox-v2-plan.md`
|
||||
- Modify: `Cargo.toml`
|
||||
- Modify: `README.md`
|
||||
- Modify: `docs/jade-discipline.md`
|
||||
- Create: `scripts/install-toolbox.ps1`
|
||||
- Create: `scripts/uninstall-toolbox.ps1`
|
||||
|
||||
- [ ] Record the Mercury Toolbox design and plan docs in the repo.
|
||||
- [ ] Rename top-level product-facing descriptions and README text from generic toolbox wording to `Mercury Toolbox` / `水星工具箱`.
|
||||
- [ ] Add an install section to the README covering:
|
||||
- `pwsh .\scripts\install-toolbox.ps1`
|
||||
- install root
|
||||
- PATH behavior
|
||||
- uninstall path
|
||||
- manual `cargo install --path` fallback
|
||||
- [ ] Write `scripts/install-toolbox.ps1` with idempotent bin copy and user PATH update behavior.
|
||||
- [ ] Write `scripts/uninstall-toolbox.ps1` with idempotent uninstall and PATH removal behavior.
|
||||
- [ ] Add focused script tests or temp-root verification coverage where practical.
|
||||
|
||||
### Task 2: Workspace Wiring For New Commands
|
||||
|
||||
**Files:**
|
||||
- Modify: `Cargo.toml`
|
||||
- Modify: `README.md`
|
||||
- Create: `fixtures/hits/rg-output.txt`
|
||||
- Create: `fixtures/diag/rust-errors.txt`
|
||||
- Create: `fixtures/diag/unity-errors.txt`
|
||||
- Create: `fixtures/logs/repetitive.log`
|
||||
- Create: `fixtures/binaries/stringscan-sample.bin`
|
||||
- Create: `crates/hitsnip/Cargo.toml`
|
||||
- Create: `crates/hitsnip/src/main.rs`
|
||||
- Create: `crates/hitsnip/src/lib.rs`
|
||||
- Create: `crates/diagpick/Cargo.toml`
|
||||
- Create: `crates/diagpick/src/main.rs`
|
||||
- Create: `crates/diagpick/src/lib.rs`
|
||||
- Create: `crates/logshape/Cargo.toml`
|
||||
- Create: `crates/logshape/src/main.rs`
|
||||
- Create: `crates/logshape/src/lib.rs`
|
||||
- Create: `crates/stringscan/Cargo.toml`
|
||||
- Create: `crates/stringscan/src/main.rs`
|
||||
- Create: `crates/stringscan/src/lib.rs`
|
||||
|
||||
- [ ] Register the four new crates in the workspace member list.
|
||||
- [ ] Add any workspace dependencies required by the new crates.
|
||||
- [ ] Add fixture files with stable, portable sample content for all four commands.
|
||||
- [ ] Add README command inventory entries for `hitsnip`, `diagpick`, `logshape`, and `stringscan`.
|
||||
|
||||
### Task 3: `hitsnip`
|
||||
|
||||
**Files:**
|
||||
- Create: `crates/hitsnip/Cargo.toml`
|
||||
- Create: `crates/hitsnip/src/lib.rs`
|
||||
- Create: `crates/hitsnip/src/main.rs`
|
||||
- Create: `crates/hitsnip/tests/hitsnip_cli.rs`
|
||||
- Use: `fixtures/hits/rg-output.txt`
|
||||
- Use: `fixtures/reading/sample.rs`
|
||||
|
||||
- [ ] Write failing tests for:
|
||||
- `rg -n` line parsing
|
||||
- merged context windows
|
||||
- JSON output shape
|
||||
- PowerShell pipeline usage
|
||||
- `--help`
|
||||
- [ ] Run `cargo nextest run -p hitsnip` and confirm the new tests fail for the expected missing behavior.
|
||||
- [ ] Implement hit parsing that correctly handles Windows drive prefixes and `path:line[:column]` shapes.
|
||||
- [ ] Implement per-file hit grouping, context expansion, max-gap merging, and compact numbered rendering.
|
||||
- [ ] Re-run `cargo nextest run -p hitsnip` until green.
|
||||
|
||||
### Task 4: `diagpick`
|
||||
|
||||
**Files:**
|
||||
- Create: `crates/diagpick/Cargo.toml`
|
||||
- Create: `crates/diagpick/src/lib.rs`
|
||||
- Create: `crates/diagpick/src/main.rs`
|
||||
- Create: `crates/diagpick/tests/diagpick_cli.rs`
|
||||
- Use: `fixtures/diag/rust-errors.txt`
|
||||
- Use: `fixtures/diag/unity-errors.txt`
|
||||
- Use: `fixtures/reading/sample.rs`
|
||||
- Use: `fixtures/reading/sample.cs`
|
||||
|
||||
- [ ] Write failing tests for:
|
||||
- Rust diagnostic extraction
|
||||
- Unity/C# diagnostic extraction
|
||||
- severity filtering
|
||||
- `--with-source`
|
||||
- JSON output shape
|
||||
- `--help`
|
||||
- [ ] Run `cargo nextest run -p diagpick` and confirm failure.
|
||||
- [ ] Implement heuristic text parsers for the supported diagnostic families plus JSONL input support.
|
||||
- [ ] Implement optional source loading with context lines and safe missing-file handling.
|
||||
- [ ] Re-run `cargo nextest run -p diagpick` until green.
|
||||
|
||||
### Task 5: `logshape`
|
||||
|
||||
**Files:**
|
||||
- Create: `crates/logshape/Cargo.toml`
|
||||
- Create: `crates/logshape/src/lib.rs`
|
||||
- Create: `crates/logshape/src/main.rs`
|
||||
- Create: `crates/logshape/tests/logshape_cli.rs`
|
||||
- Use: `fixtures/logs/repetitive.log`
|
||||
|
||||
- [ ] Write failing tests for:
|
||||
- normalization of volatile fields
|
||||
- grouping repeated templates
|
||||
- `--top` and `--min-count`
|
||||
- JSON output shape
|
||||
- PowerShell pipeline usage
|
||||
- `--help`
|
||||
- [ ] Run `cargo nextest run -p logshape` and confirm failure.
|
||||
- [ ] Implement conservative normalization and frequency grouping logic.
|
||||
- [ ] Implement compact text summaries plus JSON records with count and sample metadata.
|
||||
- [ ] Re-run `cargo nextest run -p logshape` until green.
|
||||
|
||||
### Task 6: `stringscan`
|
||||
|
||||
**Files:**
|
||||
- Create: `crates/stringscan/Cargo.toml`
|
||||
- Create: `crates/stringscan/src/lib.rs`
|
||||
- Create: `crates/stringscan/src/main.rs`
|
||||
- Create: `crates/stringscan/tests/stringscan_cli.rs`
|
||||
- Use: `fixtures/binaries/stringscan-sample.bin`
|
||||
|
||||
- [ ] Write failing tests for:
|
||||
- printable string extraction
|
||||
- category detection for URL, DLL, namespace, Unity, IL2CPP, and BepInEx-like strings
|
||||
- summary mode vs detail mode
|
||||
- JSON output shape
|
||||
- PowerShell path pipeline
|
||||
- `--help`
|
||||
- [ ] Run `cargo nextest run -p stringscan` and confirm failure.
|
||||
- [ ] Implement printable-string scanning and category classification helpers.
|
||||
- [ ] Implement summary output, filtered detail output, and JSON payloads.
|
||||
- [ ] Re-run `cargo nextest run -p stringscan` until green.
|
||||
|
||||
### Task 7: Full Documentation And Install Verification
|
||||
|
||||
**Files:**
|
||||
- Modify: `README.md`
|
||||
- Modify: command `after_help` sections in the four new crates
|
||||
- Modify: install/uninstall scripts as needed by verification
|
||||
|
||||
- [ ] Ensure every new command has PowerShell examples in README and `--help`.
|
||||
- [ ] Verify the install script copies all toolbox commands, not just the new ones.
|
||||
- [ ] Verify PATH update behavior against an isolated temp install root when possible.
|
||||
- [ ] Verify uninstall removes the Mercury Toolbox bin path entry without touching unrelated PATH content.
|
||||
|
||||
### Task 8: Final Verification
|
||||
|
||||
**Files:**
|
||||
- Modify only as required by verification failures
|
||||
|
||||
- [ ] Run `cargo fmt --all`.
|
||||
- [ ] Run `cargo nextest run --all-targets --all-features`.
|
||||
- [ ] Run `cargo clippy --all-targets --all-features -- -D warnings -W clippy::pedantic -W clippy::nursery`.
|
||||
- [ ] Run `pwsh -NoProfile -File .\scripts\check-jade.ps1`.
|
||||
- [ ] Fix any regressions and repeat until the full workspace passes.
|
||||
@@ -0,0 +1,96 @@
|
||||
# Toolbox Reading V2 Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Add four new reading-oriented CLI commands that let PowerShell and AI workflows inspect files precisely without dumping whole files.
|
||||
|
||||
**Architecture:** Keep each command as its own crate in the workspace and keep language or file-type heuristics inside focused helpers rather than shared heavyweight parsers. Reuse the existing common CLI contract for stdin behavior, JSON output, and exit codes, while letting each command define small, stable domain-specific JSON payloads.
|
||||
|
||||
**Tech Stack:** Rust stable, `lexopt`-style manual CLI parsing, shared `common` crate, `regex`, `serde_json`, `toml`, `assert_cmd`, and existing workspace verification scripts.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Manifest And Documentation Wiring
|
||||
|
||||
**Files:**
|
||||
- Create: `docs/superpowers/specs/2026-04-21-toolbox-reading-v2-design.md`
|
||||
- Create: `docs/superpowers/plans/2026-04-21-toolbox-reading-v2-plan.md`
|
||||
- Modify: `Cargo.toml`
|
||||
- Modify: `README.md`
|
||||
|
||||
- [ ] Record the reading-tool batch scope in repo docs.
|
||||
- [ ] Register `crates/snip`, `crates/outline`, `crates/fileprobe`, and `crates/chunkcat` in the workspace.
|
||||
- [ ] Add any workspace dependencies needed by the four crates.
|
||||
- [ ] Update README command inventory and usage examples.
|
||||
|
||||
### Task 2: `snip`
|
||||
|
||||
**Files:**
|
||||
- Create: `crates/snip/Cargo.toml`
|
||||
- Create: `crates/snip/src/lib.rs`
|
||||
- Create: `crates/snip/src/main.rs`
|
||||
- Create: `crates/snip/tests/snip_cli.rs`
|
||||
- Create: `fixtures/reading/sample.rs`
|
||||
- Create: `fixtures/reading/sample.cs`
|
||||
|
||||
- [ ] Write failing tests for `--lines`, `--around`, `--symbol`, and `--help`.
|
||||
- [ ] Run targeted `snip` tests and confirm failure for the new behavior.
|
||||
- [ ] Implement snippet selection, numbered text rendering, and JSON output.
|
||||
- [ ] Add heuristic symbol extraction for Rust and C# plus safe fallback behavior.
|
||||
- [ ] Re-run targeted `snip` tests until green.
|
||||
|
||||
### Task 3: `outline`
|
||||
|
||||
**Files:**
|
||||
- Create: `crates/outline/Cargo.toml`
|
||||
- Create: `crates/outline/src/lib.rs`
|
||||
- Create: `crates/outline/src/main.rs`
|
||||
- Create: `crates/outline/tests/outline_cli.rs`
|
||||
- Create: `fixtures/reading/config.json`
|
||||
- Create: `fixtures/reading/config.toml`
|
||||
- Create: `fixtures/reading/config.yaml`
|
||||
|
||||
- [ ] Write failing tests for Rust/C#/config outline extraction, JSON shape, and `--help`.
|
||||
- [ ] Run targeted `outline` tests and confirm failure.
|
||||
- [ ] Implement heuristic outline extraction and depth limiting.
|
||||
- [ ] Re-run targeted `outline` tests until green.
|
||||
|
||||
### Task 4: `fileprobe`
|
||||
|
||||
**Files:**
|
||||
- Create: `crates/fileprobe/Cargo.toml`
|
||||
- Create: `crates/fileprobe/src/lib.rs`
|
||||
- Create: `crates/fileprobe/src/main.rs`
|
||||
- Create: `crates/fileprobe/tests/fileprobe_cli.rs`
|
||||
- Create: `fixtures/reading/minified.js`
|
||||
- Create: `fixtures/reading/generated.lock`
|
||||
- Create: `fixtures/reading/binary.bin`
|
||||
|
||||
- [ ] Write failing tests for text/binary detection, heuristic flags, JSON output, and `--help`.
|
||||
- [ ] Run targeted `fileprobe` tests and confirm failure.
|
||||
- [ ] Implement lightweight probing, line stats, and file family heuristics.
|
||||
- [ ] Re-run targeted `fileprobe` tests until green.
|
||||
|
||||
### Task 5: `chunkcat`
|
||||
|
||||
**Files:**
|
||||
- Create: `crates/chunkcat/Cargo.toml`
|
||||
- Create: `crates/chunkcat/src/lib.rs`
|
||||
- Create: `crates/chunkcat/src/main.rs`
|
||||
- Create: `crates/chunkcat/tests/chunkcat_cli.rs`
|
||||
|
||||
- [ ] Write failing tests for chunk inventory, selected chunk rendering, JSON output, and `--help`.
|
||||
- [ ] Run targeted `chunkcat` tests and confirm failure.
|
||||
- [ ] Implement deterministic chunk planning and selected chunk extraction.
|
||||
- [ ] Re-run targeted `chunkcat` tests until green.
|
||||
|
||||
### Task 6: Final Verification
|
||||
|
||||
**Files:**
|
||||
- Modify only as required by verification failures
|
||||
|
||||
- [ ] Run `cargo fmt --all`.
|
||||
- [ ] Run `cargo nextest run --all-targets --all-features`.
|
||||
- [ ] Run `cargo clippy --all-targets --all-features -- -D warnings -W clippy::pedantic -W clippy::nursery`.
|
||||
- [ ] Run `pwsh -NoProfile -File .\scripts\check-jade.ps1`.
|
||||
- [ ] Fix any regressions and repeat until the full workspace passes.
|
||||
@@ -0,0 +1,95 @@
|
||||
# Toolbox V1.5 Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Upgrade the existing four toolbox commands with better UX and capability, then add a production-grade `binmeta` binary metadata inspector.
|
||||
|
||||
**Architecture:** Keep the workspace shape flat and consistent: each command remains its own crate, shared behavior stays in `common`, and every new behavior is covered by command-local tests plus workspace verification. `binmeta` will follow the same input/output contract as the existing tools and keep format-specific parsing behind focused helpers.
|
||||
|
||||
**Tech Stack:** Rust stable, `lexopt`-style manual CLI parsing, shared `common` crate, targeted parsing crates for PE inspection, workspace tests via `assert_cmd`.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Design And Manifest Wiring
|
||||
|
||||
**Files:**
|
||||
- Create: `docs/superpowers/specs/2026-04-21-toolbox-v1-5-design.md`
|
||||
- Create: `docs/superpowers/plans/2026-04-21-toolbox-v1-5-plan.md`
|
||||
- Modify: `Cargo.toml`
|
||||
- Modify: `README.md`
|
||||
|
||||
- [ ] Record the approved v1.5 scope in repo docs.
|
||||
- [ ] Add any new workspace dependencies needed for `binmeta`.
|
||||
- [ ] Register `crates/binmeta` in the workspace.
|
||||
- [ ] Update README command inventory and usage examples.
|
||||
|
||||
### Task 2: `jsonlgrep` Upgrade
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/jsonlgrep/src/lib.rs`
|
||||
- Modify: `crates/jsonlgrep/tests/query_and_cli.rs`
|
||||
|
||||
- [ ] Write failing tests for dotted field paths, `!=`, and `!~=` queries.
|
||||
- [ ] Run targeted `jsonlgrep` tests and confirm the new cases fail for the right reason.
|
||||
- [ ] Implement nested path resolution and negative predicate parsing/matching with minimal code changes.
|
||||
- [ ] Add `after_help` examples and test representative help output.
|
||||
- [ ] Re-run targeted `jsonlgrep` tests until green.
|
||||
|
||||
### Task 3: `recent` Upgrade
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/recent/src/lib.rs`
|
||||
- Modify: `crates/recent/tests/recent_cli.rs`
|
||||
|
||||
- [ ] Write failing tests for `--name` regex filtering and help output examples.
|
||||
- [ ] Run targeted `recent` tests and confirm failure.
|
||||
- [ ] Implement basename regex filtering and help text polish.
|
||||
- [ ] Re-run targeted `recent` tests until green.
|
||||
|
||||
### Task 4: `pathshadow` Upgrade
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/pathshadow/src/lib.rs`
|
||||
- Modify: `crates/pathshadow/tests/pathshadow_cli.rs`
|
||||
|
||||
- [ ] Write failing tests for explanation/rank fields and help examples.
|
||||
- [ ] Run targeted `pathshadow` tests and confirm failure.
|
||||
- [ ] Implement richer JSON/text explanation output while preserving existing contract fields.
|
||||
- [ ] Re-run targeted `pathshadow` tests until green.
|
||||
|
||||
### Task 5: `portping` Upgrade
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/portping/src/lib.rs`
|
||||
- Modify: `crates/portping/src/main.rs`
|
||||
- Modify: `crates/portping/tests/portping_cli.rs`
|
||||
|
||||
- [ ] Write failing tests for `--method`, `--expect-status`, and help examples.
|
||||
- [ ] Run targeted `portping` tests and confirm failure.
|
||||
- [ ] Implement request method selection and explicit status expectation handling.
|
||||
- [ ] Re-run targeted `portping` tests until green.
|
||||
|
||||
### Task 6: `binmeta` Command
|
||||
|
||||
**Files:**
|
||||
- Create: `crates/binmeta/Cargo.toml`
|
||||
- Create: `crates/binmeta/src/lib.rs`
|
||||
- Create: `crates/binmeta/src/main.rs`
|
||||
- Create: `crates/binmeta/tests/binmeta_cli.rs`
|
||||
- Create: `fixtures/binmeta/plain.txt`
|
||||
|
||||
- [ ] Write failing tests for non-PE fallback, PE metadata extraction against a compiled workspace binary, JSON output, and help examples.
|
||||
- [ ] Run targeted `binmeta` tests and confirm failure.
|
||||
- [ ] Implement stdin/path loading, metadata hashing, PE parsing, heuristic hints, and text/JSON rendering.
|
||||
- [ ] Re-run targeted `binmeta` tests until green.
|
||||
|
||||
### Task 7: Full Verification
|
||||
|
||||
**Files:**
|
||||
- Modify only as required by verification failures
|
||||
|
||||
- [ ] Run `cargo fmt --all`.
|
||||
- [ ] Run `cargo nextest run --all-targets --all-features`.
|
||||
- [ ] Run `cargo clippy --all-targets --all-features -- -D warnings -W clippy::pedantic -W clippy::nursery`.
|
||||
- [ ] Run `pwsh -NoProfile -File .\scripts\check-jade.ps1`.
|
||||
- [ ] Fix any regressions and repeat until all commands pass.
|
||||
@@ -0,0 +1,329 @@
|
||||
# Mercury Toolbox V2 Design
|
||||
|
||||
## Goal
|
||||
|
||||
Promote the repository from a generic AI-friendly CLI toolbox into the formally named `Mercury Toolbox` / `水星工具箱`, add four new high-leverage commands for search-to-context, diagnostics, log reduction, and binary string triage, and ship a first-class Windows + PowerShell installation path that automatically places the toolbox on the user PATH.
|
||||
|
||||
## Scope
|
||||
|
||||
- Rename the toolbox brand in repository-level docs, workspace metadata, install-facing text, and user help from the generic placeholder wording to `Mercury Toolbox` / `水星工具箱`.
|
||||
- Add four new commands:
|
||||
- `hitsnip`
|
||||
- `diagpick`
|
||||
- `logshape`
|
||||
- `stringscan`
|
||||
- Keep natural binary names such as `snip`, `outline`, `hitsnip`, and `diagpick`; do not introduce a global binary prefix.
|
||||
- Add a user-facing PowerShell installer that:
|
||||
- builds the workspace release binaries
|
||||
- installs all toolbox binaries into `$env:LOCALAPPDATA\MercuryToolbox\bin`
|
||||
- automatically adds that directory to the user PATH if missing
|
||||
- reports what changed and how to verify the install
|
||||
- Add a complementary uninstall path and installation documentation.
|
||||
- Preserve the shared CLI contract:
|
||||
- compact ASCII text output by default
|
||||
- `--json` for machine consumption
|
||||
- PowerShell-friendly stdin behavior
|
||||
- stable exit code mapping
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- No GUI, TUI, or long-running background service.
|
||||
- No package publishing, Scoop bucket, installer EXE, or code signing in this iteration.
|
||||
- No heavyweight parser stack such as tree-sitter, Roslyn, or full symbol servers.
|
||||
- No single “master” control binary; the toolbox remains a workspace of small independent commands.
|
||||
|
||||
## Product Direction
|
||||
|
||||
The existing toolbox already covers “find a file”, “probe the environment”, and “read a known target”. The new batch fills the remaining gaps that are especially painful for AI-assisted terminal workflows:
|
||||
|
||||
- `hitsnip` answers “I already have search hits; now give me the smallest useful context.”
|
||||
- `diagpick` answers “this compiler or runtime log is noisy; show me the actionable diagnostics.”
|
||||
- `logshape` answers “this log is too repetitive; summarize the patterns before I read it.”
|
||||
- `stringscan` answers “this binary or generated blob is opaque; show me the strings that reveal identity, runtime, and ecosystem.”
|
||||
|
||||
The installation work is part of the feature set, not a side quest. A toolbox that is annoying to install has near-zero real-world value even if the commands are good.
|
||||
|
||||
## Branding And Naming
|
||||
|
||||
### Repository Name
|
||||
|
||||
- Primary English name: `Mercury Toolbox`
|
||||
- Primary Chinese name: `水星工具箱`
|
||||
- Documentation should treat these as the formal product name, not as aliases for the old placeholder wording.
|
||||
|
||||
### Binary Names
|
||||
|
||||
- Keep natural command names:
|
||||
- `jsonlgrep`
|
||||
- `recent`
|
||||
- `pathshadow`
|
||||
- `portping`
|
||||
- `binmeta`
|
||||
- `fileprobe`
|
||||
- `outline`
|
||||
- `snip`
|
||||
- `chunkcat`
|
||||
- `hitsnip`
|
||||
- `diagpick`
|
||||
- `logshape`
|
||||
- `stringscan`
|
||||
- Do not add `mercury-` prefixes.
|
||||
- The installer owns product-level grouping; the binaries stay small and task-specific.
|
||||
|
||||
## Installation Design
|
||||
|
||||
### Main Installation Path
|
||||
|
||||
- Add `scripts/install-toolbox.ps1`
|
||||
- Default behavior:
|
||||
- verify `cargo` exists
|
||||
- run `cargo build --release --workspace`
|
||||
- collect all known toolbox executables from `target\release`
|
||||
- create `$env:LOCALAPPDATA\MercuryToolbox\bin` if needed
|
||||
- copy all toolbox executables into that directory
|
||||
- inspect the user PATH
|
||||
- append the install directory if missing
|
||||
- print a compact summary of installed commands and PATH status
|
||||
|
||||
### PATH Policy
|
||||
|
||||
- PATH updates target the user-level environment variable, not machine-wide PATH.
|
||||
- The script should avoid duplicate path entries by normalizing path comparison.
|
||||
- Current-session PATH may also be patched so the commands are usable immediately after install without opening a new shell.
|
||||
|
||||
### Safety And UX
|
||||
|
||||
- Support `-NoPathUpdate` for explicit opt-out.
|
||||
- Support `-InstallRoot <PATH>` for advanced/manual installs, while defaulting to `$env:LOCALAPPDATA\MercuryToolbox`.
|
||||
- Support `-Configuration Debug|Release`, defaulting to `Release`.
|
||||
- Text output should clearly distinguish:
|
||||
- install root
|
||||
- bin directory
|
||||
- copied binaries
|
||||
- whether PATH was already configured, updated, or skipped
|
||||
|
||||
### Uninstall Path
|
||||
|
||||
- Add `scripts/uninstall-toolbox.ps1`
|
||||
- Default behavior:
|
||||
- remove installed binaries from the Mercury Toolbox bin directory
|
||||
- remove the Mercury Toolbox bin directory from the user PATH when present
|
||||
- leave unrelated files untouched
|
||||
|
||||
### Fallback Installation Path
|
||||
|
||||
- README also documents manual per-command install via `cargo install --path crates/<command>`.
|
||||
- This is a fallback path for Rust-native users, not the primary recommendation.
|
||||
|
||||
## Command Designs
|
||||
|
||||
### `hitsnip`
|
||||
|
||||
#### Purpose
|
||||
|
||||
Convert search hits into deduplicated, compact context windows so neither humans nor AI have to manually reopen each hit.
|
||||
|
||||
#### Inputs
|
||||
|
||||
- Explicit files plus line references in one of these forms:
|
||||
- `path:line`
|
||||
- `path:line:column`
|
||||
- stdin text from tools such as `rg -n`
|
||||
- JSONL hit records with at least `path` and `line`
|
||||
|
||||
#### Output
|
||||
|
||||
- Text mode:
|
||||
- one header per merged snippet
|
||||
- numbered lines under each snippet
|
||||
- compact reason metadata such as hit count and merged line span
|
||||
- JSON mode:
|
||||
- one JSON array of snippet objects
|
||||
- fields:
|
||||
- `path`
|
||||
- `start_line`
|
||||
- `end_line`
|
||||
- `hit_lines`
|
||||
- `hit_count`
|
||||
- `lines`
|
||||
|
||||
#### Behavior
|
||||
|
||||
- Merge nearby hits in the same file when the distance between hit windows is at most `--max-gap`.
|
||||
- Expand each hit by `--context` lines.
|
||||
- Deduplicate repeated identical hit lines.
|
||||
- Ignore malformed lines with clear usage/runtime errors rather than panicking.
|
||||
|
||||
#### Key Flags
|
||||
|
||||
- `--context <N>`
|
||||
- `--max-gap <N>`
|
||||
- `--limit <N>`
|
||||
- `--input-format auto|lines|jsonl`
|
||||
- shared `--json`
|
||||
|
||||
### `diagpick`
|
||||
|
||||
#### Purpose
|
||||
|
||||
Extract actionable diagnostics from compiler, build, and runtime logs and optionally attach source context.
|
||||
|
||||
#### Inputs
|
||||
|
||||
- Plain text logs from stdin or files
|
||||
- JSONL records with fields such as `path`, `line`, `column`, `severity`, `message`
|
||||
|
||||
#### Recognized Text Patterns
|
||||
|
||||
- Rust diagnostics:
|
||||
- `error[E0425]: ...`
|
||||
- `--> path:line:column`
|
||||
- MSVC/C#/Unity-style:
|
||||
- `path(line,column): error CSxxxx: ...`
|
||||
- `path:line:column: error: ...`
|
||||
- Generic runtime stack/log references where a path and line are present
|
||||
|
||||
#### Output
|
||||
|
||||
- Text mode:
|
||||
- one compact record per diagnostic
|
||||
- optional source snippet below when `--with-source` is enabled
|
||||
- JSON mode:
|
||||
- array of diagnostics with stable fields:
|
||||
- `path`
|
||||
- `line`
|
||||
- `column`
|
||||
- `severity`
|
||||
- `code`
|
||||
- `message`
|
||||
- `source`
|
||||
- `tool_hint`
|
||||
|
||||
#### Key Flags
|
||||
|
||||
- `--with-source`
|
||||
- `--context <N>`
|
||||
- `--limit <N>`
|
||||
- `--severity error|warning|note|all`
|
||||
|
||||
### `logshape`
|
||||
|
||||
#### Purpose
|
||||
|
||||
Collapse repetitive logs into pattern groups before anyone spends tokens reading the raw stream.
|
||||
|
||||
#### Inputs
|
||||
|
||||
- Plain log lines from stdin or files
|
||||
|
||||
#### Heuristic Normalization
|
||||
|
||||
- Replace volatile fragments with placeholders:
|
||||
- timestamps
|
||||
- decimal and hex numbers
|
||||
- UUID-like tokens
|
||||
- long paths and addresses
|
||||
- Preserve level-like prefixes such as `INFO`, `WARN`, `ERROR` when possible
|
||||
|
||||
#### Output
|
||||
|
||||
- Text mode:
|
||||
- one compact line per template
|
||||
- includes count and a representative sample
|
||||
- JSON mode:
|
||||
- array of groups with fields:
|
||||
- `pattern`
|
||||
- `count`
|
||||
- `first_line`
|
||||
- `last_line`
|
||||
- `sample`
|
||||
|
||||
#### Key Flags
|
||||
|
||||
- `--top <N>`
|
||||
- `--min-count <N>`
|
||||
- `--show-samples`
|
||||
- `--keep-level`
|
||||
|
||||
### `stringscan`
|
||||
|
||||
#### Purpose
|
||||
|
||||
Expose the high-signal strings inside binaries, generated files, and opaque artifacts for reverse engineering and quick triage.
|
||||
|
||||
#### Inputs
|
||||
|
||||
- One or more file paths from argv or stdin
|
||||
|
||||
#### Output
|
||||
|
||||
- Text mode:
|
||||
- one compact summary line per file in summary mode
|
||||
- optional per-string output when filtering by category
|
||||
- JSON mode:
|
||||
- per-file reports with:
|
||||
- `path`
|
||||
- `is_binary`
|
||||
- `string_count`
|
||||
- `categories`
|
||||
- `matches`
|
||||
|
||||
#### Heuristic Categories
|
||||
|
||||
- `url`
|
||||
- `path`
|
||||
- `dll`
|
||||
- `namespace`
|
||||
- `unity`
|
||||
- `dotnet`
|
||||
- `il2cpp`
|
||||
- `bepinex`
|
||||
- `generic`
|
||||
|
||||
#### Key Flags
|
||||
|
||||
- `--min-len <N>`
|
||||
- `--kind all|url|path|dll|namespace|unity|dotnet|il2cpp|bepinex|generic`
|
||||
- `--unique`
|
||||
- `--limit <N>`
|
||||
- `--details`
|
||||
|
||||
## Shared Fixtures
|
||||
|
||||
Add fixtures that support these commands without introducing heavyweight dependencies:
|
||||
|
||||
- `fixtures/hits/rg-output.txt`
|
||||
- `fixtures/diag/rust-errors.txt`
|
||||
- `fixtures/diag/unity-errors.txt`
|
||||
- `fixtures/logs/repetitive.log`
|
||||
- `fixtures/binaries/stringscan-sample.bin`
|
||||
|
||||
The binary fixture should embed obvious markers such as URLs, DLL names, Unity namespace fragments, and BepInEx-like strings so category detection can be tested reliably.
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
- Unit tests:
|
||||
- hit parsing and merge planning for `hitsnip`
|
||||
- diagnostic parsing helpers for `diagpick`
|
||||
- normalization and grouping for `logshape`
|
||||
- string extraction and category heuristics for `stringscan`
|
||||
- installer path normalization and PATH edit helpers
|
||||
- Integration tests:
|
||||
- `--help` examples for all new binaries
|
||||
- PowerShell pipeline flows
|
||||
- JSON output shape
|
||||
- install/uninstall script dry-run or temp-root behavior
|
||||
- Verification gates stay under the existing Jade standard, including `check-jade.ps1`.
|
||||
|
||||
## Risks
|
||||
|
||||
- Search-hit parsing can get messy when input mixes drive-letter paths and colon-separated line syntax.
|
||||
- Prefer a path-aware parser that handles Windows drive prefixes before splitting on the final line/column segments.
|
||||
- Diagnostic parsers can drift when tools change exact wording.
|
||||
- Keep the patterns heuristic and additive rather than pretending to be a full parser for each toolchain.
|
||||
- Over-normalizing logs can merge meaningfully different failures.
|
||||
- Keep the normalization rule set conservative in v1.
|
||||
- Binary string scanning can explode in output volume.
|
||||
- Default to concise summaries and require explicit detail flags for large per-string dumps.
|
||||
- PATH editing is user-hostile if it duplicates entries or stomps unrelated content.
|
||||
- Use minimal, append-only edits with idempotent detection.
|
||||
@@ -0,0 +1,137 @@
|
||||
# Toolbox Reading V2 Design
|
||||
|
||||
## Goal
|
||||
|
||||
Add four new PowerShell-friendly, AI-friendly reading commands to the toolbox so the workflow can move from "find a file" to "read the right part" without falling back to `Get-Content` or wasting tokens on full-file dumps.
|
||||
|
||||
## Scope
|
||||
|
||||
- Add `snip` for precise snippet extraction from files or stdin.
|
||||
- Add `outline` for heuristic structure summaries of common source and config files.
|
||||
- Add `fileprobe` for fast file-type and usefulness heuristics before opening a file.
|
||||
- Add `chunkcat` for deterministic chunk listing and chunk extraction from large text files.
|
||||
- Add help examples and README usage for all four commands.
|
||||
- Reuse the existing workspace CLI contract: shared `--json`, `--input-format`, exit codes, and PowerShell pipe support.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- No TUI, pager, fuzzy picker, or interactive mode.
|
||||
- No tree-sitter, Roslyn, or heavyweight language parser in this iteration.
|
||||
- No write or patch workflow in this batch.
|
||||
- No token estimation command, preview patch command, or replacement engine in this batch.
|
||||
|
||||
## Design Direction
|
||||
|
||||
- Prefer heuristic parsing over heavyweight parsing.
|
||||
- This keeps the binaries small, predictable, and fast enough for ad hoc shell use.
|
||||
- False positives are acceptable when they are clearly labeled as heuristic output.
|
||||
- Prefer compact text output that is useful to both humans and AI.
|
||||
- Text mode should avoid banners and avoid dumping redundant metadata.
|
||||
- JSON mode should expose stable machine fields so scripts can select the next action.
|
||||
- Treat these commands as a chain rather than isolated tools.
|
||||
- `fileprobe` answers "what is this file and should I read it?"
|
||||
- `outline` answers "where is the interesting structure?"
|
||||
- `snip` answers "show me the exact region."
|
||||
- `chunkcat` answers "how do I traverse this large file safely?"
|
||||
|
||||
## Command Designs
|
||||
|
||||
### `snip`
|
||||
|
||||
- Input:
|
||||
- one or more file paths from argv
|
||||
- or stdin content when piped
|
||||
- Selectors:
|
||||
- `--lines <START[:END]>`
|
||||
- `--around <REGEX>`
|
||||
- `--symbol <NAME>`
|
||||
- exactly one selector must be present
|
||||
- Supporting flags:
|
||||
- `--context <N>` for `--around` and `--symbol`
|
||||
- `--max-matches <N>` for `--around`
|
||||
- Output:
|
||||
- text mode emits `path:start-end reason=...` followed by numbered lines
|
||||
- JSON mode emits an array of snippets with `path`, `start_line`, `end_line`, `reason`, and `lines`
|
||||
- Heuristic behavior:
|
||||
- `--symbol` uses language-aware regexes for Rust and C# plus generic fallbacks for other text files
|
||||
- when possible, symbol extraction expands to a balanced block instead of a single line
|
||||
|
||||
### `outline`
|
||||
|
||||
- Input:
|
||||
- one or more file paths from argv
|
||||
- stdin paths in line mode
|
||||
- Supported heuristic families:
|
||||
- Rust: `mod`, `struct`, `enum`, `trait`, `impl`, `fn`, `const`, `static`, `type`
|
||||
- C#: `namespace`, `class`, `struct`, `enum`, `interface`, `record`, method-like members
|
||||
- JSON: object keys traversed by depth
|
||||
- TOML: tables and keys
|
||||
- YAML: indentation-based key outline
|
||||
- Flags:
|
||||
- `--depth <N>` to cap nested output
|
||||
- `--kind all|code|config`
|
||||
- Output:
|
||||
- text mode emits one compact line per item: `line depth kind name`
|
||||
- JSON mode emits per-file objects with stable `items`
|
||||
- Heuristic behavior:
|
||||
- items are marked by file-relative line number and depth, not claimed as exact AST nodes
|
||||
|
||||
### `fileprobe`
|
||||
|
||||
- Input:
|
||||
- one or more file paths from argv or stdin
|
||||
- Output fields:
|
||||
- path, extension, size, modified time
|
||||
- `exists`, `is_dir`, `is_binary`, `encoding_hint`
|
||||
- `family` such as `source`, `config`, `data`, `binary`, `archive`, `unknown`
|
||||
- `language_hint`
|
||||
- `line_count`, `blank_lines`, `longest_line`
|
||||
- heuristic flags such as `likely_generated`, `likely_minified`, `likely_test`, `likely_lockfile`, `likely_vendor`
|
||||
- optional `container_hint` such as `pe`, `zip`, `sqlite`, `pdf`
|
||||
- Heuristic behavior:
|
||||
- detect binary vs text from bytes and UTF-8 validity
|
||||
- infer family and language from extension plus lightweight content checks
|
||||
- do not parse PE deeply here; `binmeta` remains the dedicated PE inspector
|
||||
|
||||
### `chunkcat`
|
||||
|
||||
- Input:
|
||||
- one text file path at a time in v1
|
||||
- Flags:
|
||||
- `--max-lines <N>` defaulting to a budget-friendly size
|
||||
- `--overlap <N>` for deterministic overlap between chunks
|
||||
- `--chunk <INDEX>` to emit a specific chunk
|
||||
- Output:
|
||||
- without `--chunk`, emit chunk inventory only
|
||||
- with `--chunk`, emit the selected chunk with numbered lines
|
||||
- JSON mode emits chunk metadata, and chunk content only when `--chunk` is set
|
||||
- Behavior:
|
||||
- chunking is line-based and deterministic
|
||||
- line ranges are stable for the same file contents and options
|
||||
- overlapping chunks use a fixed stride of `max_lines - overlap`
|
||||
|
||||
## Shared Testing Strategy
|
||||
|
||||
- Add unit tests for:
|
||||
- selector parsing
|
||||
- heuristic file classification
|
||||
- chunk calculation
|
||||
- outline extraction helpers
|
||||
- Add integration tests for:
|
||||
- `--help` examples
|
||||
- PowerShell pipeline scenarios
|
||||
- JSON output shape for automation
|
||||
- Add fixtures for:
|
||||
- Rust source with multiple symbols
|
||||
- C# source with Unity-style class structure
|
||||
- JSON, TOML, and YAML config files
|
||||
- minified/generated-ish text and simple binary-like samples
|
||||
|
||||
## Risks
|
||||
|
||||
- Heuristic structure detection can misclassify edge-case syntax.
|
||||
- This is acceptable if outputs are useful and clearly heuristic.
|
||||
- `snip --symbol` block expansion can drift on malformed files.
|
||||
- Prefer a safe fallback to line-only snippets over panics or empty output.
|
||||
- `chunkcat` can become noisy if default chunk size is too small.
|
||||
- Pick a conservative default tuned for AI reading, not log streaming.
|
||||
@@ -0,0 +1,80 @@
|
||||
# Toolbox V1.5 Design
|
||||
|
||||
## Goal
|
||||
|
||||
Promote the existing four commands from "usable prototypes" to daily-driver tools by pairing feature upgrades with first-class help and documentation, and add a production-grade `binmeta` command for fast binary triage in AI-heavy terminal workflows.
|
||||
|
||||
## Scope
|
||||
|
||||
- Upgrade `jsonlgrep` with nested field paths and negative field predicates.
|
||||
- Upgrade `recent` with regex-based name filtering.
|
||||
- Upgrade `pathshadow` with clearer shadowing explanations in text and JSON output.
|
||||
- Upgrade `portping` with selectable HTTP method and expected status checks.
|
||||
- Add concrete examples to `--help` output for all commands.
|
||||
- Update `README.md` with scenario-driven examples and PowerShell pipeline usage.
|
||||
- Add `binmeta` as a new CLI command for binary metadata inspection on Windows PE files, with graceful handling for non-PE files.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- No GUI or TUI work.
|
||||
- No installer, release automation, or Scoop packaging in this iteration.
|
||||
- No multi-format binary analysis beyond production-grade PE support plus safe fallback summaries for unknown formats.
|
||||
|
||||
## Command Changes
|
||||
|
||||
### `jsonlgrep`
|
||||
|
||||
- Extend query syntax to support `field!=value` and `field!~=regex`.
|
||||
- Resolve dotted field paths such as `event.user.name`.
|
||||
- Treat scalar JSON values consistently so exact matches and text projections also work for numbers, booleans, and null.
|
||||
- Improve `--help` with copy-pasteable file and pipeline examples.
|
||||
|
||||
### `recent`
|
||||
|
||||
- Add `--name <REGEX>` to filter by file or directory basename.
|
||||
- Preserve `.gitignore`-aware traversal and current sorting behavior.
|
||||
- Improve `--help` with examples for recent Rust work, recent directories, and JSON pipeline usage.
|
||||
|
||||
### `pathshadow`
|
||||
|
||||
- Append stable explanation fields to JSON output so scripts and AI can tell why an entry won or lost.
|
||||
- Include PATH rank and winner path context where applicable.
|
||||
- Keep text output compact but clearer about origin and shadowing reason.
|
||||
- Improve `--help` with explicit PATH diagnosis examples.
|
||||
|
||||
### `portping`
|
||||
|
||||
- Add `--method GET|HEAD` for HTTP(S) requests.
|
||||
- Add `--expect-status` to turn mismatched HTTP status codes into probe failure while keeping actual status visible.
|
||||
- Preserve TCP behavior unchanged.
|
||||
- Improve `--help` with health-check and HEAD examples.
|
||||
|
||||
### `binmeta`
|
||||
|
||||
- Input: one or more file paths from argv or stdin lines.
|
||||
- Text output: one concise summary line per file for human scanning.
|
||||
- JSON output: one object per file with stable fields for automation and AI use.
|
||||
- PE analysis:
|
||||
- file kind, machine/architecture, subsystem, executable vs DLL hint, PE timestamp
|
||||
- section summaries
|
||||
- imported DLLs and imported symbol count summary
|
||||
- exported symbol count summary when available
|
||||
- SHA-256, file size, modified time
|
||||
- heuristic hints for `.NET`, Unity, IL2CPP, Mono, and BepInEx-related artifacts
|
||||
- Non-PE fallback:
|
||||
- still emit hash, size, modified time, file extension, and "not_pe" kind without failing the whole command
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
- Add focused unit tests for new parsing helpers and negative query logic.
|
||||
- Expand integration tests to cover help examples and new CLI flags.
|
||||
- Add `binmeta` integration tests against:
|
||||
- a known text fixture for non-PE fallback
|
||||
- compiled workspace binaries for PE parsing
|
||||
- Keep the existing Jade verification gate unchanged.
|
||||
|
||||
## Risks
|
||||
|
||||
- PE parsing APIs can be noisy; the implementation should wrap parser details behind small helpers.
|
||||
- `binmeta` hints must remain heuristic and clearly labeled as hints, not definitive claims.
|
||||
- Help text examples must stay in sync with actual behavior, so tests should assert representative help fragments.
|
||||
Reference in New Issue
Block a user