MercuryToolbox/docs/superpowers/specs/2026-04-21-toolbox-reading-v2-design.md

Toolbox Reading V2 Design

Goal

Add four new PowerShell-friendly, AI-friendly reading commands to the toolbox so the workflow can move from "find a file" to "read the right part" without falling back to Get-Content or wasting tokens on full-file dumps.

Scope

• Add snip for precise snippet extraction from files or stdin.
• Add outline for heuristic structure summaries of common source and config files.
• Add fileprobe for fast file-type and usefulness heuristics before opening a file.
• Add chunkcat for deterministic chunk listing and chunk extraction from large text files.
• Add help examples and README usage for all four commands.
• Reuse the existing workspace CLI contract: shared --json, --input-format, exit codes, and PowerShell pipe support.

Non-Goals

• No TUI, pager, fuzzy picker, or interactive mode.
• No tree-sitter, Roslyn, or heavyweight language parser in this iteration.
• No write or patch workflow in this batch.
• No token estimation command, preview patch command, or replacement engine in this batch.

Design Direction

• Prefer heuristic parsing over heavyweight parsing.
  • This keeps the binaries small, predictable, and fast enough for ad hoc shell use.
  • False positives are acceptable when they are clearly labeled as heuristic output.
• Prefer compact text output that is useful to both humans and AI.
  • Text mode should avoid banners and avoid dumping redundant metadata.
  • JSON mode should expose stable machine fields so scripts can select the next action.
• Treat these commands as a chain rather than isolated tools.
  • fileprobe answers "what is this file and should I read it?"
  • outline answers "where is the interesting structure?"
  • snip answers "show me the exact region."
  • chunkcat answers "how do I traverse this large file safely?"

Command Designs

snip

• Input:
  • one or more file paths from argv
  • or stdin content when piped
• Selectors:
  • --lines <START[:END]>
  • --around <REGEX>
  • --symbol <NAME>
  • exactly one selector must be present
• Supporting flags:
  • --context <N> for --around and --symbol
  • --max-matches <N> for --around
• Output:
  • text mode emits path:start-end reason=... followed by numbered lines
  • JSON mode emits an array of snippets with path, start_line, end_line, reason, and lines
• Heuristic behavior:
  • --symbol uses language-aware regexes for Rust and C# plus generic fallbacks for other text files
  • when possible, symbol extraction expands to a balanced block instead of a single line

outline

• Input:
  • one or more file paths from argv
  • stdin paths in line mode
• Supported heuristic families:
  • Rust: mod, struct, enum, trait, impl, fn, const, static, type
  • C#: namespace, class, struct, enum, interface, record, method-like members
  • JSON: object keys traversed by depth
  • TOML: tables and keys
  • YAML: indentation-based key outline
• Flags:
  • --depth <N> to cap nested output
  • --kind all|code|config
• Output:
  • text mode emits one compact line per item: line depth kind name
  • JSON mode emits per-file objects with stable items
• Heuristic behavior:
  • items are marked by file-relative line number and depth, not claimed as exact AST nodes

fileprobe

• Input:
  • one or more file paths from argv or stdin
• Output fields:
  • path, extension, size, modified time
  • exists, is_dir, is_binary, encoding_hint
  • family such as source, config, data, binary, archive, unknown
  • language_hint
  • line_count, blank_lines, longest_line
  • heuristic flags such as likely_generated, likely_minified, likely_test, likely_lockfile, likely_vendor
  • optional container_hint such as pe, zip, sqlite, pdf
• Heuristic behavior:
  • detect binary vs text from bytes and UTF-8 validity
  • infer family and language from extension plus lightweight content checks
  • do not parse PE deeply here; binmeta remains the dedicated PE inspector

chunkcat

• Input:
  • one text file path at a time in v1
• Flags:
  • --max-lines <N> defaulting to a budget-friendly size
  • --overlap <N> for deterministic overlap between chunks
  • --chunk <INDEX> to emit a specific chunk
• Output:
  • without --chunk, emit chunk inventory only
  • with --chunk, emit the selected chunk with numbered lines
  • JSON mode emits chunk metadata, and chunk content only when --chunk is set
• Behavior:
  • chunking is line-based and deterministic
  • line ranges are stable for the same file contents and options
  • overlapping chunks use a fixed stride of max_lines - overlap

Shared Testing Strategy

• Add unit tests for:
  • selector parsing
  • heuristic file classification
  • chunk calculation
  • outline extraction helpers
• Add integration tests for:
  • --help examples
  • PowerShell pipeline scenarios
  • JSON output shape for automation
• Add fixtures for:
  • Rust source with multiple symbols
  • C# source with Unity-style class structure
  • JSON, TOML, and YAML config files
  • minified/generated-ish text and simple binary-like samples

Risks

• Heuristic structure detection can misclassify edge-case syntax.
  • This is acceptable if outputs are useful and clearly heuristic.
• snip --symbol block expansion can drift on malformed files.
  • Prefer a safe fallback to line-only snippets over panics or empty output.
• chunkcat can become noisy if default chunk size is too small.
  • Pick a conservative default tuned for AI reading, not log streaming.