chore(release): prepare public source release

2026-07-18 15:41:59 +08:00
commit e365e5df4d
508 changed files with 163373 additions and 0 deletions
@@ -0,0 +1,137 @@
+# Toolbox Reading V2 Design
+
+## Goal
+
+Add four new PowerShell-friendly, AI-friendly reading commands to the toolbox so the workflow can move from "find a file" to "read the right part" without falling back to `Get-Content` or wasting tokens on full-file dumps.
+
+## Scope
+
+- Add `snip` for precise snippet extraction from files or stdin.
+- Add `outline` for heuristic structure summaries of common source and config files.
+- Add `fileprobe` for fast file-type and usefulness heuristics before opening a file.
+- Add `chunkcat` for deterministic chunk listing and chunk extraction from large text files.
+- Add help examples and README usage for all four commands.
+- Reuse the existing workspace CLI contract: shared `--json`, `--input-format`, exit codes, and PowerShell pipe support.
+
+## Non-Goals
+
+- No TUI, pager, fuzzy picker, or interactive mode.
+- No tree-sitter, Roslyn, or heavyweight language parser in this iteration.
+- No write or patch workflow in this batch.
+- No token estimation command, preview patch command, or replacement engine in this batch.
+
+## Design Direction
+
+- Prefer heuristic parsing over heavyweight parsing.
+  - This keeps the binaries small, predictable, and fast enough for ad hoc shell use.
+  - False positives are acceptable when they are clearly labeled as heuristic output.
+- Prefer compact text output that is useful to both humans and AI.
+  - Text mode should avoid banners and avoid dumping redundant metadata.
+  - JSON mode should expose stable machine fields so scripts can select the next action.
+- Treat these commands as a chain rather than isolated tools.
+  - `fileprobe` answers "what is this file and should I read it?"
+  - `outline` answers "where is the interesting structure?"
+  - `snip` answers "show me the exact region."
+  - `chunkcat` answers "how do I traverse this large file safely?"
+
+## Command Designs
+
+### `snip`
+
+- Input:
+  - one or more file paths from argv
+  - or stdin content when piped
+- Selectors:
+  - `--lines <START[:END]>`
+  - `--around <REGEX>`
+  - `--symbol <NAME>`
+  - exactly one selector must be present
+- Supporting flags:
+  - `--context <N>` for `--around` and `--symbol`
+  - `--max-matches <N>` for `--around`
+- Output:
+  - text mode emits `path:start-end reason=...` followed by numbered lines
+  - JSON mode emits an array of snippets with `path`, `start_line`, `end_line`, `reason`, and `lines`
+- Heuristic behavior:
+  - `--symbol` uses language-aware regexes for Rust and C# plus generic fallbacks for other text files
+  - when possible, symbol extraction expands to a balanced block instead of a single line
+
+### `outline`
+
+- Input:
+  - one or more file paths from argv
+  - stdin paths in line mode
+- Supported heuristic families:
+  - Rust: `mod`, `struct`, `enum`, `trait`, `impl`, `fn`, `const`, `static`, `type`
+  - C#: `namespace`, `class`, `struct`, `enum`, `interface`, `record`, method-like members
+  - JSON: object keys traversed by depth
+  - TOML: tables and keys
+  - YAML: indentation-based key outline
+- Flags:
+  - `--depth <N>` to cap nested output
+  - `--kind all|code|config`
+- Output:
+  - text mode emits one compact line per item: `line depth kind name`
+  - JSON mode emits per-file objects with stable `items`
+- Heuristic behavior:
+  - items are marked by file-relative line number and depth, not claimed as exact AST nodes
+
+### `fileprobe`
+
+- Input:
+  - one or more file paths from argv or stdin
+- Output fields:
+  - path, extension, size, modified time
+  - `exists`, `is_dir`, `is_binary`, `encoding_hint`
+  - `family` such as `source`, `config`, `data`, `binary`, `archive`, `unknown`
+  - `language_hint`
+  - `line_count`, `blank_lines`, `longest_line`
+  - heuristic flags such as `likely_generated`, `likely_minified`, `likely_test`, `likely_lockfile`, `likely_vendor`
+  - optional `container_hint` such as `pe`, `zip`, `sqlite`, `pdf`
+- Heuristic behavior:
+  - detect binary vs text from bytes and UTF-8 validity
+  - infer family and language from extension plus lightweight content checks
+  - do not parse PE deeply here; `binmeta` remains the dedicated PE inspector
+
+### `chunkcat`
+
+- Input:
+  - one text file path at a time in v1
+- Flags:
+  - `--max-lines <N>` defaulting to a budget-friendly size
+  - `--overlap <N>` for deterministic overlap between chunks
+  - `--chunk <INDEX>` to emit a specific chunk
+- Output:
+  - without `--chunk`, emit chunk inventory only
+  - with `--chunk`, emit the selected chunk with numbered lines
+  - JSON mode emits chunk metadata, and chunk content only when `--chunk` is set
+- Behavior:
+  - chunking is line-based and deterministic
+  - line ranges are stable for the same file contents and options
+  - overlapping chunks use a fixed stride of `max_lines - overlap`
+
+## Shared Testing Strategy
+
+- Add unit tests for:
+  - selector parsing
+  - heuristic file classification
+  - chunk calculation
+  - outline extraction helpers
+- Add integration tests for:
+  - `--help` examples
+  - PowerShell pipeline scenarios
+  - JSON output shape for automation
+- Add fixtures for:
+  - Rust source with multiple symbols
+  - C# source with Unity-style class structure
+  - JSON, TOML, and YAML config files
+  - minified/generated-ish text and simple binary-like samples
+
+## Risks
+
+- Heuristic structure detection can misclassify edge-case syntax.
+  - This is acceptable if outputs are useful and clearly heuristic.
+- `snip --symbol` block expansion can drift on malformed files.
+  - Prefer a safe fallback to line-only snippets over panics or empty output.
+- `chunkcat` can become noisy if default chunk size is too small.
+  - Pick a conservative default tuned for AI reading, not log streaming.