sift Docs / sift 文档

English

sift is a cost-controlled open-source project auditor for dependency and repository intake. It is built around a tiered funnel: static AST dehydration first, deterministic local findings second, and large-model convergence only after the input has been reduced.

The primary product surface is a local CLI that can inspect a project path or safely fetch a GitHub repository before setup, install, build, or agent execution:

sift ./repo --agent-gate
sift ./repo --benchmark
sift github owner/repo --ref main --agent-gate

The deterministic agent gate emits a stable pre-run verdict:

VERDICT: ACCEPT | CAUTION | REJECT | INCOMPLETE
SAFE_TO_AGENT_RUN: yes | no

• English Overview
• Roadmap
• Contributor Handbook

中文

sift 是一个面向依赖引入和仓库预审的可控成本开源项目审计器。它先做静态 AST 脱水，再生成确定性本地发现，最后只把压缩后的输入交给大模型收敛。

主要入口是本地 CLI：可以审本地项目，也可以在 setup、install、build 或 agent 执行之前安全获取 GitHub 仓库并做门禁判断。

sift ./repo --agent-gate
sift ./repo --benchmark
sift github owner/repo --ref main --agent-gate

确定性 agent gate 输出稳定预运行 verdict：

VERDICT: ACCEPT | CAUTION | REJECT | INCOMPLETE
SAFE_TO_AGENT_RUN: yes | no

• 中文概览
• 路线图
• 贡献者手册

sift

English | 中文

Cost-controlled open-source project auditor: tiered funnel + compute mismatch + ReACT scheduling. Before adopting a dependency, get a file/line-level risk ledger without force-feeding tens of thousands of lines into a frontier model.

• Grunt work (structure extraction / deterministic coarse filtering) -> tree-sitter + local rules
• Logic convergence -> frontier large model, orchestrated by a ReACT state machine over deterministic findings
• Single binary, zero-config; audits a whole project or a single module; sift must pass its internal release gates

See Roadmap for full design.

Usage

sift ./repo --scan-only        # scan layer only (no key needed)
sift ./repo --agent-gate       # deterministic pre-run gate (no key needed)
sift ./repo --agent-gate --format json
sift ./repo --benchmark        # scan/model budget telemetry JSON (no key needed)
sift github owner/repo         # safe GitHub intake, defaults to --agent-gate
sift github owner/repo --ref main --scan-only
sift eval-corpus               # run the checked-in repo-intake precision corpus
sift ./repo --module src        # audit a submodule
SIFT_API_KEY=<KEY> sift ./repo  # full pipeline
sift ./repo --api-key-file ~/.sift/key
sift ./repo --report-language zh # request a Simplified Chinese Markdown report
sift ./repo --debug              # print extra diagnostics to stderr
sift doctor                    # check config, key_env, and endpoint/key mismatches

--agent-gate is a local, deterministic repo-intake gate for agents and wrapper scripts. It writes only this stable contract to stdout:

VERDICT: ACCEPT | CAUTION | REJECT | INCOMPLETE
WHY:
- <top evidence>
BLOCKERS:
- <file:line evidence or coverage blocker>
SAFE_TO_AGENT_RUN: yes | no

The command exits 0 only when SAFE_TO_AGENT_RUN: yes; CAUTION, REJECT, and INCOMPLETE exit non-zero so callers can stop before setup, install, build, or run steps.

Use --format json with --agent-gate for automation. The JSON contract contains verdict, safe_to_agent_run, exit_reason, coverage, findings, blockers, artifact inventory, truncation details, and policy actions.

The deterministic supply-chain layer currently flags npm install lifecycle scripts, manifest/lockfile reproducibility gaps, git/path/http dependency sources, Rust build.rs command boundaries, shell/Dockerfile download-execute patterns, base64 decode-to-execute flows, GitHub Actions permission/trigger risk, secrets coupled to shell execution, unpinned GitHub Actions, Dockerfile root/remote repository patterns, and suspicious binary/archive artifacts.

sift github accepts owner/repo or https://github.com/owner/repo, fetches a temporary checkout with git, resolves the commit SHA, then runs the local scan/gate/benchmark pipeline against that checkout. It never runs repository code, package manager commands, build scripts, hooks, install commands, or submodules. The checkout is inspected for file/byte limits, .gitmodules, and Git LFS indicators before scanning. Temporary checkouts are removed by default; use --keep-checkout only when you need to inspect the fetched tree.

Project-local policy lives in sift-policy.toml. It supports max_candidate_files, [[allowlist]], [[denylist]], and [[severity_override]] entries keyed by path, rule, severity, and reason; applied policy decisions are shown in text and JSON gate output.

The current full-audit path does not call small-model Map by default. It converges from the deterministic ledger with the configured large model, while the small-model Map implementation remains an experimental diagnostic path.

Supported Languages

The scan layer currently dehydrates Rust, Python, Go, JavaScript, TypeScript/TSX, HTML, CSS, Zig, Bash-compatible shell files (.sh, .bash, .zsh), Dart, Kotlin, Java, C/C++, C#, PHP, Swift, Ruby, SQL, Dockerfile/Containerfile, YAML, HCL/Terraform, Vue, Svelte, package.json, common package manifests/lockfiles, Makefile, and Markdown install snippets.

Install

make ci
make install

More

• Roadmap
• Contributor Handbook

sift Project Profile & Roadmap

English | 中文

North star + guardrails + phased build boundaries. Defines what it should become / what it must never do / what each phase ships / when internal gates apply. Name: sift (CLI is sift). Language: Rust.

Overview

A cost-controlled open-source project auditor. Before adopting a library, get a file/line-level risk ledger without trial-running it or force-feeding tens of thousands of lines into a frontier model.

Core: tiered funnel + compute mismatch + ReACT scheduling. Grunt work (structure extraction, coarse filtering) currently goes to zero-cost static parsing and deterministic local rules; heavy logic convergence goes to a frontier model; a ReACT state machine orchestrates the Reduce pass over deterministic findings. Ships as a single binary, zero-config, auditing a whole project or a single module. sift itself must pass its internal release gates.

• Architecture

  CLI key file / ENV / ~/.sift/config.toml ──(fallback resolve, exit if no key)
        ▼
  Scan      ignore::Walk → bounded channel (consume & drop)        [P0 ✓]
        ▼
  Tier-0    tree-sitter dehydrate (sig/import/calls) → JSON → drop AST  [P1 ✓]
        │   cross-boundary refs marked [EXTERNAL_BLACKBOX]
        ▼
  Models    multi-model registry · per-call hard timeout · breaker+backoff  [P2 ✓]
        ▼
  ReACT scheduler (tool protocol, deterministic findings, retry≤N)   [P3 ✓]
        │  └─ large model (Reduce convergence) ─────────┘
        ▼
  Report    stdout Markdown risk list (line/call-chain)            [P4 started]
        ▼
  Internal gate  scored source checks + release evidence          [P5/P6]

Project Profile (target state)

• Zero-friction cold start. sift ./repo --scan-only just runs; missing ~/.sift/config.toml is created with non-secret defaults; no interactive prompts; exits with an injection hint if the key is missing.
• Cost-controlled & budgetable. The deterministic baseline is local; the large model only sees the dehydrated skeleton when full audit is requested.
• Model orchestration. A ReACT state machine chains deterministic findings and large-model convergence; skills are compile-time local functions.
• Multi-model + concurrency. Multiple endpoints are configurable; scan/model concurrency remains bounded and observable.
• Never grind blindly. Every external call has a hard timeout; repeated failures trip the breaker; on trip, back off / degrade or emit a partial report — never hang.
• Engineering-grade by default. A clean-looking but incomplete audit is a defect. Any skipped input, truncation, fallback, partial model result, or invalid config must be visible and testable.
• Stable machine contracts. Scan JSONL, final Markdown, diagnostics, and generated reports have separate channels. Downstream scripts must be able to consume stdout without guessing whether it contains mixed formats.
• Memory decoupled from scale. Stream and drop; resident memory stays low.
• Internally gated. The project must pass its own maintainer-only release gates; modular, TDD-guarded, clear boundaries.
• Priority on conflict: robust > usable report > cheap > fast > small.

Non-goals (hard rules)

• No vector DB / embeddings / RAG. For one-shot low-frequency audits, index upkeep costs more than prompt assembly; plain-text pipeline, read once and discard.
• No runtime plugins / dynamic skills. Skills = compile-time enum + match local fns; extend by editing and recompiling.
• No service / Web UI / multi-tenant. One-shot CLI only.
• No process panics. Dirty data dropped & logged; hallucinations/bad JSON tripped; Result/Option throughout, no unwrap/expect.
• No unbounded blocking. Any subprocess/network/model call must have a deadline.
• Module audit must not balloon to global. Cross-boundary refs marked and handed to the large model; no chasing.
• No trial-run instead of audit. Value is the pre-adoption verdict.
• No scaffold masquerading as product. Placeholders are allowed only inside explicitly unfinished phases; they must not produce reports that look production-complete.
• No silent fallback. Invalid config, truncated seed, skipped files, missing model roles, and degraded model paths must fail loudly or be shown in the report.

Code Map

Every src/*.rs carries unit tests; new subsystem ⇒ tests built alongside (TDD). Module boundaries are responsibility boundaries.

src/main.rs       entry wiring: parse→Config→schedule→report→exit code
src/config.rs     fallback resolve, multi-model config         [P0✓→P2]
src/scanner.rs    Walk + bounded channel                       [P0✓]
src/extract.rs    tree-sitter dehydrate → AstSummary           [P1✓]
src/query.rs      stateless evidence query (rescan + regex)    [P1✓]
src/model.rs      multi-model registry/client trait/timeout    [P2✓]
src/react.rs      ReACT state machine + skill enum/match       [P3 ✓]
src/skills.rs     local skill fns (coarse filter / reduce)     [P3 ✓→P4]
src/report.rs     Markdown risk-list renderer                  [P4]
src/audit.rs      internal gate dimension scoring              [P5]

Multi-model & concurrency (config schema)

concurrency = 8          # scan/model concurrency cap
[[model]]
role = "small"           # reserved for experimental Map diagnostics
endpoint = "..."
key_env = "SIFT_SMALL_KEY"
timeout_ms = 8000
max_retries = 1
[[model]]
role = "large"
endpoint = "..."
key_env = "SIFT_API_KEY"
timeout_ms = 60000
max_retries = 1

Resolve order: CLI key file > ENV > toml > default; no large key ⇒ exit. The current full-audit path does not call small-role models by default; missing small models do not change the deterministic-ledger Reduce path. The default user config path is ~/.sift/config.toml; it is created on first run from config.example.toml-equivalent defaults and must not contain raw secrets.

Timeout, breaker & recovery (never grind)

• Per-call deadline: time out and drop; no unbounded wait.
• Breaker counter: consecutive failures / bad JSON / unknown skill ≥ N ⇒ break, stop I/O.
• Backoff recovery: transient errors retry with exponential backoff to budget; non-transient degrade (small→AST, large→partial).
• Budget cap: global token/time ceiling; on hit, force-converge a [TRUNCATED] report.

Engineering Contract

• A phase marked done must have behavior-level proof, not only type-level plumbing or happy-path unit tests.
• Full audit stdout is the final report stream. --scan-only is the JSONL stream. Diagnostics stay off stdout.
• Report coverage must disclose how much input was scanned, dehydrated, sent to models, skipped, or truncated.
• Config files are part of the trust boundary. Missing user config is auto-created from safe defaults; if a config file exists but is invalid, the process fails instead of reverting to defaults.
• Program source under src/ is English-only for runtime text, prompts, and comments; bilingual documentation stays in docs.

Phased Roadmap

Each phase: feature list / boundaries / internal gate. All-green gate ⇒ next phase; next steps set by gate evidence. For a point-in-time done/partial/pending snapshot of every item below against real evidence, see CHECKLIST.md.

P0 Scaffold — done ✓

Features: clap fallback resolve, bounded scanner, exit on missing key, minimal wiring. Bounds: no net/parse/tree. Gate: cargo build green, 0 unwrap, --scan-only scans, missing key exit1.

P1 Tier-0 AST dehydrate — done ✓

Features: tree-sitter Rust/Python/Go/JavaScript/TypeScript/HTML/CSS/Zig/Bash/Dart/Kotlin/Java/C/C++/C#/PHP/Swift/Ruby/SQL/Dockerfile/YAML/HCL/Vue/Svelte, extract sig/import/calls → flat AstSummary JSON; cross-boundary [EXTERNAL_BLACKBOX]; drop AST. Bounds: omit bodies/comments; tolerate malformed syntax without panicking and account for incomplete coverage in downstream reporting. Gate: 100MB repo memory stable & no crash; extract.rs tests cover typical+broken.

P2 Model layer (multi-model + breaker) — done ✓

Features: ModelClient trait, registry, role routing; per-call timeout, breaker, backoff. Bounds: no cache/persist; keys env/file only, never logged. Gate: timeout/bad-response simulated, breaker trips; no plaintext keys.

P3 ReACT scheduler — done ✓

Features: enum state machine, initial tool protocol prompt, large model emits <TOOL_CALL>, match-routes local skills via $SEED; retry≤N then partial. Bounds: compile-time skills, no dynamic load. Gate: bad JSON/unknown skill/N errors all trip; react.rs tested.

P4 Deterministic Reduce+report

Features: deterministic AST coarse ledger, Markdown renderer, real [[model]] TOML parsing, explicit input coverage, stable JSON agent-gate output, policy controls, artifact inventory, eval corpus, and clean stdout boundaries. Bounds: module mode slices root only; truncation and degraded model paths must be visible. Gate: hits seeded risks; module/project don’t bleed; full-audit stdout contains only the report; invalid config fails; fake-endpoint full audit smoke proves the user-facing path.

P5 Internal Quality Gate — done ✓

Features: audit.rs scores trimmed dimensions and writes maintainer-only reports to reports/ (gitignored). Gate: no FAIL/WARN for hard rules, including no broad dead-code allows, no Chinese source strings/comments, clean report stream boundary, and visible seed truncation.

P6 Release hardening

Features: ReleaseSafe single binary, Makefile install path, macOS Homebrew tap publishing, more grammars, stable JSON. Gate: single-file dist, internal gates pass, docs↔code consistent, brew install jamiesun/tap/sift backed by release checksums.

Definition of done

• Zero-config run; ~/.sift/config.toml auto-created; missing key exits with hint; never hangs.
• 100MB repo stable memory; no crash on dirty input.
• Report cites line numbers + cross-module deps + concurrency/resource risk.
• Report declares input coverage and truncation state; incomplete coverage never looks like a complete verdict.
• Every external call times out; failures trip to partial, never grind.
• One binary audits project and --module without bleed.
• Internal release gates have no FAIL or hard-rule WARN.

Suggestions (not rules): rayon, exact timeout/size/latency numbers per benchmark. Hard rules: single binary, fallback resolve, bounded channel, hard-timeout breaker, no unwrap, TDD, bilingual docs (EN default, ZH twin), passing internal gates.

sift Acceptance Checklist

English | 中文

A point-in-time acceptance snapshot of every feature and gate promised in ROADMAP.md, scoped strictly to its phases (P0–P6) and its non-goals. Nothing outside that boundary is graded here — see AGENT.md for the hard rules this checklist assumes. Line numbers are as of the snapshot commit below and may drift; prefer the named function/test when they disagree.

Snapshot

Legend

Mapped to the three buckets this checklist is meant to answer: 完成 = ✅, 待定 = ⏳, 未完成 = 🟡 / ⬜.

Type column: F = Feature bullet, G = Gate/acceptance-criterion bullet, B = Boundary constraint, taken verbatim from each phase’s ROADMAP.md text.

P0 — Scaffold

ROADMAP status: done ✓

Phase verdict: ✅ Done, with one test-coverage gap (#7).

P1 — Tier-0 AST dehydrate

ROADMAP status: done ✓

Phase verdict: 🟡 Mostly done. The only unverified gate is the 100 MB memory-stability claim, and macOS (a supported CI/release target) currently has no working resident-memory metric at all.

P2 — Model layer (multi-model + breaker)

ROADMAP status: done ✓

Phase verdict: ✅ Done.

P3 — ReACT scheduler

ROADMAP status: done ✓

Phase verdict: ✅ Done.

P4 — Deterministic Reduce + report

ROADMAP status: not marked done; README self-reports “in progress.” This is the phase carrying the most feature growth, so it is split into three groups below.

P4a — Deterministic ledger & Markdown report

P4b — Agent gate & policy

P4c — Operational modes

Phase verdict: 🟡 Mostly done — matches the project’s own “P4 in progress” self-report. The two genuinely open engineering items are #10 in P4a (no CI-automated full-audit smoke) and #3 in P4c (doctor untested); the small-model Map question (#7 in P4c) is an intentional open decision, not a bug.

P5 — Internal Quality Gate

ROADMAP status: heading now carries the ✓ (updated this session); the feature and its gate are fully built and green.

Phase verdict: ✅ Done. ROADMAP.md/ROADMAP.zh.md P5 headings were updated to — done ✓ this session to match this evidence. Remaining suggestion: tighten the BT check toward function-level coverage.

P6 — Release hardening

ROADMAP status: no checkmark in the heading; substantial evidence exists.

Phase verdict: 🟡 Mostly done. Two open threads: docs↔code consistency has no automated guard, and “more grammars” is an intentionally unbounded target rather than a gate to close.

Cross-cutting: Engineering Contract (ROADMAP.md)

Cross-cutting: Definition of Done (ROADMAP.md)

Non-goals guardrail

Confirms none of ROADMAP.md’s hard “must never do” rules have been crossed.

Self-audit dogfood check

AGENT.md states “sift itself must pass its internal release gates.” That claim covers two different gates, which this snapshot deliberately keeps separate:

1. Internal quality gate (SIFT_INTERNAL_GATE=1, i.e. make internal-gate) — sift’s own code-quality gate. Result: 13/13 PASS, 0 FAIL, 0 WARN. ✅ This is the gate ROADMAP.md and AGENT.md are talking about, and it is green.
2. Agent gate (sift . --agent-gate) — the product feature meant to screen arbitrary third-party repositories before an agent runs setup/build/install. There is no roadmap requirement that sift accepts its own repository under this gate, but running it is a useful dogfood check.

First run (start of this session): two real bugs found

VERDICT: CAUTION
SAFE_TO_AGENT_RUN: no
coverage: candidate_files=69 dehydrated_files=62 unsupported_files=7
          record_truncated=12 seed_bytes=148402

• Rule false positive: docs/ROADMAP.zh.md and other prose files were flagged dynamic-shell-eval (scope=docs, MEDIUM) purely because the English phrase “eval corpus” (sift’s own eval-corpus feature name) contains the substring "eval ", which looks_like_dynamic_shell_eval (src/report.rs) and looks_like_shell_command (src/extract.rs) both matched unconditionally. Not a real shell-eval risk.
• Unreviewed but legitimate artifacts: .githooks/pre-commit (an extensionless, real, committed executable) and two committed test fixtures (archive-payload/assets/payload.tar.gz, binary-extension/bin/tool.dylib) tripped the suspicious-artifact rule. There was no project-local sift-policy.toml (only sift-policy.example.toml), and even with one, policy [[allowlist]] matching only applied to RiskFindings, never to coverage.suspicious_artifacts — so these blockers had no suppression path at all.

Fixes landed this session

1. Added looks_like_eval_invocation (word-boundary + shell-substitution-token check) in both report.rs and extract.rs, so eval only flags a real invocation — the standalone word immediately followed by a command substitution, backticks, or a $variable — and never English/Chinese prose mentioning “eval corpus”/“retrieval”. Covered by flags_real_dynamic_shell_eval_invocation, ignores_eval_used_as_an_english_word, markdown_prose_mentioning_eval_corpus_is_not_a_command.
2. Extended the policy engine so [[allowlist]] also suppresses suspicious_artifacts blockers, matching rule against the artifact’s reason tag (apply_policy_to_artifacts, policy_match_artifact in report.rs; handles comma-joined multi-reason artifacts too). Covered by policy_allowlist_suppresses_matching_suspicious_artifact, policy_allowlisting_every_artifact_reaches_accept, policy_allowlist_matches_one_tag_within_a_combined_artifact_reason.
3. Added a real root sift-policy.toml (previously only sift-policy.example.toml existed) allowlisting .githooks/pre-commit and the tests/fixtures/repo-intake/ synthetic artifacts, each with a written reason. sift-policy.example.toml was extended with a documented example of the new artifact-allowlist form.

Second run (after fixes): blockers gone, verdict still (correctly) CAUTION

VERDICT: CAUTION
SAFE_TO_AGENT_RUN: no
coverage: candidate_files=72 dehydrated_files=64 unsupported_files=8
          record_truncated=12 seed_bytes=148542
BLOCKERS: none
POLICY:
- suppressed artifact extensionless_or_binary_executable at .githooks/pre-commit by allowlist (...)
- suppressed artifact binary_or_archive_extension at tests/fixtures/repo-intake/archive-payload/assets/payload.tar.gz by allowlist (...)
- suppressed artifact binary_or_archive_extension at tests/fixtures/repo-intake/binary-extension/bin/tool.dylib by allowlist (...)

(Note for future editors of this very section: describing these two rules’ trigger shapes in a literal, directly reproducible way can make this file itself trip them. Keep any such illustrative examples suitably paraphrased.)

Both root causes are fixed and verified: the eval false positive is gone (the one remaining dynamic-shell-eval finding is a real shell-invocation fixture — bash with an inline -c command interpolating a secret — exactly as intended), and all three unreviewed-artifact blockers are now suppressed with a written, reviewed reason.

The verdict nonetheless stays CAUTION, and this is now understood to be correct — not a defect to chase. All 40 remaining findings are Severity::Low, none Medium/High, and every one traces to one of two intentional, by-design sources:

• The 21 synthetic attack-pattern fixtures under tests/fixtures/repo-intake/ (the same corpus sift eval-corpus scores). They exist specifically to prove the supply-chain rule engine detects npm-lifecycle-script, download-execute, dependency-git-source, workflow-write-all, etc. If self-scanning made these disappear, the rules would be broken, not fixed.
• panic-edge (.expect()/.unwrap()) findings inside tests/*.rs. Hard Rule #1 forbids unwrap()/expect() only in src/; using them in tests is normal and correct, and PathScope::classify already caps these to Low — they still show up as findings (informational), they just cannot be silently hidden.

The agent gate’s verdict rule (render_agent_gate) only returns ACCEPT when findings is completely empty. Forcing that for sift’s own repository would require either deleting its own regression corpus or blanket-allowlisting every rule across tests/, both of which would remove the evidence this checklist’s P4a/P4b rows cite. The honest, durable dogfood claim is therefore: 0 High findings, 0 unexplained blockers, every Low finding accounted for — not a literal ACCEPT.

Consolidated open items

Everything not marked ✅ Done above, in one place. Two items from the previous snapshot were resolved this session and are omitted here (agent-gate self-CAUTION root causes fixed; ROADMAP P5 heading refreshed) — see Self-audit dogfood check for the former.

Refreshing this snapshot

cargo build
make ci                                   # fmt-check + test + clippy -D warnings + internal-gate
cat reports/internal-gate.md              # P5 gate detail (gitignored, local only)
cargo run --quiet -- . --agent-gate --format json   # live self-scan (dogfood check above)
sift eval-corpus                          # repo-intake precision table

This file reflects one commit in time. Re-run the commands above and update the Snapshot table, the phase tables, and the Consolidated open items whenever a phase’s evidence changes — do not hand-edit a status mark without re-checking its evidence.

AGENT.md - sift Contributor Handbook

English | 中文

The implementation handbook for humans and agents working on sift. Source of truth for hard rules, layout, and habits. Profile/boundaries live in Roadmap.

What sift is

A cost-controlled, single-binary open-source auditor: tree-sitter dehydration -> deterministic coarse ledger -> large-model convergence (Reduce), orchestrated by a ReACT state machine. Audits a whole project or one module. Small-model Map code is retained as inactive diagnostic scaffolding until behavior-level gates reintroduce it. sift must pass its internal release gates.

Hard Rules

1. No unwrap() / expect() in src/. Dirty data takes a Result/Option branch and is dropped+logged; the main process never panics.
2. Every external call has a hard timeout. Unbounded blocking is a bug. Repeated failure trips a breaker; on trip, back off, degrade, or emit partial output.
3. Single binary, low deps. No vector DB, embeddings/RAG, DB, or cache.
4. Compile-time skills only. Skills are an enum plus match to local functions.
5. Streaming, memory decoupled from scale. Bounded channel, drop the AST after dehydrating.
6. Fallback key resolution. CLI key file > ENV > project .env > ~/.sift/config.toml > default.
7. Secrets via env/file only. Never compiled in, committed, printed, or logged.
8. Module audit must not balloon to global. Cross-boundary refs are marked [EXTERNAL_BLACKBOX]; do not chase.
9. TDD. Each src/*.rs carries unit tests; build tests alongside new subsystems.
10. Bilingual docs, English default. Every user-facing doc has a Chinese counterpart, and commands/rules must match across languages.
11. No toy gates or fake capability claims. Scaffold code must be named as scaffold and isolated behind explicit modes.
12. Stable output contracts. --scan-only writes JSONL to stdout; full audit stdout is reserved for the final report.
13. No silent degradation. Truncation, skipped files, model fallback, partial reports, invalid config, and parse failures must be visible.
14. Program source is English-only. Runtime strings, prompts, and comments in src/ are English.

Code Map

Workflow

cargo build
cargo test
cargo fmt && cargo clippy
make ci
rg 'unwrap\(|expect\(|panic!' src
rg '[\p{Han}]' src

• One concern per commit.
• Before adding a feature, check it does not cross a roadmap non-goal.
• A phase is not done until its internal gate and at least one behavior-level smoke are green.
• Reports go to reports/, which is gitignored.

sift

English | 中文

可控成本的开源项目审计器：分级漏斗 + 算力错配 + ReACT 调度。引入开源库前，不必生吞数万行代码进前沿大模型，就能拿到定位到文件/行号的风险账本。

• 脏活（结构提取/确定性粗筛）→ tree-sitter + 本地规则
• 逻辑收敛 → 前沿大模型，ReACT 状态机基于确定性发现统一调度
• 单二进制、零配置、可审项目或模块；sift 自身必须通过内部发布门禁

详见 ROADMAP.zh.md。

用法

sift ./repo --scan-only        # 仅扫描层
sift ./repo --agent-gate       # 确定性预运行门禁，无需模型 Key
sift ./repo --agent-gate --format json
sift ./repo --benchmark        # 扫描/模型预算 telemetry JSON，无需模型 Key
sift github owner/repo         # 安全 GitHub intake，默认 --agent-gate
sift github owner/repo --ref main --scan-only
sift eval-corpus               # 运行内置 repo-intake 精度样本集
sift query ./repo --calls 'exec|spawn'          # 无状态证据检索 → file:line
sift query ./repo --imports reqwest --lang rust # 谁引入了 reqwest，仅看 rust 文件
sift query ./repo --any 'curl|wget' --format json
sift ./repo --module src        # 审子模块
SIFT_API_KEY=<KEY> sift ./repo  # 全链路
sift ./repo --api-key-file ~/.sift/key
sift ./repo --report-language zh # 输出中文 Markdown 报告
sift ./repo --save               # 同时保存报告到 reports/sift-audit-result-YYYYMMDD-NNN.md
sift ./repo --save-to out/audits # 保存报告到指定目录（隐含 --save）
sift ./repo --debug              # 向 stderr 打印更多诊断
sift doctor                    # 检查配置、key_env 与 endpoint/key 错配

--agent-gate 是给 agent 和包装脚本使用的本地确定性 repo-intake 门禁。它只向 stdout 写入以下稳定契约：

VERDICT: ACCEPT | CAUTION | REJECT | INCOMPLETE
WHY:
- <top evidence>
BLOCKERS:
- <file:line evidence or coverage blocker>
SAFE_TO_AGENT_RUN: yes | no

自动化集成可对 --agent-gate 使用 --format json。JSON 契约包含 verdict、safe_to_agent_run、exit_reason、coverage、findings、 blockers、artifact inventory、截断明细和 policy actions。

只有 SAFE_TO_AGENT_RUN: yes 时命令退出码为 0；CAUTION、REJECT 和 INCOMPLETE 都返回非零，方便调用方在 setup、install、build 或 run 之前停止。

sift query 是对 --scan-only 同一份脱水证据的无状态检索视图。每次 调用都重新执行本地扫描（秒级、无需 Key、无索引无缓存），并用平面 regex 旗标过滤证据：--calls、--imports、--signatures、 --external、--any，外加 --lang 与 --path 记录过滤。多个旗标在 文件级做 AND。文本输出是 grep 风格的 path:line: kind: text 证据； --format json 输出单个文档，包含 schema_version、回显的 query、 coverage、匹配计数和 matches。输出证据由 --limit（默认 200） 封顶且截断可见。退出码遵循 grep 惯例：0 有命中，1 无命中，2 用法或配置错误。

确定性供应链规则目前会标记 npm 安装生命周期脚本、manifest/lockfile 可复现性缺口、git/path/http 依赖来源、Rust build.rs 命令边界、 shell/Dockerfile 下载后执行模式、base64 解码后执行流、GitHub Actions 权限/触发器风险、secrets 与 shell 执行耦合、未 pin 到 commit SHA 的 GitHub Actions、Dockerfile root/远程仓库模式，以及可疑二进制/归档 artifact。

sift github 接受 owner/repo 或 https://github.com/owner/repo， 用 git 获取临时 checkout，解析 commit SHA，然后对该 checkout 运行本地 scan/gate/benchmark 管线。它不会运行仓库代码、包管理器命令、build script、hook、install 命令或 submodule。扫描前会检查 checkout 文件数/ 字节上限、.gitmodules 和 Git LFS 指示。临时 checkout 默认清理；只有 需要人工查看取回的源码树时才使用 --keep-checkout。

项目本地 policy 使用 sift-policy.toml。它支持 max_candidate_files、 [[allowlist]]、[[denylist]] 和 [[severity_override]]，可按 path、 rule、severity、reason 配置；命中的 policy 决策会出现在文本和 JSON 门禁输出中。

首次运行时，sift 会自动创建 ~/.sift/config.toml 默认配置文件。默认配置只包含非密钥项；模型密钥放在环境变量里，或通过 --api-key-file 传入。

完整审计的 stdout 只保留最终 Markdown 报告；进度、状态和 debug 诊断都走 stderr，长任务不会看起来像卡死，也不影响下游工具安全消费 stdout。

当前完整审计默认不会调用小模型 Map。它会把确定性账本交给配置的大模型收敛；小模型 Map 实现保留为实验性诊断路径。

--benchmark 是本地 telemetry 模式，用于 release note 和成本核算。 它不会调用模型；默认向 stdout 输出稳定 JSON，也可以用 --benchmark-output <path> 写入文件。报告包含候选/脱水/跳过计数、 扫描耗时、可用的 resident memory 指标、seed 字节数、计划 Reduce 批次、模型调用计数、近似 token 数，以及可选 USD 成本估算。价格必须 显式传入，不会自动猜测：

sift ./repo --benchmark \
  --benchmark-input-1m-cost 0.25 \
  --benchmark-output-1m-cost 1.00 \
  --benchmark-estimated-output-tokens 2000

支持语言

扫描层目前支持 Rust、Python、Go、JavaScript、TypeScript/TSX、HTML、CSS、Zig、Bash 兼容 shell 文件（.sh、.bash、.zsh）、Dart、Kotlin、Java、C/C++、C#、PHP、Swift、Ruby、SQL、Dockerfile/Containerfile、YAML、HCL/Terraform、Vue、Svelte、package.json、常见 package manifest/lockfile、Makefile 和 Markdown 安装片段。

安装

源码构建：

make ci
make install

安装本地 git hooks：

make githooks-install

pre-commit hook 会在每次提交前运行 make local-ci。确需临时跳过时，可执行 SIFT_SKIP_LOCAL_CI=1 git commit ...。

测试样本

tests/fixtures/repo-intake/ 包含合成的恶意与良性仓库树，用于 确定性 --agent-gate 回归测试。sift eval-corpus 会基于这些 fixture 输出 release 级别的精度表。这些 fixture 命令只是惰性样例，绝不能当作安装脚本执行。

macOS release 通过已有 tap 安装：

brew install jamiesun/tap/sift

状态：P0 脚手架 + P1 AST 脱水 + P2 模型层 + P3 ReACT 调度器（工具协议、编译期技能、retry→半成品）已完成。P4 进行中：本地 AST 风险账本、Markdown 渲染、[[model]] 配置解析、稳定 JSON 门禁、policy、artifact inventory 与 eval corpus 已接线。内部发布门禁会为维护者在 reports/ 下写入本地报告。

sift 项目画像与开发路线图

English | 中文

北极星 + 护栏 + 分阶段施工边界。说清“做成什么样 / 绝不做什么 / 每阶段交付什么 / 内部门禁何时生效“。 项目名：sift（CLI 即 sift）。语言：Rust。

项目概述

一个可控成本的开源项目审计器。引入开源库前，不必直接试用、也不必让前沿大模型生吞数万行代码，就能拿到一份定位到文件/行号的风险账本，据此决定是否引入。

核心是 分级漏斗 + 算力错配 + ReACT 调度：脏活（结构提取、粗筛压缩）当前交给零成本静态解析和确定性本地规则；高强度逻辑收敛交给前沿大模型；一个 ReACT 状态机基于确定性发现调度 Reduce。整个工具编译为单一二进制，零配置即可审整个项目或单个模块。sift 自身必须通过内部发布门禁。

• 架构图

  CLI key file / ENV / ~/.sift/config.toml ──(降级寻址, 缺 Key 即退)
        ▼
  扫描层  ignore::Walk → 有界 channel(消费即丢)         [P0 ✓]
        ▼
  零阶    tree-sitter 脱水(签名/import/调用) → JSON → drop AST  [P1 ✓]
        │  跨界引用打 [EXTERNAL_BLACKBOX]
        ▼
  模型层  多模型注册表 · 每调用硬超时 · 熔断+退避恢复    [P2 ✓]
        ▼
  ReACT 调度器(工具协议, 确定性发现, retry≤N)             [P3 ✓]
        │  └─ 大模型(Reduce 收敛) ─────┘
        ▼
  报表层  stdout Markdown 风险清单(行号/调用链)           [P4 已启动]
        ▼
  内部门禁  源码评分检查 + 发布证据                         [P5/P6]

项目画像（目标状态）

• 零摩擦冷启动。 sift ./repo --scan-only 直接跑；缺 ~/.sift/config.toml 时自动创建不含密钥的默认配置；不交互追问；缺 Key 立退给注入提示。
• 成本可控可预算。 确定性 baseline 本地完成；完整审计才把脱水骨架交给大模型。
• 模型调度。 ReACT 状态机把确定性发现与大模型收敛编排成一条链，技能是编译期写死的本地函数。
• 多模型 + 并发提速。 可配置多个模型端点；扫描/模型并发保持有界且可观测。
• 绝不无脑死磕。 每个外部调用有硬超时；连续失败触发熔断；熔断后退避恢复或降级，到顶则输出半成品而非挂死。
• 默认工程级。 一份看起来完整但实际不完整的审计报告就是缺陷。跳过输入、截断、回退、半成品模型结果、无效配置都必须可见且可测试。
• 稳定机器契约。 扫描 JSONL、最终 Markdown、诊断信息和生成报告各走清晰通道。下游脚本消费 stdout 时不应该猜里面是否混了多种格式。
• 内存与规模脱钩。 流式处理、处理完即丢，常驻内存压低位。
• 内部门禁约束。 项目必须通过维护者专用发布门禁；代码模块化、TDD 守护、边界清晰。
• 品质冲突优先级： 鲁棒不崩 > 报表可用 > 成本低 > 速度快 > 体积小。

非目标（铁律）

• 不做向量库 / Embedding / RAG。 单次低频审计，索引成本大于直接拼 prompt，纯文本管道阅后即焚。
• 不做运行时插件 / 动态技能注册。 技能 = 编译期 enum + match 本地函数；扩展靠改码重编译。
• 不做服务化 / Web UI / 多租户。 一次性 CLI，无常驻、无界面。
• 不允许 panic 主进程。 脏数据静默丢弃记日志；幻觉/坏 JSON 走熔断；全程 Result/Option，无 unwrap/expect。
• 不允许无超时阻塞。 任何子进程/网络/模型调用必须有 deadline，无界等待视为 bug。
• 模块审计不膨胀成全局。 跨界引用打断点交大模型脑补，不盲目追链。
• 不靠“直接试用“替代审计。 价值在引入前判断。
• 脚手架不得冒充产品能力。 占位实现只能存在于明确未完成的阶段内；不能产出看起来像生产完成的报告。
• 不允许静默回退。 无效配置、seed 截断、跳过文件、缺模型角色、模型路径降级，必须明确失败或写入报告。

模块化结构（Code Map）

每个 src/*.rs 自带单测；新子系统建则单测同建（TDD）。模块边界即责任边界，禁跨层乱伸手。

src/main.rs       入口装配：解析→Config→调度→报表→退出码
src/config.rs     降级寻址、多模型配置加载            [P0✓→P2扩]
src/scanner.rs    Walk + 有界 channel                  [P0✓]
src/extract.rs    tree-sitter 脱水 → AstSummary        [P1]
src/query.rs      无状态证据检索(重扫+regex过滤)        [P1✓]
src/model.rs      多模型注册表/客户端 trait/超时熔断    [P2✓]
src/react.rs      ReACT 状态机 + 技能 enum/match        [P3 ✓]
src/skills.rs     本地技能函数(粗筛/reduce收敛)         [P3 ✓→P4]
src/report.rs     Markdown 风险清单渲染                 [P4]
src/audit.rs      内部门禁维度评分(借鉴 scoot, 裁剪)    [P5]

多模型与并发（config schema）

concurrency = 8          # 扫描/模型并发上限
[[model]]                # 可多条；role 决定用途
role = "small"           # 保留给实验性 Map 诊断
endpoint = "..."
key_env = "SIFT_SMALL_KEY"
timeout_ms = 8000
max_retries = 1
[[model]]
role = "large"
endpoint = "..."
key_env = "SIFT_API_KEY"
timeout_ms = 60000
max_retries = 1

寻址降级：CLI key file > ENV > toml > 默认；无 large key 即退。当前完整审计默认不调用 small role 模型；小模型缺失不会改变确定性账本 Reduce 路径。 默认用户配置路径为 ~/.sift/config.toml；首次运行从等价于 config.example.toml 的安全默认值创建，不能写入明文密钥。

超时熔断与恢复（绝不死磕）

• 每调用 deadline：超时即弃，不无界等待。
• 熔断计数器：单链连续失败/坏 JSON/未注册技能达阈值 → break，停 I/O。
• 退避恢复：瞬时错指数退避重试到预算；非瞬时错降级（小模型回退 AST、大模型回退半成品）。
• 预算上限：全局 token/时长封顶，触顶强制收敛输出 [TRUNCATED] 报表。

工程契约

• 标记完成的阶段必须有行为级证据，不能只有类型接线或 happy-path 单测。
• 完整审计 stdout 是最终报告流；--scan-only 是 JSONL 流；诊断信息不得进入 stdout。
• 报告必须披露输入覆盖：扫描、脱水、送入模型、跳过、截断的规模。
• 配置文件属于信任边界。用户配置缺失时从安全默认值自动创建；配置文件存在但无效时，进程必须失败，不能退回默认值。
• src/ 下程序源码的运行时文本、prompt 和注释只用英文；双语文档保留在 docs。

阶段路线图

每阶段含：功能清单 / 边界约束 / 内部门禁。门禁全绿才进下阶段，并据门禁证据定下一步。 下方每一项对照真实证据的完成/部分完成/待定快照，见 CHECKLIST.zh.md。

P0 脚手架 — 已完成 ✓

• 功能：clap 降级寻址、有界通道扫描、缺 Key 熔断退出、最小装配。
• 边界：不连网、不解析、不留内存树。
• 门禁：cargo build 绿 / 0 unwrap / --scan-only 能扫 / 缺 Key exit1。

P1 零阶 AST 脱水 — 已完成 ✓

• 功能：tree-sitter 接 Rust/Python/Go/JavaScript/TypeScript/HTML/CSS/Zig/Bash/Dart/Kotlin/Java/C/C++/C#/PHP/Swift/Ruby/SQL/Dockerfile/YAML/HCL/Vue/Svelte，提签名/import/调用，输出扁平 AstSummary JSON；跨界打 [EXTERNAL_BLACKBOX]；解析即 drop。
• 边界：丢注释与代码体；遇到残缺语法不 panic，并在下游报告披露覆盖不完整；不评价质量。
• 门禁：百兆库内存稳定低位、坏文件不崩；extract.rs 单测覆盖典型/残缺样本。

P2 模型层（多模型+超时熔断） — 已完成 ✓

• 功能：ModelClient trait、注册表、role 路由；每调用硬超时、熔断、退避恢复；可配多端点。
• 边界：不写缓存、不持久化；密钥仅 env/文件、不入日志。
• 门禁：超时/坏响应有测试模拟，熔断必触发不死磕；无明文密钥。粗筛/收敛接线留 P3。

P3 ReACT 调度器 — 已完成 ✓

• 功能：enum 状态机，初始工具协议提示，大模型出 <TOOL_CALL>，经 $SEED match 路由本地技能；retry≤N 熔断半成品。
• 边界：技能编译期写死；无动态加载。
• 门禁：注入坏 JSON/未注册技能/连错 N 次能熔断；react.rs 单测覆盖。

P4 确定性 Reduce+报表

• 功能：确定性 AST 粗筛账本、Markdown 渲染、真实 [[model]] TOML 解析、显式输入覆盖率、稳定 JSON 门禁、policy、artifact inventory、eval corpus 与干净 stdout 边界。
• 边界：模块审计只切根；跨界不追；截断和模型降级路径必须可见。
• 门禁：审已知样本命中预埋风险；模块/项目模式不串；完整审计 stdout 只含报告；无效配置失败；fake-endpoint full audit smoke 证明用户路径可用。

P5 内部质量门禁 — 已完成 ✓

• 功能：audit.rs 跑裁剪维度评分，并把维护者专用报告写入 reports/(gitignore)。
• 门禁：硬规则无 FAIL/WARN，包括无 broad dead-code allow、无中文源码字符串/注释、报告流边界干净、seed 截断可见。

P6 发布加固

• 功能：ReleaseSafe 单二进制、Makefile 安装路径、macOS Homebrew tap 发布、多语法扩展、JSON 输出稳定。
• 门禁：单文件分发、内部门禁通过、文档↔功能一致，brew install jamiesun/tap/sift 由 release checksum 支撑。

完成的样子

• 空配置可跑，自动创建 ~/.sift/config.toml，缺 Key 即退给提示；不挂起。
• 百兆库内存稳定、扫坏不崩。
• 报表定位行号、含跨模块依赖与并发/资源风险，可直接拍板。
• 报表声明输入覆盖和截断状态；覆盖不完整时绝不能看起来像完整结论。
• 任一外部调用必超时；连错熔断出半成品而非死磕。
• 同一二进制审项目与 --module 子目录不串。
• 内部发布门禁无 FAIL，硬规则无 WARN。

建议非铁律：rayon/具体超时阈值/体积耗时数字按基准定，别当验收红线锁死。已确立铁律：单二进制、降级寻址、有界通道、硬超时熔断、无 unwrap、TDD、内部门禁达标。

sift 验收清单

English | 中文

针对 ROADMAP.md 承诺的每一项功能与门禁的时间点验收快照，严格限定在其阶段边界（P0–P6）与非目标范围内；边界之外的内容本清单不予评分——铁律见 AGENT.md。 行号以下方快照提交为准，后续可能漂移；行号与函数/测试名冲突时，以函数/测试名为准。

快照信息

图例

对应本清单要回答的三分类：完成 = ✅，待定 = ⏳，未完成 = 🟡 / ⬜。

Type 列：F = 功能条目，G = 门禁/验收标准条目，B = 边界约束，均逐字取自各阶段 ROADMAP.md 原文。

P0 — 脚手架

ROADMAP 状态：已完成 ✓

阶段结论：✅ 完成，仅 #7 存在测试覆盖缺口。

P1 — 零阶 AST 脱水

ROADMAP 状态：已完成 ✓

阶段结论：🟡 基本完成。 唯一未验证的门禁是百兆内存稳定性声明；且 macOS（一个受支持的 CI/发布目标）目前完全没有可用的常驻内存指标。

P2 — 模型层（多模型 + 熔断）

ROADMAP 状态：已完成 ✓

阶段结论：✅ 完成。

P3 — ReACT 调度器

ROADMAP 状态：已完成 ✓

阶段结论：✅ 完成。

P4 — 确定性 Reduce + 报表

ROADMAP 状态：标题未标 ✓；README 自述「进行中」。这是功能增长最多的阶段，下面拆成三组呈现。

P4a — 确定性账本与 Markdown 报告

P4b — Agent gate 与 policy

P4c — 运行模式

阶段结论：🟡 基本完成——与项目自述的「P4 进行中」一致。 真正悬而未决的工程问题是 P4a 的 #10（没有 CI 自动化的完整审计 smoke）和 P4c 的 #3（doctor 无测试）；小模型 Map 的去留（P4c #7）是一个明确的待决策问题，不是 bug。

P5 — 内部质量门禁

ROADMAP 状态：标题已在本会话中补上 ✓，功能与门禁本身早已完整落地且全绿。

阶段结论：✅ 完成。 ROADMAP.md/ROADMAP.zh.md 的 P5 标题已在本会话中改为「— done ✓ / 已完成 ✓」以匹配现状，剩余建议是把 BT 检查收紧到函数级粒度。

P6 — 发布加固

ROADMAP 状态：标题未标 ✓，但已有相当充分的证据。

阶段结论：🟡 基本完成。 两条悬而未决的线：文档↔代码一致性没有自动化守卫；「更多语法」是有意保持无上限的目标，而不是一个可以关闭的门禁。

横切检查：工程契约（ROADMAP.md）

横切检查：完成的样子（ROADMAP.md）

非目标护栏

确认 ROADMAP.md 里「绝不做」的铁律没有被越界。

自我审计 dogfood 检查

AGENT.md 写道「sift 必须通过内部发布门禁」。这句话其实涉及两个不同的门禁，本清单刻意把它们分开：

1. 内部质量门禁（SIFT_INTERNAL_GATE=1，即 make internal-gate）——sift 自身的代码质量门禁。结果：13/13 PASS，0 FAIL，0 WARN。 ✅ 这正是 ROADMAP.md 和 AGENT.md 所说的门禁，且是全绿的。
2. Agent gate（sift . --agent-gate）——用来在 agent 执行 setup/build/install 之前，筛查任意第三方仓库的产品功能。ROADMAP 并没有要求 sift 用这个门禁审自己的仓库必须得到 ACCEPT，但拿它做一次 dogfood 检查很有意义。

第一次跑分（本会话开始时）：发现两个真实 bug

VERDICT: CAUTION
SAFE_TO_AGENT_RUN: no
coverage: candidate_files=69 dehydrated_files=62 unsupported_files=7
          record_truncated=12 seed_bytes=148402

• 规则误报： docs/ROADMAP.zh.md 等行文文件被标成 dynamic-shell-eval（scope=docs，MEDIUM），原因纯粹是英文短语 “eval corpus”（正是 sift 自己的 eval-corpus 功能名）包含子串 "eval "，而 looks_like_dynamic_shell_eval（src/report.rs）和 looks_like_shell_command（src/extract.rs）对这个子串都是无条件匹配。这不是真正的 shell-eval 风险。
• 未加白但合法的 artifact： .githooks/pre-commit（一个没有扩展名、真实存在且已提交的可执行文件）和两个已提交的测试样本（archive-payload/assets/payload.tar.gz、binary-extension/bin/tool.dylib）触发了可疑 artifact 规则。项目根目录没有真正的 sift-policy.toml（只有 sift-policy.example.toml），而即使有，policy 的 [[allowlist]] 匹配也只适用于 RiskFinding，从未覆盖过 coverage.suspicious_artifacts——所以这些 blocker 根本无法被压制。

本会话落地的修复

1. 在 report.rs 和 extract.rs 中都新增了 looks_like_eval_invocation（单词边界 + shell 替换词 token 检查），让 eval 只在这个独立单词后面紧跟 command substitution、反引号或 $变量 时才命中，永远不会误读提到 “eval corpus”/“retrieval” 的英中文行文。测试：flags_real_dynamic_shell_eval_invocation、ignores_eval_used_as_an_english_word、markdown_prose_mentioning_eval_corpus_is_not_a_command。
2. 把 policy 引擎扩展为 [[allowlist]] 也能压制 suspicious_artifacts blocker，用 rule 匹配 artifact 的 reason 标签（apply_policy_to_artifacts、policy_match_artifact，均在 report.rs；也处理了逗号拼接的多 reason 情况）。测试：policy_allowlist_suppresses_matching_suspicious_artifact、policy_allowlisting_every_artifact_reaches_accept、policy_allowlist_matches_one_tag_within_a_combined_artifact_reason。
3. 新增了一个真正的根目录 sift-policy.toml（之前只有 sift-policy.example.toml），为 .githooks/pre-commit 和 tests/fixtures/repo-intake/ 下的合成 artifact 加白，每条都写了理由。sift-policy.example.toml 也同步补充了 artifact 加白写法的文档示例。

第二次跑分（修复后）：blocker 消失，但 verdict 仍然（正确地）为 CAUTION

VERDICT: CAUTION
SAFE_TO_AGENT_RUN: no
coverage: candidate_files=72 dehydrated_files=64 unsupported_files=8
          record_truncated=12 seed_bytes=148542
BLOCKERS: none
POLICY:
- suppressed artifact extensionless_or_binary_executable at .githooks/pre-commit by allowlist (...)
- suppressed artifact binary_or_archive_extension at tests/fixtures/repo-intake/archive-payload/assets/payload.tar.gz by allowlist (...)
- suppressed artifact binary_or_archive_extension at tests/fixtures/repo-intake/binary-extension/bin/tool.dylib by allowlist (...)

（给未来编辑本节的人一个提醒：如果把这两条规则的触发形状写得过于具体、可直接复现，就会让这个文件自己触发它们。请让任何这类示例都保持适度改写。）

两个根因都已修复并验证：eval 误报消失了（唯一剩下的一条 dynamic-shell-eval 是一个真实的 shell 调用样本——bash 内联 -c 命令并插值了一个 secret，正是预期中应该被标出的），且三个未加白 artifact blocker 现在都已被写有理由的白名单压制。

但 verdict 仍然是 CAUTION，现在已确认这是预期中的正确结果——不是需要追逐消除的缺陷。 剩余的 40 条发现全部是 Severity::Low，没有 Medium/High，且每一条都能追溯到下面两个有意设计的来源之一：

• tests/fixtures/repo-intake/ 下 21 个合成攻击模式样本（与 sift eval-corpus 评分用的是同一份语料）。它们的存在就是为了证明供应链规则引擎能检测到 npm-lifecycle-script、download-execute、dependency-git-source、workflow-write-all 等。如果自扫让这些发现消失，那说明规则坏了，而不是修好了。
• tests/*.rs 里的 panic-edge（.expect()/.unwrap()）发现。铁律 #1 只禁止在 src/ 里用 unwrap()/expect()，在测试里用它们完全正常且正确，PathScope::classify 也已经把这些封顶到 Low——它们依旧会以发现形式出现（信息性的），只是不能被静静藏起来。

Agent gate 的 verdict 规则（render_agent_gate）只有在 findings 完全为空时才返回 ACCEPT。强行让 sift 自己的仓库做到这一点，只能靠删除自己的回归语料，或者对 tests/ 下所有规则一概加白，这两种做法都会抹掉本清单 P4a/P4b 行引用的证据。因此诚实、经得起推敲的 dogfood 结论应该是：0 条 High 发现、0 条无法解释的 blocker、每一条 Low 发现都有归属——而不是字面上的 ACCEPT。

汇总：待办事项

把上文所有非 ✅ 完成的条目汇总在一处。上一份快照中的两项已在本会话中解决，此处不再列入（agent gate 自审 CAUTION 的根因已修复；ROADMAP P5 标题已刷新）——前者详见自我审计 dogfood 检查。

如何刷新本快照

cargo build
make ci                                   # fmt-check + test + clippy -D warnings + internal-gate
cat reports/internal-gate.md              # P5 门禁细节（已 gitignore，仅本地）
cargo run --quiet -- . --agent-gate --format json   # 现跑一次自我扫描（对应上文 dogfood 检查）
sift eval-corpus                          # repo-intake 精度表

本文件反映的是某一个提交时间点的状态。每当某个阶段的证据发生变化，请重新执行上面的命令， 并更新「快照信息」表、各阶段表格与「汇总：待办事项」——不要在没有重新核对证据的情况下手改状态标记。

AGENT.md — sift 贡献者手册

English | 中文

给参与 sift 的人与 agent 的实现手册：铁律、结构、习惯的事实来源。画像/边界见 ROADMAP.zh.md。

sift 是什么

可控成本的单二进制开源审计器：tree-sitter 脱水 → 确定性粗筛账本 → 大模型收敛(Reduce)，由 ReACT 状态机调度。审项目或单模块。小模型 Map 代码暂作为未激活的诊断脚手架保留，直到有行为级门禁再重新接入。sift 必须通过内部发布门禁。

铁律

1. src/ 内禁 unwrap()/expect()。 脏数据走 Result/Option 分支丢弃+记日志；主进程绝不 panic。
2. 每个外部调用（子进程/网络/模型）必有硬超时。 无界阻塞即 bug；连错触发熔断；熔断后退避/降级/出半成品，绝不死磕。
3. 单二进制、低依赖。 无向量库、无 embedding/RAG、无数据库、无缓存；纯文本管道，阅后即焚。
4. 技能仅编译期写死。 技能 = enum + match 本地函数；无动态加载、无运行时插件。
5. 流式、内存与规模脱钩。 有界通道，脱水后即 drop AST，常驻内存压低位。
6. 密钥降级寻址。 CLI key file > ENV > 项目 .env > ~/.sift/config.toml > 默认；缺大模型 Key 立退给提示，绝不挂起或交互追问。缺用户配置时自动创建不含密钥的默认配置。
7. 密钥仅 env/文件。 不编译进、不提交、不打印、不入日志。
8. 模块审计不膨胀成全局。 跨界引用打 [EXTERNAL_BLACKBOX]，不追链。
9. TDD。 每个 src/*.rs 自带单测；新子系统单测同建。
10. 中英双语、默认英文。 每文档有 ZH 副本(docs/*.zh.md)；英文为准，跨语言范围/命令/规则须一致。
11. 禁止玩具门禁或虚假能力声明。 脚手架代码必须明确标成 scaffold，并隔离在显式模式后面；只有行为级门禁证明后，才能算阶段完成。
12. 输出契约稳定。 --scan-only 可以向 stdout 写 JSONL；sift query 向 stdout 写 grep 风格证据行或单个 JSON 文档；完整审计的 stdout 只留给最终报告。进度、诊断、模型遥测走 stderr 或 reports，不能混进报告流。
13. 禁止静默降级。 截断、跳过文件、模型回退、半成品报告、无效配置和解析失败，必须体现在输出、退出码或内部门禁证据里。无效配置文件必须失败，不能悄悄回默认值。
14. 程序源码只用英文。 src/ 内运行时字符串、prompt 和源码注释使用英文；双语用户文档保留在 docs/*.zh.md。

任一铁律违反即内部门禁 FAIL。

模块地图

工作流

cargo build                    # 必须绿
cargo test                     # 必须过
cargo fmt && cargo clippy      # 提交前清
make ci                        # 对齐本地发布门禁
rg 'unwrap\(|expect\(|panic!' src  # 必须为 0
rg '[\p{Han}]' src             # 必须为 0

• 一次提交一个关注点；带 Co-authored-by: Copilot trailer。
• 加功能前查是否越 ROADMAP 非目标；越界先改铁律。
• 阶段内部门禁和至少一个用户路径行为 smoke 不绿，不算完成。
• 如果阶段使用脚手架，文档和代码必须明确写出仍未完成的部分。

习惯

• 全程 Result/Option；每个等待都有界；临时数据尽早 drop。
• 模块各守责任，不跨层乱伸手。
• 优先生态 crate，但拒重依赖。
• 报表入 reports/(gitignore)；审计不脏化跟踪文件。
• 宁可明确失败，也不要输出一份看起来完整但实际不完整的审计报告。