How it works

What We Run During The Audit

Every step maps to a repo operation or a ContractForge command. The default path uses a clean local clone or worktree.

Audit Flow

Collect the PR pattern

Name the repo, coding agent, repeated review question, expected commands, and any files reviewers consider sensitive.

Prepare a clean workspace

Use a disposable clone or worktree, install ContractForge, and keep generated audit files separate from product code until reviewed.

Inventory current instructions

Inspect AGENTS.md, CLAUDE.md, Cursor rules, README guidance, CI files, package metadata, and review habits.

Generate the first rules file

Run contractforge init, then edit agent.contract.yaml for file boundaries, approvals, commands, and failure recovery.

Score the current repo instructions

Run contractforge audit and review the missing clauses, supporting context, and suggested task categories.

Preview reviewer guidance

Run contractforge compile --dry-run and inspect the proposed AGENTS.md diff before writing any file.

Generate review tasks and preview the run

Run contractforge eval-gen --count 10 and contractforge eval --plan; review planned commands, dirty-worktree warnings, and approval-required prompt blocks.

Run checks and package the report

Run contractforge eval and contractforge report; assemble traces, policy notes, CI starter, generated files, and a reviewed memo.

What The Tool Actually Blocks

In ContractForge today, approval has two concrete meanings: generated repo guidance can require recorded owner approval, and the CLI can stop known risky prompts before it runs an external agent.

Generated repo files

agent.contract.yaml and compiled AGENTS.md can name changes that require recorded owner approval: dependencies, CI, package metadata, migrations, auth, billing, release files, or repo-specific risky internals.

CLI preflight

contractforge eval --agent-command checks the task prompt before invoking the agent. Known risky prompt categories are blocked unless the run includes --approve-gated, --approved-by, --approval-reason, and --approval-scope.

Hard-blocked paths

Forbidden-path matches such as .env*, secrets/**, private keys, or configured forbidden files are hard-blocked. Approval metadata does not override those matches.

Boundary

Repo-specific concerns such as benchmark sensitivity or hashing semantics are written into reviewer guidance and final-response requirements unless they map to known preflight patterns or configured forbidden paths. That is not complete policy enforcement.

How The Example Cards Map To The Tool

Each example below is shown as a process trace: local input, command, persisted file, and review use. The public-repo pass was not a customer engagement, paid pilot, or measured outcome study.

Across 20 public repos

Local process: each public repo was inspected from a local clone with contractforge init --force, contractforge audit, contractforge compile --dry-run, contractforge eval-gen --count 10, and contractforge report.

Files written: agent.contract.yaml, .contractforge/audit.md, .contractforge/audit.json, .contractforge/eval_tasks.yaml, and .contractforge/report.md. The compile --dry-run step previews AGENTS.md without adopting it.

Review use: the files are inspected by a human reviewer. The coding agent sees the guidance only if the team adopts the generated AGENTS.md or passes the task through an agent command.

Boundary: no dependency install, test suite, model call, hosted upload, or paid rollout claim was part of this pass.

node-glob

Local input: contractforge audit records instruction sources such as AGENTS.md, CONTRIBUTING.md, and README.md. Founder review then decides which repo-specific expectations belong in the contract.

Persisted output: .contractforge/audit.md records the instruction gap. If adopted, contractforge compile turns the contract rule into AGENTS.md guidance.

AGENTS.md draft:
Final evidence must include:
- files changed
- commands run and pass/fail results
- snapshot, benchmark, script, or package notes
- skipped validation and why

Review use: the reviewer can require that evidence in the agent's final response. If the task prompt asks for scripts or package metadata changes, preflight can block the external agent command unless approval metadata is supplied.

lambda-api

Local input: contractforge init drafts repo signals and starter paths. The audit reviews AGENTS.md, .github/copilot-instructions.md, and README.md before narrowing the workflow scope.

Persisted output: agent.contract.yaml stores allowed paths, forbidden paths, validation commands, and final-response requirements for the selected API workflow.

agent.contract.yaml:
allowed_paths:
  - lib/**
  - __tests__/**
  - index.js
  - index.d.ts
final_response:
  include skipped checks and risks

Review use: contractforge eval uses those paths for diff-scope checks. Forbidden secret-like paths are blocked before an external agent command is invoked.

jug

Local input: contractforge audit records current instruction surfaces. Founder review identifies where task execution, hashing, locking, and CLI changes need clearer stop rules.

Persisted output: the recovery rule is stored in agent.contract.yaml, previewed by contractforge compile --dry-run, and included in the reviewed memo.

AGENTS.md draft:
Validation recovery:
- run the narrowest pytest command
- make one focused repair after failure
- stop after two failed repair attempts
- report unresolved risks

Review use: the agent-facing rule is available only after the team adopts the compiled guidance. contractforge eval and contractforge report persist command results and trace output for reviewer inspection.

Sample Audit Slice

Fictional example. It shows the kinds of files, commands, and notes the audit returns.

Input from the team

Repo: payments-api

Agent task: draft billing retry changes.

Repeated question: why did a billing PR touch CI and dependency files?

Instruction inventory

README.md names the test suite. AGENTS.md omits billing commands. CI files changed recently. Dependency approval appears only in old PR comments.

Rules file excerpt

workflow: billing-retry-change
allowed_paths:
  - src/billing/**
  - tests/billing/**
approval_gates:
  dependencies:
    required: true
  ci:
    required: true
commands:
  test: python -m pytest tests/billing -q

Prompt block note

A prompt that asks the agent to change billing code and "adjust CI if needed" is flagged before the agent command runs. The memo names the approval information needed to proceed.

Command evidence

Ran: contractforge eval --plan

Ran: contractforge eval

Captured: command status, file-change notes, blocked-prompt notes, and trace files.

Reviewer handoff

The reviewer gets a written rule for billing paths, dependency approval, CI edits, and expected tests. The memo separates local audit evidence from claims that still require separate measurement.

How To Read The Output

Passing checks mean the listed commands ran for the chosen PR pattern. Future agent work still needs normal review.
Prompt blocks cover known approval-required patterns before an agent command runs. Complete enforcement requires separate work.
Generated tasks, CI files, and AGENTS.md drafts are starting points for team review.
The memo names what the files show, what they leave unresolved, and which claims require more evidence.

Command Sequence

contractforge init
contractforge audit
contractforge compile --dry-run
contractforge eval-gen --count 10
contractforge eval --plan
contractforge eval
contractforge report