Correctness

The correctness gate. A sandbox that runs commands faster but produces different results than running them normally is worse than useless — it silently breaks builds. Correctness beats speed, always; this is the largest test surface in the package.

The bar: behaviourally identical to native

For any command, the sandbox must produce the same observable result as running it natively: same exit code, same stdout/stderr (modulo explicitly-normalized nondeterminism), same files produced (content-identical), same dependency resolution (lockfile honored). The vfs backend's no-native-subprocess limit is a capability boundary, not a correctness excuse — within its scope it must match native exactly; outside it, it must fail loudly (fall back), never silently differ.

Test layers

flowchart LR
    subgraph gate["CI coverage shards (hard fail)"]
        unit["1 · unit\nFS provider, backend wiring"]
        int["2 · integration/acceptance\nreal pnpm install + native postinstall in RAM"]
        diff["3 · differential\nsame command: sandbox vs native, normalize, assert identical"]
        equiv["4 · equivalence\nwarm fork ≡ cold install · persist ≡ native disk · replay ≡ re-run"]
        prop["5 · property/fuzz\nfast-check vs node:fs oracle · randomized commands vs isolation invariants"]
    end
    corpus["differential corpus\n(grows with every gap a real repo exposes)"] --> diff

1. Unit — FS provider (read/write/overlay/symlink/module-load), exec backend wiring, snapshot addressing. Fast, deterministic, run everywhere; lives beside the code (*.test.ts).
2. Integration/acceptance — a real pnpm install with a native postinstall (sharp, esbuild) completing fully in RAM — the os backend's reason to exist (*.acceptance.test.ts).
3. Differential (golden) — the core technique: run the same command through the candidate backend and the native baseline, normalize both, assert identical. Shared infrastructure under services/exec/differential/; each backend's *.differential.test.ts feeds its corpus to one helper. Nothing is normalized implicitly — each case carries explicit { pattern, placeholder } rules, so a real divergence is never hidden. Every reported correctness bug becomes a permanent golden case before it's fixed.
4. Equivalence — forkSnapshot.equivalence.test.ts proves a forked warm run is observably identical to a cold in-place install; persistRun.equivalence.test.ts proves a persist run leaves the host disk exactly as native would (write-back); taskCache.equivalence.test.ts proves a replay matches a real re-run.
5. Property/fuzz — two halves. The vfs seam: fast-check drives randomized read/write/exists sequences against the provider and a real node:fs temp dir in lockstep and diffs the full trace (node:fs is the oracle, so no fs semantics are re-implemented; failures shrink to a minimal counterexample). The os half: randomized command sequences through the ephemeral RAM overlay assert the isolation invariants under every ordering — the host disk is never mutated, no write leaks across fresh-per-exec uppers, every command yields a well-formed result. Host-gated, small run counts (each op is a real subprocess).

Key files

Paths relative to packages/virrun/src/.

Notes

• The full matrix to keep covering as the corpus grows: package managers × {with, without native deps} × backends × cache states (cold, warm store, warm snapshot) × hosts (Linux native, WSL2 bridge). A pass on one cell is not a pass on the matrix.
• The differential suite is plain Vitest, so it hard-fails the CI coverage shards; speed is tracked separately by the committed bench artifacts (benchmarking) — a hard wall-clock CI gate is rejected as runner-noise-flaky (CI wall-time gate).