Benchmarks

Performance work is measured on the factory VM and in CI against the system git binary. Micro-benchmarks live in grit-lib (Criterion) and command-level scenarios in grit-bench (grit-utils).

Command-level numbers below come from the committed grit-bench JSON baselines listed in content/docs/site.toml. Each scenario runs hyperfine against the system git binary and a release grit build on the same machine; times are wall-clock milliseconds (mean and spread from timed runs).

grit-lib: parallel object hashing

Criterion group hash/batch_parallel (cargo bench -p grit-lib --bench objects hash/batch_parallel) hashes 20 000 Git blob objects of 16 KiB each using hash_objects_parallel (SHA-1, release build).

Speedup (8 vs 1 thread): ~3.9× on the factory VM (2026-10-07).

Parallel hashing falls back to a serial loop when there are fewer than PAR_HASH_MIN_ITEMS (32) objects or less than PAR_HASH_MIN_TOTAL_BYTES (256 KiB) of payload, so small batches avoid thread overhead.

grit-lib: ODB backend (pluggable object store)

Criterion group odb_backend compares post-refactor Odb against the saved baseline odb-before (cargo bench -p grit-lib --bench odb_backend -- --baseline odb-before). Built-in layouts route hot reads through FilesSource on the primary store and CompositeStore for alternates so behaviour matches the pre-refactor oracle tests while keeping a pluggable ObjectStore boundary.

Pluggable object store overhead (factory VM, 2026-10-09): Re-run Criterion with --baseline odb-before after ODB changes; command-level grit-bench odb-backend baselines are in odb-backend-before.json (pre-refactor capture, scenario ids suffixed -pre-refactor) and odb-backend-after.json. Post-refactor Grit medians for cat-file batch/check are much lower than the pre-refactor capture on the same fixture; rev-list wall time is similar while Grit/Git ratios can drift when system git speeds up between runs.

Criterion measures Odb::read, read_info, exists (hit and miss), and write on four deterministic layouts built in grit-lib/benches/fixture.rs:

An additional benchmark, odb_backend/exists_miss_loop_10k, calls exists on 10 000 distinct missing OIDs against the 100k pack fixture (models add/status miss scans).

Command-level ODB scenarios on the repacked 100k-file / 1000-commit synthetic repo live in grit-utils/baselines/odb-backend-before.json and grit-utils/baselines/odb-backend-after.json (grit-bench odb-backend): library drivers (grit-bench drive …) vs git cat-file --batch, git cat-file --batch-check, and git rev-list --objects --all on the same sorted OID stdin list where applicable.

$ cargo build --release -p grit-utils
$ ./target/release/grit-bench odb-backend --format json --output grit-utils/baselines/odb-backend-after.json
$ cargo bench -p grit-lib --bench odb_backend -- --baseline odb-before
$ make docs

Run grit-bench odb-backend once before the Criterion command (or leave a populated GRIT_BENCH_ODB_CACHE) so the pack_100k fixture reuses the repacked 100k repo instead of rebuilding 100k commits inline.

Recorded on the factory VM (2026-10-09, repacked 100k synthetic repo). Pre-refactor numbers are in odb-backend-before.json; post-refactor in odb-backend-after.json (also drives the generated ODB rows below).

Rev-list ratios can rise when system git speeds up between captures even if Grit wall time improves; compare absolute Grit medians when judging regressions.

The rev-list gap is dominated by history/object enumeration in grit-lib, not bulk pack I/O; cat-file scenarios exercise ODB read and read_info paths directly.

grit-bench: reachability bitmaps (git.git)

Scenario group bitmaps measures rev-list counting and a full-ref stateless upload-pack clone against a bare clone of upstream git/git at a pinned tag (v2.47.0 by default), prepared with git repack -adb and git commit-graph write --reachable. Library workloads run through hidden grit-bench drive commands (rev-list-count, rev-list-count-objects, serve-clone).

$ cargo build --release -p grit-utils -p grit-cli
$ ./target/release/grit-bench bitmaps --format json --output grit-utils/baselines/bitmaps-before.json

Baseline bitmaps-before.json (factory VM, 2026-10-10): rev-list-count-git.git — git ~63 ms vs grit ~207 ms; rev-list-count-objects-git.git — git ~24 ms vs grit ~546 s (no pack bitmap reader yet); serve-clone-git.git — git ~785 ms, grit failed (memory_cap / OOM under the serve time and RSS caps). Re-run with --repo after the cached fixture exists under GRIT_BENCH_ODB_CACHE.

grit-lib: parallel index-pack (in-memory)

Hyperfine on a ~90 000-object depth-50 pack (git fast-import + git repack -adf --depth=50, factory VM 2026-10-07). Grit uses pack_index_records_with_threads via cargo run --release -p grit-lib --example index_pack_bench (see GRIT_INDEX_PACK_BENCH_PACK / GRIT_INDEX_PACK_THREADS).

Speedup (8 vs 1 thread): ~1.83× on this fixture. Multi-threaded builds discover object boundaries with a discard decompress pass, then parallel workers inflate and hash whole objects; delta chains inflate, apply, and hash in parallel rounds. Grit does not yet beat git index-pack --threads=8 on this pack (Git also writes the on-disk .idx).

Integration coverage uses a ≥ 50 000 object fixture built with git fast-import and shared across tests via OnceLock (grit-lib/tests/index_pack_parallel.rs); the default release suite completes in about 41 s on the factory VM.

Hashing

Raw digest throughput uses grit_lib::hash::ObjectHasher (release build, Criterion). OpenSSL numbers are from openssl speed -evp sha1 -bytes 16384 and openssl speed -evp sha256 (16 KiB blocks where shown). Git blob hashing uses git hash-object on a 256 MiB file vs the gritx-hash-file example (same canonical blob object id, no object-database write).

Environment (2026-10-07): Intel Xeon (SHA-NI present), Linux, git 2.43.0, OpenSSL 3.0.13, Rust stable. hash-info --json reports {"sha1":"x86_sha_ni","sha256":"x86_sha_ni"}.

SHA-1 at 1 MiB and above meets the project bar (≥ 95% of OpenSSL on the same CPU). SHA-256 meets ≥ 90% of OpenSSL. End-to-end blob id for a large file beats stock Git 2.43, which still uses sha1dc for object hashing.

Commands

# Backends selected on this CPU
$ cargo run -p grit-examples --bin hash-info -- --json

# Criterion throughput (64 B … 64 MiB)
$ cargo bench -p grit-lib --bench hash

# OpenSSL + hyperfine vs git (creates a 256 MiB temp file)
$ ./scripts/bench-hash.sh

grit-bench vs Git

Methodology

• Fixtures use synthetic repos at large file counts and deep history, built deterministically in a scratch directory.
• Drivers are CLI invocations of grit and git with equivalent semantics (not argv-for-argv compatibility).
• Ratios are Grit median ÷ Git median (stored in each scenario’s ratio field; hyperfine also records mean and spread). Values below 1.00× mean Grit is faster; values above 2.00× are highlighted as regressions worth investigating.
• Spread shows Grit’s standard deviation across timed runs (hyperfine --min-runs / warmup settings from the baseline capture).

Reproduce locally

Build release binaries, install hyperfine, then run matching grit-bench scenarios from the repo root (see grit-utils/README.md for the current subcommands):

$ cargo build --release -p grit-cli -p grit-utils

# Example: hot-path scenarios at L/H tiers (switch, pick, merge)
$ ./target/release/grit-bench hot-paths --sizes 10000,100000 --format json --output /tmp/grit-bench-hot-paths.json

When roadmap item 2 adds more baseline JSON files on main, list their paths under [benchmarks].baseline in content/docs/site.toml and regenerate the site.

Compare a fresh run against the committed baseline (exit non-zero if any scenario ratio drifts beyond tolerance):

$ ./target/release/grit-bench compare grit-utils/baselines/hot-paths-before.json /tmp/grit-bench-hot-paths.json --tolerance 0.10

Hot-path acceptance (maintenance plan step 386)

Recorded on the factory VM after stacking pick/merge/stash perf work on origin/main. Before numbers are in grit-utils/baselines/hot-paths-before.json (2026-10-07); after in grit-utils/baselines/hot-paths-after.json (2026-10-08). Config isolation (GIT_CONFIG_NOSYSTEM, empty global config) applies to hot-path, suite commit, and related scenarios. FSMN variants: grit-utils/baselines/hot-paths-after-fsmn.json.

Machine (2026-10-08 acceptance): Intel Xeon (factory VM), 4 physical / 4 logical CPUs, 16 GiB RAM, Linux 6.12.94+, scratch filesystem ext4, rustc 1.99.0, git 2.43.0, release grit 0.5.1, hyperfine 2.x (see machine in each baseline JSON).

Machine (2026-10-09 refresh): same factory VM (Intel Xeon, 4 / 4 CPUs, 16 GiB RAM, Linux 6.12.94+, ext4 scratch), rustc 1.99.0, git 2.43.0, release grit 0.5.3 at 01a254dc, hyperfine 2.0.0. Committed baselines were regenerated with ./target/release/grit-bench {hot-paths,commit,all} --sizes 10000,100000 (default warmup / min-runs).

2026-10-09 refresh vs 2026-10-08 baselines: grit-bench compare reports ratio drift beyond the default 0.10 absolute tolerance on most scenarios (many improved). Regressions more than 20% worse on the Grit/Git ratio: switch-wide-100000 (2.98× → 22.54×), switch-wide-10000 (2.29× → 3.42×), merge-10000, pick-10000, pick-100000. status-dirty-100000 remains ~53× (unchanged). Suite status and add ratios improved at L/H except dirty-100k.

Suite at L/H (grit-utils/baselines/suite-after-LH.json): status scenarios remain far above 2× (dirty/clean at 10k and 100k files).

H/L acceptance (1.25× bar): after-run medians fail the scaling check for switch, switch-wide, pick, merge, and pick-series (not only switch/pick/merge). commit and add at H are within 2× of Git but add at L/H both exceed 2× at 10k and 100k files.

Two back-to-back hot-path after runs (hot-paths-after-run1.json vs run2.json) agree within 10% relative on every scenario except merge-100000 and pick-100000 (~13–14% relative spread). The default grit-bench compare tolerance is an absolute 0.10 on the ratio, which is tighter than ±10% relative for large ratios (e.g. pick-series).

Follow-up tracking (step 386)

No GitHub issues were filed from this factory run (file_bug_report is dogfooding-only). Track these against roadmap item 4:

• L >2×: switch, switch-wide, merge, pick-series, add (10k and 100k), status at L/H.
• H/L >1.25×: switch, switch-wide, pick, merge, pick-series.
• Repeatability: second hot-path after-run drift on merge-100000 and pick-100000 (~13–14% relative).
• CLI gaps: grep, rebase, reset, stash push (deferrals documented above).

Criterion (cargo bench -p grit-lib --bench hot_paths, release): see factory run log — apply_stash_L ~52 ms median; pick_2000_paths_L ~621 ms (library merge/checkout path, not CLI).

Roadmap commands without a grit engine yet

Object reads

The grit-bench odb suite (library drivers in grit-bench drive … vs system git) measures cat-file batch reads, rev-list --objects --all, and log -p on a cached bare git.git clone and a repacked 100k-file / 1000-commit synthetic repo. JSON reports include peak RSS per scenario (each workload measured in a fresh grit-bench measure-rss helper process via getrusage(RUSAGE_CHILDREN) on that run only) alongside wall time.

Reproduce:

$ cargo build --release -p grit-cli -p grit-utils
$ ./target/release/grit-bench odb --format json --output grit-utils/baselines/odb-read.json

Measured outcomes: the Object reads table below is generated from committed grit-utils/baselines/odb-read.json (hyperfine ≥5 runs per scenario, grit vs system git medians and peak RSS). Refresh with two invocations and grit-bench compare run1.json run2.json --tolerance 0.10 before updating the baseline file and running make docs.

grit-bench: network (clone, fetch, push, ls-remote)

The grit-bench network suite compares grit and system git on the same remote URLs. Cached fixtures live under GRIT_BENCH_NETWORK_CACHE (default /tmp/grit-bench-network-cache):

Scenarios include clone over file://, git http-backend (local CGI), and grit-http-server smart HTTP; incremental fetch and no-op fetch over file://; push over file://; git ls-remote vs grit remote refs on the many-refs fixture; and a server-side comparison that times system git clone against git http-backend vs grit-http-server on the same published bare repo. Hermetic git config matches other suites (GIT_CONFIG_NOSYSTEM, empty global config).

$ cargo build --release -p grit-cli -p grit-http-server -p grit-utils
$ ./target/release/grit-bench network --format json --output grit-utils/baselines/network.json
$ ./target/release/grit-bench compare grit-utils/baselines/network.json /tmp/network-rerun.json --tolerance 0.10
$ make docs

Integration smoke (tiny fixtures, eight scenarios; requires hyperfine on PATH): cargo test -p grit-utils network_scenario_smoke_end_to_end.

Acceptance bars for step 480 are ≤1.2× git wall time and ≤1.5× git peak RSS on the large fixture; compare the table ratios to those bars rather than this prose.

Results

Summary by operation

add

bitmaps

blame

commit

delta_encode

merge

network-clone

network-fetch

network-ls-remote

network-push

network-server

object_reads

odb_backend

pick

status

switch

To refresh this page after updating a baseline JSON file, regenerate the static site:

$ make docs

The docs --check step fails if baseline numbers change without re-running make docs.

For other library surfaces under test, see the grit-lib API on docs.rs.