Parallelism

Grit centralizes worker counts and batch thresholds in grit_lib::parallel so embedders and the CLI share one policy.

Thread policy

ThreadPolicy holds optional overrides:

• threads_override — explicit worker count (0 = auto / all logical CPUs, 1 = serial). The grit CLI sets this from global --threads.
• min_items_override — minimum input length before spawning a pool. Setting it to 0 forces the parallel path on tiny inputs (integration tests use this).
• min_total_bytes_override — optional byte threshold for hashing-style maps (Some(0) disables the byte gate).

Call ThreadPolicy::resolve with a ParallelOp and a merged ConfigSet to obtain ParallelOptions (threads + batch thresholds).

Git value semantics for index.threads

Matches Git's repo_config_get_index_threads:

• unset — auto (all logical CPUs for hash-style refresh paths)
• true or 0 — auto
• false or 1 — serial (one worker)
• integer n > 1 — up to n workers (clamped to available CPUs)
• other integers (for example -3) — preserved as serial (Git only parallelizes when nr_threads > 1 after treating 0 as auto)

When core.preloadIndex is false, index refresh/load operations stay serial regardless of index.threads.

EOIE / IEOT (parallel index read and write)

When index.threads is explicitly set to more than one worker, Git may also write EOIE (end-of-index-entries) and IEOT (index entry offset table) extensions so entry blocks can be parsed on worker threads while extension payloads load in parallel. Grit reads and writes these extensions when the same config keys are enabled:

• index.recordEndOfIndexEntries / index.recordendofindexentries
• index.recordOffsetTable / index.recordoffsettable

If those booleans are unset, they default to on when index.threads is explicitly configured and not serial (1 / false).

Use Index::parse_with_algo_and_config with IndexParseContext (thread policy from Repository::thread_policy) and the repository config. Invalid EOIE data falls back to the serial entry parse. Writes go through Index::write_with_config_and_policy so split indexes and sparse indexes keep working.

pack.threads (index-pack / pack-objects)

Unset or 0 selects auto (all logical CPUs). Positive integers cap worker count. Git rejects booleans, non-numeric strings, and negative values with a config error; use ThreadPolicy::try_resolve for ParallelOp::IndexPack to surface those errors. Fetch and clone pass CloneOptions::thread_policy (and the repository Environment) into received-pack indexing so global --threads overrides pack.threads and command-scope GIT_CONFIG_* / -c pack.threads=N still applies when no override is set.

Stat preload during status/diff uses Git's ~500 entries per thread rule with a 20-thread cap (ThreadPolicy::index_stat_preload_threads), still honoring core.preloadIndex, index.threads, and the global thread override.

The stat refresh in status, diff, reset and stash has to re-hash racily clean entries: files whose mtime falls in the same second as the index file, which is every file right after a clone, switch or reset --hard. On Windows, where core.usenanosec defaults to on, an entry is racy only when its mtime is at or after the index file's to the 100 ns, so a fresh checkout has none. It collects those entries from the stat scan and hashes them on the IndexRefresh pool before applying any stat updates. It uses the scan's lstat results instead of a second one per file. This step keeps the 32-entry gate but has no byte threshold, because opening each small file costs more than hashing it. A stat-thread cap (DiffIndexToWorktreeOptions::stat_parallel_threads, Some(1) for serial) also caps this pool.

The untracked walk in status (no pathspecs) uses the IndexRefresh worker count and runs one task per directory entry. With at least 1000 index entries it first splits directories that only recurse (they hold tracked files and are not a gitlink or nested submodule) into their children, until each worker has about eight directories, so a work tree whose files sit under a single top-level directory still walks in parallel. Splitting changes neither the reported paths nor their order.

Optional pack.windowMemory caps sliding-window retention during delta search (Git pack.windowMemory, default unlimited).

Global --threads (CLI)

grit --threads sets RepositoryOptions::thread_policy for the whole process:

Embedders pass the same semantics via ThreadPolicy::with_threads_override.

Determinism

Parallelism must not change Git-compatible results, only latency:

• try_par_map / par_map return outputs in input order even when workers finish out of order.
• Pack writing (PackWrite) emits the same bytes as the serial writer (header order follows the pack plan).
• Windowed delta search (PackDeltaSearch) partitions work by fixed segments so the chosen delta base for each object is independent of thread count.
• Index-pack builds the same object ids and offsets as the serial path; only scheduling differs. Each worker resolves the whole delta tree under one base at a time, depth-first, so memory follows the deepest chain (capped per worker) rather than the number of deltas, and adding threads adds only one chain's worth of memory each. Workers read compressed deltas with positioned file reads and hand their records over in small batches, so neither the pack file nor per-thread result buffers grow with the pack.
• Checkout records the same index entries and file contents; parallel creation is ordered before post-processing.

There is no global thread pool and no environment reads inside grit_lib::parallel; pass policy explicitly (via RepositoryOptions::thread_policy or a local ThreadPolicy).

Ordered parallel maps

try_par_map_force skips batch thresholds (used for index-pack). Hash-specific helpers in grit_lib::objects::hash remain thin wrappers over these APIs.

Progress and cancellation

Long-running library work reports through ProgressSink and accepts a separate Cancel signal. Call poll_cancel at loop boundaries (or rely on built-in checks in fetch pack demux and status scans); either the cancel handle or ProgressSink::should_cancel stops the operation with Error::Cancelled. Pass NeverCancel when interruption is not needed.

Network and long-running entry points that accept Cancel today include fetch_remote, fetch_local_with_progress, Remote::fetch / Remote::push, RepositoryRemotes::fetch_all, clone (fetch and post-fetch checkout), pull, push_local, status, switch_repository, reset_to_commit (keep/hard worktree apply), apply_rewrite, checkout helpers (checkout_between_trees_with_progress), bundle fetch/unbundle (fetch_from_bundle_with_progress, Bundle::unbundle_with_progress), and received-pack indexing (install_pack_path_with_progress / ingest_received_pack_path_with_progress). Local file:// / path clones poll cancel while linking or copying objects into the new repository. The coordinator thread calls poll_cancel at ingest phase boundaries (before/after parallel object hashing, between idx write, verify, and rename); index-pack worker threads observe only an AtomicBool snapshot, not ProgressSink. Not yet cancellable mid-flight: thin-pack repair during fetch, a single ordered delta-resolution pool batch, and v2 idx sorting inside write_v2_pack_index_with_trailer — cancel is checked before/after those steps. Sideband demux during fetch uses the same cancel handle as the transport. Pass NeverCancel when interruption is not needed.

A push checks for a cancel all the way through: while it walks the objects to send, enumerates and delta-searches them (search workers stop at the next chunk), and streams the pack, and while a local remote indexes it. A cancel before the objects are stored returns Error::Cancelled with no remote ref moved and no pack or temporary file left in the remote, so the same push can simply be retried. Over smart HTTP, a cancel mid-pack abandons the request body (the server sees a truncated request and updates nothing), and a cancel while the server is still unpacking returns within a fraction of a second; at that point the whole request was sent, so the server may still apply it, as with an interrupted git push. Waiting for the report on git:// and ssh connections is not interruptible.

A cancelled (or otherwise failed) checkout, switch, hard reset, or work-tree restore_paths with --source leaves refs, the index and the work tree as they were: until the index is written, every file it created is removed again, files it deleted or overwrote are restored, and paths it could not put back are listed in Error::CheckoutRollbackIncomplete.

Work-tree sessions take a sink and a cancel handle too: staging (stage, stage_on_index, stage_for_commit), commits (commit_with_scope, commit_worktree, create_commit, continue_merge), RepositoryStash::push / apply / pop, integrate_branch, start_rebase / continue_rebase / skip_rebase, replay_commit and restore_paths. Hashing work-tree files reports ProgressPhase::StagingFiles (no total: the changed set is found while scanning); work-tree updates report ProgressPhase::CheckingOutFiles. Staging's parallel hash workers stop on the shared flag of a FlagCancel, so a cancel from another thread lands within one file per worker. The rule for where a cancel lands is the same everywhere: before the commit object, stash entry or ref moves, nothing visible changes (the on-disk index is not written; blobs already hashed stay as unreachable loose objects, as after an interrupted git add). Once a commit object is written, its branch update always follows; once a merge checkout finished, the branch moves. A cancel during a checkout inside these sessions is rolled back as described above. Files a session writes itself stay written for reset --hard: the content-merged files of a stash apply onto a moved HEAD, and the paths restore_paths already restored. A stash push keeps the stored entry when its reset (full, --keep-index, --staged, pathspec or hunk selection) is cancelled. A rebase that stops on a conflict checks for a cancel before it records the stop; once the stop state and HEAD are written, its checkout always finishes, so continue_rebase never sees a stop without its conflict stages.

Checkout resolves the Checkout worker policy before splitting eligible jobs into cancellation chunks. The parallelism threshold applies to that set of jobs, so even a small cancellation chunk can use the selected pool. Chunks start at 8 files and double up to 2048; cancellation is checked between chunks and during serial reads and writes. Populated-tree checkout already walks in 2048-entry chunks and starts at full size after its first walk chunk. Prepared contents use a separate 1 MiB flush threshold; parallel readers run in bounded batches, so growing cancellation chunks does not retain all their file contents at once. Streamed files consume part of that budget too, limiting each flush to at most eight streams before cancellation is checked again. Bulk checkout reuses cached small loose objects (up to 64 KiB each); direct worktree writes and the one-pass local-object walk bypass that cache bookkeeping.

Tests

• Policy unit tests live in grit-lib/src/parallel/policy.rs.
• Public API coverage: grit-lib/tests/parallel_api.rs.
• Hash wrappers: grit-lib/tests/hash_parallel_api.rs (unchanged behavior).
• grit_test_support::forced_parallel_policy sets threads = 4 and min_items = 0 for deterministic parallel paths in integration tests (see the grit-test-support crate in the workspace).
• CI test-threaded runs cargo test --release --workspace with index.threads, pack.threads, checkout.workers, and checkout.thresholdForParallelism=0 forced via GIT_CONFIG_COUNT (see TESTING.md).