# Writing storage backends

> Implement object and ref stores, open a repository on them, test with conformance suites, and know what still needs a .git directory on disk.

Embedders replace grit’s default files layout with their own **object store**, **ref store**, or both. This page is the end-to-end guide: traits to implement, how to wire [`RepositoryOptions`](https://docs.rs/grit-lib/latest/grit_lib/config/environment/struct.RepositoryOptions.html), optional capabilities, performance hooks, and how to prove correctness with the shared conformance suites. For lookup order, alternates, pack ingest, and benchmark tables on custom primaries, see [Object stores](https://grit-scm.com/docs/library/storage-backends/object-stores.md/index.md).

## What to implement

### Object storage (read path)

[`ObjectStore`](https://docs.rs/grit-lib/latest/grit_lib/objects/odb/store/trait.ObjectStore.html) is the minimum surface:

| Method | Contract |
| ------ | -------- |
| `read` / `read_info` / `open_stream` | Return `Ok(None)` when the id is absent; propagate I/O or corruption errors (do not map outages to “missing”). |
| `contains` | Must match what `read` would return for the same id. |
| `hash_algo` | Must match `extensions.objectformat` / repository init. |
| `for_each_object` | List every stored id; release internal locks before calling the callback (callbacks often call back into the store). |
| `lookup_prefix` / `contains_many` | Override when a full scan per call is too expensive (see [Performance](#performance-checklist)). |

Optional streaming helpers: [`ObjectStore::read_into`](https://docs.rs/grit-lib/latest/grit_lib/objects/odb/store/trait.ObjectStore.html#method.read_into) (default copies from `read`), [`ObjectStore::with_stream`](https://docs.rs/grit-lib/latest/grit_lib/objects/odb/struct.Odb.html#method.with_stream) on the repository `Odb` handle for large blobs.

### Object storage (write path)

[`WritableObjectStore`](https://docs.rs/grit-lib/latest/grit_lib/objects/odb/store/trait.WritableObjectStore.html): idempotent `write`, plus `flush` / `rollback` when you batch transactions. Pack fetch, push, bundle install, and commits all end in `flush` on the primary.

### Pruning (optional)

[`PrunableObjectStore`](https://docs.rs/grit-lib/latest/grit_lib/objects/odb/store/trait.PrunableObjectStore.html): implement `remove`, `as_prunable`, and optionally `object_mtime` for grace-period prune. Without it, `gc`, `prune`, and `Odb::gc` return [`RepositoryError::UnsupportedObjectStore`](https://docs.rs/grit-lib/latest/grit_lib/repo/error/enum.RepositoryError.html#variant.UnsupportedObjectStore) with operation `prune` — never touch `objects/` on disk silently.

### Ref storage

[`RefStore`](https://docs.rs/grit-lib/latest/grit_lib/refs/store/trait.RefStore.html): `read_raw`, `resolve`, iteration (`for_each_ref` with the empty-prefix rule from [`RefStore`](https://docs.rs/grit-lib/latest/grit_lib/refs/store/trait.RefStore.html)), reflog read/write, and **transactions** via `prepare` → `commit` / `abort`.

You do not reimplement Git’s ref rules: use [`refs::store::backend`](https://docs.rs/grit-lib/latest/grit_lib/refs/store/backend/index.html) helpers (`check_transaction`, `apply_transaction`, `RefBatchState`, `SingleWriterGate` for in-process writers). Report backend failures as [`RefStoreError::Backend`](https://docs.rs/grit-lib/latest/grit_lib/refs/store/enum.RefStoreError.html#variant.Backend).

## Opening and initializing

### Inject stores when opening

[`RepositoryOptions::with_object_store`](https://docs.rs/grit-lib/latest/grit_lib/config/environment/struct.RepositoryOptions.html#method.with_object_store) is the only supported way to set a custom primary object store. Add read layers, an attached commit-graph, or alternates with [`RepositoryOptions::with_odb_builder`](https://docs.rs/grit-lib/latest/grit_lib/config/environment/struct.RepositoryOptions.html#method.with_odb_builder) ([`OdbBuilder`](https://docs.rs/grit-lib/latest/grit_lib/objects/odb/struct.OdbBuilder.html)).

[`RepositoryOptions::with_ref_store`](https://docs.rs/grit-lib/latest/grit_lib/config/environment/struct.RepositoryOptions.html#method.with_ref_store) replaces loose/packed/reftable refs under `refs/` and the on-disk `HEAD` file. Pair it with [`with_initial_branch`](https://docs.rs/grit-lib/latest/grit_lib/config/environment/struct.RepositoryOptions.html#method.with_initial_branch) when `HEAD` should start on an unborn branch.

```rust
use std::sync::Arc;

use grit_lib::config::environment::RepositoryOptions;
use grit_lib::objects::odb::store::MemoryStore;
use grit_lib::objects::HashAlgo;
use grit_lib::refs::store::MemoryRefStore;
use grit_lib::repo::{InitOptions, Repository};

let dir = tempfile::tempdir()?;
let (repo, _) = Repository::init(
    dir.path(),
    &InitOptions::new()
        .with_initial_branch("main")
        .with_repository_options(
            RepositoryOptions::empty()
                .with_object_store(Arc::new(MemoryStore::new(HashAlgo::Sha1)))
                .with_ref_store(Arc::new(MemoryRefStore::new()))
                .with_work_tree(dir.path()),
        ),
)?;
# Ok::<(), grit_lib::error::Error>(())
```

### What `Repository::init` writes

[`Repository::init`](https://docs.rs/grit-lib/latest/grit_lib/repo/struct.Repository.html#method.init) with [`InitOptions::with_repository_options`](https://docs.rs/grit-lib/latest/grit_lib/repo/struct.InitOptions.html#method.with_repository_options) creates only what your stores do **not** own:

| Injected | On disk under the git directory |
| -------- | -------------------------------- |
| Object store only | Everything except `objects/` |
| Ref store only | Everything except `HEAD` and `refs/` |
| Both | `config` only (plus the work tree beside it when not bare) |

Object format comes from the store’s `hash_algo`; do not fight it with [`InitOptions::object_format`](https://docs.rs/grit-lib/latest/grit_lib/repo/struct.InitOptions.html#structfield.object_format) unless you mean to error with [`RepositoryError::ObjectFormatMismatch`](https://docs.rs/grit-lib/latest/grit_lib/repo/error/enum.RepositoryError.html#variant.ObjectFormatMismatch).

## Capability table

### `PrunableObjectStore`

| API | On a prunable custom primary |
| --- | --- |
| [`Maintenance::prune_loose_objects`](https://docs.rs/grit-lib/latest/grit_lib/maintenance/struct.Maintenance.html#method.prune_loose_objects) | Deletes unreachable objects past `gc.pruneExpire` |
| [`Maintenance::gc`](https://docs.rs/grit-lib/latest/grit_lib/maintenance/struct.Maintenance.html#method.gc) | Pack-refs + reflog expiry + prune (skips on-disk repack / commit-graph) |
| [`Odb::gc`](https://docs.rs/grit-lib/latest/grit_lib/objects/odb/struct.Odb.html#method.gc) | Deletes all unreachable objects (no grace) |

### `FilesOnlyOperation` (typed refusal)

Operations that need the on-disk `objects/` layout return [`RepositoryError::UnsupportedObjectStore`](https://docs.rs/grit-lib/latest/grit_lib/repo/error/enum.RepositoryError.html#variant.UnsupportedObjectStore) with a [`FilesOnlyOperation`](https://docs.rs/grit-lib/latest/grit_lib/repo/error/enum.FilesOnlyOperation.html) variant — match on the enum, not strings. Grit’s fallbacks on a custom primary:

| Operation | Fallback on custom primary |
| --------- | -------------------------- |
| Repack / `.pack` install | Pack stream ingested object-by-object ([`ingest_pack_into`](https://docs.rs/grit-lib/latest/grit_lib/objects/unpack_objects/fn.ingest_pack_into.html), [`ingest_received_pack`](https://docs.rs/grit-lib/latest/grit_lib/objects/index_pack/fn.ingest_received_pack.html)) |
| Commit-graph file | Build in memory with [`Maintenance::build_commit_graph`](https://docs.rs/grit-lib/latest/grit_lib/maintenance/struct.Maintenance.html#method.build_commit_graph); attach bytes via [`OdbBuilder::commit_graph`](https://docs.rs/grit-lib/latest/grit_lib/objects/odb/struct.OdbBuilder.html#method.commit_graph) |
| Promisor / partial clone ODB | Not supported (no silent filesystem promisor) |
| Prune / `gc` object deletion | Only with `PrunableObjectStore`; otherwise typed `prune` refusal |

The generated variant table lives on [Object stores](https://grit-scm.com/docs/library/storage-backends/object-stores.md/index.md#what-works-with-a-custom-primary) (`| Name | Variant | Description |
| --- | --- | --- |
| `gc` | [`Gc`](https://docs.rs/grit-lib/latest/grit_lib/repo/error/enum.FilesOnlyOperation.html#variant.Gc) | Garbage collection without a repository: [`Odb::gc`](https://docs.rs/grit-lib/latest/grit_lib/objects/odb/struct.Odb.html#method.gc) on an `Odb` built standalone, with no git directory to find reachability roots in. |
| `prune` | [`Prune`](https://docs.rs/grit-lib/latest/grit_lib/repo/error/enum.FilesOnlyOperation.html#variant.Prune) | Deleting unreachable objects from a custom primary that cannot delete ([`WritableObjectStore::as_prunable`](https://docs.rs/grit-lib/latest/grit_lib/objects/odb/trait.WritableObjectStore.html#method.as_prunable) is `None`): [`Maintenance::prune_loose_objects`](https://docs.rs/grit-lib/latest/grit_lib/maintenance/struct.Maintenance.html#method.prune_loose_objects), [`refs::gc::prune_loose_unreachable`](https://docs.rs/grit-lib/latest/grit_lib/refs/gc/fn.prune_loose_unreachable.html), [`Odb::gc`](https://docs.rs/grit-lib/latest/grit_lib/objects/odb/struct.Odb.html#method.gc), [`Maintenance::gc`](https://docs.rs/grit-lib/latest/grit_lib/maintenance/struct.Maintenance.html#method.gc). Implement [`PrunableObjectStore`](https://docs.rs/grit-lib/latest/grit_lib/objects/odb/store/trait.PrunableObjectStore.html) to support them. |
| `repack` | [`Repack`](https://docs.rs/grit-lib/latest/grit_lib/repo/error/enum.FilesOnlyOperation.html#variant.Repack) | Repacking into `objects/pack/`: [`Maintenance::repack`](https://docs.rs/grit-lib/latest/grit_lib/maintenance/struct.Maintenance.html#method.repack) and [`Maintenance::repack_all`](https://docs.rs/grit-lib/latest/grit_lib/maintenance/struct.Maintenance.html#method.repack_all). |
| `commit_graph_write` | [`CommitGraphWrite`](https://docs.rs/grit-lib/latest/grit_lib/repo/error/enum.FilesOnlyOperation.html#variant.CommitGraphWrite) | Writing `objects/info/commit-graph`: [`Odb::write_commit_graph`](https://docs.rs/grit-lib/latest/grit_lib/objects/odb/struct.Odb.html#method.write_commit_graph) and [`Maintenance::write_commit_graph`](https://docs.rs/grit-lib/latest/grit_lib/maintenance/struct.Maintenance.html#method.write_commit_graph). A custom store builds its graph in memory with [`Maintenance::build_commit_graph`](https://docs.rs/grit-lib/latest/grit_lib/maintenance/struct.Maintenance.html#method.build_commit_graph) instead. |
| `midx_write` | [`MidxWrite`](https://docs.rs/grit-lib/latest/grit_lib/repo/error/enum.FilesOnlyOperation.html#variant.MidxWrite) | Writing `objects/pack/multi-pack-index`: [`Maintenance::write_midx`](https://docs.rs/grit-lib/latest/grit_lib/maintenance/struct.Maintenance.html#method.write_midx). |
| `pack_store` | [`PackStore`](https://docs.rs/grit-lib/latest/grit_lib/repo/error/enum.FilesOnlyOperation.html#variant.PackStore) | The files backend’s pack and MIDX read cache: [`Odb::pack_store`](https://docs.rs/grit-lib/latest/grit_lib/objects/odb/struct.Odb.html#method.pack_store). |
| `pack_install` | [`PackInstall`](https://docs.rs/grit-lib/latest/grit_lib/repo/error/enum.FilesOnlyOperation.html#variant.PackInstall) | Installing a pack file and its `.idx` under `objects/pack/`: [`index_pack::install_pack_bytes`](https://docs.rs/grit-lib/latest/grit_lib/objects/index_pack/fn.install_pack_bytes.html) and [`index_pack::install_pack_path`](https://docs.rs/grit-lib/latest/grit_lib/objects/index_pack/fn.install_pack_path.html). Use [`index_pack::ingest_received_pack`](https://docs.rs/grit-lib/latest/grit_lib/objects/index_pack/fn.ingest_received_pack.html) to write a pack into any primary. |
| `largest_pack_index` | [`LargestPackIndex`](https://docs.rs/grit-lib/latest/grit_lib/repo/error/enum.FilesOnlyOperation.html#variant.LargestPackIndex) | Finding the largest pack index in `objects/pack/`: [`Odb::largest_pack_index_path`](https://docs.rs/grit-lib/latest/grit_lib/objects/odb/struct.Odb.html#method.largest_pack_index_path). |
| `loose_write` | [`LooseWrite`](https://docs.rs/grit-lib/latest/grit_lib/repo/error/enum.FilesOnlyOperation.html#variant.LooseWrite) | Writing loose object files into `objects/`: [`Odb::write_local`](https://docs.rs/grit-lib/latest/grit_lib/objects/odb/struct.Odb.html#method.write_local), [`Odb::write_raw_local`](https://docs.rs/grit-lib/latest/grit_lib/objects/odb/struct.Odb.html#method.write_raw_local) and [`Odb::write_loose_materialize`](https://docs.rs/grit-lib/latest/grit_lib/objects/odb/struct.Odb.html#method.write_loose_materialize). [`Odb::write`](https://docs.rs/grit-lib/latest/grit_lib/objects/odb/struct.Odb.html#method.write) works with any primary. |
| `files_source` | [`FilesSource`](https://docs.rs/grit-lib/latest/grit_lib/repo/error/enum.FilesOnlyOperation.html#variant.FilesSource) | The files-backed primary source itself: [`Odb::primary`](https://docs.rs/grit-lib/latest/grit_lib/objects/odb/struct.Odb.html#method.primary). Use [`Odb::primary_writable`](https://docs.rs/grit-lib/latest/grit_lib/objects/odb/struct.Odb.html#method.primary_writable) for the primary of any kind. |
| `offload` | [`Offload`](https://docs.rs/grit-lib/latest/grit_lib/repo/error/enum.FilesOnlyOperation.html#variant.Offload) | Moving blobs to a large-object promisor: [`Maintenance::offload_blobs`](https://docs.rs/grit-lib/latest/grit_lib/maintenance/struct.Maintenance.html#method.offload_blobs), for the main repository and for the promisor repository. |
| `promisor_rewrite` | [`PromisorRewrite`](https://docs.rs/grit-lib/latest/grit_lib/repo/error/enum.FilesOnlyOperation.html#variant.PromisorRewrite) | Rewriting packs to leave out promised objects, as offload and dropping promised blobs do. |
| `partial_fetch` | [`PartialFetch`](https://docs.rs/grit-lib/latest/grit_lib/repo/error/enum.FilesOnlyOperation.html#variant.PartialFetch) | Fetching or cloning with an object filter (a partial clone) into a custom primary. |
| `receive_quarantine` | [`ReceiveQuarantine`](https://docs.rs/grit-lib/latest/grit_lib/repo/error/enum.FilesOnlyOperation.html#variant.ReceiveQuarantine) | The receive-pack quarantine directory under `objects/`. A custom primary receives pushes through an in-memory overlay instead. |
| `drop_large_blobs` | [`DropLargeBlobs`](https://docs.rs/grit-lib/latest/grit_lib/repo/error/enum.FilesOnlyOperation.html#variant.DropLargeBlobs) | Dropping promised blobs from promisor packs: [`Maintenance::drop_promised_blobs`](https://docs.rs/grit-lib/latest/grit_lib/maintenance/struct.Maintenance.html#method.drop_promised_blobs). |
`).

## Performance checklist

| Hook | Why |
| ---- | --- |
| [`read_into`](https://docs.rs/grit-lib/latest/grit_lib/objects/odb/store/trait.ObjectStore.html#method.read_into) | Rev-list, merge-base, and change-id scans reuse one buffer per commit instead of allocating every `read`. |
| [`contains_many`](https://docs.rs/grit-lib/latest/grit_lib/objects/odb/store/trait.ObjectStore.html#method.contains_many) | Write-tree and fetch/push negotiation ask about many ids at once; default is one `contains` per id. |
| [`lookup_prefix`](https://docs.rs/grit-lib/latest/grit_lib/objects/odb/store/trait.ObjectStore.html#method.lookup_prefix) | Short-id and `RevWalk::abbreviate`; default scans all objects. Use [`ObjectIdPrefix::lowest`](https://docs.rs/grit-lib/latest/grit_lib/objects/struct.ObjectIdPrefix.html#method.lowest) / [`highest`](https://docs.rs/grit-lib/latest/grit_lib/objects/struct.ObjectIdPrefix.html#method.highest) for sorted stores. |
| Attached commit-graph | Walks skip commit reads; see [Object stores — Commit-graphs for a custom store](https://grit-scm.com/docs/library/storage-backends/object-stores.md/index.md#commit-graphs-for-a-custom-store) and the benchmark table under `<!-- grit:benchmark-tables -->` on [Benchmarks](https://grit-scm.com/docs/library/benchmarks.md/index.md) for rev-list / merge-base rows. |

## Testing

Three workspace conformance crates exercise backends before you wire a full `Repository`:

| Suite | Crate module | What it checks |
| ----- | ------------ | -------------- |
| Object read/write | `grit_test_support::odb_conformance` | `run_read_suite`, `run_write_suite` on `ObjectStore` / `WritableObjectStore` |
| Ref store | `grit_test_support::refstore_conformance` | `run_suite` on `Arc<dyn RefStore>` |
| Repository | `grit_test_support::repository_conformance` | `BackendFixture` + `run_repository_suite`, `run_parity` for typed `RepositorySnapshot` parity |

Example pattern (object store):

```rust
use grit_test_support::odb_conformance::{run_read_suite, run_write_suite};

#[test]
fn my_store_passes_odb_conformance() {
    run_read_suite(|| seed_my_store(&FIXTURE_OBJECTS));
    run_write_suite(|| MyStore::new(HashAlgo::Sha1));
}
```

End-to-end SQLite + in-memory refs (parity against the files backend):

- Source: [`grit-examples/tests/sqlite_repository_conformance.rs`](https://github.com/gitbutlerapp/grit/blob/main/grit-examples/tests/sqlite_repository_conformance.rs)
- Library tests: [`grit-lib/tests/repository_conformance_memory.rs`](https://github.com/gitbutlerapp/grit/blob/main/grit-lib/tests/repository_conformance_memory.rs), [`repository_conformance_files.rs`](https://github.com/gitbutlerapp/grit/blob/main/grit-lib/tests/repository_conformance_files.rs)

After changing integration tests under `grit-examples/tests/`, run `python3 scripts/sync_integration_tests.py`.

## What still needs a `.git` directory on disk

When **both** stores are injected, [`Repository::open`](https://docs.rs/grit-lib/latest/grit_lib/repo/struct.Repository.html#method.open) does not require a populated git directory; config can come from [`Environment::push_config_override`](https://docs.rs/grit-lib/latest/grit_lib/config/environment/struct.Environment.html#method.push_config_override) only. Even then, several subsystems still read or write paths under the git directory (or require a work tree):

| Area | Without on-disk git layout |
| ---- | --------------------------- |
| Objects, refs, config (injected) | Fully served by your stores + environment |
| Log, rev-parse, merge trees, notes, bundles to `Write` | Work through stores |
| Fetch (local) | Works; pack spool uses temp files; `FETCH_HEAD` not written |
| Index, staging, `COMMIT_EDITMSG`, merge/rebase state | Need git-dir files |
| Work-tree commands (status, checkout, stash) | Need index + work tree |
| Linked worktrees, rerere, sparse-checkout | Need their admin/state files |
| Hooks expecting another `git` to read refs/objects | Not supported for purely injected stores |
| `gc` / repack / on-disk commit-graph | Typed `UnsupportedObjectStore` unless prunable / in-memory graph (see above) |

The same matrix appears in [Object stores — No git directory at all](https://grit-scm.com/docs/library/storage-backends/object-stores.md/index.md#no-git-directory-at-all). Injecting **only** the object store leaves refs and reflogs on disk as usual.

## Example: SQLite object database

`grit_examples::sqlite_odb::SqliteOdbStore` keeps zlib-compressed store bytes in SQLite (one row per object id). The runnable demo commits through a SQLite primary, walks history, and exports loose objects for system `git fsck`:

```rust
//! SQLite object store wired through [`RepositoryOptions::with_object_store`](https://grit-scm.com/docs/library/storage-backends/grit_lib::config::environment::RepositoryOptions::with_object_store/index.md).
//!
//! Writes a small commit graph in SQLite, walks history with [`rev_list`],
//! then exports objects as loose files for interoperability with system Git.

use std::env;
use std::path::PathBuf;

use grit_examples::sqlite_odb;

fn main() -> grit_lib::error::Result<()> {
    let (root, _tmpdir) = match env::args().nth(1) {
        Some(path) => (PathBuf::from(path), None),
        None => {
            let dir = tempfile::tempdir().map_err(grit_lib::error::Error::Io)?;
            let root = dir.path().to_path_buf();
            (root, Some(dir))
        }
    };
    let log = sqlite_odb::run_sqlite_object_store_demo(&root)?;
    for oid in log {
        println!("{oid}");
    }
    Ok(())
}
```

```bash
cargo run -p grit-examples --example sqlite-object-store
```

Run the conformance-backed integration tests with:

```bash
cargo test -p grit-examples --features full-tests sqlite_repository_conformance
```

## Related pages

- [Object stores](https://grit-scm.com/docs/library/storage-backends/object-stores.md/index.md) — `Odb` layering, alternates, pack ingest, benchmarks
- [Refs](https://grit-scm.com/docs/library/storage-backends/refs.md/index.md) — `References` handle, reflog, branch tracking
- [Embedding](https://grit-scm.com/docs/library/storage-backends/embedding.md/index.md) — `Environment`, diagnostics, subprocess injection
