Evidence

Measured, with receipts

Every number on this page names the file it came from, the host it ran on, and the commit it measured. When Hyphae loses to a specialized baseline, that number is printed with the same prominence as a win.

The evidence culture

Hyphae does not publish a number without a receipt underneath it. A receipt is a committed file under docs/gates/evidence/ that names the host, the exact source commit, the baseline versions and configuration, the workload definition, and the raw harness output. Receipts are never hand-edited after the run; where a number is wrong, the fix is a new receipt, not a correction to the old one.

Losses are published, not buried

The bare-metal receipt below reports that SQLite's prepared point read is 11× faster than Hyphae's and that Tantivy ingests 41× faster. Both numbers sit in the same tables, at the same font size, as the numbers where Hyphae leads. Hyphae's public wording policy forbids combative language about named systems ("beats", "crushes", "dethrones") in any direction and requires every comparison to record the other system's stronger results with equal prominence.

Environment classes

Performance statements carry the class of environment they were measured on. Only class 3 supports a comparative statement against a named baseline.

ClassDefinitionComparative claims
1 — development observation Warm, concurrency-1, developer hardware. Never quotable externally.
2 — virtualized operational scale The closed G7 C-60 authority: a DigitalOcean VM exercising all 11 surfaces at C1/C8/C32 concurrency. Quotable only with the explicit "virtualized, no latency certification" qualifier.
3 — dedicated hardware Bare-metal receipts produced by the baseline harness with pinned baselines, byte-identical deterministic workloads, and like-for-like durability. The only class that supports a comparative statement — and every one names its receipt.

Source: docs/product/claims.md, “Performance-evidence claims”. The main receipt on this page is class 3: AWS i7i.metal-24xl bare metal.

Gates G0 through G8

The ordered Native Phase 1 program closes eight gates before a release ships, plus one release-closure gate. "Closed" means an exact source commit passed every requirement of that gate under hosted, reviewed CI — not that the underlying capability is unbounded. G7, for example, closed on a pre-1.2 commit and certifies the operational-scale matrix without per-operation authorization; every release since 1.2 must either cite that scoped profile or wait for a G7 re-run on a release commit.

GateStatus
G0Closed at 14b4ec9 — 8/8 hosted and governance exact-SHA closure
G1Closed at 14b4ec9 — 7/7 hosted exact-SHA substrate closure
G2Closed at a839037 — 8/8 hosted bounded relational closure; no universal SQL or official benchmark claim
G3Closed at a839037 — 11/11 hosted suite-bound structure closure
G4Closed at 0059fce — 12/12 hosted corpus-bound search closure
G5Closed at b7cf651 — 8/8 hosted exact-SHA convergence closure
G6Closed at c57cc07 — 14/14 requirements, 42/42 exact-SHA platform cells
G7Closed at ff188af — 11/11 surfaces, 33/33 C-60 operational-scale cells; environment-bound, no dedicated-hardware or interference certification
G8Closed per release (first at e88f2ea for 1.0.0; 3.0.0 at 24bce1a) — nine-requirement exact-SHA release closure: Linux/macOS/Windows functional evidence, crash/corruption/resource/power-loss matrices, four signed packages, SBOM/provenance, migration, soak, independent restore

Source: docs/gates/native-gate-status.md. G0–G7 form the ordered product-readiness prefix; G8 closes independently per release. 3.0.0's G8 closure ran as readiness run 33836088262, signed release run 33838703304, and closure run 33836655173, retained byte-for-byte at closures/native-g8-24bce1a.json.

Bare metal, 2026‑09‑03

Host: AWS i7i.metal-24xl (bare metal, Intel Xeon Platinum 8559C, 96 logical CPUs, 755 GiB RAM, six 3.75 TB local instance-store NVMe disks, ext4 noatime, performance governor, hypervisor flag absent), Ubuntu 24.04, kernel 7.0.0-1011-aws, rustc 1.96.0, region us-east-2c. Source commit a443c52e68a5c5c69cd62455c04ad313bc8d7dc3 (workspace version 3.0.0). Environment class 3. This is the third receipt on this host class in eight days (8aeb6ea, 2ff8a4b, a443c52); every table below carries all three so the effect of each commit reads against run-to-run noise rather than a single earlier number.

1. SQL point workload — 1M rows

Phase (p50 / p99)a443c522ff8a4b8aeb6ea (08-30)SQLiteDuckDB
Point SELECT prepared20.3 µs / 36.3 µs34.0 / 36.530.8 / 33.31.8 / 2.1191 / 235
UPDATE, fsync per commit1.75 ms / 2.43 ms1.85 / 2.561.82 / 2.5121.9 µs / 402 µs915 µs / 1.38 ms
UPDATE, batched ×10030.6 ms / 31.4 ms33.7 / 34.229.4 / 30.0724 µs / 4.34 ms23.7 / 30.8 ms
Load 1,000-row batches4.0/s3.9/s4.3/s963/s13.1/s

Read checksums matched across all three engines (200,000 hits each). The prepared point read is 40% faster than either earlier receipt. SQLite's point read is still 11× faster — published here at the same weight as every number Hyphae leads on.

2. Keyspace point workload — 1M keys

Phase (p50 / p99)a443c522ff8a4b8aeb6eaRedis UDS alwaysRedis UDS everysec
GET2.2 µs / 16.3 µs11.1 / 14.113.5 / 16.07.9 / 10.88.2 / 9.5
SET, fsync-per-write vs Strict1.81 ms / 2.53 ms1.82 / 2.501.82 / 2.50509 µs / 910 µs
SET, no-fsync-ack vs everysec371 µs / 404 µs357 / 376376 / 4149.6 µs / 11.5 µs

The embedded GET is 3.6× faster than Redis over its UDS round trip at p50 (2.2 µs against 7.9 µs); at p99 (16.3 µs) it is the slower one — the pool-miss tail, printed as-is. Strict and Memory SETs are unchanged: root publication plus two fsyncs, as before.

3. Lexical BM25 — 100k documents

Phase (p50 / p99)a443c522ff8a4b8aeb6eaTantivy
Query top-10111 µs / 181 µs255 / 8564.09 ms / 6.55 ms78.3 µs / 82.9 µs
Ingest 1,000-doc durable batch1.12 s / 1.73 s2.19 / 2.8916.6 / 17.927.0 ms / 45.7 ms

Hit totals matched (100,000 vs 100,000). Against the 2.2.0-era receipt the query is 37× faster and durable ingest 15× faster. Tantivy still leads by 1.4× on query latency (was 50×) and 41× on ingest (was 580×); the p99 gap closed from 10× to 2.2×.

4. Durability ablations and the delta-vs-materialized transaction

Hyphae-only, identical single-SET commits on the materialized path, 10,000 commits per phase:

Classa443c52 p502ff8a4b p508aeb6ea p50Receipt clocks, p50 (a443c52)
Strict4.39 ms35.5 ms4.98 msexecution 1.58 ms, wal_append 49 µs, page_sync 536 µs, wal_sync 589 µs
Memory3.86 ms34.2 ms3.62 msexecution 366 µs, wal_append 41 µs
Group, 8 producers12.4 ms40.4 ms16.1 ms497 commits/s vs strict 196/s (2.5×)

The regression published in the 2ff8a4b receipt is closed on the same host class: the materialized single-SET commit is back at the 2.2.0 level and slightly under it. The receipt clocks themselves were unchanged across all three runs — the roughly 31 ms that lived outside the clocks was the complete-state load reading a tree of one-entry leaves.

Transaction shape (1 INSERT + 1 SET + 1 indexed document, Memory durability, 2,000 commits), by engine composition:

Rowa443c522ff8a4b8aeb6ea
Materialized batch p50 / p9946.6 ms / 95.8 ms65.0 / 12755.1 / 101
Delta batch p50 / p991.02 ms / 1.09 ms1.10 / 1.161.13 / 1.26
SQL only200 µs218205
SQL + structure371 µs386444
SQL + structure + search1.02 ms1.071.13

The delta-staged transaction across all three engines is roughly 45× cheaper than the materialized path at this shape. Standalone group-commit smoke (256 commits, 8 producers, delta path): strict p50 1.54 ms at 587/s; group p50 3.14 ms at 2,263/s — 3.86× strict throughput.

5. Delta-transaction scaling sweep

CPU 0, median of three runs. Version depth (prior committed versions of the same keys):

Prior versionsTotal p50StageCommitPage readsAppendsFull-state loads
1194 µs32.4 µs161 µs930
32193 µs32.4 µs161 µs930
256202 µs38.8 µs163 µs930
1,024201 µs38.8 µs162 µs930

Unrelated items sharing the engine, per commit:

Unrelated items per engineTotal p50StageCommitPage readsAppends
0346 µs34.2 µs272 µs175
256676 µs33.6 µs577 µs3011
4,096893 µs51.1 µs764 µs4114

Version depth stays flat — 193 to 202 µs across three orders of magnitude of history. The population sweep is 16% and 15% cheaper than on 2ff8a4b at 256 and 4,096 items (805 → 676 µs, 1,045 → 893 µs), with fewer page appends per commit.

6. Document-cap ladder — 250k and 1M

Fresh loads on NVMe, ladders on reopened directories (p50, 16 samples, limit=10, candidate limit 1,000); the 1M rung ran with the collection bound lifted on this host only.

RungIngest docs/sIngest wallVacuumDirectoryReopenBM25Filtered+facetPhraseFuzzy(1)
250k a443c524,26558.6 s13.8 s236 MB7.7 s6.2 ms10.6 ms7.3 ms12.0 ms
250k 2ff8a4b4,06961.4 s14.0 s236 MB7.7 s6.2 ms10.7 ms7.2 ms12.0 ms
1M a443c523,638275 s64.3 s1.01 GB34.6 s23.2 ms42.6 ms24.2 ms54.2 ms
1M 2ff8a4b3,755266 s62.9 s1.01 GB34.5 s51.6 ms71.1 ms52.7 ms89.0 ms
Ratio 250k→1M, a443c520.85×4.5×3.7×4.0×3.3×4.5×

The 1M query ladder is now linear in document count (3.3–4.5× for 4× the documents, against 6.6–8.3× on 2ff8a4b); at 250k nothing moved, since the 1,024-frame pool already held a 250k query's segments. The first bm25 sample after a fresh load is still a cold-cache outlier (p95 17 s at 1M, 3.8 s at 250k), reported and not excluded. The collection bound stays 250,000; the 1M rows measure above the bound.

Buffer-pool bound, same host, same 1M directory:

HYPHAE_BUFFER_POOL_FRAMESDurable scorerIntegrated MatchAllRange + facetFuzzy(1)
1,024 (2.x default)50.8 / 51.2 ms52.0 / 51.9 ms78.4 ms90.6 ms
8,192 (3.0 default)22.4 / 22.4 ms23.6 / 23.4 ms49.7 ms56.4 ms

2.3× on the scorer from residency alone. The BLAKE3/CRC32C page-verification cost that was 44% of samples on 2ff8a4b is absent from the top 40 of the perf profile at 8,192 frames.

7. Scorer equivalence at 1M

One oracle round on the 1M directory of §6, after the ladder:

stage=durable_scorer round=0 hits=1000 ms=39.5 terms=2 segments=1562 physical_entries=372418
stage=model_scorer   round=0 hits=1000 ms=12711486.6
stage=scorer_equivalence hits=1000 bit_identical=true

The retained-model scorer reproduces the durable scorer's 1,000 ranked hits — ids, order, and score bits — at 1,000,000 documents, on a tree whose physical leaf layout the B+tree fix changed. The model took 3.5 hours; the durable scorer took 39.5 ms cold and 22 ms warm. This is the fourth rung (100k, 250k, 1M ×2) on which the two scorers agree bit for bit.

Scope and non-claims

  • Single host, one run per phase (three for the delta sweeps); same host class as the two earlier receipts but a different physical machine each time — deltas under ~10% are not claimed.
  • Shipped defaults throughout: 8,192-frame pool, no governor or execution pool (workers=1), --scale full baselines at their documented durable defaults.
  • The collection bound stays 250,000; the 1M rows measure above the bound and the R5 vector conditions remain unmeasured.
  • Synthetic corpora; no relevance claim.
  • No universal superiority claim. On this host Hyphae's point reads and lexical queries are within 1.4–11× of the specialized baselines and its embedded GET is faster than Redis over UDS at p50, while durable writes remain the honest cost of copy-on-write publication plus two fsyncs. The differentiator is unchanged: one CSN across three engines with a model-checked commit protocol and receipts for every number.

Full receipt: docs/gates/evidence/hyphae-3.0-metal-a443c52-2026-09-03.md. Raw harness JSON: sql, keyspace, lexical, ablation, group-commit.

Related receipts

Virtualized document-cap ladder — 250k rung (class 2)

DigitalOcean c-16 droplet (16 vCPU Xeon Platinum 8168, 32 GB, 200 GB local disk, Ubuntu 24.04), dedicated, release profile. Raised MAX_PRODUCT_SEARCH_COLLECTION_DOCUMENTS from 100,000 to 250,000. Ingest went from 1,261 docs/s at 100k to 949 docs/s at 250k (2.1 GB directory before maintenance); the query ladder scaled 1.9–3.2× for 2.5× the documents (bm25 16 → 39 ms, fuzzy 19 → 61 ms). The scorer-equivalence oracle matched bit-for-bit at both 100k and 250k. Source commits between eec0784 and the commit that raised the cap.

Source: docs/gates/evidence/collection-cap-250k-2026-09-02.md (2026-09-02).

Chunked manifest and the first 1M lexical ladder

Same DigitalOcean c-16 host class. The 250k receipt above named the collection manifest — one 4 MB value rewritten per batch — as the blocker for the next rung. The chunked HYPSMAN2 format plus lazy eligibility cut the per-batch write to a header plus the touched chunks (39,056-byte header at 1M, against 16 MB before) and moved bm25 from 39 ms to 24.2 ms at 250k. The first complete lexical/doc-value ladder at 1,000,000 documents measured bm25 at 171.9 ms and fuzzy at 308.2 ms (6–7× the 250k stage for 4× the documents). This receipt does not raise the collection bound past 250,000: the R5 vector conditions (ANN consolidation cost, RSS at 1M × 768-dim vectors) remain unmeasured because the ladder corpus carries no vectors.

Source: docs/gates/evidence/collection-manifest-chunked-1m-ladder-2026-09-03.md (2026-09-03).

Cross-host determinism

The identical BEIR FiQA-2018 harness (57,638 documents, 648 test queries) ran on two deliberately different machines — different distribution, kernel line, glibc, CPU generation, and Python interpreter. Both hosts reported ndcg@10 0.235959, recall@10 0.298234, mrr@10 0.293643, a transient directory peak of 55,562,927,163 bytes, and a final maintained directory of 564,961,635 bytes — identical to the byte. Hyphae's committed state is a pure function of ingested content and protocol constants on x86-64 Linux; this does not claim cross-architecture identity (aarch64 is unmeasured).

Source: docs/gates/evidence/rag-cross-host-determinism-2026-08-22.md (2026-08-22, engine 1.2.2).

Formal model checking (TLA+/TLC)

docs/formal/HyphaeCommit.tla models the cross-engine commit protocol: 3 transactions, 2 keys, 3 engines, up to 2 crashes. TLC2 2.19, 96 workers, run to exhaustion:

79,063,806 states generated, 43,885,299 distinct states, depth 30
Model checking completed. No error has been found.  (1min 41s)

All six invariants held over the complete bounded state space: TypeOk, Atomicity (no partial cross-engine commit in any reachable state, including every crash/recovery shape), StrictDurability (an acknowledged Strict commit survives every crash), FirstCommitterWins, VisiblePrefixComplete, CsnBounded. This is evidence about the protocol as specified, not a proof of the Rust implementation — implementation fidelity is carried separately by the physical crash matrices (tests/all_engine_transaction_g5.rs).

Source: docs/gates/evidence/baseline-i7i-metal-2026-08-30.md, §1 (first run, 2026-08-30); reproduced with matching state counts against the exact spec/config/tool digests in the 2026-09-03 re-measurement.

G8 release closure for 3.0.0

3.0.0 tags release-v3.0.0-crates at source commit 24bce1accdff8d14127797afe6f237a57c1cd4f3. The nine-requirement exact-SHA G8 closure ran as readiness run 33836088262, closure run 33836655173, and signed release run 33838703304; the retained aggregate native-g8-aggregate-24bce1ac… has SHA-256 41dacc41bde4420ec3f2d735828669231966dd53c545a2fdd7a6bf0691205ebf and declares closure_declared=true, status=passed across three required platforms. crates.io publication ran separately as run 33859889168, approved by a required reviewer.

Source: docs/release/receipts/3.0.0.md and docs/gates/evidence/closures/native-g8-24bce1a.json.

How to reproduce

The bare-metal suite is a standalone workspace, deliberately not a member of the product's root workspace, so SQLite, DuckDB, Redis, and Tantivy never enter the product dependency graph:

# on the metal host (Ubuntu 24.04, root), with the repo at /root/hyphae:
bash benchmarks/baseline-harness/scripts/run-metal.sh

The script formats the spare local NVMe instance-store disk, pins the performance governor, starts two UDS-only Redis servers (one per fsync policy), builds the harness in release mode, and writes one JSON receipt per suite. A quick local smoke without dedicated hardware or Redis:

cargo run --release --manifest-path benchmarks/baseline-harness/Cargo.toml -- \
  all /tmp/hyphae-bench-scratch /tmp/hyphae-bench.json --scale small

Raw JSON outputs are attached to the evidence documents verbatim — never post-processed by hand.

Source: benchmarks/baseline-harness/README.md.

How to read a receipt

  1. Find the environment class in the receipt's opening lines. Only class 3 supports a comparative statement.
  2. Check the exact source commit against the repository history — every receipt names one.
  3. Read the "Scope and non-claims" section before the numbers. It states what was not measured as plainly as what was.
  4. Follow the raw-receipt links for verbatim harness JSON; the narrative document never edits those numbers by hand.
  5. Where a receipt supersedes an earlier one, both stay in the repository — nothing is deleted, only superseded in place.

The complete evidence index, including every gate-closure and source-bound observation referenced above, is docs/gates/evidence/README.md.