Hermit compatibility scorecard
Current determinism, parity, and coverage evidence for Hermit's Linux execution backends.
1 test attempt from 1 validation run is not counted on this site. Each validation run keeps its own record of every test attempt it made, and this site publishes a history of those attempts. The site builder compares the two. These attempts are in their run's record but not in the published history; each line below says why. The green, red and never-measured cell results on this site come from the published history, so these attempts are not counted in them. Each validation run's page still shows the per-cell verdicts that run recorded.
- 1 test attempt from run validate-coord-84da7b816939-1790723396555428157-708227-fd736ab2 is not counted on this site because it is not in the published test history; the series writer refused 1 input from this run, the first with MalformedRow: cell attempt 31 matched report is internally inconsistent. Attempts: c-programs/fp-reduction-nondeterminism/chaos/ptrace attempt 1
The page of rows the site did not count, and why lists every such run.
Hermit makes Linux execution repeatable by controlling scheduling, time, and randomness. This scorecard compares its backends: determinism asks whether repeated runs agree, while parity asks whether a backend agrees with the reference ptrace backend. Each cell is one workload, execution mode, and backend; results distinguish passes, failures, and unmeasured cells. Latest selected measurement: ; latest history entry: . Totals may combine qualifying results from multiple runs; rebuilding this website does not rerun the tests. For context, start with the introduction to Hermit, or visit the README for setup and usage.
Overview
1,930 runnable cells out of 925 tests × 16 cells per test = 14,800 cells in the manifest. The 16 are 3 modes (chaos, replay, verify) × 5 backends (dbt, kvm, liteinst, ptrace, sabre) + 1 naked/native.
12,807 cells are never run by manifest policy: each carries a recorded not-applicable reason. 63 more are cells this site leaves out on purpose, with no cell row. 1,930 + 12,807 + 63 = 14,800.
The difference between cells in the manifest and rows with a cell page is retained as exact arithmetic. It is not assigned a cell result and has no cell row on this site.
How to read these. Green and red are counted over the 1,557 cells with a recorded result, not over the 14,800 in the manifest; quoting red against the manifest would understate the rate. None of the 73 red cells is selected by full in this snapshot. These retained failures fall outside this snapshot’s full-profile selection. Of the 14,800 cells in the manifest, 1,557 have a recorded result, 616 runnable cells have never been measured, 12,564 are never run by manifest policy and have no result and 63 are deliberate omissions with no cell row on this site. 243 of the 1,557 results are on cells that manifest policy no longer runs; they were recorded earlier. The manifest holds far more cells than any path selects, which is the intended arrangement and not a backlog.
86,160 admitted canonical series rows representing 93,009 Hermit runs. 170 admitted validation ledger rows. 189,497 excluded physical series rows representing 191,031 Hermit runs. 5 custom commands selected by full. The captured input contained 275,657 physical series rows representing 284,040 Hermit runs.
Determinism and Backend Parity
Determinism
Recorded outcomes and comparison evidence
KVM and LiteInst in this artifact. Recorded outcomes (kvm: 260 recorded Green, 1 recorded failures; liteinst: 338 recorded Green, 10 recorded failures) remain in Cells. These are positions in this artifact, not changes since an earlier snapshot and not counts of cross-backend matches. A Green outcome alone does not establish either comparison below.
Every determinism percentage uses the same distinct test-and-mode population with a recorded ptrace result, including ptrace failures and leaving out the stripped-comparison cells counted below. The w/partial column adds qualifying partial fractions to full matches. Missing and failed cells stay in the denominator. Parity is not part of this table: Parity (measured after determinism), below it, is the only parity display.
189 test-and-mode cells with a recorded ptrace result are outside every denominator: every recorded check of the ptrace cell used the stripped comparison its manifest declares (comparator: stripped: exit status, streams and stripped logs), which is never a bit-for-bit match.
How comparison evidence earns credit
Numerators count demonstrated comparisons, not generic Green labels. An absent or empty reference population is unavailable. Partial determinism uses the recorded first divergent scheduler turn and a complete same-commit ptrace execution; no missing measurement is inferred as partial. A column with no supported observations is unavailable.
| BackendRecorded instrumentation backend; native and e9patch preprocessing are excluded. | DeterminismCanonical repeat matches / all distinct test-and-mode cells with a recorded ptrace result, less those checked only with the stripped comparison. | Det. w/partialFull matches plus recorded qualifying partial fractions / the same common denominator. | Evidence coverageDeterminism failures and missing/unexecuted cells stay in the determinism denominator. Missing comparison evidence earns no credit. |
|---|---|---|---|
ptrace | 98.7%370 / 375 cells | 98.7%370 / 375 cells | 3 failed; 0 missing/unmeasured; 5 without determinism evidence; 2,205 in test-and-mode cells with no recorded ptrace result |
kvm | 68.5%257 / 375 cells | 68.5%257 / 375 cells | 1 failed; 116 missing/unmeasured; 118 without determinism evidence; 2,210 in test-and-mode cells with no recorded ptrace result |
sabre | 64.0%240 / 375 cells | 64.0%240 / 375 cells | 54 failed; 74 missing/unmeasured; 135 without determinism evidence; 2,208 in test-and-mode cells with no recorded ptrace result |
liteinst | 38.9%146 / 375 cells | 38.9%146 / 375 cells | 9 failed; 30 missing/unmeasured; 229 without determinism evidence; 2,205 in test-and-mode cells with no recorded ptrace result |
dbt | 6.4%24 / 375 cells | 6.4%24 / 375 cells | 0 failed; 298 missing/unmeasured; 351 without determinism evidence; 2,211 in test-and-mode cells with no recorded ptrace result |
Parity (measured after determinism)
After the determinism checks, the retained verify log of each candidate backend whose inputs can be equalized with the ptrace reference's (kvm, liteinst, sabre) is compared with the ptrace reference log. A cell's credit is the matched prefix of the two logs over the longer one, in [0, 1]; only a complete match earns 1. A cell with no deterministic golden log has no parity and is outside both credits.
validate run validate-ops-tick-9af999677494-4ea4d2816eaf
Data as of: last parity record emitted ; Hermit commit 9af999677494 measured; parity store at ledger commit 051dde73e48f (scorecard/parity.json, sha256 39fdb02ad300); summary written by Hermit tool commit 9af999677494.
parity: 0/297 matched; selected 297 of 297 committed; mean 0.043 over 278 measured; floor 0.043 over 281 of 297 selected (counted as 0: 3 unmeasured: no-result-row 3; excluded: 0 no golden; 16 not compared)
| Candidate backend | Selected | Measured | Matched | Diverged | No parity (no deterministic golden) | Not compared | Unmeasured | Record-missing | Refused | Mean credit (over measured) | Floor credit (over floor cells) | Floor cells |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
kvm | 91 | 90 | 0 | 90 | 0 | 0 | 1 | 0 | 0 | 0.1020 | 0.1009 | 91 |
liteinst | 99 | 98 | 0 | 98 | 0 | 0 | 1 | 0 | 0 | 0.0142 | 0.0141 | 99 |
sabre | 91 | 90 | 0 | 90 | 0 | 0 | 1 | 0 | 0 | 0.0160 | 0.0158 | 91 |
dbt | 16 | 0 | 0 | 0 | 0 | 16 | 0 | 0 | 0 | n/a | n/a | 0 |
| TOTAL | 297 | 278 | 0 | 278 | 0 | 16 | 3 | 0 | 0 | 0.0432 | 0.0427 | 281 |
pressure-test run pressure-c470fa213ee8-20261003T180650Z
Data as of: last parity record emitted ; Hermit commit c470fa213ee8 measured; parity store at ledger commit 051dde73e48f (scorecard/parity.json, sha256 39fdb02ad300); summary written by Hermit tool commit 9af999677494.
parity: not compared: 0 measured of 1 selected (inputs cannot be equalized); selected 1 of 205 committed (partial)
| Candidate backend | Selected | Measured | Matched | Diverged | No parity (no deterministic golden) | Not compared | Unmeasured | Record-missing | Refused | Mean credit (over measured) | Floor credit (over floor cells) | Floor cells |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
kvm | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | n/a | n/a | 0 |
liteinst | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | n/a | n/a | 0 |
sabre | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | n/a | n/a | 0 |
dbt | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | n/a | n/a | 0 |
| TOTAL | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | n/a | n/a | 0 |
173 historical cross-backend parity failure(s) from the retired ptrace rerun are not counted above. The parity details page describes them.
Parity details: how each credit is computed, the cells with no parity and the unmeasured cells by class, the most common first divergence of each backend, every cell that is not a full match, the other parity runs in the store and the retired ptrace rerun.
Backend history
Latest recorded state at each UTC date for the current catalogue and current main ancestry, within that date’s reference population (recorded ptrace results, less the stripped-comparison cells). Percentages and counts can fall after regressions. Steps carry recorded state; they are not interpolated measurements. Gaps mean no supported comparison evidence, including legacy periods. The lines are determinism; parity is shown only in Parity (measured after determinism).
Use the chart: Hover over or focus a point to see its exact date and percentage. Clear or select a checkbox under Visible lines to hide or show that series.
Ledger anchor: 051dde73e48f6489eb242cdf3413a2600a71a66a. Latest recorded date: 2026-10-04. Machine-readable history and source provenance.
Recorded values by date
| Date (UTC) | Backend | Determinism green | Determinism percentage | Ptrace reference |
|---|---|---|---|---|
| 2026-08-30 | dbt | Not available | Not available | 113 |
| 2026-08-30 | kvm | Not available | Not available | 113 |
| 2026-08-30 | liteinst | Not available | Not available | 113 |
| 2026-08-30 | ptrace | Not available | Not available | 113 |
| 2026-08-30 | sabre | Not available | Not available | 113 |
| 2026-08-31 | ptrace | 344 | 96.9% | 355 |
| 2026-08-31 | sabre | 111 | 31.3% | 355 |
| 2026-08-31 | liteinst | 28 | 7.9% | 355 |
| 2026-08-31 | kvm | 2 | 0.6% | 355 |
| 2026-08-31 | dbt | Not available | Not available | 355 |
| 2026-09-01 | ptrace | 344 | 96.9% | 355 |
| 2026-09-01 | sabre | 111 | 31.3% | 355 |
| 2026-09-01 | liteinst | 28 | 7.9% | 355 |
| 2026-09-01 | kvm | 2 | 0.6% | 355 |
| 2026-09-01 | dbt | Not available | Not available | 355 |
| 2026-09-02 | ptrace | 344 | 96.9% | 355 |
| 2026-09-02 | sabre | 111 | 31.3% | 355 |
| 2026-09-02 | liteinst | 28 | 7.9% | 355 |
| 2026-09-02 | kvm | 2 | 0.6% | 355 |
| 2026-09-02 | dbt | Not available | Not available | 355 |
| 2026-09-03 | ptrace | 344 | 96.9% | 355 |
| 2026-09-03 | sabre | 111 | 31.3% | 355 |
| 2026-09-03 | liteinst | 28 | 7.9% | 355 |
| 2026-09-03 | kvm | 2 | 0.6% | 355 |
| 2026-09-03 | dbt | Not available | Not available | 355 |
| 2026-09-04 | ptrace | 344 | 96.9% | 355 |
| 2026-09-04 | sabre | 111 | 31.3% | 355 |
| 2026-09-04 | liteinst | 28 | 7.9% | 355 |
| 2026-09-04 | kvm | 2 | 0.6% | 355 |
| 2026-09-04 | dbt | Not available | Not available | 355 |
| 2026-09-05 | ptrace | 344 | 96.9% | 355 |
| 2026-09-05 | kvm | 189 | 53.2% | 355 |
| 2026-09-05 | sabre | 111 | 31.3% | 355 |
| 2026-09-05 | liteinst | 28 | 7.9% | 355 |
| 2026-09-05 | dbt | Not available | Not available | 355 |
| 2026-09-06 | ptrace | 344 | 96.9% | 355 |
| 2026-09-06 | kvm | 189 | 53.2% | 355 |
| 2026-09-06 | sabre | 111 | 31.3% | 355 |
| 2026-09-06 | liteinst | 28 | 7.9% | 355 |
| 2026-09-06 | dbt | Not available | Not available | 355 |
| 2026-09-08 | ptrace | 343 | 96.6% | 355 |
| 2026-09-08 | kvm | 219 | 61.7% | 355 |
| 2026-09-08 | sabre | 111 | 31.3% | 355 |
| 2026-09-08 | liteinst | 28 | 7.9% | 355 |
| 2026-09-08 | dbt | Not available | Not available | 355 |
| 2026-09-09 | ptrace | 343 | 96.6% | 355 |
| 2026-09-09 | kvm | 219 | 61.7% | 355 |
| 2026-09-09 | sabre | 111 | 31.3% | 355 |
| 2026-09-09 | liteinst | 28 | 7.9% | 355 |
| 2026-09-09 | dbt | Not available | Not available | 355 |
| 2026-09-11 | ptrace | 344 | 96.9% | 355 |
| 2026-09-11 | kvm | 220 | 62.0% | 355 |
| 2026-09-11 | sabre | 111 | 31.3% | 355 |
| 2026-09-11 | liteinst | 28 | 7.9% | 355 |
| 2026-09-11 | dbt | Not available | Not available | 355 |
| 2026-09-12 | ptrace | 344 | 96.6% | 356 |
| 2026-09-12 | kvm | 220 | 61.8% | 356 |
| 2026-09-12 | sabre | 111 | 31.2% | 356 |
| 2026-09-12 | liteinst | 28 | 7.9% | 356 |
| 2026-09-12 | dbt | Not available | Not available | 356 |
| 2026-09-16 | ptrace | 344 | 96.6% | 356 |
| 2026-09-16 | kvm | 220 | 61.8% | 356 |
| 2026-09-16 | sabre | 111 | 31.2% | 356 |
| 2026-09-16 | liteinst | 28 | 7.9% | 356 |
| 2026-09-16 | dbt | Not available | Not available | 356 |
| 2026-09-22 | ptrace | 346 | 96.9% | 357 |
| 2026-09-22 | kvm | 241 | 67.5% | 357 |
| 2026-09-22 | liteinst | 146 | 40.9% | 357 |
| 2026-09-22 | sabre | 112 | 31.4% | 357 |
| 2026-09-22 | dbt | Not available | Not available | 357 |
| 2026-09-28 | ptrace | 353 | 98.3% | 359 |
| 2026-09-28 | kvm | 242 | 67.4% | 359 |
| 2026-09-28 | liteinst | 146 | 40.7% | 359 |
| 2026-09-28 | sabre | 112 | 31.2% | 359 |
| 2026-09-28 | dbt | Not available | Not available | 359 |
| 2026-09-29 | ptrace | 365 | 98.1% | 372 |
| 2026-09-29 | kvm | 242 | 65.1% | 372 |
| 2026-09-29 | liteinst | 146 | 39.2% | 372 |
| 2026-09-29 | sabre | 112 | 30.1% | 372 |
| 2026-09-29 | dbt | Not available | Not available | 372 |
| 2026-09-30 | ptrace | 366 | 98.7% | 371 |
| 2026-09-30 | kvm | 242 | 65.2% | 371 |
| 2026-09-30 | liteinst | 146 | 39.4% | 371 |
| 2026-09-30 | sabre | 112 | 30.2% | 371 |
| 2026-09-30 | dbt | Not available | Not available | 371 |
| 2026-10-01 | ptrace | 366 | 98.7% | 371 |
| 2026-10-01 | kvm | 242 | 65.2% | 371 |
| 2026-10-01 | liteinst | 146 | 39.4% | 371 |
| 2026-10-01 | sabre | 112 | 30.2% | 371 |
| 2026-10-01 | dbt | Not available | Not available | 371 |
| 2026-10-02 | ptrace | 366 | 98.7% | 371 |
| 2026-10-02 | kvm | 242 | 65.2% | 371 |
| 2026-10-02 | liteinst | 146 | 39.4% | 371 |
| 2026-10-02 | sabre | 112 | 30.2% | 371 |
| 2026-10-02 | dbt | Not available | Not available | 371 |
| 2026-10-03 | ptrace | 369 | 98.7% | 374 |
| 2026-10-03 | kvm | 254 | 67.9% | 374 |
| 2026-10-03 | liteinst | 146 | 39.0% | 374 |
| 2026-10-03 | sabre | 112 | 29.9% | 374 |
| 2026-10-03 | dbt | 24 | 6.4% | 374 |
| 2026-10-04 | ptrace | 370 | 98.7% | 375 |
| 2026-10-04 | kvm | 257 | 68.5% | 375 |
| 2026-10-04 | sabre | 240 | 64.0% | 375 |
| 2026-10-04 | liteinst | 146 | 38.9% | 375 |
| 2026-10-04 | dbt | 24 | 6.4% | 375 |
Record/Replay
A replay cell records the program under ptrace (record start --strict --verify, in an isolated recording directory), then replays that recording; the harness compares the two runs. Green and red count the latest recorded result; determinism matched counts the cells whose repeated runs agreed. No replay cell is runnable on dbt, kvm, liteinst, sabre: manifest policy never runs them.
Build provenance
Website build started: . This is the website generation time, not the age of its measurements.
Sources used to build this website. Commit dates are Git committer dates.
| PurposeThe role this exact source revision played in producing the site data. | Source commitA short hash linked to the complete immutable Git commit. | Commit dateThe commit object’s committer timestamp; not a measurement or website build date. |
|---|---|---|
| Hermit catalogue | 9af999677494 | |
| Hermit reference revision | 238e148313e1 | |
| Website data generator (private repository) | 072d84b209ce | |
| Cell result ledger | 051dde73e48f |
Measured Hermit revisions and commit depth
Each row identifies a revision used by the selected cell results. Its measurement date is the latest selected result at that revision; its commit date describes the source, not a test run.
Measured revisions, newest measurement first.
| Hermit commitThe exact Hermit revision used for these selected test results. | Commit dateThe measured commit object’s committer timestamp; not a test execution date. | Latest measurementThe latest selected cell measurement at this exact Hermit revision. | First-parent depthRecorded commits along this Hermit revision’s first-parent chain, including itself. | All reachable commitsAll recorded Hermit commits reachable from this revision, including merged side branches. |
|---|---|---|---|---|
9af999677494 | 3,751 | 3,826 | ||
8503b4fc2c30 | 3,690 | 3,765 | ||
f0f2808c9a0c | 3,648 | 3,723 | ||
c3f7b2bcfa00 | 3,551 | 3,626 | ||
ba7b784903be | 3,258 | 3,333 | ||
05dcd35a78b3 | 2,647 | 2,722 | ||
c7520a6c3873 | 2,646 | 2,721 | ||
a58e9ecfb037 | 2,594 | 2,669 | ||
e12c472d22d2 | 2,496 | 2,571 |
First-parent depth counts commits along the revision’s first-parent chain, including the revision itself. All reachable commits also counts merged side branches. These are recorded Hermit history counts, not elapsed time or a claim that a revision is that many commits behind main. A distance requires a verified ancestry relationship. Missing or conflicting depths are not inferred.
Full provenance
- Hermit commit used for manifest policy
9af999677494fbb189d7dc4cc2e139cebba1f9f8- Hermit commit used for ancestry classification
238e148313e152d501669b13e0cfacdd669ea609- dev-hermit source commit (private repository)
072d84b209ce4a425b2f091d7499ad6d0b33012a- Manifest digest
210ed98c8629032b7f05c5a9d74f03e673d149cdfbddf74af5768713e831ce52- Captured series source digest
8c50f41858015b4624f91e08cae75cd82bf33eb323ba9fec76e0218db5eeab19- Digest of the series rows this site shows
def4ee88b711438ca4b7393778425326880d555a4f3a56e08b1d9386686c245d