Hermit compatibility scorecard
Current determinism, parity, and coverage evidence for Hermit's Linux execution backends.
1 test attempt from 1 validation run is not counted on this site. Each validation run keeps its own record of every test attempt it made, and this site publishes a history of those attempts. The site builder compares the two. These attempts are in their run's record but not in the published history; each line below says why. The green, red and never-measured cell results on this site come from the published history, so these attempts are not counted in them. Each validation run's page still shows the per-cell verdicts that run recorded.
- 1 test attempt from run validate-coord-84da7b816939-1790723396555428157-708227-fd736ab2 is not counted on this site because it is not in the published test history; the series writer refused 1 input from this run, the first with MalformedRow: cell attempt 31 matched report is internally inconsistent. Attempts: c-programs/fp-reduction-nondeterminism/chaos/ptrace attempt 1
The page of rows the site did not count, and why lists every such run.
Hermit makes Linux execution repeatable by controlling scheduling, time, and randomness. This scorecard compares its backends: determinism asks whether repeated runs agree, while parity asks whether a backend agrees with the reference ptrace backend. Each cell is one workload, execution mode, and backend; results distinguish passes, failures, and unmeasured cells. Latest selected measurement: ; latest history entry: . Totals may combine qualifying results from multiple runs; rebuilding this website does not rerun the tests. For context, start with the introduction to Hermit, or visit the README for setup and usage.
Overview
1,932 runnable cells out of 925 tests × 16 cells per test = 14,800 cells in the manifest. The 16 are 3 modes (chaos, replay, verify) × 5 backends (dbt, kvm, liteinst, ptrace, sabre) + 1 naked/native.
12,807 cells are never run by manifest policy: each carries a recorded not-applicable reason. 61 more are cells this site leaves out on purpose, with no cell row. 1,932 + 12,807 + 61 = 14,800.
The difference between cells in the manifest and rows with a cell page is retained as exact arithmetic. It is not assigned a cell result and has no cell row on this site.
How to read these. Green and red are counted over the 1,559 cells with a recorded result, not over the 14,800 in the manifest; quoting red against the manifest would understate the rate. None of the 74 red cells is selected by full in this snapshot. These retained failures fall outside this snapshot’s full-profile selection. Of the 14,800 cells in the manifest, 1,559 have a recorded result, 616 runnable cells have never been measured, 12,564 are never run by manifest policy and have no result and 61 are deliberate omissions with no cell row on this site. 243 of the 1,559 results are on cells that manifest policy no longer runs; they were recorded earlier. The manifest holds far more cells than any path selects, which is the intended arrangement and not a backlog.
83,053 admitted canonical series rows representing 89,902 Hermit runs. 167 admitted validation ledger rows. 181,559 excluded physical series rows representing 183,093 Hermit runs. 5 custom commands selected by full. The captured input contained 264,612 physical series rows representing 272,995 Hermit runs.
Determinism and Backend Parity
Determinism
Recorded outcomes and comparison evidence
KVM and LiteInst in this artifact. Recorded outcomes (kvm: 260 recorded Green, 1 recorded failures; liteinst: 338 recorded Green, 10 recorded failures) remain in Cells. These are positions in this artifact, not changes since an earlier snapshot and not counts of cross-backend matches. A Green outcome alone does not establish either comparison below.
Every determinism percentage uses the same distinct test-and-mode population with a recorded ptrace result, including ptrace failures and leaving out the stripped-comparison cells counted below. The w/partial column adds qualifying partial fractions to full matches. Missing and failed cells stay in the denominator. Parity is not part of this table: Parity (measured after determinism), below it, is the only parity display.
189 test-and-mode cells with a recorded ptrace result are outside every denominator: every recorded check of the ptrace cell used the stripped comparison its manifest declares (comparator: stripped: exit status, streams and stripped logs), which is never a bit-for-bit match.
How comparison evidence earns credit
Numerators count demonstrated comparisons, not generic Green labels. An absent or empty reference population is unavailable. Partial determinism uses the recorded first divergent scheduler turn and a complete same-commit ptrace execution; no missing measurement is inferred as partial. A column with no supported observations is unavailable.
| BackendRecorded instrumentation backend; native and e9patch preprocessing are excluded. | DeterminismCanonical repeat matches / all distinct test-and-mode cells with a recorded ptrace result, less those checked only with the stripped comparison. | Det. w/partialFull matches plus recorded qualifying partial fractions / the same common denominator. | Evidence coverageDeterminism failures and missing/unexecuted cells stay in the determinism denominator. Missing comparison evidence earns no credit. |
|---|---|---|---|
ptrace | 98.4%369 / 375 cells | 98.4%369 / 375 cells | 4 failed; 0 missing/unmeasured; 6 without determinism evidence; 2,205 in test-and-mode cells with no recorded ptrace result |
kvm | 68.5%257 / 375 cells | 68.5%257 / 375 cells | 1 failed; 116 missing/unmeasured; 118 without determinism evidence; 2,210 in test-and-mode cells with no recorded ptrace result |
sabre | 64.0%240 / 375 cells | 64.0%240 / 375 cells | 54 failed; 74 missing/unmeasured; 135 without determinism evidence; 2,208 in test-and-mode cells with no recorded ptrace result |
liteinst | 38.9%146 / 375 cells | 38.9%146 / 375 cells | 9 failed; 30 missing/unmeasured; 229 without determinism evidence; 2,205 in test-and-mode cells with no recorded ptrace result |
dbt | 6.9%26 / 375 cells | 6.9%26 / 375 cells | 0 failed; 296 missing/unmeasured; 349 without determinism evidence; 2,211 in test-and-mode cells with no recorded ptrace result |
Parity (measured after determinism)
After the determinism checks, the retained verify log of each candidate backend whose inputs can be equalized with the ptrace reference's (kvm, liteinst, sabre) is compared with the ptrace reference log. A cell's credit is the matched prefix of the two logs over the longer one, in [0, 1]; only a complete match earns 1. A cell with no deterministic golden log has no parity and is outside both credits.
validate run validate-ops-tick-8503b4fc2c30-f0c6c4842871
Data as of: last parity record emitted ; Hermit commit 8503b4fc2c30 measured; parity store at ledger commit 7763d6a31f6d (scorecard/parity.json, sha256 cb7abc76bcac); summary written by Hermit tool commit 8503b4fc2c30.
parity: 0/297 matched; selected 297 of 297 committed; mean 0.043 over 278 measured; floor 0.043 over 281 of 297 selected (counted as 0: 3 unmeasured: no-result-row 3; excluded: 0 no golden; 16 not compared)
| Candidate backend | Selected | Measured | Matched | Diverged | No parity (no deterministic golden) | Not compared | Unmeasured | Record-missing | Refused | Mean credit (over measured) | Floor credit (over floor cells) | Floor cells |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
kvm | 91 | 90 | 0 | 90 | 0 | 0 | 1 | 0 | 0 | 0.1020 | 0.1009 | 91 |
liteinst | 99 | 98 | 0 | 98 | 0 | 0 | 1 | 0 | 0 | 0.0142 | 0.0141 | 99 |
sabre | 91 | 90 | 0 | 90 | 0 | 0 | 1 | 0 | 0 | 0.0160 | 0.0158 | 91 |
dbt | 16 | 0 | 0 | 0 | 0 | 16 | 0 | 0 | 0 | n/a | n/a | 0 |
| TOTAL | 297 | 278 | 0 | 278 | 0 | 16 | 3 | 0 | 0 | 0.0432 | 0.0427 | 281 |
pressure-test run pressure-c470fa213ee8-20261003T180650Z
Data as of: last parity record emitted ; Hermit commit c470fa213ee8 measured; parity store at ledger commit 7763d6a31f6d (scorecard/parity.json, sha256 cb7abc76bcac); summary written by Hermit tool commit 8503b4fc2c30.
parity: not compared: 0 measured of 1 selected (inputs cannot be equalized); selected 1 of 205 committed (partial)
| Candidate backend | Selected | Measured | Matched | Diverged | No parity (no deterministic golden) | Not compared | Unmeasured | Record-missing | Refused | Mean credit (over measured) | Floor credit (over floor cells) | Floor cells |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
kvm | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | n/a | n/a | 0 |
liteinst | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | n/a | n/a | 0 |
sabre | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | n/a | n/a | 0 |
dbt | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | n/a | n/a | 0 |
| TOTAL | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | n/a | n/a | 0 |
173 historical cross-backend parity failure(s) from the retired ptrace rerun are not counted above. The parity details page describes them.
Parity details: how each credit is computed, the cells with no parity and the unmeasured cells by class, the most common first divergence of each backend, every cell that is not a full match, the other parity runs in the store and the retired ptrace rerun.
Backend history
Latest recorded state at each UTC date for the current catalogue and current main ancestry, within that date’s reference population (recorded ptrace results, less the stripped-comparison cells). Percentages and counts can fall after regressions. Steps carry recorded state; they are not interpolated measurements. Gaps mean no supported comparison evidence, including legacy periods. The lines are determinism; parity is shown only in Parity (measured after determinism).
Use the chart: Hover over or focus a point to see its exact date and percentage. Clear or select a checkbox under Visible lines to hide or show that series.
Ledger anchor: 7763d6a31f6d39bcd32777112b8f6efaec9d5959. Latest recorded date: 2026-10-04. Machine-readable history and source provenance.
Recorded values by date
| Date (UTC) | Backend | Determinism green | Determinism percentage | Ptrace reference |
|---|---|---|---|---|
| 2026-08-30 | dbt | Not available | Not available | 113 |
| 2026-08-30 | kvm | Not available | Not available | 113 |
| 2026-08-30 | liteinst | Not available | Not available | 113 |
| 2026-08-30 | ptrace | Not available | Not available | 113 |
| 2026-08-30 | sabre | Not available | Not available | 113 |
| 2026-08-31 | ptrace | 344 | 96.9% | 355 |
| 2026-08-31 | sabre | 111 | 31.3% | 355 |
| 2026-08-31 | liteinst | 28 | 7.9% | 355 |
| 2026-08-31 | kvm | 2 | 0.6% | 355 |
| 2026-08-31 | dbt | Not available | Not available | 355 |
| 2026-09-01 | ptrace | 344 | 96.9% | 355 |
| 2026-09-01 | sabre | 111 | 31.3% | 355 |
| 2026-09-01 | liteinst | 28 | 7.9% | 355 |
| 2026-09-01 | kvm | 2 | 0.6% | 355 |
| 2026-09-01 | dbt | Not available | Not available | 355 |
| 2026-09-02 | ptrace | 344 | 96.9% | 355 |
| 2026-09-02 | sabre | 111 | 31.3% | 355 |
| 2026-09-02 | liteinst | 28 | 7.9% | 355 |
| 2026-09-02 | kvm | 2 | 0.6% | 355 |
| 2026-09-02 | dbt | Not available | Not available | 355 |
| 2026-09-03 | ptrace | 344 | 96.9% | 355 |
| 2026-09-03 | sabre | 111 | 31.3% | 355 |
| 2026-09-03 | liteinst | 28 | 7.9% | 355 |
| 2026-09-03 | kvm | 2 | 0.6% | 355 |
| 2026-09-03 | dbt | Not available | Not available | 355 |
| 2026-09-04 | ptrace | 344 | 96.9% | 355 |
| 2026-09-04 | sabre | 111 | 31.3% | 355 |
| 2026-09-04 | liteinst | 28 | 7.9% | 355 |
| 2026-09-04 | kvm | 2 | 0.6% | 355 |
| 2026-09-04 | dbt | Not available | Not available | 355 |
| 2026-09-05 | ptrace | 344 | 96.9% | 355 |
| 2026-09-05 | kvm | 189 | 53.2% | 355 |
| 2026-09-05 | sabre | 111 | 31.3% | 355 |
| 2026-09-05 | liteinst | 28 | 7.9% | 355 |
| 2026-09-05 | dbt | Not available | Not available | 355 |
| 2026-09-06 | ptrace | 344 | 96.9% | 355 |
| 2026-09-06 | kvm | 189 | 53.2% | 355 |
| 2026-09-06 | sabre | 111 | 31.3% | 355 |
| 2026-09-06 | liteinst | 28 | 7.9% | 355 |
| 2026-09-06 | dbt | Not available | Not available | 355 |
| 2026-09-08 | ptrace | 343 | 96.6% | 355 |
| 2026-09-08 | kvm | 219 | 61.7% | 355 |
| 2026-09-08 | sabre | 111 | 31.3% | 355 |
| 2026-09-08 | liteinst | 28 | 7.9% | 355 |
| 2026-09-08 | dbt | Not available | Not available | 355 |
| 2026-09-09 | ptrace | 343 | 96.6% | 355 |
| 2026-09-09 | kvm | 219 | 61.7% | 355 |
| 2026-09-09 | sabre | 111 | 31.3% | 355 |
| 2026-09-09 | liteinst | 28 | 7.9% | 355 |
| 2026-09-09 | dbt | Not available | Not available | 355 |
| 2026-09-11 | ptrace | 344 | 96.9% | 355 |
| 2026-09-11 | kvm | 220 | 62.0% | 355 |
| 2026-09-11 | sabre | 111 | 31.3% | 355 |
| 2026-09-11 | liteinst | 28 | 7.9% | 355 |
| 2026-09-11 | dbt | Not available | Not available | 355 |
| 2026-09-12 | ptrace | 344 | 96.6% | 356 |
| 2026-09-12 | kvm | 220 | 61.8% | 356 |
| 2026-09-12 | sabre | 111 | 31.2% | 356 |
| 2026-09-12 | liteinst | 28 | 7.9% | 356 |
| 2026-09-12 | dbt | Not available | Not available | 356 |
| 2026-09-16 | ptrace | 344 | 96.6% | 356 |
| 2026-09-16 | kvm | 220 | 61.8% | 356 |
| 2026-09-16 | sabre | 111 | 31.2% | 356 |
| 2026-09-16 | liteinst | 28 | 7.9% | 356 |
| 2026-09-16 | dbt | Not available | Not available | 356 |
| 2026-09-22 | ptrace | 346 | 96.9% | 357 |
| 2026-09-22 | kvm | 241 | 67.5% | 357 |
| 2026-09-22 | liteinst | 146 | 40.9% | 357 |
| 2026-09-22 | sabre | 112 | 31.4% | 357 |
| 2026-09-22 | dbt | Not available | Not available | 357 |
| 2026-09-28 | ptrace | 353 | 98.3% | 359 |
| 2026-09-28 | kvm | 242 | 67.4% | 359 |
| 2026-09-28 | liteinst | 146 | 40.7% | 359 |
| 2026-09-28 | sabre | 112 | 31.2% | 359 |
| 2026-09-28 | dbt | Not available | Not available | 359 |
| 2026-09-29 | ptrace | 365 | 98.1% | 372 |
| 2026-09-29 | kvm | 242 | 65.1% | 372 |
| 2026-09-29 | liteinst | 146 | 39.2% | 372 |
| 2026-09-29 | sabre | 112 | 30.1% | 372 |
| 2026-09-29 | dbt | Not available | Not available | 372 |
| 2026-09-30 | ptrace | 366 | 98.7% | 371 |
| 2026-09-30 | kvm | 242 | 65.2% | 371 |
| 2026-09-30 | liteinst | 146 | 39.4% | 371 |
| 2026-09-30 | sabre | 112 | 30.2% | 371 |
| 2026-09-30 | dbt | Not available | Not available | 371 |
| 2026-10-02 | ptrace | 366 | 98.7% | 371 |
| 2026-10-02 | kvm | 242 | 65.2% | 371 |
| 2026-10-02 | liteinst | 146 | 39.4% | 371 |
| 2026-10-02 | sabre | 112 | 30.2% | 371 |
| 2026-10-02 | dbt | Not available | Not available | 371 |
| 2026-10-03 | ptrace | 369 | 98.7% | 374 |
| 2026-10-03 | kvm | 254 | 67.9% | 374 |
| 2026-10-03 | liteinst | 146 | 39.0% | 374 |
| 2026-10-03 | sabre | 112 | 29.9% | 374 |
| 2026-10-03 | dbt | 26 | 7.0% | 374 |
| 2026-10-04 | ptrace | 369 | 98.4% | 375 |
| 2026-10-04 | kvm | 257 | 68.5% | 375 |
| 2026-10-04 | sabre | 240 | 64.0% | 375 |
| 2026-10-04 | liteinst | 146 | 38.9% | 375 |
| 2026-10-04 | dbt | 26 | 6.9% | 375 |
Record/Replay
A replay cell records the program under ptrace (record start --strict --verify, in an isolated recording directory), then replays that recording; the harness compares the two runs. Green and red count the latest recorded result; determinism matched counts the cells whose repeated runs agreed. No replay cell is runnable on dbt, kvm, liteinst, sabre: manifest policy never runs them.
Build provenance
Website build started: . This is the website generation time, not the age of its measurements.
Sources used to build this website. Commit dates are Git committer dates.
| PurposeThe role this exact source revision played in producing the site data. | Source commitA short hash linked to the complete immutable Git commit. | Commit dateThe commit object’s committer timestamp; not a measurement or website build date. |
|---|---|---|
| Hermit catalogue | 32053b6d4645 | |
| Hermit reference revision | 13437c69bd97 | |
| Website data generator (private repository) | 7648ffd3d5c4 | |
| Cell result ledger | 7763d6a31f6d |
Measured Hermit revisions and commit depth
Each row identifies a revision used by the selected cell results. Its measurement date is the latest selected result at that revision; its commit date describes the source, not a test run.
Measured revisions, newest measurement first.
| Hermit commitThe exact Hermit revision used for these selected test results. | Commit dateThe measured commit object’s committer timestamp; not a test execution date. | Latest measurementThe latest selected cell measurement at this exact Hermit revision. | First-parent depthRecorded commits along this Hermit revision’s first-parent chain, including itself. | All reachable commitsAll recorded Hermit commits reachable from this revision, including merged side branches. |
|---|---|---|---|---|
8503b4fc2c30 | 3,690 | 3,765 | ||
f0f2808c9a0c | 3,648 | 3,723 | ||
c3f7b2bcfa00 | 3,551 | 3,626 | ||
ba7b784903be | 3,258 | 3,333 | ||
05dcd35a78b3 | 2,647 | 2,722 | ||
c7520a6c3873 | 2,646 | 2,721 | ||
a58e9ecfb037 | 2,594 | 2,669 | ||
e12c472d22d2 | 2,496 | 2,571 |
First-parent depth counts commits along the revision’s first-parent chain, including the revision itself. All reachable commits also counts merged side branches. These are recorded Hermit history counts, not elapsed time or a claim that a revision is that many commits behind main. A distance requires a verified ancestry relationship. Missing or conflicting depths are not inferred.
Full provenance
- Hermit commit used for manifest policy
32053b6d46457b44d009ea8278fa94fe0970de6c- Hermit commit used for ancestry classification
13437c69bd971799f6938b056c254c66df3a2fa2- dev-hermit source commit (private repository)
7648ffd3d5c4529dcf9d752b650f32ad26cfc066- Manifest digest
7b569d5a716dcae37b1f6525c9cece6d5b764009f37d9ed02748c4823b12708d- Captured series source digest
809276d5d8c66a7efc6832c01f7f14e23f61581f61f92c546ca6aa6ea0427bb8- Digest of the series rows this site shows
336a9e4261ea72ee578318edb440779b8a6fd98e60b97815663eb8836f327539