Conformance
Cruft runs the full test262 suite: every section, zero skips. Cruft v0.0.10 passes 53,519 / 53,578 against corpus 3655e7464d (measured 2026-08-13), and every remaining failure is individually classified: none is an unresolved spec-conformance gap.
These classifications are a snapshot, not a fixed score. Which tests pass, fail, or time out depends on the test harness, the test262 corpus revision, the per-test time budget, and the machine. A different runner or a newer corpus can move tests between categories. Reproduce or re-measure it yourself with cruft-verify.
What test262 covers
test262 is the official ECMAScript conformance suite. It is organized into five areas, all of which Cruft runs:
| Section | What it verifies |
|---|---|
| Language | Core grammar, scoping, and execution semantics: expressions, statements, functions, classes, generators, and modules. |
| Built-ins | The standard library: global objects, prototypes, and their methods (Object, Array, RegExp, Promise, TypedArray, Atomics, Temporal, and the rest). |
| Intl (intl402) | ECMA-402 internationalization: locales, number and date formatting, collation, and calendars. |
| Staging | TC39 proposals still in flight, ahead of standardization. |
| Annex B | Legacy web-compatibility features the specification requires of browser-facing engines. |
Zero spec-conformance gaps
Every residual failure is classified; none is an unresolved spec gap:
| Count | Class | What these are |
|---|---|---|
| 36 | Deliberate deviation | Legacy Function.prototype.caller / arguments on strict-mode functions. Cruft ships the spec-pure %ThrowTypeError% poison pill by default; CRUFT_DEVIATION_LEGACY_CALLER_ARGUMENTS=1 flips all 36 to passing. |
| 22 | Wall-clock budget | SpiderMonkey stress tests that exceed the runner's 10-second per-test cap; each passes given more time, so they measure interpreter dispatch cost, not a spec gap. Each is timed below. |
| 1 | Deliberate deviation | Intl.supportedValuesOf("calendar") enumerates orthodox, a calendar Cruft ships by design; one proposal test asserts the exact standard set and so rejects the extra member. Shown below. |
The 36 legacy caller and arguments tests
Every test in the deviation row above, grouped by directory under tc39/test262 at 3655e7464d with each path shown relative to its group. All 36 flip to passing under CRUFT_DEVIATION_LEGACY_CALLER_ARGUMENTS=1.
| # | Test |
|---|---|
| built-ins/Function (21) | |
| 1 | 15.3.5.4_2-12gs.js |
| 2 | 15.3.5.4_2-14gs.js |
| 3 | 15.3.5.4_2-75gs.js |
| 4 | 15.3.5.4_2-76gs.js |
| 5 | 15.3.5.4_2-77gs.js |
| 6 | 15.3.5.4_2-78gs.js |
| 7 | 15.3.5.4_2-79gs.js |
| 8 | 15.3.5.4_2-80gs.js |
| 9 | 15.3.5.4_2-81gs.js |
| 10 | 15.3.5.4_2-82gs.js |
| 11 | 15.3.5.4_2-83gs.js |
| 12 | 15.3.5.4_2-84gs.js |
| 13 | 15.3.5.4_2-85gs.js |
| 14 | 15.3.5.4_2-86gs.js |
| 15 | 15.3.5.4_2-87gs.js |
| 16 | 15.3.5.4_2-88gs.js |
| 17 | 15.3.5.4_2-89gs.js |
| 18 | 15.3.5.4_2-90gs.js |
| 19 | 15.3.5.4_2-91gs.js |
| 20 | 15.3.5.4_2-92gs.js |
| 21 | 15.3.5.4_2-93gs.js |
| language/arguments-object (2) | |
| 22 | 10.6-13-a-2.js |
| 23 | 10.6-13-a-3.js |
| staging/sm (13) | |
| 24 | Function/function-caller-restrictions.js |
| 25 | extensions/arguments-property-access-in-function.js |
| 26 | extensions/censor-strict-caller.js |
| 27 | extensions/function-caller-skips-eval-frames.js |
| 28 | extensions/function-properties.js |
| 29 | regress/regress-577648-1.js |
| 30 | regress/regress-577648-2.js |
| 31 | regress/regress-584355.js |
| 32 | regress/regress-586482-1.js |
| 33 | regress/regress-586482-2.js |
| 34 | regress/regress-586482-3.js |
| 35 | regress/regress-586482-4.js |
| 36 | regress/regress-586482-5.js |
The 22 wall-clock-budget tests and their times
Run in isolation (one test, no sweep cap), all 22 pass and return the correct result. They land in the budget bucket only under the full parallel sweep, where each must finish inside a 10-second per-test slot. The times below were measured on an Apple M1 Pro (16 GB) against tc39/test262 at 3655e7464d, and are machine-dependent rather than a fixed property of the tests.
| Test | Result in isolation | Wall-clock |
|---|---|---|
test/staging/sm/Date/dst-offset-caching-8-of-8.js | PASS | 116.2s |
test/staging/sm/Date/dst-offset-caching-6-of-8.js | PASS | 116.1s |
test/staging/sm/Date/dst-offset-caching-5-of-8.js | PASS | 115.5s |
test/staging/sm/Date/dst-offset-caching-4-of-8.js | PASS | 115.2s |
test/staging/sm/Date/dst-offset-caching-3-of-8.js | PASS | 114.7s |
test/staging/sm/Date/dst-offset-caching-7-of-8.js | PASS | 114.5s |
test/staging/sm/Date/dst-offset-caching-2-of-8.js | PASS | 114.1s |
test/staging/sm/Date/dst-offset-caching-1-of-8.js | PASS | 91.3s |
test/staging/sm/TypedArray/sort_large_countingsort.js | PASS | 64.8s |
test/built-ins/decodeURIComponent/S15.1.3.2_A2.5_T1.js | PASS | 34.1s |
test/built-ins/decodeURI/S15.1.3.1_A2.5_T1.js | PASS | 33.9s |
test/staging/sm/regress/regress-610026.js | PASS | 27.5s |
test/staging/sm/Array/toSpliced-dense.js | PASS | 24.9s |
test/language/literals/regexp/S7.8.5_A1.4_T2.js | PASS | 23.5s |
test/language/literals/regexp/S7.8.5_A2.4_T2.js | PASS | 22.4s |
test/language/literals/regexp/S7.8.5_A2.1_T2.js | PASS | 21.9s |
test/annexB/built-ins/RegExp/RegExp-leading-escape-BMP.js | PASS | 21.1s |
test/annexB/built-ins/RegExp/RegExp-trailing-escape-BMP.js | PASS | 20.2s |
test/language/literals/regexp/S7.8.5_A1.1_T2.js | PASS | 20.1s |
test/staging/sm/expressions/nullish-coalescing.js | PASS | 18.5s |
Two of them complete in milliseconds standalone and reach the budget bucket only under parallel-sweep contention, not because the work is slow:
| Test | Result in isolation | Wall-clock |
|---|---|---|
test/built-ins/Atomics/wait/bigint/waiterlist-order-of-operations-is-fifo.js | PASS | 0.35s |
test/built-ins/RegExp/S15.10.2.8_A3_T17.js | PASS | 0.03s |
The 1 calendar Cruft adds to Temporal
Cruft ships the orthodox calendar (the Orthodox Paschalion) in its Temporal calendar set, with a full Intl.DisplayNames entry, so Intl.supportedValuesOf("calendar") returns it alongside the standard members. One proposal test checks that list against the exact standard set and rejects anything extra. This is Cruft offering more than the specification requires, not failing to meet it.
| Test | test/intl402/Intl/supportedValuesOf/calendars-required-by-intl-era-monthcode.js |
|---|---|
| The test expects | the exact standard set of 16 calendars |
| Cruft returns | those 16 plus one: orthodox |
The full list Intl.supportedValuesOf("calendar") returns in Cruft:
| Calendar | In the standard set |
|---|---|
buddhist | Yes |
chinese | Yes |
coptic | Yes |
dangi | Yes |
ethioaa | Yes |
ethiopic | Yes |
gregory | Yes |
hebrew | Yes |
indian | Yes |
islamic-civil | Yes |
islamic-tbla | Yes |
islamic-umalqura | Yes |
iso8601 | Yes |
japanese | Yes |
orthodox | Cruft addition |
persian | Yes |
roc | Yes |