Performance and tuning

The practical guide to making a Cruft program fast and measuring whether it is. Covers the two execution tiers (Exegesis and LeJIT), the GC tuning environment variables, worker-pool memory scaling, and how to get reliable startup and throughput numbers.

This is the practical guide to making a Cruft program fast and, more importantly, to measuring whether it is. It is adopter-facing: knobs, rules of thumb, and how to get reliable numbers. For the engine internals behind these knobs, follow the links to LeJIT and the garbage collector.

One framing up front: Cruft is a young runtime. It has a real baseline JIT and a memory model that scales well for worker-heavy workloads, but it is not yet tuned to V8-parity throughput across every workload. The right posture is to measure your own program, not to assume a speed class.

The execution tiers

Cruft runs your code in two tiers:

1. Exegesis executes bytecode directly. Everything starts here, and most code that runs only a few times stays here.
2. LeJIT compiles hot code to native machine code via Cranelift. A function that is called many times, or a loop that iterates many times, becomes a candidate for compilation; a loop that heats up mid-flight can be entered as compiled code without waiting for the next call (on-stack replacement).

You do not opt code into the JIT. Hotness is detected automatically and the transition is transparent. The guarantee the whole design rests on is that correctness never changes between tiers, Exegesis is the semantic ground truth, and compiled code either proves it can produce the same result or exits back to Exegesis (deoptimization). The JIT changes only speed; behavior is identical across tiers. See LeJIT for how admission, speculation, and deopt actually work.

The practical consequence: short-lived scripts spend their time in the Exegesis and in startup, while long-running loops and hot server handlers are where the JIT earns its keep. If your workload is startup-dominated, profile startup (below) rather than reaching for throughput knobs.

GC tuning: throughput versus memory

Cruft's collector sizes its next collection at roughly HEADROOM × the live object count, so steady-state memory tracks your working set rather than total allocations. Two environment variables tune the trade-off; a third is a diagnostic you should never ship.

CRUFT_GC_HEADROOM (default 2.0, floor 1.1)

How much a heap may grow between collections. Higher means collect later: more throughput on allocation-churny code, higher resident memory. Lower means collect sooner: lower memory plateau, some throughput cost. Values are clamped to the 1.1× floor, so you cannot force collect-on-every-allocation.

Concrete guidance:

In worker-pool sweeps this knob moves RSS far more than it moves latency (p99 stayed in the same band from headroom 1.5 to 4.0), which is why the default is usually right for a service, reach for it only when memory is the thing you are actually fighting.

CRUFT_GC_TARGET_MB (soft cap)

Instead of picking a headroom, state the memory goal directly:

CRUFT_GC_TARGET_MB=8000 cruft server.mjs   # keep RSS near 8 GB

It behaves like Go's soft GOMEMLIMIT: full headroom while well under the target, tightening smoothly toward the floor as RSS approaches it. It bounds reclaimable memory (garbage). It cannot push RSS below your live working set, and it is read process-wide so every worker in a pool tightens together.

CRUFT_GC_STRESS, diagnostic only

Forces maximal collection frequency to surface memory bugs deterministically. It is a correctness diagnostic (and useful for classifying "garbage vs. leak", see below). Do not set it on a real workload.

Full behavior, measured RSS curves, and the decision procedure for high RSS are on the garbage collector page.

Worker scaling

Cruft's Imogen workers keep each worker's mutable heap local and collectible while sharing only interned strings and explicit byte buffers, so a worker pool scales sub-linearly in memory and converges to a plateau under sustained load rather than climbing. On a webhook workload with 2 KB payloads:

Sixteen times the workers costs well under twice the memory, because most of the per-worker cost is a fixed per-realm baseline rather than a duplicated VM. CRUFT_GC_HEADROOM applies per worker heap, so the throughput/RSS trade-off you pick affects the whole pool.

Use workers when you have genuinely parallel, isolatable work: many independent requests, a fan-out of CPU-bound tasks, or a tenant boundary you want enforced. They are cheap enough to use as an architectural primitive (thousands per process). They are not a way to speed up a single sequential computation, and they carry an irreducible live floor (~0.3–0.4 MB per worker) that no GC setting touches. See Imogen workers for the transport model and scaling internals.

Measuring

Guessing about performance is how "fast but wrong" and "fast but bloated" slip in. Measure.

Memory

process.memoryUsage().rss reports real resident memory, matching /usr/bin/time. Track your plateau with it directly:

console.log((process.memoryUsage().rss / 1048576).toFixed(1) + " MB");

Know the gap: rss is real, and heapTotal/heapUsed report a real (coarse — heapTotal == heapUsed) figure that tracks the working set. Only external and arrayBuffers report 0, so that per-category breakdown isn't available today. Use rss (or the coarse heap figure) and cross-reference the debugging guide, which carries the full garbage-vs-leak decision procedure.

Startup phases

CRUFT_PROFILE=1 writes a phase breakdown to stderr after the run, parse, compile, eval, and CJS/module-resolution internals. The summary line looks like this (from a small loop):

cruft-profile: modules=2 parse=0.1ms compile=0.5ms eval=6.2ms … total_phases=6.9ms

followed by per-statement-kind timings (cruft-profile-stmt: …). It is verbose and engine-oriented rather than user-friendly, but it answers "where did startup time go?", parse and compile cost, module resolution, require overhead. It is a startup-phase instrument; it does not measure throughput or profile the JIT.

Rigorous throughput numbers

For real throughput and JIT-level questions, use a proper benchmark harness rather than a single profile line. The discipline that matters for adopters: a one-off CRUFT_PROFILE line or a single timing is only a hint. Run multiple times, control the input, back a claim with a trace or counter rather than a hunch, and write down what you ran before you draw a conclusion.

Practical do and don't

• Keep hot call sites monomorphic. LeJIT specializes property access and calls on the shape of the objects it sees. When a hot site sees one object shape, the access compiles to a shape-check plus a fixed-offset load; when it sees many unrelated shapes (megamorphic), that specialization is lost and the site stays on the general path. Construct objects the same way (same property order), and avoid funneling structurally different objects through one hot function. The shapes-and-inline-caches mechanism is described in LeJIT.

• Let hot loops be loops. Tight numeric loops are exactly what the JIT is built to admit and unbox. Loops whose bodies do something too dynamic for the compiler to represent in its fast path will either stay interpreted or take deopt exits; that is correct, just not fast.

• Don't micro-optimize cold code. Code that runs a handful of times never leaves Exegesis, so shape discipline there buys nothing. Spend the effort on the paths your profiling shows are hot.

• Don't assume a speed class. Cruft is young and not yet tuned to V8-parity throughput for all workloads. For memory under many-worker isolation it does unusually well; for raw single-threaded throughput on arbitrary code, measure before you promise.

Source note

CRUFT_PROFILE=1 output shape and process.memoryUsage() (rss and the coarse heapTotal/heapUsed real, external/arrayBuffers 0) come from a small loop. GC-knob behavior and RSS curves are drawn from the garbage collector; the worker RSS figures (~350 MB @ 64, ~420 MB @ 256, ~580 MB @ 1024, 2 KB webhook payloads) from Imogen workers; execution tiers and shapes/ICs from LeJIT.