deflate and gzip

rusty-js-deflate is Cruft's DEFLATE, gzip, and zlib implementation behind node:zlib, DecompressionStream, and the gzip handling in HTTP and tar. Both decode and encode are complete, and the inflater caps output at 256 MiB and bounds back-references so a few kilobytes of input cannot allocate gigabytes.

rusty-js-deflate is Cruft's implementation of DEFLATE, gzip, and zlib, replacing flate2 / miniz_oxide. It backs node:zlib, DecompressionStream, and the gzip handling used by HTTP and tar. A decompressor is a parser of attacker-controlled input that also allocates: a few kilobytes of input can ask for gigabytes of output, so this page covers what the crate refuses as much as what it decodes.

Alpha. rusty-js-deflate (Cruft 0.0.10) stands in for flate2 / miniz_oxide. It has limited interop testing against the wider ecosystem and is not production-ready. Do not place it on an adversarial-input path in production.

Decode and encode are both real

The crate splits into a decoder and an encoder, and both are full implementations.

The decode side is a complete RFC 1951 inflater: stored, fixed-Huffman, and dynamic-Huffman blocks (inflate), wrapped for RFC 1952 gzip (gunzip, checking CRC32 and ISIZE) and RFC 1950 zlib (zlib_inflate, checking Adler-32). An http_deflate_inflate entry point tries zlib framing first and falls back to raw DEFLATE, because HTTP Content-Encoding: deflate is ambiguous across servers. The decoder round-trips against system gzip output, including highly repetitive input that exercises LZ77 back-references and English text that exercises dynamic Huffman.

The encode side is a real compressor. compressed_deflate and its gzip/zlib siblings run genuine LZ77 (a hash-chained match finder over 3-byte hashes) plus canonical, length-limited Huffman coding (length_limited_huffman, packages the code lengths to the 15-bit limit). It builds a dynamic-Huffman block (encode_dynamic) and a fixed-Huffman block (encode_fixed), and compressed_deflate emits whichever of dynamic, fixed, or stored is smallest, so the output is always valid, universally-decodable DEFLATE that actually compresses: gzipSync of a 10 KB repetitive buffer comes back at 44 bytes.

Decompression-bomb defenses

The dangerous input is the small one: a tiny, valid stream whose output is enormous. Two bounds in the inflater stop it.

  • Output cap. MAX_OUTPUT is 256 MiB, checked before every growth in decompress_block and before each stored-block copy. Past the cap it returns DecodeError::OutputTooLarge instead of allocating, and the check runs before the copy rather than after.
  • Back-reference bound. A DEFLATE copy can name a distance pointing before the start of output, which a naive decoder would read out of bounds. A distance > out.len() check returns DecodeError::DistanceTooFar.

The decode path has no unsafe and no unwrap() on untrusted bytes: malformed streams return a DecodeError variant (UnexpectedEnd, InvalidBlockType, InvalidHuffmanCode, and so on) rather than panicking. The bit reader is bounds-checked on every read.

Caller-configurable limits

The 256 MiB constant is a hard ceiling; a per-request server wants something lower and configurable. Every decode entry point has a _with_limit twin (inflate_with_limit, gunzip_with_limit, zlib_inflate_with_limit, http_deflate_inflate_with_limit), and the StreamCodec adapter behind DecompressionStream exposes finish_with_limit. On the host side, node:zlib defaults to a 64 MiB decode ceiling (below the 256 MiB crate maximum), the whole-buffer stream adapter carries a 16 MiB compressed-input cap, and stream decode applies a 128x expansion-ratio cap.

Limitations

  • The stream adapter is whole-buffer. StreamCodec::push accumulates all input and defers decode to finish; the DEFLATE core is not incremental, so a DecompressionStream here does not stream in the flow-control sense. The compressed-input side is bounded by the 16 MiB cap above, not by true streaming. For bounded-memory streaming of an unbounded compressed source, this is not that.
  • Encode ratio is not tuned to zlib. The encoder is a real LZ77 + Huffman compressor (best-of dynamic/fixed/stored), so output is compressed and universally decodable, but the match finder and heuristics are simpler than zlib / miniz_oxide, so ratios trail theirs on some inputs. For the last few percent of ratio on a ratio-critical workload, the mature encoders remain ahead.
  • Interop breadth is thin. Decode round-trips against system gzip and covers corrupt-CRC, invalid-magic, and truncated-stream rejection, but has not been run against a broad corpus of third-party encoders, unusual window sizes, or the full zlib conformance surface. miniz_oxide has years of fuzzing and ecosystem exposure behind it; this crate does not. A fuzz lane over deflate/gzip/zlib decode is scaffolded, not a long-running campaign.
  • Maturity. flate2 / miniz_oxide is the choice for production today. rusty-js-deflate lets Cruft own its compression layer with no C or third-party Rust dependency, with decompression-bomb and back-reference defenses in place.