URL parsing

rusty-js-url is the URL parser behind new URL and fetch, implementing the WHATWG URL state machine in place of the crates.io url crate. It parses schemes, hosts, paths, and queries, with careful host handling (IPv4 number forms, IPv6, IDNA domains), because which string is the host is a routing and SSRF decision.

new URL(req.query.next) and fetch(url) sit on top of a parser that rarely gets a second thought. But new URL is a WHATWG URL state machine, and every host it produces is a routing decision: http://[email protected]/, http://①.example, http://0x7f.1/. If two parsers disagree about which of those is the host, an allowlist check on one and a connection on the other is an SSRF. The url crate is the Rust ecosystem's answer, battle-tested against that. rusty-js-url is what Cruft ships in its place, implementing the same state machine.

Alpha, unaudited. rusty-js-url (Cruft 0.0.10) stands in for the crates.io url crate. It has not been externally audited. Do not rely on it for production URL handling of adversarial input.

A URL library is a state machine

new URL does far more than input.split('/') with sugar. WHATWG URL is a specified state machine: scheme state, authority state, host state, path state, each with defined transitions. rusty-js-url implements that machine literally, with the host grammar and the Url record and its serialization in separate modules. The public surface is small and Node-shaped: parse, parse_to_href, can_parse, join_to_href, set_component_href, backing a Url with protocol(), hostname(), pathname(), search(), hash(), origin(). The URL interface from Node carries straight over to this API.

The state machine matters because the wrong-model version ("just split on / and :") is exactly the version that disagrees with the browser about where the host ends, and that disagreement is the vulnerability class. So the question is never "does it parse" but "does it parse the same string the same way the spec does."

Host parsing is the security surface

A path-parsing bug gives you a wrong file. A host-parsing bug gives you a wrong server. So look at the host grammar first. It implements the pieces that decide identity:

  • IPv4 (parse_ipv4, serialize_ipv4): the full WHATWG number grammar, including octal (0177), hex (0x7f), and dotless/short forms, then canonical re-serialization. This is the exact surface where 0x7f.1 must resolve to 127.0.0.1 and not stay a "domain." new URL("http://0x7f.1/").hostname and new URL("http://0177.0.0.1/").hostname both return 127.0.0.1.
  • IPv6 (parse_ipv6, serialize_ipv6): compression, embedded IPv4 tail, the full [...] bracket form. new URL("http://[::1]/").hostname returns [::1].
  • Domains (domain_to_ascii): delegated to rusty_js_idna::to_ascii, the UTS-46 layer, not reimplemented here. A Unicode host ACE-encodes: new URL("http://例え.日本/").hostname returns xn--r8jz45g.xn--wgv71a.
  • Forbidden host code points: on opaque hosts (non-special schemes) the parser rejects the WHATWG forbidden-host set before percent-encoding. new URL("non-special://ho st/path") throws Invalid URL.

The host and general-parse behavior is checked against the upstream WPT urltestdata.json corpus, vendored into the test suite. The gate pins the corpus at 891 cases and holds a pass floor of 705, plus targeted known-answer tests for numeric hosts, IDNA, WPT encoding, and edge cases. A fuzz target exercises parse, base-relative parse, serialize-reparse, and origin projection. Because the corpus size is asserted, a silent shrink fails the build.

The host grammar has zero external dependencies and no unsafe, and the parse paths carry no unsafe, todo!, unimplemented!, or panic! (only rusty_js_idna and a JSON manifest crate for the test harness are pulled in).

The scope of the corpus floor

The floor is 705 of 891, not 891 of 891. About 186 corpus cases are not matched today, and they are edge shapes rather than the http/https SSRF core: opaque-host and non-special-scheme canonicalization corners, some serialization details. The pinned floor means the crate cannot silently regress below what it passes now, and it also means this is not yet a byte-for-byte drop-in for url across the whole spec surface.

Two specific behaviors were once weak, both now fixed:

  • The opaque-host branch used to only percent-encode C0 controls; it now rejects the forbidden host code-point floor (C0/space plus ` #%/:<>?@[\]^| ) before encoding. Both parse_to_href("non-special://ho st/path", ...) and the %00 variant return Err`, matching the behavior above.
  • blob: origin used to return an empty string; it now derives from the embedded URL's origin, with invalid inner URLs returning null. new URL("blob:https://example.com/abc").origin returns https://example.com.

With the full corpus vendored, its size and floor pinned, and the fuzz target added, that was the last open weakness on the crate itself.

Limitations

This is the strongest crate in the family, and the tested/unproven split is genuinely favorable: host parsing, numeric hosts, IPv6, credentials, and the WPT corpus floor are all backed by in-tree tests. Three real caveats:

  1. The 186-case gap is not zero. url on crates.io passes the WPT corpus at a far higher rate and has years of adversarial exposure. On the exotic-scheme and canonicalization corners, url leads on maturity. For arbitrary hostile input, the reference crate is the safer bet today.
  1. The homograph defense is not in this crate. domain_to_ascii delegates to rusty_js_idna, and that crate's validity layer is where the host-confusion risk actually lives. Its IDNA validity rejection was once weaker than the reference idna crate: partial bidi, no full ContextJ/ContextO, no CheckHyphens, Latin-only NFC. That is now fixed (contextual-validity floor plus a generated Unicode 17.0.0 bidi table), but the point stands: this crate's host-confusion safety is only as strong as rusty-js-idna's validity layer, which it does not own. Read that page before trusting mixed-script hostnames.
  1. Fuzzed, but young. The fuzz target is compile-checked and scaffolded; it has not run the corpus-hours that the reference crate's fuzzing represents. "No known differential on http/https" is a statement about what testing has found, not a proof that no differential exists.