Node streams aren't hard anymore

Node streams have a reputation. James Halliday wrote a "stream-handbook" repo over a decade ago that became canonical, and the existence of a handbook told you everything. For years, asking a Node developer about backpressure was a way to find out which job they were about to quit.

The reputation was earned, but it's mostly obsolete now. The API got fixed in stages between 2018 and 2021, and nobody made a clean announcement of it, so the cultural memory stayed at "streams are scary" while the actual code became, for most uses, boring.

Why they were hard

The short version: three eras of API coexisting in one type, errors that didn't propagate through .pipe(), backpressure as an advisory boolean that ninety percent of code ignored, and four different events all roughly meaning "done" ('end', 'finish', 'close', plus a null from read()). The longer version is the stream-handbook itself.

Pre-2018, writing correct stream code meant remembering, for every pipeline, that you had to attach 'error' to every stage because .pipe() didn't propagate errors and a downstream failure would orphan the upstream stages, hanging the process or leaking file descriptors. You had to check the return value of .write() and listen for 'drain', because otherwise your writable buffer grew until the process OOMed. You had to know which mode (paused or flowing) your readable was in, because attaching a 'data' listener flipped it, and once flowing you could lose chunks if you attached late. And you had to distinguish 'end' (no more reads) from 'finish' (all writes flushed) from 'close' (resource released), because they fired at different times and meant different things.

What changed

The fix came in pieces.

In 2018, Node 10 shipped stream.pipeline(). It propagates errors, destroys all stages on failure, and cleans up resources, the single biggest improvement to streams in Node's history, and it arrived without fanfare. The same release made Readables async iterables, so for await (const chunk of stream) started working, hiding the entire mode/event apparatus behind one line.

In 2020, Node 15 added stream/promises, giving you await pipeline(...) and await finished(stream). Promise-native, no event handlers.

In 2021, Node 16.5 added Web Streams as experimental (they became stable in Node 21, late 2023), a different model entirely, promise-based, never inheriting from EventEmitter, for code that wants out of the legacy.

Most stream tutorials today still walk you through these chronologically: events first, then .pipe(), then pipeline(), then for await, then Web Streams. You finish with five overlapping mental models and the feeling that streams have too many APIs.

There's a better framing.

The map

Every Node stream API fits into one 2×2:

Four cells. Every API you've ever seen on a stream lives in exactly one of them. The complexity of Node streams comes from each cell having multiple layers of API stacked on top, accreted over different versions, and the clarity comes from seeing the cells as separate problems.

Let's fill them in.

Consume Readable, "how do I take data out of this thing"

Three layers, low to high:

// Layer 1: paused mode, manual
const chunk = readable.read();
readable.on("readable", () => {
  /* try .read() again */
});

// Layer 2: flowing mode, push
readable.on("data", (chunk) => handle(chunk));
readable.on("end", () => done());

// Layer 3: async iteration (Node 10+)
for await (const chunk of readable) handle(chunk);

Layer 3 hides everything below it. Backpressure is automatic because await blocks the loop, errors throw out of the for await, and stream mode is irrelevant because you never see it. For consumption, this is the only API you need now.

Consume Writable, "how do I put data into this thing"

// Layer 1: manual, with backpressure
const ok = writable.write(chunk);
if (!ok) await once(writable, "drain");
writable.end();
writable.on("finish", () => done());

// Layer 2: pipeline (Node 10+, promise variant Node 15+)
await pipeline(source, writable);

There's no for await equivalent for writables, you can't async-iterate a sink. The modern answer is to never call .write() yourself, let pipeline() do it. The boolean-return-value-plus-drain dance is still the underlying protocol, but your application code shouldn't be running it.

Implement Readable, "I want to produce data"

// Layer 1: raw
class MyReadable extends Readable {
  _read(size) {
    this.push(chunk);     // returns false when buffer is full
    this.push(null);      // signal end
  }
}

// Layer 2: from an async iterable (Node 12+)
const r = Readable.from(async function* () {
  for await (const row of db.query(...)) yield row;
}());

Readable.from covers most cases. You write a generator, sync or async, and Node wraps it into a Readable that handles _read, push, backpressure, and end-of-stream for you. For most custom Readables, this is what you reach for.

The raw _read/push API is still current, it's what you fall back to when you need fine control, things like multiple data sources or integration with a non-iterable producer.

Implement Writable, "I want to receive data"

const w = new Writable({
  write(chunk, encoding, callback) {
    sink.send(chunk).then(callback, callback);
  },
});

This is the cell that didn't get sugar. There's no Writable.from(asyncFn) that wraps a consumer function into a properly backpressured Writable. You either implement the write(chunk, enc, cb) callback form or you extend the Writable class and override _write. Calling callback() participates in backpressure, Node won't call write again until you signal completion.

Most application code doesn't need to implement Writables, the built-in ones (fs.createWriteStream, the HTTP response, network sockets) cover the common sinks. When you do need one, the raw API is verbose but fine.

What the map shows

The two consume cells got modern wrappers, for await and pipeline(). Most application code lives in those two cells, which is why most stream code is now easy.

The two implement cells got partial sugar (Readable.from) or none (Writable). This is where the historical complexity still lives, and it's narrower than the reputation suggests, you have to be writing your own stream type to hit it.

Web Streams are the same 2×2 with different method names. Consumption goes through getReader().read(), production through a controller's enqueue() from inside the source. The Writable side has its own surface, a WritableStream constructed with a write() method on its underlying sink. The shape is identical, what's missing is the EventEmitter row beneath each cell. Web Streams never accumulated that layer, so there's one API per cell instead of two or three.

What this means in practice

If you're consuming streams in application code, you're in the top row of the map, and you should be using:

// Reading
for await (const chunk of readable) {
  /* ... */
}

// Writing / piping
await pipeline(source, transform, destination);

Everything below that, the raw event API and .pipe() without pipeline, is the legacy stack. It still works, and you'll encounter it in older code, in Express middleware, in libraries that haven't updated. You should be able to read it, but you shouldn't be writing it.

If you're implementing custom streams, you're in the bottom row, where the API is still raw. Readable.from covers most Readable cases via generators. For Writables, you write new Writable({ write(c, e, cb) {} }) and it's fine, the verbosity is the cost.

If you're starting a new project that might run outside Node, on Workers or in browsers, skip the Node stream API entirely and use Web Streams. Same model, smaller surface, portable.

The reputation, revisited

Node streams were hard because the API accumulated rather than redesigned. Each fix added a new layer without removing the old ones, and for a long time, that meant you had to learn all the layers to write correct code.

That's no longer true. The top of the stack, for await and pipeline(), handles the common cases cleanly. The bottom is still there for performance-critical paths and custom stream implementations, but it's no longer on the daily path.

The handbook isn't wrong, it's just no longer the prerequisite it was.