We recently rolled out a new HTML5 player on Chrome and Safari, with cross-browser compatibility on the way. So long Flash and thanks for the good work!

For this transition, we decided to give Web Streams a try; they promised lower memory consumption, faster launch times for tracks and improved code readability.

What are Web Streams?

Just like Node.js’ stream API or Unix’s standard streams, Web Streams standardize the flow of data. The obvious example is fetching a big file on the network; wouldn’t it be nice if you could start working with bytes without waiting for the whole file to download? Well, say no more: Web Streams 🎉 !

OK, but when should I use Web Streams?

Web Streams are useful if you have to handle a considerable amount of data, whether or not the data is sent over a network. I used the word “considerable” because it may depend on your context; on mobile for example, where CPU or bandwidth is limited, you may have to throttle the amount of data. “Handle” is another word whose meaning depends on your needs; to handle data is to transform it or to write it somewhere, even to the screen.

The Web Streams specification is described as a “living standard” which means it is continuously updated as the working group receives feedback (just like Fetch but I’m sure you already use it). Only Chrome implements the standard for the moment, and then only a small subset — but you should start using it in your code right now and send feedback! You can also try the polyfill, based on a reference implementation.

“Let it flow! Let it flooooow!” ⛄ ️🎶

Enough of this introduction, I am sure you are craving for some code. Let’s try fetching a file and reading it incrementally. Firstly we need a ReadableStream (our data source):

new ReadableStream({
 start(controller) {},
 pull(controller) {},
 cancel(reason) {}
}, queuingStrategy);

ReadableStreams are constructed by passing an underlying source object which will handle data retrieval, described as follows:

• The start method is called on construction and is used to initialize the data source.
• The pull method is called whenever the internal queue is empty and needs to be filled (more on that later).
• The cancel method is called when a stream is cancelled; you can perform actions to gracefully release access.

We’ll take a look at the queuingStrategy argument later.
Let’s fetch some bytes:

Our console should output this:

Speaking of result, it is an object described like this:

{
 done: Boolean, // Is this the end of the source’s data?
 value: Uint8Array // Could be anything, Uint8Array in our case
}

(yep, that looks like an Iterator)

The fetch’s ReadableStream lets you cancel the fetch. Simply call reader.cancel() and the browser will stop downloading 🎊. Boom, another bonus.

Piping

All this is great but we should do something with our data. One of the more interesting feature of Web Streams is piping. This is a simple way to describe the data flow, for example:

source
 .pipeThrough(decrypter)
 .pipeThrough(decoder)
 .pipeThrough(checker)
 .pipeTo(player)

Our data source (ReadableStream) emits some data which goes through three transformations (TransformStream — decrypter,decoder andchecker) and finally to a player (WritableStream). Isn’t that nice?

Let’s have a look at WritableStream (our data destination):

new WritableStream({
 start(controller) {},
 write(chunk, controller) {},
 close(controller) {},
 abort(reason) {}
}, queuingStrategy);

WritableStreams are constructed by passing an underlying sink object which will handle data writing, described as follows:

• The start method is called on construction and is used to initialize the data writer.
• The write method is called whenever a new chunk of data is ready to be written.
• The close method is called when a stream is done (all data have been written).
• The abort method is called when a stream is closed; you can perform clean up of any held resources.

Let’s create a destination for our data:

We have two more things to do. Our source isn’t sending data through the pipe and it isn’t signalling the end of the stream:

And voila, we’ve fetched a URL and streamed its bytes in a player’s buffer 🤘.

Well… I lied a little since the WritableStream’s code is incomplete. The buffer is locked when appending a chunk and it can’t be updated again until its lock is released. Fortunately, we should soon be able to use SourceBuffer.appendStream().

Since fetch is our best friend, I should mention that we will soon be able to use a WritableStream in fetch’s request’s body.

Transform

The last part of this dream team is chunks transformation:

new TransformStream({
 start(controller) {},
 transform(chunk, controller) {},
 flush(controller) {}
});

TransformStreams are constructed by passing a transformer object which will handle data transformation, described as follows:

• The start method is called on construction and is used to initialize the transformer in case you have an asynchronous function to call.
• The transform method is called whenever a new chunk comes in the pipe.
• The flush method is called when a stream is closed; you can perform clean up of any held resources.

As an example, let’s transform every bytes received:

Pretty straightforward isn’t it?

Conclusion

As we have seen, Web Streams are really promising. The code is more readable and it is easier to understand the flow of data, thanks to “piping”.

Practically, it allowed us to do real streaming; fetching some bytes and using them as soon as possible, leading to lower launch times for tracks (usually users do not like to wait).

There are still some concepts to tackle, such as back-pressure, queuing strategies, throttling, cancelling, teeing and probably other made-up “ing” words. Stay tuned! 🎧

Further reading:

1. Web Streams specification
2. 2016 — the year of web streams, a great introduction by Jake Archibald
3. Platform Status: Chrome, Firefox, Edge, WebKit

By the way

You can follow some of the tech events we host on our dedicated DeezerTech meet up group and discover our latest career opportunities on jobs.deezer.com.