Systems | Development | Analytics | API | Testing

Backpressure in AI streaming can build anywhere from inference to the client, but delivery is where handling it can change the response itself. The three usual approaches there (buffering, batching, and rewriting a stored record) can change how the response is stored, or what storing it costs. Anyone who joins mid-answer then gets fragments to stitch together, or your database pays for a full write at every flush.

Refresh the page, lose signal, switch tabs. Users expect the AI conversation to still be there when they return, with the answer picking up where it left off. Resumable token streaming makes that possible: a returning client catches up with the in-progress response instead of starting over. Making a stream resumable is an architecture decision. Starting generation, storing the response, and delivering it need to happen in separate parts of your stack.

I spoke about Ably's approach to resilience at the AWS Cloud Resilience Roadshow in Dublin on 22 September 2026, and the questions afterwards covered CDNs, regional failover, outages, message ordering, and capacity. They got me reflecting on which parts of our approach are common across the industry, which are less so, and how choices made early, well before my time at Ably, have put us in a strong position today.

WebSockets and Server-Sent Events (SSE) solve the same immediate problem: getting data from a server to a browser without the client asking for it first. Most comparisons stop at how their connection types and browser support compare, to help you decide which one to use. This one covers that ground too, but goes further, into what neither protocol solves for you and whether to build that yourself or hand it to a managed platform.

How you store a streamed AI response decides what a client sees when the client joins partway through. A human agent taking over a support conversation, a refreshed browser, and a second device all need the response generated so far, and depending on how the response was stored, that late joiner gets the full response, fragments, or nothing. This article compares four common ways to store a streamed response, what each one gives a late joiner, and what each one costs your team.