alt.hn

5/6/2026 at 2:31:42 PM

How to make SSE token streams resumable, cancellable, and multi-device

 https://zknill.io/posts/everyone-said-sse-token-streaming-was-easy/

by zknill

5/8/2026 at 10:38:18 AM

In JS land, this problem (streaming, resuming, recovering, multi-client, etc) has been fully solved by https://durablestreams.com - and it can be self-hosted, or managed via Cloudflare DO.

by e1g

5/8/2026 at 1:47:24 PM

Cloudflare Sessions API and Anthropic Routines have a really similar model. Where they are hosting the 'session store' for you, and giving you access to it over long-polling (or sometimes websockets).

It's a bit harder to do agent presence ('is the agent still there') with this model without heartbeats, but possible.

It's good to see the industry starting to address the "durable sessions" problem, because it sucks.

by zknill

5/8/2026 at 7:27:21 AM

I built this Clojure lib for robust high scale LLM calls wherein the consumer is usually a http request waiting on an SSE stream. https://github.com/jhancock/aimee

The article states: "Most applications are built on an architecture like the one above, where there are a number of stateless horizontally scaleable server replicas that can handle client requests."

Using the library I built, I have yet to worry about this as Clojure core.async, http libs and Java VM are so rock solid, I don't have a fragile set of stateless servers. Sure, at some point there are rare edge cases but it's nice to get very far along without worrying about them.

by jhancock

5/8/2026 at 10:36:03 AM

Im not sure what you mean by fragile stateless servers. If they are stateless, what is fragile about them?

by dgellow

5/8/2026 at 9:21:11 AM

> Stop reading here if you just wanted the how-to. Because I’m going to talk about what I think is better, and that is probably too ‘commercial’ for some folks.

> I work for Ably, and I’m building a dedicated transport for AI applications that...

by the_gipsy

5/8/2026 at 10:05:56 AM

It's honest. If they genuinely think it's better it's fair to say so. The article up to that point seems well written over the domain (I've solved much of the same set of problems.)

by vintagedave

5/8/2026 at 11:40:00 AM

Should be at the top of the post, not at the end.

by the_gipsy

5/8/2026 at 10:34:52 AM

Agreed, I found it informative and did appreciate the reading, even with the assumption it was AI generated

by dgellow

5/8/2026 at 11:48:30 AM

Cancellation was the painful one for me. I have SSE streaming in a React Native app and there's no proper AbortController — ended up with a ref + interval hack to detect when the user navigates away mid-stream. Still don't have a good answer for "connection dropped, show partial response and let them retry from where it left off." Would've loved something like this.

by TodorGrudev

5/8/2026 at 12:45:03 PM

This is way too complex!

We have developed a simple API that can produce tokens and events in various formats like jsonl, sse, even csv. Cancelation can happen when the socket is closed when streaming - fully automated - or when you push an event from another endpoint so that we stop the stream midway.

Background task are also subscriable and cancelable.

see https://cbk.ai

by _pdp_

5/8/2026 at 1:39:52 PM

> This is way too complex!

100% - the argument of the article is that building any feature beyond chat-based-demos on HTTP SSE streaming is super complex. But a lot of folks still want to do it, because that's what their tech stack is. I think it's still a valuable thing to be talking about how you might do that.

by zknill

5/8/2026 at 7:03:16 AM

> HTTP is just not a good transport for streaming LLM tokens and for building async agentic applications

I don't know if I agree if this is a problem with SSE or HTTP. Something like a Redis Streams-backed SSE would solve most of the 'challenges' presented in the post.

by ekojs