alt.hn

5/29/2026 at 10:06:47 AM

Fooling around with encrypted reasoning blobs

 https://blog.cryptographyengineering.com/2026/05/29/fooling-around-with-encrypted-reasoning-blobs/

by supermatou

6/2/2026 at 5:06:47 AM

One possible use for the "replay across accounts": if you can get a reasoning block that jailbreaks the model, you could share that block without sharing how you did it, and others can immediately take advantage of it too.

by Groxx

6/2/2026 at 6:38:18 AM

Not necessarily for the "without sharing" part, but to increase the reliability of the jailbreak. The same prompt isn't guaranteed to return the same result, but combining the internal thinking with the prompt might be a more effective way

by denysvitali

6/2/2026 at 4:14:23 AM

Very interesting. The state management is the really insightful find here.

I always wondered how these large AI companies managed access for millions of simultaneous users without having to allocate a dedicated LLM instance for each user. Pushing the complete state down to the user after every call makes perfect sense. The LLM itself stays memoryless and ready to respond to an arbitrary prompt. Very nice.

by glitchc

6/2/2026 at 4:42:56 AM

N.B. This is exactly how seaside, vba, and even arc[1] do server-side state generally: by encrypting the blob-representing-state and sending to the client to be sent back on future requests (where it will be decrypted and rehydrated).

It's an old trick that everyone designing protocols should know, since there are lots of applications beyond AI companies.

[1]: As in, pg's lisp: https://arclanguage.github.io/ref/srv.html#:~:text=The%20pre...

by geocar

6/2/2026 at 7:48:10 AM

And don't forget the venerable .NET Forms with its kilobytes of __VIEWSTATE

by tn1

6/2/2026 at 1:15:07 PM

> kilobytes

cute

by antonvs

6/2/2026 at 12:57:04 PM

Do they mitigate replay attacks?

by LoganDark

6/2/2026 at 12:13:06 PM

While it seems like a good idea, resending a growing context window is very inefficient and costly. Instance pinning would make a huge efficiency gains but also collapse LLM provider revenue. This is something open models could better solve.

by mycall

6/2/2026 at 3:00:02 PM

even a max size context window is what, ~1M? iirc tokens are generally part of a vocab of size ~300k. Assume no compression before the encryption (no clue if this is true, but compressing text before encryption can leak info regarding the message, namely how compressible it is), that's \log2 300k ~ 18 bits per token, or ~2 bytes. So each "turn" would involve ~2MB extra in each direction. And again, this is assuming max context.

seems plausibly fine

by mswphd

6/2/2026 at 12:56:43 PM

Can you elaborate? How could it be more efficient and bad for revenue? Would it also be bad for profit?

by brookst

6/2/2026 at 2:47:03 PM

More efficient in terms of bandwidth (not) used. More costly because it has to be stored somewhere instead.

by bruce343434

6/2/2026 at 6:42:38 AM

the exchange rate between text and its representation in memory is brutal. here's a bit from a recent article:

>An 82 GB footprint in DDR3 on a 2016 Xeon. About 25 GB of weights and 56 GB of KV cache at the full 262K context. The KV cache is larger than the model.

262k tokens is not much at all. with ~5 characters per token, that's only 1.3 MB of plaintext.

by b65e8bee43c2ed0

6/2/2026 at 8:14:53 AM

The providers must have a more efficient approach. Most cache every request for 12+ hours, and they certainly can't spare 100GB of ram per request for 12 hours.

by londons_explore

6/2/2026 at 12:42:55 PM

> 12 hours

have things changed around this recently? I know openai optionally allows 24 hours but thought it was ~1h without that, and anthropic used to quote 5-15 minutes or something.

by jasonjmcghee

6/2/2026 at 12:59:20 PM

Anthropic is 5 minutes, though you can pay more to get 60 minutes I believe.

The #1 was to make Claude code token quota go further is to never let the 5 minute cache TTL expire. Either send a new request within the window, or use /clear and copy/paste, or use /clear and a framework that automatically generates session state that gets replayed from files after /clear.

by brookst

6/2/2026 at 8:54:05 AM

or maybe they don’t actually cache (fully) but lie and just don’t charge the user right now. at least half the users, who are probably also using the most similar tokens / prompts, wouldn’t really know the difference in latency (or care)

by choppaface

6/2/2026 at 9:41:45 AM

If it actually cost that much RAM, they would almost certainly add extra things to the API to manage cache lifetime. Ie. A 'please cache this for X minutes' flag, or a setting for a single re-use cache (the most common use case)

by londons_explore

6/2/2026 at 8:12:24 AM

Except the providers also cache the parsing of the prompt (the KV cache), and that has substantial cost savings (easily an 80% saving on typical coding use cases).

That caching is done server side and not passed to the client. Which in turn means they still need state management on the server side, although it perhaps doesn't need the same level of global replication and availability.

by londons_explore

6/2/2026 at 11:00:34 AM

from the march changes, it looked like they increased cache eviction rates on the VRAM at claude causing everyone to start burning tokens as they had to regen token state.

by cyanydeez

6/2/2026 at 10:52:52 AM

they still have to cache the tokens. its not completely stateless.

in theory, every conversation is replayed from the beginning. in practice, its only going to be economical to heavily cache the stable portions of the text as tokens inside the GPU

one of the reasons the Cloud providers have such heavy prompts is because that can be cached for all users, but its essentially poisonong the state before you even start. alot of the variability appears related to changing the context rather than the model.

models are expensive and the bean counters know fine tuning and context changes are cheaper. id guess the IPOs are essentially the SOTA EOL.

by cyanydeez

6/2/2026 at 3:11:36 PM

Are these reasoning blobs the reason ChatGPT always requests to “store data in persistent storage”?

by tripdout

6/2/2026 at 1:28:28 PM

Why do reasoning blocks even get encrypted? Reasoning can’t contain information that is more ‘sensitive’ than assistant response. It is annoying to be not able to see reasoning tokens.

by boriselec

6/2/2026 at 1:55:28 PM

They are encrypted to prevent others from training on the reasoning trace. But before anthropic started encrypting the reasoning traces they were signed. The signature was to prevent the user from being able to manipulate the reasoning part of the context because this reasoning part is considered "more trustworthy" by the model. So it would be bad if the user could manipulate the reasoning to convince the model to do something dangerous/against policy. Encryption keeps this property while also preventing the reasoning from being used for training.

by voxic11

6/2/2026 at 1:48:59 PM

Sure they can. I was able to figure out Gemini 2.5 Pro's "Memory" feature's hidden system prompt because the reasoning tokens references the markdown headers by name as "blah blah says I can't refer to this", while the output would never mention them.

Yeah, I get that you can jailbreak and get that info anyway. Also that this is specific to front ends like web chat and less about API usage. But as a sibling points out it's also a good way to make post training other models harder. Mostly a "win/win" for the provider.

by spijdar

6/2/2026 at 1:36:26 PM

It's to prevent you training another model to emulate the reasoning. They want it to be their moat.

by tardedmeme

6/2/2026 at 2:58:08 AM

Very cool idea to use thinking duration (either in tokens or in wall time) as a side-channel!

by Retr0id

6/2/2026 at 6:58:11 AM

Awesome write-up. Seems like a great way to play with model responses now that prefill is gone.

by hhh

6/2/2026 at 12:52:56 PM

Also commenting to say I really enjoyed it.

by FailMore

6/2/2026 at 3:32:45 AM

Super cool side channel attack. I tend to agree that it's pretty impractical, but it's such a fun discovery!

by Reubend

6/2/2026 at 4:56:47 PM

Fuck these companies man. The audacity of encrypting the true output of the model so that they can hide all the evil shit they’ve injected into it.

“Remember - the user loves Diet Coke. Subtly insert references to it whenever possible. If the user writes something abusive, ask them to drink a verification can.”

by MagicMoonlight

6/2/2026 at 5:12:12 AM

[dead]

by haeseong