alt.hn

7/23/2026 at 5:06:22 PM

Show HN: Claude-thermos keeps your Claude session warm for you

 https://github.com/izeigerman/claude-thermos

by s0ck_r4w

7/23/2026 at 8:58:02 PM

This is just making it more expensive for everyone else, right?

How Claude handles its sessions is none of my business. I'm going to let them do the best they can to provide good service for everyone, and if they can't/won't, I'll switch to a provider that can.

Using these massive models is already pretty danged extravagant, I'm not going to demand to be at the front of the queue at all times, too.

by SwellJoe

7/24/2026 at 4:05:56 AM

This has nothing to do with queueing.

Cached input tokens cost 10% of uncached. So if you’re model runs for 45 minutes, generates 300k output tokens and asks you a question, it costs 10x more if you wait 5.01 minutes to answer.

Sure, you may be willing to pay 10x more (or get 10x less for your subscription). But the time limit is arbitrary and has nothing to do with other peoples’ workloads. So I think your point is a non sequitur.

by brookst

7/24/2026 at 4:50:28 AM

> But the time limit is arbitrary and has nothing to do with other peoples’ workloads

No it has to do with others workloads, now you keep their cache for longer so others will get less. And no its not arbitrary, they run out of memory, if more people do this they will have the dial it down further or run out of capacity.

by Jensson

7/24/2026 at 1:15:16 PM

Explain to me how “every gets 5 minutes, 24/7, regardless of load” is remotely compatible with your statement?

Are you imagining this a fixed MRU where duration scales with usage? Becasue that is not at all what Anthropic documents: https://platform.claude.com/docs/en/build-with-claude/prompt...

You would not get more than 5 minutes if you were the only user in the world. You would not get less at their peak hours.

by brookst

7/24/2026 at 5:35:45 AM

1) To be clear, it costs you 10x more for uncached input tokens _for the next call_, which are still 5x cheaper than output tokens.

2) Now imagine Anthropic or OpenAI now charge your per minute of reserved VRAM time. It would be more fair if they did. Would you still want to run a tool like this?

by searealist

7/24/2026 at 1:15:50 PM

Totally agree that if the economics changed, the economics would change.

by brookst

7/24/2026 at 7:12:09 AM

> has nothing to do with other peoples’ workloads.

Because ram/memory is free and not in demande at all these days?

by toasty228

7/24/2026 at 2:39:34 AM

No? You still pay for cache reads... Actually this is actually MORE expensive than just redoing prefill if your context is below 100k.

by himata4113

7/24/2026 at 1:54:07 AM

You PAY for cache writes. Holding the prefill in VRAM for a certain period of time is part of what you're paying for.

by dannyw

7/24/2026 at 2:22:38 AM

Right, but also using the cache refreshes/extends it, which means you can pay for a write once, then have it extended for very cheap (1 token).

by gruez

7/24/2026 at 4:00:06 AM

You dont have to repay per hour? It seems like that is a design issue - Like yes we will keep these in whatever cache you want, but you will have some sort of renewal period

by spacemanspiff01

7/23/2026 at 11:53:41 PM

I agree with you. And sadly, there's nothing we can do about it because it's an arms race. People will try to exploit loopholes to get more benefits and companies will try to prevent exploits--sometimes by making the service worse for normal users.

In this specific case, Anthropic can avoid keeping the cache if it detects this kind of prompt (i.e., if max tokens < some number).

by GMoromisato

7/23/2026 at 9:53:38 PM

From the README:

> Detect the danger window. When the main lineage goes idle and a subagent is actively running, the main prefix is at risk of expiring.

So, this is not demanding to be at the front of the queue, it's just paying someone to take the place you already had in the queue, when you want to take a leak.

by devnonymous

7/23/2026 at 10:01:09 PM

A subagent running doesn't mean that much, though. I often walk away while the subagents are chewing.

by SwellJoe

7/23/2026 at 10:13:35 PM

The defaults seem sensible. This isn't keep-alive-forever by default solution.

by devnonymous

7/23/2026 at 9:22:23 PM

How do a few extra requests with the same prefix make it more expensive for everyone else?

by s0ck_r4w

7/23/2026 at 9:33:08 PM

Keeping a conversation with a very large model active requires hundreds of GB of memory. If my conversation can never be swapped out, like when I go to lunch or take an afternoon walk, that's several hundred GB not available for other users. If everybody does that, Anthropic needs even more infrastructure than the quarter trillion dollars in infra they're already using.

by SwellJoe

7/24/2026 at 1:55:54 AM

But you pay for cache writes, it's not like you're abusing anything, you're simply choosing to call a billed API for the explicitly intended purpose of writing/refreshing cache.

If you keep this running for hours without doing anything, it will drain your limits and API. The use case of keeping the main thread cache warm while subagents work is very genuine and legitimate.

by dannyw

7/23/2026 at 9:41:12 PM

The idea here is that your conversation is being swapped out for no good reason. The session is still active, as the subagents continue to make progress.

How is this comparable to going to lunch or taking a walk?

by s0ck_r4w

7/23/2026 at 9:48:33 PM

Is that actually happening, though?

by SwellJoe

7/23/2026 at 9:59:40 PM

Yes https://github.com/izeigerman/claude-thermos/blob/main/src%2...

by devnonymous

7/23/2026 at 11:24:38 PM

CC actually prompts Claude about this by default in the ~20k system prompt and instructs it to avoid 300s timeout and to be mindful of the 300s cache expiration.

by idonotknowwhy

7/24/2026 at 2:19:46 AM

CC also will also block sleeps longer than eg 300s; the harness handholds the model quite a bit.

by dannyw

7/23/2026 at 11:48:43 PM

The prefix cache is a resource shared by all users. This is basically a tragedy of the commons.

by kakugawa

7/24/2026 at 2:17:44 AM

Compute is also a resource shared by all users. When your prompt isn’t cached, your input needs to go through prefill; which is compute-bound.

The cache is discounted for a reason. They WANT you to use it.

by dannyw

7/24/2026 at 5:44:27 AM

They could always just extend the cache timeout beyond 5 minutes themselves. They don't do that because it is an expensive resource and there is a trade-off bewteen saving computation and reserving VRAM. Running a tool like this will force them to do one of the following:

1) Start charging for VRAM reservations.

2) Charge _other_ customers more.

3) Eat the cost themselves.

by searealist

7/24/2026 at 6:54:08 AM

They charge you for cache writes already. There's a price assigned to it. Even if you're on subscription, it counts against your usage (very easy to observe with a big cache miss).

Anthropic (and now OpenAI too for 5.6) prompt caching is not free.

by dannyw

7/25/2026 at 2:26:53 AM

Holding your cache in VRAM for 5 minutes or 3 hours have very different costs to them.

"They already charge me to park my car, why can't I leave it there for a year for the same price as 1 week?"

by searealist

7/24/2026 at 4:08:04 AM

So why is it 5 minutes at both peak and off peak hours? If they can guarantee 5 minutes at peak hours for everyone, why is it only 5 minutes at off peak?

I don’t think your understanding works.

by brookst

7/24/2026 at 12:08:39 AM

If true, I suspect Anthropic will begin suspending accounts that use it.

by fearmerchant

7/24/2026 at 2:19:20 AM

I highly doubt that.

Keeping your cache warm is a good thing, caching saves compute and electricity.

Cached input is cheap for a reason, it is in everyone’s mutual interests to maximise cache hit rates.

by dannyw

7/24/2026 at 2:22:53 AM

If that was true then Anthropic would have built this functionality in themselves.

by OccamsMirror

7/24/2026 at 4:09:32 AM

They did. On any long running query with large context, tell Claude “give me a status report every 4 minutes to keep the cache warm”. It’s entirely happy to oblige and usually says “good idea, that will keep costs down”

by brookst

7/23/2026 at 9:28:11 PM

you use more memory?

by sznio

7/24/2026 at 1:42:55 AM

You’re already paying for the subagent, why should you pay twice? (Once for subagent, and again for waiting for the subagent). That’s double billing.

by engineer_22

7/23/2026 at 10:06:55 PM

Eh, I assume Anthropic prices queries somewhere in line with their actual costs. If they don't, and they misalign incentives instead, that's kinda on them.

by pkulak

7/24/2026 at 2:31:24 AM

hey guys dario’s here!

by icase

7/24/2026 at 2:35:12 AM

There's no need for insults.

by SwellJoe

7/23/2026 at 6:27:01 PM

I've directly inspected calls for pro/Max plans and as of today they have 1hr cache expiries. This has definitely degraded to 5 min in the past but that's the behavior today.

If you're paying API rates, you can choose 5m or 1hr yourself (and pay different rates).

Keeping a 1hr cache warm could still be useful, sure, but outside that, I don't see much use of this today.

by unholiness

7/23/2026 at 6:51:10 PM

When using API keys the default is still 5 minutes. Setting 1 hour for the entire session is actually a lot more wasteful considering the 2x rate that it comes with

by s0ck_r4w

7/24/2026 at 1:14:23 AM

FYI, on Pro and Max plans caching lasts for one hour, not five minutes, unless you're currently using Extra Usage.

https://code.claude.com/docs/en/prompt-caching#on-a-claude-s...

(Thank you to EliasWatson for giving me this link just a few days ago, as I was previously confused too.)

by Wowfunhappy

7/23/2026 at 6:03:20 PM

Feels like will be shut down real quick

by alukin

7/23/2026 at 6:14:46 PM

Yes, this wastes cycle. They dump your cache and deallocate the VMs so that others can use it. This will result in tokens wasted and downtime for others.

by gogobio

7/23/2026 at 7:53:20 PM

Can you expand on this? I can't imagine this is holding up an entire machine or materially impacting capacity. Isn't it common to have a tiered cache and to ultimately evict (or move to another tier) if there's an active inference request and no available capacity elsewhere?

by supern0va

7/23/2026 at 8:00:20 PM

Yes, however if a tool like this commoditizes a way to stay "higher" in the cache hierarchy to avoid getting dropped, then for the same total capacity, Anthropic have to become stricter about retention.

by Lalabadie

7/23/2026 at 8:32:29 PM

It's unfortunate that there isn't some better way to signal that there's a high chance that a turn is coming (ie, due to active sub-agents). If CC could send a ping saying "I don't need a turn, but please keep this cached at some tier because there will be a turn soon." that would probably help with cache efficiency. Instead, it seems like there's just the shotgun one-hour solution.

by supern0va

7/23/2026 at 6:06:10 PM

Why? Seems fine if people want to burn more cache reads.

by l1n

7/23/2026 at 6:10:26 PM

Because it makes Anthropic earn less money

by agluszak

7/23/2026 at 6:40:25 PM

> Because it makes Anthropic burn more money

Fixed that for you.

by munk-a

7/23/2026 at 6:31:28 PM

"Earn"?

Cache duration is arbitrary. What it actually does (if used en masse) is decrease the amount of oversubscription their infra can handle..

by cadamsdotcom

7/23/2026 at 7:54:02 PM

Earn is synonymous with profit, and this hurts their profits. Introducing latency can benefit much more people and actually make usage more efficient. Reducing latency can sieze up a functioning system.

Lately, I’ve been thinking about how this related to fractional banking. If you were to eliminate fractional banking introduced in the US by Hamilton, you would destroy a lot of current prosperity.

by janderson215

7/24/2026 at 3:04:58 AM

It’s unknown whether our current economic model will work out long term. It mostly relies on the US being the default reserve currency for most nations. Hopefully that lasts a long time.

by sillysaurusx

7/23/2026 at 8:45:05 PM

Agreed, the Thermos company's lawyers will probably fire off a terse letter once they find out.

by Cyberdog

7/23/2026 at 9:25:14 PM

Interested in how the critics of approaches like this defend an agentic session (with Fable, for example) that stops and runs a multi-hour ML training session. It's a script, so the actual LLM convo goes stale, but then when the results get returned to the main thread you get an expensive cache hit without doing anything.

You would have avoided that cache hit if the LLM session was kept "alive" for those few hours. Why not automate the part where you keep the large main thread alive until you're ready to analyze the results?

by boc

7/23/2026 at 11:59:38 PM

I think you mean miss?

by PcChip

7/24/2026 at 1:51:45 AM

If you're running the subscription, by default you are paying 2x for cache writes and you're getting an hour for expiry. So refreshing based on 5 min is wasteful. You need to detect whether you are in 5 min or 1 hour mode.

by leemoore

7/23/2026 at 6:23:46 PM

Is the 5-minute expiration correct? I thought it was more like ~1 hour.

by jonas21

7/23/2026 at 6:36:38 PM

You can set the retention globally (whole session) to 1 hour which will actually make it more expensive. The default is 5 minutes.

*UPD:* actually it appears the default is authentication-dependent. API key gets 5 minutes, subscriptions - 1 hour.

by s0ck_r4w

7/23/2026 at 9:18:26 PM

Claiming that it's more expensive isn't true, it's workload dependent. Filling the cache (worst case) 12 times in an hour is much more expensive than caching it for an hour, and the cheapest is to have your 5m cache refilled by a ping like this is doing.

Imo it's their fault for not having pricing that aligns incentives.

by foota

7/23/2026 at 9:42:34 PM

I assumed (perhaps incorrectly, but it was a guess since I never dug into it) that less used "hot pockets" of previous inference gradually got more stale as time went on, and the conversation went elsewhere and didn't need those bits.

Hearing one byte refreshes the whole thing is huge! 5min is wayy too slow, because sometimes I want to spent more than 5 min looking at a diff before choosing where to go next.

Kind of outrageous, I hope this kind of feature gets built into claude code =D

by purpleidea

7/23/2026 at 7:10:24 PM

How will this not lead to tragedy of the commons?

by cosmotic

7/23/2026 at 7:18:37 PM

Exactly. I really wish people wouldn't use this. If this becomes popular, anthropic will just modify their cache policy to be much less fair. It's not like they have infinite cache.

by davesque

7/24/2026 at 1:16:55 AM

Well, it's not a free lunch, you're still paying for the cache-warming request. Most of that request will be cached by definition, but cached requests merely cost less, they aren't free.

by Wowfunhappy

7/23/2026 at 6:23:47 PM

This is a good idea if it could manage it in an acceptable way.

For example, there might be something I intended to complete in one sitting, but took two sittings in the same day unexpectedly. Maybe it could just be a few cache delays per day or something, tagged in advance somehow.

by j45

7/23/2026 at 9:33:09 PM

They should increase the cache to 10 minutes. 5 is just too low, you can even miss it by taking time to select a response from a question.

by jaimehrubiks

7/23/2026 at 10:03:20 PM

10 mins would be amazing

by gabigrin

7/24/2026 at 1:30:47 AM

Yeah I do this manually by compacting the convo. The TTL is about an hour though, definitely not 5 minutes.

by skeledrew

7/23/2026 at 6:29:51 PM

Glad this exists. It will force anthropic to fix their flawed cache mechanism.

by ATMLOTTOBEER

7/23/2026 at 9:01:00 PM

While Anthropic is far behind on caching and efficiency (made more dramatically apparent by how much cheaper it is to use GPT 5.6 Sol at API rates than even Opus 4.8, much less Fable), a bunch of people forcing their way to the front of the queue at the expense of everyone else isn't going to solve that.

by SwellJoe

7/24/2026 at 1:41:14 AM

They definitely aren't. Over 98% of my tokens over the last 6 month have been cache reads (did an analysis just a couple days ago), which has made my Pro plan highly valuable.

by skeledrew

7/24/2026 at 6:26:52 AM

I certainly don't see much in the way of cached token rates when I use the API, whereas when I use DeepSeek via API, it's mostly at the cached token rate.

by SwellJoe

7/23/2026 at 6:44:20 PM

How should they fix it?

by cortesoft

7/23/2026 at 7:17:06 PM

Make the cache TTL more adaptive. Have more tiers than just 5m and 1h. Long-running subagents become increasingly ubiquitous. There's no reason for Anthropic to not do a better job for this use case.

by s0ck_r4w

7/23/2026 at 7:53:05 PM

There are a lot of workflows where cache is likely to be consumed exactly once (e.g. anything chat-like where a single thread is extended; if a prefix cache hit is found, and a set of new longer prefixes is inserted in the cache, it is unlikely that the original shorter prefix will be consumed in later turns). I could see allowing each session key a small number (1?) of 24-hour cache entries, where inserting a new cache entry (perhaps the maximum-length prefix, perhaps an API-tagged specified-length prefix) consumes that slot and either boots the rest of the items from cache, or demotes them to 5 minutes or something. Basically: workflow awareness, not uniform handling.

by addaon

7/23/2026 at 10:18:09 PM

Default to 1h. Allow setting it to longer or shorter, granularly. Add /pause to mark it for eviction, for a token refund.

by randomblock1

7/23/2026 at 10:09:33 PM

I thought the cache length was 1 hour, not 5 minutes?

by 2001zhaozhao

7/24/2026 at 1:53:36 AM

it could be either. subscription defaults to 60 min with 2x cache writes. extra usage and api defaults to 5 min with 1.25 cache writes. You can override either with settings

by leemoore

7/24/2026 at 6:13:43 AM

Yes! Accelerationism cuts the other way too.

by sublinear

7/23/2026 at 6:27:29 PM

Don't we pay for cache input though?

by razodactyl

7/23/2026 at 6:57:15 PM

You can send and receive 1 byte and refresh the cache. Keeping the cache alive isn't free, but it's close.

by broodbucket

7/24/2026 at 1:55:11 AM

if you are refreshing a cache of 120k tokens, you have to input the same 120k tokens. you can't bounce the cache with 1 byte. A proper cache hit requires your entire context that is cached. if you change 1 byte in it, everything after that byte is a cache write and no longer a cache hit

by leemoore

7/24/2026 at 2:07:21 AM

Yes, you're paying the cache read, not the cache write, which is much more expensive. You can do the math for how many 5min refreshes you can be afk for until it starts costing instead of saving

by broodbucket

7/23/2026 at 6:44:08 PM

The idea is this keeps it in cache so you don’t have to pay to re-input.

by cortesoft

7/24/2026 at 12:43:01 AM

Nice! Codex’s TTL on 5.6 is, I believe, 30 minutes, and when combined with how generous they are with resets of weekly limits and having completely gotten rid of 5hr limits, means Claude Code is just a total rip off right now. Not to mention their silent Fable nerfing and Chinese fearmongering.

by edot

7/24/2026 at 9:35:35 AM

[dead]

by IrfanD

7/24/2026 at 3:10:39 PM

[flagged]

by smokeeaasd

7/23/2026 at 6:25:38 PM

[dead]

by szin