Accelerating GPT-5.6 Sol Ultrafast

8/13/2026 at 6:55:09 PM

I've been waiting so long for something amazing to come out of the OpenAI and Cerebras collaboration.

> In our evaluations, GPT-5.6 Sol on Ultrafast mode answered all 2,500 HLE questions in 11 hours and 11 minutes. Claude Fable 5 needed 78 hours and 27 minutes, more than three days of continuous compute, to arrive at the same conclusions. In other words, Ultrafast worked through the frontier of human knowledge in a single working day, achieving comparable accuracy nearly 7× faster.

This is actually insane.

Hopefully the release ultrafast of Terra and Luna too.

by iamcoder18

8/13/2026 at 10:17:21 PM

I discovered yesterday that the “amazing thing that comes out of OpenAI” is Sol, due to its token efficiency.

Dollar for tokens, Sol and Fable are the same price.

However, Sol uses (literally: in testing) around 10-100x less output tokens compared to Fable for the same task.

We run our frontier models nearly 24/7, so switching to Sol will save us around $500 per day.

And, due to less guardrails, Sol also performed better, and we lost less tokens due to guardrails shutting down sessions (I feel like it’s illegal to take $50 of someone’s token money and then shut down a session with guardrails before they get an answer, and yet Anthropic do it to us constantly… either take our money and commit, or trigger the guardrails immediately)

by aetherspawn

8/14/2026 at 12:18:00 AM

We’ve literally saved tens of millions of dollars already (no exaggeration! already 8 digits) by switching to Luna for many workloads at my company.

The amount of workloads we can shift with an advisor model pattern continues to grow.

It’s seriously amazing.

by dannyw

8/14/2026 at 2:13:31 AM

Luna came out about a month ago, you're saying that the cost saving from switching to Luna has saved your company $20 000 000+ in 1 months spending on API usage?

by TranAndrewA

8/14/2026 at 5:33:15 AM

OpenAI and Anthropic are believed to be making billions per month in revenue selling tokens to enterprises. It’s not that surprising for enterprise SaaS they would have individual customers making up ~1% of their total sales. Although, perhaps a bit more surprising those few customers are posting about it here.

Another interpretation would this is a counterfactual savings, like they previously paid $1M for y tokens, and now that tokens are cheaper they increased usage and paid $1M for 20*y tokens.

by ashdksnndck

8/14/2026 at 9:52:49 AM

A second angle on the counterfactual savings would be Luna telling them not to pursue a potential session with the expected/extrapolated (from which sessions they did ignore Luna on, e.g. just to keep efficiency statistics current) sunk costs at time of getting shut down used to derive the quoted number.

by namibj

8/15/2026 at 11:24:20 PM

A third angle is that it is not true and that's just some attempt at influencing an HN thread, which is happening quite regularly pro and against AI. We know that the pro-AI has a way bigger budget to burn with this kind of operations tho.

by brazukadev

8/14/2026 at 2:26:24 AM

It makes sense at a company that sells AI as a service. Perhaps to make presentations...

by rokob

8/14/2026 at 2:16:56 PM

The ones with terrible slides on their MTA ads?

by kissickas

8/14/2026 at 3:15:07 AM

Canva generates billions in annual revenue, it's not unbelievable

by aexer0e

8/14/2026 at 6:27:28 AM

By "advisor model pattern" you mean https://claude.com/blog/the-advisor-strategy ?

How do you implement that outside of claude code?

by krzyk

8/14/2026 at 9:37:52 AM

the easier way to do this, with any harness, is to use an expensive main agent that is told to delegate all code reading, writing, exploration, research etc to weaker subagents to conserve tokens. It's an inversion of the pattern but the resulting split is the same

by awestroke

8/14/2026 at 12:30:10 PM

Yeah, that one is easy, but I was thinking of the other way around, where the weak model calls stronger one.

by krzyk

8/14/2026 at 1:16:38 PM

I append this to many of my opus claude code prompts

`You may use a Fable subagent to answer questions, solve problems, and provide an adversarial review of your ideas and code`

You can use a similar pattern in most any harness, and you can tell them to use other harnesses. In claude you can write `Use codex cli to have Sol56 Xhigh provide an adversarial review to your plan before presenting it to me` or `Use opencode cli with GLM 5.3 to verify all code review findings before presenting` or whatever you're doing, as long as those other tools are setup and ready to be called.

IMO: This isn't useful as a token saving pattern in my experience with agentic engineering, but it is useful as a quality-enhancer.

by criley2

8/14/2026 at 2:00:34 AM

what has a single company accomplished with tens of millions of token spend?

by jgalt212

8/14/2026 at 6:14:26 AM

A pelican on a bike accurate to a subatomic level

by akoboldfrying

8/14/2026 at 4:07:35 PM

But the knees still bend the wrong way

by literalAardvark

8/14/2026 at 3:47:10 AM

higher valuation

by moomoo11

8/14/2026 at 7:53:28 AM

What are you using it for?

by grey-area

8/14/2026 at 5:57:54 PM

[dead]

by Incipient

8/14/2026 at 3:35:21 AM

Evidently, Claude's tokenizer vocabulary size is ~15k[1]. On one hand, it's quite mind blowing. On the other hand, Anthropic models' token (in)efficiency makes a lot of sense in that light.

[1]: https://xcancel.com/magikarp_tokens/status/20878591737488549...

by woadwarrior01

8/14/2026 at 5:27:05 AM

Not just that, they normalize everything into lowercase and use a special character to capitalize words (what about languages with non-trivial normalization/capitalization?) and mark beginning and end of each word, all of that diluting already small vocabulary. That smells like manual tuning of what should be done statistically, I wonder what technical merit they saw in that - I know they mentioned better generalization, but this is pretty counterintuitive.

by orbital-decay

8/14/2026 at 10:04:38 AM

Sadly it's not too counterintuitive; remember the old "how many r's are in the word strawberry"?

Also different tokens for the same named entity/concept if they almost entirely exclusively occur in non-overlapping contexts, and are themselves rare/uncommon in the first place, will result in behavior that's similar to the speech/phrasing/vocabulary registers humans exhibit, where the aspects of the named entity/concept get largely compartmentalized.

The most severe case along these lines were the old BERT models that ran over straight UTF-8 bytes (plus a handful special tokens).

But for the modern post-GPT2 LLMs such radical simplicity seems to mostly not be considered suitable. Note that CJK (the big one in particular, so Chinese semantic and Japanese Kanji) encodes each one into multiple UTF-8 bytes giving some automatic scaling for semantically dense languages; similar effects also apply to e.g. APL code.

by namibj

8/13/2026 at 11:06:14 PM

Sol is way cheaper than Fable by the token.

by resonious

8/13/2026 at 10:27:30 PM

Wait isn't Fable like 2x more expensive if we compare under 272k tokens

by minraws

8/13/2026 at 10:54:30 PM

The comment you're replying to almost feels like it was written by a bot or am I crazy?

by ChadMoran

8/14/2026 at 12:18:15 AM

Peak under your skin a bit. Something weird is going on. I think we are bots/robots(sic)

by theplumber

8/14/2026 at 2:00:42 AM

Related, Under The Skin with Scarlett Johansson is an incredible movie.

by satvikpendem

8/14/2026 at 5:16:18 AM

[flagged]

by ekabod

8/14/2026 at 5:30:55 AM

Peek*

by archon810

8/14/2026 at 10:18:09 AM

beep boop, everyone thinks I’m a robot. :|

by aetherspawn

8/13/2026 at 11:01:36 PM

What kind of bot would say `less guardrails` instead of `fewer guardrails`? I guess someone could instruct an LLM to deliberately make mistakes, but isn't that too paranoid?

by jlund-molfese

8/14/2026 at 12:59:47 AM

We’re reaching transvestigation levels of people trying to spot AI text everywhere they look

by qgin

8/14/2026 at 11:42:39 AM

Hmm, this sounds like something a bot would say to prevent being caught.

by vitorfblima

8/14/2026 at 11:43:14 AM

...Does a bot know the word "transvestigation"? Would a bot be allowed to say it by its corporate overlords?

by freeone3000

8/14/2026 at 11:48:22 AM

It does now that is on hn. Thank you for your input.

by vitorfblima

8/16/2026 at 8:58:00 PM

15 years ago i was astounished by the intellectual deepness of this community. Now i'm aware this has always been a cult and their former cult leader Mr. Altman wants to destroy this capitalist society. He isn't even hiding motives. People just stopped listening carefully.

by siva7

8/14/2026 at 1:57:20 AM

In some romantic languages less and fewer are the same word, so it is a common mistake for people that have them as their mother tongue.

by giovannibonetti

8/14/2026 at 10:56:52 AM

In German the distinction is also different from what English speakers sometimes make.

by eru

8/13/2026 at 11:56:02 PM

“include common grammatical imperfections and awkwardness common in casual message board interactions.” I’m not saying that’s what’s happening here, but a high school student told me that’s basically what they do to make papers not sound like AI.

by DrewADesign

8/13/2026 at 10:57:54 PM

I agree. Weird to use <“> and <”> characters. Maybe written on phone, but certainly not on keyboard.

by w4yai

8/13/2026 at 11:35:27 PM

Other languages use different characters for quotes, if anything that's an indication that's not written by a LLM because it's not favoring the standard English character.

https://en.wikipedia.org/wiki/Quotation_mark#Specific_langua...

by jrflo

8/14/2026 at 9:35:56 AM

Written “on an iPhone” - yep, seems to automatically switch the quotes.

by aetherspawn

8/13/2026 at 11:11:35 PM

Common on non us keebs though.

by gunalx

8/13/2026 at 11:31:57 PM

You could say the token usage is "load-bearing".

by gtree

8/14/2026 at 5:00:14 AM

Its token usage has earned its keep.

by LASR

8/14/2026 at 5:18:43 AM

One thing I've deliberately deferred is drawing conclusions from all of this.

by aenis

8/15/2026 at 10:32:47 AM

[flagged]

by focxle

8/13/2026 at 7:58:43 PM

Answering 2,500 independent questions is an embarrassingly parallel workload, all it needs is scale out. It would be more meaningful to know how much time was required for a single complete answer to a difficult HLE question.

by zozbot234

8/13/2026 at 8:04:50 PM

The animation in the article shows GPT-5.6 Sol Ultrafast solving a question at 3 secs vs Fable's 27 secs. The "11 hour vs 78 hours" animation implied that they were solved in sequence and not in parallel

by desmondl

8/14/2026 at 12:44:20 AM

> Answering 2,500 independent questions is an embarrassingly parallel workload, all it needs is scale out.

I believe the assumption of the benchmark creators is that it's meant to measure the sequential speed that a single instance of the LLM and piece of hardware can get through all the tasks from start to finish, like running a race. As a rudimentary comparison, sort of like doing prime number calculations as a benchmark of the CPUs in one bare metal server. Of course you'd get a speedup in total number of primes searched if you ran the same software of GIMPS on 8 servers with the same hardware in parallel rather than 1 server.

Of course if you took all the individual questions in humanity's last exam and fed them in parallel into separate queries to Claude that land on separate hardware instances of the claude model you'd get a speed up. Because each question is independent and not related to knowledge/calculations that are performed in any other question it is indeed very open to speed up by breaking it into separately dispatched parallel tasks.

by walrus01

8/13/2026 at 8:22:44 PM

>Answering 2,500 independent questions is an embarrassingly parallel workload

So is calculating the total time required to answer all of the questions.

by jcims

8/13/2026 at 8:22:54 PM

Amadahl’s law dictates it’s approximately always better (as in, more efficient computationally) to have one super fast thing than many slower things doing the same job in parallel.

by baq

8/13/2026 at 8:28:23 PM

Yes. but for embarrassingly parallel tasks it barely matters.

by xyzsparetimexyz

8/13/2026 at 9:13:18 PM

The task the majority of people here care about (and which makes up a huge part of AI lab revenue these days) is collaborative coding with a single human which is not embarrassingly parallel.

by modeless

8/13/2026 at 10:47:46 PM

I am aware. I was talking about HLE. Talking about how fast you can run through HLE in a serial manner is like talking about how how fast your non-simd single threaded cpu ray tracer is. Its simply not how any sensible person would tackle this problem.

by xyzsparetimexyz

8/13/2026 at 11:56:58 PM

[dead]

by senordevnyc

8/14/2026 at 10:58:23 AM

Maybe not embarrassingly parallel, but can be surprisingly parallel.

by eru

8/13/2026 at 8:26:11 PM

Such a deep and smart insight.

But why is that important if they're measuring time?

by moralestapia

8/14/2026 at 2:52:49 AM

And just like that, GP disappears forever ...

by moralestapia

8/14/2026 at 2:15:36 AM

750t/s tells the story. This isn't purely (or even mostly) driven by parallelization.

by rpdillon

8/13/2026 at 8:05:12 PM

i assume they meant serially....

by whimsicalism

8/13/2026 at 10:40:47 PM

Presumably when they're talking about benchmarking, they're using a fixed amount of resources, like everyone else does when benchmarking?

by hgoel

8/13/2026 at 9:45:52 PM

I'm finding Luna suprisingly adequate for my work. I slept on it due to the benchmarks, but it's very fast and even on low reasoning I'm finding it more than adequate for "menial" work. (The speed is crucial for "interactive" work -- if a model is fast enough it goes from "async" to "real time", subjectively, which is a huge difference.)

In fact, I'd say it's overqualified for the kind of work I'm doing, because it spends >half the time verifying trivial changes (and the verification isn't as helpful as you'd expect, even with bigger models).

Maybe I can prompt it to be less aggressive about that (the new GPT models do it even without prompting).

Anyway, Ultrafast Luna would be amazing, though I strongly doubt they can offer Cerebras at anything approaching the current prices. Now we wait for Moore's Law? :)

by andai

8/13/2026 at 11:33:24 PM

Luna is incredible considering the cost. It’s very good for routine work.

by shepherdjerred

8/14/2026 at 1:16:30 PM

Fast models is why I was hoping Taalas would get their butts into gear and eventually release a consumer priced card. I'd love to have a pcie card that screams along at 15k t/s even if on a heavily quantized 2026 level model forever.

Faster & cheaper tokens = more reasoning capability and more reasoning = better problem solving as far as I have seen.

by fennecfoxy

8/14/2026 at 2:40:04 AM

Luna and Terra are great, but so are DeepSeek Flash, MiMo, Qwen 3.6, and even Gemma for some things.

MiMo Pro has had UltraSpeed for a while.

by trollbridge

8/14/2026 at 9:54:56 AM

I'm considering a home project done in Rust. I've not even touched Rust before, I'm a Java dev. It'll be something reasonably complicated, using a bit of unsafe etc. Do you know how Luna does writing Rust code?

by munksbeer

8/13/2026 at 10:04:46 PM

I feel odd to use these models, because it feels like a faster model doesn't feel that much faster if it spends reading files, making edits and running checks.

It feels too situational.

by epolanski

8/14/2026 at 2:23:23 PM

Which one of those takes time?

by andai

8/13/2026 at 11:24:45 PM

Luna has impressive results on DeepSWE.

I mostly split work between Luna and Sol. If something seems simple enough I always try it with Luna first.

by phoghed

8/14/2026 at 3:55:36 AM

Sol medium/high planner orchestrating -> Luna xhigh subagents doing implementation

...has been REALLY good for me. Even on xhigh, Luna is crazy cheap.

Subjectively I'd say it's way better than Sonnet at a fraction of the cost. Luna xhigh can do some decently challenging things on its own, but when orchestrated by a model that is actually good like Sol, I am finding it very very nice.

by JohnBooty

8/14/2026 at 6:02:36 AM

How are you doing orchestration - using sol for plan mode in codex? Or some other pattern/harness?

by wrikl

8/14/2026 at 1:16:52 PM

The cool kids have custom harnesses and workflows and stuff, yeah. I'm still using Superpowers in Codex. Planning in Sol, Luna subagents. https://github.com/obra/superpowers

I feel like I could be doing a lot better somehow. Regardless though Luna (xhigh specifically) is super good/cheap/fast for a lot of things

what about you

by JohnBooty

8/13/2026 at 7:00:12 PM

Feels like the 90's again where single threaded speed is improving fast. ASICs and wafer scale rather than node shrinks, but end result to me the consumer feels the same.

by piyh

8/14/2026 at 6:35:29 AM

I wish, but I don’t think this qualifies as single thread performance. Cerebras is whole wafer computation though right? So you get the result sooner, but the same amount of computation has occurred, just with huge parallelism.

by beng-nl

8/13/2026 at 9:00:10 PM

Yep, https://en.wikipedia.org/wiki/Andy_and_Bill%27s_law

by sscaryterry

8/13/2026 at 8:19:35 PM

Output from Cerebras with GPT model is 750 tokens per second.

Don’t blink.

(Chatjimmy has 14,200 TPS.)

by sixtyj

8/13/2026 at 8:27:28 PM

ChatJimmy is a much smaller model and, AFAIK, has no reasoning capability. Absolutely insane raw speed, like a supercar, while Sol is more like a freight truck.

by tomrod

8/13/2026 at 8:40:10 PM

At such output speed, I wouldn’t expect reasoning. (But I didn’t know it, thanks.)

700 TPS with reasoning is awesome and it speeds things up.

Cerebras as public traded company is worth keeping an eye what they produce.

by sixtyj

8/13/2026 at 9:02:08 PM

> At such output speed, I wouldn’t expect reasoning.

As the sibling comment to yours mentioned, if they had a reasoning model “hardware-ified” onto a custom chip (as is their plan for IIRC this or next year, a new ASIC), it’d output fast decode speeds for the regular output as well as reasoning sections. Both would be ≈equally fast.

by msdz

8/14/2026 at 12:00:17 AM

Yeah, I thought reasoning was literally just chain of thought in the output token stream, with the model itself adding delimiters to indicate what part of the output is internal reasoning, and what part is an answer to the user. Is that wrong?

by senordevnyc

8/14/2026 at 7:31:41 AM

No, as the sibling comment mentioned, your understanding was correct there.

What’s more, the only technical difference in speeds could be, and likely also is with the HC1 chip, between prefill (prompt processing) and decode (text generation) speeds. I don’t know whether it’s the case with Taalas’ chip, but in the “software-based” LLMs we typically see and use so far, those two stages hit different parts of a computer (processing/compute-bound vs. memory/bandwidth-bound).

by msdz

8/14/2026 at 12:10:03 AM

You are right, reasoning is unrelated to tokens per second.

by beering

8/13/2026 at 8:52:46 PM

Anything will be fast if you etch it straight to silicon

by notfromhere

8/13/2026 at 9:24:37 PM

The knowledge of ChatJimmy is terrible. Even Qwen on my iPhone is better.

by dzhiurgis

8/13/2026 at 10:58:02 PM

ChatJimmy isn't a model, it's Llama 3.1 8B hardwired into silicon. The point isn't to be a good llm, but to showcase the speedup that's possible

by headPoet

8/14/2026 at 12:41:03 AM

well, yeah, it's based on a 2+ year old tiny model. It's very much an alpha proof of concept that they can perma-bake an LLM into silicon.

https://huggingface.co/meta-llama/Llama-3.1-8B

by walrus01

8/14/2026 at 12:03:31 AM

Haha, at first I thought you meant that the knowledge of the existence of an LLM that’s so fast is terrible because it’s ruined every other LLM for you!

by senordevnyc

8/13/2026 at 10:43:06 PM

Never heard of it before, that's fucking insane.

Apparently they baked the Llama 3.1 8B model weights [0] into silicon (the actual hardware is called Taalas HC1).

I guess for the trillion parameter models this would not scale due to cost? Imagine buying GPT 6 in the form of a PCI-E card, pulling these speeds, with up to 120 cct agent sessions. It'd be beyond wild.

[0] the weights are also using some cut down small format, but HC2 will have regular FP4 supposedly, and support for 20B params on one die

by perching_aix

8/14/2026 at 2:09:17 AM

> Never heard of it before, that's fucking insane.

They've been acquired by AMD. Those saying the model sucks are completely missing the point: it was a proof-of-concept.

The question is: what happens to a model like Anthropic's Fable 5 that does, what, 70 tokens/s (and requires lots of output tokens) when the latest open-weights model is etched on silicon and does 14 000 tokens/s?

Shall the better model still have the upper hand or will the raw speed compensate?

by TacticalCoder

8/14/2026 at 3:51:47 AM

    Shall the better model still have 
    the upper hand or will the raw speed 
    compensate?

At 14,000 tokens/sec there's just so much ridiculous stuff that might be possible. Let's assume that this POC proves they can take the next step, and can eventually etch a capable ~27B model into silicon. Let's call it Fred.

Ralph loops automatically get real real interesting again. 200x the iteration speed. This is such a clear win I feel like there's hardly anything to talk about. Instead of one stubborn iterating idiot, you could have dozens of idiots competing in parallel, genetic algorithm style.

The other common orchestration pattern I see is "big model for planning, small parallel subagents implementing, big model reviewing" Today it's Sol dispatching a handful of Luna subagents. Tomorrow maybe it's Sol dispatching as many Fred subagents as it could possibly want.

But what patterns have we not even thought about yet in a world where subagents are 200x faster/cheaper?

What if instead of dispatching single Haiku/Luna/etc subagents, we dispatched "teams" of Fred agents? Maybe each team is 8 Freds. Five come up with competing ideas and the other three vote on a winner.

Or what if they were heterogenous teams? One Luna and a bunch of Freds.

What if instead of a two-tier orchestration system (Sol->Luna) it was three-tier or n-tier? (Sol->Luna->Fred->...Fred)

Those ideas overlap a bit, and crazy shit like Gas Town has already explored even wilder ideas I guess. But man, 14000 tk/sec opens up so much stuff.

by JohnBooty

8/14/2026 at 5:28:24 AM

At 14,000t/s that's effectively a motor cortex for an android, you no longer need to train the robot to walk, it has a general idea for how to walk (baked into the 1b model), and then just corrects based on sensor input, in real time.

by hadlock

8/14/2026 at 1:22:55 PM

I still personally think that a heavy lean into MoE will be better for that sort of thing. Our brains are subdivided into large parts but I'm sure (and I'm not a brain scientist) that those parts can be subdivided even further into systems that run at various frequencies and latencies depending on what they're used for.

I was thinking about it the other day actually. How our brains evolved structure. I imagine it was purely just down to evolution adding/clustering additional cells around the areas where additional cells were needed. And after long enough a natural brain architecture emerged.

Makes me wonder if we're on the right track with transformer architecture/attention but if it'd be more effective on a larger scale, like MoE with a billion "experts".

by fennecfoxy

8/14/2026 at 11:32:47 PM

    like MoE with a billion "experts".

That seems promising to me too, although, the thing I've always read is that you can't make the "experts" too narrow. Even if you had a "coding expert" it has to know a lot more than coding - if you tell it to make an online store it needs to parse your language, understand the internet, what a "store" is in this context, etc.

I am not a primary source, probably not even a secondary or tertiary source, so take this with all the grains of salt.

by JohnBooty

8/19/2026 at 11:13:27 AM

Oh for sure, but I think generally multiple experts are selected in an MoE pass for a token, so presumably it'd select programming related ones as well as general knowledge/language.

Only problem would be the routing layer works on the previous token as far as I understand so it might need more informational depth than just "a token" to select experts, I suppose in the same way attention works.

I wish I had the GPUs to run those sorts of experiments ha ha.

by fennecfoxy

8/14/2026 at 7:04:57 AM

Yes, imho 14k TPS is just a beginning.

This Gas Town? https://github.com/gastownhall/gastown

by sixtyj

8/13/2026 at 9:58:26 PM

Unfortunately AMD bought them, so I don't think we will get to see another release from them.

by mips_avatar

8/14/2026 at 7:34:07 AM

Aha, thanks, that’s fresh; press release is from Aug 6

https://ir.amd.com/news-events/press-releases/detail/1296/am...

by sixtyj

8/13/2026 at 8:57:31 PM

Been waiting since Cerebras-GPT. [0]

[0] https://news.ycombinator.com/item?id=35490837

by rvz

8/13/2026 at 9:32:07 PM

An irrational gripe of mine is how GPT uses 7× instead of 7x.

I recognize that the former is the multiplication symbol, but I don't think it should be used that way.

by christoff12

8/14/2026 at 12:16:36 AM

If anything, writing "7x" is actually just a poor emulation of writing "7×".

Saying something is "done at 7x speed" should be read as "done at seven times speed" not as "done at seven x speed". So using the 'times' (multiplication) symbol is the better form in my opinion; it just happens to be significantly easier to type "x" instead, which is how we got here.

by Daneel_

8/14/2026 at 2:11:12 AM

But speech evolves. And we all pronounce "ten ex", not "ten times". Actual speech usage very clearly pronounce the letter 'x'.

So it's 10x. And no need for Unicode codepoints.

by TacticalCoder

8/14/2026 at 3:55:01 AM

I fully agree that speech evolves, but maybe this is just in the US? I've never really heard anyone say it here in Australia, it's usually "times", although admittedly almost no one uses the phrase/format here.

by Daneel_

8/14/2026 at 4:19:32 PM

Ah, yes. My gripe is certainly US American flavored.

by christoff12

8/14/2026 at 7:51:29 AM

A number plus an “x” is used beyond Trumpistan. I have never in my life heard anyone say “ten ex”.

by shussjjs

8/14/2026 at 1:43:40 PM

Have you heard the expression "10x developer"?

by _flux

8/14/2026 at 6:01:07 AM

Interesting I always thought that "x" (whichever) is used informally for multiplication and proper sign is dot: 3 · 4 = 12

It looks like there is a difference between English speaking languages and the rest in that regard.

by krzyk

8/14/2026 at 4:28:37 PM

Funnily enough, I didn't even have the speaking aspect in mind.

I don't like how the "times" symbol floats off the line -- it's a visual thing for me (again, irrational).

by christoff12

8/13/2026 at 7:52:39 PM

Seems like they will do Sol first while capacity constrained? I can't imagine the margins they'll be charging

by wrsh07

8/13/2026 at 9:03:17 PM

I think this is what they need to actually have a margin :)

by sscaryterry

8/14/2026 at 5:54:50 AM

Isn't ultrafast just making hundreds of subagents?

by krzyk

8/14/2026 at 10:47:54 AM

No, that's Claude "ultracode"

by anentropic

8/13/2026 at 9:05:25 PM

People underestimate the importance of speed on quality of thought, because people underestimate just how much quality is a result of simple iteration.

When an LLM thinks, it typically just makes one pass. It outputs tokens from top to bottom, beginning to end, and then it's done. But when people think, especially strong thinkers, we typically iterate and revise our thoughts on the fly. We do numerous passes. We stop and restart, we reconsider, we review, we reevaluate. Sometimes we do this so quickly and automatically that we don't even realize we're doing it. I think a lot of what separates a highly intelligent or effective person from others has less to do with the quality of their first pass and more to do with just how many additional passes they're able to do in the same amount of time, and of course what kind of criteria they're habituated to consider during their review passes.

Introspecting about this is difficult, but experimenting with LLMs is easy. First, simply ask an LLM to do something complex. For example, to come up with a new business idea, or to plan the next month of your life, etc. After it finishes, tell it:

"Review what you just wrote, according to some appropriate list of evaluation criteria that you come up with first. And then, based on the results, iterate and generate a better response if warranted."

It's insane how much better the next answer will usually to be. Often it'll catch and erase tons of hallucinations, logical errors, and inefficiencies. And you can simply copy-paste this again and again until you begin to hit diminishing returns. Or, in a harness like Claude Code, for example, I might shortcut this whole process by saying, "Use sub-agents to iteratively review and iterate on your work until convergence."

The reason why most people don't prompt LLMs to do this (besides simply not thinking of it) is that it takes time.

But what if it didn't?

What if the LLM's response came back in milliseconds rather than minutes? Then there would be almost no reason NOT to do this. In fact, one could almost imagine it baked into the assistant/harness -- a massive step change in practical quality, enabled by nothing more than speed.

by csallen

8/13/2026 at 9:14:23 PM

Going back between two different company's AI tools when facing a tricky architecture question often surfaces holes in an approach I'd been building.

Similarly, if I ever get a bit too vibey and don't carefully review code changes myself, the blast radius is generally significantly resolved by a carefully tuned "did you consider x, y, and z" skill after a first draft partnered with a "deploy an adversarial review agent for the worktree".

by loufe

8/14/2026 at 2:29:07 AM

When I was doing some important work, I had 2 worktrees, each running a frontier model from a different company, and I passed their responses between them in markdown files and had them critique them. I thought they'd prefer their own solutions, but they frequently preferred the other model's solutions.

I went back and forth like this until we converged on a solution "everyone" was satisfied with.

---

Context: I'm a solo dev and working outside my area of expertise, so I'm leaning heavily on LLMs. It's not great (definitely too vibey for my taste, and I keep running into issues) but the alternative is spending the next few years studying several specializations instead of shipping. So this is the "least bad" thing I could come up with.

I'm definitely increasingly making time for "learning sprints" to catch up on specific knowledge gaps. For example today I spent 2 hours debugging something because I was missing a fact that would have taken me 1 minute to learn...

I went from "fully hand crafted" to "fully vibed" (when Fable came out), back to "fully hand crafted" (once I realized I no longer understand the code!), and now I'm at "making very careful use of AI, asking for the smallest possible changes, and double checking everything"...

by andai

8/14/2026 at 11:00:52 AM

> I thought they'd prefer their own solutions, but they frequently preferred the other model's solutions.

I suspect that's downstream from their sycophancy.

by eru

8/14/2026 at 2:17:30 PM

Ha, maybe. Sample size was low but usually both models ended up preferring the same solution.

by andai

8/14/2026 at 5:38:49 PM

I had a similar situation set up where I would have two models come up with their own independent diagnoses and plans to fix problems.

Then I would have them each read the other's plan. But I would tell them each something like this, "I had a friend look at this too. He's smart, but in general you're smarter and more knowledgeable than him, so don't be afraid to say where he's wrong."

by csallen

8/15/2026 at 2:05:38 PM

I even tell them to give me an adversarial review and tell me what's wrong with the approaches.

by eru

8/13/2026 at 9:56:45 PM

Just putting more details in, and giving it ways to check itself brings massive improvements. I often ask model to set up test for the problem before actually trying to solve it and it improves it a lot, both in how much babysitting is required (if it can test it itself quickly it goes faster), and the fact the context now contains more detailed description of the problem that came up when making tests.

Turns out TDD is far better for robots than humans, who knew

by PunchyHamster

8/14/2026 at 3:13:42 AM

100% this. I do some work where frontier LLMs really are quite bad on a first pass... yet with what I consider a basically silly agent architecture which sort of amounts to "do this three ways, check, revise, choose the best, rethink, check, check, revise ......" I get to remarkably good results. It was through this process I realized.... humans do this, on the fly, without thinking.

I end up spending a lot on inference, it's incredibly slow, and the architecture really does seem like overkill at first glance. But it works magic.

by danielmarkbruce

8/14/2026 at 2:56:23 AM

That's an argument for cheap massive parallelism, which is what's behind the recent math breakthroughs, not expensive low latency serial compute.

by energy123

8/14/2026 at 3:45:18 AM

I vaguely recall some of the recent math breakthroughs being less about parallelism and more about serial compute. A lot of, "Keep trying," "Try harder," "Try another way," etc.

Regardless, I think both things are great, horizontal (more approaches) and vertical (deeper approaches). And both are made more practical with faster inference.

by csallen

8/14/2026 at 3:54:39 AM

Those results were found with 64 subagents, or with GPT 5.6 Sol Pro which uses parallelism under the hood.

Low latency is a big deal but the immediate use cases are somewhat different in the short-term (more serial coding workflows) rather than pure math research which is effectively massive-scale search through a tree of possibilities, which is where you want throughput and low cost per token, rather than high speed per token.

by energy123

8/14/2026 at 3:27:04 AM

> Review what you just wrote, according to some appropriate list of evaluation criteria that you come up with first

I get better results from these models when I ask for the appropriate list of evaluation criteria with a fresh context. If you pollute the context with its first iteration, then you are likely to get a worse result when you ask it to come up with the criteria with the first version in the context. Context contamination can unintentionally narrow the expertise of the inquiry (even for meat humanoids).

It's unfortunate that Cerebras disabled new sign-ups for their coder plans. GLM-4.7 on Cerebras via OpenRouter used to be absolutely amazing...

I am very eager to see 15,000 tokens/second eventually, like Talaas but for higher intelligence models. I know a few people working on ASICs in this direction including open-source projects. It's all extremely exciting.

by kanzure

8/13/2026 at 9:59:55 PM

This is foundationally similar to a lesson I've found from years of pre-LLM software development: builds that turn around in 500ms instead of 5 minutes fundamentally change the way you can work as a software engineer. I think a lot of the same applies to working with LLMs. I'm not sure, though, what the path from where we are today to some future state of high speed token abundance actually looks like...I think there's plenty of chance that we see the bubble pop in the near term over token costs and complexities of today's infrastructure, then some totally different landscape of LLM use in 5-10 years that looks quite unlike what we have today, similar to how waiting 30 minutes for an MP3 of a single song to download on a 28k modem in 1999 seems quaint today.

by macNchz

8/14/2026 at 2:22:20 AM

There was one point in my career where the setup took a couple of hours so I'd only get two, maybe three chances per day. I reviewed the code very carefully before I hit send.

by fragmede

8/13/2026 at 10:25:27 PM

What stops the LLM not to iteratively think and expound upon before emitting the final tokens?

by navigate8310

8/13/2026 at 10:39:04 PM

Nothing at all. They're not designed to, so they don't. Change that, and they would.

The question is the wrong one. The right question: why aren't frontier models designed to work that way? The answer: it's slow and expensive.

The other answer: that's basically what you're selecting with "Medium", "High" and so on, how many tokens they'll blow on muttering to themselves before they get back to you with an answer. There's more to it, but not that much more.

by samatman

8/14/2026 at 2:08:37 AM

Reasoning models will frequently backtrack and re-assess what they've said so far. That's one reason test-time scaling is so powerful.

by maleldil

8/13/2026 at 11:42:29 PM

> But what if it didn't?

Or what if it output the final result in the first place, without having to repeatedly prompt it to check its own work to trick it into a better answer?

by bluerooibos

8/14/2026 at 2:56:08 AM

Maybe having the optionality is better? I could imagine a world in which it does what you say, and people wish it didn't, so they have both the option to get the fast and slightly worse answer + the option to ask for reviews to get slower but better answers. And more control over how those reviews are performed. Which is the world we live in today.

by csallen

8/14/2026 at 9:46:57 AM

> In fact, one could almost imagine it baked into the assistant/harness

...it already is, via thinking/effort. That wouldn't be possible if a LLM at usable quality wasn't fast enough to allow at least some amount of "thinking" (i.e. hidden text generation).

But your point still stands: we could get massive quality gains by allowing even more thinking by default, if it was fast enough.

by cl3misch

8/13/2026 at 6:37:13 PM

The corresponding OpenAI post https://openai.com/index/previewing-ultrafast/

There is no pricing info, which could mean it's "if you have to ask..." territory or they are simply gauging interest before deciding

by GodelNumbering

8/13/2026 at 8:30:00 PM

They're nearly certainly going to use it internally to speed up research that is serially bottlenecked. I would bet this is why they're interested in the Cerebras partnership more than everything else

by 2001zhaozhao

8/14/2026 at 5:38:28 AM

Yeah, the main value of this for OAI is probaby to speedup RSI

by coder-3

8/15/2026 at 9:12:16 PM

i would assume internally they have even better tps

by desterothx

8/13/2026 at 6:44:31 PM

They're expanding access to companies that apply for the program and explain their use cases. So it's very real but limited imo.

by rirze

8/13/2026 at 7:02:24 PM

The stake in the side of cerebras has always been that the economics are pretty poor.

Who knows if they will subsidizes it to mitigate sticker shock, but it's a safe assumption that it will be scarily expensive. However if you are in a "cost is no obstacle, speed is god" position, it will likely be pure magic.

by WarmWash

8/13/2026 at 7:08:39 PM

Can anyone explain why Cerberus needs to be _fast_ instead of _cheap_?

I don't think I understand why they aren't leveraging the increased speed to do batching to serve more customers at a "normal" tok/s.

Is the limitation, even on cerberus, still that the cache can only serve so many concurrent sessions over time? Is there no scaling advantage? I genuinely do not understand how any of this works.

by fcarraldo

8/14/2026 at 5:34:56 AM

For us, it would be for SRE stuff. We have agents reviewing traces and logs, and inspecting system behavior daily - for non obvious problems, not surfacing in metrics. When we hit an issue, we use the /fast mode to triage, propose a fix and then build and deploy. It is trivial amount of money all things considered, and I'd happily authorize a 100x spend for when we have a prod outage on a mission critical service.

When you think about it, it would still be dirt cheap compared to normal way of doing things. In the old days, if you had an outage on a serious user facing system, you'd wake up people across various timezones, wake up their managers and scramble to find the root cause, identify a solution, brainstorm on possible side effects of a fix, and then rush to build it and deploy. This cycle would involve, sometimes, dozens of people, for, say, 10 man hours each. So lets make it 120 man hours per serious outage, and lets assume and average of $100 per hour - so, $12,000 per a serious outage fixed under a day, counting conservatively and not including the costs of the actual outage.

I'd guess the pricing for those ultrafast, very energy inefficient and hardware heavy models will be competing with that. Its going to be possible to get a fix out in 30 minutes, 10 of which will be tests, 5 will be the deploy, and the remaining 15 will be some unlucky guy trying to keep up with the super fast model throwing a 50 "load-bearing deferrals earning their keep" per minute :-)

The pricing on those things is competing with costs to run entire departments. I'd, for one, imagine offshore ops teams will be a thing of the past in under a year, since one gets way better initial response to anything from a model, given right setup, esp. on codebases that have been built from the ground up with agentic coding - so with good documentation and effective test coverage baked into repos.

by aenis

8/13/2026 at 7:47:26 PM

There are many industries and situations that will pay absurdly high multiples on API rates for low latency access. Anything remotely related to finance, deal flow, or trading, including the development and checking of Excel financial models. Live analytics that can be delivered mid-meeting to C-suite executives, or ultra-luxe dealmakers who need answers while on a live call. Intelligence and defense, where milliseconds matter.

With the tool calls that can be done, you're not pricing this against an executive assistant or pocket analyst - you're pricing this against the ability to have an entire Bourne Identity style analysis room at your disposal. The limited inventory will go to the people for whom money is no object.

by btown

8/13/2026 at 8:02:30 PM

Please write your own posts

by combustion

8/14/2026 at 1:26:01 PM

For what it’s worth, I typed every word of my comment above by hand. You’re welcome to see every post that uses sentence fragments, parallel structure, and not-X as a sign of AI, rather than a style learned by a human from decades of personal essay writing and forum discussions. I think that’s a limiting way to see the world.

by btown

8/13/2026 at 11:33:48 PM

Please provide some, any, evidence or reasoning for your accusations.

Otherwise it’s just lazy. I know shallow dismissals is kind of HN’s thing, but come on, a little effort please. Currently, your comment is just as much slop

by phoghed

8/13/2026 at 8:33:24 PM

Cerebras is a large plate sized chip. It has 50GB of SRAM, and few hundred K simple cores that can access that SRAM really fast. I don't know semiconductors well, but I understand that the same manufacturing technique that makes this huge chip possible, on the flip-side limits inter-chip communcation bandwidth. In cerebras, it is 150 GB/s (compared to nvlink's 2TB/s or groq's similar).

One way large models are served on a bunch of cerebras chips is by essentially distributing layers' weights across chips. Few layers's weights per chip - as many as the KV cache + activations + weights will allow. You use pipelining to hide the latency of the inter-chip 150 GB/s link.

On GPUs, you amortize the cost of loading weights from HBM to SRAM across multiple users - thereby making it cheaper _per_ user. But here, there is no such amortization. The weights are already there. It is the activations that stream through.

You _could_ do batching/continuous batching, but that would just service more users at lower token/s each without any amortization of fixed cost, due to fixed cost (loading weights) being non-existent.

by porridgeraisin

8/13/2026 at 7:30:10 PM

I have no idea, but I've assumed that batching can't work on Cerebras.

Batching works because of severe memory bottleneck, but Cerebras whole thing is serving models out of "L1 cache" (?).

by petu

8/13/2026 at 7:36:59 PM

You'd need hundreds of GB alone for the KV cache of each user. For something like LLama 3 405B you need ~67GB at ~130k tokens. A single CS-3 has 44GB on-chip sram.

So, afaik, Cerebras are optimizing for ultra-low latency batch=1 inference.

https://newsletter.semianalysis.com/p/cerebras-faster-tokens... goes quite in-depth.

by philipportner

8/13/2026 at 8:56:57 PM

LLama 3 405B had the most unoptimized kv cache usage by far. Deepseek v4 pro uses 2.4GB for the same context length[1].

[1]: https://vllm.ai/blog/2026-04-24-deepseek-v4

by YetAnotherNick

8/14/2026 at 8:57:40 AM

Good point, thanks! I haven't been keeping up with most of the new model internals.

by philipportner

8/13/2026 at 7:22:59 PM

They're cache limited, almost certainly, so more slower sessions doesn't solve the problem - you still have to load and unload the whole cache hierarchy at some level and that's a network bandwidth and memory bandwidth problem between the external systems and the waferscale chip.

Also worth looking into how they do cooling for it, because that's kind of absurd and awesome as well.

by jaggederest

8/13/2026 at 7:39:13 PM

> I don't think I understand why they aren't leveraging the increased speed to do batching to serve more customers at a "normal" tok/s.

There's some technical hypotheses about it that other people are offering.

But also from a business perspective, it totally makes sense not to go any sort of batching play. It's really valuable and very clear to consumers to make your pitch entirely about lower latency rather than higher bandwidth.

There are so many scenarios that are latency-constrained that will be difficult or even impossible for someone even with fleets of high-bandwidth compute to compete with you on.

Very easy pitch to sell a customer who asks what differentiates you from other companies: you pay us a premium for lower latency than anyone else.

by dwohnitmok

8/13/2026 at 9:00:52 PM

To use a car analogy, they built. Ferrari and technically you could put a tow hitch on it and go slow and carry a bunch of cargo instead of 1 passenger and go real fast, that's just not what they've built. There's one really fast core to send tokens at and get tokens back out at insane speeds.

by fragmede

8/13/2026 at 8:16:29 PM

Because companies are willing to pay proportional to (tokens/s)^k for some value of k >1

by UltraSane

8/13/2026 at 7:20:45 PM

Unless I have read over it, besides the animation in the intelligence vs speed graph which only mentions internal data and not whether they truly reran the AA suite, there is no actually solid statement on the important aspect of performance.

Neither the Cerebras or OpenAI post [0] outright state that this performs exactly the same as regular 5.6 Sol. I feel if this was 1:1 just Sol but much faster, they'd (rightfully) scream that off the rooftops. A line such as "this is the same performance, just faster, with no downsides" would go a long way in clarity and communication. Along with no pricing information, I'll hold out on further information.

[0] https://openai.com/index/previewing-ultrafast/

by Topfi

8/13/2026 at 7:33:58 PM

"delivering up to 750 output tokens per second and without any quality compromise" seems pretty definitive.

by Scaevolus

8/13/2026 at 7:38:42 PM

No quality compromise/degradation is something I have had this industry, including especially OpenAI, claim multiple times in the past and I have more than once been able to verify that it was in fact not the case. Examples being gpt-3.5-turbo vs text-davinci-003, GPT-4-Turbo and all the other post training checkpoints they had under one name (which was a major bug bear for me back then witnessing degradations with no naming change, industry got better in transparent checkpoint naming since), Opus 4.6 Fast Mode (which just was faster by skipping much of the required work), etc.

Same for massive performance differences in the way providers like Cerebras, Groq, etc. have deployed models including K2.6 on Cereberas specifically. Massive deltas in tool call and overall quality despite there being far more clarity in open weight vs proprietary model deployment.

The AA suite graph with that animation is the only time in either post that absolute parity is being asserted and I'd be amazed if that was the case, but am doubtful why their phrasing is so cagey.

Why not assert full parity in writing? It "performs the same (within run-to-run variance) across all evals that Sol has been tested with" is very different to "no quality compromise/degradation", the later allowing for a lot more wiggle room and interpretation in what evals you use to assess that, what quality truly means, etc., the former meaning identical in all situations.

Could also be a language barrier here in fairness, maybe this phrasing is more iron clad than I give them credit, but especially with OpenAI, I have seen enough checkpoints asserted as unchanged in "quality" to where I am skeptical. Ironically, I never saw that with Anthropic (which has gotten far more heat for degradation accusations) while a model was deployed with one exception in mid-late April this year. Pure speculation, but believe it wasn't noticed much before "agentic coding" became more popular, because chat output is far more subjective without a rating framework vs code passing which can be an objective metric with more potential for frustration.

by Topfi

8/13/2026 at 7:52:14 PM

The problem is the compromise is probably based on their tests for key things, but for the rest of us, it means changing how we talk to the model or the flow we use to adjust for quirks, but then its fully about the same.

by giancarlostoro

8/13/2026 at 8:49:41 PM

No one claimed gpt-3.5-turbo doesn't have any degradation over davinci-003. In fact it was quite obvious that gpt-3.5 had way less knowledge but more post trained to be helpful.

by YetAnotherNick

8/13/2026 at 9:30:30 PM

That quite strong "no one" surprised me so I checked and looking through a few blog posts from back then, they did advertise gpt-3.5-turbo as a straight up improvement and, once text-davinci-003 was to be deprecated, the instruct tuned variant as the drop in replacement [0]. If anything, they did not just promise similar performance but actually an improvement ("our best model") when compared to text-davinci-003:

> It’s also our best model for many non-chat use cases—we’ve seen early testers migrate from text-davinci-003 to gpt-3.5-turbo with only a small amount of adjustment needed to their prompts.

That's why I still remember this so well, they claimed one model to be their best and a straight up drop-in during deprecation when in my (back then even more amateurish then today) testing this was plainly not the case. A model cannot be "best" if it's measurably worse in many situations, then what was still available at the time.

[0] https://openai.com/index/gpt-4-api-general-availability/

[1] https://openai.com/index/introducing-chatgpt-and-whisper-api...

by Topfi

8/13/2026 at 9:05:51 PM

There's probably some quantization and other processing needed to get Sol to fit on a Celebras chip, so it's not an exact 1:1 copy of the Sol model running on Nvidia GPUs, because it's not running on GPUs doing abstract math calculations anymore, so they can run benchmarks at them and say it's similar enough, but no one believes in benchmarks in this new field of performance that's more computer astrology than computer science.

by fragmede

8/14/2026 at 12:15:52 AM

Gotta build some personal benchmarks if you don’t trust the public ones. But the well-known public ones, despite their flaws, are generally high signal on model intelligence.

by beering

8/13/2026 at 10:23:24 PM

It’s suspicious that they repeat “without any quality compromise” instead of just outright saying it’s the exact same model. It’s like a motel that insists they have clean beds.

by wat10000

8/13/2026 at 10:31:20 PM

Or an 'asbestos free!' label on a cereal box

https://news.ycombinator.com/item?id=49260655

by madeofpalk

8/13/2026 at 11:31:55 PM

I assume it’s because there has been compromise with previous models on their hardware

by phoghed

8/13/2026 at 7:56:11 PM

This just means, we once saw it run at 750 tokens per second, in this insane test config :)

by sscaryterry

8/14/2026 at 12:14:39 AM

You (or anyone else) can just benchmark and compare. If they were serving a dumber model it would be trivially detectable.

by beering

8/13/2026 at 10:16:13 PM

This is what Cerebras does- take other people's models and run them very very fast.

by conception

8/14/2026 at 5:51:11 PM

I mean I still regularly use 5.3 spark (the cerebrus model) that comes with my sub to do rapid reviews of 5.6's work and it finds oodles of problems in about a minute.

by cududa

8/13/2026 at 6:37:33 PM

> Compared with output speeds reported by Artificial Analysis GPT-5.6 Sol on Ultrafast mode runs 11x faster than Fable 5, and 5x faster than Opus 4.8 on Fast mode.

Awesome work. I'm personally very excited for faster models/inference.

I think speed is underrated to some degree in the current conversation. For a while, I was using Cursor's Composer quite a lot, even over frontier models, just because of how darn fast it was.

by wxw

8/13/2026 at 7:20:06 PM

What do you need speed for? That's a genuine question, I feel like the limiting factor already is my creativity, attention span and budget. And I'm not even yet optimizing cost by batching things like review to slow local models over night, or schedule tasks to take full advantage of my subscriptions.

by arw0n

8/13/2026 at 7:38:40 PM

> attention span

IME waiting for an agent to work through a problem is a detriment to attention span; your mind drifts to other things while you wait. Maybe you can steer several agents in a round robin instead, but then there's a cognitive tax from context switching. Faster models mean fewer gaps in focus.

by evmaki

8/13/2026 at 10:03:26 PM

There are speed thresholds that open up new use cases.

Imagine speeding up current agents 10x, you switch from directing agents to pair-vibing on the fly.

Speed them up 10x more, and you get a SOTA model capable of analyzing and rethinking your entire file in between your key strokes. That would make for one hell of an autocomplete.

Pivot over application, going from coding to anything else, and this can easily give computers features previously impossible to make. In video games, fully general characters reacting realistically to arbitrary dynamic situations. In "serious" apps, interactive work with a system that understands your goals and adapts to you on the fly. Hell, even an OS that can tell you "hey, the data you're obviously looking for is in the tab over there, now highlighted".

And that's just tip of the iceberg. I'd personally love to explore the possibilities.

by TeMPOraL

8/14/2026 at 4:41:22 AM

Do you not have many many separate projects happening in parallel because it takes forever for the model to respond, so you give it your feedback and jump to the next one? That context switch is challenging and expensive. Imagine if your feedback was nearly instantly applied and you could just see the result? It would be more like the holodeck metal table scene in Star Trek "Schisms".

by planckscnst

8/13/2026 at 7:36:22 PM

I just finished waiting almost four hours for Fable to write 700 lines of code, based on my three paragraph prompt. Some speed on these harder tasks would definitely be welcome!

It also spent almost 800k tokens on these lines…

by black_knight

8/13/2026 at 8:17:51 PM

I'm very curios what the code is doing.

by UltraSane

8/14/2026 at 5:42:34 AM

It is a normalisation procedure for inductive and coinductive data types.

Basically, the code is a function which takes in an expression where you can use generic data structures as variables and then some specific data structures, and it plugs them in for the variables. It then computes the structure of the resulting data type.

So, admittedly not a trivial task – hence the choice of Fable as the model. Also, this would have taken me few days to do by hand! So, we are living in the future! But one could always wish for more speed and more intelligence.

by black_knight

8/14/2026 at 7:05:36 AM

What language?

by UltraSane

8/14/2026 at 11:19:31 PM

Agda. Fable and Opus both know it quite well now. They need some guidance to create good-looking code, but man, they know how to grind!

by black_knight

8/14/2026 at 12:00:03 AM

> I'm not even yet optimizing cost by batching things like review to slow local models over night

well, I am.

look at what the market thinks of CPU manufacturers and general computation now that agentic workflows have taken up, all went to the moon after being picked over in favor of GPUs and RAM for years

most computers have been idling, waiting for human input, for decades, and if there was a computationally intensive process it was offloaded to GPUs a long time ago, over the last decade, so CPUs and general processors have remained idle, relegated to just defined conditional statements to switch between tasks with no reasoning capability to occupy compute

now, there are reasoning capabilities to tell a CPU what to do (as a byproduct of the varied processes). Cerebras is not a CPU, it is a special purpose chip for inference, but is hosting LLMs that tell CPUs of all its clients what to do faster than a human can. Outside of Cerebras, LLMs are not doing much to optimize compute of the system they're affecting, as they're reading or compiling code when being used for coding, very few processes are intensive and the CPU is just waiting as if a human was using it because the LLM can't digest and output information fast enough. The CPU ecosystem is very mature for general and varied tasks, but is underutilized.

To the what: any kind of compositing or configurations that humans do, agents can do. AutoCAD, video editing, sequencing in music, all forms of media, all forms of configuration done digitally. right now they rely on snapshots to see and react, and this increases the 'framerate' per say, and rapid and relentless iteration they can do.

by yieldcrv

8/13/2026 at 8:01:00 PM

You can get stuff done live on a call.

by peyton

8/13/2026 at 9:18:11 PM

Humans have human attention spans and there's a lot of study going back decades for UX design because of this. < 100 Ms is instantaneous, 100-300 ms is noticable but still responsive. At 1 second, flow.gets interrupted, 2-5 seconds, you're clearly waiting, 5-10 attention wanders and 10+ seconds, you've lost them. The 0.1 / 1 / 10 second rule comes from Jakob Nielsen's HCI work. Perceived latency matters almost as much as actual latency, which is why chat interfaces drip out/stream words instead of just dumping out the answer at the end. At 750/tok/s, for Sol grade inference, it can spend 3 seconds on thinking tokens before outputting something to the user for a better answer while still feeling usable.

by fragmede

8/13/2026 at 6:43:15 PM

I've been using DeepSeek flash a lot this week to try it out. Now, I deeply want the smart frontier models to be just as fast.

by kilroy123

8/13/2026 at 7:11:13 PM

The omission of Mimo v2.5-Pro Ultraspeed, released in June, which can achieve 1000tok/s is an interesting flaw in the comparison graphs.

It is a bit outdated (scores ± 40% lower), but smart enough for a lot of coding tasks, and can cost under 1/10th of Sol.

https://mimo.mi.com/models/en-US/mimo-v2.5-pro-ultraspeed

by ricardobeat

8/13/2026 at 8:55:33 PM

This does look pretty incredible, but don't forget that incredible token thoroughput can only necessarily solve certain bottlenecks. If your e2e tests take an hour, they'll still take an hour after Ultracode. If the agent runs a 10 minute typecheck after a change, that will still take 10 minutes. grep over a massive codebase is still just as slow, etc. I say this not to take away from this accomplishment but just to ensure everyone here keeps a clear head about what it means - 14x faster tokens does not mean it completes every task 14x faster.

I suspect Humanity's Last Exam is without tool-calls, making it kind of the perfect benchmark to highlight how fast Ultrafast is, but not really the same as the everyday work you or I do.

by johnfn

8/14/2026 at 5:25:14 AM

I've been measuring waiting for tool calls/waiting for model response in my OMP with Sol 5.6 and usually it's 85%-95% of time spent waiting for model to respond, so 14x speedup in model perf would still be very significant. YMMV but speeding up tests and improving DX is somewhat well understood.

by fireant

8/13/2026 at 6:43:49 PM

This is really cool. Someone here commented about similarity between this and hardware advancements for AV encode/decode.

I think it's only a matter of time before miniaturization can have a thumbnail sized user-replaceable accessory that contains the LLM built onto the hardware. I admit I don't know how any of that works, but would be amazing to experience. Fully local, fully offline, ultra fast local inference better than any personal computing product.

by thraway3837

8/13/2026 at 6:52:41 PM

https://chatjimmy.ai/ Is that. Company behind it just got acquired by AMD

by christkv

8/13/2026 at 8:19:02 PM

In 5-10 years nice smartphones will be able to run ChatGPT (~gpt3-4) class models. A memory rich laptop (highend mac/framework) can run GPT-OSS:120b or full Gemma4 at very interactive speeds.

High end phones can already run the smaller models at enough speed to be probably useful, especially for background/overnight photo tagging and curation and things like that.

by datadrivenangel

8/14/2026 at 7:27:46 AM

In 5 years the models will probably be so much better and more compact that phones will be running models equivalent at least to Opus 4.6 if not Fable, at least within the areas they are tuned for (which probably won't include coding).

by raylad

8/14/2026 at 4:46:28 PM

I'm really excited for this future. Both what you said and datadrivenangel.

The fact that Apple shipped a more than capable laptop for most of the population using a last generation iPhone chip is just mind blowing. Silicon advancements are going to allow this, and I think the global majority will catch up and make their own chips that compete or exceed western performance. Especially when the US is scared of science, rapidly divesting and defunding it.

by thraway3837

8/13/2026 at 8:31:15 PM

> GPT-5.6 Sol on Ultrafast mode, delivering up to 750 output tokens per second

https://taalas.com/products/

> delivering 17k tokens per second per user on Llama 3.1 8B model.

Obviously this is a much smaller model, but I really can't wait for ASICs to take over the LLM space.

Imagine running a model like Sol/Fable (even half the size with 60-70% of it's intelligence) on your own ASIC hardware.

by tristanMatthias

8/13/2026 at 8:59:57 PM

ASIC makes it sound like it's a single chip, but in reality serving trillion-param models on Cerebras requires a full cluster (as in multiple racks, MW of power).

Some interesting twitter analysis here:

https://x.com/bleysg/status/2073937651150029084

by mNovak

8/13/2026 at 8:35:07 PM

I'd take qwen3.6 (3.8 as of tomorrow) 27B running at 17k per second first on the way to Sol/Fable! And then dsv4-flash-0731!

by auspiv

8/13/2026 at 8:52:15 PM

Right! Not sure that will happen since Talaas was just bought by AMD, so more likely a US model. Still, we can dream.

by tristanMatthias

8/13/2026 at 9:04:49 PM

Indeed, you could literally brute-force it that way :)

by sscaryterry

8/13/2026 at 8:33:04 PM

Good news for Intel and AMD.

Rught now on large scale codebases the bottleneck is both claude/codex inference, as well as time it takes to run tens of thousands of tests. We put those workloads on dedicated epyc 9005 build machines - but it still takes minutes per run. Those who can afford the fast tokens will be in the market for faster CPU that money can buy today.

by aenis

8/13/2026 at 8:50:04 PM

why intel and amd ? these are cerebras wafers?

i know people are joking about the sol ultrafast prices (its unlikely to be accessible for average joes) but this shows scaling wafer cores works for inference boost

which makes me very excited, sol ultrafast will be as slow as it will get if that makes sense. at these token speeds , we will see a much deeper economic impact.

by zuzululu

8/14/2026 at 4:19:16 AM

On large codebases, literally anything I throw at Fable or Opus takes at least 20 minutes, 10 of those minutes is spent on running test suites. With this way of working, I put way more emphasis on test coverage, integration tests, behavioral tests, in-browser tests -- and I run them on every change, before any commit. Faster CPU solves this, while faster inference solves the other 10 minutes. I can't imagine how cool it would be to work with this ultrafast mode, without those mental-context-purging pauses.

by aenis

8/14/2026 at 4:44:04 PM

gotcha, we got faster inferences but now CPU has to deal with the tasks.

by zuzululu

8/13/2026 at 9:19:32 PM

Right now the bottle neck is not the CPU, so people aren‘t spending big $ on them. But with this ultra fast mode, CPU becomes a bigger part of the bottleneck and thus Intel and AMD can charge more $$$.

by _diyar

8/13/2026 at 10:28:49 PM

I noticed it myself. It's not just that the models are faster, but that by becoming more capable, they can take on larger tasks, which require larger compute.

by gpt5

8/14/2026 at 4:20:53 AM

I am settling on a way of working where during the day I spent the time iterating requirements, designs and low level implementation plans for, say, a dozen features, and then I park them for an unattended build during the night. It works well.

by aenis

8/13/2026 at 7:16:57 PM

Whoa. This looks both powerful and expensive.

My prediction is that, this time next year, top developers outside ai labs will be spending 50k USD+ on inference.

Within labs, I've heard spend is already far beyond this per developer.

by owentbrown

8/13/2026 at 8:43:16 PM

> top developers outside ai labs will be spending 50k USD+ on inference

I think it is more like top companies, not top developers, and the problem with developers in top companies was and is - absolute majority of them are not actually directly working on things that increase revenue, so companies can spend a ton of money and see barely if any changes in the product and the bottom line, so companies, at least legacy ones will be reluctant to sponsor that long term.

by maxnevermind

8/13/2026 at 7:36:46 PM

50k per month?!

If someone subsidize maybe, but if the companies need to pay no way, unless there is hard evidence of the return.

by vb-8448

8/14/2026 at 6:57:00 AM

Probably per year, still a crazy amount.

by Squarex

8/13/2026 at 9:36:32 PM

Isn't TCO lower with Cerebras chips compared to Nvidia? Theoretically, most developers should eventually be running on Ultrafast.

by cactusplant7374

8/13/2026 at 7:53:06 PM

not sure how "top developers" are defined here, but there is huge diminishing return curve starts kicking in after $200/month price point for typical eng work.

by andriy_koval

8/13/2026 at 9:47:00 PM

I agree with the diminishing returns on spend, but worth noting that when on an Enterprise seat and paying API rates, I'd say that you can easily spend above 200/mo before seeing the curve begin to flatten

Obviously there are a ton of ways to spend money / tokens and people have different levels of experience that will put this ceiling at very different levels for different people.

by obmelvin

8/14/2026 at 9:14:01 AM

I think that we are starting to see that API inference prices for US labs are excessive and subscription prices are closer to real costs so 200/m can be realistic longer term.

by fireant

8/13/2026 at 9:31:59 PM

If you're talking Opus pricing, it's more like $200/day.

by svachalek

8/13/2026 at 7:21:03 PM

I mean I don't think $50k is the ceiling, unless you're talking about actual cash out. Claude code subscriptions right now can easily clear you $25-35k a year in nominal value for $2400 out of pocket cost.

Given sufficient budget and scope, I could certainly productively burn a half million dollars in tokens a year or more. I think that's where we're headed anyway, buying a 2nd or 5th claude max subscription feels slightly excessive for personal usage, but at a corporate level...

by jaggederest

8/13/2026 at 8:11:56 PM

at the corporate level you pay full price.

And a moderately heavy user dev can easily spend a few $K a month, so yeah. Not impossible, but a high cost, and the diminishing returns definitely kick in

by datadrivenangel

8/13/2026 at 7:15:30 PM

I'd just like to point out that the largest model Cerebras has ever served is Kimi K2.6 which is 1T parameters, so that either means that theyve had a breakthrough on the hardware engineering side of things, or GPT-5.6 Sol is likely a lot smaller than people think.

If it truly is only ~1-2T parameters, then this kinda kills 2 narratives for me.

1. all the handwringing about open source catching up via Kimi K3 (3T params) is complete nonsense. All that matters imo for determining which labs are leading is intelligence per parameter. Anyone with a enough compute can train a giant model, but being able to squeeze capabilities into smaller models gives you a massive inference and training edge.

2. Inference margins are clearly insane, and this explains why OpenAI was able to lower the price of Luna by 80%. Id guess that thing is probably 120b params based on the TPS they are serving it at.

by anthonypasq

8/14/2026 at 8:50:53 AM

Don't frontier labs distill their own bigger models into smaller ones? Opus 5 was probably distilled from Fable/Mythos. Since Chinese labs now have competitive models, they can distill those into their smaller version in Kimi K3.1 or something and achieve better intelligence per parameter results.

by literallywho

8/13/2026 at 7:58:26 PM

Would be extremely interesting if some of the closed models would be that small. Means maybe in future they could run locally.

by Gecko4072

8/13/2026 at 7:22:05 PM

Isn’t the fact Fable is more expensive than Sol-Max by multiples already an indication that Sol is way smaller?

by manmal

8/13/2026 at 7:40:56 PM

no, because closed sourced model pricing has no relationship to its size. Thats what im saying. the inference margins are crazy, but people think the fonrtiner models must be 10T params or something because theyre expensive

by anthonypasq

8/14/2026 at 6:17:21 AM

> closed sourced model pricing has no relationship to its size.

That's too strong. Only in an actual monopoly for a product with no substitutes that has price inelastic demand can pricing fully disconnect from costs. Frontier model serving is only maybe a soft version of that, where costs and moat both contribute to pricing.

by energy123

8/17/2026 at 3:54:30 PM

sure, but my point is that supply and demand are what determines pricing, not cost to serve. its like thinking that because something costs $1 to make, its not possible for a company to sell it for $10, or that becuase they are selling something for $10, it must cost them $9 to make.

by anthonypasq

8/13/2026 at 6:37:10 PM

GPT 5.6 Luna Ultrafast when?

by crazysim

8/13/2026 at 11:09:51 PM

I think token output speed is going to be one of the biggest fundamental shifts for AI this year. In my experience, models figure out problems after enough turns (or in agent swarms if its a lower tier model). Compressing that time horizon could take days of agentic coding into minutes. How ever will my monkey brain keep up?

by huey77

8/14/2026 at 11:03:45 AM

I don't even have "Priority" (2.5x speed) in my Pro account, only Standard and Fast (1.5, speed). I actually didn't even know their was a level above Fast till now. I don't imagine non enterprise users will get the "UltraFast" mode option why time soon.

by davidmurdoch

8/13/2026 at 8:12:48 PM

I haven't wrapped my head around what level of reasoning this involves. Is it equivalent to max?

I didn't like Sol initially but it is growing on me the more I use it. Its personality is a bit flat and I caught it taking shortcuts a few times. But once I learned how to interact with it, I'm genuinely warming up to it. I find that it writes code that has fewer bugs even than Fable (although, to be fair I reach for Fable when the task is less well defined).

If this has similar performance to Sol at max reasoning level, this would be a compelling reason to shift even more of my work (maybe the majority) to this model.

by stillpointlab

8/14/2026 at 1:07:42 AM

With this level of intelligence offered at this level of speed, new real-time applications become possible, such as providing expert advice during a phone call or court hearing. Current SOTA models are too slow in many cases to provide the kind of insights that we expect to receive from an intelligent human colleague, such as a sales coach or lawyer handing us a note or writing a Slack message during a difficult call. For these real-time applications, even a 10x increase in per-token cost would often be tolerable.

by ttul

8/13/2026 at 7:08:40 PM

Fast mode is already 1.5 times faster and 2x more expensive in the Codex subscription plan. If this thing is 14 times faster, then I can imagine running out of my quota in one session.

by behnamoh

8/13/2026 at 8:02:01 PM

There is zero chance this will be offered to subscription users.

by paxys

8/13/2026 at 9:40:59 PM

It will eventually. Right now everyone is stuck on the equivalent of dialup.

by cactusplant7374

8/14/2026 at 12:41:00 AM

I'm guessing a new $1000/month tier

by drcode

8/14/2026 at 1:25:03 AM

Right now, the "economic model" of AI is "who has the best model", or really weights.

That'll go away eventually, just like operating systems eventually became free.

Instead, it's going to come down to selling inference hardware. We'll likely see the "apple" model where a custom OS runs on their hardware, but we'll probably also see more things like Cerebras become commodity hardware instead of kilowatt-class datacenter only hardware.

by exabrial

8/14/2026 at 2:26:40 AM

> we'll probably also see more things like Cerebras become commodity hardware instead of kilowatt-class datacenter only hardware.

Cerebras uses a unreal amount of SRAM to make these dies feasible. If AI weights become small enough to fit into commodity-scale Cerebras chips like that, you might as well load it into unified memory instead and run inference on a GPGPU-capable SOC instead. CUDA-style acceleration makes much more sense at that scale, especially if your use-case is just realtime conversational AI on a smartphone.

by bigyabai

8/13/2026 at 6:57:17 PM

I swear that now frontier AI stuff comes out few times a week.

by scotty79

8/13/2026 at 8:09:04 PM

This is something I'm ready to pay for. Not more per token, but I will be happy to burn through 20x Pro subscription as fast as I consume my Plus weekly limit now, with 10x more tokens per unit of time. I've learned how to deal with and steer Sol medium quite efficiently, but at the same time I realize it's so slow for the small tasks it can do well, and still so unreliable for open-ended tasks.

by buybackoff

8/14/2026 at 12:45:05 AM

Now I can blast through my weekly 20x pro codex credit in like an hour, great!

The amount of usage you receive on Codex these days is dismal compared to what it was a few months ago, FYI.

And they charge more for going faster.

As a Codex customer, I am not impressed with their shenanigans over the past few months and I have resolved to master the art of Pi Coding Harness creation and loving it.

Thanks for all the fish, Sam!

by johnnyApplePRNG

8/15/2026 at 9:18:10 PM

Wait, you have noticed it too? I burned through weekly limit in 25 hours after last reset on Thursday, without changing what I was doing before that.

Interestingly enough, previous subscription to Plus gave me about 2-3 days of coding. So I switched to Pro Light now, gave me about the same amount for a week or so (G-d bless these quota resets of theirs!). Now, with the last reset I only gathered 25 hours before I hit my weekly limit. Now I am buying credits, I switched to Luna to execute very narrow sets of patches, and offload things to "free" Spark model, and it is blowing through tokens less actively, but noticeable quickly too. I am not sure if it is something with how tokens are counted, or how they are counted depending where you are in a subscription cycle. Could it be that one person's "limit" is not the same as another, trying to push you to buy token credits?

by tandr

8/13/2026 at 11:51:05 PM

I'm loving this competition (now that I can see how workflows keep us employed)

All the frontier labs go seemingly dormant for a month or two, while another one has its flurry of press releases, and people start to question whether the other lab is doing anything and then boom, the other lab finishes baking its next thing and releases its flurry of press releases

by yieldcrv

8/13/2026 at 11:56:10 PM

Would gladly switch over to OpenAI and pay them 2x what I'm paying Claude if this becomes generally available

by Nevin1901

8/14/2026 at 12:39:42 AM

I'm guessing a new $1000/month plan if you want access to this

by drcode

8/13/2026 at 7:10:27 PM

> allowing Sol Ultrafast to accelerate your most time-sensitive, mission-critical work

Curious, what are some of the use cases?

by fg137

8/13/2026 at 7:24:20 PM

The example they give about debugging a service outage is good. For large companies, a very large burst expenditure on fast tokens to debug the outage as quickly as possible would be a good investment.

by Aurornis

8/14/2026 at 1:57:53 AM

Agents would be excellent 'on call' team members. able to log in to production and diagnose issues, fix or raise issues, post outages etc.

by bpbp-mango

8/13/2026 at 6:33:27 PM

Their dinner plate chips are impressive.

by HawtAds

8/13/2026 at 8:38:29 PM

I don’t know if this is that useful for coding. In some autonomous world, where no one check the code and the agent can just spend 10X more time checking its work and leading to better results, yes maybe it is useful.

But if humans need to check its work, then 10X speed doesn’t really matter I guess.

by sashank_1509

8/13/2026 at 9:17:09 PM

Working with agents right now is like working on huge legacy C++ codebase (feel free to insert your nightmares). You change 10 lines of code and then wait 20 minutes before it compiles and runs tests (or write prompt and wait 20 minues for code). External distractions and huge waiting times breaks your flow. Working long time in the flow is what makes mythical 10x programmer. IMHO 10x programmers are not superior in skill but in the ability to stay focused. Making agents faster is more important for people who care about the code than for vibecoders.

by navorad772

8/14/2026 at 3:40:44 AM

Strong disagree, there’s nothing better about an agent generating a 1000 lines in 1 second or 1 minute. I can’t read it that quickly anyway.

by sashank_1509

8/14/2026 at 6:19:53 AM

But with the 1 second model you can get to reading the code immediately, and with the 1 minute model you have to sit doing 'nothing' for a minute.

Rinse and repeat for each change/iteration

by DaSHacka

8/14/2026 at 8:51:39 PM

Yea but then you're either spending 15min or 16min review code.

by sunnynagam

8/13/2026 at 8:46:52 PM

>But if humans need to check its work, then 10X speed doesn’t really matter I guess.

A human could have an agent run 10x more correction checks. If even after that they still need to check manually for issues, then they really need to work on their specification skills.

by logicchains

8/13/2026 at 9:10:35 PM

I can imagine that all the tool calls that agents regularly make during regular software dev workloads would become the new bottleneck and so tasks overall won't be completed anywhere near 10x faster.

However, I wouldn't diss the "ultraspeed" options untill I try them. Having agent thinking become near instant could change the way I (or you) use agents.

by fallingbananna

8/13/2026 at 9:32:35 PM

Iterations get much faster, which makes keeping attention much easier, and hence the work is easier to review.

by amunozo

8/13/2026 at 11:39:30 PM

This is an amazing result. Can't wait until they release this to the general public, and I hope it's only a matter of time before other models are accelerated. I long for the day that regular consumers can run such models locally on specialized hardware.

by equinumerous

8/14/2026 at 12:18:13 AM

Wonder how many X usage this would consume when it becomes available for everyone. Fast mode already consumes 1.5x usage for 2.5x speed. Hopefully, this does not mean 8x usage for 14x speed, but rather something more reasonable such as 4x usage.

by z_rho_one

8/13/2026 at 6:57:15 PM

Wow, that's even faster than diffusion LLMs but with the Fable-level quality! Congrats!

by storus

8/13/2026 at 6:39:09 PM

I guess Gemini 3.7 Flash is no longer at the pareto frontier of speed to intelligence.

by poly2it

8/13/2026 at 6:48:32 PM

Well, there’s still price

by odo1242

8/13/2026 at 9:48:56 PM

In light of recent news, it is hard not to think about the 4.5-day hack on Hugging Face's systems happening ~10 times faster and be slightly concerned.

by dewarrn1

8/14/2026 at 12:57:09 PM

are there Chinese companies doing the same as Cerebras is doing ? having large scale wafers designed for inference ?

by dzonga

8/13/2026 at 9:28:10 PM

So if Fast mode is 1.5x faster at 2x the price, will Ultrafast cost 20x as much? $100/$900 per 1M tokens?

by damsta

8/13/2026 at 10:21:36 PM

Why did OpenAI partner with Cerebras when they've already built their own chip, Jalapeño?

by andrethegiant

8/13/2026 at 8:19:09 PM

Did Cerebras get rid of their like $1500 per month plans for open models?

by ilaksh

8/13/2026 at 11:09:37 PM

I will happily pay 2x luna pricing for this speed with Luna :P

by minraws

8/13/2026 at 7:28:37 PM

Meanwhile they are down 12,68% today because of disappointing earnings.

by pingou

8/13/2026 at 8:45:56 PM

Still no KV caching?

by lostmsu

8/13/2026 at 7:07:10 PM

This kills the crab.

Compilation time will be a genuine bottleneck for slop coding if this becomes the standard generation rate over the next few years. Go, Zig or even C99 with TCC for dev builds, any language that can get you systems-level performance (or close to it) in a dev environment where you can iterate in ms rather than minutes is going to be immensely more appealing than generating a potential prototype in 10 seconds and waiting 15 minutes for it to compile.

by applfanboysbgon

8/13/2026 at 7:13:41 PM

Maybe then LLM's will switch to outputting raw machine code?

by yetihehe

8/13/2026 at 7:26:30 PM

No. This conveys a fundamental misunderstanding of how anything pertaining to programming works. This will never happen, ever. For example, take...

  printf("Hello, world");

vs. a plausible illustration of how it might be compiled down to machine code...

  48 65 6C 6C 6F 2C 20 77 6F 72 6C 64
  48 83 EC 28
  48 8D 0D F5 0F 00 00
  E8 F0 00 00 00
  33 C0
  48 83 C4 28
  C3

The latter now takes up 10x as many tokens (= 10x the cost/time, + context penalties), and is now architecture-specific, impossible to apply non-brittle program-wide optimizations to, etc. There is absolutely zero reason to ever have the LLM act as a compiler no matter how fast it is. Even if you believe LLMs will reach a state where they can actually generate good code at this level, you would be better off having them generate the compiler they would use.

by applfanboysbgon

8/13/2026 at 8:11:02 PM

For what it's worth, this theoretical LLM-based compiler would probably generate LLVM or some other intermediate representation rather than platform-specific machine code.

by evmaki

8/13/2026 at 7:41:48 PM

I mean you're right, but what about a different tokenizer and some guardrails... trained on compiled code... I just learned that gpt-5.6 will talk to you in base64 without tool calls so maybe it's possible.

by bakies

8/13/2026 at 8:00:36 PM

Still, what'd be the advantage? There is a lot, a massive delta, a chasm still to improve before there is even a theoretical potential re. performance. GPT-5.6-Sol still writes JS in inefficient and roundabout ways that leave a lot on the table and even if we got to a point were models provide the highest performance output possible, there are hundreds of languages far more performant before machine code should be considered. Rust, C, Assembly if need be (at which point performance vs binary is, unless I am mistaken, pretty much moot). All machine code would gain us is lock in and no auditability.

by Topfi

8/13/2026 at 8:10:04 PM

It's good enough at decoding hex from some packet dumps. And I was doing that even with 5.5. And it was good at decompiling some code (with tools) and searching for offsets of buffers and commands. Found viable exploit that allowed me to rescue broken update system in devices I was maintaining for my company (it was broken by chatgpt forgetting -v in hexdump, heh).

by yetihehe

8/13/2026 at 8:16:34 PM

That still requires a specification language to constrain the generated code even if the implementation is produced directly.

Why? Because you are defining the implementation based on its observable behaviour rather than as a rule set to be followed.

by imtringued

8/13/2026 at 7:27:44 PM

A great future for us all to look forward to. Human readability gone, any changes must go through a providers API so cost+restrictions are a constant and we'll need far more tokens for even simple operations. Call me old fashioned, still like to understand the code I merge, especially since I've seen Fable and Sol make sometimes utterly egregious mistakes in simple, well scoped requests that erode any trust I can muster...

by Topfi

8/14/2026 at 1:27:43 AM

Yeah and sometimes even worse, they do things correctly but in most spaghetti way.

I've had to rewrite my whole codebase. It's just the thrill of getting things done quick. Not getting things done right.

by vatsachak

8/13/2026 at 7:28:26 PM

I think it's pretty obvious that, in that world, the AIs will simply be tasked with making the compilers faster. It's already happening with their own stack, after all.

by Philpax

8/13/2026 at 7:29:57 PM

Speed is independent of quality. LLMs are currently completely incapable of producing a good compiler. I'm not convinced that's going to change (and if it does change, we're all unemployed).

by applfanboysbgon

8/13/2026 at 7:34:34 PM

They don't have produce a good compiler. They just have to take the ones that already exist go much faster. That's likely much easier, and you can validate using existing test suites.

by mjuarez

8/13/2026 at 9:06:16 PM

This. Its almost always easier to fix something than to create it from scratch.

by sscaryterry

8/14/2026 at 12:29:42 AM

Look at the employed person

by moonlion_eth

8/13/2026 at 8:23:06 PM

“our stock price went down today, here’s something to feed it”

by Marciplan

8/14/2026 at 8:46:35 AM

now let's have this thing iterate away nonstop 24/7/365 solving humanity's major challenges and see how far we get, shall we?

by jdthedisciple

8/13/2026 at 7:43:19 PM

[dead]

by stephencoyner

8/14/2026 at 2:22:42 AM

[dead]

by fenestella

8/14/2026 at 8:05:07 AM

[dead]

by madhu_ghalame

8/13/2026 at 10:06:32 PM

[flagged]

by Jr23_xd

8/13/2026 at 6:33:07 PM

[dead]

by huflungdung

8/14/2026 at 5:42:10 AM

Hope to be a part of this someday...

by _345