alt.hn

8/18/2026 at 10:06:10 PM

GLM-5.3 Artificial Analysis Benchmarks

https://artificialanalysis.ai/models/glm-5-3

by apitman

8/18/2026 at 11:20:58 PM

I like to compare models with a similar score on cost per task and output tokens per task since those measure two things I'm interested in: cost efficiency and token efficiency. Here's how GLM-5.3 compares to other models in a similar score and against GLM-5.2 to save a few clicks for others who care about these metrics:

    Model                        Score    Cost / Task    Output Tokens / Task
    -------------------------------------------------------------------------
    GLM-5.3 (max)                 59.5          $0.68                  41,107
    GLM-5.2 (max)                 53.0          $0.56                  32,200
    Claude Opus 5 (high)          61.5          $1.52                  21,353
    GPT-5.6 Sol (max)             60.9          $1.23                  16,879
    Grok 4.6 (high)               60.9          $0.84                  21,735
    Kimi K3 (max)                 59.7          $0.84                  25,474
    GPT-5.6 Sol (xhigh)           59.0          $0.87                  11,098
    Claude Opus 5 (medium)        58.6          $0.98                  12,459
    Qwen3.8 Max                   58.1          $1.13                  38,287
    Qwen3.8 2.4T A95B             57.7          $0.95                  32,472
    Claude Opus 4.8 (max)         57.3          $1.65                  33,557
    GPT-5.6 Sol (high)            57.3          $0.52                   7,545
    Muse Spark 1.2 (xhigh)        56.8          $0.40                  30,430
    GPT-5.6 Terra (max)           56.6          $0.51                  20,838
    GPT-5.5 (xhigh)               56.3          $0.69                  16,893
    Gemini 3.7 Flash (high)       56.0          $0.40                  36,847
Edited for accuracy and more models.

by scotttrinh

8/18/2026 at 11:27:42 PM

Muse Spark has a nice balance. not to mentions the Contribs version is old deepseek flash prices.

by sourcecodeplz

8/19/2026 at 12:30:54 AM

Tested muse spark 1.2 because it was rated so high on design arena, and I've missed a model that can do nice UI in the hands of an operator with no UI skills.

It produced worse UI mockups than GPT and GPT models are already the bottom of the barrel here. The only model that performed well was Kimi K3 - insanely good, but expensive.

It's hard to trust benchmarks these days.

by glub

8/19/2026 at 3:26:27 AM

If you just want it to generate UI out of nothing, the benchmarks aren't really for that.

If you want to generate a UI based on specific user input of some kind, then they are.

I'd suggest using one model for UI and another model for tacking onto that UI. LLMs are great at pattern matching, and benchmarks don't really capture one-shotting desirable UI.

That said, benchmaxxing is a thing and your experience with models is a thing. Benchmarks are fuzzy and should be taken with a grain of salt.

by samtheprogram

8/18/2026 at 11:35:35 PM

I found the sweetspot here: GPT-5.6 Sol (high) 57.3 $0.52 7,545

(Edit: TLDR; It gets on with it, makes the same mistakes you would, without overthinking and overengineering, most of the time)

by sscaryterry

8/19/2026 at 12:22:16 AM

It would make reading and comparing a bit easier if the data was sorted by a dimension.

by dudeinhawaii

8/19/2026 at 1:23:00 AM

Cost per task:

  Model                        Score    Cost / Task    Output Tokens / Task
  -------------------------------------------------------------------------
  Muse Spark 1.2 (xhigh)        56.8          $0.40                  30,430
  Gemini 3.7 Flash (high)       56.0          $0.40                  36,847
  GPT-5.6 Terra (max)           56.6          $0.51                  20,838
  GPT-5.6 Sol (high)            57.3          $0.52                   7,545
  GLM-5.2 (max)                 53.0          $0.56                  32,200
  GLM-5.3 (max)                 59.5          $0.68                  41,107
  GPT-5.5 (xhigh)               56.3          $0.69                  16,893
  Grok 4.6 (high)               60.9          $0.84                  21,735
  Kimi K3 (max)                 59.7          $0.84                  25,474
  GPT-5.6 Sol (xhigh)           59.0          $0.87                  11,098
  Qwen3.8 2.4T A95B             57.7          $0.95                  32,472
  Claude Opus 5 (medium)        58.6          $0.98                  12,459
  Qwen3.8 Max                   58.1          $1.13                  38,287
  GPT-5.6 Sol (max)             60.9          $1.23                  16,879
  Claude Opus 5 (high)          61.5          $1.52                  21,353
  Claude Opus 4.8 (max)         57.3          $1.65                  33,557
Benchmark score:

  Model                        Score    Cost / Task    Output Tokens / Task
  -------------------------------------------------------------------------
  Claude Opus 5 (high)          61.5          $1.52                  21,353
  GPT-5.6 Sol (max)             60.9          $1.23                  16,879
  Grok 4.6 (high)               60.9          $0.84                  21,735
  Kimi K3 (max)                 59.7          $0.84                  25,474
  GLM-5.3 (max)                 59.5          $0.68                  41,107
  GPT-5.6 Sol (xhigh)           59.0          $0.87                  11,098
  Claude Opus 5 (medium)        58.6          $0.98                  12,459
  Qwen3.8 Max                   58.1          $1.13                  38,287
  Qwen3.8 2.4T A95B             57.7          $0.95                  32,472
  Claude Opus 4.8 (max)         57.3          $1.65                  33,557
  GPT-5.6 Sol (high)            57.3          $0.52                   7,545
  Muse Spark 1.2 (xhigh)        56.8          $0.40                  30,430
  GPT-5.6 Terra (max)           56.6          $0.51                  20,838
  GPT-5.5 (xhigh)               56.3          $0.69                  16,893
  Gemini 3.7 Flash (high)       56.0          $0.40                  36,847
  GLM-5.2 (max)                 53.0          $0.56                  32,200

by ac29

8/19/2026 at 2:50:42 AM

This matches my experience with Sol. Read and thought for a while, and edited files, tested, edited again, then ran out of budget in a relatively short time. But its solution was very good and was done quickly, so all things equal I prefer that over something much more verbose like Deepseek.

by salamo

8/19/2026 at 1:05:32 AM

this is not very useful.

for over 1 billion real world users living in China, they don't have the option of paying $1.52 per task to use Opus 5, they are banned doing that due to US politics.

by tw1984

8/19/2026 at 1:32:47 AM

What’s the reseller situation?

by Barbing

8/18/2026 at 11:40:14 PM

these $/task figures aren't very useful in my experience. it doesn't tell you how well it did the task.

generally I choose models by their intelligence and then personal preference from direct experience.

by teravor

8/18/2026 at 11:55:15 PM

they make a ton of sense for tasks that are achievable with no prob by all models - e.g. writing scripts that do a specific thing etc.

by kolinko

8/19/2026 at 12:08:22 AM

    > writing scripts
you can use a Gemini model completely for free doing that. no agent. aistudio.google.com

by teravor

8/18/2026 at 11:54:11 PM

I've tested GLM 5.3 on the release day and Artificial Analysis is spot on. It's a really good model.

But my main takeaway was something else. I've used closed weight models for long enough that I've forgotten how good it feels to see reasoning tokens.

With GPT/Claude, you kind of hope that intent was captured well, that agent had all the information, all the tools it needed, because you won't see "hmmm it seems like nix flake isn't available here and I shouldn't install something globally" until it slopped out millions of tokens and wasted hundreds of dollars for 8 hours. With GLM and the likes, you just stop the disease right where it begins.

by glub

8/18/2026 at 11:58:08 PM

Yes, not necessary often but being able to stop something that is going off the rails is super useful. Especially if the root cause is prompt ambiguity - inject a clarification & it recovers

by Havoc

8/19/2026 at 12:25:04 AM

It's also starting to go beyond reasoning and it's becoming much more problematic. Reasoning is one thing, but codex, for example now encrypts agent-to-agent messages as well, and compaction. I've no idea what subagents are instructed to do, or what they reported back in native codex.

The only thing that's keeping me is the value $200 subscription provides. If that value disappears, I see no reason why not to switch to something that isn't a black box.

by glub

8/19/2026 at 1:06:48 AM

With GPT/Claude, hiding those from users to waste their tokens is a feature, not a limitation.

by tw1984

8/19/2026 at 1:45:40 PM

Generally, are closed sourced models hiding their traces? I was making an agent to develop and deploy apps and fed the traces to dispel time-consuming detours and made it a few times faster.

by aitchnyu

8/19/2026 at 1:54:14 AM

Sol is an underappreciated model. Dropped Claude today and went to codex. None of that god awful prose Claude used for me any longer.

by Escapade5160

8/18/2026 at 11:21:28 PM

Beware of the benchmarks listed. SciCode and EnterpriseOps for instance: https://shukla.io/blog/2026-08/gym.html

by BinRoo

8/18/2026 at 11:49:04 PM

The Chinese models also like to cut corners on stuff like science. Their scores on stuff like biotech and scientific knowledge is far from ChatGPT unfortunately. (Claude is pretty good but it just refuses all prompts).

by Onavo

8/19/2026 at 2:54:21 AM

I understand that running these benchmarks can get expensive, but it would be really nice to see AA include more benchmarks of models at reasoning settings other than the maximum, at least for the biggest releases. They have that nice graph of cost vs. composite benchmark score with the Pareto frontier line, but who knows if those are actually the optimal choices? There are already a few non-max-reasoning models on the Pareto line, among the few that were tested.

by AnodicElegy

8/19/2026 at 3:02:31 AM

You can turn on various levels of some of many of the models in the UI

by apitman

8/19/2026 at 10:32:48 AM

Yes, they have multiple levels of Claude, GPT, Gemini, and Kimi, but not the other top models (I would put GLM, Qwen, Muse, Grok, and Deepseek in that bucket).

by AnodicElegy

8/18/2026 at 11:18:18 PM

Is it worth using these models if I have a claude code subscription already? The appeal of lower cost is nice but I haven't gotten over the switching cost yet.

by Zaheer

8/19/2026 at 12:49:01 AM

FYI, you can use your Claude subscription pricing with OpenCode via Meridian[0], which also makes it easier to try out other models when they come out. You can also use your other subscriptions in OpenCode with CLIProxyAPI[1]. The switching cost was relatively high, mostly from claude code plugins but completely worth it. I'm now mostly using GLM-5.3 and Codex models via OpenCode and barely using Claude which seemed unfathomable less than two months ago.

[0] https://github.com/rynfar/meridian

[1] https://github.com/router-for-me/CLIProxyAPI

edit: reworded for clarity

by robertn702

8/19/2026 at 12:57:05 PM

I'm sometimes tempted to use these sorts of proxies, but I always worry that the hacks they have to use to wrap the upstream tools and APIs is going to mess with my downstream harness/agent.

by apitman

8/18/2026 at 11:22:57 PM

Use a unified proxy that lets you switch between models seamlessly. We are far from an equilibrium in this market and you will continue to have FOMO no matter who you pick if you go all in on one company

by culi

8/19/2026 at 12:11:42 AM

I use the $200 plan w/ Anthropic and run out of tokens half way through the week and supposedly they are progressively reducing the limits on all their subs even further.

At some point I will switch, $200 buys a lot of tokens on OpenRouter.

by oceanplexian

8/19/2026 at 12:57:21 AM

Is the conventional wisdom that the subscription price/token is better than the API price/token not valid any more? Or is access to model diversity worth the increased per token costs?

by Sabinus

8/19/2026 at 12:15:26 AM

Same here. I'm switching to the Codex plan. It just doesn't go very far now. Especially if you use fable at all.

by kilroy123

8/18/2026 at 11:21:19 PM

At least by API usage, they aren't yet lower cost than subscriptions. Not sure about GLM's subscription plans though.

by colingauvin

8/19/2026 at 12:14:19 AM

GLM subscription is better than API, but significantly worse than Codex, even when used outside peak hours.

by glub

8/19/2026 at 12:11:58 AM

If anything, it's going to be more expensive. Price/performance ratio isn't there yet for frontier open weight models.

But regardless, you definitely should use a harness where switching models on the fly is easy. There's a reason why Anthropic uses their own proprietary formats/conventions anywhere they can - to lock you in when inference eventually commoditizes.

by glub

8/18/2026 at 11:26:22 PM

no, at subscription prices claude is a better value than GLM.

They're only a better value if you're paying API rates

by notatoad

8/19/2026 at 10:57:05 AM

You can have a subscription for GLM.

by scotty79

8/20/2026 at 2:48:01 AM

yes, but the GLM subscription gives you barely more usage than you'd get just paying API rates. it's not subsidized like the claude or chatgpt subscriptions are.

by notatoad

8/20/2026 at 10:46:14 AM

It changed recently but previously subscription had prompt allowance, not token allowance. So probably accidentally was very genrous.

by scotty79

8/19/2026 at 1:54:22 AM

Does Artificial Analysis use OpenRouter for model access to do their benchmarks?

by swingboy

8/18/2026 at 11:09:26 PM

Very impressive score for the size, though token use is higher than k3 and far higher than proprietary models, and its price to performance isn't all that far ahead of k3 as a result

by markasoftware

8/18/2026 at 11:18:45 PM

>token use is higher than k3 and far higher than proprietary models

GLM sets effort to max by default historically.

by Havoc

8/18/2026 at 11:24:19 PM

Aa also benchmarked k3 at max

by markasoftware

8/19/2026 at 8:50:55 PM

I ran the same Mac SVG drawing prompt through GLM 5.2 and 5.3 across every reasoning effort level, and 5.3 showed improved performance

https://sumedh.info/models/glm-5-3

by sumedh

8/19/2026 at 4:32:10 AM

Still yet, I cannot justify switching from dirt-cheap Luna model, which is pretty damn "intelligent" and works well for my flow

by yipinwong

8/18/2026 at 11:22:52 PM

Tied for #1 by agentic index (with Opus 5).

by colingauvin

8/19/2026 at 9:34:22 AM

To me one of the biggest limitations of GLM is the lack of multi-modality.

For web dev is just a must to have, and offloading that part to a secondary model doesn't work really well in my experience.

by gdorsi

8/19/2026 at 1:43:33 AM

And reminder: it's less than a quarter the size of Kimi K3!

by scosman

8/18/2026 at 10:56:28 PM

...do I take out a double mortgage to buy a 4 Spark cluster?

by colingauvin

8/18/2026 at 11:17:02 PM

No, you use openrouter and spend 10% as much as using a proprietary model.

by nvme0n1p1

8/19/2026 at 1:45:24 AM

Qwen3.8 27B doing a lot of lifting right now, and people seem to run it pretty well on 1-2x 3090 setups...

by jtbaker

8/19/2026 at 10:55:55 AM

20k is credit card territory

by killingtime74

8/18/2026 at 11:03:50 PM

$20k is personal loan territory, not a second mortgage lol

by lisplist

8/18/2026 at 11:08:12 PM

Not with my credit!

by colingauvin

8/19/2026 at 3:35:41 PM

[flagged]

by lluisantoni

8/19/2026 at 12:19:24 PM

[flagged]

by ushiro35

8/18/2026 at 11:52:39 PM

[flagged]

by fenestella

8/19/2026 at 3:51:03 AM

[flagged]

by qqt