alt.hn

4/15/2026 at 5:19:06 AM

Google Gemma 4 Runs Natively on iPhone with Full Offline AI Inference

 https://www.gizmoweek.com/gemma-4-runs-iphone/

by takumi123

4/15/2026 at 12:11:40 PM

Is it me, or does the article sound like LLM output?

The pattern "It's not mere X — it's Y", occurs like 4 times in the text :v

by temp7000

4/15/2026 at 2:18:42 PM

I can't believe you'd impugn the high moral standards of "gizmoweek dot com".

by Andrex

4/15/2026 at 2:07:13 PM

I don't care if it's written by an LLM.

The problem with the article is the complete lack of details. No benchmarks on the iPhone capable models. No details, whatsoever.

Human or LLM - the article is a whole lot of nothing.

by BeetleB

4/15/2026 at 2:13:55 PM

Funnily enough, to me these aphorisms (?) sound almost like the replicant test in Blaze Runner. Like these are the unit bit of "nudging"

by doliveira

4/15/2026 at 6:56:36 PM

LLM, recite your baseline:

"It's not just X – it's Y." Slop. "You're absolutely right!" Slop. "And this is key –" Slop. "This is a nuanced topic." Slop.

https://www.youtube.com/watch?v=vrP-_T-h9YM

by nozzlegear

4/16/2026 at 7:29:43 PM

The problem is not authorship. It's the lack of substance

by nextaccountic

4/17/2026 at 8:32:37 AM

This is just prompting an LLM and just dumping it on the site (which is clearly what is happening here, all the articles show the same signs of AI output, no human writing, no style, as far as I can tell).

If this is the level of care that goes into news articles, then we're doomed. What will ultimately happen is that AI summarizes AI articles, which got summarized from another AI article, which got summarized from another AI article, .. and after enough rewriting all facts will be gone from articles. I don't care to read this slop, and I'm shocked people are so readily accepting this new state of affairs.

by dax_

4/15/2026 at 3:32:41 PM

This article is all fluff because real benne marketing. If they mentioned that a 4B model on an iPhone 16 drains 15% of the battery for a single long prompt and triggers hard thermal throttling after 20 seconds, nobody would be clicking on headlines about "commercial viability" fwiw

by veunes

4/15/2026 at 3:57:43 PM

I ran several Gemma 4 quants on my 24gb mac mini, and with proper context size tuning they're quick enough I guess, but I would really love to see them working well on an iphone with 2/3gb of ram...

by Domenic_S

4/15/2026 at 1:06:22 PM

Ran it through Claude, Grok, whatever...for me, they all flagged issues (no sources, punchy phrases with repetition,...) with these content farms.

My favorite: couldn't even prove the author is a real person. They all found no record!

by caminante

4/15/2026 at 1:16:50 PM

As someone said we live in a strange but amazing era, where although it has never been easier to be deceived, but its _also_ much easier to uncover said deception especially on the internet.

by itissid

4/15/2026 at 2:21:38 PM

Or at least think you've uncovered deception. It's not clear to me yet that any of these "AI detectors" are reliable, and if they are, it's just an arms race.

by ryandvm

4/15/2026 at 2:18:32 PM

It's much faster and simpler to assume everything on the internet is crooked

by walthamstow

4/15/2026 at 12:13:28 PM

> :v

I guess I found the millennial. I haven't seen that in so long!

by figmert

4/15/2026 at 12:30:18 PM

:<

by Den_VR

4/15/2026 at 1:38:59 PM

:')

by neals

4/15/2026 at 2:19:02 PM

>_>

by Andrex

4/15/2026 at 3:47:37 PM

\o/

by xiconfjs

4/15/2026 at 2:13:58 PM

Analog emojis FTW

by yangm97

4/17/2026 at 6:37:23 PM

Neither analog nor emojis. An analog emoji would just be a picture printed on paper.

by Gormo

4/18/2026 at 1:12:46 PM

¬_¬

by yangm97

4/15/2026 at 7:35:02 PM

(╯°□°)╯︵ ┻━┻

by mannycalavera42

4/15/2026 at 3:47:09 PM

¯\_(ツ)_/¯

by Melatonic

4/15/2026 at 3:20:59 PM

It is like the AI is training us to avoid certain language patterns. I rebel at the hostage of weak language: for strong language is next.

by altruios

4/15/2026 at 3:44:53 PM

The mighty semi colon prepares for its return !

by Melatonic

4/15/2026 at 12:19:47 PM

An AI slop pattern so widespread it’s now referred to as “it’s not pee pee it’s poo poo”.

by mtremsal

4/15/2026 at 3:41:42 PM

It's not just a widespread pattern –––––––––––––––– it's a sign of things to come.

by lynndotpy

4/15/2026 at 3:58:43 PM

You didn't just nail it ------------ you cut to the core of the issue.

by Domenic_S

4/15/2026 at 2:14:50 PM

I haven't heard that—that's good.

by Cider9986

4/15/2026 at 4:19:38 PM

It does in fact sound like LLM output

by odo1242

4/15/2026 at 4:03:28 PM

Smells like slop to me, looks like the site exists solely to garner search hits.

by wtyvn

4/15/2026 at 12:13:55 PM

You would be correct. Ran the article through GPTZero, 100% AI.

by kbouw

4/15/2026 at 12:45:13 PM

These detectors are a scam falsely flagging non-native English speakers: https://plagiarismcheckerai.app/ai-detector-false-positives-...

At this point relying on their judgement is beyond folly.

by subscribed

4/15/2026 at 1:18:28 PM

It's both ironic an confusing that this website itself promotes an AI detector.

by cubefox

4/17/2026 at 9:13:57 AM

Yeah, I admit I lazily chose one of the first results reporting on this study instead on the best one, so the irony is not lost on me.

Sorry for making you snort and shake your head in amusement :D

by subscribed

4/15/2026 at 12:22:19 PM

Would not trust any of these tools in the slightest.

by 71bw

4/15/2026 at 12:35:01 PM

AI detectors that use text as a basis are not real. It is fundamentally impossible for them to exist.

by devmor

4/15/2026 at 1:40:29 PM

Huh?

LLM output doesn't have the variety of human output, since they operate in fixed fashion - statistical inference followed by formulaic sampling.

Additionally, the statistics used by LLMs are going be be similar across different LLMs since at scale its just "the statistics of the internet".

Human output has much more variety, partly because we're individuals with our own reading/writing histories (which we're drawing upon when writing), and partly because we're not so formulaic in the way we generate. Individuals have their own writing styles and vocabulary, and one can identify specific authors to a reasonable degree of accuracy based on this.

It's a bit like detecting cheating in a chess tournament. If an unusually high percentage of a player's moves are optimal computer moves, then there is a high likelihood that they were computer generated. Computers and humans don't pick moves in the same way, and humans don't have the computational power to always find "optimal" moves.

Similarly with the "AI detectors" used to detect if kids are using AI to write their homework essays, or to detect if blog posts are AI generated ... if an unusually high percentage of words are predictable by what came before (the way LLMs work), and if those statistics match that of an LLM, then there is an extremely high chance that it was written by an LLM.

Can you ever be 100% sure? Maybe not, but in reality human written text is never going to have such statistical regularity, and such an LLM statistical signature, that an AI detector gives it more than a 10-20% confidence of being AI, so when the detector says it's 80%+ confident something was AI generated, that effectively means 100%. There is of course also content that is part human part AI (human used LLM to fix up their writing), which may score somewhere in the middle.

by HarHarVeryFunny

4/15/2026 at 3:20:43 PM

> LLM output doesn't have the variety of human output, since they operate in fixed fashion - statistical inference followed by formulaic sampling.

This is the wrong thing to look at; your chess analogy is much stronger, the detection method similar (if you can figure out a prompt that generates something close to the content, it almost certainly isn't human origin).

But to why the thing I'm quoting doesn't work: If you took, say, web comic author Darren Gav Bleuel, put him in a sci-fi mass duplication incident make 950 million of him, and had them all talking and writing all over the internet, people would very quickly learn to recognise the style, which would have very little variety because they'd all be forks of the same person.

Indeed, LLMs are very good at presenting other styles than their defaults, better at this than most humans, and what gives away LLMs is that (1) very few people bother to ask them to act other than their defaults, and (2) all the different models, being trained in similar ways on similar data with similar architectures, are inherently similar to each other.

by ben_w

4/15/2026 at 8:31:44 PM

An LLM is just computer function that predicts next word based on the input you give it. It doesn't make any difference what the input is (e.g. please respond in style X) - the function doesn't change, and the statistical signature of how it works will still be there.

If you don't believe me, try it for yourself. Ask an AI to generate some text and give it to the AI detector below (paste your text, then click on scan). Now ask the AI to generate in a different style and see if it causes the detector to fail.

https://app.gptzero.me/

by HarHarVeryFunny

4/16/2026 at 8:02:40 AM

I can't use that linked app, paywall immediately. Unlike the person you were replying to here[0], I do not claim that this is impossible:

LLM is indeed just computer function that does stats. And our brains are just electro-chemistry that does stats. This is why stylometric analysis of human writing is a thing.

My previous experience with things such as you have linked to, is they used to be quite poor. I assume they're better since then, but then again so are the models.

[0] https://news.ycombinator.com/item?id=47778171

by ben_w

4/16/2026 at 2:27:35 PM

> I assume they're better since then, but then again so are the models.

Yes, but "better" means different things for each of these.

Detectors are trying to get better at distinguishing human from LLM-generated text.

LLMs are being improved to generate more useful (and benchmark maxxing) outputs, not to attempt to avoid detection.

LLMs are in fact explicitly trained to be as predictable as possible. The training goal is to minimize continuation prediction errors, which means they are in effect being trained to generate output where each word can be predicted by what came before it (which we can contrast to a human who tries to spice it up and keep it interesting by not being too predictable!).

RL post-training, which is especially used for computer code and math, is going to change this word-by-word predictability (detectability) a bit since the focus is now on a longer term goal rather than next word, but to some extent you could also view it as just steering/narrowing the output of the model towards that goal, not totally overriding the next-word statistics.

I don't know if there are AI detectors specifically trained to detect AI code rather than prose, but I'd expect that is more difficult to do, both because of the RL factor, and because computer code is so predictable in the first place - adhering to rigid syntax etc.

by HarHarVeryFunny

4/15/2026 at 3:56:21 PM

What if the prompt includes, "Produce output that doesn't sound like an AI generated it."?

by newsoftheday

4/15/2026 at 6:05:47 PM

I got curious and tried: https://claude.ai/share/3af7bd6a-15f8-4533-9dc3-a44adef255b3

by js8

4/16/2026 at 8:07:12 AM

Basically the same on ChatGPT. DeepSeek managed to generate output rather than meta-discussion about how to generate output.

by ben_w

4/15/2026 at 9:06:58 PM

That's actually interesting, thanks. It's like AI is tattling on itself.

by newsoftheday

4/15/2026 at 10:26:11 PM

A human can easily produce output that looks like anything an LLM can produce, therefor an LLM detector that can say "this is 100% written by AI" cannot exist. It's really that simple.

> Can you ever be 100% sure? Maybe not

The commenter I was replying to claimed exactly this. Their AI detector showed that the text was "100%" AI generated.

by devmor

4/16/2026 at 2:47:44 AM

I was just expressing some caution. Saying you are 100% certain of anything when the evidence is statistical seems a bit too certain, especially if it was just from a short text sample.

Compare to flipping a coin, counting heads vs tails, and trying to assess if it's a fair or biased coin. After 1000 flips if it's not close to 50/50 you would rightfully be suspect, and if it was 10/90 you should be almost certain it's biased. But you can never be 100% sure.

by HarHarVeryFunny

4/15/2026 at 3:17:18 PM

[dead]

by goodmythical

4/15/2026 at 1:10:26 PM

[flagged]

by watsonL1F7

4/15/2026 at 3:27:50 PM

I noticed the inference is routed through the gpu rather than the Apple neural engine. Google’s engineers likely gave up on trying to compile custom attention kernels for Apple’s proprietary tensor blocks iirc. While Metal is predictable and easy to port to, it drains the battery way faster than a dedicated NPU. Until they rewrite the backend for the ANE, this is just a flashy tech demo rather than a production-ready tool

by veunes

4/15/2026 at 4:37:44 PM

Are the Apple neural engines even a practical target of LLMs?

Maybe not strictly impossible, but ANE was designed with an earlier, pre-LLM style of ML. Running LLMs on ANE (e.g. via Core ML) possible in theory, but the substantial model conversion and custom hardware tuning required makes for a high hurdle IRL. The LLM ecosystem standardized around CPU/GPU execution, and to date at least seems unwilling to devote resources to ANE. Even Apple's MLX framework has no ANE support. There are models ANE runs well, but LLMs do not seem to be among them.

by jonathaneunice

4/15/2026 at 5:12:57 PM

It is possible but requires a very specific model design to utilize. As this reverse engineering effort has shown [0] "The ANE is not a GPU. It’s not a CPU. It’s a graph execution engine." To build one requires using a specific pipeline specifically for CoreML [1].

[0] https://maderix.substack.com/p/inside-the-m4-apple-neural-en... [1] https://developer.apple.com/documentation/coreml

by Yukonv

4/16/2026 at 3:37:37 PM

That's the best "what is ANE, really?" investigation / explanation I've seen. Directly lays out why LLMs aren't an ideal fit, its "convolution engine" architecture, the need for feeding ANE deep operation sequence plans / graphs (and the right data sizes) to get full performance, the fanciful nature of Apple's performance claims (~2x actually achievable, natch), and the (superior!) hard power gating... just _oodles_ of insight.

by jonathaneunice

4/15/2026 at 5:49:16 PM

It will be interesting to see how things change in a couple of months at WWDC, when Apple is said to be replacing their decade old CoreML framework with something more geared for modern LLMs.

> A new report says that Apple will replace Core ML with a modernized Core AI framework at WWDC, helping developers better leverage modern AI capabilities with their apps in iOS 27.

https://9to5mac.com/2026/03/01/apple-replacing-core-ml-with-...

by GeekyBear

4/15/2026 at 5:31:20 PM

ANE is OK, but it pretty much needs to pack your single vector into at least 128. (Draw Things recently shipped ANE support inside our custom inference stack, without any private APIs). For token generation, that is not ideal, unless you are using a drafter so there are more tokens to go at one inference step.

It is an interesting area to explore, and yes,this is a tech demo. There is a long way to go to production-ready, but I am more optimistic now than a few months back (with Flash-MoE, DFlash, and some tricks I have).

by liuliu

4/15/2026 at 3:50:31 PM

I'm certainly fine with it drawing some power.

Running background processes might motivate the use of NPU more but don't exactly feel like a pressing need. Actively listen to you 24/7 and analyze the data isn't a usecase I'm eager to explore given the lack of control we have of our own devices.

by tjoff

4/15/2026 at 4:02:33 PM

> Google’s engineers likely gave up on trying to compile custom attention kernels for Apple’s proprietary tensor blocks iirc.

The AI Edge Gallery app on Android (which is the officially recommended way to try out Gemma on phones) uses the GPU (lacks NPU support) even on first party Pixel phones. So it's less of "they didn't want to interface with Apple's proprietary tensor blocks" and more of that they just didn't give a f in general. A truly baffling decision.

by the_pwner224

4/15/2026 at 5:31:04 PM

Edge Gallery does have NPU support, it needs you to install the beta of AICore on the Play Store, the Edge Gallery app has instructions.

by satvikpendem

4/15/2026 at 5:33:52 PM

Huh I didn't see those instructions when I tried it last week. Must not have looked closely enough. I do remember it not having NPU support (confirmed by other people) back at the Gemma 3 launch a while ago.

by the_pwner224

4/15/2026 at 5:37:26 PM

Yes they added it for Gemma 4. Maybe it detects whether your phone has an NPU or not too. I have a OnePlus 15 which does have it.

by satvikpendem

4/15/2026 at 9:01:59 PM

It won't even let you try Gemma4 until you install a beta update to AICore as of today.

by andrewblossom

4/16/2026 at 6:55:04 PM

Where do you see that? In my app I can see both AICore and non-AICore versions.

by satvikpendem

4/15/2026 at 5:31:58 PM

Edge Gallery app on Android has NPU support but it requires a beta release of AICore so I'm sure the devs are working on similar support for Apple devices too.

by satvikpendem

4/15/2026 at 5:48:57 PM

Isn’t Apple paying Google billions of dollars to license these things? Surely they should make it easier to compile for their native engines…

by chatmasta

4/15/2026 at 5:26:40 PM

On my iphone i can choose CPU or GPU in edge gallery. What would be the difference if I used CPU?

by InMice

4/15/2026 at 5:36:50 PM

The ANE is not a fast or realistic way to infer modern LLMs.

by bigyabai

4/15/2026 at 1:54:25 PM

I made this offline pocket vibe coder using Gemma 4 (works offline once model is downloaded) on an iPhone. It can technically run the 4B model but it will default to 2B because of memory constraints.

https://github.com/blixt/pucky

It writes a single TypeScript file (I tried multiple files but embedded Gemma 4 is just not smart enough) and compiles the code with oxc.

You need to build it yourself in Xcode because this probably wouldn't survive the App Store review process. Once you run it, there are two starting points included (React Native and Three.js), the UX is a bit obscure but edge-swipe left/right to switch between views.

by blixt

4/15/2026 at 10:40:45 AM

Unfortunately Apple appears to be blocking the use of these llms within apps on their app store. I've been trying to ship an app that contains local llms and have hit a brick wall with issue 2.5.2

by codybontecou

4/15/2026 at 5:34:04 PM

In case someone don't know, this is the full text:

> 2.5.2 Apps should be self-contained in their bundles, and may not read or write data outside the designated container area, nor may they download, install, or execute code which introduces or changes features or functionality of the app, including other apps. Educational apps designed to teach, develop, or allow students to test executable code may, in limited circumstances, download code provided that such code is not used for other purposes. Such apps must make the source code provided by the app completely viewable and editable by the user.

Why is this related to local LLMs in app?

by liuliu

4/15/2026 at 6:05:23 PM

A vibe coding app that generates new executable code and runs it would:

> execute code which introduces or changes features or functionality of the app,

by GeekyBear

4/15/2026 at 11:28:58 AM

I think Apple will become increasingly draconian about LLMs. Very soon people won't need to buy many of their apps. They can just make them. This threatens Apple's entire business model.

by Gareth321

4/15/2026 at 1:22:34 PM

It came out in the Epic trial that 90% of App Store revenue comes from in app purchases of loot boxes and other pay to win mechanics.

Apple doesn’t care about revenue from a random TODO app.

by raw_anon_1111

4/16/2026 at 9:30:47 AM

From games? Why would people pay to play those games when they can make their own?

by Gareth321

4/16/2026 at 11:55:52 AM

You’re not going to make a game using AI.

by raw_anon_1111

4/16/2026 at 4:27:58 PM

Not yet, which is the point of this discussion.

by kaliqt

4/15/2026 at 5:48:10 PM

truly a k-shaped economy we live in

by thinkthatover

4/15/2026 at 12:10:10 PM

But… why would I put the effort into getting an llm to make me an app when a there’s an existing app that I don’t have to maintain? I don’t want to have to make every app I use?

by mrkpdl

4/15/2026 at 12:21:40 PM

There's a huge difference between local apps that cost one time 3-10$ and apps that ask for a subscription between 5 to 20$ per month. the first category will remain and might become more popular as quality increases, the second category will be oblitereated as the value isn't there, even if all the buyers are rich. The second group takes up a much larger part of the pie than the first though, so apple's revenue will decrease.

by orrito

4/15/2026 at 1:25:05 PM

All apps that don't have a tangible component, legal protection (like music, tv, movies), or a personality behind it will trend towards $0.

by davidmurdoch

4/16/2026 at 9:34:24 AM

It depends on the value of your time relative to the price of the application. Many of us here are well paid and time poor. It makes sense to pay for well built apps. For the world, we are truly the top 1%. 99% of people would be happy to spend a few hours building an app they need to occasionally maintain. Especially if the app is subscription based.

We should also remember that the effort of building and maintaining apps is dropping precipitously as LLMs get smarter, faster, and cheaper. OpenClaw signalled the direction in which we're heading, and within a year, Anthropic will no doubt have cheap and competent agents which can handle the maintenance autonomously in the background.

This is why SaaS valuations are getting hammered.

by Gareth321

4/17/2026 at 3:22:52 PM

> 99% of people would be happy to spend a few hours building an app they need to occasionally maintain. Especially if the app is subscription based.

With people slowly abandoning dedicated computers they fully control (if we can even call windows/macOS that) and going towards mobile/tablet computing more every year, which is a far more locked down device run by companies that are becoming increasingly hostile to “side-loading,” I just don’t see how this can become reality.

by Forgeties79

4/15/2026 at 11:54:16 AM

Apple's business model isn't really affected by 2% of its users choosing not to spend $100/yr on the App Store. That isn't even a blip on the radar.

A kid playing Roblox can spend more than that in a good weekend.

by StilesCrisis

4/15/2026 at 12:22:21 PM

VibeOS. It’s just an LLM from which all other userspace is vibed.

by borborigmus

4/15/2026 at 3:55:38 PM

vibe-ls(1) - often list directory contents, but maybe do something else.

Where can I get this amazing technology?

by username223

4/15/2026 at 12:03:04 PM

I guess I am not seeing why would I want to abandon most (if any) of my simple, small, purpose-built apps that always do the exact thing I want for a private company’s ever-changing LLM that will approximate what I’m asking and approximate its response utilizing far more resources.

I’m sure there are things on my phone it could replace (though I struggle to think of them) but there are plenty it can’t. My black magic camera app, web browsers, local send, libby/hoopla…

I can’t really think of any apps I use every day - or every week - that an LLM would replace. I’m not coding on my smartphone and aside from that an LLM is basically a more complex, somewhat inconsistent search engine experience right now for most people. Siri didn’t replace any of my apps, for instance. Why would chatGPT?

TL;DR: what apps would an LLM replace on my iPhone?

by Forgeties79

4/15/2026 at 11:27:12 AM

Though of course Apple's rules aren't always consistent, I have 2 separate apps currently on my phone that can/are running this (Google's Edge Gallery and Locally AI)

by CubsFan1060

4/15/2026 at 3:41:52 PM

They've been slowly cutting them off of updates and/or taking them off the app store entirely.

See Anywhere and Replit. Anywhere was the #1 or #2 app and was taken off the app store entirely before being put on and then taken off again.

Last I checked, Replit hasn't received an update on the iOS app store in over two months due to reviews denying them.

by codybontecou

4/15/2026 at 11:44:58 AM

Can't be just a SaaSpocolypse. LLMs with the right harness could obliterate much of the TODO+ apps with a general assistant.

But it's more likely it's just walled garden + security theatre that'll keep them from allowing outside apps.

by cyanydeez

4/15/2026 at 12:07:20 PM

Wouldn't trust AI to run TODO, especially weak models. They can hallucinate tasks, forget to remind etc.

by varispeed

4/15/2026 at 12:35:37 PM

LLMs are stateless. But given an actual database of task-shaped items and some work, I could see the potential.

With a canonical source of truth, and set input/output expectations, the potential blast radius is quite small.

by tapvt

4/15/2026 at 1:37:51 PM

And the end results is.....? What? A todo app that takes 16GB of RAM?

by wpm

4/15/2026 at 2:12:40 PM

Nothing that Mac and Windows users aren't already used to.

by bigyabai

4/15/2026 at 3:44:54 PM

It’s tempting to be flippant about MacOS/windows but in all seriousness, the resources required for an LLM to do the job of a typical lighter weight app/software is a serious consideration. No amount of bloat matches what an LLM needs.

by Forgeties79

4/15/2026 at 5:34:24 PM

> No amount of bloat matches what an LLM needs.

I don't think that's necessarily true. For instance, LinkedIn uses more memory than Gemma E2B inference does.

by bigyabai

4/15/2026 at 6:36:01 PM

LinkedIn is an entirely different category and an extreme case at that. We’re not talking about LLM’s replacing LinkedIn either. It’s an entirely different comparison/discussion.

by Forgeties79

4/15/2026 at 8:38:45 PM

Finally, we've fully documented the Singularity-is-actually-just-bloated software.

by cyanydeez

4/16/2026 at 7:18:18 AM

I disagree. My iphone app ships with an ANE optimized LLM that runs fully offline. Sailed straight through the App Store review same day after only one minor correction. It could be possible that Apple gives apps that use LLMs for it's core functionality a pass as long as it has nothing to do with vibe coding. The recent removal of the the Anything Vibe code app supports the thesis that Apple wants to prevent a flood of Ai slop apps; at least in theory as Apple can't block people from building with Claude Code & Co.

https://www.macrumors.com/2026/03/30/apple-pulls-vibe-coding...

by marvin-hansen

4/15/2026 at 1:01:52 PM

Is this an issue with Cactus compute stuff as well?

by pj_mukh

4/15/2026 at 12:20:10 PM

What is your app doing? Just LLM inference?

by MillionOClock

4/15/2026 at 3:43:09 PM

It's a custom agent harness with on-device models and the ability to swap between models.

Basically, a "toy" app to showcase where we are with coding agents on-device.

by codybontecou

4/15/2026 at 11:27:34 AM

Use of the LLMs to do what?

by saagarjha

4/15/2026 at 11:30:51 AM

[dead]

by throwaway613746

4/15/2026 at 2:43:24 PM

Seriously, how do people put up with being nannied by Apple?

Come on folks, their IT hardware may be nice but supporting them is not worth it.

by amelius

4/15/2026 at 10:41:23 AM

Related: Gemma 4 on iPhone (254 comments) - https://news.ycombinator.com/item?id=47652561

by karimf

4/15/2026 at 11:16:58 AM

Another related submission from 22 days ago : iPhone 17 Pro Demonstrated Running a 400B LLM (+700pts, +300cmts): https://news.ycombinator.com/item?id=47490070

by redbell

4/15/2026 at 12:03:17 PM

That's very impressive but it's streaming in weights from flash storage. That's not really viable in a mobile context, it will use way too much power. Smaller models are way more applicable to typical use, perhaps with mid-sized models (like the Gemma4 26A4B model) using weights offload from SSD for rare uses involving slower "pro" inference.

by zozbot234

4/15/2026 at 5:49:41 PM

10 minutes a day of extreme power usage is probably fine for people asking for directions to the store, setting calendar reminders, timers, checking for important emails etc. AI on your phone will be incredibly useful but power usage doesn't matter when total usage is less than 15 minutes per day. I don't think the average person expects to vibe code on the phone for 8 hours a day.

by hadlock

4/15/2026 at 6:39:28 PM

10 minutes a day or 15 minutes a day is what the inference workload is like on fairly small models. Once you start streaming in weights from SSD, things slow down quite a bit and become quite power hungry.

by zozbot234

4/15/2026 at 10:56:51 AM

For those who would like an example of its output, I'm currently working through creating a small, free (cc0, public domain) encyclopedia (just a couple of thousand entries) of core concepts in Biology and Health Sciences, Physical Sciences, and Technology. Each entry is being entirely written by Gemma 4:e4b (the 10 GB model.) I believe that this may be slightly larger than the size of the model that runs locally on phones, so perhaps this model is slightly better, but the output is similar. Here is an example entry:

https://pastebin.com/ZfSKmfWp

Seems pretty good to me!

by logicallee

4/15/2026 at 11:45:28 AM

What's your goal? Do you have a project you want the encyclopedia for?

by everyday7732

4/16/2026 at 3:03:35 PM

I've now published the encyclopedia.[1] I wrote the goals on that page, which are educational. Or you can use it to judge the state of Gemma 4 and its knowledge of these areas.

[1] https://stateofutopia.com/encyclopedia/

by logicallee

4/15/2026 at 1:16:10 PM

Strangely, it is super fast on my 16 Plus, but with longer messages it can slow down a LOT, and not because of thermal throttling. I wish I could see some diagnostic data.

by mfro

4/15/2026 at 1:23:54 PM

Inference from an LLM is O(tokens^2)

by steve-atx-7600

4/15/2026 at 3:53:59 PM

Only in the naive implementations of attention

by halJordan

4/15/2026 at 12:28:44 PM

I’m pretty excited about the edge gallery ios app with gemma 4 on it but it seems like they hobbled it, not giving access to intents and you have to write custom plugins for web search, etc. Does anyone have a favorite way to run these usefully? ChatMCP works pretty well but only supports models via api.

by conception

4/15/2026 at 12:24:19 PM

I just installed Google Ai Edge Gallery on my iPhone 16 pro, here are the results of the first benchmark with GPU, Prefill Tokens=256, Decode Tokens=256, Number of runs: 3. Prefill Speed=231t/s, Decode Speed=16t/s, Time to First Token=1.16s, First init time=20s

by Chrisszz

4/15/2026 at 5:40:23 PM

Careful with using these small models. The other day, I asked it "Can dogs eat avocado" and answer was emphatic Yes.

This is not meant as a criticism, but people should be aware of their limitations.

by abc_lisper

4/15/2026 at 5:41:36 PM

well, technically they can ...

by jacobr1

4/15/2026 at 8:02:03 PM

...once

by DaiPlusPlus

4/15/2026 at 3:04:36 PM

Offline or not, I'm sure Google uploads every keystroke, phone orientation, photo, WiFi endpoints and your shoe size when you interact with it. To enhance your experience.

by rich_sasha

4/15/2026 at 3:53:03 PM

They released the source (well, currently only the Android version) at https://github.com/google-ai-edge/gallery .

At a glance, I see they do gather analytics about how much the app is used (model downloads, model invocations etc) without message content, pretty much just the model used.

by adrian17

4/15/2026 at 3:19:12 PM

> ...your shoe size

The funny thing is that a lot of Google's internal training content uses an imaginary product "gShoe", and discusses the privacy implications of data that such a shoe might collect :D

by tsycho

4/15/2026 at 3:27:57 PM

Apple is paying Google $1billion for an AI strategy that runs on device. We're seeing the preview of what that will look like.

by declan_roberts

4/15/2026 at 9:52:46 AM

It runs on Android too, with AI Core or even with llama.cpp

by mistic92

4/15/2026 at 12:54:36 PM

Its more impressive when Apple does it because they are so far behind.

I remember being excited when Apple got widgets because then I could add my 'Next Alarm time' to my home screen. Made my company work phone usable on trips.

I wonder when they are going to get NVIDIA cards or CUDA? Then they can actually run LLMs and not just trick people into buying it under the 30 year old idea of 'Unified Memory'.

by srslyTrying2hlp

4/15/2026 at 1:14:51 PM

It's kinda funny that macOS supported CUDA when it was a tech demo, but then ideologically objects to it once it's a $3 trillion business.

They've had to be dragged kicking and screaming away from the NPU model only to admit that GPGPU tech was the right choice.

by bigyabai

4/15/2026 at 1:34:06 PM

Yeah I remember that. Very Apple of them.

'Cool demo' -> Doesnt convert to tangible things.

Wont attempt to compete with companies better than them, but go their own route. "oh look it consumes low power!" (Things no one cared about).

They are the Nintendo of tech.

by srslyTrying2hlp

4/15/2026 at 11:02:59 AM

ESET is blocking this site saying:

Threat found This web page may contain dangerous content that can provide remote access to an infected device, leak sensitive data from the device or harm the targeted device. Threat: JS/Agent.RDW trojan

by usmanshaikh06

4/15/2026 at 1:21:17 PM

Same on my device.

by zache6

4/15/2026 at 1:51:33 PM

I feel like UX and API design are very under explored.

What are the possibilities of an Android or iOS device where the OS is centered around a locally running LLM with an API for accessing it from apps, along with tools the LLM can call to access data from locally running apps? What’s the equivalent of the original Mac OS?

Do apps disappear and there’s just a running dialog with the LLM generating graphical displays as needed on demand?

by jimbokun

4/15/2026 at 10:40:05 AM

> edge AI deployment

Isn't the "edge" meant to be computing near the user, but not on their devices?

by pabs3

4/15/2026 at 11:07:29 AM

No it does not. This is about as “edge” as AI gets.

In a general sense, edge just means moving the computation to the user, rather than in a central cloud (although the two aren’t mutually exclusive, eg Cloudflare Workers)

by stingraycharles

4/15/2026 at 1:40:05 PM

For sure. 1000%. Anyone disagreeing with this has lost their marbles.

For those that have lost their marbles: sure, people use words incorrectly, but that does mean we all have to use those words incorrectly.

In compute vernacular, "edge" means it's distributed in a way that the compute is close to the user (the "user" here is the device, not a person); "on device" means the compute is on the device. They do not mean the same thing.

by davidmurdoch

4/15/2026 at 10:55:25 AM

Your device is the ultimate edge. The next frontier would be running models on your wetware.

by pgt

4/15/2026 at 11:14:16 AM

Not just running it on your wetware, but charging you for it.

Can't wait until AI companies go from mimicking human thought to figuring how to licensing those thoughts. ;)

by elcritch

4/15/2026 at 11:06:48 AM

Man can't wait for AI in my brain. And then intelligence will be pay to win.

by acters

4/15/2026 at 11:04:11 AM

It depends, because edge is a meaningless term and people choose what they want for it. In 2022, we set up a call with a vendor for ‘edge’ AI. Their edge meant something like 5kW, and our edge was a single raspberry pi in the best case.

by hhh

4/15/2026 at 4:09:39 PM

Gemma4 is still power hungry since it tends to activate pretty much every weight.

qwen3-coder-next uses a lot less since it seems to only activate ~3B parameters at a time.

My guess is that this is still close to tech demo, and a lot of performance is left on the table.

by juancn

4/15/2026 at 11:33:01 PM

The article is about two models which have either 2B or 4B parameters. Both are dense models. The 2B version will certainly use less power than qwen3-coder-next.

The models are quite good. They aren't just a tech demo.

by fancyfredbot

4/15/2026 at 2:01:10 PM

They still don’t render the markdown (or LaTeX) it outputs.

by deckar01

4/15/2026 at 12:13:09 PM

Would love to see a show down of performance on iPhone vs Googles Tensor G5, which in my experience the G5 is 2 full generations behind performance wise.

by bearjaws

4/15/2026 at 3:26:55 PM

I really hope this is a preview of the replacement for Siri that Google is creating bc these models are fantastic for their size!

by declan_roberts

4/15/2026 at 3:48:23 PM

Google is not creating a replacement for anything.

Apple is getting a base Gemini model (not a Gemma), and it will run on Apple private compute. Apple foundational models will remain the on device model

by halJordan

4/15/2026 at 12:59:04 PM

does anyone know of a decent but low memory or low parameter count multilingual model (as many languages as possible), that can faithfully produce the detailed IPA transcription given a word in a sentence in some language?

I want to test a hypothesis for "uploading" neural network knowledge to a user's brain, by a reaction-speed game.

by DoctorOetker

4/15/2026 at 1:13:03 PM

Espeak-ng.

You don't need a neural network. Traditional NLP is far better at this task. The keyword you're looking for is "phoenemizer"

by estimator7292

4/15/2026 at 1:23:54 PM

can Espeak-ng provide the IPA transcription? or does it produce sound?

I'm surprised traditional NLP being better than ML models for this task, can you point me to a benchmark analysis pointing out that non-neural Espeak-ng is better than ML models?

Also, I asked for a neural model for another reason as well, I still want semantic knowledge present, I want more than pronunciation, but before I use myself as a test subject, I want to make sure I get the proper pronunciation in case the highly speculative "uploading game" works... I don't want to early systematically mis-train myself on pronunciation...

by DoctorOetker

4/15/2026 at 10:09:11 AM

Is the output coherent though? I am yet to see a local model working on consumer grade hardware being actually useful.

by bossyTeacher

4/15/2026 at 11:50:57 AM

I have a 128 GB Strix Halo tablet (same as the other commenter here with the Framework Desktop). I'm using the larger Gemma 4 26B-A4B model (only 28 GB @ Q8) and it's been working great and runs very fast.

It's a 100% replacement for free ChatGPT/Gemini.

Compared to the paid pro/thinking models... Gemma does have reasoning, and I have used the reasoning mode for some tax & legal/accounting advice recently as well as other misc problems. It's worked well for that, but I haven't tried any real difficult tasks. From what I've heard re. agentic coding, the open weight models are ~18-24 months behind Anthropic & Google's SOTA.

Qwen 3.5 122B-A10B should just fit into 128 GB with a Q4/5 and may be a bit smarter. There's apparently also a similar sized Gemma 4 model but they haven't released it yet, the 26B was the largest released.

by the_pwner224

4/15/2026 at 12:15:13 PM

There's a 31B dense model in the Gemma 4 series that's obviously going to be smarter (though a whole lot slower) than the MoE 26A4B.

by zozbot234

4/15/2026 at 12:18:45 PM

I tried it and it was unusably slow at ~5-6 TPS. 26A4B gets close to 40 TPS which is faster than you can read, and still pretty quick with reasoning enabled.

by the_pwner224

4/15/2026 at 3:58:07 PM

Sure, 26B models on beefy desktop silicon are finally nipping at the heels of commercial APIs, but this is a mobile thread. On a phone with 8GB of RAM and passive cooling, your tokens per second (t/s) are going to fall off a cliff after the first minute of sustained compute

by veunes

4/15/2026 at 11:13:06 AM

It's highly coherent (see my other comment for an example of its text output) and yes, it's useful. I am starting to use Gemma 4:e4b as my daily driver for simple commands it definitely knows, things that are too simple to use ChatGPT for. It is also able to code through moderately difficult coding tasks. If you want to see it in action, I posted a video about it here[1] (the 10 GB one is at the 2 minute mark and the 20 GB one says hello at 5 minutes 45 seconds into the video.) You can see its speed and output on simple consumer grade hardware, in this case a Mac Mini M4 with 24 GB of RAM.

[1] https://youtube.com/live/G5OVcKO70ns

by logicallee

4/15/2026 at 11:19:09 AM

Google's models work quite well on my Android phone. I haven't found a use case beyond generating shitposts, but the model does its job pretty well. It's not exactly ChatGPT, but minor things like "alter the tone of this email to make it more professional" work like a charm.

You need a relatively beefy phone to run this stuff on large amounts of text, though, and you can't have every app run it because your battery wouldn't last more than an hour.

I think the real use case for apps is more like going to be something like tiny, purpose-trained models, like the 270M models Google wants people to train and use: https://developers.googleblog.com/on-device-function-calling... With these things, you can set up somewhat intelligent situational automation without having to work out logic trees and edge cases beforehand.

by jeroenhd

4/15/2026 at 3:11:47 PM

I'm working on a visual description app for the blind. even Gemma 4 E2B can give very useful image descriptions while at the same time taking questions as audio. It's also much faster than most of the current popular cloud based apps like Be My Eyes.

by RobMurray

4/15/2026 at 10:12:48 AM

I run qwen3.5 122b on a Framework Desktop at 35/ts as a daily driver doing security and OS systems and software engineering.

Never paid an LLM provider and I have no reason to ever start.

by lrvick

4/15/2026 at 10:42:32 AM

What spec of Framework Desktop do you run this on?

by mixermachine

4/15/2026 at 11:36:10 AM

If you're looking to buy new hardware, also consider the Asus Rog Flow Z13. It has the same chip as the Framework desktop and is ~20% cheaper ($2,700) for the 128 GB spec while coming in a tablet/laptop form factor. It's capped at a slightly lower power but Strix Halo scales down very well in TDP - I never even use the max power mode on my Z13 because you don't really get any extra perf.

The only downside is that I suspect the Framework would be a decent bit quieter under load (not that this thing is abnormally loud). As well as you're limited to a single M.2 2230 internal SSD slot in this (I believe Micron recently launched a 4 TB model, but generally you'll max out at 2 TB without using an external enclosure).

I don't have anything against the Framework, I'm sure it's a great machine, but the Z13 is an incredible portable all-in-one device that can handle everything from general PC use to gaming to tablet/entertainment to LLMs & high perf.

by the_pwner224

4/15/2026 at 6:37:48 PM

The main reason to get the framework is the DIY edition for the mini-itx form factor board and support. If you do not care about those then any cheap proprietary format box with the Strix Halo chip will do.

I put my boards in mini itx rack mounts personally so framework is the only option.

by lrvick

4/15/2026 at 11:09:02 AM

There is only one and for this model you need the one with 128GiB RAM.

by breisa

4/15/2026 at 6:36:38 PM

I have the 128 but for Qwen3.5 122b XS quant you only need 64GB

by lrvick

4/15/2026 at 10:46:19 AM

Qwen3.5-9b and Qwen3.5-27b are pretty coherent on my 24G android phone

by fsiefken

4/15/2026 at 10:51:54 AM

Which android phone has 24G?

by dpacmittal

4/17/2026 at 8:01:27 PM

For example: ASUS ROG Phone 9 Pro, Nubia RedMagic 10 Pro, OnePlus 13, Realme GT 5

by fsiefken

4/15/2026 at 10:19:13 AM

It can write (some) code that works. Just roughly guessing from my use, but I think of it as being a bit like ChatGPT circa-2024 in terms of capability & speed.

Disappointing if you compare it to anything else from 2026, but fairly impressive for something that can run locally at an OK speed.

by jfoster

4/15/2026 at 10:13:26 AM

I can try it for you

by a_paddy

4/15/2026 at 5:37:28 PM

This is HN clickbait. No details or evidence, this is just generated for votes.

I think this should be flagged.

by politelemon

4/15/2026 at 12:31:53 PM

You are referring to the edge models, right? E2B and E4B, not the bigger ones (26B, 31B)...

by the_inspector

4/15/2026 at 11:05:12 AM

There are many apps to run local LLMs on both iOS & Android

by ValleZ

4/15/2026 at 12:55:30 PM

Once you realize Apple and Nintendo have 'understandings' with media outlets, you will start to realize this is just marketing.

by srslyTrying2hlp

4/15/2026 at 5:59:26 AM

is there a comparison of it running on iPhone vs. Android phones?

by andsoitis

4/15/2026 at 11:54:56 AM

Running Gemma-4-E2B-it on an iPhone 15 (can't go higher than that due to RAM limitations) versus a Pixel 9 Pro, I don't really notice much of a difference between the two. The Pixel is a bit faster, but also a year more recent.

The model itself works absolutely fine, though the iPhone thermal throttles at some point which really reduces the token generation speed. When I asked it to write me a business plan for a fish farm in the Nevada desert, it slowed down after a couple thousand tokens, whereas the Pixel seems to just keep going.

by jeroenhd

4/15/2026 at 3:49:32 PM

It’s likely a llama.cpp backend issue. On the Pixel, inference hits QNN or a well-optimized Vulkan path that distributes the SoC load properly. On the iPhone, everything is shoved through Metal, which maxes out the GPU immediately and causes instant overheating. Until Apple opens up low-level NPU access to third-party models, iPhones will just keep melting on long-context prompts

by veunes

4/15/2026 at 10:13:34 AM

You can run Android on just about anything so it boils down to Linux GPU benchmarks.

by lrvick

4/15/2026 at 10:55:18 AM

That doesn't answer the question, I'm curious too. I think there's a speed and battery advantage on the A19 Pro chip compared to the Snapdragon 8 Elite Gen 5 chip, but to know for sure one has to run the same model used in the most efficient way on both machines (flagships ios and android).

by fsiefken

4/15/2026 at 1:01:01 PM

I dont think you should have been downvoted. Processing and memory are the only thing that matters. (Unless we are being so nontechnical now that we just say things like Pixel 9 is great...)

by srslyTrying2hlp

4/15/2026 at 12:41:39 PM

Do you know of a way of running these models on Android? Also, what does the thermal throttling look like?

by grimmai143

4/15/2026 at 12:53:16 PM

Edge Gallery by Research at Google

by robmccoll

4/15/2026 at 12:34:07 PM

[dead]

by grimm7000

4/15/2026 at 10:29:47 AM

[flagged]

by camillomiller

4/15/2026 at 1:58:29 PM

I'd rather ban comments complaining about it. Judge the content by its merits.

by macintux

4/15/2026 at 12:59:09 PM

My only issue with AI is that its so verbose.

What we should encourage are people to simplify with AI.

There was that literal 9 page rant by the other day and people had no idea what he was trying to say (too technical language, a poor writer, and seemingly unedited). That could have benefited greatly.

by srslyTrying2hlp

4/15/2026 at 1:18:08 PM

[dead]

by tom2026hn

4/15/2026 at 11:10:01 AM

I find it fascinating that after all this time reporters still don’t even bother to proofread content for obvious AI tells. I guess nobody really cares anymore?

by stingraycharles

4/15/2026 at 11:06:34 AM

That bugged me too, so I started looking at other articles - they all look AI generated to me. Whole website should be banned.

by dax_

4/15/2026 at 4:28:35 PM

I don't see the value in this post, are hacker news post being upvoted by bots?

by abstracthinking

4/15/2026 at 4:30:35 PM

[dead]

by sieabahlpark