alt.hn

4/6/2026 at 12:14:37 AM

Show HN: Gemma Gem – AI model embedded in a browser – no API keys, no cloud

 https://github.com/kessler/gemma-gem

by ikessler

4/6/2026 at 2:31:16 AM

There's also the Prompt API, currently in Origin Trial, which supports this api surface for sites:

https://developer.chrome.com/docs/ai/prompt-api

I just checked the stats:

  Model Name: v3Nano
  Version: 2025.06.30.1229
  Backend Type: GPU (highest quality)
  Folder size: 4,072.13 MiB
Different use case but a similar approach.

I expect that at some point this will become a native web feature, but not anytime soon, since the model download is many multiples the size of the browser itself. Maybe at some point these APIs could use LLMs built into the OS, like we do for graphics drivers.

by avaer

4/6/2026 at 6:41:58 AM

That’s exactly where we’re headed. Architecturally it makes zero sense to spin up an LLM in every app's userspace. Since we have dedicated NPUs and GPUs now, we need a unified system-level orchestrator to balance inference queues across different programs - exactly how the OS handles access to the NIC or the audio stack. The browser should just be making an IPC call to the system instead of hauling its own heavy inference engine along for the ride

by veunes

4/6/2026 at 11:49:12 AM

FWIW - I did a real world experiment pitting the built in Gemini Nano vs a free equivalent from OpenRouter (server call) and the free+server side was better in literally every performance metric.

That's not to say that the in browser isn't valuable for privacy+offline, just that the standard case currently is pretty rough.

https://sendcheckit.com/blog/ai-powered-subject-line-alterna...

by michaelbuckbee

4/6/2026 at 2:11:19 PM

It's worth mentioning that "Gemini Nano 4" is going to be Gemma 4, and presumably when it becomes the default Nano model, it should improve performance quite a bit.

(It's currently available for testing in Android's AICore under a developer preview)

by spijdar

4/6/2026 at 6:08:53 AM

The Summarizer API is already shipped, and any website can use it to quietly trigger a 2 GB download by simply calling

    Summarizer.create()
(requires user activation)

by sheept

4/6/2026 at 5:25:51 AM

Interesting!

by oyebenny

4/6/2026 at 6:36:26 AM

It’s a neat idea, but giving a 2B model full JS execution privileges on a live page is a bit sketchy from a security standpoint. Plus, why tie inference to the browser lifecycle at all? If Chrome crashes or the tab gets discarded, your agent's state is just gone. A local background daemon with a "dumb" extension client seems way more predictable and robust fwiw

by veunes

4/6/2026 at 8:24:29 AM

> but giving a 2B model full JS execution privileges on a live page is a bit sketchy from a security standpoint.

Every webpage I've ever visited has full JS execution privileges and I trust half of them less than an LLM

by shawabawa3

4/6/2026 at 10:27:30 AM

Note that every webpage does not have full JS execution privileges on other parts of the web.

by saagarjha

4/6/2026 at 6:39:24 PM

At least in this case (not so sure about the Prompt API case mentioned in another thread) the agent is "in" the page. And that means that the agent is constrained by the same CORS limits that constrain the behavior of the page's own JS.

If you think about it, everything we've done to make malicious webpages unable to fiddle around with your state on other sites using XHRs, are exactly and already the proper set of constraints we'd want to prevent models working with webpages from doing the same thing.

by derefr

4/6/2026 at 7:17:17 AM

There's indexed db, opfs, etc. Plenty of ways to store stuff in a browser that will survive your browser restarting. Background daemons don't work unless you install and start them yourself. That's a lot of installation friction. The whole point of a browser app is that you don't have to install stuff.

And what you call sketchy is what billions of people default to every day when they use web applications.

by jillesvangurp

4/6/2026 at 1:05:44 PM

I was thinking the same thing: better to run models using a local service not in the wen browser. I use Ollama and LM Studio, switching between which service I have running depending on what I am working on. It should be straight forward to convert this open source project to use a different back end.

That said this looks like a cool project. It is so valuable writing projects like this that use local models, both for tool building and self education. I am writing my own “Emacs native” agentic coding harness and I am learning a lot.

by mark_l_watson

4/8/2026 at 3:25:08 PM

even Anthropic messed up LLM in a Chrome extension, not sure I would want this in my browser.

by siddbudd

4/6/2026 at 4:08:19 AM

I would love to see someone build it as some kind of an SDK. App builders could use it as a local LLM plugin when dealing with data involving sensitive information.

It's usually too much when an app asks someone to setup a local LLM but this I believe could solve that problem?

by emregucerr

4/6/2026 at 7:29:21 AM

It's not too hard to code together with an LLM. I've been playing with small embeddings models in browsers in the last weeks. You don't really need that much. The limitation is that these things are fairly limited and slow to begin with and they run slower in a browser even with webgpu. But you can do some cool stuff. Adding an LLM is just more of the same.

If you want to see an example of this, https://querylight.tryformation.com/ is where I put my search library and demo. It does vector search in the browser.

by jillesvangurp

4/6/2026 at 4:51:48 AM

Which apps have you seen ask for someone to setup a local LLM? Can't recall having ever seen one

by winstonp

4/6/2026 at 3:33:05 PM

[dead]

by randomexceptio

4/6/2026 at 12:38:52 PM

[dead]

by xkbear89

4/6/2026 at 4:13:40 AM

Not sure if I actually want this (pretty sure I don't) -- but very cool that such a thing is now possible...

by montroser

4/6/2026 at 8:45:33 AM

I have this written a a project I will attempt to do in the future, I also call it "weapons grade unemployment" in the notes I was proposing to use granite but the principle still stands. You beat me to it.

by dabrez

4/6/2026 at 6:35:36 AM

it would be awesome if a local model would be directly embeded to chrome and developer could query them.

Anyone know if this is somehow possible without going through an extension?

by eric_khun

4/6/2026 at 3:31:48 PM

[dead]

by randomexceptio

4/6/2026 at 10:09:09 PM

"Show HN: Gemma Gem – AI model embedded in a browser – no API keys, no cloud"

by kabir_daki

4/6/2026 at 3:18:10 AM

[flagged]

by Morpheus_Matrix

4/6/2026 at 3:28:47 PM

[dead]

by agdexai

4/7/2026 at 7:32:01 AM

[flagged]

by aimadetools