alt.hn

3/24/2026 at 2:58:27 PM

Show HN: Gemini can now natively embed video, so I built sub-second video search

https://github.com/ssrajadh/sentrysearch

by sohamrj

3/24/2026 at 5:56:59 PM

This is a really cool implementation—embeddings still often feel like magic to me. That said, this exact use case is sort of also my biggest point of concern with where AI takes us, much more so than most of the common AI risks you hear lots of chatter about. We live in a world absolutely loaded with cameras now but ultimately retain some semblance of semi-anonymity/privacy in public by virtue of the fact that nobody can actually watch or review all of the video from those cameras except when there is a compelling reason to do so, but these technologies are making that a much more realistic proposition.

The presence of cameras everywhere is considerably more concerning than the status quo, to me at least, when there is an AI watching and indexing every second of every feed—where camera owners or manufacturers or governments could set simple natural language parameters for highly specific people or activities notify about. There are obviously compelling and easy-to-sell cases here that will surely drive adoption as it becomes cost effective: get an alert to crime in progress, get an alert when a neighbor who doesn't clean up after his dog, get an alert when someone has fallen...but the potential implications of living in a panopticon like this if not well regulated are pretty ugly.

by macNchz

3/24/2026 at 7:07:52 PM

It's being built as we speak. I attended at a city council meeting yesterday, discussing approving a contract for ALPR cameras. I learned about a product from the camera vendor called Fusus[0], a dashboard that integrates various camera systems, ALPRs, alerts, etc. Two things stood out to me: natural-language querying of video feeds, and future planned integration with civilian-deployed cameras. The city only had budget for 50 ALPRs, and they stressed how they're only deploying them on main streets, but it seems like only a matter of time before your neighbor is able to install a camera that feeds right into the local PD's AI-enabled systems. One council member raised concerns about integrations with the citizen app[1] specifically (and a few others I didn't catch the names of). I'm very worried about where all this is heading.

[0]: https://www.axon.com/products/axon-fusus [1]: https://citizen.com/

by citruscomputing

3/24/2026 at 6:33:59 PM

Totally valid concern. Right now the cost ($2.50/hr) and latency make continuous real-time indexing impractical, but that won't always be the case. This is one of the reasons I'd want to see open-weight local models for this, keeps the indexing on your own hardware with no footage leaving your machine. But you're right that the broader trajectory here is worth thinking carefully about.

by sohamrj

3/24/2026 at 10:28:41 PM

How are you getting to $2.50/hr ? The price sheet says its 0.00079 per frame.

https://ai.google.dev/gemini-api/docs/pricing#gemini-embeddi...

by jimmySixDOF

3/24/2026 at 11:46:43 PM

From what I see the code downsamples video to 5 fps, so 1 hour of video is 3600 seconds * 5 fps = 18,000 frames. 18,000 frames * $0.00079/frame = $14.22. A couple dollars more with the overlap.

(The code also tries to skip "still" frames, but if your video is dynamic you're looking at the cost above.)

by jjwiseman

3/25/2026 at 1:35:00 AM

you're right that the code uses ffmpeg to downsample the chunks to 5fps before sending them, but that's only a local/bandwidth optimization, not what the api actually processes.

regardless of the file's frame rate, the gemini api natively extracts and tokenizes exactly 1 fps. the 5 fps downscaling just keeps the payload sizes small so the api requests are fast and don't timeout.

i'll update the readme to make this more clear. thanks for bringing this up.

by sohamrj

3/25/2026 at 4:35:34 AM

Thanks for the details and correction.

by jjwiseman

3/24/2026 at 7:06:42 PM

It's 2.50 an hour because Google has margins. A nation state could do it at cost, and even if it's not a huge difference, the price of a year's worth of embeddings is just $21,900. That's a rounding error, especially considering it's a one time cost for footage.

by mpalmer

3/24/2026 at 7:12:01 PM

Right? $2.50 an hour is trivial to a Government that can vote to invent a trillion dollars. Even just 1 million dollars is the cost of monitoring 45 real time feeds for a year. I'm sure just many very rich people would pay that for the safety of their compound.

by wholinator2

3/24/2026 at 11:57:45 PM

Once the hardware to run inference for something like the vision understanding module of this can be run on a low / medium power asic drones are going to be absolutely horrifying weapons.

by FuckButtons

3/24/2026 at 7:19:30 PM

Yeah, the panopticon is now technically very feasible it's just expensive to implement (for now).

by cake_robot

3/24/2026 at 11:48:42 PM

Its very cheap to target an individual though so they dont need to look everywhere

by whattheheckheck

3/24/2026 at 7:27:45 PM

Most cameras are also not queryable by any one person or organization. They are owned by different companies and if the government wants access they have to subpoena them after the fact.

The problems start cropping up when you get things like Flock where governments start deploying cameras on a massive scale, or Ring where a single company has unrestricted access to everyone's private cameras.

by Ajedi32

3/24/2026 at 7:56:22 PM

I think Flock is just a symptom of the underlying tech becoming so cheap that "just blanket the city in cameras" starts to sound like a viable solution when police rely so heavily on camera footage.

I don't think it's a good thing but it seems the limiting factor has been technological feasibility instead of any kind of principle against it.

by Spivak

3/25/2026 at 12:42:59 AM

> this exact use case is sort of also my biggest point of concern with where AI takes us, much more so than most of the common AI risks you hear lots of chatter about.

I've been hearing warnings that AI would be used for this since well before it seemed feasible.

by zahlman

3/25/2026 at 2:24:24 AM

Not claiming to have hit on something unique here, but I think it’s realistic and often drowned out in favor of sci-fi nonsense.

by macNchz

3/24/2026 at 7:44:39 PM

For specific people they probably wouldn’t use general embeddings. These embeddings can let you search for “tall man in a trenchcoat” but if you want a specific person you would use facial recognition.

by janalsncm

3/24/2026 at 8:01:38 PM

I think a general description is better for surveillance/tracking like this, no? If they're at a weird angle or intentionally concealing their face then facial recognition falls apart but being able to describe them naturally would result in better tracking IMO.

by hypeatei

3/24/2026 at 10:22:01 PM

Presumably the ideal is some kind of a fusion. Upload or tag some images/videos and link someone's social profiles and the system can look out for them based on facial recognition, gait recognition, vehicle/pets/common wardrobe items in combination.

by macNchz

3/24/2026 at 8:42:57 PM

All the major cloud providers offer some form of face detection and numberplate reading, with many supporting object detection (ie package, vehicle, person) out of the camera itself.

by greggsy

3/24/2026 at 10:26:09 PM

It's definitely creeping into things, though most of the features I've seen are fairly simplistic compared to what would be possible if the video was being reviewed + indexed by current SoTA multimodal LLMs.

by macNchz

3/25/2026 at 4:01:11 AM

Multimodal AI will lead to an interesting arms race in ad detection vs ad insertion. I played around with AI ad removal with older Gemini models, but it seems like this would be even more powerful to instantly identify ads (and potentially mute or strip them out).

https://notes.npilk.com/experiments-with-ai-adblock

by npilk

3/25/2026 at 7:07:51 AM

Nice article. I saw someone depicting the future of web search with AI. The conclusion was not the bright future. Simply put, ads will never go away. Either AI providers will get paid for whitelisting ads, or even worse these AI will directly promote advertised products.

by sbinnee

3/24/2026 at 7:52:18 PM

I picked up a Rexing dash cam a few months back and after getting frustrated with how clunky it is to get footage of it, I decided to look into building something out myself to browse and download the recordings without having to pull the SD card. While scrolling through the recordings, I explicitly remember thinking it would be nice to just describe what I was looking for and run a search. Looking forward to incorporating this into my project.

Thanks for sharing!

by rigrassm

3/24/2026 at 7:39:08 PM

Could this be used for creating video editing software?

Imagine a Premiere plugin where you could say "remove all scenes containing cats" and it'll spit out an EDL (Edit Decision List) that you can still manually adjust.

by cloogshicer

3/24/2026 at 8:07:10 PM

Yeah, this is a great idea, I’ve actually been thinking about exactly this as the next logical step.

SentrySearch already returns precise in/out timestamps for any natural-language query and uses ffmpeg to auto-trim clips. Turning that into an EDL (or even a direct Premiere plugin that exports an editable cut list) feels natural.

I’m not a Premiere expert myself, but I’d love to see this happen. If you (or anyone) wants to sketch out a quick EDL exporter or plugin, I’ll happily review + merge a PR and help wherever I can. Just drop a GitHub issue if you start something!

by sohamrj

3/24/2026 at 4:56:54 PM

Where is the Exit to this dystopia?

by emsign

3/24/2026 at 6:18:40 PM

Well, with data analysis powers like this a few treasonous words in front of a flock camera will show you the way.

by nclin_

3/24/2026 at 5:42:20 PM

In the matrix the exit was pay phones, which perhaps explains why our overlords are removing them

by RobotToaster

3/25/2026 at 7:57:47 AM

You don’t wanna live in Night City?

by moomoo11

3/24/2026 at 10:16:18 PM

https://pauseai.info/

by anxoo

3/25/2026 at 7:10:37 AM

Thanks for sharing. They say "pause", not stop. Assume that we pause now. When should we resume then? How do we know?

by sbinnee

3/24/2026 at 5:59:27 PM

I don’t think this means we’re in a dystopia

by jama211

3/24/2026 at 6:27:47 PM

You might not have been paying attention

by zwirbl

3/24/2026 at 6:49:16 PM

I think Radiohead said that

by 52-6F-62

3/24/2026 at 6:06:46 PM

The dystopia of searching for video clips and finding them? What?

by draw_down

3/24/2026 at 10:14:37 PM

Yes? Right now it is relatively expensive to search video. As embedding tech like this advances and makes it even cheaper it just increases the ability to search and analyze every movement. “Locate speech patterns that indicate dissident activity using the dissident activity skill”

by bitexploder

3/24/2026 at 5:13:34 PM

The Matrix style human pods: we live in blissful ignorance in the Matrix, while the LLMs extract more and more compute power from us so some CEO somewhere can claim they have now replaced all humans with machines in their business.

by BrokenCogs

3/24/2026 at 5:26:48 PM

I was thinking more of the season 3 episode of Doctor Who titled Gridlock where everyone lives in flying cars circling a giant expressway underground, while all the upper class people on the surface died years ago from a pandemic.

by throwup238

3/24/2026 at 5:37:57 PM

Ever get the feeling that the universe is reading your mind? Maybe there's some truth to that after all.

by ting0

3/25/2026 at 8:39:26 AM

> Check if a video chunk contains mostly still frames. Extracts 3 evenly-spaced frames as JPEG and compares file sizes.

I believe you could use a combination of select and scene parameters in ffmpeg to do this automatically when a chunk of video is created each time.

by bob1029

3/24/2026 at 10:29:47 PM

Damn, I need to going with my embeddings project. I've currently got a prototype for using embeddings (not gemini in my case) for making a game that's kinda reverse connections:

collections.lwarfield.dev

by lwarfield

3/24/2026 at 4:23:03 PM

Very interesting (not for a dashcam, but for home monitoring).

by mdrzn

3/25/2026 at 5:30:23 AM

Most home monitoring only records when there is movement though? So that already compresses the search space a lot. And just zipping forward and back it's pretty easy to quickly find the 30 seconds where there is a figure wallking up to your front door.

by SoftTalker

3/25/2026 at 12:51:20 AM

this function will be a must-have for all home security systems. I used to spend hours going through home security cameras to check if our cat went out the house when the door was accidentally left open (turned out it was just really good at hiding within the house).

by fhe

3/25/2026 at 7:53:35 AM

In the demo bro shows how to search for "a car with a bike rack on the back that cut me off at night." Given the grudge he must've held from being cut off, I strongly suspect that finding this specific car was his main motivation for building the project in the first place

by novoreorx

3/25/2026 at 8:32:08 AM

Can I give it a photo of a person and ask it to search for the person in the video?

by subhashp

3/24/2026 at 6:13:57 PM

I work in content/video intelligence. Gemini is great for this type of use case out of the box.

by danbrooks

3/24/2026 at 5:49:31 PM

Very impressive! A webhook could be configured to trigger an alarm if a semantic match to any category of activities is detected, and then you basically have a virtual security guard and private investigator. Well played.

by simonreiff

3/24/2026 at 6:38:00 PM

Thanks! Yeah that would be pretty cool, but continuous indexing would be pretty expensive now, because the model's in public preview and there are no local alternatives afaik.

This very well might be a reality in a couple years though!

by sohamrj

3/25/2026 at 4:47:37 AM

This seems like something that would be very expensive to run. Do you have some representative figures at a particular resolution and frame rate?

by febed

3/24/2026 at 8:43:51 PM

I wonder if the underlying improvements in visual language learning will allow for even more efficient search. The First Fully General Computer Action Model -> https://si.inc/posts/fdm1/

by bobafett-9902

3/24/2026 at 9:51:11 PM

I don't quite understand the 5 second overlap. I assume it's so that events that occur over the chunk boundary don't get missed, but is there any examples or benchmarking to examine how useful this is?

by WatchDog

3/24/2026 at 10:32:54 PM

yea, it's so events on a chunk boundary still get captured in at least one chunk. i haven't had the chance to do formal benchmarks on overlap vs. no-overlap yet. the 5s default is a pragmatic choice, long enough to catch most events that would otherwise be split, short enough to not add much cost (120 chunks/hr to ~138). also it's configurable via the --overlap flag.

by sohamrj

3/25/2026 at 1:44:08 AM

Is there a decent open video embedding model out there? I’d love to play with this without uploading video.

by rao-v

3/24/2026 at 5:38:02 PM

What a brilliant idea! is this all done locally? That's incredible.

by nullbyte

3/24/2026 at 5:44:08 PM

While the vector store is local, it is sending the data to Gemini's API for embedding. (Which if using a paid API key is probably fine for most use cases, no long term retention/training etc.)

by apwheele

3/24/2026 at 9:03:56 PM

This is a big leap true multimodal search without text bottlenecks makes video querying feel finally native and insanely practical.

by QubridAI

3/24/2026 at 11:15:37 PM

This is great, thanks for sharing

by cat-turner

3/25/2026 at 2:09:42 AM

Total aside here but is that you driving the pickup I assume?

by sans_souse

3/25/2026 at 2:12:35 AM

haha no i'm driving the tesla and that clip is from the left repeater camera (teslas record from all around the car)

by sohamrj

3/24/2026 at 4:07:24 PM

That's quite interesting, well done! I haven't thought of this use case for embeddings. It open the door to quite many potential applications!

by ygouzerh

3/24/2026 at 4:34:39 PM

Man, the surveillance applications for this are staggering.

by stavros

3/25/2026 at 2:05:06 AM

I wonder how well this would work with dance videos.

by crashabr

3/24/2026 at 5:18:36 PM

Does anyone know of an open weights models that can embed video? Would love to experiment locally with this.

by kamranjon

3/24/2026 at 5:37:35 PM

Not aware of any that do native video-to-vector embedding the way Gemini Embedding 2 does. There are CLIP-based models (like VideoCLIP) that embed frames individually, but they don't process temporal video. you'd need to average frame embeddings which loses a lot.

Would love to see open-weight models with this capability since it would eliminate the API cost and the privacy concern of uploading footage.

by sohamrj

3/24/2026 at 7:53:20 PM

What is your experience so far with the quality of the retrieved pieces?

by totisjosema

3/24/2026 at 7:57:20 PM

I've found I have to be very specific to get the clip I'm searching for. For example, "car cuts me off" just returned a clip of a car driving past my blindspot. But, "car with bike rack on back cuts me off at night" gave me exactly the clip I was looking for.

by sohamrj

3/24/2026 at 4:11:48 PM

Nice use of native video embedding. How do you handle cases where Gemini's response confidence is low? Do you have a fallback or threshold?

by dev_tools_lab

3/24/2026 at 4:37:37 PM

as of now, no threshold but that is planned in the future.

for example, for now if i search "cybertruck" in my indexed dashcam footage, i don't have any cybertrucks in my footage, so it'll return a clip of the next best match which is a big truck, but not a cybertruck

by sohamrj

3/25/2026 at 9:37:42 AM

Makes sense for now. Thresholding becomes critical at scale though — good luck with the next iteration!

by dev_tools_lab

3/24/2026 at 4:35:32 PM

very cool, anybody have apparent use cases for this?

by Aeroi

3/24/2026 at 11:03:38 PM

Indexing all your porn and skipping all the filler.

by mannyv

3/25/2026 at 12:12:27 PM

isn't the "fill her" the point of porn?

by iso1631

3/24/2026 at 4:48:48 PM

dashcam and home security footage are the 2 main ones i can think of.

a bit expensive right now so it's not as practical at scale. but once the embedding model comes out of public preview, and we hopefully get a local equivalent, this will be a lot more practical.

by sohamrj

3/24/2026 at 5:16:34 PM

I think a good use case would be searching for certain products or videos across social media (TikTok and Instagram). especially useful for shopping, maybe

by giozaarour

3/24/2026 at 5:28:53 PM

Branding/marketing monitoring companies would be all over this.

by vidarh

3/24/2026 at 5:00:47 PM

State surveillance

by hebelehubele

3/24/2026 at 5:28:14 PM

Worker surveillance

by wahnfrieden

3/25/2026 at 4:09:03 AM

Trail and game cams come to mind. "Create a montage of all deer encounters," "Find first appearance of black bear this year," that sort of thing.

by CamperBob2

3/24/2026 at 10:02:06 PM

Why just the dash cam?

by thegabriele

3/24/2026 at 10:25:48 PM

dashcam is just one of the use cases and the one i tested on. but this could theoretically work with any kind of video footage like home security footage

by sohamrj

3/24/2026 at 5:25:40 PM

> No transcription, no frame captioning, no intermediate text.

If there is text on the video (like a caption or wtv), will the embedding capture that? Never thought about this before.

If the video has audio, does the embedding capture that too?

by SpaceManNabs

3/24/2026 at 5:33:41 PM

Yes to both. The embedding is over raw video frames, so anything visible (text, signs, captions) gets captured in the vector. And Gemini Embedding 2 extracts the audio track and embeds it alongside the visual frames. So a query like 'someone yelling' would theoretically match on audio. My dashcam footage doesn't have audio though, so I haven't tested that side yet.

by sohamrj

3/24/2026 at 5:00:00 PM

Today I learned that Gemini can now natively embed video..

Cool Project, thanks for sharing!

by 7777777phil

3/24/2026 at 4:27:21 PM

why not skip the text conversion? is it usable at all?

by klntsky

3/24/2026 at 4:32:28 PM

gemini embedding 2 converts straight video to vectors. in this case, dashcam clips don't have audio to transcribe and even if they did, it would be useless in the search

by sohamrj

3/24/2026 at 4:57:32 PM

What are the SoA audio models right now?

by password4321

3/25/2026 at 7:10:32 AM

[dead]

by hikaru_ai

3/25/2026 at 1:18:39 AM

[dead]

by rkaliupin

3/24/2026 at 8:08:21 PM

[dead]

by matzalazar