alt.hn

6/5/2026 at 2:54:37 PM

Playing with Vision Embeddings

 https://prestonbjensen.com/posts/playing-with-vision-embeddings

by prestoj

6/8/2026 at 9:06:25 AM

Beautiful illustrations I find, 'Playing' is just the free and motivated version of 'exploration'.

One thought on your nicely illustrated "key observation [is] that neural networks tend to place features along directions": my guess is that the neural net was TOLD to behave that way by choosing e.g. Cosine Loss?

by markusMB

6/8/2026 at 3:34:50 PM

Fantastic article.

For readers interested in this, let me point to the somewhat similar "Activation Atlas" interactive paper, published in the sadly now-defunct scientific journal Distill.

https://distill.pub/2019/activation-atlas/

by jacomyma

6/8/2026 at 4:08:53 PM

Great article, including the interactive parts that were just simple enough to illustrate the point at high level.

I never got that far, but I once built a little page that just visualized embedding vectors generated from a few hundred cat, dog and plane photos as heatmaps. I used this for demonstrating to database & storage folks what embedding vectors physically are, at low level. The point of the heatmap (of different vectors) was to show that there are visually observable "vertical bands" standing out when plotting out many embeddings of the same types of objects (like different cats with different backgrounds) in a single heatmap.

I then also took a single cat photo, rotated it by one more degree 360 times and created a heatmap of these vectors to illustrate the point of what the embedding models really detect (you have to uncheck the "Normalized" checkbox in the "same cat rotated by 360 degrees" page to see the vertical bands show up).

The web-app is here:

https://tanelpoder.com/catvector/

by tanelpoder

6/8/2026 at 11:15:24 AM

Nice article! The generated images make me so nostalgic for the early days of AI image generation. DeepDream and others had such uncanny, interesting generations.

by archermarks

6/8/2026 at 1:58:54 PM

Yeah, generative AI used to be wild, alien creativity and not something that made art kids furious.

I wonder if models can be trained for "high-temperature" purposes. I'd rather have a model which can surprise me than one which can predicably produce generic mediocre results. I mean you can run them on high temperature of course, but it doesn't seem like it's optimized for that.

by vintermann

6/8/2026 at 9:56:33 AM

For some reason, the uncanniness of the feature pictures are deeply unsettling for me. It just stirs intense unease. A bit amusing, to be honest.

by RealityVoid

6/8/2026 at 5:50:45 PM

I was asking nano banana to modify pictures of me to reflect if I dressed or wore my hair differently.

It ended up inserting alien hands touching my hair with smooth pale skin holding weird wormy things.

Every once in awhile you get a peek of the Eldritch horror lurking below our helpful assistants

by piyh

6/8/2026 at 7:21:58 AM

Very nice visualizations, thanks for that!

One thing I still struggle with in my head is how these vision embeddings can then be used to give LLMs eyes.

Because you somehow need a giant training set which describes images in natural language, no? Is that actually how it works, or is there some smart trick so you don't need to pay labellers a bunch of money to look at pictures and describe them.

by jcattle

6/8/2026 at 8:08:27 AM

> Because you somehow need a giant training set which describes images in natural language, no?

That's definitely one way - they train a text encoder together with an image encoder on a labelled set of images. WL & 3b1b made a nice video on it: https://www.youtube.com/watch?v=iv-5mZ_9CPY

by dilyevsky

6/8/2026 at 8:40:53 AM

Thanks I'll check out that video

by jcattle

6/8/2026 at 1:24:24 PM

This article is very well structured and provides just the right amount of details for non-practitioners to enjoy it.

Mechanistic interpretability is a fun topic to "play with" (good title there). I recommend watching videos featuring Neel Nanda or Chris Olah

by joaquincabezas

6/8/2026 at 8:43:23 AM

This reminds me of GANs and generating directly from a latent vector, as well as exploring them. I wonder how many of the original tricks used with GANs (like extrapolation stuff, optimizing for the latent vector to get a result etc) are easy to replicate with it.

by SkitterKherpi

6/8/2026 at 1:04:46 PM

Awesome project! Preserving and sharing knowledge like this is incredibly valuable. Thanks for making these resources accessible to everyone.

by agentbraker

6/8/2026 at 10:56:15 AM

[flagged]

by cdogukank