alt.hn

3/30/2026 at 3:58:11 PM

An NSFW filter for Marginalia search

 https://www.marginalia.nu/log/a_134_nsfw/

by speckx

3/30/2026 at 6:13:50 PM

This was a very meandering project, and trying to corral it into some sort of coherent narrative was a bit of an undertaking on its own. Hopefully it makes some sense.

by marginalia_nu

3/30/2026 at 6:41:43 PM

Hi Viktor! Really cool write-up, thanks! Uruky is already using the `nsfw` param, but set to `0` or `1`, and I see in your example this looks like a new value option (`2`) that's "better" than `1`? How "safe" is it to implement it as the value to send when someone wants SFW results?

by BrunoBernardino

3/30/2026 at 6:52:40 PM

0 disables all filtering

1 filters 'harmful' sites per the UT1 blacklists

2 is 1 + the new NSFW filter.

The new filter works pretty good in my assessment. It's not infallible, but it gives significantly cleaner results.

And if you do find queries it fails to sanitize, I'd love to hear about them.

by marginalia_nu

3/30/2026 at 7:58:13 PM

Can you add 3, which only returns content flagged as NSFW?

So I can make sure I know what sites to stay away from, of course

by IncreasePosts

3/30/2026 at 8:24:46 PM

Wouldn't work very well, in that you'd get awful recall.

The way the filter is implemented, it runs after the query has been executed. I'd have to run it at document processing time, code in a pseudo-keyword for the label, and then add that to the query.

It's doable, but I question whether the juice is worth the squeeze.

by marginalia_nu

3/31/2026 at 1:41:11 AM

Asking the real questions.

by cosmicgadget

3/30/2026 at 8:22:15 PM

Or perhaps -2

by VectorLock

3/30/2026 at 7:22:24 PM

Thanks, already implemented and tested a couple of queries and it does look good!

by BrunoBernardino

3/30/2026 at 8:44:10 PM

Looks like a cool search engine! Hadn't heard about it before.

But the search page says "Simple technology, no AI". With this change, that is no longer true though, is it? Of course the definition of "AI" is extremely vague. Once upon a time, A-star search was considered AI after all.

by VorpalWay

3/30/2026 at 9:51:39 PM

The 'No AI' statement is about gen AI, which is I think what most people think of when you say AI.

But sure if I was looking for government or research funding, then for sure this would be AI. Not just AI, but the literal state of the art AI. Dario wakes in a flop sweat every night, terrified of my breakneck advances in single hidden layer classifiers that are probably at least 30% sentient. It would be so much AI I can't even hold all this AI.

by marginalia_nu

3/30/2026 at 11:16:12 PM

I think sentient autonomous cybernetic organisms. Still haven't encountered any yet.

by cykros

3/30/2026 at 7:55:18 PM

Does marginalia_nu not use embedding models as part of search? I guess I assumed it would. If you have embeddings anyway, decision trees on the embedding vector (e.g. catboost) tend to work pretty well. Fine-tuning modernbert works even better but probably won't meet the criteria of "really fast and run well on CPUs". That said, the approach described in the article seems to work well enough and obviously provides extremely cheap inference

by ChadNauseam

3/30/2026 at 8:08:08 PM

It does not use any transformer models right now. I've made experiments with BERT-adjacent methods, but not found them fast enough to be useful. Basically, whatever approach is used, it needs to do inference at ~10us latencies to either make real-time result filtering viable, or <1ms not add unreasonable overhead to processing-time result labeling.

by marginalia_nu

3/30/2026 at 6:52:22 PM

Have you seen many examples of websites labeling themselves, perhaps using rating meta tags (<meta name="rating" ...>)? Self-labeling seems valuable in some ways, but I don't think I've seen it catch on.

by 8organicbits

3/30/2026 at 6:54:55 PM

Meta tags are almost universally garbage, but the presence of '18 USC 2257' (or U.S.C.) is a very strong NSFW signal.

by marginalia_nu

3/30/2026 at 7:12:21 PM

Does this comment make this page NSFW on Marginalia?

by Wingy

3/30/2026 at 7:16:32 PM

[dead]

by GenericDev