alt.hn

6/9/2026 at 1:49:37 PM

US publishers tell Common Crawl to stop scraping and delete archive

 https://pressgazette.co.uk/media_law/common-crawl-ai-news-publishers-scraping-cease-and-desist-letter/

by thm

6/9/2026 at 2:54:55 PM

Hard to see any practical benefit to go after common crawl. The situation of freely accessible crawled data is bad enough as it is with archive.org and CC being pretty much the only available sources. We need more initiatives like them, not less. The scary thing is how the anti-AI sentiment is being used to lock things down further.

by Stagnant

6/9/2026 at 10:03:37 PM

Scary how AI chokes off the very goodwill-oriented societies that it needs as its lifeblood, eh

by conartist6

6/9/2026 at 3:37:26 PM

Its like they don't understand the problem common crawl solved rather neatly. You think the skid scrapers are bad? Wait till the competent players lose access to CC.

by Grimblewald

6/9/2026 at 3:50:09 PM

Let's fucking go, asshats! Destroy the internet to show how valuable your product is.

by conartist6

6/9/2026 at 3:55:28 PM

These guys (the publishers) are fighting last year's war. Nobody (to a first approximation) gives a shit about going to the NY Times website, or The Guardian website, or the BBC website, etc. to find information. They expect to use search engines and AI services to find stuff, and then maybe click through to the source site(s) for more details or whatever.

The publishers need to rethink their entire take on how the Internet works or any "victory" they earn is going to be extremely Pyrrhic.

by mindcrime

6/9/2026 at 1:58:01 PM

Crawling will go underground à la Anna’s Archive.

by toomuchtodo

6/9/2026 at 2:38:15 PM

This is shady. Copyrighters absolutely not get to control use of their copyrighted material when people mentally, sonically, or physically reproduce it for personal use.

It's absurd to say "you can't record this book to a friend or robot".

Nobody seems to actually reproduce the copyrighted materials.

High-dimensional eigendecompositions which underpin AI similarity are some of the most literally derivative materials of texts that you can imagine.

by khelavastr

6/9/2026 at 3:43:07 PM

So you record a copy "for a friend" and then you sell lots of those copies as your business. All within your rights! What's mine is yours, my Comrade!

(my point being that it would be different if the product CommonCrawl provides were trained models, but this is not the case: its product is unlawful reproductions of copyrighted data for commercial use)

by conartist6

6/9/2026 at 4:15:07 PM

> then you sell lots of those copies as your business

Common Crawl is not a business and is not selling anything.

by joshuaissac

6/9/2026 at 4:45:32 PM

It's awfully hard to claim you aren't selling anything when you're giving other people's stuff away for free and they would be selling it if you hadn't, um, "saved them the trouble"

by conartist6