3/27/2026 at 8:34:57 AM
Is there a practical way to enumerate all the registered internet domains? EG by asking DNS servers for all the domains they know about, and repeating over all DNS servers they know about?EDIT: apparently, "asking DNS servers for all the domains they know about" is not something you can really do anymore for security reasons. Guess that idea won't fly lol
by voidUpdate
3/27/2026 at 9:09:47 AM
Enumeration of the entire DNS space is not available in general, but it does appear that some TLDs offer complete zone files for legitimate research purposes, see for example https://czds.icann.org/help#zone-filesby 0x0
3/27/2026 at 7:05:35 PM
This https://www.whoisxmlapi.com/whois-database-download.php is a start, but there's new ones every second so you're going to need to update a lot.by bit1993
3/27/2026 at 6:03:21 PM
Check out the "Root Zone Database",https://www.iana.org/domains/root/db
(this doesn't have .yu)
by ks2048
3/27/2026 at 9:39:26 AM
> apparently, "asking DNS servers for all the domains they know about" is not something you can really do anymore for security reasons. Guess that idea won't fly lolThere are actually a few nameservers that will just give all their domains to anyone who asks [0], but they are very much in the minority.
[0]: https://github.com/acidvegas/mdaxfr#tlds-that-allow-axfr
by gucci-on-fleek
3/27/2026 at 8:43:38 AM
You could probably extract a lot from https://commoncrawl.org/by onion2k
3/27/2026 at 5:59:49 PM
Here's the latest (although, it looks truncated at those having > 1M pages),https://commoncrawl.github.io/cc-crawl-statistics/plots/tld/...
by ks2048
3/27/2026 at 6:32:08 PM
The complete list hides in the web graph:https://data.commoncrawl.org/projects/hyperlinkgraph/cc-main...
and the specific file that's every host we've seen in the latest 3 crawls is:
https://data.commoncrawl.org/projects/hyperlinkgraph/cc-main...
by ccgreg