8/3/2026 at 7:17:00 AM
I run a website that's in a similar situation, and it sounds like the author is close to the correct solution, but needs to take things one step further:There needs to be a Single Point of Truth, and it should not be your little website.
BigCentralThing should be the official database that knows about where all the Whatevers are. Your little Whatever Locator site should only ever _pull_ from them. Need a new Whatever in the database? Get BigCentralThing to add it, then pull it in.
As an example, I run https://bettybeta.com/, a select guide to bouldering in Fontainebleau tailored for Women and short folk. We have a list of boulder problems in our database, but no way for anybody (even an admin) to add a new one. Every boulder problem record includes a link to its page at https://bleau.info/, who are dedicated to cataloging every climb in the forest.
Bleau.info is the Single Point of Truth. If one of our users does a First Ascent and wants to log it with us, we'll point them at that other site with instructions on how to get their climb in there, and tell them how to get us to import it. Any other path would lead to our database and the "official" database eventually diverging.
There are half a dozen other websites doing similar things to us. Some do what we do, but the ones that don't quickly find themselves out of sync and end up with a lot more work on their hands.
It's tough because, as the author says, writing the flow to import at our end is really easy (I've written it, before coming to this realization and hiding it again). But doing it is a trap.
by jasonkester
8/3/2026 at 1:25:00 PM
This can sometimes be a challenge with OSM, if the way you want to structure data doesn’t align with the way the community has decided to structure data.I was looking into building a canoeing map based on OSM data. An important thing to map is “is a particular stream between two lakes navigable by canoe?” OSM has a defined canoe=yes/no tag, but it’s supposed to be used for legal access, not navigability.
In the place I wanted to map, canoeing is basally legal everywhere, but there are many streams that no one ever canoes down, because it would be an enormous pain. There isn’t a way to represent this is OSM today, and community tends to be resistant to adding tags that map “subjective” data.
The solution would probably be for me to keep my own database of navigable streams with references to the OSM features, except that OSM features don’t really have stable ids. If another mapper comes along and decides to map an existing stream in more detail, they might choose to delete an existing stream feature and replace it, split it up into multiple features, or join several existing features together — all of which will result in id changes.
by mikeocool
8/3/2026 at 3:24:33 PM
It sounds like the real issue is the "stable ids" one. You need a Single Point of Truth that matches up stretches of waterway with an ID that won't change. You can then sprinkle .isNavigable and such onto your record along with a reference to that magic ID. But yeah, if they don't provide is, they're not a very good point of truth.I have to deal with this in my little bouldering database as well, since the source I mentioned does in fact change or reuse its IDs every so often. There's already a periodic update that has to happen on any given boulder problem, to pull down changes from the other end. That also needs a way to self-heal if a record at the other end splits off into two pieces, and my ID now points to the wrong half of it.
It's all stuff you have to keep on top of, but your idea of keeping your own table with each record pointing back to OSM seems like the only path that avoids madness. Even it it does take some work to keep those references up to date.
by jasonkester
8/3/2026 at 8:54:03 AM
I agree that that's the ideal way to do this, but it's just not always an option.I run valideparkeren.nl, a crowd sourced overview of all accessible parking spots in the Netherlands. Ideally, there should be a government-owned single source of truth, but the whole reason for starting this is because that source doesn't exist yet. I'm starting to work with the government to eventually hand this project over to them, which was the goal from the start, but the only way to make that happen is by first becoming the source of truth.
Edit: I was also planning to add my data to OSM at some point, but I'm reconsidering that after reading this. I completely understand that they only want high quality data, but in my case I'm convinced that high coverage and slightly lower quality is better than low coverage and high quality. Of course, transparency of how sure we are about the data is vital, so users can decide the risk they want to take.
by Moghammed
8/3/2026 at 1:49:56 PM
> I completely understand that they only want high quality dataI worked on "openopeningstijden", where my aim was to improve the data for businesses in OSM and to disclose that in apis and uis.
There were other reasons why I had to pull the plug, but a big difficulty was the understandable conservatism of OSM. I didn't even want any tagging schemes changed, or the insane dsl for "opening hours" improved, at that time.
All I wanted was to allow obviously "wrong" data to be flagged with a note. Where "wrong" could easily be proven with links to other resources. "The shop is permanently closed. See this url at archive org about their announcement" or "contact details wrong. See their website at url and cross check with BigFoodDeliveryPlatform entry at ..."
Again. Reluctance for such automation is understandable wrt intelectual property rights. It becomes too easy for editors to copy in data that's not allowed.
But the result is that businesses on OSM are poorly represented, often many are missing. That data is hopelessly outdated. And that consumers use proprietary sources and apps to find e.g. a vegetarian restaurant in an unknown city, or to find out if the hairdresser is open at noon.
by berkes
8/3/2026 at 9:11:58 AM
Yeah, it's tough because to be the Single Point of Truth, you'd need to be the List of Parking Places in the Netherlands, not just the accessible ones that you care about. You might even need to be The List for Europe.Being The List of anything is a big, thankless job filled with nonstop toil to keep it up to date. It's not something you want to do unless it's truly your passion or somebody is paying you (a lot) to do it.
The problem is that building to tools to Be The List is pretty straightforward and fun. You only learn about the toil you've signed yourself up after the fact, once people start relying on it.
Good luck!
by jasonkester
8/3/2026 at 12:33:55 PM
Curious. You’re curating from the single point of truth (spot). You direct new additions for your site over to spot. How does a submission get into your subset?Does the submitter add to spot and then shoot you a message to pull by some unique id (author, submitter)? If you’re curating from their database, then spot doesn’t have the schema features to resolve to your subset by filtering alone, right?
by xtiansimon
8/3/2026 at 3:36:01 PM
Pretend the SPOT is youtube, for an example. Upload your video there, then drop a link to (any variation of) the url into this text field. A quick import script runs, and now we have the video in our database too. Or at least, a record with a link to it and whatever info we need to pull from it, along with extra fields we need to do whatever our website does.It still lives in both places, but (assuming other websites also use the same Single Point Of Truth) if the title changes or whatever, everything stays synced up everywhere.
In my concrete example, bleau.info doesn't need to know that a given line has never had an ascent by anybody shorter than 165cm. So I write that down on my copy. But I don't need to track anything else about that line to use it and to link to Boolder, 27 Crags and 8a.nu's pages about it.
by jasonkester
8/3/2026 at 3:09:18 PM
> Get BigCentralThing to add itMaking a workflow central to your business dependent on the goodwill and efficiency of an external entity is a very bad idea!
It's good that you've found a partner who is willing to put in the work, but that is luck rather than what normally happens.
by ddevnyc
8/3/2026 at 11:48:59 AM
This way you get less contributions.by inigyou
8/3/2026 at 3:27:43 PM
I haven't seen this in practice. For my site, roughly 100% of the new additions to our database come from people climbing a route that's in the Big Database that we just don't know about yet.It's just a matter of dropping a link to that route's page into an importer to scrape it in and have it be usable at our end.
But yeah, for a space where the Big Database didn't often have a record for the things your users want to add, I could see it being more of an issue.
by jasonkester