8/19/2026 at 2:34:35 PM
This isn't just theory either, for example: Revolut is a bank that does all its event persistence and streaming on top of postgres. No traditional message queues/brokers in their stack.https://medium.com/revolut/recording-more-events-but-where-w...
by HighlandSpring
8/19/2026 at 4:19:57 PM
If you start here, with the "Postgres will take you wherever you need to go" meme, without thinking extremely deeply about your schema and how you expect to evolve it in the future, you can easily paint yourself into a very difficult and expensive corner.It's easy to use Postgres poorly in ways that result in painful centralized bottlenecks.
(Obviously this is largely true for anything, but I think that in 2026, where there's also a lot of more-specialized/less-fleible but much-easier-to-scale well-supported mature alternatives, you should be VERY wary of making everything have a single central SPOF. What are your users going to expect in terms of maintenance windows, etc.)
I'd be cautious with articles that say things like "All cloud providers allow you to run (and scale!) PostgreSQL by clicking a single button." with no mention of how long that will take and what options should be set to make it faster, or the costs of those things.
by majormajor
8/20/2026 at 12:53:14 AM
A counter anecdata. We transitioned from a postgres job queue to Rabbit. We had never ending problems after that, many of them were misunderstandings, some where just wrong-fit. We migrated because we had some time on our hands and thought we would alleviate some high pressure jobs. Not only did it not solve the problem, but having written all the code that decides when to pull the next message and what to do with it, and how to dead-letter it - just worked great for us on Postgres. It was so easy to understand and doing things like reprocessing just using a standard postgres DB interface was much easier.Ultimate the entire processing got removed from our team and no longer needs to do these deployments (acquisition transitions)...
by throwitaway222
8/20/2026 at 1:56:00 AM
I tried very hard to use postgres as a queue, it was robust but slow once I started to push from more than a few processes/servers. Moving to zeromq initially and sqs after solved all my perf issues, and was still solid.by dorfsmay
8/20/2026 at 6:39:34 AM
I think that’s the way to go. Start with Postgres and only if there are problems, then think about something more specialized. Same for microservices. Start simple and introduce a service when really needed.I hate it when people already start out with 10 or more different systems/services for a few messages per second.
by vjvjvjvjghv
8/20/2026 at 9:56:27 AM
it's not the way to go if you hit the limits very quickly and have to waste immense time migrating.by cheikhcheikh
8/20/2026 at 12:04:15 PM
From my experience the limits are way higher than people think.by vjvjvjvjghv
8/20/2026 at 10:16:39 AM
I think that using the right tool for the job is important and saves a lot of time in the long run. There are expensive headaches that we have to resolve.by egwor
8/21/2026 at 4:30:10 AM
Postgres is the right tool for most practical data storage jobs.by inigyou
8/20/2026 at 12:05:03 PM
I never know what to think about the phrase “ right tool for the job”. It almost feels like a platitude at this point.by vjvjvjvjghv
8/20/2026 at 11:55:35 AM
What were your experiences with ZeroMQ?by KronisLV
8/20/2026 at 5:40:48 PM
It worked fine, we had no issue with it. We moved to SQS because it was one less thing we needed to spend time on.by dorfsmay
8/20/2026 at 9:54:24 AM
Honestly Rabbit sucks more than it shinesAlso it is very "unconventional". Everything has to be done in its weird and quirky way
by raverbashing
8/19/2026 at 7:23:33 PM
Having done that, e.g. used rabbitmq plus postgres, honestly I wish I had just used postgresql for both messages and data. It would have been easier to manage by an order of magnitude, especially at scale and needing to satisfy enterprise requirements. Also the flexibility of postgres would have solved problems that we ran into because of limitations of rabbitmq.by tensor
8/19/2026 at 8:02:34 PM
Without knowing any specifics of your uses, my usual starting point on that sort of design is that "messages AND data" is it's own special little way of ending up with a hard-to-debug-and-operate system. ;)It's very hard to best-of-both worlds event-driven system + RDBMS-storage, it's very easy to end up with worst-of-both-worlds. Hello distributed transactions!
Again, you just should think about all the ways you want to use it and the maintenance/uptime requirements your users are going to have in advance.
by majormajor
8/19/2026 at 8:48:05 PM
I think messages + database are extremely common in any sort of large application where you have data processing nodes. Postgresql actually has very good mechanisms to support message style communication, and as long as you design your message tables independently you shouldn't have horrid issues around locking and transactions. Message queues don't save you from thinking about that anyways, they just replace transactions with acknowledgements.Trying to manage a highly available and durable rabbitmq or other message system that can also be recovered from backup to an offsite mirror infrastructure in the worst case is actually incredibly difficult. Usually these systems are designed with the assumption that you can just regenerate messages based on database state anyways in worst case scenarios.
In this use case your database already is highly available and can recover on an offsite backup if you have suitable wall shipping going on. So you've done all the hard work once, may as well reuse it unless you truly have some mind bogglingly large message throughput needs.
Finally, we had a need of a queue that was more than just first in first out. We wanted to fairly balance workloads across users and tenants. Whenever you have such a need postgresql lets you design this type of queue far easier than trying to do some elaborate multi-queue setup with a traditional queue.
by tensor
8/19/2026 at 7:33:22 PM
And now there’s pgmqby cpursley
8/19/2026 at 5:20:43 PM
I dunno. I think the main takeaway here is that you can do 80-95% of your stuff in Postgres and eschew all the unnecessary, unproven stores.by pbreit
8/20/2026 at 2:00:25 AM
It’s also entirely possible that nothing you do in the eventual history of your company hits a scale where this matters.by jghn
8/20/2026 at 2:34:18 AM
Sometimes to scale is "continue to satisfy SLAs as service usage increases" and other times to scale is "successfully evolve functional capabilities over time."While the GP may have been referencing the former, embracing "PostgreSQL for Everything" often prohibits the latter.
by AdieuToLogic
8/20/2026 at 11:55:21 AM
It *can* do that. I wouldn't say it's necessarily "often". Again, it's hard to predict the future.I think that the "use Postgres for everything" messaging was a necessity, even if it is overstated. Use it until you can demonstrate it doesn't meet your near term needs. When that happens, shift. It wasn't that many years ago when I'd enter situations where people were knee deep in FAANG level infrastructure when postgres on a relatively small instance would have more than been sufficient. I'd suggest they look at converting to postgres to save money & all the energy they spend maintaining their soup. "It won't scale the way we need it!". Sometimes they were demonstrably wrong. Other times they were half-right, in that the real problems was terrible decisions made at the software layer, leading to a situation that required heavier duty infra. Almost never were they actually right* though.
Might they have been right 5 years later? Perhaps. But I know for a fact that none of the ones I encountered were.
by jghn
8/21/2026 at 1:07:28 AM
> It can do that.As the old saying goes; just because you can do something doesn't mean you should do it.
> I think that the "use Postgres for everything" messaging was a necessity, even if it is overstated. Use it until you can demonstrate it doesn't meet your near term needs. When that happens, shift.
The problem with this approach is, once "use PostgreSQL for everything" is identified as being no longer be feasible, it has already become an inextricable component underpinning system functionality. Thus making "[w]hen that happens, shift" extremely difficult.
Contrast the above with only using PostgreSQL (or any other RDBMS) to manage data and their relationships, eschewing stored procedures as well, and the "when <insert condition> happens, shift" decision becomes much more feasible to entertain.
by AdieuToLogic
8/21/2026 at 1:42:28 PM
Eh, the "it's impossible to change later!" line is the classic one for any of these types of conversations. My experience over the decades has been that while yes, this can happen, it's grossly outweighed by YAGNI-in-retrospect and the eventual shift turning out to not be nearly as painful as people think once all of the people & bureaucratic problems are out of the way.by jghn
8/20/2026 at 4:32:21 AM
I once needed to maintain an application written in everything Oracle. If I ever encounter the original author of that product: I have things to say to him.We quickly replaced part by part by easier, less costly parts.
Software development is not just writing code; I think all HN users know that.
by ivolimmen
8/20/2026 at 3:35:49 AM
> in painful centralized bottlenecks.I find the opposite to be true. I cut out the decentralization and get it all one one machine, and the bugs go away and the perf improves.
by groundzeros2015
8/19/2026 at 4:35:17 PM
It doesn't take very long (because compute and storage are separate in most of them) but good lord does it get expensive. Every time you click that upgrade button you are doubling your cost. It's really painful when you have a spiky workload that is performing fine like 95% of the time but you are watching the p99 and need to double the cost of a very expensive infra component, only to improve the experience of the heaviest 4% of your workload. This is to say nothing of the gambit you then have to play with reservations/prepays.by encoderer
8/19/2026 at 8:05:05 PM
I haven't seen a way to get guarantees of upscaling operations under like 30 seconds (with Multi-AZ RDS) with well-supported RDS stuff (leaving out active-active setups with logical replication because that's a whole other can of worms).If you know you're gonna be ok with that for a long time, go nuts. I'm just saying: think about it in advance!
The cost pain for spikes is also a thing - some of Aurora's billing models look potentially promising but I haven't used them in practice - though it's also somethings that's harder to avoid with alternatives. Distributed DBs aren't generally super friendly to dynamic scaling IME.
by majormajor
8/20/2026 at 3:16:00 AM
> If you start here, with the "Postgres will take you wherever you need to go" meme, without thinking extremely deeply about your schema and how you expect to evolve it in the future, you can easily paint yourself into a very difficult and expensive corner.Yeah, backwards compatibility is not a thing for Java, Rust, C++, etc. :eye-roll:
Meanwhile in SQL if you need to make a backwards-incompatible change to your schema you can always use VIEWs and INSTEAD OF triggers to maintain backwards compatibility for code you've not fixed yet.
by cryptonector
8/19/2026 at 3:49:15 PM
As SRE dealing with this at current company, a benefit of using well known software like Kafka is a lot of problems you will run into have solutions/guidance already available vs you having to explore solutions which a lot of time end with “Kafka could easily do this. “by stackskipton
8/19/2026 at 4:01:59 PM
100% except when Kafka goes wrong, who maintains it?by cyh555
8/19/2026 at 4:49:47 PM
There are two sizes of companies: those that can afford '1+ dedicated ____-person' and those that can't. Which should filter through to technology choices more than it does.by ethbr1
8/20/2026 at 12:26:13 PM
Or, you can do like one of my former bosses, and just rattle off a list of 60+ major projects which would require a team of 10 to make any reasonable progress on in the near future, then pin it on one underqualified person, refuse to provide a proper budget, and continually press them about why targeted deadlines are being missed.by slfnflctd
8/19/2026 at 7:29:26 PM
Often you start as the latter and grow toward the former.That transition can be super super painful as you don't quite have enough work for the dedicated person.
by mooreds
8/20/2026 at 11:24:43 AM
It's not 1+ person when a system needs to operate 24/7. To have a proper on-call rotation you need 5 to 8 people.by erispoe
8/20/2026 at 2:48:55 PM
One specialist and a group of generalists is often enough. Especially if you are allowed to contact the specialist outside normal working hours in rare emergencies.by CodesInChaos
8/19/2026 at 6:51:17 PM
This problem doesn't go away with postgres. It's totally anecdotal but this is one thing that I've noticed different in mysql shops and postgres shops - with mysql there is usually at least one person on staff who knows MySQL DBA and scaling pretty well, with postgres it's rarely the case to have someone who knows the internals well - like you said, the person capable of maintaining it when it goes wrong.You could argue it's because postgres requires less poking though I would say you don't need the DBA for when things go right.
Of course most people are just handing the management off to the cloud and that's potentially why, but it doesn't cover everything
by dwedge
8/19/2026 at 10:41:34 PM
MySQL will generally run fairly well with default tuning, assuming you've sized the buffer pool well relative to the amount of RAM you have (cloud providers do this automatically, but it's also not that hard to calculate). There are some knobs you can turn to eke out more performance in certain situations, and there are some defaults that are truly terrible (lock_wait_timeout is set to 1 year...), but all in all, it doesn't take a lot of care and feeding to run reasonably well.Postgres, on the other hand, has a million knobs, many of them interact, you'll find conflicting advice for some of them, and it can rapidly fall over if you aren't keeping a close eye on long-running transactions. It's also more performant than MySQL in _most_ situations (hello, clustered index), if you've tuned it correctly. It also of course has far more extensibility out of the box, with tons of index types that are extremely helpful, if you know how and when to use them.
This difference is why I'm always frustrated when people parrot "just use Postgres" as though that solves all problems. It's an extremely powerful tool that can replace most of your stack, yes, but it also would really, really like you to RTFM. Not random Medium blog posts, the canonical documentation.
by sgarland
8/20/2026 at 9:57:37 AM
> You could argue it's because postgres requires less poking thoughThis is the myth people who parrot "just use Postgres" believe. It is false, obviously
by raverbashing
8/19/2026 at 5:59:09 PM
I mean, with this custom thing, you have that question as well with downside is you cannot pick up the knowledge from off the street.I became the Kafka guy at my current company, it took me about a week of reading and every time I had further question, I didn't have to bother anyone, I could Google and get data I needed.
When it's some NIH thing, you have to bother coworkers and knowledge is whatever is in YOUR company knowledge base with no ability to get knowledge from outside the company.
EDIT: You could also leverage contractors or outside support if not homegrown software.
by stackskipton
8/19/2026 at 6:41:46 PM
Yup, and every time it breaks, you've got to pester someone whose job probably isn't maintaining that thing actively.I worked at a startup with massive NIH syndrome, once. We even used our own in-house programming language, because it was "better than anything else out there on the market." It did have a lot of nifty features that others don't have: a pretty novel type system, programmatic macros, a built-in build system and other fun bells and whistles -- but also not-so-fun ones like having no syntax highlighter, LSP, or debugger, and having to constantly shuffle around your code to avoid ICEs in the compiler.
The compiler wasn't the product, but we found ourselves fighting that thing more actively than any of the real problems our custom programming language was supposed to solve. The CTO found himself spending all his nights and weekends mostly trying to get the compiler to not explode.
A few years later, after I had long left (for that reason, among many) I heard they switched to Python. Can't imagine how long it took them to get that all rewritten.
by tyg13
8/19/2026 at 7:26:30 PM
> e even used our own in-house programming language, because it was "better than anything else out there on the market." It did have a lot of nifty features that others don't have: a pretty novel type system, programmatic macros, a built-in build system and other fun bells and whistlesA DSL can work, but not for the features you list. Those features you already get from existing languages anyway!
If you need general programming language features like excellent type system, programmatic macros, a build system (doesn't need to be built into the language), etc... then use a general purpose programming language.
I have a DSL for backend/endpoints, and exactly none of those are in my feature list. What it has are things like easy way to specify access-control directives[1], the SQL query to execute, mapping request variables to SQL parameters, mapping SQL results-sets to response fields, etc.
I have another DSL for a test program. Both of those DSLs have specs that's literally 2x screens of bullet points and examples. LLMs can output those DSL programs because the spec for the DSL is so small.
For general purpose programming stuff (while loops, conditionals, etc) my DSLs break out to Python.
A good indicator that you shouldn't be creating a new language for production is when you find yourself implementing conditionals, loops, etc.
===========================
[1] Limit endpoint to specific roles, or members of the same team, or both, or even to the user itself - someone calling `/user/profile/update` should only be allowed if the profile they are updating is theirs, for example.
by lelanthran
8/19/2026 at 10:12:07 PM
Ah, but you see, it was a general purpose programming language. A general purpose, functional, optionally-typed programming language, with its own optimizing compiler and build system. In fairness, it wasn't originally developed to be our in-house language, but it was the creation of the CTO.It was fun while it lasted and I had a lot of fun working on it. But it was really not a good business fit. The programmatic macro system was supposed to allow us to build customer-facing DSLs on top of it, but everybody just wanted Python anyways.
by tyg13
8/19/2026 at 10:24:37 PM
> A general purpose, functional, optionally-typed programming language, with its own optimizing compiler and build system.> The programmatic macro system was supposed to allow us to build customer-facing DSLs on top of it,
Honestly, it sounds a lot like Lisp.
As a former Lisper, I don't doubt that it was a bundle of fun :-)
by lelanthran
8/20/2026 at 9:56:33 AM
I love the naming Kafka. Either they knew what it stands for or they didn't. And the latter is the worse option.by raverbashing
8/19/2026 at 4:46:40 PM
Nose goesby striking
8/20/2026 at 6:19:59 AM
During pgConf.eu in 2016(-ish, could have been one or two years later; I don't remember too well), a representative of payment processor Adyen told the audience that they were, essentially, one big postgres cluster in their backend, too ("cluster" used as per the postgres-native meaning of the term, as in, an installation on a single host with a data directory containing any number of databases).by c0l0
8/19/2026 at 4:19:25 PM
they likely have something on top of PG to distribute data across shards, which is still untrivial task I think and require ops overhead.by andriy_koval
8/19/2026 at 4:44:43 PM
starling bank uk uses a similar kind of stack. both java based as revolut.by dzonga