alt.hn

Platform activity is surging. There were 1 billion commits in 2025. As of three months ago, it was 275 million per week, on pace for 14 billion this year if growth remains linear (spoiler: it won't.)

20% of all GitHub accounts were created in the past 6 months https://x.com/kdaigle/status/2082604368399159542

by tristanj

Eventually, though, they decided to migrate the whole thing to Azure. And they were far enough through that to be basically committed... when AI coding started hitting them with much higher workloads.

I personally think the reliability problems are more to do with the reliability of the Azure migration. But both factors are likely relevant.

by andyjohnson0

https://x.com/mariorod1/status/2085800861469495465

I really think something deeper is going wrong there, and they're not being honest with their paying customers (and enterprises) about it.

by qrush

HN Search: azure capacity - https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...

Azure Capacity Crunch Extends into 2026 Amid Data Center Constraints - https://windowsforum.com/threads/azure-capacity-crunch-exten... - October 9th, 2025

https://news.ycombinator.com/item?id=47427779

by toomuchtodo

- GitHub attempting (and seemingly failing) to move to Azure infrastucture for its website backend

- AI generated code wrecking the site due to the volume of activities.

by VCFundedGenYer

A whole lot of AI agents are presumably using Github as part of a workflow.

Would be story worthy if one of them mis interpreted their instructions and is causing havoc.

Like "Really make sure changes are saved to git" -> agent "I have to hack into github backend to really make sure the changes are saved to disk"

by pianopatrick

That was about 2 years ago. Im not kidding.

I dont know what the moral of the story is, but I found it weird at the time ( for added context - i was using AI back then about 10 hours a day, BUT I think sentiment on HN was "still" around the vibe of "you use AI to code without checking every line? I doubt your projects work" - but I had nothing better to do then wrestle with it and was surprised how I hadnt checked my code in weeks but stuff "worked". Its much better now and agents are accepted ofcourse but it sort of "snuck up" on people even in the tech community as recently as that.

I guess what Im saying is time is going quickly.

by ionwake

Couldn't they charge just a teeny tiny amount per PR or diff size or something and this would basically go away? I don't actually know what is the heaviest load. Probably actions? I understand it would be nice if it was free, but they have to pay for their infrastructure.

Or maybe pay extra for priority. Does everyone pushing llm generated code they'll rewrite or abandon in a week need all of the GitHub processes executed instantly? I'm probably way out of my depth here on what the actual bottlenecks are.

Could they break out reliability promises and reserve capacity for just the basics?

by chermi

No employee with professional ethics would tell you that (the real internal situation).

You can observe and analyze the reasons from some things.

Use it and tolerate the bad things, or on some time point go to other place (use other tool).

by lilerjee

- Addressing the VS Code retry behavior that amplified Copilot token traffic.

So basically bad code pushes that caused request amplification and then huge gaps in operational scaling and reliability standards. Oof.

by llama052

Again, I am patiently waiting for the AI bubble to burst.

by austin-cheney

Does the AI ever look back at the shit trail it left behind?

Diffs are no longer diffs, they look like largescale delete and rewrite

by rep_movsd

Pick one!

by pluc

If it sounds like I'm defending GitHub a bit, yeah, I am. I can't think of a comparable situation that any other web site has been through. This isn't "oops, we didn't plan for the Black Friday sale", this is a once-in-a-generation event focused on one, important web property that no one would have handled perfectly. (Even us all-knowing commenters here on HN.)

You know that old trope in every submarine movie, where the captain tells the driver to dive lower than they've ever gone, and someone says "I don't know if the boat can take it!" and then they switch the camera to some engineering room where the boat is groaning under the pressure and a bolt comes loose and water starts spraying everywhere but someone runs up with a giant wrench and tightens it and looks around at everyone else in the room like, "whoa, that was close..."?

Or Scotty and Star Trek, "It canna take much more, Captain!"

That's what GitHub has been going through. It's had problems, but it's still here, and usually working well.

- Let me start with: GitHub is ~3,000 talented and really nice people trying to do the right thing, with a great culture. That's the most important thing I want to communicate. While I'm sure no one at GitHub is happy about their uptime, believe me when I tell you that GitHub Engineering is exceptional. They have great leadership, depth at all levels (Distinguished / Principal / Staff / Senior / earlier-in-career) and if you ever get a chance to hire someone from GitHub, you should. I hope all of you get to work with engineers as good as GitHub's.

- GitHub has been improving all aspects of its infrastructure steadily for many years now. The GitHub Engineering blog https://github.blog/engineering/ has been documenting this the entire time. Go point your favorite LLM at it and ask for a summary of all of the major system improvements since 2020. If that work hadn't already been done, GitHub would be a smoking pile of servers at this point. There's a lot I could list, I'm not sure which are already public, but improvements on the order of using thousands fewer CPU's to serve even more traffic than before have been made, and still are, I'm sure.

- GitHub was already serving billions of requests/day before agentic coding hit. Their challenge wasn't scaling a fresh new system with a few users an order of magnitude; it was taking one of the busiest and most important sites on the Internet, and getting hit with 14x traffic in a year, and having to plan for 100x. If you think your systems and infrastructure would have survived that, if you think you would have been able to politically navigate and succeed in getting projects green-lit at a large company to prepare for 100x scaling before it hit, to get those resources for "we might have scaling problems in a year or two" instead of getting them to ramp up on AI coding and other features that were crucial to growth right now, you don't understand large organizational dynamics. That's not a complaint about GitHub or Microsoft; it's an observation about capitalism and how any mature management group prioritizes things in software. I'd expect everyone in the San Francisco/Silicon Valley Reality Distortion Field to understand that.

- We talk in terms of "14x commits" to Git but that's only part of the story. GitHub Actions, webhooks, github.com itself, and other parts of GitHub, have all been under pressure. It's the totality of it, the seams that have been exposed at scale that couldn't have been exposed without that scale, that have caused the instability. That's why architecture gets overhauled.

- Yeah, GitHub has had less consistent uptime since the 2018 Microsoft acquisition. The GitHub that existed before that had much less functionality, an order of magnitude fewer users, and had received very little improvement in the few years before. Microsoft invested and enabled GitHub to grow into something much bigger than it ever could have without them.

- Be grateful that Microsoft - with a 50-year history of shipping developer tools, and more experience operating enterprise software than any other company on the planet - acquired GitHub instead of Google, which was the other major player in contention. Spend a minute or two thinking about the product journey GitHub would have taken under Google, and then think about how many non-search, non-advertising products have succeeded there. Which amazing developer tools from Google do you use regularly? Yeah, I thought so. On behalf of Microsoft, you're welcome.

- Did you notice that GitHub swapped out one of its data centers last year for one 3x larger? No? Maybe that's because they executed it flawlessly, with no downtime. If you've ever done that on a massive web site with as much scrutiny as GitHub receives, you get a gold star.

- Did you notice that over 50% of GitHub read traffic is now being served from Azure instead of GitHub's own data centers, and growing, and that all GitHub Monolith traffic is scheduled to be served from Azure instead of GitHub's data centers by the end of CY26? This massive migration is taking place while traffic is going insane. From https://github.blog/news-insights/company-news/github-availa...: "GitHub can now serve a larger share of customer requests from independent Azure capacity, reducing reliance on any single datacenter while preserving performance. Monolith read traffic served from Azure Central US peaked at 52.75% on July 28—the first time we consistently remained above the halfway line. Git traffic in Azure reached 47%, up from 43% in June, and 29% of all repositories now have a second replica in Central US, making failover less disruptive when a region degrades."

- OpenAI and Anthropic obviously have significant scaling challenges as well, but there's a huge separation between the GPU-based inference part, and the CPU-based front-end systems. Their CPU-based systems are significantly simpler and newer than GitHub's, so I'm not surprised they have fewer outages, but it's still non-zero.

- Ruby on Rails is not the problem (and, for the record, I don't even like Ruby or Rails). Most of the performance-critical systems that were built on Ruby have been migrated to Go or some other language that takes full advantage of multithreading. The web endpoints still served by Ruby are fine.

- Spare me the "Azure sucks" comments. Please. Microsoft Azure is the second-largest computing system on the planet - only AWS is larger in terms of hardware but not in terms of the number of products they ship - and Microsoft's first-party systems that run on it, all at the same time, are among the largest, busiest, most important systems on the planet. Entra ID, Azure SQL, Service Bus, Event Hubs, Office 365, Cosmos DB, OneDrive, etc. all in the tens-to-hundreds-of-billions requests/day. I'm not even counting the massive customer-owned systems that run on it, including almost the entire Fortune 500. Nothing is perfect, everyone has downtime, we all always want more and better features, I want improvements from Azure, too, but, please, grow up. I have opinions, too, I've been programming since the Apple ][+, there are popular technologies that I don't like, but I know that's subjective. "Your favorite technology sucks, mine is better" is not an objective statement about anything.

Anyway, if you haven't shipped at GitHub's scale, in this dynamic of a world of feature churn, and one-of, if not the largest, traffic spikes in the history of the web, have fun saying whatever you're going to say. I've never worked at Google or Amazon, I have opinions about their product design and culture, but there's nothing I can say about their infrastructure and systems because I have no idea about them, and what we all do in this crazy world of programming is harder than it looks.

For some, this won't be the answer to "what's going on?" because I'm not pointing fingers at any one thing. To me, the real answer is: an unprecedented scaling event on what was already one of the busiest sites on the web exposed seams in their systems, including the need to move from their own data centers to a much more scalable cloud provider - that are being addressed as quickly as possible.

I'm not telling you not to look for alternatives, I'm not telling you what to do about it, I'm not saying that all of their problems will magically be solved soon, I have no idea. I'm even launching a new version control system myself very soon to compete with them. But I do know that the thought "GitHub doesn't know what they're doing" is wrong and unhelpful.

by SBArbeit