2/16/2026 at 12:54:37 PM
the hiding stuff is weird because the whole reason you'd want to see what Claude is doing isn't just curiosity - it's about catching when it goes off the rails before it makes a mess. like when it starts reading through your entire codebase because it misunderstood what you asked for, or when it's about to modify files you didn't want touched. the verbose mode fix is good but honestly this should've been obvious from the start - if you're letting an AI touch your files, you want to know exactly which files. not because you don't trust the tool in theory but because you need to verify it's doing what you actually meant, not what it thinks you meant. abstractions are great until they hide the thing that's about to break your buildby the_harpia_io
2/16/2026 at 1:15:39 PM
> it's about catching when it goes off the rails before it makes a messThe latest "meta" in AI programming appears to be agent teams (or swarms or clusters or whatever) that are designed to run for long periods of time autonomously.
Through that lens, these changes make more sense. They're not designing UX for a human sitting there watching the agent work. They're designing for horizontally scaling agents that work in uninterrupted stretches where the only thing that matters is the final output, not the steps it took to get there.
That said, I agree with you in the sense that the "going off the rails" problem is very much not solved even on the latest models. It's not clear to me how we can trust a team of AI agents working autonomously to actually build the right thing.
by rco8786
2/16/2026 at 1:26:21 PM
None of those wild experiments are running on a "real", existing codebase that is more than 6 months old. The thing they don't talk about is that nobody outside these AI companies wants to vibe code with a 10 year old codebase with 2000 enterprise customers.As you as you start to work with a codebase that you care about and need to seriously maintain, you'll see what a mess these agents make.
by g947o
2/16/2026 at 5:20:10 PM
Even on codebases within the half-year age group, these LLMs often do perform nasty (read: ungodly verbose) implementations that become a maintainability nightmare. Even for the LLMs that wrote it all in the first place. I know this because we've had a steady trickle of clients and prospects expressing "challenges around maintainability and scalability" as they move toward "production readiness". Of course, asking if we can implement "better performing coding agents". As if improved harnessing or similar guardrails can solve what is in my view, a deeper problem.The practical and opportunistic response is too tell them "Tough cookies" and watch the problems steadily compound into more lucrative revenue opportunities for us. I really have no remorse for these people. Because half of them were explicitly warned against this approach upfront but were psychologically incapable of adjusting expectations or delaying LLM deployment until the technology proved itself. If you've ever had your professional opinion dismissed by the same people regarding you as the SME, you understand my pain.
I suppose I'm just venting now. While we are now extracting money from the dumbassery, the client entitlement and management of their emotions that often comes with putting out these fires never makes for a good time.
by GoatInGrey
2/16/2026 at 8:12:37 PM
This is exactly why enforcement needs to be architectural. The "challenges around maintainability and scalability" your clients hit exist because their AI workflows had zero structural constraints. The output quality problem isn't the model, it's the lack of workflow infrastructure around it.by buschleague
2/16/2026 at 8:44:46 PM
Is this not just “build a better prompt” in more words?At what point do we realize that the best way to prompt is with formal language? I.e. a programming language?
by datsci_est_2015
2/16/2026 at 9:01:23 PM
No, the suite of linters, test suite and documentation in your codebase cannot be equated to “a better prompt” except in the sense that all feedback of any kind is part of what the model uses to make decisions about how to act.by semiquaver
2/16/2026 at 9:25:34 PM
A properly set up and maintained codebase is the core duty of a software engineer. Sounds like the great-grandparent comment’s client needed a software engineer.by datsci_est_2015
2/16/2026 at 11:34:18 PM
What if LLMs, at the end of the day are machines, so for now generally dumber than humans and the best they can provide are at most statistically median implementantions (and if 80% of code out there is crap, the median will be low)?Now that's a scary thought that basically goes against "1 trillion dollars can't be wrong".
Now, LLMs are probably great range extenders, but they're not wonder weapons.
by oblio
2/17/2026 at 12:56:47 AM
Also who is to say what is actually crap? Writing great code is completely dependent on context. An AI could exclusively be trained on the most beautiful and clean code in the world, yet if it chooses the wrong paradigm in the wrong context, it doesn't matter how beautiful that code is - it's still gonna be totally broken code.by lossyalgo
2/16/2026 at 10:10:05 PM
[dead]by BOOMp0wSm4sh
2/16/2026 at 2:05:21 PM
I maintain serious code bases and I use LLM agents (and agent teams) plenty -- I just happen to review the code they write, I demand they write the code in a reviewable way, and use them mostly for menial tasks that are otherwise unpleasant timesinks I have to do myself. There are many people like me, that just quietly use these tools to automate the boring chores of dealing with mature production code bases. We are quiet because this is boring day-to-day work.E.g. I use these tools to clean up or reorganize old tests (with coverage and diff viewers checking of things I might miss), update documentation with cross links (with documentation linters checking for errors I miss), convert tests into benchmarks running as part of CI, make log file visualizers, and many more.
These tools are amazing for dealing with the long tail of boring issues that you never get to, and when used in this fashion they actually abruptly increase the quality of the codebase.
by krastanov
2/16/2026 at 3:23:39 PM
It's not called vibe coding then.by g947o
2/16/2026 at 4:17:01 PM
Oh you made vibe coding work? Well then it's not vibe coding.But any time someone mentions using AI without proof of success? Vibe coding sucks.
by jmalicki
2/16/2026 at 5:25:12 PM
No, what the other commenter described is narrowly scoped delegation to LLMs paired with manual review (which sounds dreadfully soul-sucking to me), not wholesale "write feature X, write the unit tests, and review the implementation for me". The latter is vibe-coding.by GoatInGrey
2/16/2026 at 5:53:12 PM
Reviewing a quick translation of a test to a benchmark (or another menial coding tasks) is way less soul-sucking than doing the menial coding by yourself. Boring soul-sucking tasks are an important thankless part of OSS maintenance.I concur it is different from what you call vibecoding.
by krastanov
2/16/2026 at 5:50:24 PM
Sidenote, i do that frequently. I also do varying levels of review, ie more/less vibe[1]. It is soul sucking to me.Despite being soul sucking, I do it because A: It lets me achieve goals despite lacking energy/time for projects that don't require the level of commitment or care that i provide professionally. B: it reduces how much RSI i experience. Typing is a serious concern for me these days.
To mitigate the soul sucking i've been side projecting better review tools. Which frankly i could use for work anyway, as reviewing PRs from humans could be better too. Also inline with review tools, i think a lot of soul sucking is having to provide specificity, so i hope to be able to integrate LLMs into the review tool and speak more naturally to it. Eg i belive some IDEs (vscode? no idea) can let Claude/etc see the cursor, so you can say "this code looks incorrect" without needing to be extremely specific. A suite of tooling that improves this code sharing to Claude/etc would also reduce the inane specificity that seems to be required to make LLMs even remotely reliable for me.
[1]: though we don't seem to have a term for varying amounts of vibe. Some people consider vibe to be 100% complete ignorance of the architecture/code being built. In which case imo nothing i do is vibe, which is absurd to me but i digress.
by unshavedyak
2/16/2026 at 5:17:27 PM
It's not vibe coding if you personally review all the diffs for correctness.by lukeschlather
2/16/2026 at 5:07:06 PM
> According to Karpathy, vibe coding typically involves accepting AI-generated code without closely reviewing its internal structure, instead relying on results and follow-up prompts to guide changes.What you are doing is by definition not vibe coding.
by EnPissant
2/16/2026 at 4:49:05 PM
[dead]by dingnuts
2/16/2026 at 2:17:19 PM
Yeah esp. the latest iterations are great for stuff like “find and fix all the battery drainers.” Tests pass, everyone’s happy.by peyton
2/16/2026 at 4:03:17 PM
(rhetorical question) You work at Apple? :pby hp197
2/16/2026 at 2:18:29 PM
I work at a company with approximately $1 million in revenue per engineer and multiple 10+ year old codebases.We use agents very aggressively, combined with beads, tons of tests, etc.
You treat them like any developer, and review the code in PRs, provide feedback, have the agents act, and merge when it's good.
We have gained tremendous velocity and have been able to tackle far more out of the backlog that we'd been forced to keep in the icebox before.
This idea of setting the bar at "agents work without code reviews" is nuts.
by JPKab
2/16/2026 at 5:59:08 PM
If there is one thing I have seen is that there is a subset of intellectual people will still be adverse to learning new tools, hang to ideological beliefs (I feel this though, watching programming as you know it die in a way, kinda makes you not want to follow it) and would prefer to just be lazy and not properly dogfood and learn their new tooling.I'm seeing amazing result to with agents, when provided an well formed knowledge base and directed through each piece of work like its a sprint. Review and iron out scope requirements, api surface/contract, have agents create multi phase implementation plans and technical specifications in a share dev directory and to make high quality changes logs, document future consideration and any bugs/issues found that can be deferred. Every phase is addressed with a human code review along with gemini who is great at catching drift from spec and bugs in less obvious places.
While I'm sure an enterprise code base could still be an issue and would require even more direction (and opus I wont let touch java, it codes like an enterprise java greybeard who loves to create an interface/factory for everything), I think that's still just a tooling issues.
I'm not of the super pro AI camp, but having followed its development and used it throughout. For the first time I am actual amazed and bothered, and convinced if people dont embrace these tools, they will be left behind. No they dont 10-100x a jr dev, but if someone has proper domain knowledge to direct the agent, performs dual research with it to iron things out with the human actually understanding the problem space, 2-5x seems quite reasonable currently if driven by a capable developer. But this just move the work to review and documentation maintenance/crafting. Which has its own fatigue and is less rewarding for a programmers mind who loves to solve challenges and gets dopamine from it .
But given how man people are adverse...I dont think anyone who embraces it is going to have job security issues and be replaced, but here are many capable engineers who might due to their own reservations. I'm amazed by how many intelligent and capable people try llms/agents like a political straw man, there is no reasoning with them. They say vibe coding sucks (it does for anything more than a small throw away that wont be maintained), yet their examples for agents/llm not working is it can't just take a prompt and produce the best code ever and automatically and manifest the knowledge needed to work on their codebase. You still need to put in effort and learn to actually perform the engineering with the tools, but if it doesnt take a paragraph with no AGENTS.md and turn it into a feature or bug fix they are not good to them. Yeah they will get distracted and fuck up, just like if you throw 9/10 developers in the same situation and told them to get to work with no knowledge of the code base or domain and have their pr in by noon.
by hickelpickle
2/16/2026 at 2:33:43 PM
Why are you using experience and authoritative framing about a technology we’ve been using for less than 6 months?by groundzeros2015
2/16/2026 at 3:12:55 PM
The person they are responding with dictated an authoritative framing that isn’t true.I know people have emotional responses to this, but if you think people aren’t effectively using agents to ship code in lots of domains, including existing legacy code bases, you are incorrect.
Do we know exactly how to do that well, of course not, we still fruitlessly argue about how humans should write software. But there is a growing body of techniques on how to do agent first development, and a lot of those techniques are naturally converging because they work.
by kasey_junk
2/16/2026 at 3:17:32 PM
I think programming effectiveness is inherently tied to the useful life of software, and we will need to see that play out.This is not to suggest that AI tools do not have value but that “I just have agents writing code and it works great!” Has yet to hit its test.
by groundzeros2015
2/16/2026 at 3:57:13 PM
The views I see often shared here are typical of those in the trenches of the tech industry: conservative.I get it; I do. It's rapidly challenging the paradigm that we've setup over the years in a way that it's incredibly jarring, but this is going to be our new reality or you're going to be left behind in MOST industries; highly regulated industries are a different beast.
So; instead of just out-of-hand dismissing this, figure out the best ways to integrate agents into your and your teams'/companies' workstreams. It will accelerate the work and change your role from what it is today to something different; something that takes time and experience to work with.
by garciasn
2/16/2026 at 4:20:23 PM
> I get it; I do. It's rapidly challenging the paradigm that we've setup over the years in a way that it's incredibly jarring,But it's not the argument. The argument is that these tools provide lower-quality output and checking this output often takes more time than doing this work oneself. It's not that "we're conservative and afraid of changes", heck, you're talking to a crowd that used to celebrate a new JS framework every week!
There is a push to accept lower quality and to treat it as a new normal, and people who appreciate high-quality architecture and code express their concern.
by benterix
2/16/2026 at 11:41:31 PM
"Find any inconsistencies that should be addressed in this codebase according to DRY and related best practices"This doesn't hurt to try and will give valuable and detailed feedback much more quickly than even an experienced developer seeing the project for the first time.
by joquarky
2/16/2026 at 4:19:53 PM
> It will accelerate the work and change your role from what it is today to something different;
We yet to see if different is good.My short experience with LLM reviewing my code is that LLM's output is overly explanatory and it slows me down.
> something that takes time and experience to work with.
So you invite us to participate in sunken cost fallacy.
by thesz
2/16/2026 at 11:44:05 PM
Tell it to summarize?by joquarky
2/16/2026 at 4:09:08 PM
I don’t doubt that companies are willing to try low quality things. They play with these processes all the time. Maybe the whole industry will try it.I’m available for consulting when you need something done correctly.
by groundzeros2015
2/16/2026 at 7:50:45 PM
Also people need to be more specific about technologies/tasks they do. Otherwise it's apples to oranges.by democracy
2/16/2026 at 4:04:51 PM
6 months?I've been using LLMs to augment development since early December 2023. I've expanded the scope and complexity of the changes made since then as the models grew. Before beads existed, I used a folder of markdown files for externalized memory.
Just because you were late to the party doesn't mean all of us were.
by JPKab
2/17/2026 at 12:45:55 AM
> Just because you were late to the party doesn't mean all of us were.It wasn't a party I liked back in 2023. I'm just repeating the same stuff I see said over and over again here, but there has been a step change with Opus 4.5.
You can still it in action now because the other models are still where Opus was at a while ago. I recently needed to make small change to script I was using. It is a tiny (50 line) script written with the help of AI's ages ago, but was subtly wrong in so many ways. It's now become clear neither the AI's (I used several and cross checked) nor myself had a clue about what we were dealing with. The current "seems to work" version was created after much blood caused by misunderstandings was spilt, exposing bugs that had to be fixed.
I asked Claude 4.6 to fix yet another misunderstanding, and the result was a patch changing the minimum number of lines to get the job done. Just reviewing such a surgical modification was far easier than doing it myself.
I gave exactly the same prompt to Gemini. The result was a wholesale rearrangement of the code. Maybe it was good, but the effort to verify that was far lager than just doing it myself. It was a very 2023 experience.
The usual 2023 experience for me was ask an AI write some greenfield code, and get a result that looked like someone had changed variable names in something they found on the web after a brief search for code that looked like it might do a similar job. If you got lucky, it might have found something that was indeed very similar, but in my case that was rare. Asking it to modify code unlike something it had seen before was like asking someone to poke your eyes with a stick.
As I said, some of the organisers of this style of party seem have gotten their act together, so now it is well worth joining their parties. But this is a newish development.
by rstuart4133
2/16/2026 at 3:38:09 PM
If you hired a person six months ago and in that time they'd produced a ton of useful code for your product, wouldn't you say with authoritative framing that their hiring was a good decision?by dboreham
2/16/2026 at 3:51:20 PM
It would, but I haven’t seen that. What I’ve seen is a lot of people setting up cool agent workflows which feel very productive, but aren’t producing coherent work.This may be a result of me using tools poorly, or more likely evaluating merits which matter less than I think. But I don’t think we can see that yet as people just invented these agent workflows and we haven’t seen it yet.
Note that the situation was not that different before LLMs. I’ve seen PMs with all the tickets setup, engineers making PRs with reviews, etc and not making progress on the product. The process can be emulated without substantive work.
by groundzeros2015
2/16/2026 at 7:52:41 PM
Why is it always "tons of code"? Unless you are paid by the line of code writin "tons of code" makes no sense.by democracy
2/16/2026 at 7:53:11 PM
> We have gained tremendous velocity and have been able to tackle far more out of the backlog that we'd been forced to keep in the icebox before.Source? Proofs? It's not the first, second or even third round on this rodeo.
In other words, notto disu shittu agen.
by otabdeveloper4
2/16/2026 at 1:38:06 PM
That is also my experience. Doesn't even have to be a 10 year old codebase. Even a 1 year old codebase. Any one that is a serious product that is deployed in production with customers who rely on it.Not to say that there's no value in AI written code in these codebases, because there is plenty. But this whole thing where 6 agents run overnight and "tada" in the morning with production ready code is...not real.
by rco8786
2/16/2026 at 2:17:52 PM
I don't believe that devs are the audience. They are pushing this to decision makers where they want them to think that the state of the art is further ahead than it is. These folks then think about how helpful it'd be to have 20% of that capability. When there is so much noise in the market, and everyone seems to be overtaking everyone else it, this kind of approach is the only one that gets attention.Similarly, a lot of the AGI-hype comments exist to expand the scope of the space. It's not real, but it helps to position products and win arguments based on hypotheticals.
by zerkten
2/16/2026 at 2:48:22 PM
Also anything that doesn't look like a SaaS app does very badly. We had an internal trial at embedded firmware and concluded the results were unsalvageably bad. It doesn't help that the embedded environment is very unfriendly to standard testing techniques, as well.by pjc50
2/16/2026 at 11:54:02 PM
You will need to build an accessible knowledge base for the topics for which the models have not had extensive training.Proprietary embedded system documentation is not exactly ubiquitous. You must provide reference material and guardrails where the training is weakest.
This applies to everything in ML: it will be weakest at the edges.
by joquarky
2/16/2026 at 3:24:30 PM
I feel like you could have correctly stated this a few months ago, but the way this is "solved" is by multiple agents that babysit each other and review their output - it's unreasonably effective.You can get extremely good results assuming your spec is actually correct (and you're willing to chew through massive quantities of tokens / wait long enough).
by JeremyNT
2/16/2026 at 6:04:32 PM
> You can get extremely good results assuming your spec is actually correctIs it ever the case that the spec is entirely correct (and without underspecified parts)? I thought the reason we write code is because it's much easier to express a spec as code than it is to get a similar level of precision in prose.
by nicoburns
2/16/2026 at 6:22:07 PM
I think this is basically the only SWE-type job that exists beyond the (relatively near) future: finding the right spec and feeding it to the bots. And in this way I think even complete laypeople will be able to create software using the bots, but you'd still want somebody with a deeper understanding in this role for serious projects.The bots even now can really help you identify technical problems / mistakes / gaps / bad assumptions, but there's no replacing "I know what the business wants/needs, and I know what makes my product manager happy, and I know what 'feels' good" type stuff.
by JeremyNT
2/16/2026 at 7:59:09 PM
> finding the right spec and feeding it to the botsAlso known as "compiling source code".
by otabdeveloper4
2/16/2026 at 3:36:20 PM
And unreasonably expensive unless you are Big Corp. Die startups, die. Welcome to our Cyberpunk overlords.by ldng
2/16/2026 at 5:17:50 PM
Companies will just shift money from salaries to their Anthropic bill - what's the problem?by whateveracct
2/16/2026 at 6:24:33 PM
Or hey, the VCs can self-deal by funding new startups that buy bot time from AI firms the same VCs already fund.No pesky developers siphoning away equity!
by JeremyNT
2/16/2026 at 10:44:42 PM
My Claude Code has been running weeks on end churning through a huge task list almost unattended on a complex 15 yr old code base, auto-committing thousands of features. It is high quality code that will go live very soon.by flemhans
2/16/2026 at 11:36:35 PM
Awesome! Which application or service?by oblio
2/16/2026 at 7:32:47 PM
The gas town discord has two people that are doing transformation of extremely legacy in house Java frameworks. Not reporting great success yet but also probably work that just wouldn’t be done otherwise.by lanstin
2/16/2026 at 7:44:13 PM
Oh that means you don't know how to use AI properly. Also its only 2026 imagine what AI agents can do in a few years /sby democracy
2/16/2026 at 2:31:52 PM
Related question: how do we resolve the problem that we sign a blank cheque for the autonomous agents to use however many tokens they deem necessary to respond to your request? The analogy from team management: you don't just ask someone in your team to look into something only to realize three weeks later (in the absence of any updates) that they got nowhere with a problem that you expected to take less than a day to solve.EDIT: fixed typo
by pzs
2/16/2026 at 2:36:02 PM
We'll have to solve for that sometime soon-ish I think. Claude Code has at least some sort of token estimation built-in to it now. I asked it to kick off a large agent team (~100 agents) to rewrite a bunch of SQL queries, one per agent. It did the first 10 or so, then reported back that it would cost too much to do it this way...so it "took the reins" without my permission and tried to convert each query using only the main agent and abandoned the teams. The results were bad.But in any case, we're definitely coming up on the need for that.
by rco8786
2/16/2026 at 2:51:26 PM
> blank chequeThe Bing AI summary tells me that AI companies invested $202.3 billion in AI last year. Users are going to have to pay that back at some point. This is going to be even worse as a cost control situation than AWS.
by pjc50
2/17/2026 at 1:03:34 AM
Didn't you hear? Ads are coming! (well not to Claude, because I guess they plan to somehow get unlimited SV funding?!)by lossyalgo
2/16/2026 at 3:14:28 PM
> Users are going to have to pay that back at some point.That’s not how VC investments work. Just because something costs a lot to build doesn’t mean that anyone will pay for it. I’m pretty sure I haven’t worked for any startup that ever returned a profit to its investors.
I suspect you are right in that inference costs currently seem underpriced so users will get nickel-and-dinked of a while until the providers leverage a better margin per user.
Some of the players are aiming for AGI. If they hit that goal, the cost is easily worth it. The remaining players are trying to capture market share and build a moat where none currently exists.
by thephyber
2/16/2026 at 3:54:11 PM
What planet are you living on and how do I get there.Yes currency is very rarely at times exchanged at a loss for power but rarely not for more currency down the road.
by tsunamifury
2/16/2026 at 2:34:39 PM
An AI product manager agent trained on all the experience of product managers setting budgets for features and holding teams to it. Am I joking? I do not know.by Kye
2/16/2026 at 3:17:50 PM
This seems pretty in line with how you’d manage a human - you give it a time constraint. a human isn't guaranteed to fix a problem either, and humans are paid by timeby peab
2/16/2026 at 8:13:27 PM
I think this is exactly the crux: there are two different UX targets that get conflated.In operator/supervisor mode (interactive CLI), you need high-signal observability while it’s running so you can abort or re-scope when it’s reading the wrong area or compounding assumptions. In batch/autonomous mode (headless / “run overnight”), you don’t need a live scrollback feed, but you still need a complete trace for audit/debug after the fact.
Collapsing file paths into counters is a batch optimization leaking into operator mode. The fix isn’t “verbose vs not” so much as separating channels: keep a small status line/spine (phase, current target, last tool call), keep an event-level trace (file paths / commands / searches) that’s persisted and greppable, and keep a truly-verbose mode for people who want every hook/subagent detail.
by twalichiewicz
2/16/2026 at 3:02:55 PM
yeah I think that's exactly the disconnect - they're optimizing for a future where agents can actually be trusted to run autonomously, but we're not there yet. like the reliability just isn't good enough to justify hiding what it's doing. and honestly I'm not sure we'll get there by making the UX worse for humans who are actively supervising, because that's how you catch the edge cases that training data misses. idk, feels like they're solving tomorrow's problem while making today's harderby the_harpia_io
2/16/2026 at 8:07:00 PM
We run agent teams (Navigator/Driver/Reviewer roles) on a 71K-line codebase. The trust problem is solved by not trusting the agents at all. You enforce externally. Python gates that block task completion until tests pass, acceptance criteria are verified, and architecture limits are met. The agents can't bypass enforcement mechanisms they can't touch. It's not about better prompts or more capable models. It's about infrastructure that makes "going off the rails" structurally impossible.by buschleague
2/16/2026 at 2:35:51 PM
>The latest "meta" in AI programming appears to be agent teams (or swarms or clusters or whatever) that are designed to run for long periods of time autonomously.more reason to catch them otherwise we have to wait a longer time. in fact hiding is more correct if the AI was less autonomous right?
by simianwords
2/16/2026 at 8:11:25 PM
Agent teams working autonomously sounds cool until you actually try it. We've been running multi-agent setups and honestly the failure modes are hilarious. They don't crash, they just quietly do the wrong thing and act super confident about it.by fdefitte
2/16/2026 at 4:04:54 PM
If they're aiming for autonomy, why have a CLI at all? Just give us a headless mode. If I'm sitting in the terminal, it means I want to control the process. Hiding logs from an operator who’s explicitly chosen to run it manually just feels weirdby KurSix
2/16/2026 at 1:28:41 PM
Looking at it from far is simply making something large from a smaller input, so its kind of like nondeterministic decompression.What fills the holes are best practices, what can ruin the result is wrong assumptions.
I dont see how full autonomy can work either without checkpoints along the way.
by faeyanpiraat
2/16/2026 at 1:40:25 PM
Totally agreed. Those assumptions often compound as well. So the AI makes one wrong decision early in the process and it affects N downstream assumptions. When they finally finish their process they've built the wrong thing. This happens with one process running. Even on latest Opus models I have to babysit and correct and redirect claude code constantly. There's zero chance that 5 claude codes running for hours without my input are going to build the thing I actually need.And at the end of the day it's not the agents who are accountable for the code running in the production. It's the human engineers.
by rco8786
2/16/2026 at 2:20:42 PM
Actually it works the other way. With multiple agents they can often correct each others mistaken assumptions. Part of the value of this approach is precisely that you do get better results with fewer hallucinated assumptions.Still makes this change from Anthropic stupid.
by adastra22
2/16/2026 at 2:30:57 PM
The corrective agent has the exact same percentage chance at making the mistake. "Correcting" an assumption that was previously correct into an incorrect one.If a singular agent has a 1% chance of making an incorrect assumption, then 10 agents have that same 1% chance in aggregate.
by rco8786
2/16/2026 at 3:38:41 PM
You are assuming statistical independence, which is explicitly not correct here. There is also an error in your analysis - what matters is whether they make the same wrong assumption. That is far less likely, and becomes exponentially unlikely with increasing trials.I can attest that it works well in practice, and my organization is already deploying this technique internally.
by adastra22
2/16/2026 at 4:37:29 PM
How several wrong assumptions make it right with increasing trials?by thesz
2/16/2026 at 4:49:47 PM
You can ask Opus 4.6 to do a task and leave it running for 30min or more to attempt one-shooting it. Imagine doing this with three agents in parallel in three separate work trees. Then spin up a new agent to decide which approach of the three is best on the merits. Repeat this analysis in fresh contexts and sample until there is clear consensus on one. If no consensus after N runs, reframe to provide directions for a 4th attempt. Continue until a clear winning approach is found.This is one example of an orchestration workflow. There are others.
by adastra22
2/16/2026 at 5:32:10 PM
> Then spin up a new agent to decide which approach of the three is best on the merits. Repeat this analysis in fresh contexts and sample until there is clear consensus on one.
If there are several agents doing analysis of solutions, how do you define a consensus? Should it be unanimous or above some threshold? Are agents scores soft or hard? How threshold is defined if scores are soft? There is a whole lot of science in voting approaches, which voting approach is best here?Is it possible for analyzing agents to choose the best of wrong solutions? E.g., longest remembered table of FizzBuzz answers amongst remembered tables of FizzBuzz answers.
by thesz
2/16/2026 at 8:31:29 PM
We have a voting algorithm that we use, but we're not at the level of confidential disclosure if we proceed further in this discussion. There's lots of research out there into unbiased voting algorithms for consensus systems.by adastra22
2/16/2026 at 10:19:34 PM
You conveniently decided not to answer my question about quality of the solutions to vote on (ranking FizzBuzz memorization).To me, our discussion shows that what you presented as a simple thing is not simple at all, even voting is complex, and actually getting a good result is so hard it warrants omitting answer altogether.
by thesz
2/16/2026 at 7:59:05 PM
It was a scene from a sci-fi movie (i mean Claude demo to CTOs)by democracy
2/16/2026 at 2:37:05 PM
Nonsense. If you have 16 binary decisions that’s 64k possible paths.by groundzeros2015
2/16/2026 at 3:35:40 PM
These are not independent samplings.by adastra22
2/16/2026 at 3:47:00 PM
Indeed. Doesn’t that make it worse? Prior decisions will bring up path dependent options ensuring they aren’t even close to the same path.by groundzeros2015
2/16/2026 at 4:41:53 PM
Run a code review agent, and ask it to identify issues. For each issue, run multiple independent agents to perform independent verification of this issue. There will always be some that concur and some that disagree. But the probability distributions are vastly different for real issues vs hallucinations. If it is a real issue they are more likely to happen upon it. If it is a hallucination, they are more likely to discover the inconsistency on fresh examination.This is NOT the same as asking “are you sure?” The sycophantic nature of LLMs would make them biased on that. But fresh agents with unbiased, detached framing in the prompt will show behavior that is probabilistically consistent with the underlying truth. Consistent enough for teasing out signal from noise with agent orchestration.
by adastra22
2/16/2026 at 2:22:52 PM
Take a look at the latest Codex on very-high. Claude’s astroturfed IMHO.by peyton
2/16/2026 at 2:32:58 PM
Can you explain more? I'm talking about LLM/agent behavior in a generalized sense, even though I used claude code as the example here.What is Codex doing differently to solve for this problem?
by rco8786
2/16/2026 at 4:14:25 PM
>Through that lens, these changes make more sense. They're not designing UX for a human sitting there watching the agent work. They're designing for horizontally scaling agents that work in uninterrupted stretches where the only thing that matters is the final output, not the steps it took to get there.Even in that case they should still be logging what they're doing for later investigation/auditing if something goes wrong. Regardless of whether a human or an AI ends up doing the auditing.
by logicchains
2/16/2026 at 1:26:29 PM
Yes, this is why I generally still use "ask for permission" prompts.As tedious as it is a lot of the time ( And I wish there was an in-between "allow this session" not just allow once or "allow all" ), it's invaluable to catch when the model has tried to fix the problem in entirely the wrong project.
Working on a monolithic code-base with several hundred library projects, it's essential that it doesn't start digging in the wrong place.
It's better than it used to be, but the failure mode for going wrong can be extreme, I've come back to 20+ minutes of it going around in circles frustrating itself because of a wrong meaning ascribed to an instruction.
by xnorswap
2/16/2026 at 5:34:30 PM
fwiw there are more granular controls, where you can for example allow/deny specific bash commands, read or write access to specific files, using a glob syntax:https://code.claude.com/docs/en/settings#permission-settings
You can configure it at the project level
by lachlan_gray
2/16/2026 at 3:06:07 PM
oh man the going-in-circles thing - that's the worst because you don't even know how long to let it run before you realize it's stuck. I've had similar issues where it misunderstands scope and starts making changes that cascade in ways it can't track. the 'allow this session' idea is actually really good - would be useful to have more granular control like that. honestly this is why I end up breaking work into smaller chunks and doing more prompt-response cycles rather than letting it run autonomously, but that obviously defeats the purpose of having an agent do the workby the_harpia_io
2/16/2026 at 5:32:16 PM
You look at what Claude’s doing to make sure it doesn’t go off the rails? Personally, I either move on to another ask in parallel or just read my phone. Trying to catch things by manually looking at its output doesn’t seem like a recipe for success.by hippo22
2/17/2026 at 12:13:32 AM
It all depends on how much you're willing to spend.If you have an unlimited budget, obviously you will tend to let it run and correct it in the next iteration.
If you often run tight up against your 5-hour window, you're going to be more likely to babysit it.
by joquarky
2/16/2026 at 7:58:47 PM
yeah I watch it but not like staring at every line - more like checking when something feels off or when it's been chugging for a bit. if it starts pulling in files I didn't expect or the diff looks weirdly large I'll stop and check what's happening. the problem isn't that you can't catch it after the fact, it's that by then it's already burned 5 minutes reading your entire utils folder or whatever, and you gotta context switch to figure out what it misunderstood. easier to spot the divergence earlyby the_harpia_io
2/16/2026 at 1:49:55 PM
Exactly, and this is the best way to do code review while it's working so that you can steer it better. It's really weird that Anthropic doesn't get this.by aceelric
2/16/2026 at 3:04:49 PM
yeah the steering thing is huge - like when you can see it's about to go down the wrong path you can interrupt before it wastes time. or when you realize your prompt wasn't clear enough, you catch it early. hiding all that just means you find out later when it's already done the wrong thing, and then you're stuck trying to figure out what went wrong and how to fix it. it's the difference between collaborative coding and just hoping the black box does what you wantby the_harpia_io
2/16/2026 at 2:21:30 PM
I assume it's to make it harder for competitors to train on Claude's Chain-of-Thought.by charliea0
2/16/2026 at 3:03:53 PM
hm, maybe? but seems like a weird tradeoff to make UX actively worse just for that - especially since the thinking tokens are still there in the API responses anyway, just hidden in the UI. I think it's more likely they're just betting most people don't care about the intermediate steps and want faster responses, which tbh tracks with how most product teams think about 'simplification'. though it does feel short-sighted when the whole point of using these tools is trust and verificationby the_harpia_io
2/17/2026 at 1:39:13 AM
It is, but Anthropic is deluded if they think they can gain a moat from that. They gladly copy/borrow ideas from other projects that's why they are paranoid that folks are doing the same.by segmondy
2/16/2026 at 4:59:42 PM
Not true because it is still exposed in APIby simianwords
2/16/2026 at 10:49:02 PM
And saved in filesby flemhans
2/16/2026 at 1:30:52 PM
The other side of catcing going off the rails is when it wants to make edits without it reading the context I know would’ve been neccessary for a high quality change.by faeyanpiraat
2/16/2026 at 3:07:11 PM
yeah exactly - it's that confidence without understanding. like it'll make a change that looks reasonable in isolation but breaks an assumption that's only documented three files over, or relies on state that's set up elsewhere. and you can't always tell just from looking at the diff whether it actually understood the full picture or just got lucky. this is why seeing what files it's reading before it makes changes would be super helpful - at least you'd know if it missed something obviousby the_harpia_io
2/16/2026 at 6:00:38 PM
My first thought is, for the specific problem you brought up, you find out which files were touched by your version control system, not the AI's logs. I have to do this for myself even without AI.by socalgal2
2/16/2026 at 7:59:29 PM
yeah that works after the fact but the issue is more about catching it mid-run - like when Claude decides to read through 50 files to answer a simple question because it misunderstood the scope. by the time you check git you've already wasted the time and tokens. the logs help you spot that divergence while it's happening, not just afterby the_harpia_io
2/16/2026 at 6:23:07 PM
Previously you could see which files Claude was reading. If it got the totally wrong context you could interrupt and redirect it.Since it's just reading at that stage there's no tracked changes.
by graeme