alt.hn

3/30/2025 at 5:26:33 PM

All estimations are wrong, but none are useful

 https://newsletter.techworld-with-milan.com/p/all-estimations-are-wrong-but-none

by Kerrick

4/2/2025 at 12:52:09 PM

In my experience, a substantial portion of poor estimates comes from:

1. An initial estimate is made that is fairly accurate.

2. Someone in management says "that's too long, we've got to find a way to bring in that date"

3. The estimate is revised to assume the best case possible execution time in each task, that resources will be available as soon as they are needed, that everything is fully parallelizable.

4. The estimate now magically matches the target date.

5. Actual execution time is close to the original estimate.

People make bad estimates because (other) people don't want good estimates.

by variaga

4/2/2025 at 1:04:21 PM

Early in my career, when I was young and naive, I used to spend an inordinate amount of time coming up with the perfect estimate. I could pretty well get it dead on nearly all the time, at the cost of wasting a stupid amount of time to determine it.

I soon noticed that it only ever lead to "okay, thanks" from whomever made the requested and that was the end of it. So now I just pick a random number. I still get "okay, thanks" and nothing changed.

Nobody really cares. It turns out it is just something people ask when they are uncomfortable with silence, but can't think of anything else to say.

by 9rx

4/2/2025 at 9:02:44 PM

My favourite joke whenever I tell people I make up estimates is “I’m completely wrong but I’d rather spend 30s and be wrong than 3h and still be just as wrong.”

by scott_w

4/2/2025 at 1:17:59 PM

That tracks. Frequently what people learn about estimating is to optimize out steps 1 and 2 and just make the estimate match the need-by date from the beginning.

by variaga

4/2/2025 at 1:23:50 PM

I'm sure that happens a lot. In my experience, people tend to be bad at estimating things.

As a long-time SWE who eventually (also) managed teams: I found estimates particularly difficult. Some SWEs would give very optimistic estimates (and always miss). Others would "sandbag" their estimates, because they didn't want to have to explain why they were late.

I tried really hard to make sure the SWEs on my team didn't get "blamed" if they were late and asked them to generally give aggressive estimates. My rationale was that if you estimated more time than it really took, you would tend to fill up that time no matter what.

by timmg

4/2/2025 at 2:06:52 PM

It depends what you do with the estimate. If you have to deliver a product/a functionality where X people depends on it to continue. It's best, in the general case, to overestimate a little to not have the X people preparing to work when they can't. If you estimate to schedule the task for the week, it can be more useful to be more aggressive since missing the deadline is less impactful.

For example, there are a lot of task that I depends on which can be completed in one week or three month. I don't care which it will be, I just want to know when it will be done so I can schedule other tasks before.

by claudex

4/2/2025 at 1:26:56 PM

I've had similar experiences but also reversed - saying estimates are too short, adding a buffer, saying they don't seem valid and we need to account for potential issues.

But also situations where a sales person would throw an estimate to the client which would end up being the official estimate due to no pushback. Or clients stretching the scope of a feature. Or underestimating/lack of understanding of a feature. Or changes from up top in the last days of building a feature.

Or the worst - overestimating due to previous experiences, where the dev team ended up on an 8 month estimate for 3 months worth of work tops which, if true, would have probably sank the company.

They can really be correct or close to truth - and useful to know what an ideal/okay/worst case is for planning, but are often then used as the holy grail and broken due to external influences.

The most useful thing about them is actual ritual of estimating, as it requires the team to mentally go through the items in the spec and align on things they usually would handwave away.

by thecupisblue

4/2/2025 at 1:28:36 PM

We value optimism, and telling ourselves that something won't be that much work makes it easier to start and commit ourselves to finding out that it's more work than we want later.

by datadrivenangel

4/3/2025 at 3:15:38 AM

I've found in my career so far estimates are very useless but "key dates" can be very useful.

A key date could be an investor demo, marketing launch, actual physical launch if you are in hardware, etc, most importantly something with an audience outside of any of the core team. Find that important date and work backwards from it and find out what is important to finish by then.

This approach has defintiley saved me a lot of sanity compared to arbitrary "estimates" that are destined to be push forever until you run into one of these key dates.

by 3vidence

4/1/2025 at 5:54:19 PM

Construction projects get reasonably accurately estimated every day. Probably because:

1. People have been doing it long enough that estimator is a job description.

2. That it is a job description, means money is spent on estimation.

3. Money is spent on estimation because getting it wrong can cost money.

I think the problem with software estimation is that there is usually no culture of estimation, usually very little direct incentive to get it right, and no regime of formal training for those responsible.

To put it another way, software does not have standard documents for estimating.

by brudgers

4/2/2025 at 12:49:33 PM

This is more or less reverse reasoning starting with a false equivalence. Most construction projects that are a tad different than the typical go over budget and over time. Most construction projects are thesame. If you ask a software engineer to build the same project repetitively, they can give you a very reasonable estimation.

by halayli

4/2/2025 at 1:03:18 PM

The cool thing with software engineering is that we can drive the costs of building the same project repetitively down to zero. We don't have to write Postgres from scratch each time we spin up a CRUD app, we can just install it.

I think that's why estimation in software engineering is difficult. The predictable repetitive portion of the project takes up no time - all that's left are the novel unknown parts.

by yen223

4/2/2025 at 2:08:37 PM

Most construction projects are the same.

That is not the case.

It is just that there’s no drama in on-time and on-budget.

by brudgers

4/2/2025 at 8:38:53 PM

Construction projects are so rarely on time and on budget, it’s a trope.

by lazide

4/2/2025 at 10:48:46 PM

I started in AEC in 1989. Most of the built environment happens without drama because only professionals are involved. Never showing up in the news except as a press release. And then usually not.

When amateurs are involved, so are unrealistic expectations. When politicians are involved, so is political grandstanding.

Finally, when the value of a project is the future revenue generated, budgets and schedules don’t extrapolate from your household experience.

Zero is the median number of construction projects people are involved in.

by brudgers

4/3/2025 at 11:05:12 AM

So if we exclude custom homes, small scale residential, government/public, and small and medium commercial, then everything goes well?

We could make similar claims about software probably, eh?

by lazide

4/2/2025 at 1:38:22 PM

I also had to do a double-take at the blatant falsity of this statement. Software estimation is only hard when it's the first time you're doing it. Unfortunately, most software is a first-time.

There are very few specialty shops which only do the same type of project over and over. This is probably also because such repetitive work would quickly be turned into a "framework" to do more of such work, leaving only the novel stuff for each project, which would then become difficult again.

In the real world, I guess that matches up with the "prefab" type of construction.

by sjamaan

4/2/2025 at 1:54:11 PM

Most software is NOT first time! You have competition doing it as well. One website is just like another - sure there are lots of differences, but they are as alike as any two houses with different blueprints are the same. Most software is updates to the existing software of a similar type to last years update. Even ground up software is typically a rewrite of your existing software.

by bluGill

4/2/2025 at 12:40:44 PM

Software development inherently involves more unknowns than fields like construction, making estimation harder. However, when tasks are broken down into simpler, well-defined components with clear requirements, estimation accuracy improves significantly. The challenge lies in the fact that software projects often begin with ambiguous goals, evolving technologies, and dynamic stakeholder needs—factors that are less prevalent in physical construction.

by t00ny

4/2/2025 at 1:44:25 PM

> Software development inherently involves more unknowns than fields like construction, making estimation harder.

If by construction you mean only cookie cutter houses and garden sheds, maybe, but construction doesn't end there. As soon as you look to anything interesting, construction has all the unknowns software has and an environment that brings even more unknowns. Computing environments, in contrast, are highly predictable. Software has it easy.

The trouble here is that the original premise is flawed. Construction estimation is almost never accurate. Open today's newspaper and you're almost certainly going to read about yet another construction project that is over time and over budget by millions upon millions of dollars.

by 9rx

4/2/2025 at 12:59:05 PM

In addition to all of the other comments a good analogy I'd heard in the past is related to plumbing.

If 80% of the way into building a house you realize you want to swap the bathroom and the living room you'd be laughed at. The pipes are laid you and you'd have to drop back down to the 50% complete level to move them.

But this happens all the time in software. It's why so many projects get stuck at the 80/90% level. Because those defining requirements, and many times even engineers, don't have a concrete understanding of what is plumbing and what is finishing it's very difficult to know whether the ask is to hang wallpaper or move a toilet.

by roxolotl

4/2/2025 at 1:50:10 PM

Such swaps happen all the time in construction. We call them "change orders" or "remodeling" depending on when in the building lifecycle they happen. In software we have not be good about figuring out which change orders are expensive.

I've been in construction, most of them are actually really cheap because the common/likely change orders are built in - want to add a fireplace: until the drywall is installed the cost doesn't change. Want to make a doorway bigger, no problem, it will cost you $20 + the bigger door. Want to switch the tub for a shower after the pumping is roughed in - $50 to move the drain and water pipes, but once the tub is installed you need to pay for the tub you don't use as well (it has to be broken to get it out). As you say moving the bathroom to a different location will be costly.

by bluGill

4/2/2025 at 1:14:00 PM

It happens remarkably often even in big construction projects, when requirements shift for political reasons or when something unexpected is discovered about the ground it's being built on.

by pjc50

4/2/2025 at 12:39:08 PM

Never mind cost/time estimates, most software projects barely have anything resembling blueprints, bill of materials or even size estimates at the start of the project. It's hard to estimate something if you don't even know how big it'll be, or what shape.

by HPsquared

4/2/2025 at 12:47:41 PM

> most software projects barely have anything resembling blueprints, bill of materials or even size estimates at the start of the project

Producing that is the software project. Once you've delivered those things, the compiler will carry out the construction. I am sure you can estimate how long the compiler will take with reasonable accuracy.

So, you're right, construction isn't the greatest analogy.

by 9rx

4/2/2025 at 1:20:51 PM

Ha. Well, most builds take 20 minutes. So any release will take approximately 20 minutes once we get the specification written out. Will be in touch in 1 month to 3 years.

by franktankbank

4/2/2025 at 1:18:21 PM

>I am sure you can estimate how long the compiler will take with reasonable accuracy.

I'm sure you can't unless you have a pretty good idea of the size and shape of the project.

by thfuran

4/2/2025 at 1:27:45 PM

Do the size and shape of your deliverables magically change without warning? Mine never have.

Maybe you've lost it all in a hard drive crash? But in that case it is really easy to figure out how long it will take when you have nothing to compile.

by 9rx

4/2/2025 at 2:22:00 PM

You've never ended up with significantly more or fewer deliverables at the end of the project than were expected at the start? You've never had a project scrapped 50% (or 100%) of the way through? You've never thought to yourself "just a smidge of boost can't be too hard on the compile time" only to regret it for the next few years?

by thfuran

4/2/2025 at 2:26:12 PM

I fail to see the significance of the past here. Per the comment in question, the deliverables have already been delivered. The size and shape is known. All that is left to do is to get shovels in the ground (i.e. compile it).

by 9rx

4/2/2025 at 12:50:28 PM

The ease and seemingly no-cost nature of small changes in software development, causes scope creep that would have huge costs in construction.

by bliteben

4/2/2025 at 1:31:28 PM

Changing the color of a door will cost you a few hundred dollars or less. Changing all the windows may cost tens or hundreds of thousands of dollars. Rotating the foundation five degrees may be more expensive than building a new building.

Not all changes are equally huge in construction.

by datadrivenangel

4/2/2025 at 1:26:02 PM

No, even construction projects don't get accurately estimated. Software development is R&D, not a construction - lot more unknowns. It is not skill issue by any means.

by Tractor8626

4/2/2025 at 1:36:00 PM

I'm not convinced "software development is R&D, not a construction" is really true. But, I agree there are more unknowns.

Maybe more accurately... - construction project estimates are done after architecture and some engineering analysis are completed. - software projects often include the architecture and analysis

by alistairSH

4/2/2025 at 2:09:43 PM

> I'm not convinced "software development is R&D, not a construction" is really true.

Software development being development (the D in R&D) must be true, surely? It literally asserts it.

It may be that some non-development work in software is inappropriately labeled software development, granted.

by 9rx

4/2/2025 at 3:55:04 PM

Yep, that's exactly it. Traditional (meat-space) engineering has a clearer delineation between design and build (and whatever other phase you want to include).

We can see this in estimating bigger infrastructure projects (where there's an end-to-end estimate of time and budget) - at least in the US, these almost always miss as well.

But, building a housing development (once permitting/zoning is done) has a much higher likelihood of being close to estimates. Building a spec house is a "solved" problem. Similarly, if a software team is building a pile of CRUD APIs with a basic web frontend, in a domain they have experience, they should be able to estimate that pretty well.

by alistairSH

4/2/2025 at 4:04:00 PM

> Traditional (meat-space) engineering has a clearer delineation between design and build

I'm not so sure about that. But, in software, the builders - historically known as coders - got automated out of a job by compilers. Software is entirely design-side nowadays. Builds are done by machine.

by 9rx

4/2/2025 at 1:17:20 PM

I think you’re right to a degree. Certainly there is a lazy approach to estimates in software compared to other fields. But I think that’s because the estimation is hard enough to feel hopeless at times. Unless the task is truly something mundane that’s been done 100x before, as in the case of construction where many things are standardized, then the buffer for uncertainty needs to be huge, meaning everything is estimated to take a long time, making the whole process pretty useless.

Even for mundane tasks, I frequently find bugs in libraries, software versions that are incompatible with each other unless I apply just the right combination of upgrades and downgrades. SaaS vendors change their pricing structure or deprecate the APIs I am familiar with. At each step, there’s a little bit of debugging, learning, and creativity in working around problems. Each task is a small piece of art, important or otherwise. And you wouldn’t bother asking a painter how long their landscape will take to finish, would you? It could take a day or it could take months. Timebox it if you want, but that won’t have the same result.

You can save time and make better estimates by never going off the beaten path and by reducing your reliance on 3rd parties, but that only gets you so far and results in boring, homogenous software. Kind of like modern architecture and construction practices, which plenty of people don’t like.

by sholladay

4/2/2025 at 2:01:30 PM

In construction, if all you're doing is building (essentially) the same house, with minor variations, and little risk, you can make very accurate estimates.

In contrast, the construction projects that go over schedule and over budget typically involve a certain degree of novelty where you just can't say, "it typically takes me 5 months to build a house from these plans, so it'll probably take 5 months."

The thing with software is that no one writes the same program over and over again. It's more like the situation where someone is building a unique bridge and it goes over budget.

by gwbas1c

4/2/2025 at 12:39:34 PM

Hmm.

What about the difference between building a house from a blueprint vs building software from tickets?

IMO the blueprint is a much clearer plan.

EDIT: In fact, I'd argue that the software is more like the blueprint itself than the finished construction project.

by fhennig

4/2/2025 at 12:47:58 PM

Houses would be much different if they could be built on the inner side of the black hole.

by Ygg2

4/2/2025 at 12:50:43 PM

There is a better analogy between "compiling and installing software" and "building a house from a blueprint". The difference is that in software, anything that can be automated, now has automation. Which is easier with bits than it is with bricks, wires and pipes.

The process of writing software is more analogous to drawing up the house blueprints.

This is where the the whole question "If building houses is predictable, why can't writing software be?" falls apart. Because the unpredictable part of housebuilding is over when you have a bespoke or off-the-shelf blueprint. Source code is a blueprint. The software compile-test-deploy-install pipeline is actually fairly predictable.

by SideburnsOfDoom

4/2/2025 at 12:50:39 PM

In software we're always doing new things, relying on new code internal and external, so we always have to learn new things and run into unexpected problems, neither of which can not be reliably accounted for. The more routine your task is, the easier it is to provide an estimate. Construction technology and blueprints are not constantly changing, right?

by esafak

4/2/2025 at 12:56:54 PM

Are we fucking around here? There are exactly two construction projects that got estimated on time and cost in the US. Cost overruns are so routine as to not be noticeable.

In fact, if this is not the case, then this makes a good argument for the cancelation of CA HSR since it must be run by notoriously incompetent people since it is delayed decades and over a hundred billion over budget.

by renewiltord

4/2/2025 at 1:18:42 PM

Yeah. Construction projects arriving on time and on budget is almost unheard of, aside from maybe the construction of common structures, like residential homes, where the builders have already built the same building 100 times before.

by 9rx

4/2/2025 at 1:32:27 PM

One insight here is that there is a cost to estimation, and that your marginal improvement on estimate accuracy begins to level off, so too much estimation is waste.

by datadrivenangel

4/2/2025 at 12:42:43 PM

4) A house, e.g. "3 beds, 2 baths, timber frame" is a much more known quantity with standard tools and techniques that have been executed many many times before, than a typical software project. The house builder can average the last 100 times that spec has been executed. The software project cannot. It is inherently much more variable and speculative.

It's not entirely to do with "blame the people" ideas such as "no culture of estimation" and "little direct incentive". The nature of the project and the way to go about is simply much more wide open. And doesn't have such easily-defined endpoints.

by SideburnsOfDoom

4/2/2025 at 1:25:13 PM

I don't think you understand software or construction. Construction runs over all the time for many reasons in common with software, but in general, construction is a horrible analogy to use for software development.

by gilbetron

4/2/2025 at 1:45:25 PM

This is literally the FIRST bullshit I was fed when I started my software development career. Buildings are always on time because they're too important to be finished late! Buildings are expensive! Buildings are serious! If a house wasn't finished on time, a family might have to find somewhere else to live! If a sports stadium wasn't finished on time, the season would be ruined! If an office building wasn't finished on time, businesses with leases would have to find other office space! Hundreds of millions of dollars would be lost! Nobody could afford that, and it doesn't happen, because the people who build buildings understand How Important It Is! If programmers would just grow up and start acting like responsible adults like the people who build buildings, maybe start wearing collared shirts to work, then they would start shipping software on time!

Yeah, except all of those things do happen. Families spend months renting while their new house gets built. My city's new professional sports team played on the road the first several months of their first season, like the previous two expansion teams before them, because the stadium was behind schedule. Office buildings with 100% of their office space pre-leased have to push back their opening dates, leaving all their new tenants in the lurch and giving them the right to cancel their leases and go elsewhere. It turns out construction estimation is not reliable, either, despite the fact that much of construction is more analogous to a software build process than a software development process.

A lot of the reasons are pretty common-sense, and many will sound familiar to software developers:

- New building products that were expected to be drop-in replacements for older products turn out to have unexpected limitations.

- Available labor isn't as familiar with materials as expected.

- Economic conditions change, and it takes longer than expected to get materials.

- Economic conditions change, and the client decides to make changes to the project in response.

- Something about the site that was expected to be discovered during site engineering gets discovered at the beginning of construction instead.

- Another project going over schedule or over budget has a cascading effect.

- Labor becomes so scarce that construction has to be started late.

- Materials are accidentally ruined on site, and they can't be replaced quickly. (Imagine that a junior engineer arrived early one morning and ran a bunch of builds on a broken branch. Now you're scrambling to locate a supplier who can get you more build juice, and they all have a two week lead time.)

Unlike in software, money and time can often be traded off in construction. When labor and materials become scarce, you can pay extra to be the project that gets the best available, or gets them at all. This can create the appearance of more money buying better estimation, which is true to an extent, but sometimes it's simply a consequence of bigger projects have more on the line and more cash available. If you are building a $1m house, you are likely to discover that the team building a $5m house across town can afford to pay more to stay on schedule than you can. Likewise if you're building a $10m commercial building and something gets scarce, it's a good bet that the $100m project across town will offer more for it. They'll finish on time, you'll just have to wait.

by dkarl

4/2/2025 at 12:38:29 PM

Well put!

by blastro

4/2/2025 at 12:57:56 PM

Risks, risks & risks.. That's my #1 priority on communicating estimates.

Overall this is a nice short summary on the topic. The one thing I would add that I found very helpful on larger projects is communicating the risks & unknowns. I suggest listing them out at the start of the project & update their status as you work on it.

I've worked on teams where it's done with a simple color (red, yellow or green) on how confident we are on the task estimate based on risks/unknowns. This is the simplest way in my opinion.

I also like Basecamp's Hill Charts - https://3.basecamp-help.com/article/412-hill-charts

by mattferderer

4/2/2025 at 1:12:40 PM

My first job I worked was for a small agency. I was kind of a young go getter and was able to very quickly pump out bad (in hindsight) yet mostly functional code very quickly.

Sales people figured out they could come to me specifically for estimates because they would be shorter than other developers estimates and more likely to get the sale. I was young and dumb and didn't connect that often times I wouldn't be the one developing the project. The other developers got angry with me after they caught on to why they were getting impossible timelines.

I started multiplying my personal estimates by three. The sales people were less than pleased and eventually started going elsewhere for their estimates as greener developers were hired.

by donatj

4/2/2025 at 3:03:06 PM

Building software is often like building a house. When construction starts, progress appears to happen fast. You put up walls pretty fast so it starts looking like a real house in a short time. Later, construction progress appears to slow down significantly as work shifts to detail work (wiring, plumbing, etc.) which doesn't change the appearance from the street much.

With software, the basic UI can take shape quickly. Some rudimentary functionality sometimes comes along quickly as well. Then all the detail work (error handling, logging, performance enhancements, etc.) makes progress appear to slow significantly.

by didgetmaster

4/2/2025 at 1:39:43 PM

My take is (and I somwhat agree with the article on this) that if you know requirements, interfaces and tasks precisely enough to give a reliable estimate, then that means that the majority of actual development was already done.

But thats just not the point where people typically want estimates, they want them much earlier.

by myrmidon

4/2/2025 at 1:37:58 PM

You can only estimate job you’ve done before.

Everything else is just guessing.

Software engineering is not uniform: agencies working on similar projects may have good estimates.

R&D and novel approaches - usually takes whatever time it needs and then some more.

by hkpack

4/2/2025 at 1:03:24 PM

Skim read the article, why is it "But None are useful" and not "And none are useful"?

by ksec

4/2/2025 at 1:13:47 PM

It's a play on the phrase "All models are wrong, but some are useful" in reference to statistical models (but often applied more generally to scientific or mental models).

by PerilousBardom

4/2/2025 at 1:07:55 PM

It's a reference to "All models are wrong, but some are useful"

by turnsout

4/2/2025 at 3:58:05 PM

The value is in the activity itself, like planning, rather than the final document.

by hakaneskici

4/2/2025 at 1:35:40 PM

This remains great reading for software estimation: https://www.researchgate.net/publication/247925262_Large_Lim...

I'm barely even interested in talking about the subject anymore - estimating software is only useful as an alternative viewpoint to help understand what you are building. Nobody accurately estimates complex software in any meaningful way. Even min/max/most-likely estimation isn't that useful, but ok to think about.

Hidden complexity and simplicity lurks everywhere. "3 month projects" turn into 3 day projects because of an unknown tool. Or a simple problem (think fermat) turns out to be insanely complicated.

The core issue is that when you create a function, or module, or whatever, you never need to do it again. So you are always creating new things. If you are writing the same code again and again, you are bad. Now, the past 10 years or so, that was less true as we wrangled with yaml configs and gluing together pipelines and modules and services. But I think AI is really good at that stuff, so I think we'll be back to always working on original things.

And that doesn't even take into consideration legacy codebases - good luck estimating changes to those ;)

by gilbetron

4/2/2025 at 1:27:20 PM

If none are useful, then why does my manager keep asking me for them?

by amelius

4/2/2025 at 1:32:01 PM

It's a low effort way to make you consider all the necessary steps and possible contingencies for delivering the work. Creating the DB, onboarding to the API, etc., etc.

by JackFr

4/2/2025 at 4:03:33 PM

This. It's usually all the things besides writing code that takes most of the time.

by hakaneskici

4/2/2025 at 1:23:56 PM

Software estimates also often fail to account for external factors. You may have a fairly good sense for how long some feature will take to get to code-complete. However, code reviews on many times are nobody’s job, so they happen whenever (and you end up with a Volunteer’s Dilemma of completing them). Then CI, linters, etc. may have various failures. Some of these may be legitimate, others may be flakiness your team has decided to ignore, or even infrastructure issues that prevent them from running at all. There’s enough X factor in external sources alone to make any estimate you provide total nonsense.

by teeray

4/2/2025 at 1:43:24 PM

In 25 years in industry, I have never seen estimates proven to be valuable or important, mostly used for dumb purposes, and generally are a waste of time.

At this point I feel Kanban style priorities and limiting work in progress to be only useful approach.

When some product person dumps a huge task in the priorities, break it down into smaller deliverables. No estimates are really needed.

by gedy

4/2/2025 at 1:13:50 PM

I hate it when people quote Hofstadter's "law" in a literal sense. It was never made to make any statement about timeline estimation. It was just an example of a recursively-defined law. Honestly I'm not even sure how many of the other quotes are also taken out of context.

by j_maffe