Estimation Accuracy in Software Engineering

By: Carly N., Director, Application Archiecture

or… How to Advance Your Career by Creating Your Own Redundancy

Are you responsible for ensuring that software projects are completed on time, on budget, and meet all specifications? Have you read article after article related to estimation, project management and software design?  Even though these tasks seem to be more project management related, as a developer I’m still held accountable for those business objectives. To help me accomplish these goals, I’ve investigated, tested, piloted, and deployed multiple software development management tools all designed to make delivering software easier.  Nothing new here, anyone who’s been in the software industry has experience in attempting to achieve these goals. Sadly, despite sophisticated tooling to support the development process, we all have much more experience with failing to meet these goals than we would like to admit.

When you fail, your boss (or your boss’ boss, or the owner, or the customer) will ask you “Why?  What went wrong? Why couldn’t you tell me how long it would take, or how much it would cost, or why it failed?”  In a previous role, I was working in an industry that is partly software, and partly mechanical/electrical engineering. The delivered product was the result of a large industrial engineering effort to build and install the automation hardware to outfit a distribution warehouse, along with a software engineering effort to design, develop, install and test all the software that will run the automation (and deliver to your doorstep that box of Nike’s that you just bought online).

One half of the project (installing all that metal) operates like any typical construction project.  This may be a gross oversimplification, but the classic example given to illustrate project planning might be building a road.  Over time, or perhaps by scientific principles, the project manager can determine how long it should take to grade the roadbed, install the rebar, pour the concrete, let it set, test the results, line the lanes, clean up the equipment and open the road.  People are involved, but they are repeating a process that has been executed and refined over many projects, and ideally the PM has a very good idea of how long these processes should take, and also what the impacts are going to be if the graders run into a granite outcrop, or the concrete doesn’t test out when cured.

Why is estimation so hard?

Contrast this with Software Engineering.  One might say that after you’ve completed multiple projects, you gain the required data to understand how long it would take a developer to complete a task.  Partly right. Partly wrong. The partly wrong bit comes from the nature of the medium in which a developer performs his tasks. It all boils down to one thing: automation.  (Well, there are others, but we’re concentrating on the automation aspect)

When a developer finds they are doing the same task more than a few times, that task gets automated.  A task that one might think could be nailed down to consistently take four hours is now a script that automates the task and completes it unattended in 2 minutes.  Or that boilerplate code you have to write over and over again is generated by a class annotation, or provided by a maven archetype. Or the code you want to yank/paste from another project has been abstracted into a common class hierarchy, or transformed into a library call, or a REST service. Some may perceive that the net result is that there is less and less work to be done, as more and more of the predictable, simple tasks become automated.

“So what,” you might think, “that’s a sure sign of a good developer.”  And I agree, that is a totally correct assertion. But, this is precisely why it is so difficult to estimate the time to completion for the work being completed by this great developer.

Operating in an Environment of High Uncertainty

Referring to the construction project I mentioned earlier in the article, those project managers get closer and closer to accurate predictions because they benefit from empirical data that measured how long each of their past repeatable processes took to complete.  

In software projects, naturally repeatable processes are typically processes that we have previously recognized the importance to automate.  The rest is somewhat uncertain.

When all of the repeatable tasks have been automated, only the new, creative, open-ended, possibly-ill-defined-lets-figure-it-out-as we-go types of tasks remain to be done.  We can all agree that these tasks don’t have enough historical data to provide us any basis whatsoever for accurately estimating how long it will take. Consequently, these are the tasks that naturally get assigned to experienced developers to control for the lack of empirical evidence to provide a basis for delivering “on-time”.  

Experienced developers have built the tools that enable them to operate in an environment of high uncertainty, which by definition is an environment in which it is extremely difficult (ok, impossible, but allow me to hedge) to provide an accurate estimated time to completion with a high degree of certainty.  

However, the more that is automated, the more certain a delivery date can be despite this uncertainty.

The corollary to this: as a high-quality developer, don’t be afraid of automating yourself out of a job.  Most likely that will mean you will be chosen for another job, one in which you will again be expected to automate yourself out of.  In short, the interesting bits.