Software Estimation: Demystifying the Black Art

by Steve McConnell

Often referred to as the "black art" because of its complexity and uncertainty, software estimation is not as difficult or puzzling as people think. In fact, generating accurate estimates is straightforward—once you understand the art of creating them.

In his highly anticipated book, acclaimed author Steve McConnell unravels the mystery to successful software estimation—distilling academic information and real-world experience into a practical guide for working software professionals. Instead of arcane treatises and rigid modeling techniques, this guide highlights a proven set of procedures, understandable formulas, and heuristics that individuals and development teams can apply to their projects to help achieve estimation proficiency.

Genres: Programming, Software, Management, Technology, Nonfiction, Computer Science, Technical

308 pages, Paperback

First published January 1, 2006

Reviews

[Marek · Rating 3 out of 5]

"Software Estimation - Demystifying the Black Art" is a boring book. I read it because I wanted to have tools to discuss the subject, and I think this books accomplishes that.

To me, the first and last chapters which dealt with conceptualizing the problem space in general were the most interesting. The bulk of the book consists of different techniques to actual estimation, which I suppose would open up by actually trying to apply the methods described.

Overall I found the book to be interesting, although somewhat tedious and more geared toward estimating big projects on a high level. For a developer being asked for feature-level estimates it gives some tools for presentation and discussion, but not much more.

[pearl]

I've reached this point in my life I suppose.

[Ha Truong · Rating 5 out of 5]

An excellent book on software estimation.

Not like other books that scare readers at the first glance with sophisticated math equations, this book naturally comes with the practical methods (what, how, why and caveats). It also provides plenty of tips and diagrams to depict the definitions and methodologies.

Besides, it states issues in reality and how to fix such as the estimation can be impacted by the executives or the marketers, the estimator should defend and make an appropriate commitment (last chapter)

[Mad Hab · Rating 5 out of 5]

The must read book for everyone who is struggling with the soft size estimation. Very detailed and informative and entertaining book.

[Taras Fedoruk · Rating 4 out of 5]

It is a good book full of interesting formulas and laws that you are free to follow while estimating the software development project. It is 4 stars because some of the facts and approaches like counting the lines of code are outdated. Apart from this, it is a decent basis helpful for everyone.

[Marco Bizzarri · Rating 4 out of 5]

Very detailed discussion about software estimation process.

[Bartosz · Rating 5 out of 5]

The most straightforward book I’ve found on this subject. Works great as a reference, and a textbook. Highly recommended to anyone who wants to up their estimation game.

[Allisonperkel · Rating 5 out of 5]

"Software Estimation" by Steve McConnell provides a very broad overview of many ways to reduce the software estimation errors for your development cycle. Like all of Mr McConnell's books, he provides crystal clear writing with tons of techniques that are ready for application in the real world.

One of the many great things about "software Estimation" is the sheer number of methods he gives. From Lines of code, to function points, to similar projects, to industry estimates (broken down by sub category so that database is different from embedded devices), to t shirt sizing, to maintaining development history: he makes it clear that you have a lot of different options available to you. He takes great pains to emphasize that one size does not fit all. Additionally he gives rationales for when the estimate techniques work in a project's lifecycle.

With all the methods described, another point driven home is that software is something of an art and that you can reduce the amount of uncertainty but you can never fully remove it. None of the methods that improve estimation are silver bullets. I love that he draws the line in the sand here. Its very true and in fact he goes a step further, pointing out that on successful projects the "cone of uncertainty" converges as the project matures. The converse is also true. Wise words indeed.

The final chapter feels more like a tack on, however the message contained therein is something that needs to be stated again and again: marketing/management is not the enemy. It is important to remember that everyone has the same goals and that the battle really should be a collaboration. However good this chapter was, it still felt out of place.

There are a few niggling issues that I had. The biggest gripe is that he talks a lot about estimation software packages. In fact, he makes assumptions that the reader has knowledge of these packages. Working in start-ups, I've never even heard of these packages until this book. Its a small gripe, but it did detract. Another issue would be some of the examples on applying the various techniques towards the end of the book were far too glossy and far to dry. I think there was some good information there but you, as the reader, will need to make a few assumptions. Which, to me, is always a dangerous thing. Not as bad as fighting a land war in Asia, but still dangerous.

Overall though, as a software engineer/manager I found this book to be invaluable. The techniques are usable right away and really helped me convey the uncertainty I had in ways that I wasn't able to in the past. I think this should be required reading for anyone who works in the software industry.

[Gustavo Leiva · Rating 5 out of 5]

Great insights into estimation. Some of the highlights to call out in my opinion:
- Clear definitions and differentiation of: estimate, commitment, complexity assessment, schedule, confidence level, among others.
- The author brings up other topics that are usually forgotten when estimating a project, but definitely could impact it.
- The importance of being clear about the cone of uncertainty and how to use it when creating/communicating estimates and confidence levels.
- Clarity into when it could be a good time to add more people to a project without actually hurting it.
- Iterative vs sequential projects.
- Good references to other books to amplify other topics.

From the management standpoint I value that the topic calls out anti patterns that organizations fall in when requesting for an estimate.
The book definitely will give you a good tool set to look & create timelines and estimates from an unbiased stand point. And it also makes you understand how to connect the worlds of the people/stakeholders requesting the project and the teams actually estimating and executing.

My only feedback would be that it would be nice to have some more content on Agile projects. The topic is mentioned but there could be more about it. The book does recommend this other book:
https://www.amazon.com/Agile-Estimati...

[Erika RS · Rating 4 out of 5]

Estimation is difficult in software and everywhere else. In this book, McConnell provides the reader with tools to improve their estimation skills. There are no silver bullets -- most of these techniques take work and practice -- but by breaking down the process of estimation McConnell takes it from a black art to something understandable.

The first part of the book covers fundamental concepts in estimation. The heart of the book is part two which discusses many different estimation techniques, pros and cons of each, and times when a particular technique may be applicable. The final part of the book has further discussion of some other issues in estimating.

Although this book is specific to software, the core insights it contains are also useful outside of software. My husband and I are building a house, and as I read this I found myself thinking "if our builder had used some of these techniques, we might have had a more accurate budget and schedule".

This book has made it onto my short list of "must read" books for developers, especially those with no previous exposure to formal discussions of estimation.

[Michael · Rating 5 out of 5]

Another great book from Steve McConnell. Obviously, subject matter expertize and lots of historical data are key to good estimates, but this book is packed with real world examples and complete explanations when you don't have that. Highly recommended.

[Graeme Canivet · Rating 5 out of 5]

Solid principles that still apply today

Gets to the heart of real world software estimation, targets, commitments and project management challenges. Still relevant for any modern software project. Will definitely incorporate many of the tips on a daily basis.

[Daniel Godfrey · Rating 5 out of 5]

When an idea is answerable with software, a stakeholder seeks a commitment from software developers to create that answer, from which all involved can make plans. To make reasonable plans and commitments, it's good to take some time out to understand roughly what the scope of what the stakeholder wants is. This is estimation, and several different variables can be estimated, among them: size (how big will the final product be), effort (how much labor will go into the project), schedule (when the project will be finished), and cost (how much money is needed to fund the project). After introducing estimation terms, motivations, and principles, this book describes techniques for making estimates at various points in a project's lifecycle and communicating them to stakeholders.

I got this book a while back, not long after discovering the author's other book Code Complete, a treasure trove of information on how to write good code. During the pandemic, I saw charts the author was posting on social media and one of his videos about COVID and software estimation, which rekindled my interest in this topic.

Early in the book there's an illustrative test to demonstrate what 90% confidence is: The reader is asked to provide ranges for 10 random trivia questions. 90% confidence means feeling sure the answer for 9 of the questions falls in your ranges. It is a great example of uncertainty, what techniques we naturally employ to estimate (one of the 3 I got right I used an analogy for the estimate), and dealing with the psychological pressure to narrow uncertainty ranges.

Some other ideas from the book:

Best case, worst case, expected case. There's a limit on how good things can get. (The book mentions an Impossible Zone, where no matter how much effort you add, you cannot produce the software any more ahead of schedule.) There's less of a limit on how "bad" -- ideas for new features aren't bad, but they're not always part of the original estimate, commitment, or plan, and may warrant a re-estimate -- things can get. (Perhaps a dev's work, like a tailor's, is never done.) But neither extreme is necessarily likely. Thinking through what things would be like if everything went smoothly, then turning to thinking if everything went wrong, and then choosing a point somewhere in between as an estimate has helped me come up with very accurate estimates for effort.

Diseconomy of Scale. Linear relationships don't always hold the bigger a project gets. Bigger projects tend to require even more effort, take even more time, present even more risks (especially more defects, which have a vicious cycle of requiring extra effort to fix), etc., than a proportionately smaller project.

Cone of Uncertainty. In the planning-production-delivery lifecycle of software, uncertainty is high earlier but tapers as the project continues and nears completion.

Take some time out to think. Even an estimate made after 15 minutes of thinking about the problem will be better than an off-the-cuff estimate.

As you can probably tell statistics plays a major role in software estimation. (And one of my personal takeaways from the pandemic, one of my main motivations for reading this book, is just how important statistics is for decision-making in general.) I'm not a strong statistical thinker right now, but this book has been written for a wide audience and employs practical and simple statistical techniques and rules-of-thumb over complex equations or theories.

While this book is directed at estimating software projects, I think (with considerable translation) the same ideas could carry over into other domains. The author himself mentions times where he used similar techniques while writing Code Complete, a 1000-page book, using pages instead of KLOC (thousands of lines of code) and sections and chapters instead of feature points. The discussion in Chapter 15 of how the author reached a better approximation of how long that book would be using formal estimation techniques than his original "gut feel" estimate is especially enlightening.

The parts near the end about projects were tougher for me to read because they were a little out of my depth/experience, but it gave me a whole new appreciation for large-scale project management. For example, Chapter 19 talks about Space Shuttle software that was 20,000,000 lines of code, took 20,000 staff years, and had a failure probability of about 50% -- and that's just the software! Nevertheless, that project produced code at nearly the same efficiency of an operating system project, despite being about 10x larger than it in relevant areas. This example maybe also serves as a classic demonstration of the diminishing returns from Diseconomy of Scale.

The middle portion of the book (chapters 6-21) strikes me as good reference material to pull out when starting a new undertaking.

[Alec L · Rating 3 out of 5]

Although large chunks of this book are clearly 20 years out of date, the fundamental statistical ideas underlying it give some basic enduring insights. In particular, by recontextualizing the question of software estimation as a statistical problem (in the process thereby distinguishing it from the practical business problems of "when will this be done by?"), the book is then able to break the broader problem down into a raw statistical problem (where statistical techniques can be brought to bear) and the separate communication/coordination problem of making use of the estimate. The discussions of the latter are all targeted towards seemingly a very large corporate environment environment, but the discussions of the former note that one can gain significant improvements by using a couple of basic statistical techniques.

Separately, in relation to the actual practice making software estimates, there are perhaps two key insights contained in this book. The first is that one can think about estimates for the time of individual project components as independent random variables. The book does not *explicitly* give this model, but instead expounds the benefits of several things that come as corollaries. One one hand, when creating the overall project estimate, one can call on the law of large numbers to get a more accurate overall estimate. On the other, acknowledging that estimates are drawn from a *distribution* allow of to try and compute e.g. standard deviations of that distribution, and to think about confidence intervals for our estimates. The second key insight of this book is the issue of subjectivity in making estimates, and it espouses the use of extrapolating objective historical data---for example, instead of estimating time directly, it is generally much better to instead estimate the count of some measure of size (e.g. features, LOC; ideally some number >20 to increase the statistical fidelity) and then generate the time estimate by leveraging historic data to convert from size to time required. Overall, even if most of this book is no longer useful, there is clearly still a lot to be gained by really thinking through its general framing and these two estimation insights, and the book's generally easy-to-read nature means that it might just be worth going through cover-to-cover anyways.

[Adrian · Rating 3 out of 5]

A solid pick if you want to get some leverage the next time your manager has your back against the wall trying to make you commit to some project impossible to deliver in the preseneted timeframe.

You will get a good understanding on what estimation means , how are estimates different from commitments and targets. Once you realise this , your approach with non technical people from your organization will hopefully change radically

There's a lot of useful tips on topics such as estimation errors , planning , the cone of uncertainty and many tools and approaches on figuring out how in deep shit you.are when starting a new project with the given resources or how badly the ongoing one was planned.

This one gets 3 stars since i was hoping for some end to end scenarios like "you receive project X and you do Y,Z,T in this order". I was expecting a bit of handholding and more cohesion regarding the multiple tips and approaches you get familiar along the chapters.

[Ashkan Saeedi Mazdeh · Rating 5 out of 5]

I almost forgot how enjoying reading a good technical book can be before reading this. As always Steve McConnell did not disapoint. The book is about estimation so it cannot be a reading as smooth as Code Complete or professional sofware development but still it is very easy to read for an estimation book.
It has lots of good techniques and provides you with a great set of tools and the correct context for thinking correctly.

It differenciates between goals, commitments and estimates and not only teaches techniques but also how to deal with negotiations and how to present your estimates. Even if you are a small firm like us, the first and 3rd parts are extremely valuable and the second part is useful too. I highly recommend this if you need better estimates and who doesn't.

[Bibek · Rating 3 out of 5]

I read a handful of chapters in a reading group with coworkers to see if we could glean some insights for our own work. It was useful insofar as part 1 provided us with the vocabulary and mental frameworks for thinking about estimation in general. The meatier parts about how to actually go about estimating work felt less directly relevant. The programming languages and practices referred to were somewhat dated (understandable since the book was written 15 years ago) and the recommended techniques generally seemed more useful for large-scale organizations where the projects being estimated take on the order of many months to years to complete.

[Eric · Rating 4 out of 5]

Every software engineer should read at least the first ten chapters of this book. These are the most insightful chapters that talk about what estimation is for, what to be wary of, and some basic ways of estimating. The biggest, most broadly useful takeaways boil down to a few rules of thumb and a handful of equations, many of which are usefully collected in an Appendix.

The rest of the book is a broad survey of techniques, but many of them are likely only applicable in a big company that has a lot of record-keeping and is able to do things like staff 20 person projects for several years.

[Maxim Chetruşca · Rating 5 out of 5]

I would actually give this book 4.5 stars.

What I did not like:

- it's a bit old;
- too much publicity for the software estimation tool the author worked on.

What I did like:

+ it actually does its job - a list of gotchas, suggestions and rules of thumb which seem helpful. A seasoned developer will recognize most of the suggestions as thoughts which we all get at one point, then forget. Here you can find them all in one place;
+ really simple to read and easy-going;
+ no rocket science.

[Nathaniel Inman · Rating 3 out of 5]

First half of book could be called "Applied COCOMO 2", and the book largely presents most of its value to general SDLC practices stringing together a variety of good sources and connecting the dots with applicable examples. For me the largest value was the smallest chapter where he takes those results and shows how to present them to stakeholders for "negotiation". Very well cited sources and organization skills will allow this to be easily used as a reference manual later.

[Barrett Clark · Rating 2 out of 5]

I tried to hang with this book and finally gave up about halfway in. I appreciate the difficulty of estimation and the mathematical approach this describes. I disagree fundamentally with the premise that you can say Project B is similar to Project A, therefore we can use Project A as the base template and apply fudge factors from there. Project B is 12% more blue than Project A, therefore dot dot dot just doesn't make sense to me.

[Rubén · Rating 4 out of 5]

Gran parte del libro habla de métodos de estimación que pueden sonar un poco obsoletos a estas alturas (como estimar basándose en líneas de código), pero si dejas fuera esas partes, puedes encontrar mucha sabiduría. Estimar es siempre una tarea ingrata, pero me llevo unas cuantas ideas y buenas prácticas para sobrellevarlo mejor.

[Bennor · Rating 4 out of 5]

It took me an awfully long time to get through—it is about as dry as it sounds—but it was worth the effort. There's a ton of useful advice that I should be able to immediately put to practical use.

It was humbling to see how wrong I've been getting some aspects of estimation though.

[Artem Lytvynov · Rating 4 out of 5]

The task estimation always was a pain in my butt. This book is more for PMs and tech leads who estimate the whole projects from the beginning till the end, rather than individual development tasks. Still, I found some tips for devs, which will hopefully work out.

[Benji · Rating 4 out of 5]

Well-written, insightful and very well researched. Despite its age, most of this book is still completely relevant. I learned a lot about estimation and how it relates to targets, commitments, schedules and plans. Highly recommended for anyone who needs to estimate software projects.

[Steve Fenton · Rating 5 out of 5]

McConnell is always well researched and this is no exception. This book covers pretty much everything to do with estimation, including dysfunctions and abuses. A great handbook for estimation.

[Juan · Rating 5 out of 5]

This is a basic book for any software craftsman, essential after the initial steps of writing code profesionally.

[Peter Aronson · Rating 4 out of 5]

A solid, readable book on the various software estimation (time, effort, scope, cost, etc.), with a ton of useful definitions and rules of thumb and advice.

[Michał Węgrzyn · Rating 5 out of 5]

This should be handed to everyone who works in software development as a must read. I plan to read it many times over next years.

[Vander Alves · Rating 3 out of 5]

Accessible, comprehensive, still relevant, and a far more interesting read than one would expect from the subject—worthy to have on a handy bookshelf.