Recently, I met Kurt Häusler (Twitter, Blog) at the monthly meeting of the Agile User Group Rhein-Main in Wiesbaden. We had a short discussion around #NoEstimates, which brought up the idea of having a little interview via e-mail. You can read the results of this interview here. Kurt clears up many misconceptions about #NoEstimates in the following lines (Kurt blogged this here):
Fabian: "Hi Kurt, you tell me that it is a good thing to stop estimating? I'm sorry, but I cannot imagine how that could work out. How will I be able to calculate the costs for an upcoming project? As a business person, I must - at least - have a rough estimate, how expensive my project will be to know if it will pay off. How can you do this without estimation?"
Kurt: “Yes I think in general it is a good thing to stop estimating, there will always be situations where it makes sense, and of course it also depends on what we mean by estimating. For calculating costs estimation is especially bad. In fact the agile community had never even discussed estimation for this purpose. Estimation was generally safe as a tool for teams to use in planning how much work to take on each iteration and for providing rough forecasts about which features might be available in which release. For calculating costs there are a variety of approaches. Asking a team to sit down and guess how many developer hours each feature will require to type in and adding this up and multiplying these hours by various hourly rates is the poorest approach I have seen whether we are preparing an estimate for a T&M project or determining an actual price for a fixed price project.
My first comment would be that pricing an entire project at once is un-agile. It is at best hybrid, but that might be ok. Exactly how I would price the project depends on a lot of factors. How well does the customer understand about what he wants? Has the team done something similar before? Does the customer have a budget? How flexible is the customer and is he open to working in new ways or does he have certain constraints?
One simple approach (that might work in some cases but not others) would be to identify the most important feature than can be developed in one sprint. We know our team costs for a sprint so we can base the price for this “Version 0.1” or “Minimal Viable Product” on our costs. Then the customers question might be “how do I know in advance how many sprints I will need?” Then it gets interesting. I would perhaps then ask him something like “what are the minimal features we need to go live and start earning money?”. Perhaps something like the Incremental Funding Method could be an option."
Fabian: "OK. That explains a lot. I see many valuable approaches in what you say. The idea of delivering MVPs and approaching a project in small steps is nice. But as a customer I do not want to risk building one increment after another to determine that my money will not suffice to get the minimal marketable product ready when the money is already spent. So, I essentially want to know if there is a realistic chance to get my product built within my budget. How might that work with an MVP approach? Or am I and my project just not eligible for working agile? Is that what you are saying?"
Kurt: “The assumption I made, was that there was at least enough money available for a minimal viable product (One thing we like to offer customer’s is a special price for an MVP that can be completed in 1 sprint), and some extra available for further work in case it is not ready to go live (if the software development is only part of the product for example, we probably can’t go live until the rest of the product is ready anyway). It sounds like you already have a fairly detailed understanding about exactly what it is you want as well as a budget. In other words, fixed scope, with a fixed upper bound on budget. This is probably the easiest case. Without needing to get too detailed, we can compare our delivery rate of features with our pricing and see whether the budget easily covers it or not. We can probably do this with an MVP approach but it sounds like the difficult part is already done. You already know exactly what you want. (MVPs are usually used when the customer doesn’t know exactly what he wants, and wants to first test the market with an MVP and use the feedback to drive the vision of the product) In this case we can simply build it, one feature after another, and provide regular feedback about how much of the budget we will be using. This is a fairly typical approach, and definitely qualifies as agile software development. Radical people like me would consider it a hybrid situation, as the financial decisions and requirements gathering has already taken place for the whole project, and only the development itself will be carried out iteratively. So the whole “project” doesn’t sound agile, but the software development phase can certainly be carried out according to agile software development."
Fabian: “OK, I think I got that. You would just take all my stories, divide them by your average
sprint capacity and then - since you have a fixed size team - know how the costs would be. Correct?
But how do you know, that the stories, I will deliver are about the same size as the stories, you worked on in the past? If you simply take the number of stories, size should matter. Doesn't it?"
Kurt: “That is roughly right, yes. However I won’t pretend to be able to “know the costs”. The first step is to see if we are anywhere in the same ballpark. I would take your stories, and split them, which is a fairly mechanical process looking for certain keywords, steps in a workflow, or variations etc, and count those. The numbers I base my costs on are actually based on a larger group of 20 people that is usually split into 2-4 teams and works on 2-4 different things at once. So not quite sprint capacity, but close. I know over the last 12 months this group has cost roughly 130,000 Euro per month on average, and has delivered over the last 12 months roughly 180 items per month. So that means my costs are roughly 730 per item. We only need a small margin say 5% as everything else, like rent, hardware, training, maternity leave etc is already included in the 130,000 costs. That brings it up to about 760, which I would round up to 800 to provide a nice round number and cover the risk on my size if I have to commit to a price up front, say if you will only accept a fixed price offer. Oh and you don’t really need a fixed size team, you just need to know how much the typical team changes over a year affect costs. It is definitly better to have a fairly stable team though.
So we can take that 800 and multiply it by the number of stories, you have and compare it to your budget. I would recommend proceeding only if the price is significantly lower than the budget. If it is more, or comes close then the project might not be such a business success. Even if the expected price is slightly lower than the budget, I would still suggest looking for a better project, knowledge work should result in significant rather than scant returns.”
I fully expect that in any backlog the stories will vary in size. This size is not as important as many people think, once you consider the whole value stream, rather just just the development itself. I expect that the distribution of the true lead times of items I attempt in the next 12 months, and my throughput will be similar to the distribution of sizes of items I delivered and my throughput over the last 12 months. Plus every new project gives me new statistics that I can use to base the next months prices on (and one month of year old statistics drops out of the moving average). If your project turns out to result in longer lead times or a drop in throughput (whether that has anything to do with item size or some other factor), then my cost per item will go up slightly, and I will raise the prices for the next project. Possibly however another customer’s project has factors (maybe item size, maybe something else) that result in shorter lead times and an increase in throughput and they will cancel each other out."
Fabian: "OK, I see, there is a lot more thoughts in #NoEstimates than just denying to estimate ;-)
Your explanations sound reasonable. But they open up two further questions for me:
1.) If your first step is to break down stories at the beginning of a project to relatively small ones - will you not be very detailed before starting to implement? This sounds kind of like a classical project specification for me...
2.) Using your mechanics, you would implicitly reward customers, that formulate larger and larger stories over time which sounds not good for neither your understanding of the project nor the next customer, who will have to pay that bill? After all, the distribution of story sizes is not a random variable, but highly determined by your customer!"
Kurt: “With regards to question 1, my preferred approach is in fact NOT to break down all stories in a project to relatively small ones. My preferred approach is to simply pick the single most important feature, break that down if necessary, deliver it, and repeat. Even if we might know what the next most valuable features are I would ignore them for now and concentrate on the single most valuable feature. This is not too radical for agile software development or lean startup. This particular case is a bit different for these reasons: you already know exactly what you want, and have already completed the initial phases of the project in a waterfall manner, at least the requirements gathering seems to be complete. The other reason why we do this initial quick splitting is because you wanted a forecast upfront about how the expected costs of the whole project might compare with your budget. This is definitely a hybrid project but that is ok, most are.
You are right that a customer who is paying per feature might try and make them bigger to get more for his money, that is why we always split before committing together to beginning development of an item. But it could still be the case, that even after splitting one customer’s project’s items turn out to have shorter lead times or less impact on system throughput than another customer’s items. This doesn’t necessarily mean one customer is being rewarded over another. It means that customers are paying for the value they get rather than the effort we put in. Why should a customer pay more for a feature just because we have to do a bit more research than usual? Such extra investments don’t just benefit him but help the team in general, so this method spreads the costs. Why should a customer pay more for a feature just because an employee has gone on maternity leave and we had to hire another. Costs will go up, throughput will go down. This will change our cost per item, but no specific customer should be rewarded or penalised for this."
Fabian: "I think, I understand: Either I know exactly what I want to have (and am willing to invest heavily in analysis), then you would just break down stories to relatively small ones. Otherwise, you would suggest to proceed incrementally via MVPs (where you break down stories for the next increment only).
The activities in both cases seem relatively similar to what I would do when I would estimate a project (or a part of a project):
* Specify what will be delivered with the next milestone
* Break it down (to a certain degree), to see what has to be done
* Assign every item some specific effort
You just leave out the last item, which is putting a number on it, ain't you?
The amount of work for assigning efforts seems relatively low for me, if you already broke down the work. This brings me to my next question: What are the concrete advantages of not estimating? What problems are solved by not estimating?"
Kurt: “The first bit is roughly right, although I don’t think anyone should ever need to invest heavily in analysis. If you know what you want in advance then the analysis has been done. If you don’t then we can start to think in a lean startup way, and conduct experiments to test our market assumptions and generate new knowledge. I guess initial story splitting is a kind of analysis, but it is more like a simple mechanical process, looking for keywords etc.
And of course as usual, both extremes are rare, more typical is something in the middle, where the customer knows something about what he wants, but not all the details.
I like your list of three points. I would leave out the third point, but I would replace it with something like “assign a risk category”, or “assign a class of service” or “calculate the cost of delay”. Actually these things don’t need to be, and shouldn’t be, done all at once in the beginning. That is waterfall, or hybrid not agile. I think it is better to do such things as late as possible, just before the actual development.
You are right that the problem with estimates is not just that it takes time. It is unprofessional. It is like fortune-telling. It is simply guessing, and in several decades of software development we have learned that it doesn’t work. The numbers that is produces cannot be reliably used for anything, except perhaps sprint and release planning. Now we are starting to understand why it doesn’t work. You cannot reliably predict the effort involved in completing work in the complex or chaotic domain. Even if you could predict it, this number will not have any meaningful relationship to anything important. Now the agile community has talked a bit about estimating, and done a reasonable job with it so that it is useful in certain circumstances, done in a certain way. Using relative units like story points has helped scrum teams perform sprint and release planning, particularly when the variability of PBIs is not well understood. It is important that everyone understands the limitation of this approach. That these estimates are not to be understood by a customer to be a promise, they are not applicable when considering the wider value stream, they should not be used to base prices on etc.
No one in the agile community ever said that effort estimates are the correct approach to use for calculating costs in an offer or invoice. No one ever said that effort estimates be used to determine project delivery dates. Doing this has resulted in losing huge amounts of trust and money (either the customer’s or supplier’s).
So there are several problems solved by not estimating: Project failure in general, mistrust and disappointed customers, teams that feel disappointed when the actual lead time and delivery rate doesn’t match their guesses, and huge loss of money.
Imagine these slight exaggerated scenarios. Imagine a Scrum team developing software according to a backlog, and they need to know what they might achieve in a week. In this case the units of work (PBIs) are relatively large and few in number compared to the container (the sprint). It is like packing large stones in a small jar. If the stones have a variety of sizes then guessing at their rough relative sizes can help determine if we can pack in 5, 10, 15 or 20 stones in the jar. This is all the agile community has ever used estimation for, and in this specific context, it solves a specific problem.
If we consider the whole value stream, not just the development, and look at the departmental or business unit level, or even if we consider whole projects rather than individual sprints, then we aren’t packing large stones into small jars. The size of the units of work is relatively smaller compared to the container. It is more like filling larger jars with grains of sand. Not only filling the jars, but collecting them from the beach, cleaning the jars, filling them, packing them into a box and taking them to a post office. In this scenario would you bother trying to estimate the size of each grain of sand? No. You would use concrete measurements of how many boxes could be delivered to the post office over the last lear, and use that information to base future business decisions on."
Fabian: “Thanks Kurt! I think we covered many topics. And cleared up several misunderstandings about #NoEstimates. I really appreciated this dialog as I learned a lot!”
An sich selbst arbeiten ist hart. Und es funktioniert nicht ohne Feedback und Anregungen von anderen zu bekommen. Da gibt es Konferenzen, auf denen man immer wieder tolle Ideen aufschnappen kann. Oder Agile Coach Camps auf denen in lockerem Open Space Format im Prinzip alles möglich ist.
