Monday, March 26, 2007

Costing A Project Using Scrum and XP

At the last EAMUG meeting Mike Boroditsky came up with some good questions about doing an initial costing of a project. Let's say a potential customer comes along and asks you how much she would have to pay for a piece of software she has in mind. How does one go about providing such an estimate in an agile context? If agile software is about iterations and feedback, is there any way to give a customer an up-font estimate? The idea in XP is that every project balances 4 principal variables: Time, Cost, Scope, and Quality. XP fixes the time, cost, and quality. Therefore only scope can be varied during the project. Based on an initial estimate, the developer will commit to a particular schedule, but both the developer and customer must be flexible with respect to scope - the nature of the work that ends up being done. See here for a nice elaboration. I think the reality in agile projects is that the time dimension can also stretch as long as the customer is involved in managing the schedule.

Here's how it might work in context of Scrum/XP: The project starts up with one (or perhaps several) initial meetings. In these meetings the developer and the customer get together and build up a product backlog. The product backlog is essentially a list of features sorted in order of descending priority. During this process, the developer and customer work together to establish a shared understanding of what the customer is trying to accomplish. Once an initial product backlog has been created, the developer goes over each item in the backlog and provides an estimate. Since each item in the product backlog may expand into multiple stories when it's tackled as part of a sprint, such estimates as not as reliable as etimates for stories. Nevertheless, one should be able to generate reasonable estimates based on how long similar features took on past projects and on a sense of the complexity of given items. Once these estimates are added up, one can tally them up and provide a total time estimate for the
customer. These estimates will be tested against reality in the very first iteration; the burndown chart will make progress clearly visible to all. If progress is totally off-pace from the initial estimate, then that triggers an immediate signal that something is wrong.

How about fudge factors? I've heard people say they expect some estimates to be too high and others to be too low so that the average should work out to be about right. My experience suggests that estimates are far more likely to be too low than too high. Therefore it may be a good idea to apply a fudge factor to the project. In an agile context where everything is out in the open, it would be important to share this information with the customer. The developer may say to the customer, "the estimates in the product backlog suggest this project will take 3 months to develop but it's very easy for such estimates to be off by as much as a factor of 2. Let's budget for up to 6 months." Note however that the estimates are not being inflated - the budget is being adjusted as a matter of risk management but the estimates are not being changed at all.

Watching the burndown chart (which is a visual representation of velocity), one can quickly develop a sense of whether the project is on track. After each iteration, the customer can re-prioritize, add, and remove items. All changes are put back into the product backlog and the developer re-estimates all the items after every iteration. The "red line" allows the customer to clearly see what low priority items are falling out of scope. In an agile environment, the customer is not expected to be able to come up with perfect requirements up front. The customer is allowed to learn more about what she wants as the project goes on and as she is able to interact with the actual working application. The developer is also not expected to understand the requirements perfectly. It is assumed that the developer will misunderstand some requirements such that the application will need to be refined. It's a process of give and take - but again, the overall success of each iteration is assessed by the customer.

What about quality of estimates? The more familiar a developer is with a given type of requirement - the domain as well as the technology - and with a particular client, the better the estimate. If you've worked with a client before on a similar project, your estimates are going to be much more accurate. The higher the estimate, the more likely it is to be innacurate. If you think something will take 1 day, it may take a bit longer. If you think something will take a month, it becomes difficult to understand where that number is coming from. Therefore I suggest that the largest estimate for any given item on a product backlog be one week. If you think something will take more than a week, then you should find a way to break that requirement up into smaller pieces. It's important to realize anything that will require research is highly uncertain and should be approached with care.

I think the value of this approach is that it lets the customer see progress very clearly. After every iteration, the customer gets new features as part of a working application. Also, if the customer is unhappy with the progress being made, every iteration is an opportunity to change things, or in the extreme case to cancel the project before it becomes too expensive. Importantly. the developer does have the tools to provide an up-front estimate of time and money for a project.


Anonymous said...

That is an amazingly well-written article.
Need to find number of iterations upfront to estimate total budget needed. Nobody can start a project without knowing the total budget(unless it is a defense project ;)Isn't that bottom-up estimation in a way and would need a WBS upfront?

Like you said, the customer can see with the first five iterations whether the whole estimate needs tweaking.

Anonymous said...

"XP fixes the time, cost, and quality. Therefore only scope can be varied during the project."
Could you kindly say from you got that info from?

Vladimir Levin said...

I could be wrong, but I think this "cost time and quality" is from the classic "XP Explained" book. It's a bit hard to remember now...

Vladimir Levin said...

Vis. WBS (Work Breakdown Structure), I am not a fan of this kind of more traditional approach. It proposes to understand in detail what needs to be done, and how. The reality of software, based on my own experience and everyone I have every spoken to, is change. That doesn't mean that no elaboration is possible at the early stages of creating a broad estimate for a project, but one must be very circumspect about it. I prefer to create a first pass estimate that is them continually refined as real work begins. Of course if one is obligated to produce a WBS
or something similar, then I guess one must decide whether to try to do so while maintaining some semblance of agility or whether to pass on the work altogether and move on to something else.

Angelina ray said...
This comment has been removed by a blog administrator.
peeter pattinson said...
This comment has been removed by a blog administrator.