Long Term Planning for Agile Projects

The other day my boss asked me about what do I think about how to do long term planning for an agile project. Of course I hve an opinion which I gladly share :-) I think agile software development makes planning much easier that traditional projects. Though the techniques are similar. The details here are for an IT environment although they can easily be varied for other software organizations. So here is my approach (beware it is looooong):

First start with a list of Features as defined by your customer that constitutes the project's scope. Next use the wide-band delphi estimation technique. The estimation team must consist of the customer(s) who requested the Features and at least 2 members of the agile team tasked with delivery (tech lead and team lead at a minimum) and at least one expert not on that team. The non-customer representation should be 5-7 in size not counting the moderator. I like to document the following for each Feature: description,assumptions, average estimate and standard deviation. Then depending on the risk deemed appropriate for the project determine the final number based on this:

A - Mean = 50% chance of actual work exceeding the estimate
B - Mean + 1 std dev = 16% chance of actual work exceeding the estimate
C - Mean + 2 std dev = 2% chance of actual work exceeding the estimate
D - Mean + 3 std dev = ~0% chance of actual work exceeding the estimate

I would never use A because I think this is gambling. I would never use D because I don't believe any estimate has that low a chance of being wrong. Of course all of this is based on statistics and we have all heard some variation on this quote:

There are three kinds of lies: lies, damn lies, and statistics. --Benjamin Disraeli

We are just putting confidence levels on our estimate. What might these numbers look like in reality... An estimate team of 7 estimates Feature X to take 4,5,5,6,6,7,9. Using B you get 8 and C you get 9. Some of you might be tempted to use decimal places...please do not. I am sure there is quote out there, but here is my general rule: let the level of precision of a number be based on its accuracy. Accuracy of an estimate is always low so please don't use decimals (precision) If you don't understand, please read this explanation of the difference between precision and accuracy.

So do you use 8 or 9 for Feature X? As I said above "depending on the risk deemed appropriate for the project ." This means that a high risk project should use the larger number. The riskier the project the safer the estimates need to be. A risky project usually has a high number of unknowns and a high degree of pressure. Unknowns makes estimation hard and pressure tends to make estimates low (and the project hard because of them). If you succumb to the pressure at the beginning of the project you are doomed.

Another pitfall can occur when the Features are too big. You should be keeps the estimates in single digits of person weeks. I doubt any estimate of week past this. What do you do when a Feature is too big? Help the customer break it into multiple Features. Repeat until all features come in as single digits. This tool is a mainstay of agile planning and will be needed for each sprint planning session so it is critical you understand how to do this. If you need help please read Mike Cohn's book "Agile Estimating and Planning" and use the abundance of experience in Yahoo Groups. I find these two to be the best in this area:
http://tech.groups.yahoo.com/group/extremeprogramming/
http://groups.yahoo.com/group/scrumdevelopment/

Next you get the customer's to rank them in priority. If there are multiple customers then you need a moderator who knows how to get multiple parties to agree and a simple technique or two to get them to put a number on each and every Feature. You will get at least one customer who says "I don't care, just get them all done" please be patience and ask them to participate in the ranking. Do not try and convince them of the importance, if they say something like this is an open forum they are pretty much not going (or probably even try) to understand in an open forum. So now you have a ranked list of Features with estimates.

At this point some of you may be wondering what unit of measure to use. I would go with person-weeks. Features should be big enough to take a week. Weeks are easy for people to grasp. Reasons not to use other units:

• Points can mean different things to different people and will usually confuse/frustrate your customer
• Hours are too fine grained and too precise for accurate estimates of large chunks of work
• Months are too long for most projects. I do not trust any estimate that is months away. If your project is huge then use two levels of estimates. Start with groups of Features...I like to call them a Capability. One by one split these Capabilities into Features and use the technique here. Then roll the features up by Capability (Excel excels at this ;-) to get a complete picture.

So what do you do with this list. This list feeds into the agile team's work. Each sprint they take enough Features off the top of the list to ensure complete sprint coverage (meaning the total person weeks exceeds the person weeks of your sprint). So if you have a team of 7 and you use a 1 week sprint length (my recommendation) then you need at least 7 person weeks of work to start your agile planning session. Out Feature X had 9 so that one Feature would be enough. The amount over you current capacity is determined by your team's performance. If the team has no history then estimate double the capacity. You might plan more than you do, but it is always OK to have extra stories on the backlog than to finish a Feature and have no work planned out.

Now you need to add a timeline to your list of Features. We will not put dates on each Feature. Instead we will draw a line with the Y axis as Feature person-weeks and the X axis as sprints. The slope is the bur rate we expect from our team. With a team of 7 and a sprint length of one week do not expect 7 person-weeks/sprint. This presumes 100% effeciency. Remember there are planning eets, demos, doctor appointments and other non-project work. If you have historical data then use that to inform you. If you do not the 50% on task is probably a good starting point. Our fictional project has 10 features with sizes 2,5,4,4,4,7,2,5,5,2 (total 40 person weeks). That menas we should be done in the middle of sprint 12 (40/3.5 = 11.4). Remember we are using estimates for this prediction. Instead of drawing one line I would draw two - pper and lower limit, and reality should fall in betweem (give your customer the upper limit). List use a burn rate of 3 and 4 for these two lines (if you are really into numbers you can use +- some percentage..say 15?). That gives us a lower limit of 10 (40/4) and an upper limit of 14 (40/3). So tell your management 10-14 sprints and use 14 as the end date. Now the sprints can start.

There are several ways to do Agile planning within sprints. The key for me is how you break up the work. One way to break this work up can be seen here: http://www.mountaingoatsoftware.com/sprint_backlog. I recommend slicing the Feature into Feature Slices. Feature Slices are stories that slice the Feature end to end. Using slices means each story is a mini-feature so each one would be recognizable from the customer's point of view and would have distinguishable value. This allows the customer to prioritize the work in a meaningful way. The link above is developer focused and helps the team's natural inclination to silo and sub-optimize.

An example will probably help. Lets say the Feature we are working on is a shopping cart (forgive any ignorant assumptions...I have never coded a shopping cart) Lets assume we already have a way to display products. One way to slice this is:

• Select one item from the catalogue and add it to the cart
• Display an item previously selected
• Add multiple items to the cart
• Sort items in the cart by product name
• Allow the customer to select the sort order (name, price, date selected)
• Sort items in the cart by Price
• I think you get the idea

Each one of these has a different value to the customer. He can order them as he wants. He can also draw a line and say don't show me the shopping cart until it can do these things. This puts the power in the right hands. Us technical folks like to have power over the order, but we don't own the work...we just do it! These should still be stories...meaning they should take no more than 2 days to do (ideally take 8 ideal hours or else) and completion is defined by executable tests. If the team can't promise that for the above list then add more differences that will split the work. For example, think of the story w/o the database...it would not have persistence. Then add persistence as a separate story. Think of why the story takes more than 2 days and split up the chunks in a way that the customer cares. For the above example the customer cares about persistence...think about the usefulness of a stateless shopping cart...not appealing :-)

That's about all the techniques you need. The rest is presentation. I am lazy so lets do an example. Our team has 7 members as is following a one week sprint. Here is their Sprint Burndown chart (Bar height shows work left, The line represents expected burndown; used to determine estimated completion time):

Notice that the backlog is in the person-weeks from the original estimate not in actuals (much like earned-value). Many people use actuals and thus the backlog grows and shrinks each sprint and this fluctuation will seem to the uninformed to be an ever changing target and some sort of shell game. In case it isn't obvious you start with the total estimated person-weeks for the all features. Each sprint in which you complete a feature (pass all feature tests) you subtract the estimate off the backlog.

The factors that will shape this chart are the quality of the original estimates, the quality of the story estimates each sprint and the productivity during each sprint. Usually, but not always, the factors even out. The key is to look at the numbers each sprint and understand the variance. You can show you management all the data, but I recommend only showing the customer the burndown. You can use the other data for the IT management so they can understand the factors (and it gives them metrics that tend to make upper management happy :-)

Data you should keep track of:

• Person-weeks per Feature (estimates)
• Points per feature (estimated in the sprint planning)
• Sprint each feature was shown complete (actual)
• Points finished each sprint (actual)
• Total Points required to complete each feature (actual)

With this data you can show:

• How good your original estimates were and even show limits and track this as the sprints complete and re-plan if the estimates are consistently tracking out of limit.
• How good your sprint planning estimates are
• How many points done each sprint (this should start low, track up after 2-4 weeks then track down near the end as the project winds down...yes the S curve did not die) I would not do much with this chart unless it never rose, trended down over several sprints or if there was a drastic change

How to do the above I leave as an exercise for the reader. What might some examples scenarios look like? Lets look at three scenarios:

1. Team is new and runs into problems they could not foresee (Dark Matter), will miss the 14 sprint end date given to customer.
2. Customer keeps adding scope each and every sprint, and the work will push past 14 sprints.
3. Feature estimates were much too low and the team can not be done in 14 sprints.

Scenario 1
I will use the following for Feature/story points (6,17,14,11,14,18,11,18,18,8). You can track along in Excel if you dare ;-)

The team has trouble with the technology, each other and agile software development techniques. The stories completed for the first 4 sprints are 5,6,6,6. They complete feature 1 in sprint 2 and feature 2 in sprint 4. Given the performance so far the team would need to average 13 points/sprint to finish by sprint 14. At the current rate it looks like it will take 23 sprints to finish. Things do not look good.

The team should talk about whether or not they think they can double their outcome. It should be obvious to the team lead that she needs to sit down with the customer and her management to determine a course of action. Upper management should be able to tell from the numbers that this talk is needed. I would not recommend they have the talk after the first 3 sprints. The first three sprints are typically lower than the rest. They may decide to let the team try to self correct for the next 2-3 sprints.

The team completes 7 points the next two sprints and complete feature 3. This is the best they have done so far, but it is not good enough to get the scope done by sprint 14 (more likely they will take 20). Management has the options of hiring a consultant and adding a senior person or two. The customer will need to be told that there is a risk to delivery and she needs to start thinking about potentially lengthening the time line or cutting scope.

Scenario 2
I will use the following for Feature/story points (9,22,18,22,22,33,11,27,29,13). Notice these numbers are higher than scenario 1. The team is experience with agile and the technology. The fist three sprints the team completes 9,13,12 points and features #1 & #2. Their backlog is at 33 and the estimated finish is sprint 18 with a current average of 2.3 person-weeks/sprint. To finish by sprint 14 the team will have to up their average to 3. The team talks about this at their weekly retrospective and think they can do it.

The team works most nights in print 4 and does 18 points to complete feature #3 worth 4 person-weeks. Things are looking up, but the next sprint their stories are down to 11 because they are tired. Sprint 6 is back to 15, but now the picture is starting to be clearer. They are expected to finish in sprint 16 and to finish by 14 they need to up their output to 3.1. The team wises up in the retrospective for sprint 6 and starts asking about why the work isn’t lining up with the estimates. They do a ratio of points per person-week by feature. It is averaging around 4.7 and tracking up. They ask the outside expert that helped them come up with the feature estimates and he notices that several assumptions made in the planning session are being violated by the stories. The customer is asking for more that was agreed. The team lead needs to sit down with the customer and the her management to work out a solution. Something has to give.

Scenario 3
Will follow the same path as scenario 2, but when the retrospective digs into the reason for the mismatch with the estimate it is becomes obvious that the original estimates were much too optimistic. The estimates did not take into account many of the realities that the team now knows. The team lead needs to sit down with the stakeholders, but the discussion should be about two things. What do we do to satisfy the customer and what do we do about improving the estimation process for next time.

For those brave enough to stay with me I hope this was coherent.