Who estimates the stories in Scrum?

A few years ago we did a project with an outsourcing company. We gave them the current source code of a product, we gave them the backlog for the new version and we asked them to start implementing that backlog top to bottom.

Since the external team was new to both the product and the Scrum process, I decided to also do some rough estimates myself. Given my experience with both the product and the process, those estimates could serve as a rough baseline.

The progress

The remote team worked on the project for 5 sprints. During that time we saw an interesting trend.

As the team finished each sprint, some of their work was accepted and other work wasn't. Work was sometimes rejected because of its quality, but often it was rejected because it simply didn't do everything that the product owner had expected. In those cases the team put another story on the backlog for the missing parts and provided a fresh estimate for the "new" work. They also re-estimated the existing stories based on the newly gained insight of the previous sprint.

So over time the team's backlog grew in size. That is not uncommon, but look at the burnup chart:


burnup 1: based on team estimates

Based on this burnup chart, the project is in serious trouble. The team seems to be doing great, delivering more work with every sprint. But even though more and more work is delivered every sprint, the team is not getting closer to the goal.

The fifth sprint here is really disastrous: the team stopped increasing delivering more work. Did they burn out? How are we ever going to get the project out the door?

For comparison let's look at the data based on my estimates - that the team never saw and didn't commit to:


burnup 2: based on estimates by an external stakeholder

On my end, for every new story the team added to the backlog I checked if I had considered that part in my original estimate. If so, I split the story points I had originally estimated over the two stories. If the new story was functionality I had not expected in my estimate, I would provide a fresh estimate - also increasing the backlog in my part. As you can see that happened only once, but the increase was substantial (about 10%).

In my chart too the team seems to be burning out a bit in sprint 5. But it doesn't seem half as bad as in the firsts burnup chart. They are still getting closer to the goal.

And in both charts the team seems to be about 20% done with the total amount of work.

Analysis

So what's the difference between the two charts and estimators. From my perspective there are two major differences: the stability of the progress and who made the estimates.

How stable is the progress

The charts below show the velocity per sprint as derived from the burndown charts above:


velocity 1: velocity from sprint to sprint in burndown 1

velocity 2: velocity from sprint to sprint in burndown 2

The burnups don't really have a lot of data. But if you look at the first velocity chart, you can see that sprints 1 to 4 show a somewhat exponential growth in velocity (1.3x, 1.4x, 1.8x).

The scope/goal line in the corresponding burnup chart (burnup 1) shows a constant growth, mostly because I don't have the exact data anymore.

So at some point the two lines in burnup 1 are going to intersect, but it is pretty difficult to determine where they'll intersect with the naked eye.

The second burnup doesn't have this growth in velocity and the scope increase is about 10% over 5 sprints.

It is a lot easier to see where this project is going to end. And isn't scrum all about transparency and easy to use charts and data?

Who provides the estimates?

The second question I posed above is who provides these estimates. With the whole hype about lean development I learned that one way to optimize output is to eliminate waste. Anything that isn't delivered to the customer is waste and should be eliminated.

Who needs these estimates? The customer? I don't think my customer cares about estimates. He cares about getting as much software as possible for his/her money. In fact, while the team was providing these estimates they could also have been building more software.

So is it the team that needs these estimates then? After all without those estimates, how do they know what they can commit to? Well... the team does need to know how big a story is before they can commit to it. But they only need to know that at the start of each sprint. And only for the stories that they think they might do in that sprint. So although the team needs to know the size of each story it commits to, it doesn't need to know the size of all stories at the start of the project. Nor do they need to re-estimate the stories (another form of waste).

So the only person that actually needs those estimates, is the guy drawing the charts. In this case that was me, an external stakeholder who is not a part of the team. In many cases it will be the scrum master, who needs those estimates to give his stakeholders some view of the progress towards the overall goal. In other cases it will be the product owner, since he is most interested in seeing his return on investment.

Conclusion

My suggestion: let the guy who needs them come up with the numbers. And if you don't feel comfortable, do a reasonable guess. And if you don't even feel comfortable guessing, set all stories to the same size. In the end it doesn't really matter too much and you'll allow the team to focus on what really matters: building working software.

My first game of planning poker

Yesterday I took part in the first sprint planning meeting of my life. We have started up a new development project and we decided to use Scrum as the process. The project is actually quite small, so we have just two developers (myself included) and a product owner for it.

The product owner had prepared nicely and had a quite extensive product backlog. He had even filled in a "how to demo" field for a lot of the stories, which I'm not sure he's supposed to do before the sprint planning. At least it wasn't very handy to have the "how to demo" in place, as it makes it harder to discuss alternative solutions for the same functionality.

After the product owner had explained each story, we were to come up with an estimate of how much work (in ideal man days/story points) it would be to implement the story. I have done many of these estimation sessions before, but this time we decided to play a game of planning poker. Being the good scrum master that I am, I had brought two packs of (rather improvised) planning poker cards.

The other developer and I talked through the story, determining what it would take. We were basically already breaking the story down in tasks, which was a nice head start for the actual breaking down we planned to do later. After agreeing on the tasks, we would go into our poker deck and select the card matching our estimate. When we both had selected a card, we'd pull it out of the deck at the same time - revealing our estimate.

Now I must admit that I wasn't too impressed with the transparency that this estimating method brought. I guess -just as with real poker- you shouldn't play with just two players. There was actually only one story where we seemed to have a big difference in estimate: 8 vs 13 points. But as it turns out, our decks just didn't have any numbers in between 8 and 13. We had both wanted to select a 10, but since that wasn't there we just had to pick something slightly higher or lower. Being the planning pessimist that I am, I of course picked the 13. :-)

So there you have it: I played the game of planning poker. It wasn't anything special or extremely different from the ways I've done estimations before. But I guess that contrary to popular belief, being extremely different is not what Scrum is about. What is it about then, you ask? I'll let you know when I find out. Because if I answered that question now, I'd just be repeating the Scrum/Schwaber mantra.