Who estimates the stories in Scrum?

A few years ago we did a project with an outsourcing company. We gave them the current source code of a product, we gave them the backlog for the new version and we asked them to start implementing that backlog top to bottom.

Since the external team was new to both the product and the Scrum process, I decided to also do some rough estimates myself. Given my experience with both the product and the process, those estimates could serve as a rough baseline.

The progress

The remote team worked on the project for 5 sprints. During that time we saw an interesting trend.

As the team finished each sprint, some of their work was accepted and other work wasn't. Work was sometimes rejected because of its quality, but often it was rejected because it simply didn't do everything that the product owner had expected. In those cases the team put another story on the backlog for the missing parts and provided a fresh estimate for the "new" work. They also re-estimated the existing stories based on the newly gained insight of the previous sprint.

So over time the team's backlog grew in size. That is not uncommon, but look at the burnup chart:


burnup 1: based on team estimates

Based on this burnup chart, the project is in serious trouble. The team seems to be doing great, delivering more work with every sprint. But even though more and more work is delivered every sprint, the team is not getting closer to the goal.

The fifth sprint here is really disastrous: the team stopped increasing delivering more work. Did they burn out? How are we ever going to get the project out the door?

For comparison let's look at the data based on my estimates - that the team never saw and didn't commit to:


burnup 2: based on estimates by an external stakeholder

On my end, for every new story the team added to the backlog I checked if I had considered that part in my original estimate. If so, I split the story points I had originally estimated over the two stories. If the new story was functionality I had not expected in my estimate, I would provide a fresh estimate - also increasing the backlog in my part. As you can see that happened only once, but the increase was substantial (about 10%).

In my chart too the team seems to be burning out a bit in sprint 5. But it doesn't seem half as bad as in the firsts burnup chart. They are still getting closer to the goal.

And in both charts the team seems to be about 20% done with the total amount of work.

Analysis

So what's the difference between the two charts and estimators. From my perspective there are two major differences: the stability of the progress and who made the estimates.

How stable is the progress

The charts below show the velocity per sprint as derived from the burndown charts above:


velocity 1: velocity from sprint to sprint in burndown 1

velocity 2: velocity from sprint to sprint in burndown 2

The burnups don't really have a lot of data. But if you look at the first velocity chart, you can see that sprints 1 to 4 show a somewhat exponential growth in velocity (1.3x, 1.4x, 1.8x).

The scope/goal line in the corresponding burnup chart (burnup 1) shows a constant growth, mostly because I don't have the exact data anymore.

So at some point the two lines in burnup 1 are going to intersect, but it is pretty difficult to determine where they'll intersect with the naked eye.

The second burnup doesn't have this growth in velocity and the scope increase is about 10% over 5 sprints.

It is a lot easier to see where this project is going to end. And isn't scrum all about transparency and easy to use charts and data?

Who provides the estimates?

The second question I posed above is who provides these estimates. With the whole hype about lean development I learned that one way to optimize output is to eliminate waste. Anything that isn't delivered to the customer is waste and should be eliminated.

Who needs these estimates? The customer? I don't think my customer cares about estimates. He cares about getting as much software as possible for his/her money. In fact, while the team was providing these estimates they could also have been building more software.

So is it the team that needs these estimates then? After all without those estimates, how do they know what they can commit to? Well... the team does need to know how big a story is before they can commit to it. But they only need to know that at the start of each sprint. And only for the stories that they think they might do in that sprint. So although the team needs to know the size of each story it commits to, it doesn't need to know the size of all stories at the start of the project. Nor do they need to re-estimate the stories (another form of waste).

So the only person that actually needs those estimates, is the guy drawing the charts. In this case that was me, an external stakeholder who is not a part of the team. In many cases it will be the scrum master, who needs those estimates to give his stakeholders some view of the progress towards the overall goal. In other cases it will be the product owner, since he is most interested in seeing his return on investment.

Conclusion

My suggestion: let the guy who needs them come up with the numbers. And if you don't feel comfortable, do a reasonable guess. And if you don't even feel comfortable guessing, set all stories to the same size. In the end it doesn't really matter too much and you'll allow the team to focus on what really matters: building working software.