Wide-band delphi estimation for Business Value - 2

See Part 1. ***

Step by step

Let me walk through how I do it.

1. We have a product backlog.  Usually user stories.
2. We identify the right people to be the business value experts. We recommend that they participated in developing the PB.  Typically 5 people.  I call them the business stakeholders (maybe not what your company means by that word...just the key people to do this and participate in the Sprint Review) and the Product Owner.  The 5 best experts. (Sometimes you have 4 people or 6 people.)
3. We discuss BV drivers.  To make the concept of 'business value' more concrete in this specific situation.
4. We choose the highest BV card.  We give it 100 BVPs (BV points).  It becomes the reference story for BV.
5. We select cards randomly.  And compare them to the reference story.  The experts vote on BV using the Fibonacci cards.  I recommend the full true Fibonacci series up to 144.  If you want to add some cards at the high end (some, not many), I think it will not hurt.
6. They discuss and then 'secretly' vote. 1-2-3 show.  Everyone shows his poker card.  If the cards are all with 3 Fibonacci numbers of each other (eg, 21-34-55), then sum the cards and average to the nearest integer.
7. If they are not close, then discuss (especially the highest and lowest voters), enough so that the 5 voters learn something.  Then re-vote.
8. On average, allow about 1 minute per card.  Maybe slightly more. They will discuss the first few cards more slowly than that, and the last half of the story cards will go more quickly, on average.  Do not let them get take more then 75 minutes for 50 use stories.  Go fairly quickly.
9. When you are done as a team, then arrange the cards of similar BVPs.  And you ask the experts...do you see any cards that now look 'out of kilter?'  Often they will identify a few user stories to be re-voted. And then they do that.

That's the basics.  Pretty simple.

Results

I have never seen any set of experts vote all the user stories or even most of the user stories as 100 BVP (on every one).  In fact, they always use a roughly even distribution from 100 to about 5 or maybe 2 BVPs, across the set of stories.  This is quite remarkable, when you think of what we used to do.

Comments

I recommend that the BV experts think mainly in terms of 'wow factor.'  That is, BV is what will make someone (eg, the customers) say 'wow.'  Or say wow more loudly.  the louder the wow, the higher the number.  The experts often talk about dependencies (of one type or another) and want to deduce that the one card on which many other stories are dependent -- that card must therefore have the highest BV.  I discourage that way of thinking.  I recommend handling dependency issues a different way. If they need a decision about what a story is (or is not), the PO is the final decision-maker.  We hope, of course, that the PO uses common sense. We recommend you use a ScrumMaster (or someone like that) to facilitate the meeting.  Keep them at a good pace.  Let the quiet ones talk more, get the talkative ones a bit less verbose. I like to let the implementers listen.  They learn things. And I like to let the implementers, occasionally, ask questions.  "Why is that story so important?"  "I thought that story would be more important...can you explain?"  And the experts take a minute or two to answer.  But, we cannot distract the experts too much. The voting is very interesting to watch. Bring popcorn.  You could even sell tickets.  The experts will disagree with each other, and educate each other, and learn things about business value.  And they will complain that it is hard to estimate the future.  The implementers usually find it amusing when they say that, because these are the same 'jerks' who 2 weeks ago 'beat up' the implementers for not being able to estimate effort more accurately.  They start to respect each other more. The implementers learn a lot about business value, as they watch. This changes their motivation and it will change their behavior when they build the product. Are the experts really expert?  In my opinion, we can say that they are the best people we can find to make these guesses.  But they are really guessing what the customers will want when we deliver the product and for the 2-5 years after that.  No one is really an expert about the future. Still, business has always been risky.  About all real business questions, we get the best people to take their best guesses, and we hope that will give us more success than our competition.

Wide-band delphi estimation for Business Value - 1

I and many many teams have been doing wide-band delphi estimation of business value for several years now.  Let me report how we do it, which seems to be successful.  In fact, as far as we can tell, it is more successful than any other approach.

What is it?

You may want to look up Wide-band delphi estimation in Wikipedia. And I recommend you read James Greening's article on Wide-band Delphi estimation for effort. It is, most essentially, expert estimation.  It is said to have been invented by Rand Corporation in the 1950's, and they did do a lot of research then. In my opinion, it must be older. The ancient Greeks talked about the oracle of Dephi millenia ago. So, in this case, we mean gathering, for a given set of work, your best experts on business value in that area.  I recommend about 5 people (5 'experts').  And, as with the well-known Planning Poker (cf Mike Cohn), we use the Fibonacci cards, and have then estimate relative business value.  And that is done after they pick the highest business value card (user story, Product Backlog Item, etc), and the estimates (guesses, opinions) are made in comparison to that highest one.  That highest one become the 'reference story' and it is arbitrarily given 100 BVP.  (You could give it another high number.  100 BVP seems to work well.) We will explain more about the basics, but that's the main idea.  Expert estimation. I should note that Mike Cohn asked me to call this 'priority poker' to distinguish it from 'planning poker' for effort.  I like that idea.

Why?

There might be many answers to this question.  Here is why I think this technique is valuable. First, I think the number (the Business Value Points assigned to each card, or 'the guess') will be very valuable.  In several ways. But, in my opinion, the most valuable thing is that it drives more useful conversations and learning. It is my opinion that the key thing that knowledge workers do is learn, or, you might say, create knowledge.  And that happens best in a small team, where diverse perspectives enable them to 'fight', disagree, examine assumptions and alternate perspectives, and then start to agree, or at least, average their opinions. And the exercises of priority poker enables them (and those watching them) to: (a) see the same elephant (b) become more motivated (see they see better where the juice is) (c) share the tacit knowledge We will discuss the usefulness of priority poker more later.

Will it work for you?

Yes. There really is nothing more to say. It is fair to say that before they have done it, you or they will wonder "will it work for me?"  But after they have done it, there is never any question. I think I have heard accurately that if it is introduced to senior people (of some outspokenness, we will call it) by a person who does not have confidence, then the process can get derailed.  But at least I can honestly say that no team that I have worked with has ever refused to do it, or found it un-useful.  They have found, either during or after the exercise, that 'I wish that [George] had been here."  But that's a different kind of problem. So, because I have done this so often with so many different kinds of teams and situations, I am confident in saying 'yes.' Am I right?  Well, of course, almost surely, as with Newton's Laws of Physics, I am wrong to some degree. Surely out there in the wide world, there is at least one place it will not work.  So, use with some caution. *** See Part 2.  

Executive Action Team

Jeff Sutherland has defined an Executive Action Team.  This is similar to the Management Scrum idea that I discussed. You will also notice that Ken Schwaber and Mike Cohn have described roughly similar ideas to what Jeff is describing.  (I will look up later the names they gave to those teams.) Jeff Sutherland's idea is similar to what I (and some others) call an IRT.  An Impediment Removal Team.  That is, a higher level team that looks at all the impediments and works on the 'best' ones.  Best is either highest (best bang for the buck?) or the ones that IRT are best at fixing. For Jeff's discussion of the Executive Action Team, see the video and the slides here.  It is part 3 of his Scrum at Scale series.  You will notice that in addition to working on 'impediments', it implies that they work on 'transition issues' (as they are aka as a 'transition team'). Enjoy!