Wide-band delphi estimation for Business Value - 2

See Part 1. ***

Step by step

Let me walk through how I do it.

1. We have a product backlog.  Usually user stories.
2. We identify the right people to be the business value experts. We recommend that they participated in developing the PB.  Typically 5 people.  I call them the business stakeholders (maybe not what your company means by that word...just the key people to do this and participate in the Sprint Review) and the Product Owner.  The 5 best experts. (Sometimes you have 4 people or 6 people.)
3. We discuss BV drivers.  To make the concept of 'business value' more concrete in this specific situation.
4. We choose the highest BV card.  We give it 100 BVPs (BV points).  It becomes the reference story for BV.
5. We select cards randomly.  And compare them to the reference story.  The experts vote on BV using the Fibonacci cards.  I recommend the full true Fibonacci series up to 144.  If you want to add some cards at the high end (some, not many), I think it will not hurt.
6. They discuss and then 'secretly' vote. 1-2-3 show.  Everyone shows his poker card.  If the cards are all with 3 Fibonacci numbers of each other (eg, 21-34-55), then sum the cards and average to the nearest integer.
7. If they are not close, then discuss (especially the highest and lowest voters), enough so that the 5 voters learn something.  Then re-vote.
8. On average, allow about 1 minute per card.  Maybe slightly more. They will discuss the first few cards more slowly than that, and the last half of the story cards will go more quickly, on average.  Do not let them get take more then 75 minutes for 50 use stories.  Go fairly quickly.
9. When you are done as a team, then arrange the cards of similar BVPs.  And you ask the experts...do you see any cards that now look 'out of kilter?'  Often they will identify a few user stories to be re-voted. And then they do that.

That's the basics.  Pretty simple.

Results

I have never seen any set of experts vote all the user stories or even most of the user stories as 100 BVP (on every one).  In fact, they always use a roughly even distribution from 100 to about 5 or maybe 2 BVPs, across the set of stories.  This is quite remarkable, when you think of what we used to do.

Comments

I recommend that the BV experts think mainly in terms of 'wow factor.'  That is, BV is what will make someone (eg, the customers) say 'wow.'  Or say wow more loudly.  the louder the wow, the higher the number.  The experts often talk about dependencies (of one type or another) and want to deduce that the one card on which many other stories are dependent -- that card must therefore have the highest BV.  I discourage that way of thinking.  I recommend handling dependency issues a different way. If they need a decision about what a story is (or is not), the PO is the final decision-maker.  We hope, of course, that the PO uses common sense. We recommend you use a ScrumMaster (or someone like that) to facilitate the meeting.  Keep them at a good pace.  Let the quiet ones talk more, get the talkative ones a bit less verbose. I like to let the implementers listen.  They learn things. And I like to let the implementers, occasionally, ask questions.  "Why is that story so important?"  "I thought that story would be more important...can you explain?"  And the experts take a minute or two to answer.  But, we cannot distract the experts too much. The voting is very interesting to watch. Bring popcorn.  You could even sell tickets.  The experts will disagree with each other, and educate each other, and learn things about business value.  And they will complain that it is hard to estimate the future.  The implementers usually find it amusing when they say that, because these are the same 'jerks' who 2 weeks ago 'beat up' the implementers for not being able to estimate effort more accurately.  They start to respect each other more. The implementers learn a lot about business value, as they watch. This changes their motivation and it will change their behavior when they build the product. Are the experts really expert?  In my opinion, we can say that they are the best people we can find to make these guesses.  But they are really guessing what the customers will want when we deliver the product and for the 2-5 years after that.  No one is really an expert about the future. Still, business has always been risky.  About all real business questions, we get the best people to take their best guesses, and we hope that will give us more success than our competition.