I went to the Agile Ottawa group last night. Mostly I very much enjoyed the meeting. A lot of smart people, a lot of good discussions. But...
A Sad Tale
A gentleman there, a serious business guy, new to Scrum, said something like this: "It seems to me that Scrum has no up-front planning. And for those of us in the business world who must give some up-front estimate of time and dollars, this is not going to work for us. Does Scrum have some up-front planning?"
Several people answered. And I gave an answer: YES it does have up-front 'Release Planning.' Scrum does normally include up-front planning. This is somewhat controversial in Agile, but we have it. It is called Release Planning.
And I went on to say some of the things I say below. But my comments were brief, and it seemed to me they were drowned out by Agilists saying "why do you want to do up-front planning?" (and similar comments)...which is a good question, but again, it likely implied to the gentleman that all we 'scrummers' hate up-front planning. Perhaps I am wrong, but it seemed to me that he heard 'Scrum does not do much Release Planning'.
I discussed this with a person in my CSM class the next day. His comment was "yes, based on the stuff I have read [and he had read several things] Scrum does not say much about the up-front planning. I can totally see how that person had that impression."
To me this is very unfortunate.
[Aside: I call this a sad tale, with my tongue somewhat in my cheek. These are relatively normal human misunderstandings. Perhaps a bit sad, but nothing to compare to the sadness in Libya right now.]
Let me now make two references.
In the Scrum Guide (located here, among other places), Release Planning and a Release Planning Meeting are specifically mentioned and discussed.
In "Agile Project Management with Scrum" by Ken Schwaber there are only two mentions of Release Planning (according to my Kindle) in the book. Both not favorable (really about bad release planning). So, one can see why some people might get a wrong impression from that book. And some other sources as well.
What's Wrong With Up-Front Planning
Now, let me guess why Ken Schwaber minimized the mention of Release Planning in the book mentioned above. First, Schwaber and Sutherland note in the Scrum Guide that a lot of the details about how to do Release Planning are outside the purview of Scrum. Meaning: Scrum is a framework only, and as such, it is probably not useful or necessary for Scrum to give guidance on this that could be seen as Scrum's "answer" for everyone. Starting to do that risks making Scrum too big to implement in one step, relatively quickly. And risks giving great advice (for one effort) inappropriately to another effort or situation.
To me, these are quite reasonable concerns, and I think it stretches the point, though, to be so quiet about Release Planning more generally. And, indeed, the Scrum Guide, short as it is, does give a fair amount of guidance on Release Planning.
Another hypothesis I have is that Schwaber was concerned at the time he wrote that book that he was seeing too much Release Planning in 'scrum' implementations that looked too much like waterfall release planning. (I think I have seen this some too, although perhaps not as much.)
So, what's wrong with that?
Well, first, the customer sees no value in Release Planning per se. It is, to use lean terms, at most a 'necessary' waste. So, we want it to be as short as possible (but not shorter).
Second, too often people think the main (only) value from the initial release plan is the initial date and budget. Usually, as indicated above, that initial date and budget are necessary information to make some initial basic decisions. Maybe even accurate information if there were to be no changes. But anyone with much experience with projects has a near 100% experience of changes to projects after the initial Release Planning. Usually quite a number of changes and quite significant. In which case, the initial date and budget quickly become moot (in terms of accuracy; possibly still "directionally" useful, if you have a firm that thinks in those terms).
Third, often the initial Release Planning date and budget are used to drive people on Death Marches. This is an horrendous and utterly scandalous thing in our industry. It is shameful that we let stupid managers use this information this way, but that's the way it often happens. And unfortunately, this gives Release Planning a bad name too.
And partly because of this, many coaches recommend that Release Planning not be done until the Team has established a real velocity. (To me, for reasons I will not explain here, this is the wrong solution to a very real problem.)
Fourth, because of the rapidity of change in many areas, many agile people feel that the YAGNI (You Ain't Gonna Need It) principle means that up-front planning should be minimized. Very minimized. Often these people are from business situations where keeping Release Planning quite minimal will actually work. But, in my professional opinion, other business situations actually greatly benefit from some up-front planning....so long as everyone uses the plan (and all its refactorings) to drive good behavior (such as minimizing the stories that get in the real "minimum marketable feature set" of the next release).
I guess we need to be concrete here, to avoid silly discussions. I won't give all the parameters here (and, to be more accurate, one should), but let me ballpark and say that a typical 3 month release for a Scrum team can probably be planned usefully using agile methods (Release Planning) in about 3 days or less. Not a half-day, but again, not 2 weeks or a whole week either. Again, there are potentially a host of possible factors; these are fairly typical real-life situations I have in mind.
What's Right About Release Planning
OK. So we acknowledge that there are 'issues' with up-front planning. (And more than we said above.) These are not issues that either Scrum or waterfall create, but issues nonetheless.
Now, let's look at some of the positives.
A. Businesses need some way of comparing efforts, to decide which efforts should get started, and which should not start. Exact or even somewhat 'accurate' numbers are not essential here...just some rough sense of date/budget and level of accuracy. In virtually all businesses that I work with, this is (or should be, in my professional judgment) a requirement.
B. The team needs to start to see, at a middle level, the interrelationship of the benefits and costs of the work (features) it will be building. To make many other decisions (architecture decisions, among them). Again, rough numbers are sufficient usually. And even very rough numbers are better than nothing. Release Planning provides this information.
C. A sense of the rough time trade-offs is needed. Customers always want features immediately. When the initial conception of the first release looks, as an example, a long way off, it forces the mind into innovative solutions about how to scope down the first release. Release Planning can give the team enough information to start being innovative this way.
D. I think the biggest value of the initial Release Planning is that the whole team starts to see the same elephant. The same effort. In multiple dimensions. The stories (functionality), the business value (at a story level), the effort (at a story level), the risks and dependencies, etc. This is extremely valuable and, in my experience, the level of 'same-page-ness' is so very much better than what I always did in waterfall planning (and what I still hear). Again, extremely valuable, even though it will change.
E. Re-Planning. The initial Release Plan sets up a base from which re-planning canbe done easily. Each Sprint. I like to call this Release Plan Refactoring (and sometimes we use other names for it, such as: Product Backlog Grooming, Product Backlog Refactoring, etc, etc.).
Re-planning, as Eisenhower implied, is essential. The way I will put it (maybe mixing some metaphors) is: We have to use Pareto to deliver just the most essential stuff so the customer gets the coffee while it's still hot. A continual daily-hourly fight to see that most essential stuff. Mark Denne's "Software By Numbers" talks more about why it is so essential to release a "minimum marketable feature set." A great phrase.
Let me emphasize that in Scrum, we are always doing "Release Plan Refactoring" every Sprint. It includes re-planning in multiple ways, such as: adding new stories, using a new known velocity of the team, refactoring the story points on a few badly estimated stories, adjusting some business value points, slicing rising epics into smaller stories, etc, etc.
Having mentioned all these things: Scrum does not define these activities. Again, Scrum is a framework only. I have given you what I see and teach and advise for almost all situations. (And other people whom I respect also concur with these.) But the activities just mentioned are not defined in the Scrum framework. Scrum only defines a 'short' release planning, and then release plan refactoring (although the Scrum literature does not use a consistent term for release plan refactoring).
E. The final value is motivation. The team no longer feeling like they are getting the mushroom treatment. They see the 'big picture' at a more meaningful and richer middle-level. The team feels like they are respected. They see themselves as adults making an adult, well-considered commitment to build that product. This motivational impact is quite important. And is not affected by the inevitable changes much.
***
More to say, but enough for now. For those interested, see our earlier posts on Release Planning. And future ones.
Wednesday, March 9, 2011
Sunday, March 6, 2011
Additional ideas: adopting lean-agile
This topic is being discussed here. And many places. I think I will be discussing it with a senior executive in Canada on Monday. So, very pertinent for me. And likely many others.
A lot of smart people in that discussion (I know some of them personally). A number of great ideas were mentioned by these people in that discussion, and I wanted to summarize them and react to them.
1. My advice might differ greatly if I understood Mark's specific situation better -- really, if I understood it at all. So, my comments in the prior post are based on what I find is the typical situation.
2. As discussed here earlier, asking people to do something (Scrum) that they really do not (yet) feel an interest in doing, is not a recipe for success. So, typically, one must influence them to want to do Scrum. Usually a fair amount of work itself.
3. To re-phrase Steven "Doc" List's main point (I think fairly): In Scrum terms, it is always in my experience useful to do a decent Release Planning with the whole team. (We do this in the Workshop, although not always with enough time to do it 'well enough for now'. So, it may need some further work before Sprinting starts.) To me, the biggest advantage in this is that it gets the team all on what page: what is this effort all about.
In Scrum, we refactor the Release Plan every Sprint (to some degree).
4. As Eric L said: Bringing in an external person (or people) is valuable. This is one of the patterns in Fearless Change by Manns and Rising. (In fact, get that book. And do all the patterns in it. Repeatedly.) The external person can do training, coaching, a speech...many different things.
5. I know people at Rally Dev and at Thoughtworks. All the people I know are good (although not all equal). Each brings 'their thing' to the table.
6. Jay C. mentions being prescriptive. There is a paper that Jeff Sutherland contributed to called Shock Therapy, which describes an approach to this. While it is risky in some ways, I think it can be very successful to be prescriptive about what Scrum is (for a period of time). Not prescriptive about how they do their 'real work', but about Scrum.
7. Jay C. also mentions being value centered, about which I strongly agree. I start this in Release Planning. My fuller approach I call Business Value Engineering, where I steal ideas from Scrum and Lean to incrementally improve the approach to delivering higher business value. As one very small example, from the beginning I advocate Priority Poker, which is much like Planning Poker, only done with the top 5 guys and about Business Value.
8. Bob M. mentions the idea of doing agile-scrum outside of just the software building part of the overall business process. I concur. It often would actually be more valuable elsewhere. The question is how and when. Sometimes useful to demonstrate that agile-scrum is successful here (sw dev), and then expand it out.
9. Matthew H. mentions an analysis of the organization: what's working and what's not. I would definitely do one initially. (Maybe an hour or two.) I would say the key is: prioritize your battles (if you like the war metaphor). Scrum by the way can be said mainly to be a way of seeing your problems more clearly --- then it takes the guts to actually act on that new information.
10. Matthew H. talks about understanding the leadership team. This is often a key part of the organization, and analyzing and getting an action plan around leveraging their strengths and mitigating their weaknesses can be very useful. This has to be done in a timebox. Everything does not depend on these people. The most important person is really the 'agile advocate' (typically, that is you). Again, see Fearless Change by Manns and Rising.
11. Daniel G. gives a link to a very nice list of the basics of Scrum. This is a useful thing to use, particularly with more than one team, so there starts to be a good discussion around "why aren't we all doing the basics of Scrum?" Other uses too, of course.
12. Robyn D. gives some great advice about how to set up the first team at the very beginning. Get a good set of work, do some release planning, get them to collaborate, etc. Being realistic about the 'quality' of the first few sprints is very good advice. My advice (prior post) took a somewhat longer term time frame.
13. Robyn D. mentions XP practices (TDD, CI). Concur. I would implement the fixes to the biggest impediments first. Usually the value in doing these XP practices should be either: faster velocity (at sustainable pace) or reduced technical debt, or both. But, sometimes these XP practices are a great hammer, but we are not quite ready to get to the nails yet. If see what I am suggesting. Anything can be the biggest impediment; fix the biggest one first.
14. Matthew H. mentions 'silver bullets and ideology'. Umm. This is hard to explain well. On the one hand, "it depends on common sense" is always useful advice. One the other hand, common-sense is not that common. Then we observe: There is a lot of Scrum-But out there. And usually we find people acquiescing to it, with no good long-term reason (often the initial short-term compromise was quite sensible).
We do find people in agile whom I would characterize as impractical or too impatient about the initial implementation. They want a team to do all of Scrum and all of XP on day one. For example. So, being impractical in this way about existing impediments on day one seems to me wrong.
We do find people in agile who say or imply 'do Scrum [or XP or whatever] and all will be well'. Let's be clear. Scrum cannot guarantee success. Example: If you have a great team at American football, and you ask them to play soccer, they will fail at that against a world cup level soccer team. But (back to our regular situation) if the team is not sufficiently competent for the work they have been given, Scrum will enable you to see that impending failure sooner.
On the other hand, as I suggested earlier, typically we think people should be very aggressive about removing impediments (within the rule that 'a dead ScrumMaster is a useless ScrumMaster'). And we also find in very many scrum implementations a lack of aggressiveness about removing the impediments (eg, the Scrum-Buts that we had to start with). So, we need some 'fire in the belly' about eventually getting to 'better Scrum' (and better lean-agile-scrum more broadly).
Our overly simplistic motto for the right attitude is from lean: 'The relentless pursuit of perfection.' Meaning: We accept that we are imperfect. We define a 'better state' and we relentless pursue that until we are close enough to see that we can define a yet higher state of 'perfection'. In other words, we relentless pursue getting better, without the illusion that we will ever become truly perfect.
Wow, that balance between being practical and being idealistic is hard to explain. I hope you could get the idea-action that I meant.
I hope you found one or two of the ideas here actionable in the short-term. (It is important to think and then act, in a tight cycle.)
Comments or further comments welcome.
A lot of smart people in that discussion (I know some of them personally). A number of great ideas were mentioned by these people in that discussion, and I wanted to summarize them and react to them.
1. My advice might differ greatly if I understood Mark's specific situation better -- really, if I understood it at all. So, my comments in the prior post are based on what I find is the typical situation.
2. As discussed here earlier, asking people to do something (Scrum) that they really do not (yet) feel an interest in doing, is not a recipe for success. So, typically, one must influence them to want to do Scrum. Usually a fair amount of work itself.
3. To re-phrase Steven "Doc" List's main point (I think fairly): In Scrum terms, it is always in my experience useful to do a decent Release Planning with the whole team. (We do this in the Workshop, although not always with enough time to do it 'well enough for now'. So, it may need some further work before Sprinting starts.) To me, the biggest advantage in this is that it gets the team all on what page: what is this effort all about.
In Scrum, we refactor the Release Plan every Sprint (to some degree).
4. As Eric L said: Bringing in an external person (or people) is valuable. This is one of the patterns in Fearless Change by Manns and Rising. (In fact, get that book. And do all the patterns in it. Repeatedly.) The external person can do training, coaching, a speech...many different things.
5. I know people at Rally Dev and at Thoughtworks. All the people I know are good (although not all equal). Each brings 'their thing' to the table.
6. Jay C. mentions being prescriptive. There is a paper that Jeff Sutherland contributed to called Shock Therapy, which describes an approach to this. While it is risky in some ways, I think it can be very successful to be prescriptive about what Scrum is (for a period of time). Not prescriptive about how they do their 'real work', but about Scrum.
7. Jay C. also mentions being value centered, about which I strongly agree. I start this in Release Planning. My fuller approach I call Business Value Engineering, where I steal ideas from Scrum and Lean to incrementally improve the approach to delivering higher business value. As one very small example, from the beginning I advocate Priority Poker, which is much like Planning Poker, only done with the top 5 guys and about Business Value.
8. Bob M. mentions the idea of doing agile-scrum outside of just the software building part of the overall business process. I concur. It often would actually be more valuable elsewhere. The question is how and when. Sometimes useful to demonstrate that agile-scrum is successful here (sw dev), and then expand it out.
9. Matthew H. mentions an analysis of the organization: what's working and what's not. I would definitely do one initially. (Maybe an hour or two.) I would say the key is: prioritize your battles (if you like the war metaphor). Scrum by the way can be said mainly to be a way of seeing your problems more clearly --- then it takes the guts to actually act on that new information.
10. Matthew H. talks about understanding the leadership team. This is often a key part of the organization, and analyzing and getting an action plan around leveraging their strengths and mitigating their weaknesses can be very useful. This has to be done in a timebox. Everything does not depend on these people. The most important person is really the 'agile advocate' (typically, that is you). Again, see Fearless Change by Manns and Rising.
11. Daniel G. gives a link to a very nice list of the basics of Scrum. This is a useful thing to use, particularly with more than one team, so there starts to be a good discussion around "why aren't we all doing the basics of Scrum?" Other uses too, of course.
12. Robyn D. gives some great advice about how to set up the first team at the very beginning. Get a good set of work, do some release planning, get them to collaborate, etc. Being realistic about the 'quality' of the first few sprints is very good advice. My advice (prior post) took a somewhat longer term time frame.
13. Robyn D. mentions XP practices (TDD, CI). Concur. I would implement the fixes to the biggest impediments first. Usually the value in doing these XP practices should be either: faster velocity (at sustainable pace) or reduced technical debt, or both. But, sometimes these XP practices are a great hammer, but we are not quite ready to get to the nails yet. If see what I am suggesting. Anything can be the biggest impediment; fix the biggest one first.
14. Matthew H. mentions 'silver bullets and ideology'. Umm. This is hard to explain well. On the one hand, "it depends on common sense" is always useful advice. One the other hand, common-sense is not that common. Then we observe: There is a lot of Scrum-But out there. And usually we find people acquiescing to it, with no good long-term reason (often the initial short-term compromise was quite sensible).
We do find people in agile whom I would characterize as impractical or too impatient about the initial implementation. They want a team to do all of Scrum and all of XP on day one. For example. So, being impractical in this way about existing impediments on day one seems to me wrong.
We do find people in agile who say or imply 'do Scrum [or XP or whatever] and all will be well'. Let's be clear. Scrum cannot guarantee success. Example: If you have a great team at American football, and you ask them to play soccer, they will fail at that against a world cup level soccer team. But (back to our regular situation) if the team is not sufficiently competent for the work they have been given, Scrum will enable you to see that impending failure sooner.
On the other hand, as I suggested earlier, typically we think people should be very aggressive about removing impediments (within the rule that 'a dead ScrumMaster is a useless ScrumMaster'). And we also find in very many scrum implementations a lack of aggressiveness about removing the impediments (eg, the Scrum-Buts that we had to start with). So, we need some 'fire in the belly' about eventually getting to 'better Scrum' (and better lean-agile-scrum more broadly).
Our overly simplistic motto for the right attitude is from lean: 'The relentless pursuit of perfection.' Meaning: We accept that we are imperfect. We define a 'better state' and we relentless pursue that until we are close enough to see that we can define a yet higher state of 'perfection'. In other words, we relentless pursue getting better, without the illusion that we will ever become truly perfect.
Wow, that balance between being practical and being idealistic is hard to explain. I hope you could get the idea-action that I meant.
I hope you found one or two of the ideas here actionable in the short-term. (It is important to think and then act, in a tight cycle.)
Comments or further comments welcome.
Saturday, March 5, 2011
How do we start agile/scrum?
In the Agile Alliance LinkedIn group discussions, there is a discussion about 'how to adopt agile in my organization?' by Mark Lummus. This is a complex topic, with many things to say. Here is most of my first post (in that group) about this. There are many other things I might have said.
QUOTE
Mark,
Here are my opinions on this (based on much experience). And I am talking not so much to you (you probably have the experience to consider some of my remarks as obvious) as to a broader audience.
First: I think really doing lean-agile-scrum well requires lots of things. But I think I am agreeing with you to say: Start the change with Scrum.
Second: Go high and go low. Meaning: Get senior guys in support and get people in the teams to support.
2b: You need to build a team (or each one needs to be a real team), and it needs to be a team with a winning spirit. Scrum helps with this, but there are many other factors that contribute.
Third: Use some metrics to show that you are making progress. Do not let them insist on perfect metrics (never exist) and make sure people use common-sense in interpreting the metrics (common-sense is not so common).
Fourth: Being aggressive about removing impediments is key. In the team, by the ScrumMaster, by the managers. Hard to predict from afar where your biggest impediments will lie: people, rigor in doing Scrum (too much ScrumBut), engineering practices (see XP!), the flow of business info into the team (eg, weak PO), or what.
Fifth: Build off the success of the team(s). To me, success is more business success (as in: the customer is really happy) rather than technical success (we did what we said we would do X months ago). I don't know what you all do today, but I can just about guarantee, after 3 Sprints, some degree of relative success. Build on that.
Fifth: The 'change management' around good lean-agile-scrum is quite significant. It is never-ending. People think they understand lean-agile-scrum, but I think the paradigm shift is actually hard. Typically firms need coaching for the teams (and surrounding managers). We have seen a team have a full-time coach for 2 months. And I have seen a team get a coach for 3 days out of each 2-week Sprint, for a bunch of Sprints. Almost always, when I see a firm that chooses not to get coaching, it smells to me like being 'penny wise and pound foolish'. But I am sure there are also some cases of strong success without coaching...just not often, percentage-wise. (I am a coach; you might call me biased.)
Sixth: Technical debt and better technical practices will be very important, sooner or later. Start attacking them soon.
True success with Scrum should be measured as 5x-10x better than your current baseline. Some firms do this quite well and often, but most never get there. Far, far too many settle for minor success (eg, a 50% improvement). I think the secret sauce is not so much the practices of Scrum (the dance steps), but rather more people 'getting' the underlying values and principles (the music). The dance steps become truly powerful when in sync with the music. Without the values and principles, a 'daily scrum', for example, can be almost a waste of time.
What would others put as the top 6 or 7 things for 'Mark' to consider?
UNQUOTE
More on this topic in a bit.
QUOTE
Mark,
Here are my opinions on this (based on much experience). And I am talking not so much to you (you probably have the experience to consider some of my remarks as obvious) as to a broader audience.
First: I think really doing lean-agile-scrum well requires lots of things. But I think I am agreeing with you to say: Start the change with Scrum.
Second: Go high and go low. Meaning: Get senior guys in support and get people in the teams to support.
2b: You need to build a team (or each one needs to be a real team), and it needs to be a team with a winning spirit. Scrum helps with this, but there are many other factors that contribute.
Third: Use some metrics to show that you are making progress. Do not let them insist on perfect metrics (never exist) and make sure people use common-sense in interpreting the metrics (common-sense is not so common).
Fourth: Being aggressive about removing impediments is key. In the team, by the ScrumMaster, by the managers. Hard to predict from afar where your biggest impediments will lie: people, rigor in doing Scrum (too much ScrumBut), engineering practices (see XP!), the flow of business info into the team (eg, weak PO), or what.
Fifth: Build off the success of the team(s). To me, success is more business success (as in: the customer is really happy) rather than technical success (we did what we said we would do X months ago). I don't know what you all do today, but I can just about guarantee, after 3 Sprints, some degree of relative success. Build on that.
Fifth: The 'change management' around good lean-agile-scrum is quite significant. It is never-ending. People think they understand lean-agile-scrum, but I think the paradigm shift is actually hard. Typically firms need coaching for the teams (and surrounding managers). We have seen a team have a full-time coach for 2 months. And I have seen a team get a coach for 3 days out of each 2-week Sprint, for a bunch of Sprints. Almost always, when I see a firm that chooses not to get coaching, it smells to me like being 'penny wise and pound foolish'. But I am sure there are also some cases of strong success without coaching...just not often, percentage-wise. (I am a coach; you might call me biased.)
Sixth: Technical debt and better technical practices will be very important, sooner or later. Start attacking them soon.
True success with Scrum should be measured as 5x-10x better than your current baseline. Some firms do this quite well and often, but most never get there. Far, far too many settle for minor success (eg, a 50% improvement). I think the secret sauce is not so much the practices of Scrum (the dance steps), but rather more people 'getting' the underlying values and principles (the music). The dance steps become truly powerful when in sync with the music. Without the values and principles, a 'daily scrum', for example, can be almost a waste of time.
What would others put as the top 6 or 7 things for 'Mark' to consider?
UNQUOTE
More on this topic in a bit.
Subscribe to:
Posts (Atom)
