Innovation: The most important question

Steve Denning (a very interesting guy) tweeted about this article:
http://www.openforum.com/idea-hub/topics/the-world/article/innovations-most-important-question-matthew-e-may

The question (in the article) is: Is there a better way?

Read it. Nice story behind it. (Steve is a big proponent of stories. Google his books.)

Actually useful if you want your team to be innovative. (You do want your team to be innovative, right? It isn't just hard work, right?)

My only comment: It isn't a question (a thought), so much as a feeling: Goddammit, there's gotta be a better way!!!

The way we envision people is grossly over-simplified. But one way is the well-known dichotomy: thinking vs feeling. And my hypothesis (cf. Freud) is that more action (results) derive from feeling than thinking.

Middle Managers and Scrum

In a lot of organizations I work with, we need to do a better job of explaining Scrum to the middle managers.

Most of the people in the Teams get Scrum and see benefits. Most of the senior guys see their metrics get better, and, if they focus on the right feedback, hear and see good things.

But often the middle managers feel left out. Yes, they see benefits here and there, but often not for themselves. They should actually see benefits for themselves, but no one shows them how to realize those benefits. We just assume they will naturally, without any explanation, understand and adapt to, the change (to Scrum).

But, a common feeling among the middle managers is: 'Who moved my cheese?' (If you know that book.) Meaning: Before, I knew how to manage and be successful and show progress. Now, with Scrum, they moved all my levers. And how do I function successfully?

So, we need to take the time to explain it to them.

Here is a first, too quick, pass at that explanation. It is too simplistic, and too specific in assuming a prior waterfall process. But, a good start (I hope).

***

1. Your job has gotten better (not stupid).

Instead of doing something stupid (trying to manage individuals in a waterfall process), you now help teams get better in a more effective approach/framework (Scrum).

2. Your job has gotten better (clearly useful).

Instead of trying to show in waterfall that you did something truly useful (almost impossible to show, really), you now can see real progress in the success of the Scrum team. And working product (software) is a far better measure of real progress. As is team velocity.

3. Your job is different (remove impediments).

Instead of pushing the team to 'work harder', you now are a servant leader. Your main job is to help the team remove impediments.

Honestly, most of you removed impediments before. You just didn't call them that. And there was no clear way to prioritize them. Now, you get help from the team and more recognition (up and down).

For some of you, you get to join a Scrum team and lead the team in doing 'real work'. Which in general is more satisfying for most people. But for those of you who like the frustrating work of removing impediments, we have a clear, prioritized, long list of them. For you to help fix.

4. Your job is different (no command-and-control).

In almost all cases, we do not recommend the command-and-control style any more.

OK, maybe at most 1-in-20 of the guys in our shop are really lazy or fundamentally without good motivation. Yes, you can help re-purpose (ultimately, fire) those very few people.

And you can identify those people who also appear to be de-motivated, and help them get through the past de-motivators we had around here, and connect to work they really want to do.

But mostly we want you to take the coaching, servant leader style with the team.

You will find that the coaching style is much more satisfying than the command-and-control style. And the people you work with will now mostly like you and appreciate your efforts to make help them better. Instead of ducking-and-diving to avoid you.

5. More truth.

With both the people below you and the people above you (well, we want to start moving away from the hierarchy metaphor, to a more flat organization, but for now....)...for both sides, the good news is everyone is talking the truth more.

This is much more satisfying.

Although, to be honest, in the short-term, and until we get more people understanding it, it can be painful and take more courage.

But in the long-term, it is better for everyone, including you, if we are all more honest. Our work of new product development is truly hard. We, of course, make lots of mistakes. (As all scientists who discover a new thing do.) We are trying as hard as we can to learn as fast as possible from our mistakes. None of us is perfect.

Each of us tends to have a typical, repetitive weakness (and, the good news: we now have a team that helps each of us avoid our own peculiar weakness(es)).

There is nothing particularly new about this; it was true in waterfall really. But we are now going to admit it more, top to bottom (as we say).

6. Career growth.

Because we will be more successful as a firm with this approach (Scrum...or at least so we fully expect), we will be more profitable as a firm. And so, your prospects for long-term career growth are better.

Assuming it is the style of 'play' that you are comfortable with. (To be honest, not every manager fits with the style of Scrum. Some are died-in-the-wool command and control managers...they should not be 'forced' to play Scrum. Although maybe they should be 'invited' to play their style at another firm.)

***

This is a start on what to explain to the middle managers. Even at the high level, we have left out a lot. Then the middle managers start to ask: OK, what do I specifically do? While some of the specifics will indeed be very specific to your organization, we can say more on that. And will in a future post.

PS. The excellent cartoon is by Mike Baldwin. Google him.

Dear Rob - 2

Rob, as some readers may recall from the first post, is a made-up character, representing a typical senior business executive, responsible at a high level for the agile initiative, among many other things. Perhaps the CEO (although probably not a CEO at the Jack Welch level at GE.)

In this post, we talk about Quality and Technical Debt.

****
Dear Rob,

It is interesting that you want to focus on quality first. This is actually a very good thing.

In Agile, we start to do professional testing in the very first Sprint. This is much earlier than in Waterfall, as you know. Many reasons for this; among them, we can measure progress by the amount of completed, 'fully' tested code.

In Scrum, we have what we call a Definition of Done for the team. It defines, for the typical 'story' they are working on, how 'done' the story will get to. Like the meat metaphor: rare, medium rare, medium, medium-well, and well done.

Fully done, done, done means the product (and the newly finished stories) is/are in production and being used by the customer with no problems. Most teams have too many impediments to get to done, done, done in one Sprint (1-4 weeks). At first.

The Definition of Done should also make clear, especially for the Product Owner, what is not being done in the Sprint. For example, often the team can't do a full, full regression test in a fully integrated environment. Often (full) performance testing can't be done. Etc.

As Ken Schwaber suggests, this 'undone' work needs to be added to the Product Backlog somewhere.

Two reasons we don't like anything but 'done, done, done' product increments. (They are related reasons.)
1. The bad news is getting better with age. Meaning, of course, worse. For example, all the undiscovered bugs are quickly becoming harder and harder to fix.

2. The 90% problem is growing.
The 90% problem is, for example, very common in software development. A manager goes to a team and asks them how complete they are. They say 90%. And then he (usually a he) asks, 'how much longer to be 100% done?' And they say, 'oh, it took us about 9 months to get to here....ummm, that will be another 9 months.' Yogi Berra summarized this problem by saying: "It ain't over until it's over."

So, any time the Product Owner sees things in the Definition of Done that are not getting done in the Sprint, he should worry about the 90% problem and discuss them.

Technical Debt

What does this mean? Well, there is no one simple definition of technical debt. I say it is anything in or around the system that makes it harder to change.

Examples include: duplicative functions or modules in the code, spaghetti code, lack of automated tests, zero or poor documentation (in the code, perhaps), code written by George (when George has left the firm), the bug list, any upgrade (eg, to the database or the language or some middleware) that we have put off, etc, etc, etc.

Every system that is 6 months old has some technical debt. And it is starting to be obvious that it is hurting us. Older systems have more technical debt.

If your product is older than 2 years, I can just about guarantee that technical debt is a (very) serious problem in your shop. And that real velocity is (very) low.

In simple terms, technical debt decreases the velocity of Scrum teams.

We (seemingly purposefully) grow technical debt by saying "we have to deliver features now; let's put that stuff off until later." And then we do that. That 'that stuff' is identifying bugs, upgrading the XYZ, fixing bugs, etc.

Every manager must start understanding technical debt, and fight to get the team to dig out of technical debt. If you want any business agility. Meaning: We can adapt and change with the market and customers faster than our competition.

Impediments

As we discuss these matters, you will find that your shop (and every shop) will discover impediments. It is tempting to say: This is Scrum's fault! In fact, Scrum is only making the problems around quality and technical debt more visible. To blame Scrum is only to blame the messenger.

So, don't be surprised that the group identified a bunch of impediments. And that they cost money to fix. And that you will get real business results (better quality, eventually lower cost, etc.) from that.

How do we dig out of Technical Debt?
How do we improve the Definition of Done over time?
Mainly by removing or reducing (certain kinds of) impediments.

Quality Metrics

One way of minimizing and maybe reducing technical debt is to focus on quality.

Now, quality is itself a complex subject, and its full and complete nature depends on the specific product and your customers.

But in simplistic terms, one might say it is the lack of technical debt. Or one might say that it means the customer gets a perfectly fitting product (to her problem), with no added 'features' (eg, bugs), no extra delays, no extra effort.

As relatively low-level indicators of quality in Scrum, we can measure:
* no bugs escaped the Sprint this week. Meaning: For all the stories we say we completed in the Sprint, we did a professional (automated) testing effort (eg, unit and functional, at least), and any bugs identified were fixed and retested in the Sprint.
* X number of new automated tests were built this week (maybe one number at the unit level and another at the functional level)
* Y number of automated tests are passing in the regression tests this week.
* Our Definition of Done is relatively strong, meaning that, in areas 1-5, when a story is done, it means that no technical debt was built in those areas (at least). Or at least we made a serious effort to minimize the new technical debt being built.
* The list of (pre)existing bugs is prioritized, and shrunk by Z items or A% this week.
* We have prioritized the work around an increase of code coverage by the automated tests, and it increased B percentage points in the past week. (And these were meaningful, useful tests, not just baloney tests to make the coverage metrics look better.)
* We saw that in the last release, field issues decreased by C%, comparing first month to first month.
* Our velocity has increased X percent. This is in part due to less technical debt and in part due to less effort per unit of work...due to: better configuration management, better continuous integration, better testing tools, and faster testing servers. (* If you don't understand some of these terms, and how they are inter-linked, we should talk. You need to understand them a bit, since they are key to each (Scrum) team's success.)

Measuring quality is complicated, and my comments above are over-simplified. But, if the business and technology folks both know you care about quality, that will help a lot. A reasonable, and fairly frequent, discussion around quality metrics can help focus their attention on that. Be careful about getting too focused on one aspect of quality, one metric. The keys are (a) higher quality for the customer, as the customer defines quality, and (b) lower technical debt.

Regards,
Joe

***

Again, please give me feedback. Did this all make sense? Or did some of it sound too geeky?

It is actually not all that geeky, but if there are some geeky terms that we need to explain, I think almost surely you need to learn them a bit. Not become an expert in them, just understand them enough to guide the product development group.

Also, please give feedback on what you want to discuss next.

'How' to rollout Scrum - a summary

In the prior post, I noted that Tom Mellor had a basic objection to this question (How to rollout Scrum). Which was, and I hope I do not misstate it too much, don't try to rollout something to an organization that does not really want it.

He used the other pig metaphor, which I liked. He said: It will be like trying to teach a pig to sing. It will frustrate you and annoy the pig. See here.

But I think there are times when it does makes sense to rollout Scrum. Albeit, it is hard to ever know if they really want it. Still, we take action and then we inspect results. So, below is my comment in that list about that.

***quote***

Let me go back to the main thesis or question.

Let's say you are in a group of 200 people (say, a SW dev group). Let's say the group has experimented with Scrum and generally likes it. Let's say at least some 'business' people like it (they are outside this group, basically its 'customers' in some sense). Let's say you have convinced the head of the Dev group that rolling it out is a good idea.

What are the next action steps?

1. To Tom's point, it would appear that the org (the 200 people at least) mostly want this change.

2. I would actually do something to assess how much the group really wants the change and what is it exactly they want. (With a smile and some tears: Often they just want no more documentation and no more planning, let's just start writing code and complaining. Meaning: They want their own myth about agile.) An Open Space might do that trick. But there are other methods. BTW, it is not necessary that everyone like it.

3. Assuming things are still positive. Then the roll-out needs a few things. The scrum community argues about their relative priority, but mainly these.

a. identify a speed of rollout (eg, X teams per week or month)
b. identify the new Scrum teams and their work
c. train the teams in Scrum. This might include a definition of 'barely acceptable Scrum' in your context. Not unlike a Definition of Done.
d. get coaches for the teams.
e. train the top managers (of this group). (not so easy)
f. train the other managers. (probably harder)
g. set up what I call an 'Impediment Removal Team' of managers. (Other books give this team a different name. Cf Schwaber, Cohn and others.) As a Scrum team, or semi-Scrum team. Their product backlog is the main/big impediments overall. Sooner or later 'adoption issues' will be added to that list.
h. help the middle-managers understand their new job/jobs.
i. instill a continuous improvement culture and specific values, principles and practices around that (I like the teams doing an A3 every Sprint, as one example.) Relatively soon, this includes more training.
j. start attacking Technical Debt ASAP.

I say 'training' above. What I really mean is an event that almost viscerally changes them. At least, that is what you want. We call it training to get them to come.

A fairly typical list. Lots more to add, of course.

***unquote***

Again, your comments are welcome.

Is there hope?

There is an interesting discussion on the CSM Linkedin group, about rolling out Scrum to an organization. See here.
Tom Mellor gives a realistic (for him, I am sure) but somewhat depressing prognosis for many Scrum implementations. Tom is a good guy and a friend. It is ok if he and I disagree a bit. Below is my post in response, slightly edited.

***quote***

Many good comments [in the discussion chain].

And a great issue (as in, big and difficult).

I guess I am a bit more of an optimist than Tom. But I believe also realistic.

First, let me mention two books: Fearless Change by Manns and Rising, and A Sense of Urgency by John Kotter. Lots of good ideas in those.

OK, so to start, someone in the org has to have that "oh, merde, we're screwed if we don't change" moment. (You?) And then that person needs to 'acquire' others who share that view. Not intellectually so much as emotionally, or in the gut. Now, here we are talking big changes, such as to lean-agile-scrum (as I call it).

There are lots of ways to 'acquire' others to the cause. But, of course, many things in the existing org culture will fundamentally resist change. Even quite illogically sometimes (eg, the demise of the firm is quite clear to virtually everyone if they don't change, yet the org may resist change). But, if you get enough people (see Egypt), change can also be hard to resist. Miracles can happen.

Now, some groups of people and some org cultures allow more change to happen faster. Many factors. I agree with Tom that the 'Leader' (CEO) is actually more of a follower. Far less control than he (usually a he) or they (the people in the org) tend to think. Far, far less. But, if that person helps set up a culture that (partially) wants change, then usually change-oriented people tend to gravitate (more) to that org, and so change becomes easier.

So, before you try to change something (an org) assess its ability to change. This is VERY hard to do. But at least try.

Now, also assess your willingness to use every trick in the book to make miracles happen (see American Revolution, 1989, Tunisia, South Africa, etc, etc.). Are you willing to work hard and wait for a little luck?

Sometimes the strategy needs to be: "I am ok if only my immediate group (team) is doing lean-agile-scrum." "For 1 year." That actually can be a lot of change that you can feel very proud of. Then, maybe you get lucky and it starts to spread.

You [people in the discussion] have already noted that smaller orgs tend to change more easily.

It is also useful to recognize in oneself that change can feel threatening, and have more sympathy (up to a point) with those who are afraid of change.

Remember also: Surely we will all change with time...and we have some influence on that change. While time may seem an enemy, you can also make it an ally.

No doubt most of you know this one: If you can't change your organization, then you must change your organization.

Finally (for now): "Go confidently in the direction of your dreams." There really is no other choice. If you want to be alive. (Yes, of course, our dreams sometimes need clarification.) We cannot wait for that perfect day when change will be easy.

*** end of quote***

Let me add this.
Every new product development team I have seen has needed Scrum. It is a crying shame that it has not been implemented more and more effectively. Real lives (of team members and customers) are far less happy than they should be.

So, I think it is fair and just to hope that firms that cannot adopt Scrum will soon be defeated by firms that allow far more change (ie, to Scrum, in my belief). Yes, this creative destruction will be painful for some, but it is necessary in the long run.

Your comments are welcome.
The next post is a bit more practical. But I hope you will note its limitations.