Dear Rob - 2

Rob, as some readers may recall from the first post, is a made-up character, representing a typical senior business executive, responsible at a high level for the agile initiative, among many other things. Perhaps the CEO (although probably not a CEO at the Jack Welch level at GE.)

In this post, we talk about Quality and Technical Debt.

****
Dear Rob,

It is interesting that you want to focus on quality first. This is actually a very good thing.

In Agile, we start to do professional testing in the very first Sprint. This is much earlier than in Waterfall, as you know. Many reasons for this; among them, we can measure progress by the amount of completed, 'fully' tested code.

In Scrum, we have what we call a Definition of Done for the team. It defines, for the typical 'story' they are working on, how 'done' the story will get to. Like the meat metaphor: rare, medium rare, medium, medium-well, and well done.

Fully done, done, done means the product (and the newly finished stories) is/are in production and being used by the customer with no problems. Most teams have too many impediments to get to done, done, done in one Sprint (1-4 weeks). At first.

The Definition of Done should also make clear, especially for the Product Owner, what is not being done in the Sprint. For example, often the team can't do a full, full regression test in a fully integrated environment. Often (full) performance testing can't be done. Etc.

As Ken Schwaber suggests, this 'undone' work needs to be added to the Product Backlog somewhere.

Two reasons we don't like anything but 'done, done, done' product increments. (They are related reasons.)
1. The bad news is getting better with age. Meaning, of course, worse. For example, all the undiscovered bugs are quickly becoming harder and harder to fix.

2. The 90% problem is growing.
The 90% problem is, for example, very common in software development. A manager goes to a team and asks them how complete they are. They say 90%. And then he (usually a he) asks, 'how much longer to be 100% done?' And they say, 'oh, it took us about 9 months to get to here....ummm, that will be another 9 months.' Yogi Berra summarized this problem by saying: "It ain't over until it's over."

So, any time the Product Owner sees things in the Definition of Done that are not getting done in the Sprint, he should worry about the 90% problem and discuss them.

Technical Debt

What does this mean? Well, there is no one simple definition of technical debt. I say it is anything in or around the system that makes it harder to change.

Examples include: duplicative functions or modules in the code, spaghetti code, lack of automated tests, zero or poor documentation (in the code, perhaps), code written by George (when George has left the firm), the bug list, any upgrade (eg, to the database or the language or some middleware) that we have put off, etc, etc, etc.

Every system that is 6 months old has some technical debt. And it is starting to be obvious that it is hurting us. Older systems have more technical debt.

If your product is older than 2 years, I can just about guarantee that technical debt is a (very) serious problem in your shop. And that real velocity is (very) low.

In simple terms, technical debt decreases the velocity of Scrum teams.

We (seemingly purposefully) grow technical debt by saying "we have to deliver features now; let's put that stuff off until later." And then we do that. That 'that stuff' is identifying bugs, upgrading the XYZ, fixing bugs, etc.

Every manager must start understanding technical debt, and fight to get the team to dig out of technical debt. If you want any business agility. Meaning: We can adapt and change with the market and customers faster than our competition.

Impediments

As we discuss these matters, you will find that your shop (and every shop) will discover impediments. It is tempting to say: This is Scrum's fault! In fact, Scrum is only making the problems around quality and technical debt more visible. To blame Scrum is only to blame the messenger.

So, don't be surprised that the group identified a bunch of impediments. And that they cost money to fix. And that you will get real business results (better quality, eventually lower cost, etc.) from that.

How do we dig out of Technical Debt?
How do we improve the Definition of Done over time?
Mainly by removing or reducing (certain kinds of) impediments.

Quality Metrics

One way of minimizing and maybe reducing technical debt is to focus on quality.

Now, quality is itself a complex subject, and its full and complete nature depends on the specific product and your customers.

But in simplistic terms, one might say it is the lack of technical debt. Or one might say that it means the customer gets a perfectly fitting product (to her problem), with no added 'features' (eg, bugs), no extra delays, no extra effort.

As relatively low-level indicators of quality in Scrum, we can measure:
* no bugs escaped the Sprint this week. Meaning: For all the stories we say we completed in the Sprint, we did a professional (automated) testing effort (eg, unit and functional, at least), and any bugs identified were fixed and retested in the Sprint.
* X number of new automated tests were built this week (maybe one number at the unit level and another at the functional level)
* Y number of automated tests are passing in the regression tests this week.
* Our Definition of Done is relatively strong, meaning that, in areas 1-5, when a story is done, it means that no technical debt was built in those areas (at least). Or at least we made a serious effort to minimize the new technical debt being built.
* The list of (pre)existing bugs is prioritized, and shrunk by Z items or A% this week.
* We have prioritized the work around an increase of code coverage by the automated tests, and it increased B percentage points in the past week. (And these were meaningful, useful tests, not just baloney tests to make the coverage metrics look better.)
* We saw that in the last release, field issues decreased by C%, comparing first month to first month.
* Our velocity has increased X percent. This is in part due to less technical debt and in part due to less effort per unit of work...due to: better configuration management, better continuous integration, better testing tools, and faster testing servers. (* If you don't understand some of these terms, and how they are inter-linked, we should talk. You need to understand them a bit, since they are key to each (Scrum) team's success.)

Measuring quality is complicated, and my comments above are over-simplified. But, if the business and technology folks both know you care about quality, that will help a lot. A reasonable, and fairly frequent, discussion around quality metrics can help focus their attention on that. Be careful about getting too focused on one aspect of quality, one metric. The keys are (a) higher quality for the customer, as the customer defines quality, and (b) lower technical debt.

Regards,
Joe

***

Again, please give me feedback. Did this all make sense? Or did some of it sound too geeky?

It is actually not all that geeky, but if there are some geeky terms that we need to explain, I think almost surely you need to learn them a bit. Not become an expert in them, just understand them enough to guide the product development group.

Also, please give feedback on what you want to discuss next.

'How' to rollout Scrum - a summary

In the prior post, I noted that Tom Mellor had a basic objection to this question (How to rollout Scrum). Which was, and I hope I do not misstate it too much, don't try to rollout something to an organization that does not really want it.

He used the other pig metaphor, which I liked. He said: It will be like trying to teach a pig to sing. It will frustrate you and annoy the pig. See here.

But I think there are times when it does makes sense to rollout Scrum. Albeit, it is hard to ever know if they really want it. Still, we take action and then we inspect results. So, below is my comment in that list about that.

***quote***

Let me go back to the main thesis or question.

Let's say you are in a group of 200 people (say, a SW dev group). Let's say the group has experimented with Scrum and generally likes it. Let's say at least some 'business' people like it (they are outside this group, basically its 'customers' in some sense). Let's say you have convinced the head of the Dev group that rolling it out is a good idea.

What are the next action steps?

1. To Tom's point, it would appear that the org (the 200 people at least) mostly want this change.

2. I would actually do something to assess how much the group really wants the change and what is it exactly they want. (With a smile and some tears: Often they just want no more documentation and no more planning, let's just start writing code and complaining. Meaning: They want their own myth about agile.) An Open Space might do that trick. But there are other methods. BTW, it is not necessary that everyone like it.

3. Assuming things are still positive. Then the roll-out needs a few things. The scrum community argues about their relative priority, but mainly these.

a. identify a speed of rollout (eg, X teams per week or month)
b. identify the new Scrum teams and their work
c. train the teams in Scrum. This might include a definition of 'barely acceptable Scrum' in your context. Not unlike a Definition of Done.
d. get coaches for the teams.
e. train the top managers (of this group). (not so easy)
f. train the other managers. (probably harder)
g. set up what I call an 'Impediment Removal Team' of managers. (Other books give this team a different name. Cf Schwaber, Cohn and others.) As a Scrum team, or semi-Scrum team. Their product backlog is the main/big impediments overall. Sooner or later 'adoption issues' will be added to that list.
h. help the middle-managers understand their new job/jobs.
i. instill a continuous improvement culture and specific values, principles and practices around that (I like the teams doing an A3 every Sprint, as one example.) Relatively soon, this includes more training.
j. start attacking Technical Debt ASAP.

I say 'training' above. What I really mean is an event that almost viscerally changes them. At least, that is what you want. We call it training to get them to come.

A fairly typical list. Lots more to add, of course.

***unquote***

Again, your comments are welcome.

Is there hope?

There is an interesting discussion on the CSM Linkedin group, about rolling out Scrum to an organization. See here.
Tom Mellor gives a realistic (for him, I am sure) but somewhat depressing prognosis for many Scrum implementations. Tom is a good guy and a friend. It is ok if he and I disagree a bit. Below is my post in response, slightly edited.

***quote***

Many good comments [in the discussion chain].

And a great issue (as in, big and difficult).

I guess I am a bit more of an optimist than Tom. But I believe also realistic.

First, let me mention two books: Fearless Change by Manns and Rising, and A Sense of Urgency by John Kotter. Lots of good ideas in those.

OK, so to start, someone in the org has to have that "oh, merde, we're screwed if we don't change" moment. (You?) And then that person needs to 'acquire' others who share that view. Not intellectually so much as emotionally, or in the gut. Now, here we are talking big changes, such as to lean-agile-scrum (as I call it).

There are lots of ways to 'acquire' others to the cause. But, of course, many things in the existing org culture will fundamentally resist change. Even quite illogically sometimes (eg, the demise of the firm is quite clear to virtually everyone if they don't change, yet the org may resist change). But, if you get enough people (see Egypt), change can also be hard to resist. Miracles can happen.

Now, some groups of people and some org cultures allow more change to happen faster. Many factors. I agree with Tom that the 'Leader' (CEO) is actually more of a follower. Far less control than he (usually a he) or they (the people in the org) tend to think. Far, far less. But, if that person helps set up a culture that (partially) wants change, then usually change-oriented people tend to gravitate (more) to that org, and so change becomes easier.

So, before you try to change something (an org) assess its ability to change. This is VERY hard to do. But at least try.

Now, also assess your willingness to use every trick in the book to make miracles happen (see American Revolution, 1989, Tunisia, South Africa, etc, etc.). Are you willing to work hard and wait for a little luck?

Sometimes the strategy needs to be: "I am ok if only my immediate group (team) is doing lean-agile-scrum." "For 1 year." That actually can be a lot of change that you can feel very proud of. Then, maybe you get lucky and it starts to spread.

You [people in the discussion] have already noted that smaller orgs tend to change more easily.

It is also useful to recognize in oneself that change can feel threatening, and have more sympathy (up to a point) with those who are afraid of change.

Remember also: Surely we will all change with time...and we have some influence on that change. While time may seem an enemy, you can also make it an ally.

No doubt most of you know this one: If you can't change your organization, then you must change your organization.

Finally (for now): "Go confidently in the direction of your dreams." There really is no other choice. If you want to be alive. (Yes, of course, our dreams sometimes need clarification.) We cannot wait for that perfect day when change will be easy.

*** end of quote***

Let me add this.
Every new product development team I have seen has needed Scrum. It is a crying shame that it has not been implemented more and more effectively. Real lives (of team members and customers) are far less happy than they should be.

So, I think it is fair and just to hope that firms that cannot adopt Scrum will soon be defeated by firms that allow far more change (ie, to Scrum, in my belief). Yes, this creative destruction will be painful for some, but it is necessary in the long run.

Your comments are welcome.
The next post is a bit more practical. But I hope you will note its limitations.