My client, a New Zealand government department, is in the process of introducing Agile-Lean. They are currently in a trial phase to see if it is for them and during the early stages they'd like to run Agile and non-Agile projects in parallel.

Fair enough, but how to choose whether a project should be run Agile or Waterfall?

There are a few factors that are important for the decision and I thought other people in a similar context might find this checklist of things to consider useful.

A checklist

1. About the Project

• Are you delivering software? (Agile is not a good fit for e.g. infrastructure projects)
• Is there uncertainty around what you are going build?
• Is there uncertainty about the complexity of the solution? (If the work is entirely predictable, repeatable and/or has been done before Agile is not the best fit.)
• Do you want to have the opportunity to adapt to changing requirements and/or new ideas throughout the project?
• Is this a high risk project?

If you have answered yes to any of the above questions your work would be a good fit for Agile.

However, apart from the work there’s the people factor.

2. About the People

In order to use Agile methods you’ll need a team who can do the actual work.

Here’s a checklist of things to get right if you:

1. have an experienced Agile team
2. have people with Agile experience but not a team
3. are starting a new Agile team

a) If you have an experienced Agile team:

An experienced Agile team is a team that has worked together with the same team members for at least 6 months.
Do any of your experienced Agile teams have the capacity to take on the work?

If so you’re ready to go ...

b) If you have people with Agile experience but not a team:

• Is the team able to co-locate? (If not do you know how to work around this limitation?)
• Are the team members full time?
• Is the team cross-functional and includes all skills needed to deliver the project?
• Can the Scrum Master or XP-style coach be full time?
• Is the Product Owner/Business Representative able to commit enough time to the project (2-3 days per week at least)?
• Will it be possible to involve end-users throughout the project?

c) If you’re starting a new Agile team:

In addition to the above you should also consider the following:

• Will the team be able to implement the necessary tools and practice changes to allow them to deliver fully tested working software every 2-4 weeks?
• Are all team members prepared to deal with pervasive changes to the way they do their daily work?
• Do your team members have the right personalities to enjoy working in an Agile team? (small egos, team players, open, honest, happy to learn new things, etc)
• Do you have a coach assigned or will have a coach assigned before the project starts?
• If you don't have a coach does anyone of the team have at least 6 months full time Agile project experience?

Summary

Context matters: I wrote this for the context of a large organisation that is delivering relatively big projects and faces a cultural change from command and control to Agile/Lean/Radical Management (why have we made this terminology mess? ;-)

I am aware that this checklist might not apply for other contexts but I think the essence of the project and people readiness check will be useful also to people in similar situations.

Overall the question of ‘is my project suitable’ is less important than ‘can I build an Agile team’. Having the right mix of people - skills and attitudes - is key to project success.

One of my current clients, a large government agency, have recognised that their current monothilitic waterfall approach doesn’t work all that well and are trying to decide whether to change their delivery approach to Agile or “just” Iterative (mini-waterfall style).

Management have recognised that introducing Agile is a major change and that it requires a change in the way people go about their daily work, a different management style and a change in attitude and behaviour for everyone involved.

I respect my client for recognising the magnitude of the change and I also appreciate that the attraction of an iterative development approach is that it doesn’t require much change. However, I think this is a “weasel-out solution” which, while helping management to hedge its bets, will not solve any of the fundamental problems associated with waterfall projects and will cheat my client of the benefits Agile could provide.

Here are 6 reasons why I would highly recommend to choose Agile rather than Iterative:

Iterative vs. Agile:

First of all, Agile is iterative already but it is way more than just iterative. Here are a number of differences between Agile and “just” Iterative development:

• Mini-waterfall is still waterfall

Iterative is still waterfall, just on a smaller scale. A series of mini-waterfalls is certainly better and less risky than one big waterfall but mini-waterfall still is fundamentally waterfall and comes with all its known problems such as difficulty to adapt to change (“Nice idea but sorry, the requirements have been signed off months ago”), cascading delays (“Oops, we need to shorten the testing phase”) and low quality (“We don’t have time to fix those bugs. We’ll fix them in a later phase/iteration”).

• Agile is iterative AND incremental

Iterative development splits large problems into smaller chunks but without an incremental approach it requires a perfectly shaped idea and solution upfront as well as dead accurate estimation to build the right thing according to plan.

Agile assumes that the first idea is not necessarily the best and final one and that learning happens continuously throughout the process. It encourages a tight feedback loop to incorporate new knowledge, emerging requirements and innovative ideas even late in the process. Incrementing in addition to iterating towards a solution hugely increases the chances of success.

• Agile helps “building the right thing” through customer involvement

Iterative development does not address the lack of business and end user involvement usually seen in waterfall projects. Reports and PIRs time and again show that lack of business and end user involvement results in solving the wrong problem, making the wrong tradeoffs and increasing risk by not building core functionality first.

Agile bakes customer involvement in through making the customer and end-user representative part of the delivery team throughout the entire duration of the project or life cycle of the product. Agile teams don’t second guess; they ask, watch and learn.

• Agile delivers high quality

Iterative development still puts the testing phase at the end of each iteration. Testing at the end of an iteration results in defects not being detected until late in the iteration which usually makes it impossible to find the time to fix them. In addition, if any of the previous phases of analysis, design or build is delayed the test phase will be shortened and quality issues will go undetected even longer (Ever been on a project where bugs were found during systest or UAT?).

Agile on the other hand does not have phases during an iteration and ensures testing is done continuously from the very beginning. Any bugs found during the iteration will be fixed before new work is started. This guarantees that we have working software at the end of each iteration with no work left to be done (no 90% done or known defects). Should we run out of time we compromise scope but never quality.

• Agile is open to incorporating feedback and learnings

Iterative development has no mechanism built in to accommodate feedback and learning. Most of the time planning and design still happen upfront and scope is usually well-defined and fixed. Change arising from user testing, stakeholder involvement, increased understanding of the problem and an ever changing world can’t easily be incorporated into a just iterative approach without increasing project risk. Also, iterations can be as long as three to six months long which makes collecting feedback an infrequent matter and increases the risk of being out of touch with customer and business needs.

Agile limits iteration length to one to four weeks which ensures that feedback is received in a timely manner and can be used to adjust course. Scope is somewhat variable and defined in user and business goals which makes it possible to be open to changes to the solution while focusing on the desired business outcomes. In fact, Agile encourages testing all our assumptions to maximise early learning and to minimise risk. Apart from early feedback risk is managed through ensuring that core functionality is built first, a concept that is not necessarily required in iterative development.

• Agile fosters communication and collaboration

Iterative development retains a phased approach (analysis, design, build, test) and promotes the existence of functional silos. Functional silos communicate through written handovers, often “thrown over the wall”, which are a known breeding ground for misunderstandings and omissions. At best BAs, designers, developers and testers will co-ordinate their work with each other but they will never truly collaborate.

Agile relies on self-organising, cross-functional teams responsible for and authorised to reach a shared goal. Not only will this reduce the number of handovers and miscommunication it will foster real collaboration where the whole is greater than just the sum of its parts.



Overall Iterative development doesn’t force organisations to change and while some might say that’s a good thing I think Agile has so much to offer that it’s well worth the effort and learning curve.

Following last week’s interview with a newly-minted Scrum Master this week I have had a conversation with developer Mateusz Udowski. We talked about how SilverStripe’s adoption of Agile and Scrum have affected him and why he thinks SilverStripe is now more intelligent as a whole than it was before.

Mateusz, what was the most interesting thing that happened during the change? What was the most surprising?
I was expecting we would have problems with completing less exciting tasks such as testing, but that didn’t happen. Shifting the responsibility from individuals to the group gave space for everyone to work efficiently.

Also, we needed some structure and after we started working as a team we began to collaborate and everything seemed to fall into place. I think the main reason for our success was that the team members regarded each other as peers.

What's different now?
The biggest change is that there is no penalty for helping each other out anymore. The budget is relevant on the team level, but not on individual level, which means we can share problems and solutions.

One can clearly see what to work on now, and what needs to be worked on next. Tasks and impediments are clearly visible and are not being swept under the carpet.

Reducing the size of the team from 40 (whole company interacting semi-randomly) to 7 (Scrum team) makes it easier to work together, mainly because we get the chance to learn about and respect each others’ strengths, weaknesses and habits.

What is your day like now versus before?
Less stressful. It's possible to get more work done because we can focus on tasks and clearly see at any moment where we are in the sprint and in the project.

Folk wisdom has it that when you force a person's brain to focus on many things in parallel, their IQ falls considerably, and that's how it feels now - less chaotic, more intelligent.

What are you more more confident about now?
It's easier now to apply creative solutions and "refactor fearlessly". We have shifted from delivering at all costs to delivering high quality products up to capacity. Peer reviews, test coverages and testing by many people all contribute here.

It is possible to enhance how the team works both technically and from a process perspective. We can now build on top of what we have achieved in previous iterations because there is a "we" - a stable team.

What did you have to learn? What was the hardest to learn?
Not to tell people what to do and how to do it. It never worked well anyway. The other important ability is to be able to discern when is the time to say "no" to factors that would break the Scrum process.

How do you think you benefitted from working with a coach?
I've seen many of these Agile elements previously in different combinations and contexts, but never all put together.

A good coach will give you the confidence to apply all these principles now, immediately, and to achieve good result. Otherwise we would probably be stuck in the step-by-step approach which would stop halfway through and ultimately fail.

Pointing out the future possibilities is also helpful, showing that it's not the end of the road.

What's been in it for you?
There is more room for error and to try things out, hence more is possible. Short feedback loops and peer reviews provide a platform for learning and a cushion for failure.

Would you recommend Scrum and Agile to others?
Yep, without hesitation.


Conversations like these remind me why I love being an Agile coach. Whether the context or flavour is Agile, Lean, Systems Thinking, Scrum or Kanban (all very good things) - what it really boils down to is to make people’s working lives more enjoyable and purposeful and to create an environment where people are trusted to make decisions and have the freedom to succeed.

Mental note to self: Re-read this interview when I get frustrated with process, command and control and factory thinking.

Six months ago SilverStripe, an open-source Content Management System provider and Wellington web agency approached me to help them improve the way in which they deliver client and open source projects, increase employee happiness and, in general, just do the best possible job. To achieve this, we decided to move away from the existing Agile-like (fixed scope/fixed price) approach and introduce Scrum with its focus on client-driven iterations, early feedback and continuous improvement.

Silverstripe have asked me to interview some of their staff about the transition to Agile. The original posts can be found on Silverstripe's blog (Sam Minnée, Aleksandra Brewer), below are selected highlights from the interviews.

It has been interesting for me as a coach to learn how people experienced the changes including the biggest surprises, differences, and how they found the changes working out for them.

Today’s conversation is with Scrum Master Aleksandra Brewer. Alex works with one of the Agile teams at Silverstripe and has likened working with me to a visit to the dentist.

Alex, what was the most surprising thing that happened during the transition?

Some of the surprising (although maybe obvious) things were that (1) it's possible for more than one person to work on the same user story, (2) work goes faster when people collaborate, (3) sprint planning that results in greater understanding of stories and tasks necessary to complete them really speeds up the work during the sprint - everyone knows what needs to be done and can pick up a simple task and complete it.

What's different now?

I love being able to see the day to day progress of the team - it's so visible on the board, plus the work seems to be going faster, with several people going through small tasks all the time. With the acceptance criteria being defined and discussed before the start of a sprint, and with the Product Owner being available to answer any additional questions and provide feedback throughout the sprint, there is virtually no possibility for any team member to go off on a tangent.

What are you more confident about now?

Talking to clients is easier now, as they are much more involved and ultimately responsible for making decisions about priorities. We (the team) make recommendations, share our knowledge and inform the client about pros, cons and consequences of the different options, but in the end it's up to them to make a final decision.

All along the course of a project clients know exactly where we're at, what's being built, etc., which they love. The transparency of Scrum, although scary at the beginning, is really beneficial for both the team and clients.

What did you have to learn? What was the hardest to learn?

The hardest thing to learn was to give up the control over what the individual team members were doing from day to day.

How do you think you benefitted from working with a coach?

Working with you has been a bit like going to the dentist - painful at times, but all along I knew it was good for me, and I'm in better shape now than I was before. It's been good to have you keep us on track, and point out things that now seem obvious, and yet were not at first.

Would you recommend Scrum and Agile to others?

Definitely. I couldn't imagine going back to the old ways, negotiating "resourcing" among Project Managers, developers being on several different projects at the same time, and not knowing when a project would end because of the uncertainty of developer availability.





Conversations like these help me put my work into perspective: They show me where I can improve as a coach and help me be a bit less perfectionist and appreciate that, even though we are not perfect, we have come a long way and I have helped make some people's working days a bit more fun and satisfying.

Next week I will talk to Mateusz Udowski., a developer at Silverstripe.

In many companies, especially those who provide services to external clients, the main focus from a project management perspective seems to be on resource allocation and utilisation. People are viewed as individual “resources” and an important goal is to maximise people’s utilisation (Before you say this is not true, think about how important this metric is at your company and that people tend to optimise what is being measured).

I know that especially for vendors who often charge by the hour or have to keep cost down in a fixed price and scope scenario, utilisation is hugely important. That’s fair enough. However, making utilisation the main driving factor, will backfire and ultimately lower utilisation.

Why is this a problem?

By foregoing the team approach, aiming to maximise utilisation and piecing together projects through dynamically allocating and re-allocating people we:

• Take away the focus from the real end goal which is delighting our customers (people who either pay for, use or support our application).
• Inadvertedly sub-optimise individual parts of the system such as e.g. design, frontend development or database design by isolating them from the whole product.
• Miss out on optimising the development of the whole product or project.
• Ignore the fact that software is developed as one-of-a-kind, which causes variety and makes predictive planning impossible. Honestly, stuff often gets finished either earlier or later than initially planned and there’s nothing we can do about it.
• Have difficulty absorbing this inevitable variety if resource utilisation is too high as people will either be idle or unavailable as a consequence of variety and lack of slack (This is bad for utilisation and people will either be stressed out and over-worked or bored. Also, no slack makes it difficult to deal with emergencies and opportunities).
• Run the risk of cascading delays within projects: If one part of the project is delayed dependent parts will be delayed as well, leading to a cascading effect.
• Run the risk of cascading delays between projects: Projects waiting for a particular person to come free will be delayed, again potentially causing a cascading effect on other projects’ schedules.
• Create an increased need for handovers between individuals. Handovers are regarded as one of the biggest causes of waste in software development and come with the added danger of losing valuable tacit knowledge. Handovers also often cause misunderstandings through ambiguity and are plain evil.
• Miss out on the advantages of working in teams. A well-functioning team collaborates to a level where the sum is way bigger than its parts. Well functioning teams create magic - much like the All Blacks on a good day.
• Make bad decisions: Without teams each individual on a project will only have part of the picture and it is almost impossible to make good decisions without knowing the history, background and reasoning behind all the other little decisions that have been made on the project so far. Also, diverse groups of people are usually superior in decision making ability to individual experts.
• Incentivise people to care about their particular part only, which leads to behaviours that are detrimental to the success of the overall system. For example, if I reward a football player for the number of goals he scores he will most likely engage in egoistic behaviour that damages team performance. The same is true for software development teams - especially if people’s utilisation is linked to their salaries.
• Waste time allocating and re-allocating people to projects and trying to get the impossibly complex and variable puzzle to work out.

What to do?

If we, instead of the above, provide a group of people with a shared vision, collective responsibility, the authority to design their own processes and ways of working (the essence of self-organisation) while incentivising them as a team and giving them the all the support they need they will have a much greater chance in achieving the end goal of delighting the customer.

So, let’s:

• Organise people into feature teams and give them the freedom to succeed
• Focus on improving the boundary-spanning handovers within the organisation (just as Mary Poppendieck says!)

People will also be happier and love their jobs a lot more if we let them experience the magic of a team and “move work through people, not people through work”.



For further reading about building teams and organisational structures and behaviours that support teams I highly recommend Esther Derby’s blog.

Tags: agile , team, organisations, lean

“When you need me but do not want me, then I must stay. When you want me but no longer need me, then I have to go.”
— Nanny McPhee (via Lyssa Adkins)

I am an Agile coach and the goal of my job is to put myself out of a job.

My mission is to teach people Agile and to make sure they understand and correctly apply Agile values, principles, frameworks and techniques. This is quite a big deal as Agile often forces us to change the way we work on a daily basis; how we organise our work, how we collaborate, which tasks we perform and how we communicate with each other and the rest of the organisation.

Adopting Agile is quite an intense process, especially because it involves so much behavioural change. To make matters worse, learning does not progress in a linear fashion, it is more of a “two-steps-forward-one-step-back” process and it takes time. It is frustrating, painful and rewarding and as a coach it is my job to guide people through this experience.

It is also my job to know when they are ready to take off the supporting wheels and to work on their own. It is my responsibility to recognise the point in time when they are safe to fly and let (sometimes force) them to do so. If I’m still coaching the same people after two years or things unravel when I go on a 4-week holiday I have made people dependent and have failed as a coach.

So when am I done? When have I succeeded? Hard to say when you’re dealing with a process that has continuous improvement and therefore no end point at its core but I know for sure that I have succeeded when things not only work without me but keep improving after I have left.

I’m proudly leaving my current client. I won’t forget to tag off.

Last week I presented at WebDU in Sydney. The conference was excellently organised by Geoff and the Daemon guys and I met lots of interesting people. And I love Sydney! In short I had a blast.

Apart from a workshop on user stories I presented on 10 ways to fail with Agile. Judging by the Twitter stream my presentation about the most common mistakes and pitfalls during Agile adoption and how to avoid them has been well received (okay, they also tweeted I looked like Starbuck in Battlestar Galactica Season 1 ;-)

As promised here is a version of my slides which more than just pretty pictures ...

10 ways to fail with Agile on Prezi

Tags: organisations, agile, failure, inspect & adapt, adoption, transition, scrum

To get our code to production what is left to do is to turn the "potentially" releasable product into a releasable product. To do this a number of activities are required: Which ones and how much effort they require varies greatly between types of organisation.

The fastest way to perform rollout activities is to do them within the development team and to automate as much of the process as possible. While I envy and respect teams that are mature in terms of test and deployment automation and operate in an organisational context that allows them to easily move code from environment to environment I find that this is very often not possible within larger organisations.

Often, among other activities, someone outside the team has to be instructed as to how to deploy code, other parts of the organisation have to be informed about the changes (eg. end users, operations and maintenance teams, etc), and manual regression testing has to be performed.

For the kind of large organisations Mike Cohn refers to in his description of a bank with COBOL code and no automated regression testing or large organisations new to Scrum and Agile I, too, find this is best done within a so-called release sprint.

What is in the release sprint?

Projects I have worked on and organisations I have coached usually included two types of work in their release sprints:

• Work that couldn’t be performed every sprint because it was too time-consuming and/or expensive (e.g stress testing, full regression testing, getting copy text translated)
• Work that is only needed when releasing to production (e.g. bundle software, send out release notes to ops team, send comms package to change management, move code through various environments)

What is a successful release sprint like?

The following was common for successful release sprints across all projects and organisations:  

• Independent of how much functionality we had developed and how "big" the release was the release sprint was always the same length (e.g. always 2 weeks).
• Everyone on the team had a 100% focus on getting the release out of the door. 
• We never developed any new functionality during the release sprint.
• Although sometimes unpredictable we made use of the task board and tracked our work with a burn-down.

• Sometimes, on larger teams (8-9 people), we were only a subset of the original development team but most of the team stayed on to help. (People who didn't work on the project did other work for the organisation such as e.g. R&D, "business planning", etc but that was not a project issue)

What are your experiences?

Tags: organisations, transition, agile

There’s a saying in software development “If someone gets hit by the red bus ... “ which roughly translates to that if you lose a project team member or two you still want to be able to get the work done and finish the project.

Normally, red busses are rare: The realistic worst case scenario is one or two team members out with the flu or someone quitting their job. A good team can normally recover from such disruptions.

But red busses can be bigger and more damaging and wipe out large parts of a team: This usually happens if a new CEO, new management or a new government revokes funding, recession hits or an organisation just plain runs out of money.

In this case one of two things normally happen:

1. The project goes on with fewer team members
2. The project gets canned or put on hold

Waterfall
This is bad news for everyone but has a devastating effect if your project runs any flavour of the waterfall methodology:

Fewer team members:
Going on with fewer team members will not just result in major delays but will risk the success of the entire project.

Job roles in waterfall projects and organisations are highly siloed, i.e. testers test, developers code and a business analysts deal with requirements. A team losing several members will not just work at a lower capacity but some work simply cannot be done: If a project for example loses its testers testing will not be performed and the project comes to a grinding halt.

Canning the project:
Stopping the project has even more devastating effects and most likely the entire investment in the project is lost.

Waterfall methodologies use a phased approach: Requirements gathering, Design, Build, Testing and Deployment. Depending on which phase the project gets stuck in an organisation might end up having paid millions for a functional specification, technical design or developed but not yet tested software. All work on a product that cannot be rolled-out and used and that might never see the light of day.

If a waterfall project is stopped before the final phase the product cannot be used and the return on the (often considerable) investment is zero.

Agile
For an Agile project loss or reduction of funding can be bad news too but the risk when using an Agile methodology such as Scrum or XP, if implemented properly, is contained and the investment will not have gone entirely down the drain:

Fewer team members:
Going on with fewer team members will reduce a team’s capacity but as the responsibility and accountabililty for delivery lies within the entire team rather than individual job roles an Agile team can usually keep on delivering.

Agile teams organise around generalising specialists where people have both a primary and several secondary job roles and if one or more specialists have to leave the team other team members will take on their work. If an Agile team for example loses its testers the team’s business analysts, developers, scrum masters/project managers etc will share the workload and ensure that the software is tested and working properly.

Even at a slower pace the team will produce software that can be rolled out to end users.

Canning the project:
If the project is stopped or put on hold the damage is limited. Agile teams develop software incrementally and the primary measure of success is working software. Every 2-4 weeks (dependent on iteration length) the team delivers fully functional software, i.e. software that is fully designed, built and tested.

The basic rule to implement high priority features first ensures that, if the project is stopped, the most important pieces of functionality have been fully developed.

Unlike in a waterfall project project sponsors will never be left with just a functional specification, pieces of documentation or untested software. You might not get everything you initially wanted or set out to build but everything that has been worked on is fully functional and ready to provide value to the business and/or end users.

Real Life
Loss of funding and team members is what has happened after a change of government at my (now former :-) workplace, a New Zealand government entity.

My Agile (Scrum) project has already produced working and usable software in an incremental fashion. Even if it should get stopped we have delivered business value and a good return on investment.

In this case the organisation’s choice of running Agile has put them ahead of their peers.

The waterfall projects? I wish them the best of luck - they’ll need it.

Tags: organisations, agile, risk

People keep asking me whether I’d run all projects using an agile framework such as Scrum.

When I answer “of course not” they often not only expect but gently try to steer me towards an answer that excludes certain type of projects: “You certainly wouldn’t use it for a mission critical system, would you? Or a compliance project? Or an infrastructure project? “

Actually, yes! I have used Scrum for business critical systems at a major mobile phone manufacturer, have implemented mission critical systems for a publicly owned European Rail Cargo company, a friend of mine has successfully delivered a compliance project and another one has even developed fighter jet control systems for a European Air Force using scrum.

It’s not about the type of project. It’s not even about the size of your project. It’s about whether you’re willing and able to create a setup that allows agile projects to succeed. To me the rather vague terms of organisational and cultural readiness can be narrowed down to a list of primary show stoppers:

I’d NOT run agile if ...

• People on the team are not fully allocated and committed to the project
• You can’t stack a great team and only get the people that everybody else wants to get rid of (Give me some of the company “stars” even if it’s just to show commitment and get me agile personalities!)
• You don’t have the courage to let the team make decisions, to allow them to learn by making mistakes and to trust them to get the job done - you’re stuck in command control
• You don’t have a business person who has the time, courage and authority to work with the team and to make quick decisions
• You expect instant success and need a silver bullet (Sorry to be the one to bring it to you but if your project is impossible or if you don’t have good people no methodology or framework is going to help you ;-)

There are of course numerous other things that are important for agile teams to succeed but usually there are work-arounds for non-ideal situations.

If I don’t meet any of the above obstacles I’ll go for Agile any time!

What’s your list of show stoppers or essential requirements for running an agile project?

Tags: organisations, governance, obstacles , change, agile, scrum