Agile doesn’t jibe with big, so the big question is: How do you scale Agile?
The good news is, Agile is common sense, not rocket science.
You can be Agile in a big way when you pass the Agile primer stage and pay attention to three things:
- Organization as a whole,
- Organization using containers,
- Container integration.
Some Background (or skip this part)
Once upon a time, software development was done in long projects using a “waterfall” process with a few simple phases: requirements, design, code, test, and voilà . . . release.
This software development life cycle (SDLC) made sense and people were thrilled and thorough (I was one of them), taking great care in “gathering” requirements and tracing them from phase to phase to make sure the final solution would meet those carefully-gathered requirements.
Then in 2001, seventeen independent-minded software development practitioners met to “talk, ski, relax, and try to find common ground.” The get-together was a success: it resulted in the Manifesto for Agile Software Development or, as you call it, the Agile Manifesto.
The Agile Manifesto called attention to a carefully selected set of values and principles (four values and twelve principles, to be exact) and proclaimed a better way of working, and so Agile was born.
Fast-forward to today, Agile is everywhere from startups to corporate America, in part thanks to the most popular incarnation of the Agile value system: the Scrum framework.
What Is Agile About?
At its core, Agile is adaptive rather than predictive, and people-oriented rather than process-oriented.
Another tenet of Agile: breaking big, overwhelming things into small, manageable chunks.
Break big, overwhelming things into small, manageable chunks.
Breaking big into small works: it reduces complexity and increases agility when it comes to people (go from large groups of people to small teams), projects (go from lengthy requirements documents to small stories), process (go from long-drawn-out development phases to short sprints), and yes, with cautious optimism, architecture (go from monolithic applications to microservices).
Thanks to Agile, companies changed.
Before Agile, companies took many months to painstakingly develop elaborate solutions that users didn’t necessarily want. Bummer.
Now, companies deliver delightful functionality continuously, thanks to autonomous teams that change course along the way based on customer feedback. Hurrah!
So is that it?
Well, not really. If Agile were that simple, there would be nothing to write home about. Unfortunately, as the saying goes, if it seems too good to be true, it probably is.
Agile Gone Awry
Back to 2001, things certainly got off to a good start with a masterful manifesto and a magic word: “agile.”
But then, 10–15 years later, big companies jumped on the Agile bandwagon with process and tooling specialists who advertised their process or tool as the way to go to be “Agile at scale”—and things got dicey.
No doubt, there is value in process and tools. It’s just that the basic mechanics of Scrum and the bells and whistles of your tool of choice, they will only take you so far.
Not only that, at scale, process and tools are insidious: they become treated as ends in themselves and create complexity that slow down your Agile transformation, not to say undermine it, instead of accelerating it.
Before you know it, process and tools dominate as one-size-fits-all solutions that create oversimplification in some areas and overcomplication in others.
And in the end, process and tools force people to accept undesirable compromises and distract them from the real goal: be organized to act together and get the right work done, the right way.
Scrum
Take Scrum for example.
For big organizations, it’s a no-brainer to start an Agile transformation with Scrum, because Scrum has prevailed as the de facto standard of Agile frameworks.
Basically, Scrum is a safe way to go.
Scrum is not only a safe way to go, it’s also straightforward: the Scrum Guide defines Scrum with three roles, five events, and three artifacts.
Scrum is good because it’s a simple way to break monolithic organizations and overwhelming projects into small chunks. In a nutshell, (1) you break large groups of people into small teams (the Scrum Teams) and (2) you ask the teams to work with a business-minded person (the Product Owner) to turn projects into pieces of functionality (the Product Backlog items) that the team can implement within short time boxes (the Sprints).
Scrum is good but Scrum leaves a lot to be desired.
For one thing, if you apply Scrum by the book with dozens or hundreds of teams, you will certainly end up with dozens or hundreds of teams doing work in small chunks and short sprints, but there will be so many visible and hidden dependencies between the teams, chunks of work, and sprints (not to mention dependencies on non-Scrum teams and non-Scrum processes) that you will wonder how much agility you really got.
The answer: not much.
Sure, you can double down on Scrum: you can go with a Scrum variant meant for scale like Scrum@Scale, Nexus, or Large-Scale Scrum. But chances are, Scrum on steroids will still fall short of addressing your reality and meeting your expectations.
There’s only so much that any one process can do, and the same goes for tools (Jira, Rally, and the like).
So your choice:
A. Use process and tools to roll out “Agile” en masse across dozens or hundreds of teams and settle for a sub-optimal organization that’s marginally better than what you had to begin with.
B. Stand by the first statement of the Agile Manifesto: “Value individuals and interactions over processes and tools.”
Keep reading if you chose B.
—
Container Structure
If process-and-tools isn’t the answer, then back to the big question, how do you scale Agile?
This is one way: create your own multi-container structure.
Unlike one-size-fits-all process and tools, which impose methods on people and reduce interactions between them, a multi-container structure empowers people to organize themselves at different levels (from big picture to implementation details) across multiple dimensions (people, products, projects, practices).
Now bear with me, there is quite a bit to unpack.
Containers
Of course, the container construct is nothing new.
Containers come in all shapes and sizes.
Back to Scrum for a second, Scrum thrives on small containers: small teams (people container), small backlog items (work container), and short sprints (time container).
In technology, services and microservices are containers of sorts and so are, well, application containers.
Containers are everywhere and for good reasons: containers shield from external interference and increase focus.
Containers are also an effective means to break big, overwhelming things into small, independent chunks (the Agile tenet mentioned earlier).
So, can containers transform large monolithic organizations into Agile federations?
Containers certainly can, when you “containerize” all the way, step by step.
Here are 10 steps for you to consider.
Step 1: Start with One Big Container
To start off right, start with a “big container.”
With this big container, you want to carve out and encapsulate a piece of your business so you can handle it in an Agile way, as a smaller company within the company.
Whatever you call your big container, think of it as a company in its own right. A mini company or a startup, if you will.
Your big container should be as big as possible but not so big that it becomes impractical (“too big to succeed”).
And your big container should contain these prime elements:
- People
- Products (at least one)
- Projects (big chunks of work, “epics” if you prefer)
- Practices (including roles, processes and tools)
You want to start with this way because you want to improve the whole (optimization), not just the parts (sub-optimization).
Step 2: Nest Containers
Think of nested containers as matryoshka dolls.
People-wise, the big container should encompass everybody necessary. Then, within it, you can develop a structure of rightsized containers at different levels of granularity.
For example: 3–9 people make a team, 3–9 teams make an area, 3–9 areas make a group, and 3–9 groups make the “mini company.”
Be deliberate about the number of levels in your hierarchy (generally speaking, the fewer the better) and calibrate the size of the container units until you achieve something that works for you.
Interestingly enough, the military uses this type of nested container structure: 2 fireteams make a squad, 3 squads make a platoon, 3 platoons make a company, and so on.
Step 3: Create a Taxonomy of Containers
You can use the same type of nested container structure to order other discrete dimensions, not just the people dimension.
For starters, here are the four basic dimensions mentioned in Step 1: People, Products, Projects, and Practices.
In Agile transformations these days, the mere use of the word “project” can cause controversy and confusion, and that’s a different topic altogether. Here, simply take these terms as labels for different types of container—labels you can change as you see fit.
Take Spotify for example. With labels like “tribe,” “chapter,” and “guild,” Spotify created a vernacular that reinforced Spotify’s very own culture.
With and beyond labels, as ordinary or creative as they might be, the important thing is to define, group, and nest containers into categories that make sense to you.
Put another way, using a technical term, you want to create a taxonomy of containers—a taxonomy you can call your own.
The Project category is all about work: actual projects (“epics” if you prefer), inside bigger containers (“initiatives”), implemented in small chunks (“stories” if you must).
The Practice category is how you get the work done: management practices, technical practices, and tools and techniques meant to support those practices. In this dimension, consider containers to deal with Agile management practices (Scrum, Kanban, etc.), design and engineering practices (User Experience Design, Software Development, DevOps, etc.), and practice areas within practices (Domain-Driven Design, Specification by Example, Test-Driven Development, Continuous Delivery, Cloud Computing, etc.).
Finally, in the Product category, use containers to organize technology assets (applications, components, platforms) and crystallize actual products (something someone would pay for).
Just remember, the goal is to develop a taxonomy of containers that defines the most elegant and powerful multi-dimensional organizational structure you can implement.
Easy peasy.
Step 4: Align Containers
The more containers you use, the more complexity you add, and complexity is the enemy.
One way to keep complexity in check is to align containers across dimensions because when containers align, for all intents and purposes, they become one.
Indeed, it all comes together when containers align: one team, dedicated to one project, within one product area.
And it gets better when containers align at higher levels too: one group of teams, dedicated to one initiative, for one product.
Another way to keep complexity down is to keep the number of container levels to a strict minimum, say, three or four.
For example, use and align three levels for People (teams, groups of teams, mini company) with three levels for Projects (stories, epics, initiatives) and three levels for Products (product areas, products, product line).
Put another way, the span of the containers at each level should be relatively the same.
As a sidebar, this type of alignment is also helpful when working with non-Agile counterparts. In particular, if you’re transitioning from an old way of working with many levels to a new way of working with fewer levels, you will clear things up if you map old levels to new levels, at least loosely.
Step 5: Use Extra Containers
How good would it be to organize every thing into neat stacks of nested containers?
Every element of the whole would simply belong to one container inside another: people inside teams and groups of teams; work inside epics and initiatives; components inside applications and products.
Unfortunately, it’s not that simple.
In reality, as you develop your container structure and its taxonomy, you will want the flexibility to layer “extra containers” to manage additional variables and further optimize the whole.
First of all, in the People category, you can use extra containers to deal with a variety of attributes, not the least of them location: physical spaces within buildings, buildings within cities, cities within time zones.
Area of expertise is another good extra container from a people management point of view: when people report to a manager based on their area of expertise (software development for example), they can switch teams (their primary container) while staying with the same manager.
Area of interest (community of practice) is another example, this one more loose and fluid.
In the Project category, you have to manage releases across projects and might as well treat those releases as extra container.
In the Practice category, how about metrics? Good idea.
And finally, in the Product category, customer experiences across products, applications, and communication channels (digital, voice, paper) can be good extra containers too.
Basically, you can use extra containers to pay extra attention to different aspects of the overall organization.
Step 6: Use Makers and Enablers
Distinguish two basic kinds of container: “makers” and “enablers.”
Agile puts emphasis on makers.
With people for example, Agile values independent teams that build products. Sometimes called “feature teams,” these teams are makers.
Meanwhile, enablers support makers. Among other things, enablers can provide tools or specialize in functions like release automation or information security. Enablers make makers more effective and increase control.
Paul Graham differentiates managers from makers, which fits the model: managers should see themselves as enablers and they should strive to act as such. Enablers at the service of makers.
Agile glosses over enablers because it’s true, enablers don’t deliver business value directly and independently. What’s more, enablers can become bottlenecks, or worse, they can underserve and get in the way of makers.
Sure, in true Agile spirit, you want to encourage makers to take care of themselves and acquire all the skills, capabilities, or team members they need to run independently, unencumbered.
Then again, enablers make sense as you scale, and you would be remiss not to give them the attention they deserve.
Agile focuses on makers and it’s a good thing, until it is to the detriment of enablers. Neglecting enablers, or making do with enablers that fall short of fulfilling their function, will undermine makers. And leaving makers to their own devices isn’t a solution either, not at scale.
Here too, the military provides examples:
While a squad (“maker”) is better off with combat lifesaver training and equipment within the squad, a squad is also better off with the support of a medic attachment (“enabler”).
And more generally, support comes in at the platoon level to perform specialty functions (combat medic, radio telephone operator, etc.) and, as you go up the levels, support increases to fulfill staff functions (personnel, intelligence, training, supply, etc.).
Combat units and support units, makers and enablers, you get the gist.
And yes, makers can be enablers: makers can share tried and tested solutions with other makers (for example, using an in-house open source model).
Step 7: Integrate Containers
By now you have a lot of containers and, as a result, complexity moves from containers to connections between containers.
Now the challenge isn’t so much with what’s in the containers, but rather the dependencies and interactions between the containers.
As a parallel, when you integrate systems, you meticulously design and document interfaces (APIs). This way, teams can work independently on their own system, knowing how it all comes together in the end.
The same goes for containers. You want well-defined interfaces between containers so it’s clear how containers work together and form a cohesive whole, with no gray area or gap between them.
As you define interfaces between containers, it’s also a good time to confirm the boundaries of the containers. Containers should be as independent as possible, and interfaces as simple as possible. In age-old tech terms, you want containers to be well-encapsulated and loosely coupled.
Take “makers” and “enablers” for example. Maker and enabler should work out how they will interact with each other and form a solid bond: they should determine how the enabler will enable and support the maker and, the other way around, how the maker will consult and inform the enabler.
As you define interfaces, just stay true to the foundational principles of the Agile Manifesto. While you should value process and documentation and treat interfaces as contracts of sorts, you should value conversation and collaboration more.
Step 8: Operate at Different Levels
“Vision without execution is a daydream. Execution without vision is a nightmare.”
In large organizations, the situation on the ground is always more complicated than you think, and it’s no surprise to see gaps and waste between vision and execution. Large organizations also tend to be monolithic, with a lot of pieces intertwined.
One way to bridge gaps and reduce waste is to deliberately operate at different levels: from entire organizations to individuals, from strategy to implementation details, from architecture to code.
With a container hierarchy, you can explicitly define levels that you can use to decompose and compartmentalize work.
And once the levels are clear, it becomes a matter of doing “just enough” work at the right level, at the right time—with higher levels providing context, and lower levels providing validation.
It’s probably worth mentioning that while it’s bad to work at any level in a vacuum, it’s good to concentrate on one level at a time. That’s because it’s hard to work at different orders of magnitude at the same time, and you don’t want to mix levels.
Step 9: Federate
Agile encourages teams to “self-organize” but obviously, there’s a limit to how far you want to take this principle. For example, for teams working on the same application, it makes sense to share common practices and tools, starting with a common source code repository.
Along the same lines, at higher levels of organization, self-organization gets into centralized governance versus decentralized governance.
Monolithic style organizations favor centralized governance and tend to standardize on single solutions. With this organizational style, you are in many ways back to square one: one-size-fits-all processes and tools.
In contrast, Agile style organizations strive to keep central governance to a bare minimum and empower teams to do as they see fit. You want to use Scrum? Go right ahead. You want to use your own process with some Scrum and Kanban mixed into it? Go for it.
At scale, with hundreds of teams, no style is perfect. The monolithic style can turn into sclerosis, and the Agile style can turn into chaos. And a fuzzy mix of both styles is no better.
Enter containers and the “multi-container” style organization.
You started with one “big container” (Step 1). At this point, to transform an entire company, you want to use not one but several big containers.
Beyond using big containers to carve out discrete business entities (think mini companies), the goal is to create a federation of big containers, each taking control of its own internal affairs. What you are looking for is just enough overall control to form a unity, then you want to get out of the way and let people organize themselves as they choose.
And don’t stop there. You can apply the same approach at every level and manage any container as a federation of smaller containers within it, all the way down to teams.
In the end, what you have is a kind of federated, multi-container structure that you can use to deliberately give and organize control at different levels of organization.
Give and organize control at different levels.
Again, terms like “big container” aren’t the important thing here—use your own labels. What matters is the approach of using containers to break things that are too big into concrete parts that are as independent as possible.
Step 10: Be Bold
It sounds good to use containers to break monolithic organizations into independent parts, the same way it sounds good to be Agile.
It sounds good but it’s hard to do.
It’s hard to do because it’s hard to figure out where container boundaries should lie, and then, it’s hard to reorganize to make those containers a reality.
There’s also this overwhelming desire to break down silos (instead of creating them) and develop universal solutions that somehow culminate in a universal, seamless customer experience. For example, if you were Google when Google bought Waze in 2013, wouldn’t you have been tempted to go right ahead and merge Waze and Google Maps?
But you can’t have it all (and Google kept Waze and Google Maps separate).
Agility comes at a price: you have to be willing to carve out concrete entities, and you have to slice around them to make them independent.
It’s possible to do this and there are a number of approaches you can take. The obvious one is to divide the organization along business lines. Back to “makers” and “enablers” (Step 6), you can also choose to favor makers over enablers, or vice versa.
If you focus on makers, your ultimate goal is to turn the organization into a portfolio of companies that are virtually free and independent from each other.
By the way, this approach—operating as a portfolio of companies—doesn’t mean you have to give up on the goal of creating seamless customer experiences. For example, Mint.com showed when they launched in 2007 that it’s possible to create an independent solution that lets you consolidate all your bank accounts in one place, seamlessly, no matter how many banks you use.
On the other hand, if you focus on enablers, your ultimate goal is to develop solutions that are so good and so enabling that any company, not just you, will want to use them.
Amazon provides examples of how you can operate in both “maker” and “enabler” mode at the same time.
Maker mode: In 2009, Amazon bought Zappos under the agreement that the relationship between the two companies would be “governed by a document that formally recognizes the uniqueness of Zappos’s culture and Amazon’s duty to protect it.” To this day, Zappos runs as an independent part of Amazon.
Enabler mode: Since 2006, Amazon has been developing Amazon Web Services (AWS), “the world’s most comprehensive and broadly adopted cloud platform, offering over 165 fully featured services from data centers globally.” Now, AWS enables over one million active customers and is a highly successful company in its own right.
Whatever you do, have a clear strategy and communicate it so everybody is on the same page. Incremental implementation is good, not incremental thinking.
Incremental implementation is good, not incremental thinking.
Your strategy should show how you want to divide the organization into containers with clear boundaries and clear connections between them, so you don’t end up with a jumbled mess.
Also, as Ron Jeffries recommends in his article “Developers should abandon Agile,” you want to detach your thinking from any particular named “Agile” method. No Agile playbook will ever have all the answers you are looking for.
So yes, without a doubt, it’s hard to slice up a monolithic organization.
The good news is that it can be done, as shown by companies that run this way because of acquisitions (the Zappos example) or because they simply had no choice in the matter (ring-fencing legislation comes to mind).
The other good news is that it doesn’t have to happen overnight (the AWS idea took years to bake) and it’s not a once-and-done thing; containers take time to emerge and container boundaries should continuously evolve.
Just be bold.
In Summary
- Start with one big container
- Nest containers
- Create a taxonomy of containers
- Align containers
- Use extra containers
- Use “makers” and “enablers”
- Integrate containers
- Operate at different levels
- Federate
- Be bold
Final Thoughts
The term “container” is good because it makes it clear you want concrete, self-contained entities you can manage in an Agile way. “Container” also forces you to talk about boundaries instead of being fuzzy on the edges.
And a “multi-container” structure is good because it’s a simple yet powerful structure you can use to see and improve your multi-level, multi-dimensional organization as a cohesive set of containers.
See and improve your organization as a cohesive set of containers.
But obviously, all of this is easier said than done.
In reality, if your organization is big and complicated and your business is doing well, you may find it reasonable to keep things more or less as they are, and settle for one-size-fits-all solutions.
Truth be told, if you are in a messy situation, things could actually get worse if you throw “containers” into the mix without the right people to make it happen.
That said, if you want Agile and you mean it, the multi-container method is a solution worth pursuing: use containers, be bold and deliberate, and you will pioneer your own way to be Agile at scale, big time.