Why do developers at Google consider Agile development to be nonsense? (2016)

As a former Google Engineering Director, I can offer some insight.

Agile means many things to many people. The simple high level Agile Manifesto is something I think is close to the way Google engineers think about software development.

• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan

However, once you get past these high level ideas into details, the agreement starts to fade. Agile has some good ideas but it also has problematic elements which are too centered around short-term thinking for working on revolutionary engineering projects done at companies like Google. Without getting buried in details, let’s look at the themes in Principles behind the Agile Manifesto .

Let’s start with the shared ideas. Google development style exemplifies the type of individual empowerment talked about in the principles. The specific principles which are most in-line-with and probably inspired by Silicon Valley include:

• Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
• The best architectures, requirements, and designs emerge from self-organizing teams.
• At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.
• Continuous attention to technical excellence and good design enhances agility.
• Simplicity - the art of maximizing the amount of work not done - is essential.

These principles are almost common sense for smart engineers. I argue Silicon Valley created the culture of empowerment and trusting individuals that these are centered around.

However, there are other parts of the principles which are not a part of Google-style development culture. These are the parts which have led to the short-term focused Scrum process. They seem suited to particular types of development, most notably consulting or contract programming, where the customer is external to the organizations, runs the show because they are paying for development, and can change their mind at any time:

• Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
• Business people and developers must work together daily throughout the project.
• The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
• Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.
• Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

This style of short-term planning, direct customer contact, and continuous iteration is well suited to software with a simple core and lots of customer visible features that are incrementally useful. It is not so well suited to software which has a very simple interface and tons of hidden internal complexity, software which isn’t useful until it’s fairly complete, or leapfrog solutions the customer can’t imagine.

Companies like Google write revolutionary software which has never been written before, and which doesn’t work until complex subcomponents are written. Bigtable and Borg immediately come to mind. Bigtable is a widely copied design for a distributed database, and Borg was one of the first extremely large scale cluster/cloud managers. This type of innovation takes significant up-front design time, and working on components over longer than one week iterations. Because the projects have such simple external interfaces, and so much internal complexity, much of the work is not even visible to “customers”, so there is no way to write customer visible stories about it. This type of software takes 8–20 months to deliver the first working version to the customer.

Projects like Bigtable and Borg are the anti-scrum. They represent extremely long term thinking on the part of the technical leaders. Instead of working on something that would meet a small need this week, they were laying a foundation for a fundamental shift in the way cluster software was developed. That investment has not only reaped incredible rewards at Google, but has influenced the entire industry.

Other industries have similar analogs. From tax-accounting software to computer games, some software is not suited to give to end customers when partially finished.

If I was asked to rewrite the above agile principles to be more in-line with Google style development, they might be more like:

• Our highest priority is to increase customer (and programmer) productivity and access to information. Work on the biggest, most frequently used problems you can find, and create the largest net impact. Don’t give the customer what they ask for; understand them, and revolutionize their world.
• Developers should create a Google Design Document (a fairly minimal, but structured design doc), explaining the project, what goals it hopes to achieve, and explains why it can’t be done in other ways. This document should be circulated with stakeholders, to get early feedback before the project gets underway. The written record is essential, as it assures there is a clear and agreed understanding of when the project is a success and how it aims to get there.
• At all phases of the project, critical design elements for larger components should be concisely explained and captured in a design document.
• Innovate in leapfrogs. It’s more important to finish and deploy a leapfrog than to attempt perfection. There is no perfection. Instead be flexible, and plan to constantly reinvent at every level of the stack.
• Deliver working software as soon as is reasonably possible, and no sooner. “Dogfood” projects internally before they are shipped externally. Make sure products meet high quality standards before shipping. The quality of the product is more important than the time it takes to achieve it.

While the high level Agile Manifesto is flexible enough to work with these principles, these are very different than the short-iteration low documentation Agile/Scrum process which has become synonymous with the word Agile.