Review: Agile! The Good, the Hype and the Ugly

In October, I spent some time in the company of Bertrand Meyer, author of "Agile!: The Good, the Hype and the Ugly". This book was written to be an independent, impartial and objective study of the various agile methods (scrum, xp, lean, crystal) viewed against the knowledge-base of software engineering methods and principles. The author, being no stranger to software engineering, is well-known in the computer world, across both academia and industry. He took it upon himself to do the research, investigate the agile landscape breadth-and-depth, probing assertions, practices, principles and values using a scientific (and empirical) approach with searching questions, thus providing an overall assessment. This wasn't purely an academic exercise, Meyer walked the path of agile himself, is even a certified as a Scrum Master, his team are using selected methods of agile in their own product development, so it's not like Meyer is throwing the baby out with the bath water! On the contrary, Meyer tries to remain objective, unbiased and fair in his reporting and analysis.

This book may just as well be the first book to read if you're a software manager, entering the agile-space, who's potentially feeling uncomfortable with perhaps some misplaced(?) "baggage" of software engineering, old-school-style projects, as touted by some agilists. Meyer has done almost all the background work for you, covering and assessing the popular agile methods in play today.

I was quite intrigued by the book's title, who wouldn't be!!?? You must admit it is quite EDGY, axe-to-grind, in-your-face-daring-the-agile-pundits - agilistas. I just HAD to get my hands on a copy, I actually waited a long time to buy this book (due to the bad Rand/Dollar exchange rate). I have voraciously read most of the popular books on agile (Schwaber, Cohn, Poppendieck, Rubin, Appelo, Pichler, Derby et. al, you name it), that extol this new thing "agile", often claiming a silent revolution is coming to overtake the industry, that "Software Engineering" should belong to the annals of history, and instead welcome "Software Craftsmanship" as in.  And when I read these signature-series books, I do get caught up in the rush-of-it-all, excited, converted and have actually been a promoter for #agile for ten+ years...I was caught hook, line and sinker!

Then when I came across "Agile! The Good, the Hype and the Ugly" written by a person very well respected in the industry, I had to ask myself, if I may have actually fallen for some hype, maybe I didn't ask probing questions, without having empirical data to substantiate claims. I wanted to find out if I was potentially backing the wrong horse, wanted to check some of my own values, personal-biases or not, of software engineering experiences held weight or not, but most important, the title being so catchy, I was rather curious to find out what the "Hype & Ugly" bits of agile this book claimed were...

Since my background in software is in embedded systems (Set-Top-Box systems) and highly-available-systems (Real-Time-Streaming/Encryption-Services) I grew up with the scientific engineering mindset (BSc. Electronics Engineering & Masters Computer Science), so I often found myself being selective with vanilla Scrum and had in the past, cautioned people against following a particular agile method with extreme dogma, i.e. I maintained a certain amount of discipline and structure was always needed. This is primarily because of the particular domain-experience I was coming from, which wasn't high-level application non-critical development (Mobile apps, WebApps, Websites), or application development that relied on a stable SDK/engines (i.e. the expectation of a stable operating system, database, etc upon which to build applications on top of).

This book, in my view, should be essential reading for any software manager, looking to understand agile methods before diving head-first into a vanilla, textbook-implementations.

For people convinced about agile to-the-letter, this book will be a little edgy for you - one needs a cool head, and openness to accept some of the challenges that Meyer puts forward, especially when it comes to backing up assertions of values/practices/principles or citations of productivity-gains, without sound scientific and empirical data to back up those claims.  Meyer highlights such challenges from some of the books that I myself have held in high esteem for many years, so take it on the chin...

Meyer's style of writing is somewhat academic, factual, but also practical with some nerdy-humour thrown in-between. Meyer has written with sincerity, remained as open-and-unbiased-as-humanly-possible, and made a conscious effort not to promote his own personal projects, products and frameworks. Meyer cuts to the core of uncomfortable-but-some-relevant truths, especially challenging assertions and statements that lack scientific validation, or backed up by empirical studies. He writes with a depth of experience and passion for practical software methods that it forces you to think hard about the course you're on, the things you just accepted and may have taken for granted (e.g. forgoing necessary engineering practices such as a little bit of design up-front to support changing requirements).

You have to be patient with Meyer as he unpacks in some surgical, analytical detail the various topics, in fact, the selling point of the book's title, is actually left right till the last chapter, so you have to read from start-to-finish, because the essence of the Hype, Ugly, Good & Brilliant is saved for the end (building upon his arguments and case-points from the earlier chapters).

I was taken on a roller coaster ride, experiencing moments of pure resonance thinking I am on the same wavelength as this guy riding high, in-phase. Yet also, there were instances when I felt a little edgy, somewhat uncomfortable, noticeably shifting my position as I lay in bed reading at night. Stopping, putting the book aside, to sleep over it. [I am two+ years into consulting as a Systems & Software Engineering Management consultant, doing the odd agile coaching gig here and there, advising on agile systems processes - and here is Meyer taking issue with consultants!]

In keeping with my deep-review style for special books - topics struck certain nerves, either resonating (fully in agreement with Meyer) or feeling of discomfort (not sure, not convinced), so I graphed the below curve, which is how I resonated with Meyer's assertions in the last chapter, specifically the edgy bits: Meyer's UGLY & HYPED assertions:

The blue area shows the feel-good, things that resonated with me, the extent of which I agreed and was comfortable with the ideas. The amber spots show the areas that made me feel uncomfortable, my level of discomfort, that either I'm not convinced, or have some personal biases that's potentially blinding me from seeing the points. On the whole though, resonance wins over discomfort.

[Aside: Here is Meyer's blog post introducing why he wrote this book, you'll find detail about the book's table of contents too]

Here's the detail of these comments, for each topic - In what follows, read as:
Title, Level of Resonance, Level of Discomfort, Comments

The Bad and the Ugly parts of Agile