Ship Software Smaller, Deliver Better Product

Cycle Time is a powerful metric. Google's DORA team has identified it as one of the Four Key Metrics for DevOps performance that are scientifically shown to lead to better business outcomes. At Haystack, we often champion it as the first "North Star" metric for evaluating overall engineering team health.

"Cycle time compression may be the most underestimated force in determining winners and losers in tech."

Marc Andreessen

Namely, in the software engineering context, Cycle Time measures how long it takes from software to go from starting development to running in production. It measures the time it takes to develop the software itself and any subsequent steps (build/test, code reviews, idle completion time, etc). In other words - how quickly can you iterate software from a software engineer starting work to it running in production?

By then drilling down - Cycle Time can help you identify blockers in your software engineering process. Elite teams typically see a Cycle Time of just a few hours.

"Gaming" Cycle Time

I recently was listening to a webcast on software engineering metrics, and the topic of Cycle Time came up. One guest mentioned concern that Cycle Time as a metric could be "gamed" by teams dividing their work up into smaller user stories.

Far from this being an unhealthy activity, this kind of "gaming" is essential to having an agile software development lifecycle.

Students of agile production techniques like Lean production or Theory of Constraints will understand the critical importance of smaller batch sizes. Likewise, for software engineering teams, ensuring that work is broken down correctly is essential.

It is of course essential that Cycle Time isn't taken in isolation - speed metrics (like Cycle Time and Deployment Frequency) must be taken in tandem with quality metrics (like Change Failure Rate and Time To Recovery).

That said, there's nothing wrong with dividing up work into the smallest possible piece of deliverable business value. Indeed, DORA research has consistently shown that speed and quality go hand-in-hand.

Understand Risk

The CHAOS report shows us that, as projects are smaller, they are far more likely to be successful. Over 4 years, the CHAOS database found that whilst only 43% of grand projects failed, only 7% of small projects failed. Indeed, small projects are over 9 times more likely to be successful (on-time, on-budget, on-target) than their small counterparts.

We can see a lot of reasons why this is the case for software engineers. If someone is working on a large and substantial change to a software codebase, by the time they merge it in, there are likely to be merge conflicts in the source code itself which will require a lot of the software to be rewritten.

When creating small Pull Requests, there is far greater attention and understanding in the code review process - meaning defects are more likely to be found.

By instead making smaller changes, the work is less likely to conflict with the work going on elsewhere in the organisation and there is less that is likely to go wrong when something is ultimately deployed. When you deploy smaller changes, it is easier to diagnose and quicker to fix an issue when it appears.

Additionally, by using functionality like feature flags, you can get software running in production whilst isolating away the steps needed to make it generally available (which could require product announcements, training customer support staff and writing documentation).

This allows you to get the technical learnings from running your new feature in production whilst also.

Managing Work In Progress

Multiple psychological studies have shown that multitasking is harmful to productivity. Context switching has been estimated to take up anywhere from 20% to 80% of productivity depending on the number of concurrent projects.

A recent scientific study of software engineers perceptions of task switching found "that practitioners perceive task switchings are as disruptive as spontaneous and random interruptions, regardless of the source and type of the switching".

By breaking work down into smaller and more manageable components of work, there is space to allow for work to be reprioritised in an agile way without the costs of task switching.

Getting Faster User Feedback

I have however saved the best benefit for last. When you ship small, your product gets better faster. Shipping smaller means you get feedback from your users quicker. Early and quick feedback allows you to respond faster to customers changing requirements.

We live in a chaotic world and we need to test ideas to gain an understanding of what users want in the real world. By shipping fast, you get feedback fast. Before sinking huge cost into large-scale projects, you can test MVPs rapidly to get feedback fast.

These principles are codified in two of the four points of the Agile manifesto:

• Customer collaboration over contract negotiation
• Responding to change over following a plan

By using the principles of experimentation in combination with feature flags, you are able to A/B test features rigorously and incrementally rollout new functionality in a safe way. A/B tests are essentially Randomised Control Trials, the most rigorous form of scientific experiment, used to test drugs and vaccines when observational data doesn't cut it. LEAN principles make it easy to get this level of rigorous insight from your product data.

Conclusions

Shipping fast and shipping reliably go hand-in-hand. They are lessons which has been handed down to software engineers from engineers of all disciplines adopting LEAN production techniques. The processes of the Toyota Production System have been continuously refined and simplified to lead us to Agile software development and the DevOps revolution.

Shipping small allows us to prevent burnout, experiment faster and ship reliably.

For further reading on this subject, I'd recommend the following books:

• The Goal by Eliyahu Goldratt
• The Pheonix Project by Gene Kim, George Spafford and Kevin Behr
• Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations by Gene Kim, Jez Humble, and Nicole Forsgren

‍