Lyft, Netflix and Amazon have made on-demand services the status quo. Consumers tap their iPhones and expect instant gratification. This means businesses of all kinds are now expected to provide real-time products and services that offer amazing consumer experiences.

But most enterprises aren’t very good at shipping software. And in today’s competitive landscape with rising customer expectations for better, faster experiences, those organizations that fail to ship software quickly and safely will be disrupted by businesses who can outperform evolving customer demand.

This means IT is under unprecedented pressure to move faster and deliver software and services that drive business value.

Capitalizing on disruption with DevOps

According to Forrester, business technology decision-makers plan to differentiate their companies from competition primarily through innovation and IT agility. However, their software and staffing budgets are not growing significantly.

Gartner recommends, “IT leaders should start a DevOps initiative to minimize their risk of not developing capabilities needed for an increasingly competitive and time-sensitive business environment,” and proposes IT leaders justify their DevOps investment, “based on time to production and business value and impact.”

DevOps is a cultural and professional movement focused on how we build and operate high velocity organizations, born from the experiences of its practitioners. DevOps is the way IT must operate to be the business engine the digital economy requires.

How do you measure the success of DevOps initiatives?

Metrics for the sake of having metrics is a waste of time and money. Metrics are designed to facilitate insight into quality and how to improve it, so they need to effectively measure the impact of our products and services on customers and the business. The best way to ensure businesses are competing in a digital and on-demand economy is to focus on velocity — or the time from idea to when software is shipped to users.

The three critical metrics to deliver on heightened expectations include speed, efficiency and mitigating risk.

1.     Improve speed at which your organization delivers change. Speed is about two things: how often you are deploying changes to keep up with customer needs and how long it takes to deploy a finished feature. Companies can achieve speed by deploying updates frequently, approving and building changes faster and provisioning new environments quickly. A great example of improving speed through DevOps practices is Standard Bank, the largest bank in Africa. Standard Bank found they were operating three times slower than U.S. counterparts, which was costing them potential customers. So the bank adopted DevOps methodologies – which shortened cycle times – to gain a competitive edge. Standard Bank now pushes ideas from commit to deploy in roughly 18 minutes!

2.     Improve team efficiency. IT organizations are categorized based on how efficiently they manage applications. Efficiency should be measured by how often changes which are deployed fail and how quickly you are able to resolve those failures. Companies can achieve efficiency by collaborating across the organization, setting definite and executive standard workflows and regularly auditing effectiveness of automation. Intuit, the number one online accounting solution worldwide, was challenged by team siloes and redundant efforts which slowed failure resolution. By embracing DevOps, Intuit was able to reduce change failure rate by 90 percent.

3.     Mitigate risk and ensure changes adhere to compliance policies. High-performing organizations pull information security into the development process earlier by automating compliance tests. Becoming a high-performing and risk averse organization is about how frequently compliance assessments are run and how quickly vulnerabilities can be remediated. Companies can achieve risk mitigation by packaging compliance as code, automating compliance testing, and logging and auditing policy failures for compliance remediation. Equifax, a leader in workforce information solutions, organizes and assimilates data on more than 820 million consumers, but by embracing compliance automation, the company can easily scan and maintain security policies.

These metrics are one cohesive set. Companies cannot pick and choose a la carte, otherwise they will not be laddering up to the larger outcome of building a high-velocity organization. Outperformance is correlated only with organizations that achieve all of these metrics. The outcomes of effectively implementing DevOps mean increased speed, efficiency and decreased risk. Measuring DevOps performance allows teams to analyze where they are succeeding and where improvements need to be made so companies can continue to drive innovative solutions to exceed customer needs and ship better software experiences faster and more reliably.

Nathen Harvey’s InterOp ITX presentation – The Path to Next-generation DevOps,
What Really Works

Nathen Harvey presented with Chef Director of Customer Outcomes Michael Goetz on “Defining DevOps Metrics” at InterOp ITX May 15-19 at the MGM Grand in Las Vegas. Review the presentation to learn more about how to identify and measure DevOps outcomes.


Additional DevOps expertise on developerWorks


1 comment on"How – and why – to measure DevOps outcomes"

  1. > … from commit to deploy in roughly 18 minutes!

    I assume that’s commit to master, after all the dev activities, pull requests, etc.

Join The Discussion

Your email address will not be published. Required fields are marked *