At IBM, we think of DevOps as a way to provide a frictionless path for ideas to go into production. The faster you can get real feedback in a production context, the faster you can learn what you should have done and the faster you can adjust. One of the main benefits of the DevOps way is this ability to learn quickly, then create and iterate on the product that the market wants as quickly as those needs are revealed.

Much of the attention of the DevOps community so far has been around (see links at end):

  • Shifting Left–testing earlier in the development process and automating it.
  • Infrastructure as Code (IaC)–having software defined environments that can be stood up reliably and repeatedly through automation.
  • Automating the pipeline and delivery of products and services.

And we still have a way to go with these efforts, as indicated by DZone’s latest Painsylvania infographic, because these are among the top issues that teams see in being successful with DevOps.

painsylvania infographic

What might surprise you is the next thing in line: requirements issues.

As teams finish their DevOps pipelines and solidify their practices and culture around delivering quickly, the pipeline starves. In the words of Meat Loaf, they’re all revved up with no place to go. You are at risk of having a high-performing team with nothing to do. This issue with requirements was echoed in the StackOverflow 2016 Developer Survey. Over 50,000 developers responded to the survey and more than a third said “Unspecific Requirements” was one of their top 3 challenges at work.

I’m not surprised by this.

As a developer for at least a couple of decades, it has almost always been an issue. Very early in my career, I authored some of those requirements documents (that we now talk about as relics of a process long forgotten). It was difficult work. Stakeholders could only review documents full of text and diagrams, never anything working or more tangible than a screen wireframe. Even the “language” was a barrier – you needed to write for developers while getting approval from business people. Too much was lost trying to bridge communication between disparate communities as time passed and requirements shifted.

Extreme programming and agile development sought to improve this through the use of more multidisciplinary teams and shortening of the gaps between ideas and feedback from their implementation. This did improve results for many – and brought us the DevOps movement as “potentially shippable increments” piled up with nowhere to go. Continuous Integration is nice, but it is simply a necessary precondition to Continuous Delivery.

AgileScrumXP brought us user stories as a lighter weight (yet sufficient) way to capture user requirements and communicate them to the development team. A user story typically starts with a summary that follows the pattern:

As a <role>, I need to <goal> so that I can <achieve some business value>.

A typical project starts with a story storming session that generates many of these story summaries to be stuffed in the product backlog and is later prioritized by the product owner. At regular intervals, the team refines the backlog, re-ranking stories based on changing market conditions and development priorities.

A typical sprint begins with the team looking at the top-ranked stories, planning what needs to be done to complete them, then figuring out how many they can complete in the coming sprint…

And if you’ve ever been in a typical sprint planning meeting, you know what comes next. That summary statement mentioned above is all that’s in the backlog. A team of folks stares at that summary, trying to figure out how to write tests and code later today and finish that story by Thursday. But it’s not going to happen. Three hours later, they quit the sprint planning meeting to get lunch, hungry and dispirited that they are still trying to figure out what they are supposed to do. Perhaps 5% of their two-week sprint is already gone and they produced nothing.

Requirements Issues – sucking momentum from your teams since forever (and still at it). I read a quote recently (that I wish I could attribute):
“We’ve gotten really good at delivering the wrong things quickly.”

So what do you do? There are likely as many opinions on that as there are frustrated developers and coaches and consultants who want to help you.

Here’s my recommendation:

  • Respect your backlog and take it seriously so that teams can pull work.
  • Use Behavior Driven Development to define your Acceptance Criteria.
  • Automate it.

Respecting your backlog

developerWorks blog pipeline
Your backlog is the fount of ideas that feed into your DevOps pipeline. It doesn’t need to geyser, but don’t let it trickle. It must be ranked, not just prioritized. The next most important things must be at the top. Ideally, your teams should be able to go as fast as they can. This means they need to be able to confidently grab the top item from the backlog as soon as they are ready for new work.

For this to work, the top items in the backlog must be actionable (and identifiable as such). Exactly what actionable means in your context is up to you.

I think it should mean that each story has:

  • A descriptive summary with a crisp definition of business value.
  • A description of the general flow and edge cases to be concerned with.
  • A specification of any critical performance characteristics.
  • Acceptance criteria that everyone agrees will complete the story when met.

If these conditions are not met, the team should not work on the item. How can they? Without a crisp definition of what it means to be done, what’s the point? Don’t value activity over progress.

Part of respecting your backlog is to make sure that the backlog refinement meeting happens as needed. The team and product owner should look ahead and predict when they will run out of actionable stories. They then need to define the next collection before that happens. Don’t make the mistake of trying to make all of them actionable or not putting them into the backlog until they are actionable. With each iteration, you’ll learn about your product and what’s next most important might change. Keep enough stories ready so you don’t starve the team’s pipeline.

Behavior Driven Development (BDD)

BDD as a practice is intended to improve the communication of requirements between business people and developers; to create a shared understanding by using a common language. It is meant to create a directed discussion around the behavior of features and to capture it in a structured way that can (and should) lead to automated acceptance tests. It is focused on building the right thing. Coupled with DevOps, it helps you build and deploy the right thing quickly.

Serious coverage of BDD will have to wait for a future post. For now, I’ll offer my favorite books on the topic and an example from one of our workshops.

As an example, we used a Blackjack/21 card game exercise as part of our camp exercises. It was the basis for our pair programming and TDD and BDD exercises. It was also something we turned into microservices to simulate strategy evaluation.

One of the user stories was: “As Dave (a bored programmer who often goes to Las Vegas for conferences), I want to evaluate various Blackjack strategies so that I can win a lot of money, retire early, and take extravagant vacations.” Each team had to choose a strategy and implement it to play with a dealer service. BDD tests were used as acceptance criteria to prove that the code properly implemented the selected strategy.

To illustrate how this worked in a final solution, we located a strategy table online which would likely be similar to how a product owner might specify a winning strategy. The table looked something like this, with versions for when you have an ace (as a “soft” count hand) and when you don’t (a “hard” count hand).

Dealer’s card
2 3 4 5 6 7 8 9 10 A
4-8 H H H H H H H H H H
9 H Dh Dh Dh Dh H H H H H
10 D D D D D D D D H H
11 D D D D D D D D D H
12 H H S S S H H H H H
13 S S S S S H H H H H
14 S S S S S H H H H H


Using the Python Behave library and Behave’s Scenario examples, we easily mapped the table to BDD examples and code. We did a straight translation although you could probably do something more clever.

The Gherkin language (processed by Behave) uses a specific syntax for describing a concrete scenario. The basic shape is:

Given some precondition (the context)
When some action occurs (the event)
Then some testable result is achieved (the outcome)

It is easily augmented using a table of data with examples that fit the scenario. Each line of data represents a test case. Here’s a snippet of the Gherkin used for the table above:


Scenario Outline: Respond using the defined strategy for simple hand without aces
Given a player hand value of handval> and a dealer card value of
When the player app is asked to decide
Then the player app will respond with <action>
Examples: Hand values without an ace
| player handval | dealer cardval | action |
|       4        |       2        |  hit   |
|       4        |       3        |  hit   |
|       4        |       4        |  hit   |


The result was nearly 700 tests that ran on every code update and indicated if the code met the defined strategy. Knowing that your code behaves as expected is the sort of coverage that makes you comfortable shipping.

BDD works for many situations. Even if, for some reason, you don’t turn it into automated tests, the thinking and conversations required to be specific and create concrete test cases make your User Stories actionable and provably acceptable.

Automate it!

If there’s a TL;DR for DevOps it would be “automate all the things”.

In the context of requirements issues, this means having a dashboard where you can see the actionable status of the top stories in your backlog. Maybe even an alert if the number falls below some threshold. Starving your pipeline is a blocking issue and should become an all-hands-on-deck emergency.

Because you’ve written real, concrete acceptance criteria that illustrate your requirements, take the last step and automate them. Just about any serious language in production use is going to have a BDD compatible library. Find yours and use it (some links below). Leverage your DevOps pipeline to retest your acceptance criteria constantly.


You’ve done all the hard work. You’ve built a great DevOps pipeline and your team is revved up to work at top speed with high levels of quality. Don’t starve your efforts by being unprepared to provide your team actionable work.



Respecting your backlog (and your team):

Shifting left:

Infrastructure as Code:

Behavior Driven Development:


Join The Discussion

Your email address will not be published. Required fields are marked *