BDD Acceptance Criteria Pay For Themselves Multiple Times

[Note: This post is a mirror-post of Paul Hammant’s site. He is a colleague, good-friend, the lead author of this article.]

For two previous blog entries on Story writing in advance of development, here and here, Jeffrey Davidson was my collaborator and editor before publication. Jeffrey also wrote a companion piece to my argument for smaller stories here. He used to talk about these ideas at conferences with presentations like this one. Note, I have written before on BDD acceptance criteria for the well-known “Todo MVC” technology demonstrator app.

In this article Jeffrey and I are extending our conversation about Stories. We both firmly believe in using Behaviour Driven Development (BDD) as the model for acceptance criteria and we’re taking the previous “candidate stories” examples as a starting point. This sort of stuff isn’t our primary area of work anymore, but it is fun and crucial to smooth flow of business ideas into production. And from that, we believe, gives a high return on investment, regardless of whether you intend to use the same BDD acceptance criteria for test automation.

Rationale for our belief

Bottlenecks can appear in any software development production line at any time. And as you solve them, a new bottleneck will appear somewhere else in the production line.

One common bottleneck is with stories written weeks or months before developers start implementing them. Such stories can be viewed as vague and underdeveloped (under-specified?) by the time developers and test engineers take a look at them in order to estimate them. Alternatively, the elaboration of the story is missing key concepts needed to meet the business needs.

Either of those issues lead to defects during development and waste later. Even when stories are well-phrased, misunderstandings may result in longer development time and unintended defects.

Classically, the defects we focus on are those in production code that reach production. Defects can also happen in test code too. Sure, test code doesn’t reach production as such, but work being marked as “QA approved” when that wasn’t correct, is a real second-order defect. This happens when people misunderstand what they are testing. The same thing can happen for manual testing as well as test automation, of course.

Behavior Driven Development (BDD) acceptance criteria

Dan North’s BDD encourages writing ‘scenarios’ using the words “Given, When, and Then” but otherwise uses plain English. Scenarios are perfect acceptance criteria. Writing them in the story definition in the story tracker (Jira, Rally, etc.) is expedient.

For the initiate, “Given” focuses on the system’s existing condition; the state before the user roles performs a specific action. “When” describes the action the user takes. “Then” describes the results the users sees after the system responds. In all cases, this should be from the user’s perspective and written in business language. It should not be overly technical, include a host of design elements, or written from the system’s perspective.

Note: For the rest of this article we are adopting the term “gherkins” instead of BDD acceptance criteria. This term was popularized by fans of Cucumber, the most popular BDD framework.

Worked Examples

The “candidate stories” article had the two stories from the Insurance domain. In that article they were referred by number. Both #11 and #12 and are worth expanding to have acceptance criteria in a gherkin style for this article. Note below that we’re using italics highlighting to hint at variance for lines we have used more than once.

Story #11, “pro-rata refunds on cancellations”:

Narrative (as before)

In order to meet regulations concerning termination of policies,
As an underwriter, I want to be able to cancel policies, effective
mid term, and offer a pro-rata refund of premium.

Acceptance Criteria (new)

Given the insured has an in-force policy
When they cancel the policy mid-term
Then the policy status is changed to canceled
And a pro-rata return of premium is returned to them

Given the insured has an in-force policy with half the annual premium unpaid
When they cancel the policy ¾ of the way through the term
Then the policy status is changed to canceled
And the debt owed is reduced by the amount of the pro-rata return of premium
And no premium is returned to them

Given the insured has an in-force policy with half the annual premium unpaid
When they cancel the policy ¼ of the way through the term
Then the policy status is changed to canceled
And a pro-rata return of premium less the debt owed is returned to them

Given an Insured with an expired policy
When they cancel the policy mid-term
Then they are informed the policy cannot be canceled after the natural expiry date of the policy

Story 12, “pro-rata claw-back on reinstatement”:

Agile’s INVEST mnemonic suggests we should snip stories up where we can. A reinstatement story is clearly something that comes after a cancellation story, and there is an order to that. Otherwise, there is a perfect INVEST separation to the two. You could, as a budding insurance company, have gone live without either and be busy adding them as people signed as ‘new business’. Strategies to cancel policies might include “do it on paper” if the cost for that isn’t going to hockey-stick too soon. Anyway, reinstatements come after cancellations:

Narrative (as before)

In order to correct erroneously cancelled policies,
As an underwriter, I want to be able to reinstate policies effective
from their cancellation date provided it is not in the past, and
claw-back refunded premiums on reinstatement

Acceptance Criteria (new)

Given the insured has a canceled policy with an expiry date in the future
When they reinstate the policy from the date of cancellation
Then the policy status is changed to active
And they are charged the previously returned premium

Given the insured has a canceled policy with an expiry date in the past
When they reinstate the policy from the date of cancellation
Then they should be informed reinstatements must be completed before the natural expiry date of the policy

A common BDD acceptance criteria pitfall

Referring to “Login” in the Given lines is a mistake many teams new to this make. Much of the BDD community now agrees: please try as hard as you can to not begin gherkins with “Given I am logged in.” Unless your scenario is about logging into the system, you can safely assume this has already happened.

When test engineers are automating the gherkin acceptance criteria, then tests should attempt to explore the functionality of the UI in a ‘microcosm’ stack that doesn’t require an actual login to test the focus area of the story. Your Given lines should assume the UI testing technology has a way of getting straight to the rectangle of the UI that is the subject of the test. I.e. bypassing a bunch of interstitial pages, including login.

The need for a glossary and style guide

A glossary and style guide owned by the Product Owner (PO), or Business Analysts (BAs) are “must haves.” The glossary is all the terms the business and users would use to describe the system. A style guide defines look-and-feel elements and key user interface functionality. Using a style guide allows gherkin statements to be simpler and avoids unnecessary wordiness.

A primary goal of gherkin statements is understandability; anyone with the business domain knowledge should be able to understand the steps and goals. Adding a glossary and style guide increase the team’s understanding of both the domain and the system. These documents also give guidance and will help the author write better acceptance criteria, improving them before review by the larger team. If you have a glossary and style guide it is important verify that new stories adhere to them before they are up estimation, coding, and test automation.

In our insurance examples here, multiple terms would have an entry in the glossary: claw-back, pro-rata, cancellation, reinstatement, premium.

Two styles of gherkin acceptance criteria

With projects ranging across many years and multiple industries, we have found the above gherkin style to be a very good way for introducing BDD to teams. While some story writers find this to be a bit repetitive, this is an easy technique for getting started and clearly communicating system needs (up to its limits).

Let us go further. There is no better or faster way to get a new business analyst up to speed, or a mediocre analyst up to ‘good’, than BDD.

The following style of gherkin has some significant advantages over the above examples. It provides the information needed for test automation – it has variables and reduces some of the repetition found above. This approach uses data to describe the variations that would otherwise be repetitive.

Here’s an example from Cucumber’s Scenario Outlines page:

Scenario Outline: eating
   Given there are <start> cucumbers
   When I eat <eat> cucumbers
   Then I should have <left> cucumbers
 
   Examples:
     | start | eat | left |
     |  12   |  5  |  7   |
     |  20   |  5  |  15  |

Our experience has shown it is easier for new teams to start with the simpler, language-only, non-data BDD. While it is possible to jump straight to the latter method, we find this stepping stone leads to better understanding of how to use this technique and increases adoption.

Starting with the former gives the entire team the basis for building their business domain understanding. It also gives testers and test engineers additional time to develop the data required for scenarios; data needed for functional testing.

We recommend teams adept with BDD shift to the second form earlier in the story elaboration cycle. That is, instead of developing the data during or after development is complete, the data scenarios can supplement or replace the text only versions prior to development.

Having scenario data prior to development is an outstanding way of ensuring developers have a clear idea of the system expectations. We have witnessed this greater understanding leading to faster development, better code, and fewer defects.

20 Quotes about BDD and Agile Analysis

Kickstart Academy, a training organization where you get access to experts started a podcast series and I was privileged enough to be involved recently. Watch (click the image above or watch on Youtube) as Chris Matts interviews Kent McDonald, Jake Calabrese, and myself about the use and misuse of Given-When-Then, Behavior Driven Development (BDD), and business analysis in agile.

 

There were some great quotes (tweetable moments) and here are 20 of my favorite:

“Use Given-When-Then after the conversation, not during”

“You mean the special, princess, unicorn BA”

“BDD is here to make Eric Evans’ dream — everyone on the team understanding the domain — come to life”

“There are no BAs in agile … is an agile fairytale”

“The tools have made [BDD] into a developer focused thing, and not an ‘understanding’ tool”

“I have never seen a method for getting a BA from brand new or mediocre to competent or very good faster than BDD”

“BDD will help you [BA] do your job better!”

“The people you call ‘old-school’ are your organizations ‘tradition holders'”

“BAs are risk managers”

“Alistair Cockburn got something right when he said analysts should be called Value Managers”

“All the successful people do different things”

“The BA needs to shift from focusing on features and what to develop to what not to develop, maximizing what’s not done”

“I always start with the final THEN and then ask them what gets us to THEN”

“It’s an interview of what you’re doing”

“Can you think of an example where this outcome doesn’t happen? Can you think of a different outcome to this situation?” hat tip to @lunivore

“Where do you engage BDD in the [development] process?” “As soon as possible. And as late as possible.”

“Where do you engage BDD in the [development] process?” “Always. I don’t even understand how you can ask the question.”

“If the system’s not behaving, what is it doing?”

“I think, on the BA side, context is even more significant than it is on the software side, the technical side”

“The tools that are most effective are fairly simple, but have enough subtlety and nuance that they can be very powerful”

 

Here what you need to find and follow this great group:

 

If you are looking for more on Behavior Driven Development, then please check out my BDD Resource Page.

Should the Product Owner test everything?

A scrum master I’ve coached recently sent me this question and I wanted to share my answer. Would you have answered the same way? What did I miss? What do you ask (demand?) from your product owner?

 

Question: Hi, Jeffrey,

Quick question for you: Does Product Owner (PO) approval need to be on a per story basis, per feature basis, or both?

We are facing a situation where some of the system environments were not in place and completed work has remained in Dev until today. We received today that the Test environment is ready. The stage environment is due to be completed at some point in the near future. Meanwhile, our team has modified the team’s “Definition of Done” so that completion criteria are more aligned with our capabilities within the framework of system environments being incomplete. Hence, the above question.

Signed,
The Client

 

Answer: Hello, Client.

First, it makes sense the PO is included in the conversation around “Definition of Done.” I’m not sure based on the question if they are in the loop, or not. I say this because the team is building and meeting expectations for the PO. It’s the polite thing to do to notify them and explain the new definition. In some cases, it may be more appropriate to ask their permission to change rather than simply notify them of the change.

Second, this change makes sense to me; you didn’t have the right environments previously and now you do. It makes sense the definition should change to accompany the environment and how the team is working.

Third, what’s happened to date and how much trust is there between the PO and team? If the PO has already tested all the existing stories, then they may not want to do more than audit the existing stories in the new environment(s). If the PO has trust in the team and testers, they many never do more than audit the stories. If the PO doesn’t have time, they may never get to more than auditing stories. In the end, it’s a great big “it depends” kind of answer.

What do I want from the PO? I want more involvement, as much as I can get. I want the PO to test every story as it’s finished and at least audit functionality and features as they are delivered. I don’t often get it, but it’s my request.

How do I split these 40 stories?

A student from last week’s Agile Bootcamp Class taught at Harvard University (yes, that Harvard) asks,

 

Question: I have a project that involves 3 different users who want approximately 25 new fields on an application. Most of these fields are view only with the exception of 4-5 fields which allows input changes. However, user #1 wants to see all 25 fields, user #2 wants to see only 10 of the same fields, and user #3 wants to see 5 of the same fields. How should I approach writing these stories? Would I write a story for each user and each field – which could potentially be 40 stories? Or should I just combine the fields that all 3 users would want to view? For example, one field might be “name”. My user story might say“As a user #1, user #2, user #3, I want to see all names of employees eligible for an annual salary increase so that I can view all eligible employees.”

 

Answer: This sounds like an interesting set-up. My answers, of course, will be a bit general because I don’t know all the specifics. Also, I will make more stories if the developers are unfamiliar with the systems / tables / data, and probably fewer stories if the team knows quite a bit about the different systems. With caveats out of the way, let’s dig in! First, 40 stories? Yuck. Too many. Second, because you have 3 different users, I would start with stories for satisfying all of them (unless there is a reason to focus on just one). My presumption is doing this adds value the quickest, which is my guiding answer for how to break up stories.

  1. As user #3, I want to view < information from 1 field> so I can do my job.
  2. As user #3, I want to see < more information > so I can . . . .
  3. As user #2, I want to view < even more information > so I can . . . .
  4. As user #1, I want to see < all the information > so I can . . . .

The point of #1 is to prove we can display information, while the other stories add more details for the same and additional users. None of these is about editing the data. Depending on what keeps the users and developers happiest, there are a couple of options. I can insert story 1A; Edit the first field. After this I would insert more edit stories, probably grouped like the last 3 stories above, as appropriate. A different approach might be to insert the edit stories after the last story. Again, where is the value?

A couple side notes:

  • It doesn’t make much difference if you use “see” or “view,” the goal is understanding, not the worlds best grammar.
  • I would be remiss if I didn’t mention the value statement in your user story is a bit weak. What is the specific reason to view eligible employees; the actual awarding of salary increases, a validation check, a fascination with other people’s salary, something else entirely?

Analysis In the Spotlight

It’s is with great pleasure that I announce Business Analysis is getting some official attention from both the Agile and Scrum communities! The first set of attention on us comes from the Agile Alliance . . . .

 

During this year’s Agile 2013 conference I talked with many great business analysts and product owners (including co-presenters Inger Dickson and Chris Matts, as well as Kent McDonald, Jake Calabrese, Ellen Gottesdiener, “Kupe” Kupersmith, and Leslie Morse). Toward the end of the conference I remember telling Kent and Kupe we should “take over” the upcoming BBCCon. (The Building Business Capabilities conference is the primary meeting for IIBA, Business Rules, and has tracks for Biz Architecture too!)

Agile Open Jam poster for BBC ConferenceMy simple idea was to run a conference within a conference. I don’t remember drinking during this discussion, but it certainly had all the bluster of a late-night, never-gonna-happen, kind of conversation. For those who don’t know, I am also known to push those conversations into the waking hours. So much so, I generated a list of topics we could share within our “sub-conference.”

As we all know, ideas are cheap. It takes execution to make something great. That’s where Kent McDonald shines. He took our conversation (I do not recall if I ever showed him my notes) and made it real with a proposal to form an official sub-group within the Agile Alliance dedicated to Analysis and Product Management. Further, he received official blessing, permission, and funding for us to share our knowledge at BBC and upcoming events.

Here is the official announcement:

New Agile Alliance Program

Analysis and Product Management in Agile

The Agile Alliance board recently approved a new program, Analysis and Product Management in Agile, with the purpose of providing a way for practitioners in the business analysis and product management communities to share stories, questions, and puzzles about using those skills in an agile setting and to share ideas between communities. The program Chair is Kent McDonald.

 

(Un)conference in a Conference
We’re pleased to announce that the first activity of this program is to facilitate an “(un)conference in a conference” at the Building Business Capability (BBC) conference where attendees can take part in conversations about the intersection of business analysis, business process, and business rules with agile principles and techniques.

 

The members of the Analysis and Product Management in Agile program will curate the outcomes of the discussions and make them available to Agile Alliance members on our website.

 

The Building Business Capability Conference is being held at the Mandalay Bay Resort & Casino in Las Vegas NV November 11 – 15, 2013.  It is the only conference that combines insight into Business Analysis, Business Architecture, Business Process, Business Strategy & Transformation and Business Rules & Decisions to facilitate creating the agile enterprise.  The conference is filling up quickly but registration is still available on the BBC Conference Website: http://www.buildingbusinesscapability.com/

 

We’d like to thank the International Institute of Business Analysis (IIBA) and the organizers of the Building Business Capability Conference for helping us put together a means for members of the analysis, business rules, and business process communities come together to discuss agile approaches.

Agile Alliance Newsletter, Oct 23, 2013

Special kudos go out to Kent for organizing this program and Ellen, who did most of the communication and coordination between the program and BBC. The (Un)conference will run in an open area outside the sessions and keynote Wednesday through Friday. I am one of the hosts, along with Kent, Ellen, Jake, and Mary Gorman.

If you want more information, here are the write-ups by Kent, Jake, and Ellen. Also, Yamo recorded a special podcast with Ellen and Mary. I joined for the last few minutes (at the 30:05 mark) to discuss the Open Jam.

 

Shifting focus to the other good news . . .

Apparently, the Scrum Alliance is also on board with our skill set, too! The next Global Scrum Gathering has a track dedicated to Scrum Product Owners (link to their call for Papers). I’m told this is the first time this has happened, previous conferences pushed their CSPOs into a track with other topics.

 

Note: I am a member of all 3 organizations discussed; IIBA, Agile Alliance, and Scrum Alliance.

How to Improve Your Requirements with Tiny Stories

I presented How to Improve Your Requirements with Tiny Stories at ProjectSummit & BusinessAnalystWorld held in Boston on October 21-22, 2013.

I firmly believe using the Given – When – Then of Behavior Driven Development (BDD) is relevant for business analysts. This talk is probably my best presentation of the material in the last few years.

You are welcome to view the accompanying presentation below. I sill share the link for the accompanying video upon request (and after I upload it to YouTube).

https://speakerdeck.com/jeffreygoodreq/how-to-improve-your-requirements-with-tiny-stories

Skills for Success: Professional Development Day @ IIBA Minn / St. Paul

I will be giving a new version of my talk about Behavior Based Requirements at this year’s Professional Development Day in Minneapolis / St. Paul. I am excited because I have reworked the entire presentation to both show my passion and give attendees a chance to practice using the Given-When-Then requirements format.

The conference title is Skills for Success: Evolving as a Business Analyst and my presentation is “How to Use Behavior Based Requirements to Build Understanding & Save Your Project.”

The event is on Monday, April 15 at the Earle Brown Heritage Center outside of Minneapolis (registration link). This is one of the premiere PDD events in North America and I am honored to be speaking. If you are in the area, please come, engage, and learn from the outstanding speaker line-up and local BA community.

 

Post conference update: Here is the published version of my slides: