Requirements remind me one of The Simpsons episodes, where Bart reads outs loud junk mail: "Gas your termites. Freeze your termites. Zap your termites. Save your termites?" and it feels sometimes that writing good requirements is more of an art form than a teachable skill.

In the world of agile development, requirements are often seen as something cavemen used to gather, but in reality user stories serve the exact same purpose as requirements, with the intent to avoid the black magic that requirements are made of.

This post describes some of the practical usefulness in well written requirements that I found for myself and how requirements can make managing projects a bit more straightforward.

What's a requirement?

A requirement describes what an application is expected to do, but without providing any implementation details on how to achieve it. The purpose of a requirement is for Product Owners to be able to throw new ideas into the backlog, without having to spend time on figuring out how to achieve the goals behind those ideas.

For example, an online shopping website may have a requirement:

• The app shall allow visitors to select items for an immediate purchase

, which leaves out any implementation details, such as whether the selection of items should be implemented as a shopping cart or as a pending order.

Development teams pick up business requirements and design solutions for each, which may be standalone documents, Wiki pages, sets of tasks with solution implementation details, detailed comments in user stories, or even document-as-you-go approach when a solution is worked out in the code as it is being developed.

For example, the requirement above may result in these tasks.

• Implement UI to select and add items to a pending order
• Implement UI to show the pending order summary
• Implement a pending order with selected items maintained per session in the backend
• Implement API to add/remove selected items in the pending order

Once a solution is available in any form, a feature may be phrased for the release notes, including important implementation details that could not be described in requirements, such as supported payment methods, how a shopping cart can be saved, whether pending orders may expire, and so on.

A requirement should be focused on a specific desired capability and should never combine multiple topics, so each requirement can be included or excluded individually from their target milestones.

A requirement should be verifiable. Marketing-speak phrases, such as that the app shall do something efficiently or quickly or easily, do not belong in requirements.

How do user stories fit into this?

In practice, it is not always easy to avoid implementation details in requirements and still describe the desired behavior in a focused and unambiguous way.

For example, this authentication requirement:

• The app shall provide a way for users to authenticate

, lumps different types of authentication into one requirement (e.g. single sign-on, user credentials, etc), while this one:

• The app shall allow user authentication with a name and password

, describes implementation details that may get in the way (e.g. if an email address is expected to be used for user identification).

A good way to verify whether a requirement is phrased well is to ask yourself why is it needed, which in this case would be to allow human users enter their credentials interactively, so a requirement could be phrased as

• The app shall allow users to authenticate with manually entered credentials

, which leaves authentication design to the architects, but limits it to one specific form of authentication.

User stories are supposed to make requirements more relatable by spelling out how application users would benefit from the required behavior.

For example, the online shopping website requirement might be phrased as a user story

• As an online shopper I want to select items for an immediate purchase

Similarly, the authentication requirement might be phrased as either of these.

• As a user I want to authenticate with a name and password.
• As a user I want to authenticate with manually entered credentials.

Phrasing requirements from the user's point of view makes the person creating requirements focus on why this behavior is needed, rather than on how to achieve it.

Where user stories fall short

User stories are easier to phrase, compared to requirements, but they present a couple of problems that cannot be fixed with clever phrasing.

One of the problems is that the way user stories are phrased, makes it vague where in the application the work is supposed to be done, because a user story is about about the user, not the application. A typical requirement will be phrased either for the entire application or for one of its subsystems, which makes it simpler to navigate and interpret requirements in order to figure out their area of intended work.

Another problem is that sometimes business requirements need underlying functionality designed in a particular way, which may be tracked with architectural requirements, but these type of requirements cannot be easily expressed as user stories.

For example, multiple application subsystems will interact with a database management system, so a development team may create an architectural requirement

• The app shall provide a common database layer to be used by all subsystems.

The user in this case is the developer and the user story would be phrased along the lines of

• As a developer I want to use a common layer to read from and write to the database.

, which sounds odd and would be confusing when mixed in with business user stories.

Many Product Owners I worked with had a hard time tracking architectural work with user stories and often either tucked them into business user stories as overhead or avoided architectural work altogether until it was impossible to ignore those issues and then tracked them with bugs.

Contrived Requirements

A typical book about requirements would have an example on how some requirement would be written such that the system shall use a database, which is really an implementation detail for an actual requirement, which should have been phrased along the lines that the system shall use tabular data store that can be queried and updated.

While technically, this sentiment is correct, requirements for well-known designs look contrived in that the solution for such requirement is known well before the requirement may even be phrased in a good way.

In practical terms, choosing the database management system is more of a task for a system architect to select the type (e.g. SQL, NoSQL) and the distribution (e.g. Postgres, SQL Server, Mongo DB, etc) of the DBMS for the application to use, rather than to figure out how to implement some tabular data store.

Some of these well-known implementations phrased as requirements are harder to spot. For example, this requirement:

• The app shall run as a systemd service

, is actually a task that describes a specific implementation for these requirements:

• The app shall run without a user being logged in
• The app shall provide a way to stop it gracefully

The line between tasks and requirements for well-known implementations may get quite blurry and the trick in figuring out whether just to throw in a task or to design a solution, in my experience, is to distinguish actual tried-and-true implementations, like those above, from those a Product Owner spotted on some website they like. Phrasing the latter as tasks will take away the design step from the development process and will reduce the app to being a patch quilt of solutions copied from other applications.

User stories and tasks

Some of the issue tracking systems use confusing terminology when defining work items and it makes it harder to track work and organize it on agile boards.

One of the large companies I worked for used work items as if they were both - a requirement and a task, so a developer would be assigned a work item and it would appear in their To Do column on the agile board and when they were done with their work, they would move it from In Progress back into the To Do column, but assign it to a QA person, so their work could be verified. This created a huge mess for people having to hunt their work among a mix of new and implemented work items in the To Do column.

One other company came up with a clever way to keep a work item assigned to the same developer with extra custom fields for an assigned code reviewer and an assigned QA person. This way work items remained in their columns and people in those additional fields could pick up work in their agile board column, such as In Code Review or In Testing.

Tasks are quite useful in allowing multiple people to work on the same user story without having to create any custom fields or using other creative ways to track their work.

A task is a specific work assignment for one person that needs to be completed for a user story. At a minimum, a task may be in a New, Active or Completed states. Notice that a task is never verified - the parent user story is verified instead, so a task cannot be failed in any way because it just tracks some work being done, not the results. A task may only be completed or abandoned.

For example, an architect may create a task for a user story to allow users select items for purchase:

• Implement UI to select and add items to a pending order

, and assign it to a specific developer. When the developer completes this task, they change the state of their task to Completed and create a new task for QA to verify development work using the acceptance criteria from the user story. If QA find any bugs, they create one or more tasks for the developer to address those bugs.

The parent user story is typically moved back to the Active state while bugs are being fixed, which makes it easier for QA to pick their user stories and tasks among user stories in the Resolved state.

A task should never be titled using a user story rephrased as a task, such as "Implement UI to allow visitors to select items for an immediate purchase", and instead should be specific in describing the work that needs to be done, but without micromanaging the implementation.

A task may be created for the initial research work as well, which will be tracked under the user story for which the research is being done, such as to evaluate some framework or research a design approach. Contrast this with commonly used research spikes created at the same level as user stories, which makes it more difficult in tracking the link between research spikes and their user stories.

Standalone research spikes may be useful when there are no user stories in some area and some research work is needed to figure out what user stories that may be created in that area. For example, in a large dataset may be explored using various statistical methodologies in order to figure out if there are any useful trends, and maybe one or more user stories may be created to implement some form of visualization for those trends in the application.

Features

A feature is an implemented application behavior that is listed in release notes and described in detail in application documentation. This means that a user story title is not suitable for release notes because it will not have any implementation details.

This point is being missed by far too many Product Owners in that requirements are created first, then solutions are designed and tasks created and only then features emerge and will be further refined while the tasks are being worked on.

The relationship between user stories, tasks and features can be described with this diagram. Multiple user stories may contribute to the same feature and multiple tasks may be created to implement requirements and verify all the work.

In practical terms, some features can be described in generic terms soon after their user stories are created, some can be maintained as placeholders until they can be described in a meaningful way, possibly grouping temporarily a few related user stories, and some can be delayed entirely until the implementation is clear, and then a feature could be created with a title reflecting the implementation and the relevant user stories can be added to it as children.

I find that using placeholder features keeps user stories better organized while they are being worked on, and, as an added benefit, it allows aggregating the effort in included user stories by feature area, initially, and then breaking it down onto individual features.

For example, until all authentication methods are designed for their user stories, a placeholder feature named "User/client authentication" can be created and all authentication user stories can be included into this feature. As authentication design solidifies, this placeholder feature can be split onto specific features, such as one for user name and password authentication, one for single sign-on authentication, one for bearer token authentication, and so on.

Sprints and releases

Requirements may take significant implementation effort and may span multiple sprints. A well written requirement cannot be split onto smaller requirements because it describes a single specific desired application capability.

This means that it should be expected that user stories, which reflect application requirements, will move from a sprint to a sprint, as their tasks are being worked on.

Tasks, on the other hand, should be structured such that they can be completed within a sprint or two, so they serve as breadcrumbs for their user stories, as user stories move between sprints.

Each task should always have the amount of time, in days, it took to complete that task. See Five days till release for insights on days vs. hours. The time for each task will be rolled up into user stories to show how much time each user story took to implement, which in turn will roll up into their features, creating a very accurate picture of how much effort, in calendar days, was put into each feature.

Features are planned for a specific release, but unlike user stories, which stay in the sprint where they were completed, features just need a release indicator, which is typically a release version, and don't need to move between sprints.

Having a release version in features makes it easy to create release notes for a specific release - one just need to query features with that release and get a complete list of properly phrased features, along with all implementation details that are worth mentioning in release notes.

Bugs

Bugs are requirement level work items that describe application behavior that deviates from business or architectural requirements. Bugs may be created for released application features, when these features cannot be reworked within their original user stories, or for unreleased features if their primary user stories have been verified and closed.

More specifically, if a bug is found in a user story that is being actively worked on, a task may be created to address this bug. A closed unreleased user story may even be reopened to fix a newly found bug within a task or a bug can be created for an isolated issue that can be tested on its own. A released user story cannot be reopened (i.e. same functionality cannot be released twice), so creating a standalone bug is the only way to go about it.

For the purposes of release notes, it makes little sense to create a feature for each bug, so a query to list all bugs for a specific release often works out better, even though bug work items are not at the same level as features.

Worth noting that a bug created to avoid reopening and re-verifying a user story implemented in the current release may look odd in release notes because the affected feature was never listed in any of the past releases. Filtering out such bugs from release notes or avoiding to create them in the first place by using tasks within the original user stories will keep release notes cleaner.

Commit References

Having features, user stories, bugs and tasks raises the question of what should be used as a work item reference in source code commits.

Features cannot be used in commit references simply because they may not even exist when user stories that will eventually contribute to these features are being implemented.

Tasks sound like the natural reference to identify which work is being done, but this is actually a bad idea because using task numbers in commits will fragment work item references and will make it hard to trace them back to original user stories.

Using requirement level work items, such as user stories and bugs, in commit references works out naturally in grouping changes that implement a particular requirement and can be interpreted intelligently even if the original bug tracker system becomes unavailable, which happens for long-running projects more often than people realize.

In order for commit work item references to be useful, commits should be structured such that changes being committed contribute to the implementation of a single work item, such that listing commit subjects alone would paint an accurate picture of which requirement is being implemented.

Sometimes developers list multiple work items in a single commit. This is a terrible practice that creates oversized commits without clear boundaries between code changes and requirements these changes are intended to implement and it creates a mess of multiple user stories referencing the same code change as their implementation.

The almighty ticket

Some teams use just one type of work item, which is often called a ticket or an issue. A ticket may be phrased as a requirement, a task, a bug, a feature - anything really, and may be good enough for teams that want to use a simplified work tracking process. There are a few things to consider in this case.

Such teams are more likely to have technical designs originating from one or two technical leads, so there's no need to worry about having implementation-free requirements, as issues are likely to be phrased to include fairly detailed technical guidance on how things are expected be done.

Without using tasks, assigning issues may get confusing, because it is not always clear who is supposed to take the issue after the next step is completed.

For example, if an issue was assigned to a QA person for verification and was reopened, they have to dig through the history to see who was working on the issue before they got it, so they can assign it back. One workaround for this is to add custom fields, such as Assigned Tester and Assigned Code Reviewer, which work out as surrogate tasks.

Tasks also provide good time tracking for planning purposes. Using calendar days generates comparable numbers between tasks, which roll up into user stories, features and epics, so one can tell how much effort a particular feature required. Tracking time in a single issue, especially in hours, produces numbers that cannot be used to figure out which activity took time, compared reliably or used in planning for similar activities.

Preparing release notes with generic issues is not a trivial exercise and requires somebody to decipher each issue title, decide whether it should be included, and, for some of the issues, translate titles into meaningful features for the release notes. Compare this to just querying for features, which are phrased as features and can be used as is in release notes without editing.

Worth noting that using just requirement-level work items, such as user stories and bugs, and avoiding the overhead of maintaining tasks and features, isn't the same as using generic tickets for all types of work because a list of user stories and bugs still provides good consistency in what work items in such list represent, while a list of generic tickets needs to be parsed to figure out work items that do not contribute to whatever purpose such list was generated for (e.g. release notes).

Azure DevOps

Azure DevOps offers multiple ways to track requirements and features and their Agile process accommodates most of what is described in this post.

Epics in Azure DevOps allow grouping features, which makes it easy to track work time spent on specific areas, such as authentication or logging, and may include multiple features with their rolled up time for user stories and bugs rolled up from tasks.

One interesting work item type in Azure DevOps Agile process is Issue, which is intended to track unexpected implementation blockers, such as a broken computer or an inaccessible cloud account. The term issue is often used as a generic way to describe what Azure DevOps calls work items, and it surprised me at first that issues do not have a New state, but then I realized that one cannot plan for issues and they just happen, so Issue in Azure DevOps starts with an Active state.

Azure DevOps works out quite well for website type of applications, which do not maintain past versions, but will present challenges if more than one application version is being supported.

This stems from the way Azure DevOps tracks development milestones, which is called Iteration Path and reflects a milestone for a specific team, as well as a sprint.

For example, if sprints are named after an ISO-8601 week number within their year, an iteration path for a version 2.0.0 of a project Abc might look like this - Abc\2.0.0\2022\W25. User stories will be assigned a full iteration path, as they are being worked on in particular sprints, and features will be assigned just the path with the version - Abc\2.0.0, so they appear in the team backlog, but don't need to be moved between sprints.

The problem with this setup is that if there is a bug in the version 1.2.3 that needs to be fixed, the Product Owner will need to plan for this bug fix to be included in versions 1.2.4 and 2.0.0 and while the latter works out well as a milestone within the iteration path, Azure DevOps provides no way to track 1.2.4 as an additional release within a work item.

Azure DevOps does provide Release Pipelines for application releases, which are different from build pipelines, and the names of release pipeline stages with the release name are shown in work items, but this information is shown only after a release pipeline runs, so there is no straightforward way to plan application releases within work items.

Release pipelines may be tied to branches, which do appear in work items, but a release branch may be created long after a work item is planned for a specific release and this does not help much with planning the same work for multiple releases.

One of other shortcomings in Azure DevOps is a lack of customizable resolutions for work items. Azure DevOps provides a list of possible resolutions, which may work for simplified work item state transitions, but for those who want to specify accurately work item resolutions, it will present challenges.

For example, in a Jira's classic workflow, a resolution is associated with a work item, not a transition, so moving a bug through New -> Active -> Resolved or through New -> Closed pops up a form to fill out the resolution for the work item and moving from Resolved to Closed does not because the resolution has already been filled in. In Azure DevOps resolutions are associated with transitions and hard-coded with values that make little sense for some transitions, such that it is not possible to close a bug from the New state with a Won't Fix resolution.

Jira

Jira is trying to deliver everything for everyone and ends up being a very confusing solution for structuring work in the way described in this post.

That is, work items that belong in different levels, such as features, user stories and tasks, are all offered at the same level in Jira, which makes very little sense in terms of organizing them as tasks contributing to user stories, which in turn are contributing to features. It took me some time to realize that in addition to a Task Jira also has a Sub-task, which is similar to the Azure DevOps task.

It is no surprise that many Product Owners using Jira combine requirements and features in the same issue, which typically is tracked as a user story. On top of that, feature-type issues may be mixed in, which only can be connected to user stories via linking and offer no structural view onto how user stories contribute to feature. Features intermixed with user stories are hard to manage on story boards that are set up not to filter out features.

One way I found to organize Jira to track user stories in a structured way is to set up a workflow to only allow user stories, bugs, epics and sub-tasks and use epics as features to group user stories that contribute to the same feature. If a story board may be configured to filter out features, the workflow may be set up with user stories linked to features via implements/implemented by, which, even though technically incorrect (i.e. sub-tasks implement user stories), allows the team to focus on requirement-level work items on story boards and list features and their contributing user stories via filters.

[2024-04-07] Recent releases of Jira replaced the Epic link field with Parent, and allow to select features as user story parents in Premium-level subscriptions, which sounds promising for the purposes of linking user stories and features more logically.

Sub-tasks work out the same way as tasks in Azure DevOps and allow time tracking, even though there is no way to set up work time to be shown in days. However, Jira rolls up large number of hours into days, so it sort of works out the same way, except that one must ignore actual hours and always treat them as parts of a calendar day in the full day being a setting in Jira.

Jira Tasks may be useful to avoid having to come up with user stories for simple standalone activities, such as setting up a cloud VM for a one-off demo. However, standalone tasks often tend to turn into a pile of unspecified work activities that cannot be linked to any business or architectural needs and may create more of a project mess than people realize. When using Tasks, it is a good idea to review each new one to see if it should rather be a sub-task in some user story or a user story on its own, properly rephrased as such.

One area where Jira works out really well is in tracking multiple versions via the Fix versions field, so one can add 1.2.4 and 2.0.0 and track them within their releases.

Jira also maintains the work time log separately from issue comments, which makes it easier to navigate work log entries, compared to using issue comments to describe ongoing activities.

One thing I find quite messy with Jira is that for simplified workflows they renamed the Closed state to Done, which completely disregards the resolution in work items and make closed work items with resolutions other than Done look odd, such as Done, with a resolution Won't Do or Cannot Reproduce.

GitHub

GitHub issue management is as basic as it gets. Computer-managed sticky notes, really. Issues may be assigned labels (tags) and may be set up with a milestone.

Only one milestone is allowed, so it won't work for version tracking if multiple application versions are supported. Labels may be used as a workaround to track multiple versions.

One nice thing about GitHub issues is that more than one person can be assigned to it, which allows a QA person to be added without replacing the original developer and makes it easier to move the issue through the workflow between different people.

GitHub also provides agile boards and backlogs within projects, which allow issue ordering and may span repositories. Projects make GitHub usable for planning work and prioritization.

Conclusion

Requirements are often perceived as remnants of waterfall development and as something that is not necessary in agile development. In reality, agile development merely calls for requirements to be considered as a fluid list of desired application capabilities that can be modified at any time while development is moving towards a release, rather than a rigid list prepared before any implementation work can start.

Requirements, whether they are phrased as requirements or user stories, play an important role of capturing desired capabilities at early stages of planning and throughout development, without having preconceived bits and pieces of implementation getting in the way of architectural solutions for these requirements.

Writing good requirements is hard and, without a question, some projects will be delivered faster without the overhead of planning requirements and tracking their implementation with tasks. In fact, much of Open Source development operates this way, but then again most Open Source projects run on time, not on money, and may not need to track how development efforts were spent across released features and bug fixes.

Most of commercial software development teams, however, would benefit from following well documented development processes, sometimes this will even be required, such as in financial and medical software development, and having requirements, tasks, bugs and features organized in a logically structured way helps to achieve just that.