5 key attributes of requirements testing: Know before you code

Most software bugs can be traced back to the requirements phase. That's why the best way to decrease the number of new open bugs in a project is to introduce a requirements analysis stage that your teams must complete before they start coding.

Bugs that track back to requirements are expensive to fix, and the cost of doing so grows exponentially over time. Fortunately, no one knows a product as well as the quality assurance (QA) tester who works with it every day.

Here are five key attributes engineers should check all specifications—whether for requirements, wireframes, or mockups.

1. Completeness

A requirement must contain all information needed for developers and everyone else who uses it to do their jobs. And a requirement should include all the details necessary to express the users' needs.

To make testing requirements for completeness more effective, use a heuristics testing approach. This targeted testing strategy relies on past data about probabilities, and it often allows for a more intelligent investigation of where bugs or problems might occur, even in terms of requirements testing.

Heuristics can help you find out what type of errors occurred, how frequently, and in what parts of your code (the affected areas of an application). It also helps you check requirements against an accumulated base of problems (gaps) that you might otherwise miss. You can use heuristics as a way to remind yourself not to forget to test something in your app, such as SFDPO:

• Structure: What is it?
• Function: What does it do?
• Data: What does it process?
• Platform: What does it depend upon?
• Operations: How will it be used?

You can invent your own heuristics and apply them to the whole application, as well as to separate functionalities (modules) and requirements analysis. On my current project, for example, we created checklist templates for each type of functionality.

From time to time we receive similar functionalities to implement, and we leverage these so that our QA teams don’t have to reinvent the wheel every time they want to test requirements.

Let’s say you previously tested a payment method called "credit card." For testing of a new method called "invoice," you might be able to reuse some checklists from "credit card." Once you create a new service or functionality, it becomes a potential functionality template. We create checklists for every type of task so as to not forget to test requirements against them.

For example, for popups, we use the SPACES heuristic. SPACES stands for:

• Several popups: Do you have any conflicts with existing popups in the system? How should you handle cases when two or more popup windows should be displayed?
• Places: Where do you need to display the popup? On the main page? The landing page? The search results page?
• Authentication: Do you need to display the popup for authenticated users only, or for everyone?
• Close: Do users need to close the popup by clicking on its background? Should pressing the escape key close the popup?
• Enter: Should you support an enter key and submit the form located within this modal when the user presses it?
• Schedule: Do you need to display the popup again, and if so, when (after re-login in, after clearing a cache, after two weeks)?

If you missed something in the requirements analysis, add it to your heuristics checklist, and you will never miss it in terms of testing requirements for similar functionality. In most cases, heuristics testing is vastly more effective than blind testing in solving problems.

2. Clearness

Requirements should be transparent and clear for everyone, with only one interpretation possible.

Ambiguity occurs when several interpretations are plausible. Any idea or statement whose intended meaning cannot be definitively resolved is ambiguous. But ambiguity also depends on the reader's knowledge and expertise. To reduce it, include a definition of terms (those commonly used in requirements specifications) in the system's glossary.

Context also may play a role in resolving an ambiguity. For example, the same piece of information might be ambiguous in one context and unambiguous in another. The bottom line: Do not make up your own answers if something is unclear or vague. Ask the business analyst or product owner.

3. Correctness

All statements should be correct and truthful and make sense.

Testing a system against incorrect requirements is a waste of time, money, and effort. How correct is your requirement? Is this really what's required from the system, or did someone make a mistake when writing the requirements?

4. Consistency

Requirements should not contradict other requirements. Each must fully comply with external documentation, as well as internal and external standards.

Pay attention to the general wording in your requirements. For example, if all "submit" buttons should be blue throughout the application, put these on a "dangerous requirements" list, and then analyze all other requirements for any contradictions.

It helps to categorize your requirements. For example, you might categorize all requirements that regulate buttons, popups, fields, and so on and revise them for contradictions.

Distinguish all requirements that you can trace back to a single upper-level requirement and analyze that set. Usually, inconsistency refers to one top-level requirement, and you may have several lower-level requirements from different sections that contradict each other.

5. Testability

There should be a way to check whether or not an implementation meets a requirement. Is it possible to verify that requirements have been satisfied? How do you do that, and what data and tools do you need?

Testable products are easy and less costly to maintain, and the chances of achieving customer satisfaction are much higher. That's why testability is so important to the maintainability of your application.

Other attributes of good requirements

Good requirements should be clear and strict, with no uncertainty or ambiguity. Each should be measurable, in terms of values; testable, in terms of having a way to confirm that each requirement has been implemented properly; and should be complete, without any contradictions.

Other attributes of good requirements include:

• Necessity
• Priority
• Traceability
• Сonciseness

But all of these overlap with or are included in the five attributes I described above.

Requirements testing will help you make your requirements clearer. Once you understand the basics of requirements validation, you can avoid situations where you missed dependencies, implemented something you didn't have to implement, or couldn't find a way to validate something. This type of testing saves a lot of quality assurance and development team time and effort. 

All analysis must be complete before the iteration starts, and developers must not write any code until all requirements have been tested and everyone is on the same page about acceptance criteria for a given functionality.

Stop bugs at the source. Test your requirements.

Want to know more? During my STAREAST 2018 conference session, "The Four Cs and One T of Requirements 'Testing,'" I'll offer more tips on how to follow each step listed above. The conference starts on April 29, in Orlando, Florida. TechBeacon readers can enter promo code SECM to save up to $200 off registration.