Defect detectives

How to write effective software defect reports

Amy Reichert QA Engineer, RxMxUSA

Writing easily readable, detailed, and complete software defect reports is an essential skill for quality assurance, developers, and support personnel. All software development team members need to take their defect reports seriously, providing thorough research and detailed reporting to make the development job easier and more productive. Those who write great defect reports will garner credibility and respect from their software development peers.

Here's how to be an exceptional defect report writer. Below I've broken down each step into specific directions and examples that resemble the complexity of real software development work situations.

The defect report checklist

Your defect report checklist should include these four items:

Define the defect

State the platform and version
Log in and examine data (client, patient, program selected)
List any nonstandard configuration settings with values
State the workflow direction through the application

Research to narrow down the root cause and get beyond the defect symptom

Retest until you have the exact steps in the correct order to reproduce the defect
Define the who, what, where, when, and how you found the defect
Execute database queries
Scan error logs
If the defect is intermittent, say so

Add support documentation

Screenshots
Video
Step recording files
Error log data
Database query results

Format all text for readability and easy understanding

Stick to a known and familiar format when possible
List using bulleted or numbered lists
Keep textual descriptions succinct and direct
Facts only; no opinion, no suggestions, no accusations, no blame

Step 1: Define the defect

The first step is to define the defect by writing a summary in the defect title and providing a general description of the problem. When writing a summary in the defect title, include the area and function where the problem occurs. Why? Because most applications are highly integrated and therefore complex. Also, you can’t assume developers or other defect reviewers know how the application works in all instances.

For example, I’ve found a defect in a medication-management application; a user is able to enter a medication that the patient record showed a severe allergy to. The allergy button doesn’t highlight in red to visually indicate the allergy, and worse, it allows the user to enter the very medication in question, regardless.

In my summary title, I’ll enter the area and the general function:

Allergy button not highlighted in red and user able to enter medication.

This is a brief statement of the problem in a concise and understandable sentence. Next, you need to add a brief description in the body of the defect report:

Allergy button fails to highlight in red when an allergy is saved on the patient record. Additionally, allergy entry is configured to disallow medication entry of the interacting medication. User is able to enter and save the medication that the patient is allergic to.

The description is going to be followed by additional detail, so keep it short and to the point. It may be the one thing many reviewers read, so it’s essential to describe the problem effectively.

Step 2: Research the root cause

Research means making sure the defect is truly a defect. You’ll want to check configuration settings, patient settings, user settings—anything in the application that affects how it functions. Do your best to ensure you have laid down an accurate foundation. Look for error log statements, if possible.

Be sure to add any research performed to the end of your defect report in a note format. For example:

NOTE: Configuration set to block medication entry when any related allergy exists on the patient record regardless of severity value. User is an MD with full access. Database table displays the correct allergy value on the patient. No errors generated in the error log.

"Notes" are good ways to communicate to the developers what research you’ve done so they can determine where they need to start.

Step 3: Add supporting documentation

Add or attach a step recorder file or video of the defect wherever possible. If you're using Microsoft products, there's a free Step Recorder application you can use to step through the problem. It will create a screenshot-by-screenshot view of where you clicked and the location of the code. This helps developers figure out the problem more efficiently. Additionally, list any database query results or error log files. Similar to adding a note, this gives backup support that the defect exists and is not only a UI issue. The five types of supporting documentation to consider using are:

Microsoft Step Recorder files
Recorded video of the steps you take and the application's reaction (choose from several free video programs available online or just use your smart phone)
Database query and screen shot of the results
Screen shots or full text of error messages in the description or as an attachment
Error logs, which exist for most applications, regardless of whether they are mobile, web, or legacy; attach a copy of the log or copy and paste the text into your description; be sure to identify which log file, if there is more than one

Step 4: Format your report for high readability

Providing an understandable format makes your defect easier to review and more likely to be accepted. Format the defect text by separating it into the following sections:

Summary (title)
Description
Build/platform
Steps to reproduce
Expected results
Actual results
Research
Support documentation

The "build/platform" section is optional, depending on the application being tested. Include this section anytime the application runs on more than one platform or browser, and note the specific version. In the full example below, I add the exact version of the browser and, if applicable, the platform.

The "steps to reproduce" section must be accurate. If you cannot reproduce the defect every time, include that in the report. Repeat the steps to reproduce several times, and verify you that have the correct steps and actions in the correct order that are necessary to reproduce the defect. When writing the steps to reproduce, keep in mind that developers may not know how the overall application functions. Give them detailed, but concise steps so they can reproduce the defect. For example:

Setup: Set the allergy configuration to ALWAYS BLOCK and save.

Log in as a MD user with full access.

Select a patient with an existing, saved allergy to penicillin.

View the Allergy alert button in the upper right corner.

Enter a medication order for penicillin, q4h for 30 days. Save.

Be direct and to the point. If multiple ways of performing an action are possible, be sure to note which one you used. For example:

Click the Enter Medication button from the main menu to add a medication. Use the Search/Select window to pick penicillin. Click OK on the popup window to confirm medication selection.

Full defect report example

The following is a full-text example of a defect report:

Summary

Allergy button not highlighted in red and user able to enter medication.

Description

Build/Platform

Windows 10/IE 11.5.1 / OS X Yosemite/Safari 10.0.3

Steps to reproduce

Setup: Set the allergy configuration to ALWAYS BLOCK and save.
Log in as a MD user with full access.
Select a patient with an existing, saved allergy to penicillin.
View the Allergy alert button in the upper right corner.
Enter a medication order for penicillin, q4h for 30 days. Save.

Actual results

The allergy button doesn't refresh and turn red to alert users to the presence of an allergy, and it allows the user to add the medication even though the allergy is on the patient record.

Expected results

The allergy button automatically refreshes anytime a new allergy is entered and saved. The button turns red to indicate an allergy exists for the patient. The MD user is blocked from saving a medication order on the patient record when the allergy is present. The configuration setting of ALWAYS BLOCK allows the medication order to be blocked regardless of the severity of the allergy.

Evidence and supporting documentation

See the red arrow in the sample screenshot below. The allergy button should be displaying where the red rectangle is drawn.

NOTE: Configuration set to block medication entry when any related allergy exists on the patient record regardless of severity value. User is an MD with full access. Database table displays the correct allergy value on the patient. No errors generated in the error log.

Be an exceptional defect report writer

Review the whole defect report before submitting. The credibility of development team members improves when defects are clear, concise, and reproducible. If you mention an attachment, make sure you add it. Note in the description whether or not you're able to reproduce the error at will, or only on certain builds or platforms. Be concise and direct, including only relevant defect facts.

Learn more about report writing and defect discovery

Now that you’ve learned how to write a defect report, check out the information on the American Software Testing Qualifications Board (ASTQB) website, which provides multiple software testing certifications. ASTQB offers a detailed syllabus that includes a long list of books, articles, and other resources.

Finally, once you're up to speed on the basics of defect reporting, here's where to turn to learn about exploratory testing techniques, which are the most effective ways to find defects to report in the first place.

Read more articles about: App Dev & Testing, Testing