Every morning, I launch my NPR news app, press play, and—nothing happens. Each time, I must go to another screen to re-select my local NPR news station, return to the main screen, and press play again. Only then does the broadcast start. I shouldn’t have to do that. The app is supposed to remember my local station. Nevertheless, the workaround has become part of my morning routine because I am willing to put up with it to hear the news.
But when an ABC News app began interrupting me with notices that it had stopped functioning—even when I wasn’t running it—I swiped it into the trash without a second thought. My life is full of software issues that somehow must have survived the quality assurance gauntlet, and the incidence of such annoyances only seems to be increasing.
What about the apps you or your team have built and deployed? Are they simply being tolerated by irritated users? How many times did your app get swiped into the trash, without you even knowing it? And what should you do about it?
The software quality conundrum
Ask a typical consumer whether the state of software quality has improved and you may not like what you hear. Over the last six months, I’ve personally had to deal with a barrage of software-related issues, and I’m not talking about apps from mom-and-pop development shops. Some problems I've experienced come from apps built by some of Silicon Valley’s biggest software companies—the ones held up as the true innovators in software development.
In my day job, I speak with software quality experts, attend conferences, and talk to developers and software quality experts in leading-edge businesses who sing the praises of using DevOps, agile, and continuous delivery pipelines to get better-quality software to market faster. But at home, someone is always struggling with one application glitch or another. And it’s not just mobile and desktop apps. The Internet of Things has introduced its own series of problems, from my home security cameras that mistake noise for motion and issue false alerts to the smart thermostat that just can’t seem to maintain its connection to my router.
“It’s not just you,” said Michael Cooper, chief quality officer at Healthcare IT Leaders and Run Consultants, after hearing my long litany of software issues. “The issues you describe seem to have happened to a lot of people. It’s a real problem.”
Cooper, who has led QA teams at HP, T-Mobile USA, and FICO, feels the burn. He had to deal with a programmable thermostat that shut off his home heat last winter due to a software glitch and a Windows 10 upgrade that corrupted all of the data on his hard drive.
I asked him to weigh in on five of the most frustrating software-related issues I have faced recently. I wanted to know what went wrong, why these issues keep cropping up with such alarming frequency, and what software developers can do to prevent similar issues in their own applications.
1. The mobile OS upgrade from hell
Not long after I installed Google’s Android Lollipop update, things began to go horribly wrong. At first, just a few apps began to respond slowly or crash. But the situation gradually worsened over the next few days, until my phone became practically unusable. Wiping it and reinstalling the software helped initially, but then the glitches would just begin anew.
At one point, the Android phone dialer app froze in a Windows-like blue screen of death; at other times the screen went black after I initiated a call, making it impossible to hang up. The last straw was when my phone crashed in the middle of an important conference call. I powered it back up, only to see this dialog: “Optimizing app 1 of 135…” More than a half-hour later, the phone finally came back online. But by then I had missed a call with my boss, and I lost even more time after deciding to wipe and rebuild the phone—again.
For business users such as me, this is the downside of IT consumerization. The new features in Lollipop, while nice to have, did not deliver any quantifiable business value. And they introduced risks. What I value most is stability and reliability, not cool new features. I suspect I'm not alone in this preference.
For this reason, I initially resisted the nagging OS upgrade messages. But the carrier said there was no way to turn those reminders off, so I finally gave in. Only after the problems began did I realize that there was no going back, no “restore to last known good state.”
My carrier blamed the mobile operating system and the phone manufacturer for my woes. Its solution: Buy a new phone, just 14 months into a 24-month contract for a piece of business equipment that cost $700. Ouch. After several calls to tech support and three visits to the carrier’s store, I ended up dumping the phone.
Here’s Cooper’s take:
- What went wrong: Most likely there were some incompatibilities with the Android code and the Qualcomm chipset in my device. Clearing the cache might have resolved the issues. Unfortunately, that advice came too late for me.
- Why it happened: Google, device manufacturers, and wireless carriers don’t test Android upgrades on every device that’s out there. Older devices are rarely tested for compatibility with new versions of Android, and most device makers add proprietary code to Android, increasing the potential for upgrade-related issues. Even with new devices, Cooper says, “There is often a rush to market, and testing is limited.”
- How software development teams can prevent it: For Android’s software developers, it’s important to understand which devices are most popular and test the top 20 percent. That should cover more than 80 percent of your users. The best ways to get this level of test coverage are through crowdsourced testing, mobile test automation, and the use of a cloud-based mobile testing lab.
Device makers should collaborate with OS makers to test their new device lineup on the latest and upcoming OS versions, Cooper added. Carriers and device makers can reduce problems by limiting customization of the Android operating system. Apple doesn’t allow nonstandard versions of IoS, which is why such issues occur less often with iPhones, he said.
As for users, they should wait to upgrade until bugs have been worked out for their phone, Cooper advised, and wipe the cache immediately after an upgrade.
2. The online banking upgrade that threw ease of use under the bus
I loved my bank’s old bill-paying system for its simplicity. Once I had set up my payees, I brought up the list, typed in the amount to pay and the payment date in two little boxes next to each vendor’s name, clicked the “pay” button at the bottom of the window, and I was done.
With the new bill-pay system, I must click on each biller name individually to open a pop-up window, then click the buttons on a calculator image to enter the dollar amount, press the “enter” button, and then open another window to set the payment date before clicking “submit.” To pay the next biller, I must return to the main bill-pay screen and start the process all over again.
Here's what Cooper had to say:
- What went wrong: Most likely this is due to poor design, with no consideration of its impact on the user.
- Why it happened: It could be due to a lack of user experience (UX) training and, most likely, the business performed no UX testing.
- How software development teams can prevent it: Hire professional UX designers and perform UX testing with feedback. The most customer-centric companies, such as Intuit and Apple, are known for UX testing and use of focus groups.
3. E-billing Catch-22: After choosing e-billing over paper, I get neither
When I signed up to receive e-bills from one payee in my bank’s online bill pay system, the paper bills stopped arriving, but the e-bills didn’t show up in the system. When I attempted to turn off the e-billing option, the software complained: “You must receive at least one online bill before e-billing can be turned off.” I then attempted to delete the biller, only to be blocked again. The system’s response: “Biller can’t be removed until online e-billing is turned off.”
Cooper had some ideas:
- What went wrong: It’s possible that my spam filter blocked my first e-bill, or corrupted data might have prevented the system from sending the e-bill to the correct email address.
- Why it happened: It is hard to anticipate all of the issues that can go wrong with e-billing. There are usually several systems involved, and all must be integration-tested from end to end, including the actual sending of the bills and the process of switching from e-billing to paper billing and back. Often in a testing cycle, teams don’t budget enough time to run several billing cycles, including the mailing of real paper bills and the e-mailing and verification of e-bills.
- How software development teams can prevent it: End-to-end testing must include switching between e-billing and paper billing options and back. Amdocs, which tested billing for T-Mobile, created a logical date in its system so it could test multiple billing cycles with representative synthetic data in order to avoid exactly this type of situation, Cooper said.
4. Gaming software rejects payment, dumps user data
When my nephew received a popular PC game for Christmas, my brother-in-law went online to register and pay for it. He filled out the profile information for himself and his young son, entered his credit card information, and promptly received a “payment rejected” error. When he returned to the registration screen to try again, all of his personal profile and billing data had been discarded. He repeated the process using a different credit card, with the same result. As it turned out, both credit cards were fine. But this particular user ended up re-entering his profile data three times before the issue was finally resolved, and then only after making a call to technical support.
Cooper suggested the following reasons for my brother-in-law's hassle:
- What went wrong: To process a payment, the vendor must make a call to a credit card processor. The credit card transaction must pass several fraud checks, and all of the information needs to match before it can be processed. It must be clear to the user that, for example, the address entered must exactly match the billing address on file with the credit card company. A temporary outage in the connection to the payment system also could have led to the problem. While the credit card information must disappear for security reasons if the request is rejected, other data that the user has entered could have been retained.
- Why it happened: Many companies stop testing before making an actual payment because they don’t want to spend the money during the test, Cooper said.
- How software development teams can prevent it: Get test accounts from the credit card processing company so you’re not actually getting charged during testing. Also, be careful not to trigger fraud protection during credit card testing. End-to-end testing is key, and tests need to be run against all major payment types you plan to accept.
5. Software upgrade corrupts user data
My migration from Windows to the Mac version of a popular personal finance program turned into a data conversion debacle. The Mac version could not read the data file created in the Windows format without a conversion step. When that process failed, the vendor offered to do it for me, and I gladly uploaded my file. But when the Mac-ready file came back, "memorized transactions"—a feature that recognizes a transaction and automatically assigns it to the proper reporting category—didn’t remember what went where. The categories came over, but the logic on when to apply them didn’t. Tech support was unable to help, and I resigned myself to manually recreating all of my memorized transactions as I created new entries. But when I realized that all of my previous transactions were no longer correctly categorized as well, I threw in the towel.
Cooper was very definite on this one:
- What went wrong: Clearly, there was some sort of issue with the conversion program.
- Why it happened: Conversion testing typically gets little to no attention. In this situation you may have had two completely different development teams, one for the Windows version, and one for the Mac version of the program.
- How software development teams can prevent it: Perform adequate testing of the conversion processes prior to release.
Getting at the root causes
Why do such glitches continue to occur with such frequency that they’ve become a part of my everyday life? For one thing, there’s a lot of legacy code out there, and not all of it is maintained by following current best practices. But part of the problem also may be that developers are moving too fast too soon. “It used to be that upgrades happened every 18 months,” Cooper said. “Now it’s every few weeks or even every day, and there are more things that can break.”
That’s dangerous, because some issues are more than an inconvenience, particularly with the Internet of Things, he added. For example, having your smart thermostat shut down the heat in your home during the coldest day of the year can be dangerous. What if you’re not home to reboot that thermostat?
While everyone wants to speed up the development pipeline, if you start slow and follow best practices, you’ll improve software quality, retain more users, and have a big advantage over your competitors who feel pressured to rush releases and updates to market.
Do you think the number of software quality issues is on the rise? Or is it just that the explosion in the number of apps out there makes it seem so? What’s your take on the state of software quality? Is the need for speed getting ahead of quality in your organization? I look forward to hearing your stories.
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
