Certain times of year—Mother's Day, Valentine's Day, back to school, and those all-important Black Friday/Cyber Monday shopping days— are the ultimate stress test for e-commerce. Sites that aren't prepared pay the price. Last year, some notable retail websites, including Best Buy, had major outages on Black Friday weekend, likely costing hundreds of thousands of dollars.
So is your site ready for your next big sales rush? Here are the key facts you need to know:
- Even big sites have outages
- Bad testing and web infrastructure are often at fault
- Many retail sites fail to prepare at all
- Mobile traffic is bigger than desktop and growing fast
- Cloud testing is essential to avoid website crashes during busy times
- Testing can be simple and scalable
Mobile traffic is bigger than desktop traffic for e-commerce
Mobile traffic has been growing faster than desktop traffic for years. It hit a tipping point during the 2014 holiday rush, when mobile traffic outpaced desktop traffic for the first time on a Thanksgiving. Furthermore, sales completed on mobile devices were up 14 percent over 2013.
This trend isn't slowing down. A whopping 40 percent of male and 33 percent of female online shoppers aged 18-34 would buy everything online if they could. Meanwhile, the threat of DDoS and other malicious attacks during sales events remains a huge problem. Disruptions like these are often planned to hit at the worst possible times for the impacted businesses.
And yet, given the immense fortunes at play, some retailers simply blow it. Best Buy's website, a top-20 most-visited online retail web property, experienced a total of about three hours of downtime on Thanksgiving morning last year. Cabela's, an outdoor gear retailer, experienced two hours of website downtime during the prime midday shopping period on Black Friday.
Outages are expensive
Given the overall amount of traffic they would have received over those hours, both Best Buy and Cabela's are certain to have left hundreds of thousands of dollars—or more—on the table due to their simple failure to appropriately handle the crush of traffic. "As the old adage goes, time is money," says Tom Lounibos, CEO of performance analytics firm SOASTA. "This will apply more than ever during the upcoming busy online shopping seasons. Poor website performance is now being measured by retailers in terms of lost customers and revenues—so much so that many leading retail brands have spent their summer preparing their holiday websites and mobile apps with around-the-clock testing and measurement."
Poor website performance is now measured in terms of lost customers and revenues
—Tom Lounibos, CEO, SOASTA
Customers usually shop on these days with the specific intent to make purchases, often big ones. If those purchases aren't made from your store, they're almost certainly going to be made from someone else, essentially doubling the damage of your downtime through the enrichment of your competition.
And avoiding site outages isn't just about money, though there are billions of dollars on the table here. What happens to your company's brand image—especially if you position yourself as a technological leader—if you have an e-commerce crash in the middle of a big sale? What does downtime like this do to your customers' opinion of you as a business? And where do they go if your site isn't reachable?
Remember that there are no second chances with e-commerce. If your site goes down, the damage is done for good.
Why retail sites crash
Why do problems like the ones encountered by Best Buy and Cabela's persist? Because the sites weren't tested appropriately or thoroughly enough to ensure they could handle the kind of traffic that these shopping days generate. Specifically, there were likely significant performance failures in these organizations' web infrastructure, and those problems should have been addressed well before the sale dates arrived.
Cloud testing is simply the use of cloud computing environments to simulate real-world user traffic to a website or mobile app. Cloud testing lets you determine how your site will respond under a load, based on parameters you set, and from a variety of virtual device types, including desktops, tablets, and smartphones.
Conventional testing has long been the bastion of web developers, but this type of testing requires a complex and expensive infrastructure, which in many environments is only rarely used, and traditional testing has become even more difficult as business applications become more complicated. Cloud testing lets enterprises offload this testing infrastructure to third parties, which have virtually limitless resources to provide scalable, easily managed testing services through the web. Because these services can be activated only when needed, many enterprises find they represent a significant cost savings over traditional testing as well.
Finally, since cloud services run these tests from outside your firewall rather than just over your internal network, cloud testing does a much better job at realistically modeling the traffic on your website. So many traditional testing systems are ultimately useless because they aren't modeling traffic accurately.
Cloud testing is now a critical component of any major enterprise. Because retail sites never experience the level of traffic they see on major sales days, cloud testing is the only way to ensure your site will stay operational and maintain performance under that level of traffic.
Hope isn't a strategy—it's a recipe for bad performance
The sad reality is that not only do most retailers inadequately prepare for heavy traffic, most of them don't prepare for it at all. "We think we'll get lucky again this year and the site will handle the traffic" is far too common a sentiment in the industry when it comes to preparing for any huge selling event.
As any enterprise web professional can tell you, this attitude sets up a business for failure. While it's true that not every site will go down altogether for hours at a time like Best Buy's and Cabela's, but many are likely to suffer from generally slow performance as traffic increases, and many will lose customers who become frustrated with the site's sluggishness.
How to do cloud testing right
Why are people reluctant to cloud test their websites and mobile apps? Often, it's because they're afraid of complexity and cost or that the testing won't be sufficient to mimic real-world traffic.
It doesn't have to be that way. Today's cloud testing services are:
- Simple: They utilize tools, services, and infrastructure that most enterprises are widely familiar with and require minimal training to operate.
- Smart: Tap in a few parameters, hit the "go" button, and these services are underway, quickly and efficiently.
- Scalable: Cloud testing lets you scale to a level that models real-world traffic patterns, thanks to the advantages inherent in leveraging the cloud environment versus a standalone solution.
Let's dig deeper into each of the criteria we've outlined above to see why cloud testing is a smart decision for any enterprise.
Keep your cloud testing simple
One big advantage of cloud testing is that it leverages your existing IT investment. There's no need to add hardware, onboard new employees, or hire consultants to implement a cloud testing strategy. It's a true SaaS offering that can be activated only when it's truly needed.
That simplicity cascades down to the actual use of a cloud testing service. It's not just available when you want it, it's available at the level you need. Do you want to model 1,000 simultaneous connections or a million simultaneous connections? Cloud testing lets you flip the switch between the two—or anywhere in between—in a matter of seconds.
Cloud testing can also let you model legitimate traffic or denial-of-service attacks, or a combination of the two. There's no waiting on reports to be generated, either. Results come to you in real time, as the system is running, giving you a live look at what's happening on your network.
Look for smart solutions that make testing easier
A good cloud testing solution isn't just intelligent, it's predictive. These tools can help you locate where serious bottlenecks and programmatic bugs lie, and offer smart guidance on what resources might be required to manage different load scenarios.
Different end users in different parts of the world don't behave the same way. Traffic doesn't arrive all at once from a single type of device. You may want to model a 24-hour day that includes traffic from Australia that is heavily smartphone based, tablet traffic from the United States, and desktop traffic from Europe, all of which ebbs and flows throughout the day and night as users wake up, take breaks, and go to sleep.
A smart cloud testing solution can take all this into account. It can accommodate traffic shifting between time zones, from metropolitan areas to more rural locations, and many more considerations. Keep in mind that a simple change in device type (such as desktop to phone) or network type (LTE to 3G) can change results by an order of magnitude, so these capabilities are critical.
Of course, it would be foolish to think that capabilities like this didn't come without some level of expense, and the level of this expense is often determined by a service level agreement (SLA). A smart cloud testing solution will take your SLA into account. If your load testing scenarios begin to exceed the parameters of your SLA, it will stop the test and notify you to give you the ability to optimize your investment in the technology.
Scalable cloud testing is a must
Of course, none of this has any value if it can't actually do the job of accurately modeling heavy traffic and the massive scale required.
Cloud testing must work on a small scale, then allow you to add burst capabilities as needed. You might run 1,000 virtual user tests against your site as a matter of routine, but want to instantly ramp up to a million virtual users to prepare for major sales days.
Remember that the typical distributed denial-of-service (DDoS) botnet can be activated in a matter of minutes, so modeling a nearly instantaneous crush of traffic isn't a far-fetched concept. That's true scalability. And you can only really add that kind of capability by testing through the cloud.
A solid cloud testing system can ramp up from one virtual user to a million or more virtual users, accessing your network from locations all around the globe, in less than 15 minutes. Time is of the essence when you're running these scenarios, and the results you get have to be accurate and repeatable.
Scalability also applies to your test results. Cloud testing doesn't happen in a vacuum, and the success of your site at managing the load is not a binary operation. Testing results measure not just whether your retail site is up or down but how responsive the site is under various loads.
Your cloud testing system will be able to give you a long-term analysis of this responsiveness over time—letting you see whether programmatic and infrastructure changes are making things better or worse—and the library of results the system produces should be clear and manageable.
Shane Evans, a senior product manager for HP, says, "Our customers are quickly adopting burst testing in pre-production, complementing the proactive and proven approaches to optimization of their applications, and ensuring...an optimal experience for their end users. The value seen by customers leveraging common assets for testing in the lab, through to monitoring in production, is significant."
Our customers are adopting burst testing in pre-production
—Shane Evans, product manager and performance engineering evangelist, HP LoadRunner
Start cloud testing before it's too late
For any online retailer that wants to survive a busy selling season, cloud testing needs to be a key part of their enterprise IT arsenal.
It may help to think about cloud testing programmatically. That is:
IF you do nothing—or simply hope and pray for nothing bad to happen
THEN you will negatively impact your business' revenue, brand, and customers
AND there are no second chances to get this right.
ELSE you can do something by implementing a simple, smart, and scalable cloud testing solution
OR prepare for your business to be a headline the following day and to get your resume in order after the fallout.
2015 is on track to be an even bigger shopping year than 2014, with more online shopping taking place and more of those customers reaching your site through mobile devices. Start thinking about cloud testing today and leverage this simple, smart, and scalable process to stress-test your website and mobile apps before it's too late.
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
