Your business' success depends on its websites and IT systems. You cannot meet your business goals without performance engineering.
That's why you should care about performance even if "performance engineering" or "testing" isn't in your title. And it's why enterprises are making performance a priority throughout their organizations.
Performance engineering is mission-critical
With at least two decades of the Internet under our belts, you'd think we'd all be experts at constant adaptation—that we could plan for any eventuality or contingency, or that our web-based systems would be as resilient as the dining room table. Not so.
Whether you adjusted your career to the new demands of online business, or you were born into a world of screens and browsers, the pace of change in any web-related or mobile category still holds plenty of surprises. It keeps many of us up at night.
Strategies such as agile development methods, IT operations automation, and the DevOps movement itself are certainly helping us cope. But system performance can't be taken for granted, and neither can your end users.
Retaining users by providing a great experience in shopping, research, partnering, and other activities requires a consistent focus and a culture of building in performance. No doubt this is why new research shows that performance engineering practices are becoming increasingly critical to businesses' operational plans.
While performance engineering is often defined narrowly as a discipline to ensure that a system's nonfunctional requirements are met (things like connection speed, memory use, and throughput), there's a clear trend toward a much broader application of the term. This is a key finding from a blind survey commissioned in 2015 by Hewlett Packard Enterprise, conducted by YouGov, a market research firm.
The survey found that performance engineering is defining a cultural shift through practices and capabilities that build in quality and performance throughout an organization. This includes functional requirements, too, plus security, usability, technology platform management, devices, third-party services, cloud, and more.
Performance engineering is a cultural transformation that builds in performance throughout an organization.
No longer limited to just the "performance testing" checkbox at the end of your development lifecycle on the way to production, performance engineering has the potential to transform your technology, your business, and your end-user experience.
Performance starts with culture
Performance engineering is beginning to cover much more than the governance of a system's nonfunctional requirements. But because the nonfunctional aspects of a system hold a key to performance engineering's broader application, let's spend a minute on this particular aspect of system design.
For example, the time it takes a user to connect to a web application must be "acceptable." The related functional feature request is that the user must be able to connect, but how long that connection should take is noted within the nonfunctional feature request: that the wait time be acceptable.
Whether we think the connection is "fast," "slow," or "average" is, strictly speaking, dependent on our values and the culture we come from. (We're talking about different countries, languages, climates, food, styles of dress, laws, generations, and social norms.) In one culture, a three-second load time for a web page may be perfectly acceptable, but in another culture such performance might be totally unacceptable. The level of acceptability for any given aspect of a system's performance is, therefore, a matter of judgment.
Whether we think a connection is "fast" depends on our values and the culture we come from.
Of course, there are also many technical factors behind how long it takes that user to connect, as well as other variables (some which are out of our control, such as the network), which is where the practices of performance engineering come in. Considerable research shows a direct correlation between a metric such as "page response time" and "conversion rate," which takes the guesswork out of the cultural equation because you can make an objective decision to set the SLAs (service-level agreements) across your organization to a specific page response time.
The discipline of performance engineering offers the capability to establish the various thresholds, based on judgment, for what's acceptable in a given system's operations and user interactions. This often starts with desired business results, then backs into the system design and architecture, proving out models of technology that will deliver business goals while exceeding end-user expectations.
Performance engineering also includes understanding and addressing failures when a fix is necessary, as well as understanding successful results and how to incorporate those into future systems. And it includes the means to have all of this adopted throughout an organization's culture, with practices executed early and throughout, to deliver exceptional results to the end user.
Monitoring performance
Once these business goals and cultural expectations, along with SLAs, are vetted and agreed upon, monitoring the performance of a system becomes a technical matter, with a set of metrics that indicate acceptable or unacceptable performance against the agreed measurements. This requires performance monitoring criteria, which, continuing with the example, typically center on time, size, and optimization, including:
- DNS lookup speed: The time it takes your IP device to ask a DNS server for the IP address associated with a domain name
- Connection speed: How quickly an end-to-end network connection can be established
- First-byte speed: How long it takes to receive the first byte for the primary URL
- Content download speed: The time it takes between loading the first byte to the last byte of the URL you're testing
The results from this group of metrics might feed a dashboard to the broader team regarding how they're currently delivering. They then collaborate and iterate on a build-deploy-monitor cycle to achieve their goal. When performance engineering is driven throughout an organization, simple metrics like these enable a focus on business goals and end users.
User experience is the ultimate performance metric
For most users of a system, the difference between functional and nonfunctional requirements is purely academic. For example: If a bank's ATM provides cash (which meets a functional requirement) but only provides that cash after a 120-second wait time (a time interval that doesn't meet the nonfunctional requirement for "acceptable wait time"), then the user at the ATM may justifiably conclude that the system isn't "functioning." The programmers and designers of the ATM system may say "Not so fast (literally). The system works, just not at the speed you want; this is really a performance problem, which falls on operations, not us."
This example illustrates how performance engineering practices can dissect the transaction from start to finish and understand the why and the what. Examples like this may involve a hardware or configuration issue, or perhaps a third-party call, or security policies, or a whole host of other items, often combined. All point to a transaction not meeting a defined SLA and leaving an end user without the desired cash.
Putting an end to these academic distinctions, along with the day-to-day finger-pointing between dev and ops teams (not to mention technology versus business teams), is part and parcel what the DevOps movement is all about. It involves an internal cultural shift toward the end user of the system(s) who wants an update, a bug fix, and some assurance that the system has the full backing of the organization the customer is depending on.
Similarly, with performance engineering, the real focus is on end-user satisfaction and business goals, and users care nothing about feature types or how a product gets delivered. They just want their cash, in this example.
A brief history of performance engineering
There's an obvious business reason to focus on the needs of end users. If they're your customers, they're consuming your products and/or services and possibly paying you for expected results. If you're a provider in a technology chain that defines a complex solution of some sort, you're still dependent on the satisfaction of users, however indirectly.
But it wasn't so long ago, say 20 years, that users of computing systems were expected to live with the high latencies, downtimes, and bug workarounds that were common in business systems. That's because users were employees and had to put up with the buggy, slow, or unpredictable systems their employers provided. Or they were users of a standalone application, such as WordPerfect for writing or Lotus 1-2-3 for spreadsheets.
That's right, we're talking about the pre-Internet age, when very few users imagined doing actual business transactions online. But once e-commerce became a buzzword, and soon simply a word, users stopped being just users. They became customers, and it was obvious that the best web-based experiences for customers would lead to repeat business.
Fast-forward to today's web-based and mobile business climate, where:
- User experience (UX) is a red-hot topic.
- Commoditization of virtually everything is a fact of life.
- Social media is the engine that can quickly sink an online retailer, transportation provider, etc., if the UX is poor—no matter what the reason.
These days, performance is king, and your online or mobile customers can either be adoring subjects, or they can suddenly become a mob of thousands, with the social-media equivalent of torches and pitchforks, at your kingdom's gate. Or they'll just go on to the next provider, leaving you alone in the dark ages.
Performance engineering skills and roles
The viability of e-commerce quickly led technology teams into various modes of performance testing, such as load testing, stress testing, burst testing, and so on—all well-known patterns today. Effective performance testing can reveal weaknesses that may cause failure under certain conditions, or it can show that one approach to a system requirement is superior to another under different performance scenarios. However, this has always been the job of a few, and realistically just a checkbox on the way to production.
But what did performance testing mean for the development team, the business, the QA team, or the IT operations team? After all, performance testing isn't about looking for software defects against the functional requirements; that activity is focused on whether or not the software meets the basic functional specifications laid out by the client and/or the design team. Now, the added goal is to understand system readiness under load, stress, bursts, etc., and these are not the sorts of conditions that traditional QA teams were designed to understand. Here's the point:
As we move toward a sharper focus on the end user, all teams throughout an organization and business need to adopt performance as a top priority.
This means moving beyond the definition of "what I do," according to traditional roles and responsibilities, and understanding how all roles can contribute to improved system performance, which leads to higher levels of user satisfaction. In fact, from the survey, respondents clearly found that performance engineers have a broad skill set, as shown below:
The gray bars at the left show the aggregate response across all three roles surveyed to the question, "In your opinion, which of the following skills are the most important for a successful performance engineer/tester?" The blue bars at the right show the individual responses by each of these three roles.
The most important thing to note here: These are skills that performance engineers have, and they also represent performance engineering practices being adopted throughout organizations as they transform. So what does that mean? Who does the work? Answer: We all do.
These results simply reflect the key roles and responsibilities of a performance engineer from the perspective of these three key stakeholder groups. Based on the survey results, all of these skills are important in order to achieve a level of proficiency in performance engineering, which is more of an overall business goal and less of a specific job function.
Make performance a business goal
Performance engineering moves organizations beyond a pure focus on technology and takes into consideration things that can't be measured via log file analysis. While it's natural for someone conducting performance monitoring to concentrate on numbers, such as response time or transactions per second, other numbers can often be more revealing and ultimately more valuable.
I would argue, for example, that a dependence on performance monitoring in production is too late. A great performance engineering practice is to monitor both production and pre-production, so you can see what levels you're optimizing your systems for, which helps you tune to the desired level for business goal achievement.
A catalog company, for example, might focus on total revenue from a specific product line or service, then track it after specific changes were made to a promotional website to see if revenue increases. Or the value of the mobile application could be judged by the number of registered users and the frequency of them accessing your products and/or services. And, as the back-end infrastructure (including web servers, middle tiers, and databases) takes on more roles inside the corporation, the metrics that track the performance of the larger organizational goals become better reflections of the quality of the supporting and enabling technology.
There are hundreds of other metrics that can be used to show the success of a given technology solution. Whatever those metrics are for your organization, you might consider picking the top five that best reflect performance and keeping day-to-day (if not more granular) results visible for everyone.
Quality performance isn't just a management objective or the goal of someone who happens to have "performance engineering" or "tester" in their job description. It should be the goal for everyone associated with the health of an organization because success depends on keeping users happy and coming back for more.
Keep your end users (especially customers) thrilled by outperforming their expectations and alternatives, and you'll get to keep doing more of whatever made them come to you in the first place, while making them brand champions.
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
