Companies are increasingly recognizing the value of performance engineering, which goes well beyond performance testing. These organizations now require the capability to fix performance issues whenever and wherever they exist—and they're willing to pay a premium for those skills. That’s why many performance testers are stepping up their game.
While the journey from performance tester to performance engineer isn't easy to complete, it is easy to start. You'll just need to acquire a few skills and learn some new problem-solving techniques.
Here's how to start your own journey.
Performance testing is part of performance engineering
The first thing to understand about performance engineering is that performance testing is just one aspect of it. Performance engineering has a broader scope.
Why the current emphasis on performance engineering over just testing? Because testing product performance at the end of the software development cycle produces too much technical debt and causes too much rework. It's too expensive.
Companies need to figure out how to build performance into products as early as possible in the development process, and that requires more than just performance testing. When development teams are discussing functionality, security, or any topic for that matter, performance should be part of the conversation.
Here are seven areas where performance should play a key role.
Figure 1. Performance engineering plays a role in every phase of the software development lifecycle. Source: Scott Moore
Performance testing is still important
Performance testing is still a crucial part of software and infrastructure validation. Doing that accurately and efficiently takes time. This is especially true during the planning phase as well as during test results analysis.
Producing data that guides businesses to make good decisions takes experiential knowledge. Unfortunately, there aren't many people who do all of this extremely well.
Building a set of reusable documents and other deliverables ("templates") to use in the initial planning phase and final reporting will speed your initial startup time. I recommend working with or learning from others with experience to build out some of these templates.
How to make the transition to performance engineer
How do you make the transition from performance tester to performance engineer? Start with these three high-level changes:
Widen your perspective. Instead of thinking about whether a test passes or meets the requirements, think about whether the end user will be delighted. Look at the app through the user's eyes.
Include the whole lifecycle in your scope. Think about all of the areas you might need to address performance. Do the requirements include the right performance criteria? Are units of code, all the individual pieces, and the whole thing together performant? What does the end user think? How do you know the answers to these questions? What are you looking at that tells you whether performance is good or bad, or when it changes?
See a need, fill a need. As you learn to apply performance to a given situation, you'll start to notice holes, places where the team is not considering performance. Your job is to attempt to fill those holes. Because you now see the whole software development lifecycle, you aren't limited to thinking just about the testing phase. Help wherever there is an issue, even if it isn't in your formal job description.
Tech skills to master
There are also a few technical skills you’ll need to pick up.
Gain front-end performance knowledge, specifically regarding the browser itself. JavaScript, HTML, CSS, and other client-side components contribute to the performance of web page response times. Know how each affects the metrics you see in browser developer tools, such as time to first byte, idle time, and so on. Learn how the underlying protocols—such as HTTP(S), QUIC, and web sockets—work from the browser to the back-end servers.
Learn how to gather the right metrics needed to tell the performance story. This includes utilities for Windows and Linux, open-source products such as Prometheus to monitor containers and Kubernetes clusters, or commercial tools such as total application performance monitoring solutions. Combine these with end-user monitoring (timings from the user's perspective) and real-user monitoring (grabbing times from actual user activity).
Learn how continuous integration (CI/CD) products such as Jenkins, Bamboo, and others work. Set up a strategy for building out a continuous performance engineering process so that you can report metrics with each new build and you can observe trends as early as possible.
Shift performance left
As a performance engineer you need to shift left by doing these three very simple things:
Ask whether the business process workflow performs well for a single person. Baselining the application by gathering initial metrics from the browser or sites such as Web Page Test and GTMetrix is an easy way to tell. Look at the recommendations from Google Lighthouse from the audits tab in Chrome's DevTools. And find out why each suggestion in the report is being recommended.
Work with individual Scrum teams and find out who is taking responsibility for performance in their assigned features. Review a user story and determine whether it includes solid performance requirements. If possible, provide those teams with limited access to performance testing tools and teach them how to do their own browser audits (as mentioned above) as soon as a feature is working in the development environment.
Question whether performance metrics have been added to the continuous build cycles (the CI/CD pipeline). If so, who is looking at them? Is anyone reporting performance trends from the output? If not, help to make that possible.
A real-world example
Not long ago I was engaged to load-test a highly customized version of a SaaS product that was replacing all of my client's current systems. They had put more than two years of effort into their customized workflows and were just a couple of months away from starting the rollout.
The statement of work was fairly standard. They wanted to do performance testing of their critical business processes and determine user response times and how well the infrastructure held up under the expected volume. It sounded like a typical testing process.
After an initial orientation, I asked to see the application. I opened a browser window, went to the URL, logged in, and immediately saw that the page response was slow. So I opened up Chrome DevTools in the browser and started looking at the information in the various tabs for network, performance, and audits.
Figure 2. An example view of the performance tab in the DevTools feature that's built into the Google Chrome browser.
Within minutes I was telling the QA manager that his application would fail under load and that response times would not pass his service-level objective. I didn't need to run a test because I knew what to expect, having tested many applications like this one in the past. But they were skeptical as I confidently told them—without any test results to back me up—that the application needed some fundamental architectural changes.
I was ready to pivot away from the existing statement of work and start working with their developers and architects to tune the application before we did any testing, since that would have provided better value.
Unfortunately, they rejected my suggestion and insisted that I stick with the original plan of conducting basic load-testing activities. This turned into a six-week exercise, at the conclusion of which I unveiled the brutal news that the application would not scale and needed fundamental architectural changes.
The client had spent years of time, resources, and money building this application. Unfortunately, the outcome was tragic: They lost millions of dollars, and the failure to launch the application on time led to the resignation of the CIO. If that CIO had had a performance engineer acting as a trusted advisor from the beginning, all of this could have been avoided.
Performance engineering is really about asking the right questions at the right time.
How to get started
Performance testing is still important and relevant, and organizations will always need people who are very good at it. Performance engineering has a much broader mandate: It's about using additional skills to accomplish the same outcome as you would with performance testing—to make software perform optimally so it delights the end user.
But with performance engineering, you do that from beginning to end, at every opportunity throughout the software development lifecycle.
If your software is functional, that's good. But if it doesn't perform well, it won't be considered great. At the end of the day, you should not still be having conversations around how slow applications are or how long web pages are taking to display. That should be a fundamental element built into all areas of the product. It should be engineered in from the start.
The main difference between an engineer and a tester is that an engineer does not need to run a test to diagnose or assist in correcting a performance issue. (For a more in-depth discussion of the differences, listen to my podcast discussion with Perfbytes Podcast host James Pulley.) Performance engineers attack it from every angle.
To become a performance engineer, all you need is passionate curiosity and a willingness to help. Sometimes just starting a conversation can get the wheels turning in the right direction.
Your performance engineering journey will never be complete, because the technology changes continually and there's always something new to learn. Get started by becoming involved wherever you can to make things faster, continuously asking performance questions, and continuously sharing what you know to educate others on why performance is important and adds value.
Don't miss my talk on "Effective Performance Engineering" at the PerfGuild online conference, to be held June 1-2, 2020. I'll cover the basics and provide you with the documentation templates that I use when I do a standard performance testing project. See you there!
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
