There is more to performance engineering than just testing. Done right, performance engineering means understanding how all the parts of the system fit together, knowing what metrics matter, and building in performance from the first design.
Making the journey from performance testing to performance engineering isn't easy. But the proven practices established over years of observation can help you on your way.
The path to performance engineering
One of the first tasks that budding programmers are given is to write a program that produces the text "Hello world."
Next they start to play with the program and try to do more, to see how quickly it delivers data or answers queries, and try to optimize for the highest performance with the least amount of code. The requests come in, the responses go out, and you see results on a screen. Take this and add a long-time run script for performance testing, a script you run every time you push out your latest release. It's pretty easy when you're the author and the user.
Performance engineering, though, is a broad set of processes, and it's also a culture. Performance engineering is an art based on years of observation that have led to proven practices.
These practices define a culture that enables teams to deliver fast, efficient, and responsive systems architected for large-scale populations of customers, employees, regulators, managers, and more. The careful application of the principles of performance engineering makes it possible for corporations to please customers, support employees, and boost revenues, all at the same time.
But moving from performance testing to performance engineering isn't an easy process. The team must be ready to move from a) simply running a checkbox performance test script and focusing on parts, to b) studying the way that all parts of the system work together. These pieces encompass hardware, software, configuration, performance, security, usability, business value, and the customer. The process is about collaborating and iterating on the highest-value items, and delivering them quickly, at high quality, so you can exceed the expectations of your end user.
Here's a roadmap for making the trip from performance testing to performance engineering. Essentially, these are the steps to become a hero and change agent—and how you can enable your organization to deliver with proven performance engineering practices and the accompanying culture.
Define a culture
The success of a team depends heavily on the way leaders are nurturing the professional environment and enabling individuals to collaborate. Building this type of environment will inspire the formation of cross-functional teams and logical thinking.
Build a team
A performance engineering team means that technology, business, and user representatives work together. They focus on the performance nature of everything they're working on and figure out together how they can build in these capabilities. They need to know what specific area to focus on first, along with how to measure along the way. They need to agree on the desired outcome. They must constantly remind themselves that the end goal of adopting performance engineering is to benefit the organization and end user.
Add technology
Performance engineering requires a new way of thinking, related to your existing software and infrastructure, including the existing tools and capabilities. This is how you shape and form quick, automated results.
Define what your scope of effort is going to be and quickly learn what technology capabilities you already have available to you and your team. This will be an interesting experience, because you'll learn about the capabilities that other siloed teams have available to them. Now, with a shared vision of how you want to deliver performance engineering throughout the organization, you can leverage the technology to launch a single approach that aggregates these capabilities.
Perhaps there are a few technology areas you want to start thinking about from the lifecycle virtualization space, such as user virtualization, service virtualization, network virtualization, and data virtualization. These are the core capabilities that will enable your team to accelerate the transformation to performance engineering.
Choose metrics
I often encourage teams to start with a manual metrics process, perhaps a whiteboard (I know, not really high tech for a technologist) and a few key metrics, then measure them over time and see why they matter (or don't). You'll quickly get a core set of metrics that matter for you and your organization, which have grown out of your cross-functional teams. Your people have the passion and understanding behind these, so trust them. They offer a good way to judge results against the desired outcome.
Once you have figured out enough of this manually, and individuals are starting to adopt and believe in them, take a look at your existing technology capabilities and see how you can get to automated reporting of these results fairly simply. These metrics will be key to your way of measuring what you do and the results you're able to deliver. Make sure you have a solid baseline, and take regular measurements.
Build in telemetry
Now that you've started with culture, team, and technology, it's time to start integrating the telemetry and its data.
For example, how are you capturing the APM (application performance monitoring) data from production, and how about pre-production? Can you begin to examine these results and understand more about the behavior patterns of your users, systems and transactions? From a cross-functional perspective, this will also pique the interest of the IT operations manager; so you'll continue to broaden your network, and you'll enable them to reduce the number of productionincidents. This is just one example.
Think about other quick wins or simple integrations that exist for your existing technology that will enable you to build more bridges. Correlate these types of results across your team so you can promote the culture and desired outcomes of performance engineering by building in telemetry.
Look for indirect metrics
There are hundreds of metrics available that can be used to estimate the success of a new capability or feature being released. As systems take on more roles inside a company, metrics that track performance become more available, and these enable you to begin partnering with your business peers to find out what metrics they watch and how they get these results.
Start looking at and asking about indirect metrics within the business that would show results related to revenue, customers (attraction and retention), competitive advantage, and brand value. These are important to measure as you make the transition to performance engineering.
Focus on stakeholders
Get to know your stakeholders. Who on your team has the most interest in delivering the highest-value items to the end user most quickly and with great results? Find these people and get to know them well. Remember, you're looking for your executive-level sponsors and peer champions, so you can transform the practices and culture of an organization to become a performance engineering delivery machine.
Start gathering information and sharing initial prototypes for the type of results, reports, and dashboards you want to show to your stakeholders at a regular frequency. Typically, this would be a monthly show-and-tell exercise; however, as it matures it may become a set of automated results delivered with every build, consistently available if stakeholders want to review it. Also, you should consider regular, quarterly presentations to the executive board in which you share last quarter's results, talk about the current quarter, and seek funding for the next one.
Stay focused. Remember your objective. Find your champions. Deliver results.
Create stable environments
One of the earliest challenges will involve enabling teams with the capabilities they require. Some of this will come as you build these teams and the cross-functional tools, capabilities, and associated skills come together. But in the beginning, having a "like production" environment for performance engineering is key.
By leveraging the above-mentioned lifecycle virtualization—including user virtualization, service virtualization, network virtualization, and data virtualization—you can quickly recreate production environments at a significant fraction of the cost, and you can duplicate them as many times as required. There are several other stable environment proven practices that have been learned along the way, which you can also learn and share through others.
Celebrate wins
Remember the old forming, storming, norming, and performing program developed by Bruce Tuckman? He believed these were the four phases necessary to building teams. If you're a leader or a team member, you'll see this in action.
It's important to remember why you're doing this, and know it's all part of the transformation. Stay focused on the business and end-user objectives, so you can measure your progress and keep your eye on the prize.
Just imagine what it will be like once you have delivered these capabilities to your end user. Conduct proper retrospectives, track your progress with your metrics, and celebrate the wins!
Add gamification
As you mature the capabilities listed above, think about how you can add gamification into the results. In other words, how do you make it fun and visual to see the results you're delivering and make a positive impact on your end users and the organization in the process?
Of course, you also want to ensure that you highlight the opportunities for improvement and show the wins and losses. You can also set up the gamification of performance engineering itself at a team level to encourage a little healthy competition within your group, and well beyond, then broadly share the results.
Start small
When you first begin to incorporate performance engineering, you may be tackling a long-neglected maintenance list, or a new, up-and-coming hot project. Either can benefit from the focus of a performance engineering culture. Don't try to take on too much at first.
As you begin to elaborate on your requirements, stories, and features, it's important to remember that your whole team is working to define the what, why, and how of each item. As you continue down the performance engineering path, you will learn from each other's domain expertise, as long as you can see these strengths emerging through the clarity that small projects allow.
...and start early
Performance engineering works best when the team starts thinking about it from the beginning. The earlier the team begins addressing performance in the product lifecycle, the more likely the final system will run quickly, smoothly, and efficiently. But if it can't be done from the very beginning, it's still possible to add the process to the redesign and reengineering work done to develop the next iteration or generation of a product.
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
