In today’s digital world, data is constantly being generated, evaluated, and updated. It also plays an important role in the work of software engineers by providing accurate, actionable feedback that helps engineers understand where and how to make improvements to a product or process.
Data also helps IT leaders visualize how work is being done, the quality and quantity of output, and how they can improve the lives of staff. And it's a vital part of any digital transformation.
Many organizations are implementing metrics-based key performance indicators (KPIs) or objectives and key results (OKRs) that encourage teams to think about business value and strategic outcomes in their daily work.
When used correctly, KPIs and OKRs are valuable tools for data-driven software engineering. The right metrics create visibility into how the business is successful (or not) and each person can see how their individual work contributed.
But there’s much to uncover when it comes to data-driven software engineering. Here are the key components, challenges, and best practices for doing it right.
People are key
The key to successfully implementing a metrics-based program often depends more on how well your teams work together, not on focusing on numbers and charts. Individuals must engage with the data, and ideally, they should be the ones requesting more data to continue to improve their work.
This tends to happen only when the human side of engineering is taken into account at the beginning of a KPI or OKR initiative. With this in mind, a successful data-driven software engineering organization will figure out what makes its people want to succeed and what they need to do their jobs better.
There is a downside with a metrics-focused approach. Problems tend to crop up when you haven't executed KPIs properly, especially when:
- The creation and rollout is clunky and focuses on management without listening to engineers, understanding what they need or want, and addressing their fears. Teams will be afraid of a data-driven organization, so leadership must engage the fear or risk the disengagement that will follow.
- The business, department, or team has an off year/quarter/month. In this situation it's difficult to keep team members engaged and prevent KPIs or data-based management from feeling overbearing.
- The balance between the contributions of teams and individuals is out of whack. Organizations need a healthy balance between team metrics that measure (and reward) team success versus those that recognize individual contributions.
You can avoid many of these problems by carefully constructing metrics or OKR/KPI systems. To begin the process, your leadership needs to understand the four pillars that enable data-driven software and how to successfully put them into action.
Anatomy of a data-driven engineering organization
Here's how the key elements of data-enabled software engineering work together:
Figure 1. The four pillars of data-driven software engineering must be driven by your company's vision. Source: Exadata
In all, there are five major components to enabling data-driven software engineering:
Company vision
Having a strong company vision is not technically one of the pillars, but it is critical, nonetheless—and this must come from leadership. The vision strongly influences each pillar and should determine (especially in the case of KPIs and OKRs) what those pillars are. The company vision is the "why" of the organization and should be strongly reflected in your KPIs and OKRs.
KPIs
These measure your organization's ongoing business performance, including profitability and how it achieves its vision. If your KPIs miss one of these marks, your employees may become disconnected from the company vision.
OKRs
These are measurable goals that are more transitory than KPIs. Your OKRs should measure what is happening now (this month, this quarter, this year) to achieve and improve results. Strong OKRs result in improved KPIs. At their best they show that you chose the right work and did it well.
Engineering metrics
These are often elusive. What is the measure of good software development? What is the measure of a great developer? Good engineering metrics should result in agreed-upon standards for software engineers, a high bar for quality of work, and the production of more and better features in order to support more valuable work.
Positive behavioral metrics
These metrics require additional explaining that goes beyond this bulleted list in order to better understand why they are important and how they work. When you have a down quarter or a struggling team, what gives your people energy and lifts them up so they deliver anyway? What makes them carry on? What makes them feel that it is worth it to do so? Each of these is driven by positive behavioral metrics.
How to get it done
While this framework offers a way to overcome some of the key problems with data-driven software engineering, it may seem impossible for you to keep up with these pillars.
That's understandable: A leader's time and energy are precious commodities, and a bunch of extra paper-pushing doesn't help anyone. So here is a cost-effective way to approach this challenge, get started, and grow it as you determine which elements are most valuable for you:
Most KPIs should be an accounting function
Those that aren't either should be, or they should be baked into the software product as a key management function. Beware of special cases and subdivisions that make reporting on KPIs more work. Everyone should get the same measurement. Large organizations with true business divisions can subdivide.
Experiment with OKRs and scale as necessary
Tools such as Weekdone offer free-to-use options you can use to get started. Another solution is to create some basic OKRs for teams each quarter and use a spreadsheet to track them. Then periodically review and discuss the spreadsheet results.
Get an engineering metrics tool
You'll need one to track successful engineering metrics, and several are available. Engaging your development team to understand their work better is key here. An effective tool will help them in understanding their work and how to get more done. What are the roadblocks? What works well? What is challenging?
Try to get your teams to engage with the numbers, and suggest ways to use them. Homegrown metrics that your team sees as serving them or their fellow developers are the way to go. You can make suggestions, but the more that this is owned by the people building your software, the better.
For positive behavioral metrics, using a tool is helpful, but not 100% necessary
The right tool can help spread positive energy to more individuals by sending automated emails, texts, or messages. Those give managers a boost, show employees and bosses that managers spread that energy, and gives all of them a boost, too.
Just sending an email to one person and tracking the information in a spreadsheet doesn't have the same effect. I've built tools that have this type of functionality, and the core tenets of this solution offer multi-person recognition, up-chain recognition, general positive energy flow, and principle-based recognition. (I am currently developing an open-source tool that I plan to share soon). Other tools for employee recognition exist as well, and you can find them by doing a quick search online.
If there isn’t a tool in place, simply find ways to spread as much positive energy as possible and practice up-reporting (sending kudos to the person and that person's boss or boss's boss, etc.). As a CEO I once worked for said, "One of the best parts of my job is going around recognizing people for all the great things they do." So sending her emails about my team's successes was not hard for me to do. It made her feel good, it made my employees feel good, and it made me feel good. That's positive energy.
Go beyond the numbers
Data-driven software engineering produces benefits well beyond the numbers if you have an effective process or framework that reduces pain points and drives success. As your organization moves toward digital transformation, you'll need to have a greater focus on delivering value. With these metrics in place, the value you deliver will be measurable, and your employees will be engaged and happy to contribute to the organization’s success.
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
