Writing high-quality success metrics, key performance indicators, benefit hypotheses, résumés, self-assessment paperwork, and the like can be difficult. To make things easier, I created a technique I call IDEEM, after the five steps involved in the model: increase, decrease, enable, eliminate, and maintain.
My technique was inspired by an approach I learned from Mahan Khalsa and Randy Illig's book, Let's Get Real or Let's Not Play. I recommend the audio book (as well as the paperback) to team members looking to get good at measuring things that are hard to quantify. I listened to it while driving around in my car and learned a lot.
When you pair the IDEEM model with the QPPE model (introduced in a previous article on TechBeacon), you can quantify the benefit of any idea. It can be as small as writing a better user story. It can be as ambitious as adopting agile in an organization.
An IDEEM before-and-after
- Before: a need to significantly reduce planned downtime
- After: a decrease in planned downtime of x% (from y hours to z hours)
Now we need to find what y is today, what z needs to be, and voilà!
How the IDEEM model works
With the IDEEM approach, when quantifying the benefit of an idea, we ask five measurement-related questions:
- Increase: Among things important to our business, what would increase?
- Decrease: Among things bad for our business, what would decrease?
- Enable: What would be enabled that we can't currently do?
- Eliminate: What would we be able to eliminate that would benefit the business?
- Maintain: What do we need to keep healthy as we implement our new idea?
Let's take each of these in order.
What to increase
There are three syntax styles to use when describing an increase benefit:
- Increase <item>
- Increase <item> from x% to y%
- Increase <item> by x% (from x to y <units>)
Increase <item>
This syntax is useful if you cannot guess how much of an increase you'd need to break even on the cost of the idea. However, setting a target increase is better. Then you can tell if you miss, meet, or exceed the original aspiration.
Increase <item> from x% to y%
This syntax is useful if the unit you are targeting is a percentage: market share, defects per opportunity, etc.
Increase <item> by x% (from y to z units)
This syntax is useful if the unit of improvement is not a percentage. However, you still derive the percentage mathematically: (z-y) / y = z%.
To find the target number, the ideal is to figure out how much the value would need to change to pay for the cost to run the experiment to implement and validate the idea. This isn’t always possible, but when done, meeting or exceeding the target means assured ROI.
Increase possibilities include:
- Increase qualified leads by 50% (from 100 to 150 leads)
- Increase relative market share from 30% to 50%
A real-world example
We once had an increase benefit objective that said "decrease parameter exceptions by 5% (from $y to $z)" A parameter exception means having to ask for more money for a project; in this case, the amount was equal to $1 million, or 30% of the original estimate, whichever was lower.
Our client determined that if we reduced parameter exceptions by 5%, that would pay for our entire engagement at that client. In other words, $z above was equal to the cost of our statement of work. We engaged with the client for six months, then rolled off the account.
A year later we came back and the client had a presentation on the results over the last year. In the end, it reduced parameter exceptions by 95%! If I had shared that without having had the 5% target, this story would not have any emotional impact.
The lesson: Set goals on your IDEEM objectives, even if you have to guess a bit. Think about how big the change would need to be to make the cost of the experiment and the validation of it worthwhile.
What to decrease
The decrease benefit syntax is the same as the increase benefit syntax:
- Decrease <item>
- Decrease <item> from x% to y%
- Decrease <item> by x% (from x to y <units>)
Decrease examples
- Decrease resources spent per paying client by 40% (from $5,000 to $3,000)
- Decrease average conversion time by 75% (from 8 weeks to 2 weeks)
Enable
The enable benefit syntax is straightforward:
Enable <capability>
If a capability didn't previously exist, it may not be possible to state that something else will increase or decrease as a result. However, creating a capability may lead to other capabilities. So while you can capture this type of benefit, you can take it further by asking what would increase/decrease once that capability has been created.
Enable examples: Customer complaints, oxygen levels
One client had no data on customer complaints, but determined that data would be valuable. After the enable statement was written, the follow-up question was asked: What would increase/decrease should we enable this new capability?
The answer was to allow measurement of customer complaints and, ultimately, to increase customer retention. Note that both of these can and should be tracked.
Another client had no automated system for detecting low oxygen in subterranean mines. The client also asked what would increase/decrease if it was able to detect low oxygen levels. The answer to that, accidents by type, was something it wasn't tracking.
In this case, enabling automatic detection of low oxygen levels would:
- Enable tracking of accidents by type
- Decrease accidents due to oxygen deprivation
Enablers as leading indicators
Enablers are great and should be measured and tracked. They often are leading indicators that can be measured before the other measures that they tie to. Also, while not shown in my examples above, do add the "by x%" (from y to z units) to the increase/decrease items discovered during the follow-up questions.
Eliminate
The eliminate benefit syntax is like the enable syntax:
Eliminate <item>
Just like enablers, eliminated items can be helpful and are often leading indicators for value. Once you identify something to be eliminated, ask the increase/decrease questions and see if you can find the lagging indicators.
Examples
The idea was to eliminate data servers. The follow-up questions, of course, were what would increase ore decrease if we eliminated these data servers.
Eliminate three data servers
- Decrease cost of development
- Decrease cost of environment maintenance
Another idea would move a solution to rechargeable batteries.
Eliminate need for disposable batteries
- Decrease landfill waste attributed to disposable batteries by 2% from (y pounds to z pounds)
Maintain
The maintain benefit syntax is as follows:
- Maintain <item> of <current measure>
An idea sometimes can affect another, already-successful idea or metric. It is usually important not to negatively impact those successes. The maintain phrase ensures that we measure for impact on these ancillary ideas.
Maintain examples
- Increase user stories written per hour by 500% (from 8 to 40)
- Maintain sprint planning accuracy of 80% to 120%
Applying IDEEM to existing work
Armed with the IDEEM technique, you can look at experiments currently going on that may have benefit criteria that aren't as strong as they could be. You can still apply IDEEM to those objectives.
Existing objective example
- Existing objective: high-performance encryption for secure WAN
First we recognize this is an enabler. A very simple reword makes it fit the IDEEM style:
Enable high-performance encryption for secure WAN
Remember, with an enabler we then ask what will increase or decrease should we succeed in enabling the idea:
- Thought 1: Increase security. What are the units of that? May not be measurable.
- Thought 2: Decrease time for secure data transfer. From what number to what number? What is it at today? What's the target?
Increase secure data upload speed 1,000% (from 250kbps to 2.5mbps). This is more promising.
- Thought 3: What about data breaches?
If the improvement will decrease transfer speed, that may not be an idea designed to decrease breaches. Yet the solution better not make breaches easier. This is the perfect time for a maintain: Maintain security breaches of secure data at 4%.
Get clean, clear, concise and consistent
So, before IDEEM, the objective was to provide high-performance encryption for the secure WAN. After IDEEM was applied, the objectives became to increase secure data upload speed by 1,000% (from 250kbps to 2.5mbps) and to maintain breaches of secure data at 4%.
By using the IDEEM model, you can sharpen up benefit statements. Add that to the QPPE model and you can measure any experiment and do so in a clean, clear, concise, and consistent way.
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
