Managers misunderstand the minimum barrier to entry for test automation. One barrier is not having dedicated, focused test automation engineers, people who work on test automation every day. Another barrier is not having people who are strong in both programming and testing.
Here's how you make an effective business case for that hire: Understand what automation engineers are, the breadth of their responsibilities, and the scope of their day-to-day activities, so that you can make the case that hiring dedicated automation engineers is essential.
The automation engineer
What do test automation engineers do? For starters, they enable the team to understand the status of the codebase. Their test code automatically exercises new features when they are committed to source code control to see if functionality has changed. Shortly after new stories or requirements are created, they write and automate test cases that will pass when functionality is complete. Any regression in functionality is called out by test automation and signals to those responsible that something is wrong.
When customers have issues with the product, the automation engineering team quickly creates automated tests to replicate the problem and hand them off to developers, who run them to identify code paths affecting the outcome. Automation engineers alert developers to UI, service, and unit tests that indicate the source of the problem.
Test automation engineers make sure that a continuous integration (CI) system runs tests frequently against the system under test. A failed test for a build can be cross-referenced with logs from source code control to identify who may be responsible and who may have information to help with the fix.
Whether we call them software development engineers in test (SDETs), automation engineers, or test automation engineers, these specialists are used for their brains, their experience, their insight, their coding skills, and their peculiar talent for sniffing out issues with software.
Responsibilities
Automation engineers have numerous responsibilities, including working with the team to:
- Identify what to test
- Define needs for test automation
- Identify tools that can be written to help testing
- Identify how to test something with automation
- Implement a tool, script, or framework
- Test a tool, script, or framework
- Maintain tools, scripts, and frameworks
- Identify what’s going on with a tool, script, or framework
- Troubleshoot
- Test
Does this seem like a lot of context shifting?
Right. That’s my point.
Here is how the roles of tester, developer, and automation engineer compare:
| Responsibility | Tester | Developer | Automation engineer | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Write new code | √ | √ | |||||||||||
| Maintain old code | √ | √ | |||||||||||
| View SUT broadly | √ | √ | |||||||||||
| View functionality narrowly | √ | √ | |||||||||||
| Support users | √ | ||||||||||||
| Help in-house customers | √ | √ | |||||||||||
| Troubleshoot automation code | √ | ||||||||||||
| Diagnose potential issues in SUT | √ | √ | √ | ||||||||||
| Create test cases | √ | √ | |||||||||||
| Experiment with SUT | √ | √ | √ | ||||||||||
| Communicate with users | √ | √ | |||||||||||
| Use testing skills | √ | √ | |||||||||||
| Use development skills | √ | √ | |||||||||||
Day to day
The following is a typical day in the life of a test automation engineer:
| Time | Activity |
|---|---|
| 8:00 AM | Check CI runs from previous night |
| 8:15 AM | Troubleshoot failed tests |
| 9:00 AM | Scrum |
| 10:00 AM | Fix test automation defects |
| 1:00 PM | Begin coding test automation for a new feature |
| 1:15 PM | Check in on a failed build |
| 2:00 PM | Continue coding test automation for new feature |
| 4:00 PM | Engage in impromptu conversation on morning failure |
There is a lot going on here, coupled with the task of doing software development for a codebase that tests your application.
The fallacy of learning as you go
Test automation is a testing activity and a programming activity. The majority of a test automation engineer's time is spent programming. Many managers and executives do not understand this, which can contribute to their dismissal of the need for dedicated automation engineers.
I was two years into my profession as a software engineer before I began to feel proficient at it. That was two years of full-time programming, somewhere shy of 4,000 hours. That was after completing my computer science degree. And multiple side projects throughout college and those first two years. And programming during college.
Of course, hindsight and the Dunning-Kruger effect tell me now that I probably wasn't as “proficient” as I believed. It took me ten years of programming before I was able to easily turn a thought into code in my native programming language.
Too often, I see managers beginning a test automation effort with people who have coded for a few weeks. They ask them to get test automation going as a part-time activity—a couple of hours a day. But let’s think through this.
How long would it take a person with almost zero programming experience to be proficient while programming just two hours a day? Remembering that I believed I was proficient at 4,000 hours (two years of eight-hour days), then it would take this person eight years! And that’s just to get to the point where the person thinks she's proficient. Does your company have that long to wait?
So test automation engineers need focused time to learn the crafts of programming, testing, and test automation before they can produce.
What management needs to understand
I work with many of my clients' managers to help them understand that creating test automation is difficult and takes long periods of uninterrupted concentration. If your team is constantly interrupted, then you need a test automation engineer who is outside your normal team. If you need more people to be testing by hand, hire them. Don’t re-purpose test automation engineers for significant periods of time.
Managers need to know that test automation is a focused, difficult activity. Successful test automation teams hire exceptional test automation engineers.
ROI and the minimum barrier to entry
As I recently wrote, the return on investment for test automation can be huge. Nonetheless, teams are unwilling to invest in it properly. They want ROI on the cheap.
Let’s say I told you that you can create $1 million of labor a year, but it’s going to cost $200,000 per year. Would you take that deal? If you need the created labor and had the funds, absolutely!
But what I see in practice is that people want a $1 million return on an investment of just $20,000. However, there is a minimum barrier to entry in test automation. If you invest $20,000, you won’t get $1 million returned in labor. In fact, when the investment is too low, you won’t break even—you’ll lose the investment, plus you won’t be doing whatever you could have been doing instead of this activity. Additionally, you will lose the calendar time, while leaving your need unmet. To top it off, you will have a half-written automation codebase that people will think of as a major investment that they have to maintain!
The minimum barrier for entry for test automation is one full-time test automation engineer per small team. Keep in mind, you still need testers, because they work to inform the test automation engineers about what to test—sharing their expertise and domain knowledge.
One job, full time
Test automation is a software development activity. It requires long periods of uninterrupted focus. Most decision makers in software engineering would not be willing to split a software engineer’s time between projects, but for some reason we are willing to do that to test automation engineers. Moreover, we’re willing to split their time between manual and automated testing. And then we split their time again by asking them to:
- Maintain existing product (test automation) without handoff to a “customer service” equivalent
- Troubleshoot any test failure when it happens (from hour to hour with CI)
- Develop new test automation with full understanding of the domain and business logic
Test automation is a full-time role. Understanding one project team’s business scope is enough of a challenge in addition to the other roles required of them: maintenance, support, and new test automation development.
Want to learn more? Check out my webinar, "Making the Case for Full-time Test Automation Engineers."
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
