6 test automation tools developers love

Establishing a whole-team approach to testing in all phases of the software development lifecycle can be challenging. Even when testers use development-centric languages and IDEs to create automated tests, many developers still resist the notion of having to test their own code. It's tough to get your test automation right.

Scrum and agile were supposed to fix this problem, but there's often still a division between testing and development. So why are so many teams failing to meet this vision?

After speaking with test-centric developers on my TestTalks podcast—people such as Kent C. Dodds, a full-stack JavaScript engineer at PayPal—I've come to believe the answer is based on a very simple idea: Maybe developers are not testing because the testing tools we're using just aren't that fun to use. You need to know how to choose the right tools.

Here's how to make testing tools fun again, and the tools that developers do seem to enjoy using for their testing purposes: Jest, Cypress.io, Storybook, Postman, feature-flagging software and property-based testing tools.

Put the 'fun' back into 'functional'

So, why is fun a factor here? If you’re doing web-based automation, you're probably already using Selenium, the most popular tool.

Selenium is awesome, and it's open source, but it has a reputation for being difficult to use, and not every developer finds creating tests with it a fun experience. What if there were some tools that not only made automation more intuitive for your developers but also more enjoyable? 

[ Special coverage: Automation Guild online conference ]

Some ways to do this are:

• Make it easier for developers to write code.
• Make it easier for developers to debug code.
• Make it so that writing tests is seen as a time-saver instead of a time-waster.

Make it easier for developers to write code

One of the first things you can do to try to rid your team of its anti-testing attitude is to make it easier to write tests in your framework. The easier it is for developers to write tests, the better experience they will have, and the happier they will be.

Make it easier for developers to debug code

As far as testing tools are concerned, the most important problems to solve are discovering what's broken and figuring out how to fix it.

You'll want to make finding an answer as fast as possible, so improving the developers' test debugging experience is a paramount objective of any testing tool.

When there is a failure, or an error thrown somewhere, your tool should show you the testing error. It should also show you what line that error occurred on and give you the context of the code around it, with supplemental info including the exact stack trace.

Make it so writing tests is seen as a time-saver, not waster

Besides making tests fun to create and debug, you should also explain the time benefits developers would gain if they embraced automation.

Imagine if each time developers made a change to their code for a new feature or refactor they had to manually verify that they didn't break something. Doesn't sound like much fun, does it?

Instead, what if when they make a change they have tests that automatically kick off and verify that nothing was broken due to their change? Now, that sounds like fun to me.

This should also give your developers a sense of confidence, knowing that the code they introduced to the codebase will not break anything.

6 tools that fit the bill

Here is a small sampling of tools, based on interviews with many developers on my Test Talks podcast, that meet the criteria of making it easier for developers to write and debug test code, and helping them find mistakes more quickly. 

Storybook

Storybook describes itself as a development environment for UI components. It allows you to browse a component library, view the different states of each component, and interactively develop and test those components.

I first heard about Storybook from Gil Tayar, a senior architect at Applitools, at a testing conference several years ago. He told me the reason he finds Storybook so amazing is that it's one of those tools that make you ask yourself, "How can a large team survive without this?"

It has two main stand-out features. First, it's an easy-to-use tool for building a visual component library. It allows developers, product managers, and designers to collaborate, review the components, and choose the ones they want to use for which applications.

The second feature is it lets you do visual regression on all your components using visual validation testing tools.

Jest

For unit-level automated tests, Jest is a great, developer-friendly tool that's easy to configure and set up.

It has features that make it easy to do test-driven development—such as its interactive watch mode, which you can use to improve workflows. And it's intelligent enough to know which tests to run that are relevant to the files you've changed since your last commit to your code repository.

This is super helpful if you have hundreds (or even thousands) of tests that take a long time to run, since it will run only those that are relevant to your changes.

On top of that, it runs your tests in parallel. If one of your tests fails and you make another change when you rerun, Jest is smart enough to know to rerun the failing test from the previous run first. This allows you to find out as quickly as possible whether your changes fixed those tests.

(PayPal's Dodds will be speaking on how to get started with Jest at the Automation Guild 2019 online conference on February 4 at 11 am ET.)

Postman

This one might surprise you, but when I polled some developers on what tools they love, many of them mentioned Postman. This is an easy-to-use REST client, and you can get started with it quickly by using the Chrome plugin. You'll find a native version for both Mac and Windows.

Jason Weimann, an independent software game developer at Unity3dCollege, uses Postman exclusively to test his Unity games' web API calls.

Cypress.io

This is a friendly tool for developers who also need to perform browser-based automation. Upon discovering Cypress.io around two years ago, Dodds liked it so much that he hasn't felt a need to look at anything else since, he said.

Cypress is incredible because it allows you to look at the browser, open the developer tools, and pin what the application looks like at a specific point during the test run.

You can look at what the document object model looks like at the same time, making debugging broken tests a breeze.

Gleb Bahmutov, vice president of engineering at Cypress.io, will demonstrate some of Cypress' developer features in his upcoming presentation at Automation Guild 2019.

Feature-flagging software

Releasing a new feature can sometimes be scary, especially if you’re not 100% sure of how it will react in production. Using a feature flag lets you easily turn on and off certain features for a specific percentage of users. This allows you to experiment with new features and control how soon you roll out those features to all your users in production.

According to Dave Karow, an evangelist at Split.io, feature flagging gives you several advantages:

• Speed: Trunk-based development without merge madness decouples deployment from release.
• Safety: Use canary deployment release patterns without needing to route to different images of an app or roll back by reinstalling an old version. 
• Smarts: Statistically significant experiments replace debating which features will work with users.

Most of the feature-flag options that my past guests have mentioned are paid options, but there are still many available that will work with your favorite programming language.

Property-based testing tools

There is never enough time to write all the tests you'd like to run. Even if you create lots of tests, it doesn't mean you are actually covering all aspects of your application.

Good tests are difficult to write, and, even when they are good, we sometimes miss critical edge conditions with that testing. That's where property-based testing can both improve your developers' code coverage and save them time.

Eric Normand, a trainer and speaker from PurelyFunctional.tv, defines property-based testing as a kind of random testing. It means that you generate random data. For example, instead of testing for a known, deterministic value, you send random inputs to your functions.

The beautiful thing about property-based testing is you're letting the framework manage testing edge cases for you. Developer and founder of Bizmonger, Scott Nimrod, says that's one of the best things about property-based testing.

The technique also complements your developers' existing unit tests. Since developers already leverage unit tests, they can easily adopt property-based testing in their arsenal of tools.

The programming language your developers prefer will determine which framework to use. For example, for Java there is JavaQuickCheck, and for .NET you can try FsCheck. (For a fuller selection, check out Nat Pryce's list of property-based testing tools on GitHub.)

Developers' helpmates

While this is by no means an exhaustive list of available tools, it should be enough for you to get an idea of which tools you might want to introduce to your development team.

Use the criteria mentioned earlier in this article—code that's easy to write and debug, and tools that save time as a guide—and you'll be well on your way to making testing more fun for your developers.

To learn more about test automation tools, and to hear what the experts have to say, attend the Automation Guild online conference, to be held February 4-6, 2019.