Fast end-to-end tests are the next big thing. The tooling has improved tremendously, and the productivity and insight gains are too good to ignore. Modern tools such as Cypress and TestCafe are becoming quite impressive and can give you confidence in your product's quality.
As with most facets of software development, there is a balance to be strived for between speed and test confidence. The leverage point depends on the project, and the two most common types of software project these days are web services and enterprise software.
Here's why using the right testing suite matters.
Testing web services vs. testing enterprise software
Web services typically have a dynamic nature, in that the scope and requirements are constantly evolving. Features are frequently added and removed depending on what the production metrics say, or how the client feels about something.
The web is the place where a more flexible architecture makes the most sense, and end-to-end tests can be tuned to allow maximum architectural flexibility. As long as each test is focused on the unchanging elements of a module, instead of making direct reference to Document Object Model (DOM) elements or any other temporary thing, end-to-end tests can be reliable and consistent.
As for the enterprise and desktop software environments, that is where the waterfall model came to be, and it generally favors a larger scope of development, less frequent changes, and a more static and reliable architecture. Big design up front can be advantageous if the requirements are set in stone, for example in the case of a big rewrite with no new features.
That’s why enterprise software developers such as Java developers love unit tests. If the vision is clear, the rapid feedback of a unit test suite makes the code flow seamlessly.
The practical advantage of fast tests
But why is performance important in testing? Usually the concern is application performance, which can have a direct impact on the bottom line. Testing performance, on the other hand, has a direct impact on developer productivity.
Imagine what would happen to your productivity as a developer if your tests ran instantaneously. Suddenly you’d be able to save a file and instantly know that you had broken a feature on another module.
The main concept at work here is the feedback loop, and having a short feedback loop is highly motivating and makes your life easier. It’s the whole point of agile, for example. Short feedback loops enable you to learn fast so that you can adjust while the costs are low.
There is a multitude of small but frequent questions that can be easily answered with fast tests:
- Is this branch stable?
- Is that method fully deprecated?
- Does this single configuration setting fix my environment?
Slow tests can answer these questions, but the process becomes tiring and burdensome.
Even new development tools become practical once tests are fast enough. Just as code syntax–checking tools are now ubiquitous for most languages, you can run your tests on each file-save to get instant feedback on your changes.
Of course, with any of these tools, maintenance is necessary so that they stay responsive, otherwise the increasing wait times may become annoying and you may end up not using them as often.
The practical advantage of broad tests
End-to-end tests used to be slow. That’s one of the reasons why the original testing pyramid strongly favored smaller, more specialized tests. They run fast, but they also force a static software architecture onto your project. It takes time to write tests for every single method and, to avoid making tests constantly obsolete, large architectural changes are discouraged.
One time I noticed an architectural anti-pattern called "anemic domain model" and presented the issue to the lead developer on our Java team. The bottom line was that we would have to rewrite about half our unit tests after making these sweeping but definitely appealing changes.
In such cases, it is so painful to evolve the architecture that technical debt becomes permanent. And having identified a problem, new modules might be developed with a different architecture entirely, which makes the issues compound.
Architectural flexibility is great, but the most commonly mentioned benefit of broad tests such as end-to-end tests is that they give you confidence in your code. Confidence means that the results of the tests are very close to what will actually happen in the real world, once a user gets to play with what you built.
Obviously, this is a huge time-saver. No more deducing what will happen after a certain sequence of actions that would take time to verify. No more manually creating users or entities with specific properties for each case. That can all be defined in a test, encapsulated, and parametrized.
Test parametrization and reuse enable testing a multitude of cases by mostly copying and pasting method calls, clearly a powerful tool to gain confidence. That’s not specific to end-to-end tests. But while iterating over the parameters of a unit test can be productive, being able to instantiate and reuse entire collections of objects can save an immense number of hours in manual testing work.
Modern tools for writing end-to-end tests
The landscape of end-to-end testing seems to have settled on a battle between Cypress.io and TestCafe as of late.
Cypress.io is a front-end tool that runs through Chrome. It is modern and well-polished, and has a great UI.
TestCafe has the slight advantage of being cross-browser, which does not matter as much for this kind of test. But if Chrome feels too heavy for you, it can be a great option. It also has a new IDE where you can define test cases visually.
I won’t go into a detailed comparison between testing tools, but one thing that is seldom mentioned is that some back-end frameworks come with their own end-to-end testing tools.
For example, Laravel has Dusk. This type of tool has the advantage of giving you more power to define the back-end and database setup before each test case, where front-end-only tools can't help you since they have no direct integration with the back end.
Laverel Dusk comes with an end-to-end testing tool.
Using Dusk, you might define a type of user and run a case that applies to it, whereas in a front-end-only testing tool it would be necessary to set up the test through the application UI itself, resulting in slower tests.
Your mileage may vary
While a simple conclusion is to use mostly end-to-end tests for web development and mostly unit tests for desktop software, it is also important to identify how rigid your project requirements are. A more rigid and well-defined project specification that does not give much room for creativity and opinions would be a great environment for unit tests, whether on the web or not.
On the other hand, if you’re working with a client who isn't completely sure about his needs, make your tests as broad and generic as you can, and defer unit testing for the time when requirements begin to solidify.
Although unit testing has been around for some time, the right balance between different types of tests is still up for debate. You undoubtedly have your own experience and opinions on the subject, so please share in the comments.
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
