In the first five years of my career as a tester, I focused solely on automation for web and mobile web applications. In my work at GitHub over the past year, however, I have focused on testing desktop applications, and I have learned quite a bit about how automation for desktop testing differs from other forms.
Here are some lessons I've learned and tips for determining locators and running tests, plus a few popular frameworks for desktop testing.
How desktop automation is unique
Desktop applications run on a single workstation or computer. They must be downloaded to a computer and require setup based on the platform and operating system used. Desktop applications are generally for single users only, or whoever is signed in to the account at the time. In addition, desktop applications can access the local file system.
This is different from web applications, which are designed to be distributed and can be accessed via a URL. Web applications can have an unlimited number of simultaneous users and many more hardware, browser, and device combinations.
There are also differences with mobile applications; they are usually installed through an app store, work with a single user account, and have to take into account factors such as network connectivity and performance.
Tips, tricks, and tools
Here are a few ideas for how to succeed with desktop automation.
1. Decide what to automate
The approach for desktop automation should be similar to that of web automation: Start with determining exactly what it is you want to automate. I recommend focusing on the highest-priority user workflows.
As always, start with understanding the requirements before jumping into writing test automation code. Before testing the selected workflows, consider the state the application needs to be in, since you will need to navigate through a particular state in the interface. Also, decide which test data will be needed and how to best create, consume, and clean up this data.
2. Pick a framework that supports your needs
It's important to pick a desktop automation framework that will suit your requirements. Some are available for a specific operating system, and some are restricted to a particular scripting language. I'll describe a few noteworthy and free tools for desktop testing.
The SikuliX framework automates anything visible on the screen of a desktop computer. It is compatible with Windows, Mac, and Linux. It uses image recognition to identify and control GUI components. After locating images on a screen, SikuliX can use the mouse and keyboard to interact with identified UI elements. It is great because it makes it easy to write tests.
AutoIt is a scripting language that can create automation scripts for Windows programs. It is one of the oldest tools available in this space and is still used widely today. It comes with its own IDE and scripting language and can generate common Windows events such as sending keys, mouse movements, and click events.
Next up is the White framework, which helps automate a rich Windows client. It is .NET-based and does not require the use of any proprietary scripting languages. White provides a consistent, object-oriented API. The API is highly extensible and simple to use. White is not the most active project, but it still has many users and is a good framework.
The Windows Application Driver is a service to support Selenium-like UI test automation on Windows applications. It uses the JSON Wire Protocol defined by Appium, a popular mobile automation framework. This framework is still somewhat in early stages, but it is actively being worked on and has good support. I'm currently using Windows Application Driver to automate the GitHub extension for Visual Studio.
These are just a few of the test frameworks that I would recommend, each of which has its pros and cons. Check out other frameworks when determining what works for you.
3. Decide element locator strategy
In terms of determining locators to use in test scripts, a different approach is necessary than with other types of applications. With web applications, for instance, the browser's developer tools can inspect the HTML elements on the page. But for desktop applications, you need to download an application to be able to inspect UI elements on the screen.
Inspecting a particular element will show properties such as name, ID, and class name. There are many tools that will help you do this:
- Inspect is one of the most popular options. This is part of the Windows SDK and allows you to select any UI element and view the element's accessibility data.
- UI Automation Verify is another good way to view an element's accessibility data and is also a part of the Windows SDK.
- Snoop is a Windows Presentation Foundation (WPF) spy utility that allows you to view the visual tree of a running WPF application and change properties.
- Spy++ is a useful, popular utility that can display information about all current processes, their threads, and all open windows in Microsoft apps.
Choose one or a combination of these inspector tools to allow you to locate and view automation properties of UI elements.
4. Run tests locally and in CI
Running tests with desktop applications locally should be pretty straightforward with whatever tools you choose. The distinction with desktop applications is that you need to start the desktop application before running tests.
The same distinction applies for continuous testing. Instead of just making sure a particular browser is available, with desktop testing you need to make sure the application is downloaded on the machine that is running the tests and that the application is open and in the right state before running tests.
At GitHub, I use a continuous integration service called Appveyor. With it, I can spin up virtual machines for each test run that have a specific version of Visual Studio and that then install a version of the GitHub extension for Visual Studio before running tests. It's not as simple as just opening up a Chrome browser and navigating to a particular URL.
Make sure that whatever tool you choose can first download your desktop application successfully before running tests.
Keep this in mind
Desktop testing is very different from mobile and web testing. Different tools and approaches are necessary to ensure the success of desktop automation. I hope you find these tips helpful if you are new to automating standalone desktop applications.
Share your experiences below.
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
