Quality assurance and testing pros are facing a new set of challenges with the rise in adoption of scaled agile development models. While agile test strategies come with their own issues, scaling agile adds another level of complexity.
My experience leading testing in a scaled agile context is within the Scaled Agile Framework (SAFe), but no matter which scaling mechanism your organization has chosen to use, you will recognize some of the issues described below.
Here is my war story, from the trenches of scaled agile implementations.
Getting an overview
You’ll encounter your first obstacle as soon as you visit the SAFe website to learn about testing in that context. Or at least it was difficult for me. I wanted a clear structure that makes it easy to find everything about the topic, similar to the LeSS scaled agile model website, which I found to be extremely well structured.
But the later versions of the SAFe model have improved significantly with respect to testing. You will find chapters on agile testing, CI/CD, test-driven testing, behavior-driven development, and so on.
Although there is still work to do here, here are the challenges you might experience with testing in this context. Here are the big three.
1. Risk-based testing
As testers, we talk about doing risk-based testing, but how is this applied in an agile release train? You need to focus on product risks at different levels. When specifying the features, product risks must be identified, since these might influence the level of quality assurance you need to do for the feature.
Here are some questions to answer:
- Are there any risks that might require nonfunctional testing?
- Do you need more unit testing or more code review?
During the program-implement (PI) planning, the team must consider product risks in the context of individual stories so that this is known before you start estimating the stories, again to get a clearer picture of what should be done.
But you also need to focus on cross-team quality risks, which are identified when you look at the solution on the release-train level. You must also have the solution focus when it comes to identifying and mitigating the quality risks. The mitigation is often done on a team level and must be a part of the PI.
2. Test coordination between teams
Often features have dependencies between teams, so you need to coordinate both development and testing activities. Teams must address the feature dependencies identified during the PI planning during the increment. This also covers the test part—how, who, and when the testing of the dependent features is done.
One way to handle this is to be explicit around this in the definition of ready/definition of done (DoR/DOD). It is a joint responsibility of the team to ensure that it is handled, and the DoR/DoD can remind them of this.
But the team needs to be explicit when discussing the implementation of a feature. If it is the "team" in general, there might be a risk that it falls between two chairs. Make sure you specify who takes care of the cross-team coordination for testing a given feature. This is even more needed if DoR/DoD is not present.
3. System integration testing and end-to-end testing
You might argue that the team should take care of this test activity, too, but it might need assistance. In case the team needs help, or if it is not practical to have this on the team level, SAFe refers to a "system team" for this activity. This is a team that helps build and support an agile development environment, including any tools.
This team will ensure that a framework for continuous integration is established, and also that at least once per sprint an integrated version of the complete, cross-system solution is established. The system team can also take the responsibility for the specification and execution of end-to-end tests.
There are challenges with moving system integration test and end-to-end test to a separate team. One is the handover of knowledge about features. Also, the system team needs to have continuous involvement in the work done on the teams, or at least be a part of demos and get handover from the team when a feature is complete.
Context is important
These testing challenges in a scaled agile environment, but it all depends on your context and specific scaled agile environment. There are many other potential challenges, including these:
- Do we know our value stream, and do we address it in our testing?
- Who is taking care of test management across teams in the train?
- How are we handling test automation: team, train, system team?
- Do we have dedicated agile testers on the team?
Testing in a scaled agile environment has its challenges, no matter what framework you choose to use in your organization, be it SAFe, LESS, Scrum@Scale, or maybe your own mix. It is important to remember both the "small" testing on user stories and features and also the "large" testing on the solution as a whole. It must be clear for everyone working on this how and when this testing is done—and by whom.
How much help you require to get started with agile testing in the different frameworks will differ. But remember that they are only frameworks. Your team or organization needs to specify what works in your context.
Want to know more? During my EuroStar conference session, "Testing SAFely—Finding your way in the Scaled Agile Framework," I'll offer more tips on overcoming testing challenges with scaled agile. The conference runs November 17-19, 2020.
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
