Over the past 17 years, I have vetted hundreds of testing contractors, consultants, and even large-scale, multimillion-dollar testing managed services. If you want to avoid some expensive missteps in outsourcing testing, listen up and learn from my experiences.
Even when you're sure you want to outsource, have a plan, and know what tests you want to perform, the process of choosing a partner is difficult because many system integrators look and sound very similar. When I led QA and testing at T-Mobile US, for example, I received dozens of calls a month from vendors, each claiming it could save us money and improve testing efficiency. I had to develop a method for vetting the vendors and their claims, which led to the development of my evaluation list.
This simple top 10 checklist includes the most important things to consider when selecting a strategic testing partner. Each item is based on my own experiences and those of my peers.
1. Lowest rate does not equal lowest total cost
When it comes to outsourcing vendors, the old adage, "If it sounds too good to be true, it probably is," makes sense. I once received a rate card from a procurement team that had negotiated an $8-per-hour rate for offshore testing resources. When I checked into the details, the systems integrator that offered the low-priced testing was planning to open an office in China but hadn't even set up the office yet and had no testers onboard. With business-critical systems under test, I wasn't willing to make my business the guinea pig for that experiment. You may be tempted to select a vendor that is offering you a virtual army of testers for what seems like an attractive price, but from my experience, using a small team of rock-star testers is more effective than relying on a large pool of untrained workers.
2. Industry experience counts... a lot
Each industry vertical has its unique business processes, so bringing in testers who have never worked in your industry before will only slow down your testing efforts. On the flip side, a team of QA professionals who understand how your business works not only will add efficiency to testing, but also can help focus your quality efforts to ensure that the areas with the highest level of business impact get the highest testing priority.
3. Add staff, or go with a managed service?
If you just need testers for a short-term project, staff augmentation is usually the best approach. However, if your organization requires ongoing support and the skill set needed is not core to your business, a multiyear managed service can be very effective. In a well-run managed service, the testing partner will have an incentive to do the work more efficiently, driving process improvement managed by your carefully negotiated service-level agreements.
4. Does your partner have a technology framework and best practices?
Be sure to ask a prospective vendor what unique intellectual property and best practices it can bring to your project. You want a team that has done it before and distilled its years of experience and success in completing projects into a set of best practices, accelerators, methodologies, and tool kits. Having these assets will help accelerate your quality efforts and reduce time to market.
5. Cultural compatibility: Does the shoe fit?
A good cultural fit is every bit as important as every other criterion. Your partner should get to know your organization, the people, and other vendors that work on your projects. Can it work at your pace, communicate information the way you need it, and be prepared to deal with the level of chaos that exists in your work environment? If it can't handle the way you work, it's probably not a good match, no matter how good it is technically. In addition, if you need a large managed service, it is also important to have an on-site lead to ensure accountability.
6. Does your partner speak agile?
The role of testing in agile practices is well documented and understood, yet many organizations still struggle with integrating testing and quality into their agile delivery methods. Choose a partner that has similar views on how testing fits in with your development methods. If, for instance, you are using agile and your partner can't begin testing until the requirements for the entire system are finalized and delivered, you have a problem.
7. Is doing business with your partner easy?
Transparency, accurate billing, clear communication, good understanding of priorities, and the ability to respond quickly to changing needs?these qualities are essential for an outsourced partner. Be sure to ask whether a prospective vendor is planning to do the work using its own employees or outsource it to a subcontractor. This happens, and it's not a bad thing if your partner chooses to subcontract, but it should disclose that, and you need to be comfortable with the arrangement.
8. Onshore and offshore: What's the right mix?
Yoav Ziv, the head of testing at Amdocs, likens offshore testing to a lemon that has been squeezed to death. The cost savings from pure offshore labor augmentation, he says, have been exhausted. While offshoring remains an essential part of most vendors' offerings, by itself it doesn't deliver substantial benefits. Instead, pursue a "rightshoring" approach, in which you locate teams in places that make the most sense for the success of your project. That might mean that some are on site at your facility, others work remotely, and still more are offshore. As long as there's the right level of communication between the teams and they are able to work well together, it shouldn't matter where they are located.
9. Check those references
Of course you will do the due diligence of checking the references that the vendor brings to you, but go beyond that and look around. What are your competitors using? Which vendors are gaining headway in the industry? Check analyst reports and market analysis?you will get a clearer picture of your prospective vendor's position and market share. Once I get positive references, I like to give partners opportunities on smaller projects first and then increase the scope of responsibility as we begin to build trust.
10. Vision and innovation
Being ahead of the curve is a huge plus for any prospective service provider. When you are outsourcing testing, you want to find a partner that can give you guidance based on what's going on in the industry, share its experience, and propose new ideas, fresh approaches, and different angles. A vendor that can help you focus your effort on the right areas, technologies, and trends is more likely to become a long-term strategic partner.
That's my list. What's most important to you when selecting an outsourced testing partner?
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
