Remember Tay, Microsoft's experimental AI chatbot that unleashed racist commentary after learning through interaction with its Twitter followers? Then there was COMPAS, AI-based software used by law enforcement to assess the risk of recidivism in offenders, which was found to be biased against people of color.
More recently, Amazon secretly shelved a recruiting tool that was shown to unfairly discriminate against potential female would-be hires.
These high-profile examples illustrate both the potential and peril the artificial intelligence (AI) revolution presents. Software developers are already tapping AI algorithms to facilitate loan approvals in the banking industry, to improve diagnostic decisions in emergency rooms, and to streamline the hiring process. But they don't always understand the risks.
AI algorithms are only as good as the data used to train them. That data, and the training process itself, can contribute to bias. Here are the key things to focus on to avoid bias in your AI implementations.
Promise and problems
Market watchers are bullish on AI: IDC forecasts that the market for cognitive and artificial intelligence systems will reach $77.6 billion by 2022, more than three times the $24 billion projected for 2018. Last year, the market research firm pegged the category's largest area of spending ($4.5 billion) to cognitive-enabled process and industry applications that automatically learn, discover, and make recommendations and predictions.
AI and machine learning are ushering in a new era of automation and smarter decision making, including in software development, IT operations management, and testing. But the potential for human bias to seep in during development and training of algorithms that data scientists and software engineers may use in applications present challenges to the promise of redefining the nature of work and bolstering innovation.
Unintentionally biased AI models create several risks, including damage to a company's brand equity, regulatory fines and legal action, and potential loss of customers and revenue.
Brent Kirkpatrick, founder of Intrepid Net Computing, a cybersecurity company that also specializes in big data and AI algorithms, said the problem was as big as trying to limit interpersonal discrimination.
"It is absolutely a big problem. It's the same problem now represented on paper and in numbers instead of in interpersonal interactions."
—Brent Kirkpatrick
Be wary of the inputs
One of the first obstacles impeding nonbiased AI is having the right dataset. Developers must think carefully about what kinds of data they are using to train their algorithms and where the data comes from as part of the exercise of building robust models. The problem is that the average developer doesn't have the background or skill set in data science, let alone statistics, to drive this process.
Dave Schubmehl, research director for IDC's cognitive/AI systems and content analytics research, said the problem is that if your data isn't sufficiently broad or varied, you risk creating a statistical model that's skewed.
"The real issue isn’t so much injecting bias in the system; it's not having sufficient, robust data to eliminate the bias."
—Dave Schubmehl
Training against your historical data is another area where bias can slip in. Without specific rules and guidelines as to how developers should be selecting or making decisions about the data as they are building and training models, there is opportunity for them to inject their own biases, said Lewis Baker, director of data science at Pymetrics. The company uses AI and neuroscience games to change how companies attract and retain talent.
"You have to establish a series of checks to guide selection and make sure people are doing what you want them to be doing."
—Lewis Baker
It's also important to ensure that the AI development team is diverse, so they will realize when datasets and models skew toward inadvertent bias. If your AI development team is made up solely of white men, for example, the chances of a white male bias slipping into the project becomes exponentially higher.
Given the shortage of AI talent, this can be a hard problem to resolve. However, a variety of new organizations are cropping up to encourage and develop a more varied AI talent bench. Among them: Black in AI, LatinX in AI, and Women in Machine Learning, (WiML), each of which hosts workshops and forums aimed at growing the targeted talent base as a slice of the overall AI sector.
Opening up the black box
Perhaps the biggest safeguard against AI bias is transparency—opening up the black box traditionally associated with AI to give organizations and development teams more visibility into how decisions are made. Human decision makers tend to believe an algorithm over a human being, despite the fact that they have no idea how the algorithm came to its decision, said Pymetrics’ Baker. You need to strive for transparency in the models you build and use.
"People need to know this isn't magic—there aren't fairies in a box telling you what stock to pick."
—Lewis Baker
Concerned about bias in its own AI models, for both ethical reasons and due to regulatory obligations governing the employment space, Pymetrics built its own tool set designed to determine how its models performed, including detection of any inherent bias. The company decided to release its tool, Audit AI, on GitHub so any developer can use it to determine whether a specific trait or data point is favored or disadvantaged and could lead to bias.
Beyond Pymetrics, many organizations are launching efforts to make AI more transparent and to mitigate inherent bias. For example, Google aimed its GlassBox initiative at visualizing algorithmic models, while IBM's AI OpenScale brings new trust and transparency capabilities for AI on the IBM Cloud. And Microsoft is also developing a tool that detects bias in AI algorithms.
No magic bullet
While the advent of new tools will certainly help address the growing problem, IDC's Schubmehl said, as with humans, it will be next to impossible to eradicate AI bias completely. "Like anything else, we have to know what types of predictions or recommendations we want the system to make, and we tune the algorithm to that," he said. Ultimately, there will always be a chance for unconscious bias to slip in.
"At the end of the day, we’re human and we can't think of everything."
—Dave Schubmehl
Keep learning
Get up to speed fast with TechBeacon's guide to the modern data warehouse.
Download the Buyer's Guide to Data Warehousing in the Cloud.
Get up to speed on digital transformation with TechBeacon's Guide.
How important is digital transformation to your org? Take our survey and find out how you stand next to the competition.
Thinking of making a change? TechBeacon's Careers Topic Center provides expert advice to prepare you your next career move.
