As Artificial Intelligence systems become more deeply integrated into our daily lives, a worrying problem has become evident: algorithmic bias.
Despite the common perception that machines are unbiased and objective, the reality is very different. Systematic errors in the training data can be transferred directly to our applications.
Therefore, Software Development and Artificial Intelligence must face this challenge to ensure that technology does not perpetuate discrimination or logical flaws.
This raises an essential question: how do humans influence AI biases, and how can these biases then influence our decisions?
What is bias in AI?
In simple terms, bias in AI refers to systematic errors that occur in the processes or results of an algorithm due to an uneven distribution or narrow focus on the data used to train it.
These biases are not random errors, but rather follow predictable patterns based on factors such as the gender, race, age, or socioeconomic status of the people included (or excluded) in the data sets.
Main sources of bias
Non-representative data
Algorithms are trained on large volumes of historical data, reflecting past human decisions, but if that data is biased, the algorithm will replicate and amplify those errors.
For example, in the medical field, if an AI system is trained with data from patients from a predominantly white population, its predictions may not be as accurate for patients of other ethnicities, which can lead to misdiagnoses or inaccurate diagnoses.
A notable case is that of an algorithm used in the United States to prioritize medical care, which underestimated the severity of the health needs of black patients.
This was because the training data was based on past medical costs, and Black patients tend to spend less on health care due to systemic disparities.
Biased feature selection
Biases can also arise in the feature selection stage, which refers to the variables that AI system designers decide are relevant to the model.
Sometimes the characteristics that are selected can be influenced by social stereotypes.
For example, in the case of AI recruiting systems, the use of characteristics such as school of origin or address of residence can lead to an indirect bias towards candidates from less privileged backgrounds.
A recent example; Amazon developed a recruiting system that, after an analysis of historical resumes, penalized women because the data it was fed was dominated by men’s resumes.
Although the algorithm did not actively select based on gender, it indirectly learned to give preference to men based on previous patterns.
Cultural and social prejudices
AI also inherits society’s cultural biases. Because algorithms learn from human-generated historical data, they reflect existing social biases.
If an AI is trained with data that reflects structural inequalities in society, those biases are likely to be amplified in the AI’s future decisions.
A facial recognition algorithm developed by IBM, Microsoft and others showed significant errors when trying to identify the faces of women and dark-skinned people, which showed that the models had been trained predominantly with images of white men.
These errors not only perpetuate discrimination in applications such as policing, but could reinforce harmful stereotypes at a societal level.
Biases in human interaction
A less discussed, but highly relevant, aspect is how AI biases can, in turn, influence human biases.
According to recent studies, professionals who work with AI systems tend to blindly trust the machine’s recommendations, even when they are incorrect.
This can lead to people perpetuating and accepting the biases that come from algorithmic recommendations.
In a study on antidepressant prescribing, it was observed that doctors who followed incorrect AI-generated recommendations were more likely to make errors compared to those who did not receive such suggestions.
The alarming thing is that doctors began to integrate these errors into their decision making, showing that AI can not only inherit human biases, but can also amplify them in human interaction.
Consequences of bias in AI
The consequences of bias in AI can be devastating, especially in high-risk contexts, such as health or justice. A biased algorithm could lead to serious medical errors, unfair court decisions, or discriminatory practices in the workplace.
Furthermore, these biases can have long-term effects, as they can shape human decisions based on erroneous recommendations.
In the clinical setting, for example, it has been shown that AI-based diagnostic systems, despite their high accuracy, sometimes fail in particular cases affecting certain underrepresented groups.
This can lead to unequal medical care, where certain patients do not receive appropriate treatment due to a lack of representativeness in the data on which the AI was trained.
How to mitigate bias in AI?
Data diversification
It is crucial that the data sets used to train algorithms include a broader and more diverse representation of the population.
This can help mitigate some of the biases, although it will not eliminate them completely. Even more important is the validation of these models in real-world situations to ensure that biases do not persist.
Human supervision
It is essential that humans maintain an active role in making final decisions. Although AI can offer valuable recommendations, experts must be trained to identify potential errors or biases in those suggestions.
This oversight is especially important in areas where errors can have serious consequences.
Development of algorithmic audit tools
Currently, tools are being developed that allow algorithms to be audited to identify biases.
This involves reviewing AI training processes and the results of its decisions to detect and correct errors before they are applied in real-world contexts.
AI is neither infallible nor impartial
Biases in AI represent a complex, multi-layered problem, arising from the data it is trained on, but which can also influence human decision-making.
Overreliance on the recommendations of these systems, coupled with unbalanced or non-representative data, can lead to biases being perpetuated and amplified in various spheres.
It is therefore essential that both AI designers and users take a critical and conscious approach, recognizing that these technologies, although powerful, are not infallible and, like any tool, can reflect and perpetuate human biases if not handled carefully.
This post is also available in: