Understanding Bias in Machine Learning: Types, Measurement, and Mitigation

In the realm of artificial intelligence, particularly within machine learning, the term "bias" carries significant weight. It appears in various contexts, sometimes leading to confusion. This article aims to clarify the concept of bias in machine learning, exploring its different meanings, sources, measurement, and mitigation strategies.

Defining Bias in Machine Learning

The term "bias" manifests in three primary ways:

Technical Bias: In the specific domain of machine learning, "bias" quantifies the disparity between an AI model's predictions and the actual data it was trained on. An AI system designed to forecast weather, which consistently underestimates temperatures by an average of 10 degrees, exhibits bias toward lower temperature predictions. This bias is evaluated against real-world temperature observations.
Ethical Bias: In broader discussions about AI, particularly among journalists, marketers, and AI ethics communities, "bias" refers to the discrepancy between model predictions and a desired ideal state. This ideal is often a world free from prejudice, inequality, and discrimination, where AI algorithms treat everyone equitably, irrespective of race, gender, or background. When AI predictions reflect bias, it can spark indignation and public outcry. For example, imagine a qualified job candidate being rejected solely based on gender. If an AI algorithm makes such a decision, it can feel especially unjust because people tend to expect machines to be objective and unbiased.
Cognitive and Socio-cultural Bias: The third type of bias is related to human cognition and socio-cultural factors. Cognitive biases are mental shortcuts that help humans make faster decisions. Socio-cultural biases are generated by cultural stereotypes and personal experiences. These biases can infiltrate training data and influence AI systems.

Read also: Your Guide to Nursing Internships

Why Bias Matters

AI applications are increasingly deployed in sensitive areas, aiming to improve various aspects of life. Therefore, it's crucial to ensure that AI's impact is not discriminatory toward any particular idea, group, or circumstance. Moreover, given the growing commercial importance of AI, understanding the types of biases, their effects on model performance, and methods to measure and reduce them is essential.

When AI bias goes unaddressed, it can negatively impact an organization's success and limit individuals' participation in the economy and society. Businesses are less likely to benefit from AI systems that produce skewed results. AI models learn from vast amounts of data, and if that data reflects societal biases, the models will likely perpetuate those biases. This can harm historically marginalized groups in areas like hiring, policing, and credit scoring.

AI mistakes stemming from bias, such as denying opportunities, misidentifying individuals in photos, or unfairly punishing certain groups, can damage an organization's brand and reputation. Furthermore, those affected groups and society as a whole may experience harm without even being aware of it.

Types and Examples of Bias in Machine Learning

Most AI systems rely on large datasets for training. If this training data contains biases, the algorithms will learn and reflect those biases in their predictions. In some instances, algorithms can even amplify biases, leading to misleading outcomes.

Based on the "Survey on Bias and Fairness in Machine Learning" by the USC Information Science Institute, biases in machine learning can be categorized into three main groups:

Data to Algorithm Bias

This category encompasses biases present in the data that lead to biased algorithmic results:

Measurement Bias: This arises from inconsistencies in how features are assessed and measured versus how conclusions are drawn from observed patterns. For example, assuming that a minority group is more likely to commit crimes solely based on higher arrest rates is an example of measurement bias.
Sampling Bias: Also known as selection bias, this occurs when the training data is not randomly sampled from the collected data, resulting in a preference toward certain populations. For example, a face recognition system trained primarily on images of white men will likely perform poorly when identifying women and people of diverse ethnicities, even if the initial data collection was unbiased.
Representation Bias: Similar to sampling bias, representation bias stems from uneven data collection. It arises when the data collection process does not adequately consider outliers, population diversity, and anomalies. In the face recognition system example, if the collected data predominantly contains images of white men, random sampling will not eliminate bias because the bias is already inherent in the data.
Aggregation Bias: This occurs when false assumptions or generalizations are made about individuals based on observations of the entire population. It's essential that the labels used to tag the training dataset accurately capture the different conclusions that can be drawn from the data. For instance, labeling images of cats, dogs, and tigers as either "dogs" or "felines" when training a model to predict animal weight can be misleading because tigers and cats have different weight ranges.

Read also: Transfer pathways after community college
Omitted Variable Bias: This reflects the bias caused by missing variables that can influence the end result. The model may end up attributing the effects of the missing variables to the included ones.

Algorithm to User Bias

Algorithms can influence user behavior. This section focuses on algorithmic biases that can affect user behavior:

Algorithmic Bias: This is bias introduced directly by the algorithm itself, not by the data. It arises from choices made when optimizing particular functions, such as the depth of a neural network, the number of neurons per layer, or regularization. Bias can also be introduced by the prior information that the algorithm requires, as most AI algorithms need some degree of prior information to operate.
Popularity Bias: Popular items are often more visible, but they can also be subject to manipulation (e.g., spam, fake reviews). Even if the model makes correct predictions, the final conclusion can be biased due to the popularity of other possible conclusions. Similar popularity may not indicate quality but rather biased approaches that are not immediately apparent.
Emergent Bias: This type of bias develops over time due to interaction with users and can be triggered by changes in the target user base, their habits, and values, typically after the model has been designed and deployed.
Evaluation Bias: This arises during model evaluation and can result from inappropriate or disproportionate benchmarks. For example, facial recognition systems may be biased toward certain skin colors and genders. It's crucial to create unbiased training datasets and to design bias-free test datasets and impartial benchmarks.

User to Data Bias

Since much of the data used to train models is user-generated, inherent user biases can be reflected in the training data:

Population Bias: This occurs when user demographics, statistics, and data differ between the platform from which the data is extracted (e.g., social media) and the original target population. It's essentially non-representative data that affects the model's outcomes.
Social Bias: This occurs when a user's opinion is influenced by the opinions of others. For example, when rating a service, a user may be influenced by existing reviews, leading to a biased rating that is then used to train a model.
Behavioral Bias: Users react differently when exposed to the same information, leading to behavioral bias. For example, an emoji may represent different ideas to people from different cultures, leading to contrasting communication directions, which can be reflected in the dataset.

Cognitive and Socio-Cultural Biases in AI

Cognitive and socio-cultural biases can infiltrate training data in various ways:

Human Annotation Bias: Human annotators may introduce bias during the labeling process.
Data Collection Methodology Bias: Flawed data collection methods can lead to unbalanced, biased datasets.

AI systems trained on biased data can amplify these biases. This raises the question of whether AI datasets should mirror our imperfect reality or whether we should create training datasets that adjust for human psychological biases and societal injustices. While most people agree that debiasing training data is preferable, the perception of socio-cultural bias can vary among individuals and across different time periods. What may be considered innocuous in one culture may be offensive in another. A decision deemed acceptable today might trigger public outrage in the future. Bias perception also varies among individuals from the same culture due to differences in personal experiences and sensitivities.

The lack of AI explainability further complicates the situation. Current deep-learning AI systems often operate as black boxes, concealing the logic of their learning processes. For example, an AI-powered recruitment tool was found to discriminate against women for technical positions, even after gender information was removed from the training data. This highlights how AI algorithms can unintentionally learn bias from various sources.

Common Types of AI Bias

Beyond the broader categories, several specific types of AI bias are frequently encountered:

Algorithm Bias: Misinformation can result if the problem or question posed to the machine learning algorithm is not fully correct or specific, or if the feedback provided to the algorithm does not effectively guide its search for a solution.
Cognitive Bias: AI technology relies on human input, and humans are inherently fallible. Personal biases can seep in, even unintentionally.
Exclusion Bias: This occurs when important data is omitted from the data used for training, often because the developer fails to recognize new or important factors.
Measurement Bias: This is caused by incomplete data, often due to oversight or a lack of preparation, resulting in the dataset not including the entire population that should be considered.
Out-group Homogeneity Bias: People tend to have a better understanding of members of their own group (in-group) and perceive them as more diverse than members of other groups (out-group).
Prejudice Bias: This occurs when stereotypes and faulty societal assumptions find their way into the algorithm's dataset, leading to biased results. For example, an AI could return results showing that only males are doctors and all nurses are female.
Recall Bias: This develops during data labeling when labels are applied inconsistently due to subjective observations.
Sample/Selection Bias: This occurs when the data used to train the machine learning model is not large enough, not representative enough, or too incomplete to adequately train the system.
Stereotyping Bias: This happens when an AI system inadvertently reinforces harmful stereotypes. For example, a language translation system could associate certain languages with specific genders or ethnic stereotypes.

Measuring Bias: Suggestions

Several metrics can be used to measure bias, and the key considerations vary depending on the project's goals and the types of tasks involved. For classification tasks, the focus is on the accuracy of predictions. When working with location-based annotations using bounding boxes or polygons, the intersection of the units and overlap is more relevant.

Here are a few suggestions for measuring bias in supervised machine learning projects:

Track Annotation Activity Per User: Monitor each annotator's progress to identify inaccurate labeling early, determine the source of the error, and prevent further bias. This is particularly useful when outsourcing annotation services or managing large-scale labeling projects.
Identify Systematic-Error Sources, Locations, and Reasons: Gain a comprehensive view of the annotations and filter the data as needed. For example, you may want to review annotations for a specific data point, class, or attribute. This can help identify the locations and sources of errors and address them. Other reasons for bias may include:
- Inefficient instructions with few or no examples
- Lack of communication across the team
- Time of day (annotations made later in the day may be more accurate due to lighting)

Analyze your dataset, consider potential reasons for bias, and develop a strategic approach to address existing errors and prevent future ones.

Mitigating Bias in Machine Learning

Bias can creep into a model due to various factors, including poor data quality, model performance mismatch, and human factors. Here are several steps you can take while developing a machine learning model to minimize the risk of bias:

The Right Training Data: Ensure that your dataset is diverse, inclusive, balanced, and representative of your objectives. The data collection method can also introduce bias, so ensure that your data covers the cases that address the environment in which your model will operate. Be extra cautious when using public datasets and try to avoid reusing them to prevent bias.
Infrastructure-Related Issues: Problems with equipment can also introduce bias, especially when you rely on data collected from electronic devices, smartphones, cameras, etc. This type of bias can be difficult to detect, but investing in the right infrastructure can significantly benefit your model.
Deployment and Feedback: Algorithmic biases can influence user behavior. To identify these biases early and ensure that the model operates as intended, always consider feedback during deployment. Provide a channel for end-users to share their thoughts on how the model performs.

AI Governance and Bias Mitigation

Identifying and addressing bias in AI requires AI governance, which is the ability to direct, manage, and monitor an organization's AI activities. AI governance involves creating policies, practices, and frameworks to guide the responsible development and use of AI technologies. It often includes methods to assess fairness, equity, and inclusion.

Given the complexity of AI, algorithms can be black box systems with limited insight into the data used to create them. Transparency practices and technologies help ensure that unbiased data is used to build the system and that the results are fair.

tags: #bias #in #machine #learning #explained