Algorithmic Bias

Algorithmic Bias occurs when an AI system produces systematically different, and often unfair, outcomes for different demographic groups. It can arise at multiple points in the ML pipeline: historical bias in training data that reflects past discriminatory practices; representation bias when certain groups are underrepresented in training sets; measurement bias in how the target variable is defined; aggregation bias when models trained on mixed populations fail specific subgroups; and deployment bias when models are used in contexts different from those they were trained on.

Real-world examples illustrate the stakes. Amazon's automated resume screening tool penalized resumes containing the word 'women's' and downranked graduates of all-women's colleges — because it was trained on 10 years of predominantly male resumes. Facial recognition systems have shown dramatically higher error rates for darker-skinned women than lighter-skinned men, raising serious concerns about their use in law enforcement. The COMPAS recidivism algorithm was shown to incorrectly flag Black defendants as high-risk at twice the rate of white defendants.

Detecting and mitigating algorithmic bias is both a technical and organizational challenge. Technical approaches include auditing model outputs across demographic subgroups, rebalancing training data, applying fairness constraints during training, and using post-processing calibration. But technical fixes are insufficient alone — bias often reflects structural inequities in data that cannot be resolved by the algorithm alone. Meaningful solutions require diverse development teams, stakeholder engagement, ongoing monitoring post-deployment, and governance accountability.