Algorithmic Bias

Systematic errors in AI/ML systems that produce unfair outcomes, typically disadvantaging certain groups based on race, gender, age, or other protected characteristics.

On this page

What is Algorithmic Bias?

Algorithmic bias can originate from biased training data (historical discrimination), feature selection (proxy variables), model architecture choices, or feedback loops that amplify existing biases. Types include selection bias, measurement bias, representation bias, and aggregation bias. Detection methods include disparate impact analysis, equalized odds testing, and demographic parity assessment. Mitigation strategies span pre-processing (data rebalancing), in-processing (fairness constraints), and post-processing (threshold adjustment). Regulatory focus is increasing, with the EU AI Act and NYC Local Law 144 requiring bias audits.

Application to Vendor Screening

If vendors use AI for screening, scoring, or decision-making that affects people, assess their bias testing practices. Request bias audit reports, understand what fairness metrics they use, and verify testing covers relevant protected characteristics. Vendor algorithmic bias can create direct legal liability under anti-discrimination laws.

Practical Examples

  • Requesting a vendor's algorithmic bias audit for their hiring AI tool
  • Evaluating fairness metrics used in a vendor's credit scoring model
  • Assessing whether vendor AI training data is representative and unbiased

Frequently Asked Questions

How can algorithmic bias be detected?

Detection methods include statistical parity testing across demographic groups, disparate impact analysis (80% rule), equalized odds assessment, individual fairness testing, and counterfactual analysis. Regular bias audits by independent third parties provide the most credible assessment.

Who is liable for vendor AI bias?

Both the vendor and the deploying organization may be liable. Under anti-discrimination laws, the organization using the AI tool is typically responsible for outcomes affecting its customers or employees, even if the bias originates in the vendor's system.

Check the vendor. Keep the evidence.

Start free