5.6 Computing Bias

Bias can creep into computing

Computing bias happens when a program produces unfair or prejudiced results, often because of bias in the data used or the assumptions of the people who designed it. Since algorithms reflect their creators' and data's limitations, they can unintentionally favor or harm certain groups, even without anyone intending to discriminate.

Where bias comes from

Bias can originate from unrepresentative data sets (e.g., training a facial recognition system mostly on one skin tone), from the biases of developers (who may not consider all users), or from flawed algorithm design. This can lead to real-world harms like unfair loan approvals, biased hiring tools, or inaccurate facial recognition for underrepresented groups.

Reducing bias

Reducing computing bias requires diverse development teams, testing with diverse data and users, and ongoing evaluation after deployment. Since it's nearly impossible to eliminate all bias, developers should aim to identify and minimize it, and be transparent about a system's limitations so users understand potential risks.

Key terms

Computing bias

Prejudice or favoritism in an algorithm's outcomes, often caused by biased data or the assumptions of its creators.

Common source of bias

Unrepresentative or incomplete training data that doesn't reflect all users or groups.

Effect of biased hiring algorithm

Can unfairly favor certain genders, races, or groups if trained on historically biased hiring data.

Way to reduce computing bias

Use diverse development teams and test with diverse data/users throughout development.

Why bias can occur unintentionally

Developers' unexamined assumptions or limited data can create biased outcomes even without intent to discriminate.

Try an AP-style question

A facial recognition system works poorly for some groups. What is a likely cause?

1. Processors that are too fast
2. The camera's color settings
3. Unrepresentative training data
4. Too much training data overall