The invisible bias in selection

There is a widespread belief that AI eliminates bias in selection because "algorithms are objective". It is exactly the opposite. AI does not eliminate human bias. It amplifies it, systematizes it, and scales it at speeds no human could achieve.

When a human recruiter has unconscious bias, it affects the 30 CVs they review per day. When an algorithm has bias, it affects the 3,000 CVs it processes in an hour. Scale turns an individual problem into a systemic one.

The origin of the problem is simple: AI learns from historical data. If your company has historically hired 80% men for technical roles, the algorithm learns that "man + technical role = good hire". Not because it is true. Because that is what the data says.

This module teaches you to detect, measure and mitigate algorithmic bias in your selection processes. Not as a theoretical concept, but with tools, metrics and checklists you can apply tomorrow.

Types of algorithmic bias in hiring

Not all bias is equal. To correct it, you first need to know which type you are facing:

1. Historical data bias

The most common. If your dataset of "successful hires" reflects past biased decisions, the model replicates them. If for 10 years you only hired engineers from 3 universities, the model learns those universities predict success. They do not. They predict that your company chose them.

2. Proxy bias

Even if you remove protected variables (gender, age, ethnicity), the model can use proxies. Zip code predicts race in many cities. School name predicts socioeconomic level. A CV gap can indicate maternity. The model does not need the direct variable to discriminate.

3. Labeling bias

Who decides what a "successful hire" is? If the label is "stays more than 12 months", you penalize profiles that change companies frequently (more common in younger generations and certain sectors). If the label is "receives a good evaluation", you inherit the evaluators' bias.

4. Representation bias

If your dataset has 90% candidates from one demographic group, the model works well for that group and poorly for the rest. Not because it actively discriminates, but because it does not have enough data to learn patterns from the minority group.

5. Feedback loop bias

The most dangerous and the hardest to detect. The model rejects candidates of a certain profile. Since they are never hired, evidence is never generated that they could have been good. The model reinforces itself: "see, we never hired that profile and we did fine". It is a self-fulfilling prophecy.

The Amazon case: a lesson that changed the industry

In 2018, Reuters revealed that Amazon had developed an AI system to evaluate CVs that systematically discriminated against women. The system, trained on 10 years of Amazon hiring data (predominantly male in technical roles), had learned that male CVs were "better".

Indicators the system penalized:

• The word "women's" (as in "women's chess club captain")
• All-female universities
• Names associated with the female gender

Amazon tried to fix the system by removing explicit gender variables. But the proxies persisted. The system continued to identify patterns correlated with gender and penalize them. Amazon ended up withdrawing the system.

Lessons from the Amazon case

1. Removing the variable does not remove the bias. Proxies are ubiquitous. You need active fairness testing.
2. Historical data reflects past decisions, not objective truth. If you only hired men, the model will learn that men are better candidates.
3. The system needs continuous auditing, not just at launch. Bias can appear or change over time.
4. Transparency is non-negotiable. Amazon developed the system in secret. When it was discovered, the reputational damage was greater than the recruitment savings.

AI Act Annex III.4: HR as a high-risk system

The EU AI Regulation (AI Act) classifies AI systems used in employment and worker management as high-risk systems (Annex III, section 4). This has direct legal consequences for any company using AI in selection.

What Annex III.4 says exactly

The following are considered high-risk: AI systems intended to be used for:

• The recruitment or selection of natural persons, in particular for publishing job ads, screening applications, evaluating candidates.
• Making decisions affecting employment conditions, promotion or termination of contractual relationships.
• Task allocation based on individual behavior, personality traits or personal characteristics.
• Monitoring and evaluating the performance and behavior of persons in employment relationships.

Obligations for companies

If your HR AI system is classified as high-risk, these obligations apply:

1. Risk management system (Art. 9): document the system's risks and mitigation measures.
2. Data governance (Art. 10): ensure training data is representative, relevant and error-free.
3. Technical documentation (Art. 11): document how the system works, what data it uses, and how it has been validated.
4. Activity logging (Art. 12): automatic logs of the system's decisions.
5. Transparency (Art. 13): inform candidates that AI is used in the process.
6. Human oversight (Art. 14): a human must be able to intervene, override or deactivate the system.
7. Accuracy, robustness and cybersecurity (Art. 15): the system must work as expected, resist attacks and errors.

Fairness testing: how to measure bias

You cannot eliminate what you cannot measure. Fairness testing is the process of evaluating whether your AI system treats different demographic groups equitably.

Key fairness metrics

• Disparate Impact Ratio: the selection rate of the minority group divided by the majority group's rate. The 4/5 rule (EEOC): if the ratio is below 0.8, there is evidence of discrimination. Example: if you select 50% of men and 30% of women, the ratio is 0.6. There is disparate impact.
• Equal Opportunity: the model has the same true positive rates (good candidates correctly selected) for all groups.
• Predictive Parity: of the candidates the model selects, the same percentage turns out to be "good" in each group.
• Demographic Parity: the percentage selected is equal in each group, regardless of qualification. It is the strictest and most controversial metric.

How to do fairness testing

1. Define the protected groups relevant to your context (gender, age, nationality, disability).
2. Collect demographic data from your candidates (you can do this anonymously and voluntarily).
3. Calculate the fairness metrics for each pipeline stage (screening, interview, offer).
4. Identify significant disparities (ratio < 0.8 is the reference rule).
5. Investigate causes of disparities (data? model? process?).
6. Implement corrections and re-measure.

Fairness testing tools

• IBM AI Fairness 360: open-source toolkit with 70+ fairness metrics and mitigation algorithms.
• Google What-If Tool: interactive fairness visualization for ML models.
• Microsoft Fairlearn: Python library for evaluating and mitigating unfairness in models.
• Manual audit: for companies without a technical team, a quarterly analysis of selection rates by demographic groups using a spreadsheet is better than nothing.

Blind screening: the anonymous CV

Blind screening consists of removing from the CV all information that could reveal gender, age, ethnic origin or socioeconomic level before evaluation. It is one of the most effective techniques for reducing bias in the first selection phase.

What is anonymized

• Name: replaced by a code (Candidate #147).
• Photo: removed.
• Date of birth / age: removed.
• Address: reduced to city or removed.
• University: optionally, replaced by type (public/private) or removed.
• Previous company: optionally, replaced by sector and size.

AI for blind screening

AI can automate CV anonymization at scale. An LLM can read a CV, extract only the relevant information (skills, experience, achievements) and generate a standardized anonymous profile. This is faster and more consistent than manual anonymization.

Limitations of blind screening

• Only works in the first phase. In the interview, visual and verbal bias returns.
• Does not completely eliminate proxy bias (a 3-year career can reveal age).
• Some candidates want their identity known (diversity as a differentiator).
• Requires real organizational willingness. If the hiring manager asks to "see the original CV", the process loses its effect.

Audit checklist for selection systems

Use this checklist quarterly to audit any AI system in your selection process:

Data

• Is the training data representative of the target population?
• Have protected variables and known proxies been removed?
• Is the data up to date (< 2 years)?
• Does documentation of data origin and processing exist?

Model

• Has fairness testing been performed with at least 2 metrics?
• Is the disparate impact ratio >= 0.8 for all protected groups?
• Does a human override mechanism exist for each decision?
• Is the model revalidated with each significant change?

Process

• Are candidates informed of AI use (GDPR Art. 22, AI Act Art. 13)?
• Does a channel exist for candidates to challenge automated decisions?
• Are recruiters trained in algorithmic bias?
• Is there a designated person responsible for AI auditing in HR?

Governance

• Has a DPIA (Data Protection Impact Assessment) been conducted?
• Has the works council been consulted (where applicable)?
• Is the AI Act technical documentation complete?
• Does an incident response plan exist if bias is detected?

EEOC, EU anti-discrimination directive and GDPR

GDPR Art. 22: automated decisions

GDPR Art. 22 establishes that every person has the right not to be subject to a decision based solely on automated processing that produces legal effects or similarly significantly affects them. In selection, this means you cannot reject candidates fully automatically. There must be meaningful human intervention (not a human who simply approves what the machine says).

EU Directive 2000/78/EC (Equal treatment in employment)

Prohibits direct and indirect discrimination by religion, disability, age or sexual orientation in access to employment. Indirect discrimination is key: an apparently neutral practice that disadvantages persons of a protected group. An algorithm that filters by "availability to travel" can indirectly discriminate against persons with disabilities or family responsibilities.

EEOC (USA)

The US Equal Employment Opportunity Commission published specific guidance in 2023 on AI and employment discrimination. Their position: using AI in selection does not exempt the employer from liability for discrimination. If your algorithm discriminates, you discriminate. Even if you did not design it and even if it was not your intention.

Directive 2024/1760 (Corporate due diligence)

Although not AI-specific, this directive obliges companies to identify and mitigate adverse impacts on human rights in their value chain. AI selection processes fall within this scope.

Practical mitigation strategies

1. Diversify your recruitment team. If all interviewers have the same profile, historical data will reflect that bias. Diversity in the team is the first line of defense.
2. Use predefined evaluation criteria. Before seeing candidates, define which competencies you evaluate and how you score them. AI can help create scorecards. Not the other way around.
3. Implement blind screening in the first phase. It is the intervention with the best cost/impact ratio.
4. Audit quarterly. Calculate the fairness metrics. If you detect disparities, investigate before assuming the model "is right".
5. Train your recruiters. Algorithmic bias is not a technical problem. It is a problem of people using technology. Without training, the best tool is misused.
6. Document everything. The AI Act requires it, but even without regulation, documentation protects you legally and helps you improve the process.
7. Create a feedback channel. Candidates can detect bias you cannot see. Provide them a channel to report it.

Ejercicio practico

Puntos clave