Why AI ethics is a legal matter

AI ethics is not abstract philosophy. It is quantifiable legal risk. A company that deploys an AI system with racial bias in hiring faces discrimination lawsuits, administrative sanctions, and reputational damage. A bank whose credit scoring algorithm discriminates by gender violates equality regulations and financial regulation. A surveillance system with facial recognition that misidentifies a person can generate government liability.

For lawyers and compliance officers, AI ethics translates into three concrete dimensions:

• Regulatory compliance: the AI Act, the GDPR, the AI Liability Directive, and sector-specific regulations impose specific obligations of fairness, transparency, and explainability.
• Risk management: ethical risks (bias, opacity, lack of human oversight) translate into legal risks (sanctions, litigation, license loss) and financial risks (reputational damage, client loss).
• Competitive advantage: organizations that implement AI ethically and can demonstrate it gain the trust of clients, regulators, and investors. In a market where distrust of AI is growing, ethics is differentiation.

EU guidelines for trustworthy AI

In April 2019, the High-Level Expert Group on AI (AI HLEG) of the European Commission published the "Ethics Guidelines for Trustworthy AI." This document establishes 7 requirements that every AI system must meet to be considered trustworthy. These requirements are the philosophical foundation of the AI Act and the mandatory reference for any AI governance framework in Europe.

The 7 requirements for trustworthy AI

1. Human agency and oversight. AI systems must support human autonomy and decision-making. They must not manipulate, deceive, or coerce persons. Adequate human oversight must exist: human-in-the-loop (human decides), human-on-the-loop (human supervises), or human-in-command (human can intervene at any time).
2. Technical robustness and safety. Systems must be resilient to attacks (adversarial attacks), reliable, reproducible, and capable of managing errors safely. Includes a fallback plan: what happens when the system fails.
3. Privacy and data governance. GDPR and data protection compliance. Training data quality. Integrity and controlled access. Data minimization.
4. Transparency. Traceability of the system's decisions. Explainability of the results. Clear communication to users that they are interacting with an AI system and what its limitations are.
5. Diversity, non-discrimination, and fairness. Avoid unfair biases. Universal accessibility. Participation of diverse stakeholders in design and deployment.
6. Societal and environmental well-being. Consider the system's impact on society and the environment. Sustainability. Impact on employment and skills.
7. Accountability. Auditability of systems. Reporting and remedy mechanisms. Minimization of negative impacts. Continuous assessment.

Evaluate the following AI system against the EU's 7 requirements
for trustworthy AI:

System: [description of the system, sector, users, data it uses,
decisions it makes or supports]

For each requirement:
1. Current status: COMPLIANT / PARTIAL / NON-COMPLIANT
2. Evidence: what measures currently exist
3. Gaps: what is missing for full compliance
4. Legal risk: what regulation could be breached if not corrected
5. Recommendation: concrete action, owner, deadline

Prioritize recommendations by legal risk (highest to lowest).
Include a summary table at the end.

Algorithmic bias: types, detection, and mitigation

Algorithmic bias is the most tangible ethical problem of AI. It is not an abstract concept: it has real victims. People who do not get a loan, who do not pass a hiring filter, who receive an incorrect medical diagnosis, or who are wrongfully detained by a facial recognition error.

Types of bias in AI systems

• Historical bias: training data reflects past discrimination. If a hiring system is trained on the last 20 years of hiring data from a company that systematically hired fewer women in technical roles, the system will replicate that discrimination. Amazon's 2018 case is the canonical example.
• Representation bias: certain groups are under or overrepresented in the data. A facial recognition system trained predominantly on Caucasian faces will perform worse on people with dark skin. Joy Buolamwini's study (Gender Shades, 2018) demonstrated error rates of 34.7% for dark-skinned women vs 0.8% for light-skinned men.
• Measurement bias: proxy variables capture information they should not. Postal code as a credit risk predictor is a proxy for race and socioeconomic status. The system does not use race directly, but encodes it indirectly.
• Evaluation bias: the system's performance metrics do not capture the differential impact between groups. A system may have 95% overall accuracy but 70% for a minority group.
• Deployment bias: the system works correctly in the environment it was designed for, but is applied in a different context where its assumptions are not valid.
• Feedback loop bias: the system generates data used to retrain it, amplifying initial biases. Predictive policing is the classic example: more surveillance in a neighborhood generates more arrests, which feeds the model to predict more crime in that neighborhood.

Bias detection

Analyze the following AI system to detect potential biases:

System: [description]
Training data: [sources, size, period, variables]
Relevant protected variables: [gender, race/ethnicity, age,
disability, nationality, religion, sexual orientation]
Decisions it makes: [what the system decides or recommends]
Affected population: [who receives the decisions]

Requested analysis:
1. DATA BIAS:
   - Representativeness of each protected group in the data
   - Correlation between proxy variables and protected variables
   - Differential data quality between groups
2. MODEL BIAS:
   - Performance metrics by group (if data is available)
   - Variables with highest weight in the decision: is any a proxy?
3. DEPLOYMENT BIAS:
   - Training population vs actual production population
   - Contexts where the system should not be applied
4. FEEDBACK LOOPS:
   - Does the system's output generate data that feeds back into it?
   - Risk of bias amplification?
5. RECOMMENDATIONS:
   - Technical mitigation (rebalancing, fairness constraints, etc.)
   - Organizational mitigation (audit, HITL, monitoring)
   - Legal mitigation (DPIA, fundamental rights impact assessment)

Mitigation strategies

Bias mitigation operates in three phases of the system lifecycle:

• Pre-processing (data): rebalancing datasets, removing proxy variables, data augmentation for underrepresented groups, stratified sampling techniques. This is the preferred option: fixing the problem at its source.
• In-processing (model): fairness constraints in training, adversarial regularization (penalizing the model when it discriminates), multi-objective learning (optimizing accuracy AND fairness simultaneously).
• Post-processing (output): per-group threshold calibration, result adjustment to guarantee statistical fairness. This is the least preferred option: it masks the problem without solving it.

Fairness metrics and their trade-offs

Measuring the fairness of an AI system is not trivial. Multiple fairness metrics exist, and the problem is they are mutually incompatible in many scenarios. Chouldechova's impossibility theorem (2017) demonstrates that, except in trivial cases, not all definitions of fairness can be satisfied simultaneously.

Main fairness metrics

• Demographic parity: the proportion of positive outcomes is the same for all groups. If 30% of men get the loan, 30% of women must also get it. Problem: ignores whether actual qualification differs between groups.
• Equalized odds: the true positive and false positive rates are the same for all groups. The system gets it right (and wrong) at the same rate for men and women. Problem: may require sacrificing overall accuracy.
• Predictive parity: the precision of positive predictions is the same for all groups. If the system says "approved," the probability of it being correct is the same regardless of group. Problem: may generate disparate false negative rates.
• Individual fairness: similar individuals receive similar treatment. Problem: defining "similar" is subjective and requires a distance metric that may be biased itself.
• Counterfactual fairness: the decision would be the same if the individual belonged to another protected group. "If this person were male instead of female, would they get the same result?" Problem: requires modeling a counterfactual world, which is complex.

The inevitable trade-off

A concrete example. A credit scoring system for a bank:

• If you apply demographic parity, you approve the same percentage of loans for each ethnic group. But if one group historically has worse repayment capacity (due to structural inequality, not inherent ability), the bank will assume more risk with that group.
• If you apply equalized odds, the system is equally accurate for all groups. But the approval rate will differ if the underlying distributions are different.
• If you apply predictive parity, each approval is equally reliable. But the false negative rate (creditworthy people rejected) may be much higher for disadvantaged groups.

There is no universal correct answer. Metric choice is an ethical and legal decision, not a technical one. The lawyer and compliance officer must participate in that decision, not delegate it to the technical team.

An AI system of [type] uses these fairness metrics:
[list metrics applied]

Relevant protected groups: [list]

Analyze:
1. Which metrics are compatible with each other in this case?
2. Which are mutually exclusive?
3. Which metric is most appropriate according to:
   - Applicable regulation (AI Act, GDPR, sector-specific)
   - Impact on the fundamental rights of those affected
   - Legal risk for the organization
4. How to document the metric choice for an audit
5. What continuous monitoring to implement

Explainability: requirements and XAI techniques

Explainability (Explainable AI, XAI) is the ability of an AI system to provide comprehensible reasons for its decisions. It is not an academic whim: it is a right of EU citizens. Article 22 of the GDPR grants the right not to be subject to automated decisions with significant effects, and Article 15 grants the right to obtain "meaningful information about the logic involved."

Levels of explainability

• Global transparency: the model is comprehensible in its entirety. A linear regression with 5 variables is transparent: you can see exactly how each variable contributes to the result. A deep learning model with millions of parameters is not.
• Local explainability: for a specific decision, you can explain why the system reached that conclusion. "Your loan application was denied because your debt-to-income ratio (42%) exceeds the threshold (35%) and your credit history has 2 incidents in the last 12 months."
• Post-hoc interpretability: techniques that generate approximate explanations of opaque models. They do not explain how the model actually works, but provide a comprehensible approximation.

Main XAI techniques

• LIME (Local Interpretable Model-agnostic Explanations): generates a simple (linear) model that approximates the complex model's behavior around a specific decision. "For this patient, the most relevant factors were: age (40%), blood pressure (25%), and cholesterol (20%)."
• SHAP (SHapley Additive exPlanations): based on game theory, assigns each variable its marginal contribution to the result. More robust than LIME, but more computationally expensive.
• Attention maps: in language and vision models, show which parts of the input received the most model "attention." "The system detected fraud because it focused on the transaction time (3:47 AM) and the location (different from the usual country)."
• Counterfactual explanations: explain what should change to obtain a different result. "Your application was denied. If your income were 15% higher or if you reduced your debt by 5,000 EUR, it would be approved." This is the most useful technique for the affected citizen.
• Decision trees as surrogate model: train a decision tree that mimics the complex model. The tree is interpretable and provides the main decision rules.

Transparency obligations

Transparency in AI has three dimensions that are often confused:

• Interaction transparency: the person must know they are interacting with an AI system, not a human. The AI Act (Article 50) requires informing users when they interact with chatbots, AI-generated content (deepfakes, synthetic text), or biometric categorization systems.
• Decision transparency: the person affected by an automated decision has the right to know that an AI system participated in that decision, what data was used, and what logic was applied (GDPR, Articles 13-15).
• System transparency: regulatory authorities must be able to access the system's technical documentation, its training data, its performance metrics, and its impact assessments. This is what the AI Act requires for high-risk systems.

Transparency evaluation prompt

Evaluate the transparency obligations for this AI system:

System: [description]
Users: [who interacts with it]
Affected persons: [who receives the decisions]
AI Act classification: [high-risk / limited risk / minimal risk]
Sector: [health, finance, employment, education, etc.]

Analyze:
1. INTERACTION TRANSPARENCY:
   - Is the user informed they are interacting with AI? How?
   - Is there AI-generated content? Is it identified?
   - Does it comply with Article 50 AI Act?

2. DECISION TRANSPARENCY:
   - Do affected persons know AI participates in the decision?
   - Can they obtain an explanation of the logic? In what format?
   - Does it comply with Articles 13-15 GDPR?
   - Is there a contestation mechanism?

3. SYSTEM TRANSPARENCY:
   - Does complete technical documentation exist?
   - Are training data documented?
   - Are per-group performance metrics available?
   - Has an impact assessment been conducted?

4. GAPS and RECOMMENDATIONS:
   - What is missing for each level of transparency?
   - Legal priority of each gap
   - Recommended correction timeline

AI ethics committees

An AI ethics committee is the internal body of an organization responsible for evaluating, overseeing, and guiding the ethical use of artificial intelligence. It is not a direct legal requirement of the AI Act, but it is a recommended practice that demonstrates due diligence and that many sector-specific regulations are beginning to require.

Committee composition

An effective ethics committee needs diversity of perspectives. Recommended minimum composition:

• Legal/Compliance: 1-2 persons. Knowledge of AI Act, GDPR, sector-specific regulation. Their role: ensure that ethical decisions align with the legal framework.
• Technical: 1-2 persons. ML engineer, data scientist. Their role: assess the technical feasibility of ethical recommendations and explain technical limitations.
• Business: 1 person. Product or strategy director. Their role: assess the business impact of ethical decisions and balance with commercial objectives.
• HR/Diversity: 1 person. Their role: perspective on employee impact, equity in hiring, training, performance evaluation with AI.
• Independent external: 1 person. Academic, NGO, external expert. Their role: provide an independent perspective and avoid corporate "groupthink."
• User/affected person representative: when possible, include someone who represents the groups affected by the AI system's decisions.

Mandate and functions

Draft the Terms of Reference for an AI ethics committee
for an organization of [size] in the [sector] sector.

Include:
1. MISSION AND SCOPE:
   - Which AI systems does it evaluate (all or only high-risk?)
   - What decisions can it make (binding or advisory?)
   - Relationship with the Board/Senior Management

2. COMPOSITION:
   - Permanent members and their profiles
   - Appointment mechanism and tenure
   - Quorum and voting rules

3. FUNCTIONS:
   - Pre-deployment evaluation of AI systems
   - Review of ethical incidents
   - Issuance of internal guidelines
   - Training and awareness
   - Annual AI ethics report

4. EVALUATION PROCESS:
   - Who can request an evaluation?
   - Response timelines
   - Opinion format
   - Escalation mechanism if there is disagreement

5. RESOURCES:
   - Dedicated budget
   - Access to information and systems
   - Technical and legal support

Responsible AI frameworks

Major technology companies and international organizations have published responsible AI frameworks. Each has a different approach, but they share common principles. Knowing them lets you design your organization's internal framework by taking the best of each.

Microsoft Responsible AI Standard

Microsoft structures its framework around 6 principles: Fairness, Reliability and Safety, Privacy and Security, Inclusiveness, Transparency, Accountability. What differentiates it is its operationalization: each principle has goals, requirements, and concrete tools. It includes a mandatory "Responsible AI Impact Assessment" before launching any AI system. It is the most mature framework in terms of practical implementation.

Google AI Principles

Google published 7 principles in 2018, including 4 areas where it will NOT develop AI (weapons, mass surveillance, technologies violating international law, harmful purposes). Its strength is public transparency: it publishes progress reports and case studies where principles conflicted with business objectives. Its weakness: it has been criticized for the gap between declared principles and actual practices (Timnit Gebru case, 2020).

OECD AI Principles

Adopted by 46 countries, the OECD principles are the international public policy reference. 5 principles: inclusive growth, human-centered values, transparency, robustness and safety, accountability. Their impact: they are the basis of the G7 AI Code of Conduct and have influenced legislation in the EU, Canada, and Japan.

UNESCO Recommendation on AI Ethics

Adopted by 193 countries in November 2021. It is the broadest global normative instrument. It includes 10 principles and specific areas of action for governments. Relevant because many developing countries will use it as the basis for their national legislation.

Comparative for practical implementation

Compare these responsible AI frameworks for my organization:

Organization: [type, size, sector, primary jurisdiction]
AI systems in use or planned: [list]
Applicable regulation: [AI Act, GDPR, sector-specific]

Frameworks to compare:
1. Microsoft Responsible AI Standard
2. Google AI Principles
3. OECD AI Principles
4. IEEE Ethically Aligned Design
5. UNESCO Recommendation on AI Ethics

For each framework:
- Key principles
- Strengths for my organization
- Weaknesses or gaps
- Practical tools it offers
- Compatibility with AI Act / GDPR

Recommendation: which framework (or combination) to adopt and why.
3-phase implementation plan.

ISO 42001: AI management system

ISO/IEC 42001:2023 is the first international standard for artificial intelligence management systems (AIMS). Published in December 2023, it follows the high-level structure (HLS) common to ISO 9001, ISO 27001, and ISO 14001, which facilitates integration with other existing management systems.

ISO 42001 structure

• Clause 4: Context of the organization. Identify stakeholders, legal and regulatory requirements, AIMS scope. Includes an AI systems inventory and their risk classification.
• Clause 5: Leadership. Top management commitment, AI policy, roles and responsibilities. The ethics committee fits here.
• Clause 6: Planning. Risk and opportunity assessment of AI systems. Measurable responsible AI objectives. Risk treatment plan.
• Clause 7: Support. Resources, competencies, awareness, communication, documented information. Staff training in AI ethics.
• Clause 8: Operation. Operational planning and control. AI impact assessment. Data management. AI system lifecycle management (development, deployment, monitoring, retirement).
• Clause 9: Performance evaluation. Monitoring, measurement, analysis, evaluation. Internal audits. Management review.
• Clause 10: Improvement. Non-conformities, corrective actions, continual improvement.

ISO 42001 and the AI Act

ISO 42001 does not guarantee AI Act compliance, but it facilitates it enormously. The AI Act establishes the possibility that harmonized European standards (which CEN/CENELEC is developing) may align with ISO 42001. In practice, an organization certified to ISO 42001 will have significant ground covered for demonstrating AI Act conformity.

Design an ISO 42001 implementation plan for my organization:

Organization: [size, sector, number of AI systems,
existing certifications (ISO 27001, ISO 9001, etc.)]

Generate:
1. GAP ANALYSIS:
   - What do we already have that is compatible with ISO 42001?
   - What do we need to create from scratch?
   - What can we reuse from ISO 27001 / ISO 9001?

2. IMPLEMENTATION PLAN (12 months):
   - Phase 1 (months 1-3): scope definition, policy, inventory
   - Phase 2 (months 4-6): risk assessment, controls
   - Phase 3 (months 7-9): operation, training, documentation
   - Phase 4 (months 10-12): internal audit, correction, certification

3. REQUIRED RESOURCES:
   - Dedicated team
   - External consultancy (yes/no, for what)
   - Estimated budget
   - Tools

4. INTEGRATION:
   - With ISO 27001 (information security)
   - With GDPR / existing DPO
   - With AI Act compliance program

Ejercicio practico

Puntos clave