Informal presentation at the French data protection authority (CNIL), on 16 November 2018

1. Bias in algorithmic decision-making:
Standards, Algorithmic Literacy and
Governance
ANSGAR KOENE,
HORIZON DIGITAL ECONOMY RESEARCH INSTITUTE, UNIVERSITY OF NOTTIN GHAM
5TH SEPTEMBER 2018
1

2. Projects
UnBias – EPSRC funded “Digital Economy” project
◦ Horizon Digital Economy research institute, University of Nottingham
◦ Human Centred Computing group, University of Oxford
◦ Centre for Intelligent Systems and their Application, University of Edinburgh
IEEE P7003 Standard for Algorithmic Bias Considerations
◦ Multi-stakeholder working group with 70+ participants from Academia, Civil-society and Industry
A governance framework for algorithmic accountability and transparency – EP Science
Technology Options Assessment report
◦ UnBias; AI Now; Purdue University; EMLS RI
Age Appropriate Design
◦ UnBias; 5Rights
2

3. UnBias: Emancipating Users Against Algorithmic
Biases for a Trusted Digital Economy
Standards and policy
Stakeholder workshops
3
Youth Juries

4. Algorithms in the news
4

5. 5

6. Theme 1: The Use of Algorithms
Introduces the concept of algorithms
Activities include:
◦ Mapping your online world
◦ Discusses the range of online services that use algorithms
◦ What’s in your personal filter bubble?
◦ Highlights that not everyone gets the
same results online

7. Theme 1: The Use of Algorithms
Activities include:
◦ What kinds of data do algorithms use?
◦ Discusses the range of data collected and inferred by algorithms, and what happens to it
◦ How much is your data worth?
◦ From the perspective of you (the user) and the companies that buy/sell it

8. Theme 2: Regulation of Algorithms
Uses real-life scenarios to highlight issues
surrounding the use of algorithms, and asks
Who is responsible when things go wrong?
Participants debate both sides of a case and
develop their critical thinking skills

9. Theme 3: Algorithm Transparency
The algorithm as a ‘black box’
Discusses the concept of meaningful
transparency and the sort of information that
young people would like to have when they are
online
◦ What data is being collected about me?
◦ Why?
◦ Where does it go?

11. 11
Fairness Toolkit
http://unbias.wp.horizon.ac.uk
https://oer.horizon.ac.uk/5rights-youth-juries

12. 12

13. 13
IEEE P7000: Model Process for Addressing Ethical Concerns During System Design
IEEE P7001: Transparency of Autonomous Systems
IEEE P7002: Data Privacy Process
IEEE P7003: Algorithmic Bias Considerations
IEEE P7004: Child and Student Data Governance
IEEE P7005: Employer Data Governance
IEEE P7006: Personal Data AI Agent Working Group
IEEE P7007: Ontological Standard for Ethically Driven Robotics and Automation Systems
IEEE P7008: Ethically Driven Nudging for Robotic, Intelligent and Autonomous Systems
IEEE P7009: Fail-Safe Design of Autonomous and Semi-Autonomous Systems
IEEE P7010: Wellbeing Metrics Standard for Ethical AI and Autonomous Systems
IEEE P7011: Process of Identifying and Rating the Trustworthiness of News Sources
IEEE P7012: Standard for Machines Readable Personal Privacy Terms

14. Algorithmic systems are socio-technical
Algorithmic systems do not exist in a vacuum
They are built, deployed and used:
◦ by people,
◦ within organizations,
◦ within a social, political, legal and cultural context.
The outcomes of algorithmic decisions can have significant impacts
on real, and possibly vulnerable, people.

15. P7003 - Algorithmic Bias Considerations
All non-trivial* decisions are biased
We seek to minimize bias that is:
◦ Unintended
◦ Unjustified
◦ Unacceptable
as defined by the context where the system is used.
*Non-trivial means the decision space has more than one possible outcome and the choice is not
uniformly random.

16. Causes of algorithmic bias
Insufficient understanding of the context of use.
Failure to rigorously map decision criteria.
Failure to have explicit justifications for the chosen criteria.

17. 17
Algorithmic Discrimination

18. 18
Complex individuals reduced to
simplistic binary stereotypes

19. Key question when developing or
deploying an algorithmic system
19
 Who will be affected?
 What are the decision/optimization criteria?
 How are these criteria justified?
 Are these justifications acceptable in the context where the
system is used?

20. 20
P7003 foundational sections
 Taxonomy of Algorithmic Bias
 Legal frameworks related to Bias
 Psychology of Bias
 Cultural aspects
P7003 algorithm development sections
 Algorithmic system design stages
 Person categorization and identifying affected population groups
 Assurance of representativeness of testing/training/validation data
 Evaluation of system outcomes
 Evaluation of algorithmic processing
 Assessment of resilience against external manipulation to Bias
 Documentation of criteria, scope and justifications of choices

21. Related AI standards activities
British Standards Institute (BSI) – BS 8611 Ethics design and application of robots
ISO/IEC JTC 1/SC 42 Artificial Intelligence
◦ SG 1 Computational approaches and characteristics of AI systems
◦ SG 2 Trustworthiness
◦ SG 3 Use cases and applications
◦ WG 1 Foundational standards
Jan 2018 China published “Artificial Intelligence Standardization White Paper.”

22. A governance framework for
algorithmic accountability and
transparency
EPRS/2018/STOA/SER/18/002
ANSGAR KOENE, UNIVERSITY OF NOTTINGHAM
RASHIDA RICHARDSON & DILLON REISMAN, AI NOW INSTITUTE
YOHKO HATADA, EMLS RI
HELENA WEBB, M. PATEL, J. LAVIOLETTE, C. MACHADO, UNIVERSITY OF OXFORD
CHRIS CLIFTON, PURDUE UNIVERSITY
25TH OCTOBER 2018

23. Awareness raising: education,
watchdogs and whistleblowers
 “Algorithmic literacy” - teaching core concepts: computational thinking, the role of data
and the importance of optimisation criteria.
 Standardised notification to communicate type and degree of algorithmic processing in
decisions.
 Provision of computational infrastructure and access to technical experts to support data
analysis etc. for “algorithmic accountability journalism”.
 Whistleblower protection and protection against prosecution on grounds of breaching
copyright or Terms of Service when doing so serves the public interest.

24. Accountability in public sector use
of algorithmic decision-making
Adoption of Algorithmic Impact Assessment (AIA) for algorithmic systems used for public service
1. Public disclosure of purpose, scope, intended use and associated policies, self-assessment
process and potential implementation timeline
2. Performing and publishing of self-assessment of the system with focus on inaccuracies, bias,
harms to affected communities, and mitigation plans for potential impacts.
3. Publication of plan for meaningful, ongoing access to external researchers to review the system.
4. Public participation period.
5. Publication of final AIA, once issues raised in public participation have been addressed.
6. Renewal of AIAs on a regular timeline.
7. Opportunity for public to challenge failure to mitigate issues raised in the public participation
period or foreseeable outcomes.

25. Regulatory oversight and Legal liability
 Regulatory body for algorithms:
 Risk assessment
 Investigating algorithmic systems suspected of infringing of human rights.
 Advising other regulatory agencies regarding algorithmic systems
 Algorithmic Impact Assessment requirement for systems classified as causing potentially
severe non-reversible impact
 Strict tort liability for algorithmic systems with medium severity non-reversible impacts
 Reduced liability for algorithmic systems certified as compliant with best-practice standards.

26. Global coordination for algorithmic governance
 Establishment a permanent global Algorithm Governance Forum (AGF)
 Multi-stakeholder dialog and policy expertise related to algorithmic systems
 Based on the principles of Responsible Research and Innovation
 Provide a forum for coordination and exchanging of governance best-practices
 Strong positions in trade negotiations to protect regulatory ability to
investigate algorithmic
systems and hold parties accountable for violations of European laws and
human rights.

27. Age Appropriate Design
27

28. What is the Age-Appropriate Design Code?
The Age-Appropriate Design Code sits at section 123 of the UK Data Protection Act 2018
(“DPA”) and will set out the standards of data protection that Information Society Services
(“ISS”, known as online services) must offer children. It was brought into UK legislation by
Crossbench Peer, Baroness Kidron, Parliamentary Under-Secretary, Department for Digital,
Culture, Media and Sport, Lord Ashton of Hyde, Opposition Spokesperson, Lord Stevenson
of Balmacara, Conservative Peer, Baroness Harding of Winscombe and Liberal Democrat
Spokesperson, Lord Clement-Jones of Clapham.
28

29. 29

30. ICO to draft code by 25 October 2019
30

31. Thank you
31
http://unbias.wp.horizon.ac.uk

32. Biography
• Dr. Koene is Senior Research Fellow at the Horizon Digital Economy Research
Institute of the University of Nottingham, where he conducts research on
societal impact of Digital Technology.
• Chairs the IEEE P7003TM Standard for Algorithms Bias Considerations working
group., and leads the policy impact activities of the Horizon institute.
• He is co-investigator on the UnBias project to develop regulation-, design- and
education-recommendations for minimizing unintended, unjustified and
inappropriate bias in algorithmic systems.
• Over 15 years of experience researching and publishing on topics ranging from
Robotics, AI and Computational Neuroscience to Human Behaviour studies and
Tech. Policy recommendations.
• He received his M.Eng., and Ph.D. in Electical Engineering & Neuroscience,
respectively, from Delft University of Technology and Utrecht University.
• Trustee of 5Rights, a UK based charity for Enabling Children and Young People
to Access the digital world creatively, knowledgeably and fearlessly.
Dr. Ansgar Koene
ansgar.koene@nottingham.ac.uk
https://www.linkedin.com/in/akoene/
https://www.nottingham.ac.uk/comp
uterscience/people/ansgar.koene
http://unbias.wp.horizon.ac.uk/