I4ADA 2019 - Presentation Cedric Wachholz

UNESCO Event / Name / other Date
Cédric Wachholz,
Chief
ICT in Education, Culture and Science Section
UNESCO
Accountability in the Digital Age

1.Artificial Intelligence
challenges – mind the gaps!
2. Accountability: key
dimensions & requirements
3. Action points
4. What UNESCO does & will do
Accountabitliy in the digtial age

AI Challenges – accountability gaps
Need for accountability: examples

• Inclusion: Digital & knowledge divides
Percentage of individuals with ICT skills,
by development status, 2017 Geographic distribution of issuers of ethical AI
guidelines by number of documents released

• Inclusion
• Privacy

• Inclusion
• Privacy
• Biases

• Inclusion
• Privacy
• Biases
• Blackbox – explainability,
transparency and human control
• Complexity: Multiple components
and hands

• Overtrust
• Distrotion/ Disruption of governance
models
Credit:Stanfordedu/Reuters/MortezaNikoubazl,
Unsplash/MarkusSpiske,DreaSullivan

• Overtrust
• Distrotion/ Disruption of governance
models
• Portability across borders challenge
• Pace of technology innovation vs
policy responses
• How to ensure accountability in this
complex context?

Accountability (A.): key dimensions
• A.: Acknowledgement of responsability
• Pre-factum A. as an AI system feature
• Post-factum A.: Responsibility
determination
• A. as a feature in sociotechn. systems
• Algorithmic accountability and justice
• Human rights based, inclusive, relational
def.

Accountability: key requirements
• Transparency
• Explainability
Different approaches: Understanding the
• overall system (how does it work?) or a
• particular outcome (how does it
behave?)

How to translate A. concepts into reality?
• Maximize benefits, minise risks – allocate
resources in propotion to risks to society
(impact and likelihood)
• Human rights – existing mechanisms
• Governance (by EPRS): demand side market
solutions; supply side market solutions;
companies'self organisation; branches' self-
regulation; co-regulation; and state
intervention.
• Global dimensions of algorithmic governance

Ethics
A Map of Ethical
and Rights-based
approaches
From: Berkman
Klein Center for
Internet & Society
at Harvard
University
https://ai-
hr.cyber.harvard.e

Ethics
A Map of Ethical
and Rights-based
approaches
From: Berkman Klein Center for
Internet & Society at Harvard
University
(July’19 draft)
https://ai-
hr.cyber.harvard.edu/primp-
viz.html

Ethics
UNESCO
World Commission on
the Ethics of Scientific
Knowledge and
Technology (COMEST)
Working Group on AI

AI - Digital Transformation: why, what and how?
Laboratory of
Ideas,
innovative
models
Capacity
Development
Standard
Setting
Policy Advice
Communication
and
Information
• Human Rights
• Access
• Journalism
• Digital skills
Social and
Human Sciences
• Standard Setting
for Ethics of AI
• Ethical reflection
• Social implications
Culture
• Diversity of
Expression
• Inclusion
Natural Sciences
• Disaster Risk
Reduction
• Climate Sciences
Education
• AI for Teaching/
Learning
• EMIS
• Preparing for the
future of work

Tentative Roadmap for Standard-Setting Instrument
on human rights based, human centred Ethics of AI
Done: Feasibility study; Recommendation by
Executive Board to go forward
Upcoming:
Nov 2019: Decision by the 40th General Conference of
UNESCO to launch the process
Apr 2020: First meeting of Ad Hoc Expert Group (AHEG) to
prepare a preliminary draft
Apr-Jul 2020: Multistakeholder consultations on preliminary draft
(including online)
Followed by 8 steps of physical meetings and online consultations
until:
Nov 2021: Examination of final draft by the 41st General
Conference of UNESCO
More information at: https://en.unesco.org/artificial-intelligence

Cédric Wachholz,
Chief
ICT in Education, Culture and Science Section
UNESCO
Accountability in the Digital Age
Thank you for your attention!

Accountability gap-
1) of AI systems explainability,
2) pre-deployment
3) Governance decisions: When do we accept to replace human diginity,
understood as the uniqzue chartisterics of a specific individual, which is the key aspect
of discretion within decision making.
=Y governance choise about the removal, reductino or reconfiguartion of discretion

AI – affecting HR, other types of societal harms
Wider threats to all HR by new & emerging tech
• Risks to privacy
• potential for discrimation and inequality
• wider threats to all HR posed by new and emerging technologies
Address Risks throught the technology itself:
• Algorithimc accountability
• Make more transparent, more explainable

Accountability - EC
Accountability: AI systems should be auditable and covered by existing protections for
corporate whistleblowers. Negative impacts of systems should be acknowledged and
reported in advance.
-closely linked to the principle of fairness
-mechanisms be put in place to ensure responsibility and accountability for AI systems
and their outcomes, both before and after their development, deployment and use.
-Identifying, assessing, reporting and minimising the potential negative impacts of AI
systems is especially crucial for those (in)directly affected.
Due protection must be available for whistle-blowers, NGOs, trade unions or other
entities when reporting legitimate concerns about an AI-based system. The use of
impact assessments (e.g. red teaming or forms of Algorithmic Impact Assessment) both
prior to and during the development, deployment and use of AI systems can be helpful
to minimise negative impact. These assessments must be proportionate to the risk that
the AI systems pose.
-Trade-offs:
Identifying, assessing, reporting and minimising the potential negative impacts of AI
systems is especially crucial for those (in)directly affected. Due protection must be
available for whistle-blowers, NGOs, trade unions or other entities when reporting
legitimate concerns about an AI-based system. The use of impact assessments (e.g. red
teaming or forms of Algorithmic Impact Assessment) both prior to and during the

11/20/2019UNESCO 25
Dimensions of Accountability
1.Universal access
2.Liability for AI actions

11/20/2019UNESCO 26
AI Activity in Africa and LAC

11/20/2019UNESCO 27
Liability for AI Actions
• Autonomous
vehicles
• Data collection
• Algorithmic bias and
discrimination
• Disinformation and
filter bubbles

11/20/2019UNESCO 28
• Forum on AI in Africa
• Ericsson and UNESCO initiative to
scale up skill development in AI
• Deep Learning Indaba AI and
Fairness workshops
• International Conference on
Embedding AI in Education Policy
and Practice
• COMEST Study on the ethics of AI
UNESCO’s Work

Some issues to consider for addressing ethical
implications of AI
• Central ethical concerns, also for CIOs:
1. Ethical implications beyond algorithms:
include technological and social AI
ecosystems
2. The design of AI systems should be human
centred and inclusive
3. Users: AI literacy & critical thinking
4. AI scientific practice and decision-making
require human evaluation and control
5. Address gender and other biases from the
outset
Questions:
1. How will AI interact with datasets, IoT? How
will AI results be used in societies?
2. Diverse development teams? Un-/concious
biases built in?
3. Are users able to evaluate the validity of AI
results?
4. How to address blackbox challenges from
the outset? How to ensure transparency,
explainability and also: accountability?
5. Which datasets used/ gaps exist – and how
could they result in biases?

Future action: when developing and deploying AI
systems...
-Build internal capacities on AI (understanding, ethics)
-Ensure Long-term and consistent monitoring of AI
systems
• Launch an ethics review committee
• Establish a mechanism to monitor and analyze the information
produced by AI systems to identify new ethical risks
• Set-up mechanism for follow-up action
• Ensure diversity
-Join, contribute to and use the standard setting process

Accountability Gap
Mind the Gap!
Accountability is a key dimension of decision-making in human and artificial
intelligence, and it is crucial to any democratic, tolerant and inclusive society.

Policies

National AI Strategies
Source: Planes-Satorra and Paunov (2019)

National AI Strategies: Main
Objectives
• Strengthen (national) research in AI
• Strengthen capabilities in AI: infrastructures (high power
computing; data; technical standards); skills
• Support businesses’ adoption of AI (notably SMEs)
• Boost the use of AI by public administrations
• establish inclusive approaches and ethical standards

National AI Strategies: Main
Instruments
• Create new research centers
• Fund chairs in universities
• Sponsor new programs in Higher Education
• Finance special funds for research and start-ups
• Create incubators, hubs, clusters etc.
• DARPA type of support for breakthrough innovation
• Establish testing facilities
• Fund research focused on ethical & societal issues
• Create AI observatories (overall coordination and monitoring of
policies)

Provisional Conclusions
• AI will be a major shaping factor in the future for economies and
societies
• AI is a key domain for government to influence
• Governement are in the back seat at the moment
• Europe is lagging behind the US and China, it needs to catch up
fast

Human
Rights

Human rights
China’s “first cyber-dissident” has been sentenced to 12 years in jail
Huang Qi set up a website that monitors human rights abuses. (BBC)
+ Here’s what the West gets wrong about China’s social credit system. (Wired)

Ethics

AI – From Ethical Principles
to Normative Standards
Ms Dafna Feinholz,
Chief, Bioethics and Ethics of Science and
Technology
Mr Cédric Wachholz,
Chief ICT in Education, Culture and Science
Section
UNESCO

UNESCO’s Internet Universality and ROAM approach
• ROAM approach
• Protecting human Rights:
freedom of expression,
privacy, access to information
• Openness
• Inclusive Access to
information
• Multistakeholder approach
• X-cutting issues

Tentative Roadmap for Standard-Setting Instrument
on Ethics of AI
Done: Feasibility study; Recommendation by Executive Board to go
forward
Upcoming:
Nov 2019: Decision by the 40th General Conference of UNESCO to launch the process
Apr 2020: First meeting of Ad Hoc Expert Group (AHEG) to prepare a preliminary draft
Apr-Jul 2020: Multistakeholder consultations on preliminary draft (including online)
=> UNESCO will reach out to you for your input!
Followed by 8 steps of physical meetings and online consultations until:
Nov 2021: Examination of final draft by the 41st General Conference of UNESCO
More information at: https://en.unesco.org/artificial-intelligence

AI – From Ethical Principles
to Norrmative Standards
Thank you for your attention!
Ms Dafna Feinholz (d. feinholz (at) unesco . org)
Chief, Bioethics and Ethics of Science and
Technology
Mr Cédric Wachholz (c. wachholz (at) unesco .
org)
Chief ICT in Education, Culture and Science
Section
UNESCO

Seven system tenets for trustworthy AI:
(UNESCO’s approach to be defined in standard setting process)
•Human Rights and human control of technology
•Privacy and data governance
•Technical robustness and safety
•Transparency and explainability
•Diversity, non-discrimination and fairness
•Human, societal and environmental wellbeing
•Accountability

Some questions to ask when developing and
deploying AI systems
• What type of datasets are being used to train AI systems? What are the
gaps in these datasets and how this could result in bias or discrimination in
results (special attention to gender, racial and cultural bias)?
• What are the assumptions built into the design of AI algorithms? How
diverse is the development teams? Are there biases that have been
embedded into the algorithms by the developers, whether consciously or
subconsciously?
• How are the results produced by the AI systems used? Will they result in
discrimination, deepen inequalities, or create new injustices in any way?
Will they inadvertently harm our intended beneficiaries? Do users of these
systems have a critical view of the results produced, so as not to take for
granted that such results are always accurate or correctly reflect social and
cultural reality?
• These concerns require long-term and consistent monitoring of AI
systems, and it is advisable that each organization should establish a
mechanism to monitor and analyze the information produced by AI
systems to identify ethical risks that may emerge.
• Possibly through an ethics review committee, not only in the development phase, but also in the
post-deployment phase of AI systems

UNESCO COMEST Working Group on ethics
of AI
• Human rights
• inclusiveness
• explainability
• transparency
• awareness and literacy
• responsibility
• accountability
• democracy
• good governance
• sustainability
• ...
(https://unesdoc.unesco.org/ark:/48223/pf0000367823)
UNESCO COMEST
Working Group on
ethics of AI

UNESCO COMEST WG on ethics of AI: human rights, inclusiveness,
flourishing, autonomy, explainability, transparency, awareness and
literacy, responsibility, accountability, democracy, good governance,
sustainability (https://unesdoc.unesco.org/ark:/48223/pf0000367823)

AI - Digital Transformation: why, what and how?
Communication
and
Information
• Human Rights
• Access
• Journalism
• Digital skills
Social and
Human Sciences
• Standard Setting
for Ethics of AI
Culture
• Diversity of
Expression
• Inclusion
Natural Sciences
• Disaster Risk
Reduction
• Climate Sciences
Education
• AI for Teaching/
Learning
• EMIS
• Preparing for the
future of work
• ‘Pragmatic’ AI: machines that are
capable of independently
performing tasks that would
otherwise require human
intelligence and agency
• Challenges
• Inequality
• Deliberate/accidental violation
of human rights
• Transparency and
accountability (blackbox +
biases)
• Speed and scope of regulation
• Future of work

AI ethics DG EU lunch speaking points
Intelligence artificielle
Eléments de langage sur votre vision concernant les principes éthiques de l’IA
L’IA produit des changements importants dans notre façon de comprendre, de
concevoir et de mettre en pratique l’éducation, la science, la culture et la
communication. Même si l’IA offre de nouvelles possibilités, diverses questions se
posent, concernant le respect de la dignité humaine, l’équité, la démocratie, l’égalité
d'accès aux ressources et aux avantages du progrès, le changement de la société.
L’étude sur l’éthique de l’IA du Groupe de travail élargi de la Commission mondiale
d’éthique des connaissances scientifiques et des technologies (COMEST) examine les
questions d’ordre éthique relevant du mandat de l’UNESCO. Ses conclusions sont
présentées dans le document du Conseil exécutif 206 EX/42.
L’IA oblige à repenser le rôle sociétal de l’éducation. Les idées actuelles concernant «
l’apprentissage tout au long de la vie » devront peut-être être étendues à un modèle
d’éducation continue, comprenant l’élaboration d’autres types de cycles

G20 AI Principles
Section 1: Principles for responsible stewardship of trustworthy AI
1.1. Inclusive growth, sustainable development and well-being
1.2. Human-centered values and fairness
1.3. Transparency and explainability
1.4. Robustness, security and safety
1.5. Accountability

Disinfo
online hate
speech

AI deepfakes are now as simple as
typing whatever you want your
subject to say
AI deepfakes are now as simple as
typing whatever you want your
subject to say
https://www.theverge.com/2019/6/10/18
659432/deepfake-ai-fakes-tech-edit-
video-by-typing-new-words
Gesendet von Flipboard

Forget fake news—nearly all deepfakes are being made for porn
The internet hosts at least 14,678 deepfakes, according to a report by DeepTrace, a
company that builds tools to spot synthetic media. But most weren’t created to mess
with elections.
Back to the beginning: Deepfakes arrived on the scene in late 2017. The word was
originally used to describe AI-generated fake porn, but now it refers to any kind of
manipulated video. This has stoked fears about the end of truth and the potential of
deepfakes to swing elections.
Now: Most of the videos aren’t about politics. A full 96% of deepfakes are still plain
old fake porn. All the fake porn contains women, mostly famous actresses and
musicians.
Fighting back with law: The issue has caught the attention of legislators. In California,
Governor Gavin Newsom just signed into law two bills that limit what people can do
with deepfakes. One law makes it illegal to make and distribute a malicious deepfake
of a politician within two months of an election. The second gets closer to how the
manipulations are really being used. It lets people sue if their picture is used in
deepfake porn without consent.

Three threats posed by deepfakes that technology won’t solve
The harm caused by deepfakes can’t be fixed without social and legal solutions,
reports Angela Chen.
Imagine: Wouldn’t it be great if a perfectly accurate deepfake detector existed, able to
instantly label ever video that had been manipulated with AI? It might sound like just
what we need: more technology to fight the growing problem. And it’s been the
overwhelming response from technologists.
Some bad news: Technology alone cannot address the main harms: disinformation
and harassment.
Why? Deepfake detectors can’t tell us what should—and shouldn’t—be taken down.
Only better moderation can do that. They also might not help the people who need
protection most: marginalized groups like women, people of color, and LGBT people.
So we shouldn’t build any detection tools without consulting those most affected.
Finally, deepfake detection comes too late to help victims. The damaging fake material
is already out there. The answer? New laws. Read the full story here.

Deepfakes could anonymize people in videos
while keeping their personality
AI could generate faces that match the expressions of anonymous subjects to grant
them privacy—without losing their ability to express themselves.
The news: A new technique uses generative adversarial networks (GAN), the
algorithm behind deepfakes, to anonymize someone in a photo or video.
How it works: The algorithm extracts information about the person’s facial
expression by finding the position of their eyes, ears, shoulders, and nose. It then
uses a GAN, trained on a database of 1.5 million face images, to create an entirely
new face with the same expression and blends it into the original photo, retaining the
same background.
Why am I always the bad guy?: Face anonymization is typically used to protect the
identity of people such as whistleblowers, in photos and footage. This technique
would make it far safer for them and also allow them to convey their personality when
speaking. It also demonstrates a new value proposition for GANs, which have
developed a bit of a bad reputation.

Here's how social media firms should tackle online hate, according to ph
Policing online hate groups is like a never-ending game of whack-a-mole: moderators remove one neo-Nazi page on Facebook, only for a
The scale of the problem: The team examined the dynamics of “hate communities” on social media platforms Facebook and VKontakte
The current approach is broken: Their model predicts that policing within a single platform, like Facebook, can actually make the sprea
What can be done? The researchers suggested some policies which might help:
— Ban relatively small hate clusters, rather than the largest. These are easier to locate, and eliminating them can help to stop the larger c
— Ban a small number of users chosen randomly from online hate clusters. This avoids banning whole groups of users, which results in o
— Platform administrators should promote the organization of clusters of “anti-hate” users to counteract hate clusters.
— Introduce an artificial group of users to sow division between hate groups. These sorts of battles could bring down large hate clusters t
How likely is it? Some of the policies, especially the latter two, are pretty left field. But given current approaches are so profoundly ineffe

A new tool uses AI to spot text
written by AI
AI algorithms can generate text convincing enough to fool the average human—potentially
providing a way to mass-produce fake news, bogus reviews, and phony social accounts.
Thankfully, AI can now be used to identify fake text, too.
The news: Researchers from Harvard University and the MIT-IBM Watson AI Lab have
developed a new tool for spotting text that has been generated using AI. Called the Giant
Language Model Test Room (GLTR), it exploits the fact that AI text generators rely on
statistical patterns in text, as opposed to the actual meaning of words and sentences. In
other words, the tool can tell if the words you’re reading seem too predictable to have been
written by a human hand.
The context: Misinformation is increasingly being automated, and the technology required
to generate fake text and imagery is advancing fast. AI-powered tools such as this may
become valuable weapons in the fight to catch fake news, deepfakes, and twitter bots.
Faking it: Researchers at OpenAI recently demonstrated an algorithm capable of dreaming
up surprisingly realistic passages. They fed huge amounts of text into a large machine-
learning model, which learned to pick up statistical patterns in those words. The Harvard
team developed their tool using a version of the OpenAI code that was released publicly.
How predictable: GLTR highlights words that are statistically likely to appear after the

Yes, FaceApp could use your face—but not for face recognition
There’s a lot that the viral photo-editing app could do with a giant database of faces.
The context: FaceApp, a photo-editing app that uses AI to touch up your face, has been around since 2017, but a new feature
that “ages” users has catapulted it back into popularity.
The concern: It’s amassed more than 150 million photos of people’s faces since launch—and its terms of service stipulate that
the company can use the photos however it wants, in perpetuity. The company behind it, Wireless Lab, said it deletes most
images from its servers within 48 hours of upload, and doesn’t share data to third parties. Despite this, some Democratic
members of the US Congress are now calling for an FBI investigation into the company.
Should they be? It’s highly unlikely they’re being used to train face recognition algorithms. However, there are other ways to
use a giant database of faces. Read more on the possibilities here.
Does it matter? There are many other open-source databases of face photos that may or may not already include your
likeness. FaceApp just highlights how much we’ve already lost control of our digital data.
7 Privacy concerns over
FaceApp have been
overblown
If the face-ageing app scares
you, you should really stop
uploading your image
anywhere online. (Wired UK)
+ It has pretty much exactly
the same terms and
conditions as Facebook.
(Wired)
+ The US Senate Minority
leader Chuck Schumer has
called for a national security
investigation (why???)
(Reuters)

Facebook isn’t doing enough to tackle
misinformation, say fact-checkersFull Fact, a charity that has been part of Facebook’s third-party fact-checking program
for six months, says the company needs to be more transparent and expand the
program to its other platforms, like Instagram.
Background: It’s the first public report from a Facebook fact-checker so a useful
insight into how that process is working behind the scenes. You can read it here.
How the process works: Facebook users flag posts they suspect are false. They’re then
reviewed by Full Fact, which labels whether they are true or not. If false, the post
won’t disappear but it will be shown to fewer people, by appearing lower down in
their news feeds. Facebook is paying for the initiative, but it’s run by independent
non-profits and charities.
Read between the lines: Although the document is written using diplomatic language,
it’s clear that Full Fact has some major concerns. Specifically, it says it’s worried that
Facebook is rapidly increasing the scale of the fact-checking scheme to cover 42
languages worldwide, before it can adequately speed up the response, or volume of
content handled. It says Facebook does not share enough data with fact-checkers for
them to evaluate the content or impact of posts.
Recommendations: Full Fact makes 11 recommendations, which you can read here.
They’re mostly technical feedback for Facebook, including a call for a “more context
needed” rating, a rating for satire or pranks, and new tools which can flag repeatedly-
posted harmful content.

Monopoly creator/ Entrance barrier?
To which extend do laws/regulation on, e.g. 24h cleaning of hate & violence inciting
speech on social media platforms, do represent a business barrier for new market
entries, as only the big techs can address (through tech and humans) these
requirements?

Election hackers targeted 50
states in 2016. Here’s how the US
will prep for 2020.
Election hackers targeted 50 states in 2016. Here’s how the US will prep for 2020.
In 2020, all state voter databases will be protected by sensors that alert the federal
government to hacking attempts, reports Patrick Howell O’Neill.
Back in 2016: Russian hackers likely targeted all 50 states in various ways during the
2016 election, a Senate report found. The findings highlighted numerous failures on
the part of the US government, but there’s no evidence the activity resulted in any
data or changed votes.
Better planning: For the 2020 elections, 100% of voter databases will be covered by
“Albert sensors” designed to spot malicious activity. These will provide intrusion
detection and reporting.
Go further: Democratic senator Ron Wyden is calling on Congress to establish
mandatory, nation-wide cybersecurity requirements. Read the full story here.
+ Also: British hacker Marcus Hutchins avoided up to 10 years behind bars, receiving
no prison time and one year of supervised release after being charged with creating

A new tool uses AI to spot text
written by AI
AI algorithms can generate text convincing enough to fool the average human—potentially providing a way to mass-produce fake
news, bogus reviews, and phony social accounts. Thankfully, AI can now be used to identify fake text, too.
The news: Researchers from Harvard University and the MIT-IBM Watson AI Lab have developed a new tool for spotting text that
has been generated using AI. Called the Giant Language Model Test Room (GLTR), it exploits the fact that AI text generators rely on
statistical patterns in text, as opposed to the actual meaning of words and sentences. In other words, the tool can tell if the words
you’re reading seem too predictable to have been written by a human hand.
The context: Misinformation is increasingly being automated, and the technology required to generate fake text and imagery is
advancing fast. AI-powered tools such as this may become valuable weapons in the fight to catch fake news, deepfakes, and twitter
bots.
Faking it: Researchers at OpenAI recently demonstrated an algorithm capable of dreaming up surprisingly realistic passages. They
fed huge amounts of text into a large machine-learning model, which learned to pick up statistical patterns in those words. The
Harvard team developed their tool using a version of the OpenAI code that was released publicly.
How predictable: GLTR highlights words that are statistically likely to appear after the preceding word in the text. As shown in the
passage above (from Infinite Jest), the most predictable words are green; less predictable are yellow and red; and least predictable
are purple. When tested on snippets of text written by OpenAI’s algorithm, it finds a lot of predictability. Genuine news articles and
scientific abstracts contain more surprises.
Mind and machine: The researchers behind GLTR carried out another experiment as well. They asked Harvard students to identify
AI-generated text—first without the tool, and then with the help of its highlighting. The students were able to spot only half of all
fakes on their own, but 72% when given the tool. “Our goal is to create human and AI collaboration systems,” says Sebastian
Gehrmann, a PhD student involved in the work.
If you're interested, you can try it out for yourself.

Congress’s first deepfakes hearing is
today. Here's what you need to know.
With the election approaching, lawmakers need to do something about
the explosion in manipulated media, writes Karen Hao.
What’s happening: The House of Representatives will hold its first
dedicated hearing on deepfakes today. In parallel, Representative Yvette
Clarke will introduce a bill on the same subject.
A hot topic: There’s growing alarm over manipulated media after a
doctored video of Nancy Pelosi—edited to make her appear drunk—
spread online last month, and an AI-generated deepfake of Mark
Zuckerberg was uploaded to Instagram this week.
The proposed law: It would require creators of deepfakes to
automatically add watermarks to forged creations, and force social media
firms to build better manipulation detection into their platforms. Finally, it
would fine or jail people who create malicious deepfakes. Read the full
story here.

Privacy

Surveillance
3 How London became a test case for using face recognition in democracies
It’s the second most-CCTV monitored city in the world, after Beijing. (FT)
+ Another city in the US plans to ban it: Cambridge, Massachusetts. (Gizmodo)

Today, we’re looking at how competition evolves AI agents, a prediction about an
impending face recognition ban, and a technique that anonymizes people without
removing their personalities. You can view our informal archive here.
A face recognition ban is coming to the US. San Francisco and Oakland, California, and
Somerville, Massachusetts, have outlawed certain uses of face recognition
technology, with Portland, Oregon, potentially soon to follow. That trend will soon
spread to states, according to Mutale Nkonde, a Harvard fellow and AI policy advisor,
and there will eventually be a federal ban on some uses of the technology.
Exactly what kind of uses will be banned, it’s not yet clear: while some cities have
banned police departments form using it, Portland’s focus is restricting use by the
private sector. And the debate is not confined to the US. In the UK, there is growing
concern over the use of live face recognition after it emerged that a property
developer had been collecting images of people’s faces in an area of London for two
years without informing them. Read more here.

RESEARCH
Hidden in plain sight. Deepfakes are taking a lot of heat these days, so I’m highlighting an interesting positive
application of the technology: the anonymization of vulnerable people, such as whistleblowers, in photos and
videos to protect their identity—without losing their ability to express themselves.
The technique first extracts information about the anonymous subject’s facial expression by finding the positions of
their eyes, ears, shoulders, and nose. It then uses a deepfake algorithm, trained on a database of 1.5 million face
images, to create an entirely new face with the same expression. The new face is blended into the original photo,
retaining the same background.
The technique is still highly experimental. It often trips up when the face is partially occluded or turned at particular
angles; it’s also very glitchy for video. But it has the potential to replace traditional anonymization techniques, such
as blurring and pixelation, which run the risk of being incomplete (i.e., the person’s identity can be discovered
anyway) or completely stripping away the person’s personality (i.e., by removing facial expressions). Read
more here.
If you come across interesting research papers or AI conferences, send them my way to
algorithm@technologyreview.com.

A data leak has exposed the unchangeable
biometric data of over 1 million people
You can always change your password. Your
fingerprints and face are another story.
A data leak has exposed the unchangeable biometric data of over 1 million people
You can always change your password. Your fingerprints and face are another story.
A major breach discovered this week in the UK includes fingerprints of over 1 million
individuals, face recognition information, and other personal info from Suprema, a
security firm used by UK police, banks, and military contractors, according to a report
in the Guardian.
Biometrics: This incident is different to most data leaks because it strikes at the heart
of one of the big fears and criticisms about the increasing use of biometrics: You can
change your username and password with a couple of clicks. Your face is forever.
The source: Suprema runs Biostar 2, a biometric lock system controlling access and
surveillance in secured buildings. The leak was discovered by Israeli researchers Noam
Rotem and Ran Locar and the cybersecurity firm vpnMentor. Approximately one week
after discovery earlier this month, it was fixed (although Suprema did not
acknowledge it).

2 Apple contractors are listening in to Siri recordings 👂
And they hear drug deals, people having sex, and medical information. (The Guardian)
+ Amazon workers listen to people talking to Alexa, too. (TR)
+ Oh, and Google also does this. (TR)

A hacker stole the personal data of 100
million Capital One customers
.
A hacker stole the personal data of 100 million Capital One
customers
The alleged hacker behind one of the biggest ever cyberattacks on a bank boasted about what she'd done on Twitter and on
Slack, according to the FBI.
The news: A hacker stole the personal data of 106 million US and Canadian Capital One customers, the firm revealed last
night. A suspect, Paige A. Thompson, was arrested by the FBI yesterday, about two weeks after the company first identified the
breach. Thompson, who now faces up to five years in prison, allegedly breached Capital One’s data on a cloud service thanks
to a misconfigured web application firewall. Capital One is an Amazon Web Services customer and she’s a former AWS
employee.
Online trail: Thompson allegedly talked about the breach on a personal Slack account, and effectively admitted to hacking the
company in direct messages on Twitter. Prosecutors claim that she also posted data to a GitHub account that linked directly to
her real identity. A GitHub user who spotted the data alerted Capital One earlier this month which kickstarted the investigation.
The aftermath: Information dating from 2005 to 2019 was stolen including Social Security numbers, bank account numbers,
credit scores, names, and addresses. No credit card account numbers or login credentials were compromised.

Computers can’t recognize
emotions by only looking at your
face
Emotion recognition is a $20 billion industry, but a new study says the most popular
method is deeply flawed, reports Angela Chen.
How humans work out emotion: We use facial expression, but also body language,
where the person is, and much more. When computers do the same, they only focus
on the face, and that’s a major flaw. That’s why, according to an important new study,
most claims made by “emotion recognition” companies are wrong.
The industry: It’s big business. Huge companies like Microsoft and Apple, as well as
specialized startups like Kairos and Affectiva, are all taking part. It’s used for
recruitment, spotting fraud or terrorists, and advertising.
Oversimplified: There has been growing pushback against the belief that facial
expressions are easy giveaways to feelings. In fact, it’s very hard to use facial
expressions alone to accurately tell how someone is feeling, a group of scientists
concluded. Read the full story here.

What happens when a country’s entire
adult population is hacked?After a massive hack in Bulgaria, the prime minister called the attacker a “wizard,” but cybersecurity experts said the security was simply
inadequate.
The hack: A 20-year-old man was arrested in Sofia, Bulgaria, on Tuesday afternoon and charged with an unprecedented hack of the country’s tax
authority, ending with the theft of sensitive personal records from nearly every adult in Bulgaria, according to local reports. The suspect, whose
name is Kristiyan Boykov, according to Bulgarian media, faces up to eight years in prison. Police say others may have been involved.
The country’s officials have spent the week revealing and apologizing for the pillaging of Bulgaria’s National Revenue Agency (NRA) in June,
Reuters reported. Personal and financial data for millions of taxpayers was leaked by email to local journalists. The data leak includes names,
addresses, income and earnings information, and personal identification numbers, totaling 21 gigabytes and extending back over a decade.
In the email, the hacker described the Bulgarian government as corrupt. (Indeed, Bulgaria ranks as the most corrupt country in Europe, according
to Transparency International.)
The reaction: Prime Minister Boyko Borissov called Boykov a “wizard” and said the country should hire people like him. Security professionals in
Bulgaria are disputing the compliment and say the vulnerability never should have been exposed.
“It was alleged in the press that internal sources say the attack was an SQL injection,” said Bozhidar Bozhanov, an executive at the Bulgarian
security company LogSentinel. “SQL injections are easy to detect and somewhat easy to exploit. Protecting from SQLi should have been done on
many levels. First, in the software requirements. Second, during acceptance tests. And third, during operation by regularly scanning publicly
facing services for vulnerabilities. Apparently none of this has been done.”
The facts: There is a gap between the hacker’s claims and what the Bulgarian government says happened. The facts are still being determined.
The hacker claimed to have stolen data from over 5 million Bulgarians. The country’s entire population is around 7 million. Finance Minister
Vladislav Goranov said 3% of the NRA’s databases were impacted. Although the number is in the millions, it’s not clear how many individuals
Goranov believes are affected, but he said financial stability was not in danger.
Goranov apologized to Bulgarian citizens in front of the country’s parliament.
Vesselin Bontchev, a cybersecurity researcher and assistant professor at the Bulgarian Academy of Sciences, said the suspect left a mountain of
digital traces that led to his arrest.
“I can’t say the hacker was a ‘wizard,’” Bozhanov said. “If he indeed got caught so quickly, it means he was sloppy rather than a mastermind.”
The consequences: The scope of this attack is vast, and the number of unanswered questions remains significant.
The email the hacker sent to journalists with the leaked data came from a Russian email address. No one is quite sure what that means yet, but
given the tension between Russia and Europe, especially in cyberspace, it’s a detail that’s attracted immediate attention.
Closer to home, the Bulgarians are looking at their government and wondering what went so badly wrong.
“We have to note that NRA is one of the most technically advanced administrations in Bulgaria,” Bozhanov said. “This issue may or may not be
representative of the entire stack of technologies and services inside, but the fact that so much data was breached hints that few operational-
security best practices were followed.”
The big open questions include who was behind the attack, and whether it was an individual, a group, or even a nation-state. Criminals, activists,
and governments use hacked data in entirely different ways that can spell distinct forms of trouble for the Bulgarians affected by this breach.
One thing is clear: a reckoning has arrived for Bulgaria’s cybersecurity. Whether the government recognizes it or not, outside hackers certainly
will.

Facebook has promised to leave up a
deepfake video of Mark Zuckerberg
The firm's emerging policy on deepfakes seems to be to leave them up, but
flag they are fake.
The news: A deepfake video of Mark Zuckerberg was uploaded to
Instagram, which Facebook owns, earlier this week. In the clip, Zuckerberg
talks about being "one man, with total control of billions of people’s stolen
data, all their secrets, their lives, their futures." He adds: "I owe it all to
Spectre." You can watch it here.
Who made it? It was created by artists Bill Posters and Daniel Howe in
partnership with a UK-based advertising company called Canny, as part of
an exhibition at a documentary festival.
An emerging policy: Last month, Facebook decided not to remove a
video of Nancy Pelosi which had been doctored to make her seem drunk.
Instead, it said that if found to be fake, disclaimers would be added to
videos, and they’d be ranked further down people’s news feeds. This
seems to be Facebook’s new rule: leave manipulated videos up, but flag
they’re fake.
A growing problem: The speed and ease with which anyone can now
make and spread a deepfake has alarming implications, and policymakers
are increasingly wising up to the issue. Tomorrow the US will hold its first
ever hearing on deepfakes, and house representative Yvette Clarke will

Quantum computing to hack…ello, Weekend Readers!
arely a week goes by without some new mega-hack that’s exposed huge
mounts of personal data. The threat posed by such cyberattacks is forcing
overnments, militaries, and businesses to take a keen interest in the idea of
quantum internet.
o what is it? The quantum internet won’t replace the way most of us share
formation. Cat videos, witty memes, and your vacation pictures will live on in
e world wide web. But for more sensitive information like credit card details,
ealth records, and government data, quantum’s unique attributes could help
eep things far more secure.
ut with every new technology comes drawbacks. Can we address quantum
chnology’s shortcomings before it handles the world’s most sensitive data?
What is quantum computing?
Let’s start with the basics. What
is a quantum computer?
How it works, why it’s so powerful, and where
it’s likely to be most useful first.

Data

Private Data should not be owned and sold

Data from brain-Facebook is funding brain
experiments to create a device that reads your
mind.Big tech firms are trying to read people’s thoughts, and no one’s ready for the
consequences.
The news: In 2017, Facebook announced that it wanted to create a headband that
would let people type at a speed of 100 words per minute, just by thinking. Now, the
social-media giant has revealed that it has been financing extensive university
research on human volunteers.
How? It’s developing “speech decoders” able to determine what people are trying to
say by analyzing their brain signals. It’s important research as it can help show if a
wearable brain-control device is feasible.
Eventually: Facebook wants to create a wearable headset that lets users control music
or interact in virtual reality using their thoughts.
The ick factor: It’s an early example of a giant tech company being involved in getting
hold of data directly from people’s minds. To some neuro-ethicists, that means we are
going to need some rules, and fast, about how brain data is collected, stored, and
used. Read the full story here.

New
Tech:
Quantum
comput...

The fundamental nature of computing is changing.
AI, neuromorphic chips, edge computing, 5G, quantum computing, and the internet of
things are making it possible to solve problems in entirely new ways

Facebook wants to build a mind-reading wristband
Facebook is buying a brain-machine interface startup to further its ambitions in the
field
The news: Facebook has agreed to buy CTRL-labs, a New York-based startup that is
working on methods for people to communicate with computers using brain signals.
The deal is worth about $1 billion, according to CNBC.
The company: CTRL-labs only launched four years ago. There isn’t a ton of public
information about its technology and its research, but it claims it has made a
wristband which can turn brain signals into input for a computer. How well it works,
remains to be seen. Brain-computer interfaces of the sort used by people who are
paralyzed typically require a lot of training to work effectively and even then they are
sluggish and prone to errors. Elon Musk’s Neuralink is trying something similar, but its
interface is actually fitted inside the user’s brain.
What? It might seem a strange acquisition for a social media company, but in fact, it
makes a lot of sense. In July, Facebook revealed it has been financing extensive
university research on human volunteers.

Are brain implants the future of thinking?

Google researchers have reportedly achieved
“quantum supremacy”
The news: According to a report in the Financial Times, a team of researchers from Google led by John Martinis
have demonstrated quantum supremacy for the first time. This is the point at which a quantum computer is shown
to be capable of performing a task that’s beyond the reach of even the most powerful conventional
supercomputer. The claim appeared in a paper that was posted on a NASA website, but the publication was then
taken down. Google did not respond to a request for comment from MIT Technology Review.
Why NASA? Google struck an agreement last year to use supercomputers available to NASA as benchmarks for
its supremacy experiments. According to the Financial Times report, the paper said that Google’s quantum
processor was able to perform a calculation in three minutes and 20 seconds that would take today’s most
advanced supercomputer, known as Summit, around 10,000 years. In the paper, the researchers said that, to their
knowledge, the experiment “marks the first computation that can only be performed on a quantum processor.”
Quantum speed-up: Quantum machines are so powerful because they harness quantum bits, or qubits. Unlike
classical bits, which represent either a 1 or a 0, qubits can be in a kind of combination of both at the same time.
Thanks to other quantum phenomena, which are described in our explainer here, quantum computers can crunch
large amounts of data in parallel that conventional machines have to work through sequentially. Scientists have
been working for years to demonstrate that the machines can definitively outperform conventional ones.
How significant is this milestone? Very. In a discussion of quantum computing at MIT Technology Review’s
EmTech conference in Cambridge, Massachusetts, this week before news of Google’s paper came out, Will Oliver,
an MIT professor and quantum specialist, likened the computing milestone to the first flight of the Wright brothers
at Kitty Hawk in aviation. He said it would give added impetus to research in the field, which should help quantum
machines achieve their promise more quickly. Their immense processing power could ultimately help researchers
and companies discover new drugs and materials, create more efficient supply chains, and turbocharge AI.
But, but: It’s not clear what task Google’s quantum machine was working on, but it’s likely to be a very narrow
one. In an emailed comment to MIT Technology Review, Dario Gil of IBM, which is also working on quantum
computers, says an experiment that was probably designed around a very narrow quantum sampling problem
doesn’t mean the machines will rule the roost. “In fact quantum computers will never reign ‘supreme’ over classical
ones,” says Gil, “but will work in concert with them, since each have their specific strengths.” For many problems,
classical computers will remain the best tool to use.
And another but: Quantum computers are still a long way from being ready for mainstream use. The machines

Quantum Computing
The time is now to understand and plan for new capabilities in AI and quantum
computing.
We're curating a program that brings our coverage at MIT Technology Review to life and
highlights the executives, technologists, and innovators you need to know. Register for
EmTech MIT to assess emerging risks and industry impacts.

Other
Applications

Can Silicon Valley revolutionize mental health care?
Apps can democratize access, but they could also distract people who need serious
help from seeking it. (The Guardian)

AGI

Microsoft is investing $1 billion in OpenAI to
create brain-like machines
The AI lab gets to throw Microsoft’s supercomputing and cloud computing
muscle at its bid to build artificial general intelligence (AGI).
The news: Microsoft says OpenAI will help it jointly develop and train new AI
technologies for its Azure cloud computing service. They’ll also work
together on OpenAI’s mission to try to achieve AGI—machines with the
capacity to learn tasks the way human beings do.
Show me the money: OpenAI began as a nonprofit research lab in 2015 with
the mission of developing safe AGI. But AI models need mountains of data to
crunch, and that requires expensive computing power. So earlier this year,
OpenAI set up a new for-profit arm to help pay for its work.
Moonshot: Many AI researchers believe AGI is unachievable. OpenAI may try
to reach it by scaling existing deep-learning approaches rather than by
developing new ones. There’s no guarantee that will work, so Microsoft’s
billion-dollar bet is a risky one.
—Martin Giles

CI – AI team meeting – 23 May

Draft Agenda:
1. Moving forward on parallel tracks:
• Overall results framework – presentation to CI leadership
• Resource mobilisation
• Joint planning for C/4 and C/5
• AI - Information and Knowledge Sharing
2. Draft key messages (Prateek)
3. AOB (mutual updates,++)
4. Next steps

Contexte
- Vision of inclusive Knowledge Societies,
- Elevate debate –
- AI and other frontier technologies is currently and will be transforming every
aspect of our lives, and of UNESCO’s areas of competence;
- There are two aspects: How to harness technologies
- The digital revolution is harnessed for the SDGs, but raises questions on the future
of learning and work, risks to exacerbate inequalities, climate change, to reinforce
gender ++ biases, infringe on human rights (e.g. privacy) and
- Absolutely requires international, multistakeholder cooperation.
- Intersectorality
- – implementation, AI task team
- -ICTinED - S-Africa: Digital skills – computational thinking

- Vision of inclusive Knowledge Societies,
- AI and other frontier technologies is currently and will be transforming every aspect of
our lives, and of UNESCO’s areas of competence;
- There are two aspects: How to harness technologies
- The digital revolution is harnessed for the SDGs, but raises questions on the future of
learning and work, AI risks to exacerbate inequalities, can have a positive, but also
negative climate change, to reinforce gender ++ biases, infringe on human rights (e.g.
privacy) and
- Absolutely requires international, multistakeholder cooperation.
2) Access to the Internet
- Hardware, infrastructure – we are working with partners, in the BBCom
- Access requires soft component: Capacities,
- Intersectorality: meaningful, for results orientation
- -Cooperation with
- – implementation, AI task team, ethique
- -ICTinED - S-Africa: Digital skills, media and information literacy – computational thinking

IPAI
• support and guide responsible adoption of AI
• facilitate international collaboration in a
multistakeholder manner
• to monitor and draw on work being
done worldwide
• provide a mechanism for analysis, foresight, and policy
development in the area of AI.

UNESCO
work

The role of AI in the mandate of UNESCO
In 2018, UNESCO established an inter-sectoral Task Team working on artificial
intelligence, lead-facilitated by the CI Sector. The main objectives of UNESCO’s
initiative on AI are to: (1) foster a human-centred development and application of AI
that respects human rights and ethical principles (2) address the impact of AI on the
fields of competence of UNESCO; (2) promote the use of AI and its potential to
achieve the Sustainable Development Goals (SDGs).
UNESCO’s strategic positioning, compared to other international organizations
working on AI (e.g. OECD, African Union, European Commission, IEEE, Council of
Europe, etc.), lies within its universal reach, its ability to bring together the
perspectives of developing countries on an equal footing with those of developed
countries, and its multi-stakeholder approach.
UNESCO's action in the field of AI is fully aligned with the Organization's two global
priorities: Africa and gender equality. In particular, it aims to ensure that the African
continent is actively involved in the development of AI and can fully harness its
potential. UNESCO will work to develop digital AI-related skills for women and girls
and will seek to combat the reproduction of gender-related social biases in AI
applications.

UNESCO undertook with partners groundbreaking research on combatting gender bias
in applications using artificial intelligence. The publication titled ‘I’d blush if I could’
shares the first United Nations agency recommendations regarding the gendering of
AI technologies, imploring companies and governments to closer attention to how,
when and whether AI technologies are gendered and, crucially, who is gendering
them.
Serving UNESCO’s global Priority Africa, and as a follow-up to the UNESCO Forum on
AI in Africa held in Morocco, UNESCO organized capacity building workshops for the
technical community on Artificial Intelligence and Human Rights at the Deep Learning
Indaba, the annual gathering of the African Machine Learning Community at Kenyatta
University in Kenya, attended by 700 experts. (Capacity Building Workshop for Human
Centered AI in Africa )
AI is changing the landscape of work and learning and UNESCO investigated the next
Generation of Competencies required for the new digital era we are embarking on.
The intergovernmental Information for All Programme organized with partners the
15th edition of the World Conference on Intellectual Capital for Communities at
UNESCO Headquarters to discuss AI and the next generation of competencies for
employability of workers and closely linked, the future of learning.
UNESCO facilitated the consensus building of 67 Ministers, Deputy/Vice Ministers and
government officials and over 300 participants from 105 Member States, who

UNESCO’s work on shaping a human centered AI is strongly aligned with the UN
system-wide strategic approach for supporting capacity development on artificial
intelligence, the report of UN Secretary General’s High Level Panel on Digital
Cooperation and the Outcomes of UNESCO’s different AI events in HQs, Forum on
Artificial Intelligence in Africa.
Highlights include:
• Eleven awareness raising and capacity building activities have been initiated
around the world including in Africa since the last Executive Board.
• Four new AI publications in Education, Science, Communication and Information
and Gender have been published.
• Funds are being mobilized for the possible development of a standard setting
instrument on AI, capacity development and policy advice for AI.

Key messages - COM
• AI could open up tremendous opportunities for achieving the Sustainable
Development Goals (SDGs) set by the United Nations in the 2030 Agenda for
Sustainable Development. Its applications enable innovative solutions, improved risk
assessment, better planning and faster knowledge sharing.
• Many actors—businesses, research centres, science academies, United Nations
Member States, international organizations and civil society associations—are calling
for an ethical framework for AI development. While there is a growing understanding
of the issues, related initiatives need more robust coordination. This issue is global,
and reflection on it must take place at the global level so as to avoid a ‘pick-and-
choose’ approach to ethics. Furthermore, an inclusive, global approach, with the
participation of United Nations funds, agencies and programmes, is required if we are
to find ways of harnessing AI for sustainable development.
• UNESCO priorities must also guide our international action in this area. It is essential
to ensure that Africa fully participates in transformations related to AI, not only as a
beneficiary but also upstream, contributing directly to its development. In terms of
gender equality, we must fight against the biases in our societies to guarantee that
they are not reproduced in AI applications. Finally, we must empower young people by
providing them with the skills they need for life in the twenty-first century for
integration in a changing labour market.

United Nations Educational,
Scientific and Cultural Organization
Réunion d'information sur le rôle
de l'UNESCO dans le domaine
de l'intelligence artificielle
Moez Chakchouk
UNESCO Assistant Director
General
Communication & Information

UNESCO & IA
• Le domaine de compétence de l'UNESCO est centré sur l'être humain,
• L’intelligence artificielle (IA) a un potentiel énorme pour:
- Aborder les questions les plus cruciales pour l'humanité, notamment en
matière d’éducation, de sciences, de culture, de médias, d'accès à
l'information, d'égalité des sexes et de lutte contre la pauvreté.
- Promouvoir la réalisation des Objectifs de Développement Durable,
- Favoriser de nouvelles opportunités de développement tout en respectant
les valeurs humaines universelles et les principes éthiques.

Le travail de l'UNESCO en matière d'IA
• L’IA a un impact sur toutes les activités de l'UNESCO— tous les secteurs sont impliqués et
engagés.
• Coordination intersectorielle — Un groupe de travail mené par le Secteur de la
Communication & de l’Information (CI), a été créé en 2018, comme secrétariat.
• Lancement d'une étude par un groupe de travail multidisciplinaire composé d’experts, afin
d'exploiter l'importance des normes éthiques mondiales en matière d'IA.
• Advocacy and Public Information inclut l’édition spéciale du Courrier de l’UNESCO sur l’IA
(parut en juillet 2018) et le site Web intersectoriel sur l’IA (en.unesco.org/artificial-
intelligence).
• Organisation d’évènements sur l’IA afin de sensibiliser les Etats membres et de souligner
l'importance du travail à accomplir par l'UNESCO en collaboration avec d'autres parties
prenantes, notamment les organisations internationales.

Bhanu’s Key themes for the paradigm shift
 Access to information and knowledge
 Bridging the AI divide through policy development and capacity building
 The ethical implications of AI
 Opening up tools, processes and contents on AI
 Rights to freedom of expression and privacy
 Journalism and media development
 Internet governance in the AI world

Which paradigm shifts do we want to
introduce in AI?
• From Fragmentation to coherence to implementation
– Holistic and multidimensional approaches
– From policy formulation to policy IMPLEMENTATION
• From thinking ICT and AI are the solution, to knowing
when and how they can enhance
– From technology focus to needs of stakeholders
• From teaching digital skills to using AI to enhance (FoE,
learning)
– acquiring 21st century skills
• From relying on others on selfs
– From sales driven to national capacities to plan and evaluate
offers
• From imported solution to creating local, AI literate
environments
– From Push or Pull to Push and Pull
– From an abstract wish to be part of the new AI wave, to an
awareness, understanding, capacity and interest

introduce in AI?
• Awareness raising:
From AI awareness to a basic understanding of
AI opportunities and risks
• Capacity development: From random… to use
of guiding principles
• From a government developed AI
policy/strategy to a multstakeholder
involvement
• Member States are developing / adopting AI
policies in line with international frameworks
(ROAM)…

introduce in AI?
• AI improve access and knowledge
• From a simplified vision of AI, to an AI that follows ROAM
principles
• Key themes for the paradigm shift
•
• Access to information and knowledge
• Bridging the AI divide through policy development and capacity
building
• The ethical implications of AI
• Opening up tools, processes and contents on AI
• Rights to freedom of expression and privacy
• Journalism and media development
• Internet governance in the AI world
•

introduce in AI?
• Access to information and knowledge
• Bridging the AI divide through policy
development and capacity building
• The ethical implications of AI
• Opening up tools, processes and contents on
AI
• Rights to freedom of expression and privacy
• Journalism and media development
• Internet governance in the AI world

Paradigm shifts:
-From expert knowledge to policymaker awareness and a more in depth
understanding of policymakers of AI potential
-From digital divie
Address AI digital and knowledge divides:
• Processes
• tools
Upstream capacity building:
• Two chairs
• Category 2 Center
-Assess AI – readiness
-Capability approach:
• Commitment
• Analysis of policies, whitepapers
• Capacity (to use AI tools)
Which paradigm shifts?

RESEARCH:
-Mapping of standard setting work:
-IGO
-National policies
-AI – readiness:
-The state of public policy development
-multistakeholder
-Research on ongoing technological developments
-Openess source libraries
-Data localisation
-What is the
-Who owns the data
Joint observatory: -Map OECD’s and UNESCO indicators…

LONGER VERSION
PRESENTED TO ROK IN
MARCH 2019

PROPOSED ENTRY POINTS WITH ROK
ARTIFICIAL INTELLIGENCE
FOCUS AREA
Bridging the AI Digital and Knowledge Divides
FUNDING
NEEDS FOR
2020-2021
CURRENT
PARTNERS
1) Raising awareness/ building capacities of highest-level
decisionmakers on AI opportunties and risks in UNESCO’s
field of competence
a) Awareness raising about opportunties, risks, ways
forward for highest-level decision makers (Prime
ministers, 50 ministers)
b) Capacity building for policymakers, including
parliaments; developing plans for AI enabling
environments, using the AI policymaker toolkit
c) Facilitate international, regional, and national AI
cooperation for SDGs, with a focus on LDCs, Africa,
SIDS; advise on where to invest scarce resources; how
to share tools, facilitate collaboration
2,5 million
USD
- Member
States
- International
and Regional
organizations
-Private sector
-Technical
community
-Civil Society
and Academia

FOCUS AREA
FUNDING
NEEDS
FOR 2020-
2021
(USD)
CURRENT
PARTNERS
2) Resource portal (selected, trusted information)
- Mapping of existing principles and standards concerning AI
- Mapping of ecosystem of AI (and stakeholder groups)
- Development of the Indicator Suite, a tool for assessing
nationally the AI enabling environment in UNESCOs fields of
competence
- Mapping of national and regional AI strategies and their
integration into complementary (innovation++) policies
- Guidance to Open Ressources, tools, funding opportunties
- Pool of trusted experts
1,600,000
USD
- Member
States
- International
and Regional
organizations
-Private sector
-Technical
community
-Civil Society
and Academia

FOCUS AREA
FUNDING
NEEDS FOR
2020-2021
(USD)
CURRENT
PARTNERS
3) Policymaker Suite:
- Development of policymaker guidelines (UNESCO and
transversal themes, e.g. data governance);
- Decision maker essentials
- Constantly updated policymaker toolkit, including MOOC
4) Open consultations on the ethical dimensions of Artificial
Intelligence (online and regional)
(In case of EXB and GC decision, USD 1,3 million to develop a
standard setting instrument)
5) UNESCO Communication on AI (tools,++)
1,8 million
300,000
(1,3 million)
300,000
- Member
States
- International
and Regional
organizations
-Private sector
-Technical
community
-Civil Society
and Academia
-COMEST and
IBC

FOCUS AREA
FUNDING
NEEDS FOR
2020-2021
CURRENT
PARTNERS
6) Concrete AI projects, including:
-Digital AI skills for youth, particularly for girls & women in Africa
-Using AI to foster citizen science (e.g hydro-climatic monitoring);
use of big data, cloud computing and AI in geosciences,
-Ensuring inclusive and equitable use of AI in education for
vulnerable groups, leveraging AI to enhance education and
learning-Support the development of digital and AI skills with
focuses on women and girls in African countries
-Support the development of local AI experts and AI developers
through reinforcing the capacities of public TVET and HE
institutions of the developing countries in integrating AI and
coding training into the programmes
-Build the capacity of policy makers and education managers in
understanding the potential benefits and risks of AI for
education, and improve their capability of planning and
coordinating AI in education policies and programmes
4 million
- Member
States
- International
and Regional
organizations
-Private sector
-Technical
community
-Civil Society
and Academia
-COMEST and
IBC

SHORT VERSION
PRESENTED TO ROK
IN MARCH 2019

FOCUS AREA
FUNDING
NEEDS FOR
2020-2021
CURRENT
PARTNERS
1) Raising awareness/ building capacities of highest-
level decisionmakers on AI opportunties and risks in
UNESCO’s field of competence
2) Resource portal (selected, trusted information)
3) Policymaker Suite
2,5 MUSD
1,6 MUSD
1,8 MUSD
- Member
States
- International
and Regional
organizations
-Private sector
-Technical
community
-Civil Society
and Academia

FOCUS AREA
FUNDING
NEEDS FOR
2020-2021
CURRENT
PARTNERS
4) Open consultations on the ethical dimensions of
Artificial Intelligence (online and regional)
5) UNESCO Communication on AI (tools,++)
6) Concrete AI projects
1,3 MUSD
300 K USD
4 MUSD
- Member States
- International
and Regional
organizations
-Private sector
-Technical
community
-Civil Society and
Academia
-COMEST and IBC

EVENTS
1
3
0

Awareness raising and capacity development
events...
Across UNESCO Sectors events were organized for awareness raising and capacity
building for a human centered AI
Capacity Building Workshop for Human Centered AI in Africa
UNESCO organized two workshops on Artificial Intelligence and Fairness at the Deep
Learning Indaba 2019, the annual gathering of the African Machine Learning
Community at Kenyatta University in Kenya. UNESCO advocated for human Rights
centered, Open, Accessible and Multistakeholder driven development of artificial
intelligence as outlined in its study “Steering AI for Knowledge Society: A ROAM
Perspective”.
Agenda Building on AI for African Development at TICAD7
UNESCO and Japan co-organized a discussion on AI for Africa at the Seventh Tokyo
International Conference for African Development to discuss opportunities and
challenges of AI and UNESCO’s role in AI.
IFAP organized Conference on AI and Next Generation of Competencies
The 15th edition of the World Conference on Intellectual Capital for Communities,
organized at UNESCO Headquarters, highlighted the importance of human capacity
building as a critical element in efforts to ensure employability.
AI Examined from a ROAM Prism at IAMCR 2019 in Madrid
At the annual conference of the International Association for Media and

Evènements 2018 de l’UNESCO sur
l’IA
11-12 sept 2018
« L’intelligence
artificielle au service
des droits humains et
des objectifs de
développement
durable: Promouvoir
des approches multi-
acteurs, inclusives et
ouvertes »
14 nov 2018
Discussion sur
« L’intelligence
artificielle au service
du progrès des sociétés
de savoir et de la
bonne gouvernance ».
L’UNESCO à l’UM6P,
Benguérir, Maroc
« Forum sur
l’intelligence
artificielle en Afrique »
Siège de l’UNESCO,
Paris, France
13 déc 201812-13 déc 201815 nov 2018
Tables rondes:
« l’Intelligence artificielle:
réflexion sur sa
complexité et sur son
impact dans la société »
« L'édition du génome :
pourquoi l’éthique
s’impose »
Paris, France
« L’intelligence artificielle:
un nouvel environnement
de travail pour les
créateurs ? »
Paris, France
Forum sur la Gouvernance
de l’Internet (IGF)
L’UNESCO à la Fondation
Mozilla, Paris

Prochains évènements de l’UNESCO sur l’IA
22 janv 2019
Global Governance of AI
Roundtable 2019
10-12 fév 2019
Conférence
internationale
“Towards a Trusted
Artificial Intelligence
for Human
Progress”
Paris, France
Semaine de
l’apprentissage mobile
"Augmented Human
Intelligence and
Humanity"
Paris, France
17-19 avr 20194-8 mar 20194 mar 2019
"Tech Futures: Hope or Fear?”
debate on Ethics of New
Technologies and AI.
Réunion d'information sur
l'impact de l‘IA dans les
domaines de compétence de
l'UNESCO, en présence des
Etats membres et des ADGs
Paris, France
« Les innovations
digitales autour du
monde »
Forum NETEXPLO
Sommet mondial des
gouvernements en
partenariat avec l'UNESCO,
l'OCDE et al à Dubaï, EAU
Paris, France

IFAP Carribean Forum on AI
Indeed, the Caribbean Forum in Jamaica would be an excellent opportunity to scale-
up CI’s action in the field of Artificial Intelligence, reflect on related issues from the
perspective of the SIDS and to work together with our colleagues at HQ and at the
other UNESCO Offices in the sub-region.
Likewise, your possible participation in this event would be certainly one of the
highlights of the Conference and would strongly contribute to its success. In this
respect, your field visit to Jamaica would be a great honour for our Cluster Office to
learn about the latest developments on UNESCO’s programme priorities for the next
biennium. Equally, it would be a unique opportunity for us to have a direct dialogue
with you on our work in the Caribbean sub-region and to receive your good counsel.
We will liaise with our colleagues at headquarters about the possible dates and will
be pleased to submit you a joint proposal for your consideration.

UNESCO
Sectors

Coopération internationale
• Plusieurs initiatives ont déjà été menées et la coopération est nécessaire:
- Stratégie des Nations Unies sur les technologies pionnières au service du développement
durable, y compris le sommet mondial AI4Good de l'UIT à Genève,
- OCDE, « Des politiques meilleures pour une vie meilleure » — Groupe de travail d’experts
AIGO,
- Commission Européenne, « Vers un cadre éthique partagé pour l’IA » — Groupe
d’experts de haut niveau sur l’IA,
- Conseil de l’Europe, « Vers une IA éthique et responsable pour les droits de l'homme,
l'Etat de droit et la démocratie »,
- L’IEEE, « L’initiative mondiale pour les considérations éthiques en IA et des systèmes
autonomes ».

Éthique des sciences et des technologies :
• Éthique des sciences et des technologies :
- Plus de 20 ans de travail en bioéthique et éthique des sciences et des technologies.
- 2 organes consultatifs indépendants :
1. COMEST, la Commission mondiale d'éthique des connaissances scientifiques et
des technologies;
2. IBC, le Comité international de bioéthique. Le CIGB – Comité intergouvernemental de
bioéthique (36 Etats membres) examine le travail de l’IBC.
- Pouvoir de convocation de tous les États membres pour débattre d'un cadre
normatif international de coopération.
- Une philosophie mondiale, pluraliste, pluridisciplinaire et multipartite.

Universalité et gouvernance de l’Internet:
- Approche multipartite des normes et règles de gouvernance : société civile,
universitaires, secteurs public et privé.
- Fossé numérique et absence de politiques en matière d’open data = rareté des
données alimentant les avantages potentiels de l’IA.
- Aborder l’universalité de l’Internet par le biais des principes D.O.A.M. : les Droits
humains, l'Ouverture, l'Accessibilité, et la participation de Multiples acteurs.
- Mesurer les Indicateurs de l’Universalité de l’Internet (IUI) afin d’exploiter l’état de
préparation des Etats membres et d’encourager les décideurs politiques des PMA à
adopter des politiques adéquates en matière de développement et d’utilisation de
l’IA.

Education et renforcement des capacités :
- Garantir une utilisation inclusive et équitable de l'IA dans l'éducation des groupes vulnérables, tout en
promouvant l'égalité des sexes dans l'IA;
- Tirer parti de l'IA pour améliorer l'éducation et l'apprentissage, notamment en promouvant l’intelligence
collective liant l’humain à la machine, en exploitant un système d’apprentissage intelligent pour
l’apprentissage et l’évaluation, et donner aux enseignants les moyens d’enseigner l’IA efficacement.
- Promouvoir le développement des compétences en IA pour améliorer sa vie professionnelle et personnelle,
en mettant l’accent sur les innovations des jeunes dans le développement d’applications de l’IA pour
atteindre les O.D.D.
- Utilisation transparente et vérifiable des données relatives à l’éducation par le biais de partenariats
multipartites, en préservant la confidentialité des données et la vie privée des élèves.
- Publication de rapports (sur l’éthique sur la robotique, le big data et la santé) soutenant les décideurs
politiques et les différentes parties prenantes, en fournissant des conseils sur les questions éthiques.

Les sciences, la technologie et
l’innovation :
• Politique, capacité, application
• S’attaquer au fossé scientifique en promouvant la science ouverte et l’innovation ouverte
(open data, ressources scientifiques ouvertes comme l’informatique en nuage etc.) et
l’accès universel au savoir scientifique.
• Suivi des tendances politiques liées à l’IA, telles que les stratégies nationales d’IA, dans le
cadre de la fonction de suivi mondial des O.D.D. 9.5, centrées sur l’inclusion et la diversité.
• La formation et la recherche en IA dans les pays en développement doivent progresser
parallèlement aux progrès technologiques.
• L’IA doit soutenir les Etats membres dans leur gestion de l’eau, à la réduction des risques de
catastrophe, au changement climatique, à la surveillance de la biodiversité, etc.

Patrimoine culturel & créativité :
- Promouvoir la diversité des expressions culturelles dans un environnement
d’IA,
- Favoriser la créativité numérique via l’usage des technologies de l’IA,
- Contribuer à la préservation du patrimoine culturel et faire face aux
menaces par le biais de la numérisation et de l'utilisation des systèmes d'IA,
- Lutter contre le trafic illicite d'objets culturels et contre la destruction de
biens culturels.

Médias & accès à l’information
- Des médias indépendants et pluralistes afin de diffuser les bonnes informations et
de sensibiliser les citoyens aux questions liées à l'IA,
- L'éducation aux médias et à l'information (MIL) pour construire des sociétés du
savoir inclusives et faire face aux menaces qui pèsent sur les processus
démocratiques et le développement pacifique (discours haineux, désinformation,
discrimination, etc.),
- Promouvoir l'accès à l'information publique (données publiques ouvertes, REO),
- Protection des droits fondamentaux : liberté d'expression, vie privée, accès à
l'information, etc.

- Forum des femmes « Daring Circle » sur l’IA et les sexes (14 novembre 2018);
- Document sur les implications des assistants numériques pour l’égalité des sexes
(à paraître en mars 2019);
- Note d’orientation sur la promotion des compétences numériques pour les filles
et les femmes (à parution le 8 mars 2019).

Priorité Afrique et intelligence artificielle
- Déclaration finale du Forum sur l’IA en Afrique, Benguérir (Maroc) le 13
décembre 2018, comprenant la promotion d’une IA centrée sur l’être humain,
fondée sur les droits de l’homme et les valeurs éthiques et appelant à une
collaboration entre l’UNESCO, l’Union Africaine et les communautés
économiques régionales pour:
o Soutenir la création d’un forum africain d’association et d’institutions sur l’IA,
réunissant les principaux acteurs publics et privés de l’IA;
o Elaborer une stratégie continentale pour l’IA, étayée par l’Agenda 2063;
o Organiser des forums sous-régionaux sur l’IA.
- Prochain forum sous-régional sur l’AI pour l’Afrique de l’Ouest et Centrale en
2019.

- L’initiative de l’UNESCO avec Microsoft et d’autres partenaires à l’élaboration d’un indice de préparation à
l’IA dans l’éducation et à la formation des décideurs politiques sur l’IA dans l’éducation.
- Initiative avec Ericsson et d’autres partenaires à l’élaboration d’un référentiel de programmes de formation
à l’IA et soutien des institutions ETFP à développer ou réformer des programmes de formation sur l’IA.
- Du 18 au 22 février 2019, le nouveau Centre des technologies quantiques (Centre for Quantum
Technologies) du CIPT accueille sa première école avancée sur la physique quantique omniprésente
(Advanced School on Ubiquitous Quantum Physics) : la nouvelle révolution quantique, destinée aux
scientifiques des pays en développement.
- Le programme de l’UNESCO et de l’Université de Kyoto sur l’Homme et la Biosphère soutient la Réserve
naturelle intégrale du Mont Nimba en Guinée (inscrite au patrimoine mondial) à améliorer la préservation
des chimpanzés grâce à l’usage des drones pour combattre les braconniers.

- L’IA pour les sociétés du savoir du point de vue des D.O.A.M.. Une publication sera prête lors du
lancement de la Conférence internationale sur l’IA en mars.
- Traduire les principes D.O.A.M. en normes et politiques sur l’IA, recherche de base « Vers
l’universalité de l’Internet et des sociétés du savoir à l’ère du numérique: opportunités et défis de
l’IA ». La recherche sera présentée lors de la conférence du mois de mars sur l’IA.
- La faisabilité d’un centre de catégorie 2 placé sous l’égide de l’UNESCO à l’Institut Josef Stefan
(Slovénie) a été établie afin de soutenir les objectifs de l’UNESCO et d’utiliser l’IA dans le cadre des
O.D.D..
- La Chaire de l’UNESCO pour l’IA et la fondation Knowledge4All ont préparé une carte de
l’écosystème pour le développement de l’IA. Les résultats ont été présentés lors de l’IGF et lors de
la conférence sur l’IA au Maroc.

- Première version des principes directeurs en matière d’éducation, lesquels seront lancés lors de la Semaine
de l’apprentissage mobile.
- Le Programme hydrologique international de l’Université de Californie ont développé l’application GWADI
GeoServer en 2005 pour estimer les précipitations en temps réel dans le monde entier, en utilisant des
réseaux neuronaux artificiels. Il est utilisé dans le monde entier pour la planification et la gestion des
situations d'urgence, par exemple pour suivre les tempêtes et préparer des bulletins quotidiens contenant
des informations actualisées sur les conditions d'inondation et de sécheresse pour les communautés
locales.
- Un projet conjoint avec Sciences Po Paris développe une méthodologie afin de cartographier l’engagement
de l’IA, la capacité et la création de l’IA. Le projet a initié un processus de collecte de données pour le
continent africain.
- Un atelier de renforcement des capacités sur les questions techniques, les mythes entourant l'IA et l'impact
de l'IA sur les O.D.D. sera conjointement organisé par le Secteur CI et HRM. L'atelier s'inscrira dans le cadre
de la Semaine de l'apprentissage mobile et sera animé par la Chaire de l’UNESCO pour l’IA et la Chaire
UNESCO « Analytics and Big Data ».

United Nations Educational,
Scientific and Cultural
Organization
Merci de votre attention
Moez Chakchouk
UNESCO Assistant Director General
Communication & Information

Education
New skills
jobs

EDUCATION
Are You Developing Skills That Won’t Be Automated? by Stephen M. Kosslyn
The jobs of the future will depend on emotion and context.

11th & 12th July 2019 The World Conference on Intellectual Capital for Communities
- 15th Edition -
How will AI impact jobs?
151
The warehouse example
Humans are not replaced
→ Their activity is modified
Who receives the benefit ?
Sources: McKingsley Global Institute, E. Reynolds
MIT
The robot nurse
Lack of medical staff
Humans are not directly
replaced
→ job was not occupied
TARDIS EU project
Training young NEET
No recruitement
Human-AI collaboration

- 15th Edition -
How will AI impact jobs?
Two situations can be observed:
1. Robots that replace human workers
• Does what a human being could do
• Not necessarily cheaper
• Pre-existing cost reduction policies
2. Creation of new jobs, transformation
• Market extension
• New skills and competencies
152

- 15th Edition -
Conclusions
• AI still needs us
• Domain experts
• Programmers
• Data collection
• AI will not replace us, but…
• Companies invest in AI products to increase their profit
• People must decide what to do with AI
• Transformations will occur
• New jobs, new skills
• No massive disappearance of jobs due to AI only
153

UNESCO Event / Name / other Date154
1 How has skills demand changed historically?
2 What skill shifts can we anticipate in the next 15 years?
3 How can companies and policy makers address
the skill mismatch?

There have been large-scale
upskilling episodes in the past
century
155
SOURCE: Claudia Goldin, “America’s graduation from high school: The evolution and spread of secondary schooling in the twentieth century,” Journal of Economic History, volume 58, number 2, June 1998; National Center for Education Statistics;
McKinsey Global Institute analysis
High School Movement (1910–40) GI Bill (1944–present)
80
100
0
20
6
2
4
0
8
40
10
60
301900 40 197010 20 50 60
Million
college
students
College enrollment
High school enrollment
% of
enrolled 14-
to 17-year-
olds

All jobs now increasingly
require digital skills
156
NOTE: Figures may not sum due to rounding; 1 Brookings defines a digital score based on 2 digital components of O*NET (overall knowledge of computers and electronics required by a job and the centrality of computers to the overall work activity of the occupation) as
an index from 0 to 100. High score is above 60, medium score is between 33 and 60 and low score is lower than 33
SOURCE: Brookings analysis of Occupational Information Network (O*NET) and Occupational Employment Statistics (OES) data
Employment by levels of job digitization, 2002–2016
56%
30%
40%
48%
23%
5%
2002 2016
Medium1High1 Low1
Example
occupations
Nurses
Lawyers
Auto mechanics
Construction
Cooks
Software developers
Financial managers
Assembly line workers

Looking ahead, by 2030 AI will
alter the nature of most jobs
157
SOURCE: McKinsey Global Institute analysis
of FTE time is at risk
Only ~15%
of FTE time needs
significant retraining
About ~45%

UNESCO Event / Name / other Date158
Higher cognitive, socio-
emotional and tech skills
will be in high demand

Our unique bottom-up
methodology is based on 3
input factors
159
Physical and manual
Basic cognitive
Higher cognitive
Social and emotional
Technological
We mapped workforce activities
to skills
We defined 5 workforce skill
categories
We collected qualitative inputs to
corroborate our results
>3,000 business leaders
From 7 countries
Across 14 sectors
~800 occupations
Across all sectors
Skills assessment
25 skills in total
~2,000 activities
10 activities for a retail sales
person

AI will require more
technological, social and
emotional skills…
160
All sectors
+67% in basic digital skills
+91% in advanced IT skills and programming
United States and Western Europe
-14
-15
8
24
55
Change in number of hours
2016-30, %
-26% in general equipment operation and navigation skills
-14% in craft and technician skills
-7% in basic numeracy, literacy and communication skills
-21% in basic data input and processing skills
+18% in complex information processing and interpretation skills
+34% in creativity skills
+29% in leadership and managing other skills
+33% in entrepreneurship and initiative taking skills
Total hours, billion
Technological skills
Higher cognitive skills
Social and
emotional skills
Physical and manual skills
Basic cognitive skills
650
31
18
22
18
11
Hours in 2016
% of time
Large portion time spent today in physical
and basic cognitive skills…
…but in the future, time spent on these skills is going to decrease while
skills such as tech, social and emotional will grow
NOTE: Western Europe: Austria, Belgium, Denmark, Finland, France, Germany, Greece, Italy, Netherlands, Norway, Spain, Sweden, Switzerland, United Kingdom. Figures may not sum due to rounding

The impact varies by sector
161
-40% less 80% more
Number of hours worked
United States and Western Europe, Percent change in time spent using skill
Retai
lSkill category
Technological skills
Higher cognitive skills
Social and emotional skills
Physical and manual skills
Basic cognitive skills
Manufacturin
g
Banking and
insurance
Energy and
mining
Healthcar
e

Cédric Wachholz, Chief of ICTs in Education, Science and Culture Section,
UNESCO
Challenges and Opportunities of
Artificial Intelligence for SDG 4

Content
2 Improving Learning outcomes
3 Preparing for an AI driven world
4 Six Challenges in Education and AI
5 UNESCO work
1 SDG 4 – key statistics
Conclusion6

Sustainable Development Goal 4
Ensure inclusive and equitable quality education
and promote lifelong learning opportunities for all

Why Education matters?
Education - a Human Right!
Helps achieve other SDGs
Breaks Cycle of Poverty, reduces Inequalities
Empowers people to live healthy and sustainable lives
Fosters Tolerant and Peaceful Societies

Key Statistics
Access and Quality

I4ADA 2019 - Presentation Cedric Wachholz

I4ADA 2019 - Presentation Cedric Wachholz

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (19)

Ähnlich wie I4ADA 2019 - Presentation Cedric Wachholz

Ähnlich wie I4ADA 2019 - Presentation Cedric Wachholz (20)

Mehr von Paul van Heel

Mehr von Paul van Heel (8)

Kürzlich hochgeladen

Kürzlich hochgeladen (17)

I4ADA 2019 - Presentation Cedric Wachholz