Digital ethics and ensuring fair and unbiased AI systems are important priorities for VDAB. They have developed principles of trust, transparency and benefit and are working to operationalize them. This includes qualitative and quantitative assessments of AI systems to identify any biases and ensure fair treatment of all users. VDAB aims to be a leader in the ethical development and use of AI to best serve citizens and employers.

1. 1
Digital ethics in practice
AI 4 GOOD

2. “ TECHNOLOGY IS NEITHER GOOD NOR BAD; NOR IS
IT NEUTRAL.”
FIRST LAW OF TECHNOLOGY, MELVIN KRANZBERG (1917-1995), PROFESSOR, AND
CO-FOUNDER, SOCIETY FOR THE HISTORY OF TECHNOLOGY
https://www.youtube.com/watch?reload=9
&v=rPKrdxiEkQ0
73% van de werkzoekenden in Zweden denkt dat ze tijdens het interview worden gediscrimineerd.
Door de mens te vervangen door AI-robotkop Tengai, denken de ontwikkelaars dat ze het
screeningproces eerlijker en eerlijker kunnen maken.

3. Human digital
Proactieve en reactieve ondersteuning
Face to face
Wanneer nodig en met een meerwaarde
Digital first
24/7/365 online dienstverlening
3

4. WAAROM DOEN WE DIT ?
Aanreiken van informatie en tools waarmee werkzoekende én
werkgever zelf aan de slag kan
Interactie met de klant op zijn maat en verwachtingspatroon
Consulenten en instructeurs zijn geen dossierbeheerders. Het
zijn vakmensen, met een hart voor hun metier.
De eerste inschatting van de klant gebeurt met behulp van
data
Proactief detecteren van obstakels in arbeidsmarktintegratie
= slim gebruik van onze
rijkdom aan data
Stijging aantal online inschrijvingen
Stijging online zoekgedrag (12=>26%)
88% tevredenheid mbt eerste inschatting
60% tevredenheid over de jobsuggesties
Snellere doorstroom bij nood aan persoonlijke
dienstverlening (+20%)
Op 6 maand na inschrijving is 68,3% terug
aan het werk

5. OPINIE
SIHAME EL KAOUAKIBI
VROUW? COMPUTER SAYS NO

6. ● ✦Opinie
● ‘De VDAB is een voortrekker van
het ethisch gebruik van artificiële intelligentie’

7. Digital Ethics en Privacy : a “must have”
DPO team lead
Privacy
AVG
Ethische AI

8. AI 4 GOOD
WAAROM DOEN WE DIT ?
> Verantwoordelijkheid tegenover de Vlaamse overheid
en de burgers
> Evolutie naar een samenleving ‘Human + Machine’
> Gebruik van persoonlijke data en AI voor
geautomatiseerde besluitvorming
> Bias in de data en algoritmes kunnen leiden tot
ongewenste impact op burgers

9. Europese en lokale overheden definiëren richtlijnen voor het
ethisch ontwikkelen en gebruik van AI systemen
> Europa: ‘Ethics guidelines for trustworthy AI’ en requirements waaraan AI systemen
zouden moeten voldoen
> België: coalitie AI 4 Belgium werkt een AI plan uit voor o.a. het ethisch gebruik van
data
> Frankrijk: ethische principes om AI in dienst te stellen van de mensen
> Duitsland: nationale AI strategie die stelt dat AI op een ethische manier geintegreerd
moet worden in de maatschappij
> Verenigd Koninkrijk: richtlijnen voor het ontwikkelen en gebruik van AI systemen in
de publieke sector
AI 4 GOOD
WAAROM DOEN WE DIT ?
VDAB wil een voorbeeld zijn

10. Trust Transparency Benefit
AI 4 GOOD
AI systems which have impact on people must be devellopped and continuously monitored following our three
principles

11. FROM THEORY TO PRACTICE
MAKE THE PRINCIPLES REAL
Principes Begrijpen Meten
Actie &
preventie
• Trust
• Transparency
• Benefit
• Interpretatie VDAB
& begrijpbaar voor
de burger
• Bruikbaar voor
interne en externe
communicatie
• Kwalitatief meten
tijdens het
ontwikkelings-
process
• Kwantitief meten
van ‘fairness’ in
data & model
• Korte termijn acties
voor “Kans op
Werk”
• Lange termijn
aanbevelingen voor
nieuwe AI use cases
• Workshops & co-creatie met het
Core team met medewerkers uit:
• Arbeidsmarktbemiddeling
• Ondersteunende diensten
• I&T
• Evalueren van Trust bij AI use
cases, data & modellen door
bevraging van medewerkers
• Evaluatie van gebruikte data &
model “Kans op Werk”
• Interpretatie van de metingen
• KT acties voor “Kans op Werk”
• LT aanbevelingen voor nieuwe AI
use cases
WAT?
HOE?

13. CONTRETE OUTCOMES
USE CASE : KANS OP WERK
Meten
“TRUST” one-pager & detail
document gevalideerd door
het Core team
Principes BegrijpenPrincipes Begrijpen
Kwalitatieve evaluatie van
“Kans op Werk”, AI process en
bias d.m.v. interviews en
vragenlijsten
Meten
Kwantitatieve analyse “Kans of
Werk” data en model
MetenMeten

14. Principles & playbook Discover with impact
Output from the ethical AI practice
Interpreted principles – this can be used for
oversight & communication
Assessment results & results template
MVP v2 to assess data & model in qualitative and quantitative way
One pager:
Transparency
One pager: Benefit AI 4 Good summary
Full playbook
+ Communication pilot plan,
AI communication template,
channel & frequency
described
Full playbook, describing the responsible AI
organization

15. AI 4 GOOD JOURNEY

16. > Creation of ambassadors
> Evangelize on critical thinking from the idea and aks
the right questions when developping AI systems
> Internal communication about the ethical dev and use
of AI
> Digital ethicist role and governance board
Initiatives
AI 4 GOOD AMBITION
> Continuous monitoring of bias in data and models
> Ethical Governance board with Veto right and
empowers Ethics by design
> External commnucation about ethical development and
the use of AI
Ambition

17. Putting AI 4 Good into practice
Exploit
Experiment
Explore
Experimental
bias
Design
issues
Execution
issues
Data bias
Model bias
Communication
failures
Operational
bias
Conception
flaws
Use case is known
(from business) vs.
use case is not
known

18. Explore
Exploit
Experiment
Define expected outcome, potential
value and impact of AI use case
(ensure this is aligned with VDAB
mission & vision)
1
Define the sensitive variables
used in the experiment / AI use
case and verify Governance is
needed.
2
Perform qualitative assessment
(identify conception flaws or
experimental bias)
3
Select fairness
metrics & define
fairness for the
use case
Create & share
internal
communication
material
Share lessons learned,
and incorporate changes
in Playbook, ensure
collaboration & diversity
across teams
1
Perform qualitative &
quantitative assessments
(These can run in parallel
& at different stages of
the project)
2
Receive & review
assessment results
& provide feedback
34
5
1
Create & share
external
communication
material
Share lessons learned,
and incorporate
changes in Playbook,
ensure collaboration&
diversity across teams
Monitor using the quantitative
assessment & perform qualitative
assessment checks (execution issues,
communication failures & operational
bias)
Receive & review
assessment results
& provide
feedback
2 3
4
The AI 4 Good organization - overall actions
Based on the initial playbook, we have created a light version of the key steps for execution during the AI lifecycle
= Integrate feedback improvements when required

19. Ethics oversight defined roles
Role Brief description
Executive Committee Is responsible to underwrite the advice of the Digital Ethicist which has been overruled by the Director Innovation & Architecture
Director Innovation &
Architecture
Has oversight of all AI initiatives in VDAB, knowledgeable of their status and ethics assessment results. Has the authority to overrule
advice from the Digital Ethicist.
Digital Ethicist / Informational
Ethicist
An ethics expert that provides ethical guidance to employees or teams defining AI use cases and developing or maintaining AI
systems. Is responsible to evaluate the ethics assessment results at each gate of the AI delivery lifecycle and provides an advice to
proceed or to address ethical issues.
GDPR expert Ensures that AI system development and data used in that process is compliant with the GDPR regulation.
Legal Expert Well-versed in the legal dimensions relevant to the types of applications, technologies, policies, and projects that fall within the
committee’s purview. Legal experts are crucial to ensuring legal compliance, as well as for identifying areas where current legal
guidance is absent, inadequate or ambiguous.
Data Architect / Data Expert Ensures that the data that is being collected can be collected as per the ethical guardrails and is collected in a secure way. Aligns
with the Digital Ethicist on project specific data needs and purpose to ensure this can be collected in an ethical and secure manner.
Analytics & AI team lead Has oversight of the AI delivery lifecycle and manages the AI portfolio. Responsible to
Project Manager Responsible to plan the project activities, allocate resources, ensure the methodology is applied, organize project team meetings,
report project progress to sponsor, ensure the necessary deliverables are in place to proceed through the AI delivery lifecycle.
Specifically for Ethical AI, the Project Manager needs to ensure that the Qualitative and Quantitative Algorithmic Assessment results
(deliverables) are produced in the various steps of the project.
Data Scientist Works within the Analytics and AI team and part of a project team developing AI systems. Is responsible to execute the Machine
Learning lifecycle: data preparation model creation, model evaluation, model refinement and model deployment. As part of that
lifecycle, the data scientist is responsible to run the Algorithmic Assessment steps on the input data, model output and mode itself.
The Data Scientist needs to discuss and review the results with a Business Expert. He is also responsible to peer review the
assessment results of colleague data scientists working on in a different project team.

20. PURPOSE OF JOBNET
To make it easier for CITIZENS (and EMPLOYERS) to find suitable match of JOBS (or
CANDIDATES), through relevant recommendations reflecting the current market
situation driven by jobseekers’ and employers’ interests.

21. SOURCES OF UNFAIRNESS IN AI SYSTEMS
What we will not assessWhat we will
assess
Training Data
The dataset for training the
algorithm may have
unintentional bias due to
unbalanced sample
representativeness.
Engineered Features
Engineered features
(embeddings) that are used to
convert raw data into algorithm
compatible data may be biased.
The modelling process may
lead to incorporation of data
bias which will then be
propagated to future
predictions.
Model Model Outcome
The outcomes generated by
the model might amplify the
bias due to training data or
model.
A B C D

22. Making a success of the AI 4 Good organization, we
need to ensure we have the right teams
CommunicationProject team* Analytics & AI team
Oversight
get data protection
& ethics right
Collaboration
build the right reflexes
Measuring
data driven ethics
Communication
promote a culture to
speak up
Data Protection
& Ethics team

23. You need a sustainable operating model
Ethical board structure
DPO – digital ethicist
Privacy & ethics
Director labor market
management
Business & Operations
Director Innovation
Innovation & analytics
University ethics & AI expert
External, research (NEW ideas)
Director Strategy & Policies
HR / Values & Strategy
CEO / or Executive Committee
?

24. • Transparency with external parties ( blackbox effect)
• Enable and keep innovation capacity
• Governance model => keep it light
• Data Governance is key
• Open data
• Guarantee neutrality
• Explain vs justify vs responsibilise
• Ethics by design integrated with Privacy and Security by
design
• Communication internal en external ( reputation
impact)

25. VINCENT BUEKENHOUT
DIGITAL ETHICIST & DATA PROTECTION OFFICER
VINCENT.BUEKENHOUT@VDAB.BE
0497 39 43 88
Tapping the ethical
forces of change