EU projects and The Learning Health System

1. EU Projects and The Learning Health
System.
Brendan Delaney
Chair in Medical Informatics and Decision Making
Director Centre for Patient Safety and Service Quality
Imperial College London

2. What is a ‘Learning Health System’?
‘A learning healthcare system is [one that] is designed
to generate and apply the best evidence for the
collaborative healthcare choices of each patient and
provider; to drive the process of discovery as a natural
outgrowth of patient care; and to ensure innovation,
quality, safety, and value in health care.’
Institute of Medicine. 2007

3. www.transformproject.eu
€7.5M European Commission
March 2010-Nov 2015
Funded under the Patient Safety Work Program of FP7
• To support clinical diagnosis
• To support clinical trials
• As part of a ‘Rapid Learning Healthcare System’
3

4. 21 Partners from 10 Member States

5. Aims of TRANSFoRm
To develop methods, models, services, validated
architectures and demonstrations to support THE
LEARNING HEALTH SYSTEM:
•Epidemiological research using primary care records,
including genotype-phenotype studies and other record
linkages
•Research workflow embedded in the EHR
•Decision support for diagnosis

6. TRANSFoRm clinical requirements (2010-15)
Part of a Learning Health System
Prevalent and incident case identification from live EHR
systems in primary care
Real time alerting via EHR
Pre-population of CRFs displayed “within” the EHR
Data capture in EHR (eSource)
PROMS data in EHR for Safety monitoring
Data provenance – towards compliance with 21 CFR
Part 11 and European regulation
Full evaluation in 5 EU member states and real world
RCT

7. TRANSFoRm technical requirements
To use ontologies to maintain models of meaning for
the LHS (CRIM, CDIM, Provenance)
To use CDISC foundational standards ODM, SDM
To enable connection to multiple country, multiple
language, multiple vendor systems with MINIMAL
vendor input – No ‘hackathons’
Vendor requirements:
• Standard Terminology used in EHR (LexEVS)
• Sample EHR data set
• Represent local database metadata as a model (DSM) and
map to TRANSFoRm Clinical Data Integration Model
(ontology)
• API and a demo installation for testing

8. Limitations of existing CDISC standards for clinical trials
Operational Data Model operates as a holder and can
contain a wide variety of content.
Therefore clinical meaning and context can vary between
clinical and research domains in unpredictable ways.
CDASH offers a means of assembling libraries of well-
defined forms, but does not implicitly translate to a
clinical setting
8

9. Approach to Meaning
TRANSFoRm uses common data models expressed as
ONTOLOGIES to maintain meaning across the
Learning Health System
Archetypes are used to link ‘meaning’ as a reference to
the ontologies (Research and clinical meaning) at
individual data element level
9

10. A software platform for the Learning Health
System is built using tools to author,
exchange and act upon these basic
information constructs. It serves all three use
cases.
10

11. Four points of eCRF/EHR integration
1. Extracting study data from the EHR to facilitate data
entry
2. Asking physician to complete the study data that are
not present in the eHR
3. Presenting all study data to the physician for approval
before sending to the research databasein order to
increase control and data quality
4. Adding care-relevant study data to the EHR, so that the
physician needs to fill out the information only once

12. High level system architecture

13. ODM in TRANSFoRm

14. SystmOne pop up

15. SystmOne Eligibility

16. SystmOne Agreement

17. User Interface Extension to ODM
ODM extended with GUI elements
When an ODM is created, the QuestionType attribute is added to every
ItemDef object in ODM
Platform-agnostic, rendered appropriately for each device

18. 18

19. A Learning Health System for Diagnosis

20. The power of early hypotheses
Hypotheses formulated quickly and with little information.1
Disproportionate influence of early hypotheses on subsequent
judgements and decisions.2-4
Strong association between physicians’ initial diagnostic
impressions and their final diagnoses (and management) in
common presentations that could indicate cancer.5
Baron (2000);1
Asch (1946);2
Crano (1977);3
Forgas
(2011);4
Kostopoulou, Sirota et al MDM 2016 5

21. The principle of early diagnostic support
• Suggest possible diagnoses early in the consultation,
before physicians start gathering information to test their own
hypotheses.
• Diagnostic suggestions based on patient’s age, sex, risk
factors and reason for encounter (RfE).
• Principle tested in 2 RCTs: UK1
and Greece2
(D2.1)
• Online, interactive, simulated consultations with 9 clinical
scenarios

22. Trigger DSS in Vision EHR

23. Early support

24. Coded data entry

25. Information transferred to Vision EHR

26. DSS prototype evaluation in a high-fidelity simulation
• 34 GPs – users of Vision EHR system
• 12 standardised patients (actors)
• 2 sessions (within-participant design)
» 1st
session: (baseline performance) Vision EHR – no DSS.
» 2nd
session: Vision EHR with integrated DSS.

28. Results
MEASURE
Baseline
Mean [95% CIs]
DSS
Mean [95% CIs]
Odds ratios or
stand. coefficients
[95% CIs]
Diagnostic
accuracy
0.50
[0.42 to 0.57]
0.58
[0.52 to 0.65]
OR 1.41
[1.13 to 1.77],
P<0.01
Appropriate
management
0.59
[0.52 to 0.66]
0.66
[0.52 to 0.65]
OR 1.34
[1.01 to 1.78],
P<0.05
Diagnostic
certainty (0-10
VAS)
7.61
[7.28 to 7.94]
8.01
[7.79 to 8.23]
Beta 0.39
[0.12 to 0.67],
P<0.01
Data items coded
into the EHR
1.64
[1.10 to 2.18]
12.35
[10.82 to 13.87]
Beta 10.71
[9.06 to 12.35],
P<0.01
95% CIs are adjusted for clustering on GP

29. Results
MEASURE
Baseline
Mean [95% CIs]
DSS
Mean [95% CIs]
Standardised
coefficients
[95% CIs]
Investigations
ordered
2.51
[1.92 to 3.09]
2.83
[2.33 to 3.33]
Beta 0.33
[-0.54 to 1.20]
Consultation length
(mins)
13.73
[12.61 to 14.85]
14.42
[13.05 to 15.79]
Beta 0.69
[-0.28 to 1.67]
Patient satisfaction
(5-point Likert)
3.26
[3.07 to 3.45]
3.26
[3.10 to 3.42]
Beta 0.001
[-0.18 to 0.18]

30. Web service for DSS
3

31. Reusable ontology model of evidence
4

32. Data Mining: Steps
5

33. Conclusions
• Improvement in diagnoses and decisions
• Without more investigations ordered/time taken
• A lot more data coded into the EHR
• GPs recorded during the consultation (not at the end)
• Opportunity to capture rich (and less biased) routine
data
• A Learning Health System for diagnosis

40. Thank you
Brendan Delaney
www.transformproject.eu
Brendan.delaney@imperial.ac.uk