Illawarra Shoalhaven Cancer and Haematology Network (ISCaHN) has been using an oncology information system (OIS) as a complete electronic record for over 4 years. There has been both considerable and valuable treatment data generated at the point of care. Are we able to rapidly assess the outcomes of our own treatment data, and use this outcome data to help inform the delivery of care to our patients?
3. Aim
To describe the development of a treatment dashboard at
Illawarra Shoalhaven Cancer and Haematology Network
(ISCaHN)
4. Treatment Dashboard
The aim of the dashboard is to present data extracted in
real time from our Oncology Information System (OIS) that
is accessible and actionable for clinicians
This data can then be used to inform and support treatment
decisions
7. Foundation - RLS
Etheredge defined a rapid learning health care model as
one that generates as rapidly as possible the evidence
needed to deliver quality patient care 1
Users learn as much as possible as soon as possible
through the collection of data at the point of care that can
then be used to inform clinical care and service delivery
Whilst this model has been developed around the concept
of “big data”, it is also possible to apply it at a localised
level to achieve similar outcomes
1. Abernethy et al, 2010
9. Challenges for Big Data and RLS
Data correctness
Data completeness
Data consistency
Data storage
10.
11.
12. Development
Dashboard not developed in isolation
Result of experience from multiple extraction projects
including:
– CINSW Enhanced Medical Oncology Reporting Project
– Oncology Day Care Enhanced Scheduling Project
– Activity Based Funding Extract
– PROMPT care pilot project
13. Development
From lessons learnt in extract projects we were able to
develop an extract with relevant clinical data
This data is then displayed in a dashboard for clinicians to
access in a readable and accessible form
14.
15. Intake Data
You can only pull out what you've put in
Ensure quality and completeness of data
– Use of manual and automated QA's
– Regular staff training, education and support
Data needs to be accessible and actionable for clinicians
16.
17. Data Transformation and Aggregation
Treatment dashboard
Chemotherapy protocols – different protocols dependent
upon diagnosis, stage, co-morbidities
Gold standards in curative disease, greater variability in
palliative setting
Dash board not solely a tool for clinicians, we aim to
develop an option for patient viewing, so that they can be
walked through treatment options, empowering them in
their own treatment decision
24. Carboplatin/Gemcitabine in NSCLC
D1 or D8 Carboplatin?
Carboplatin combined with Gemcitabine has an established
role in the treatment of advanced NSCLC
In 2009 Crombie et al evaluated Two 21 day gemcitabine-
carboplatin schedules
Phase II study where 40 patients were given Gemcitabine
on D1 and Day 8 of a 21 day cycle, with patients being
randomized to having Carboplatin on either D1 or D8 of
their treatment
Results of the study showed that Carboplatin administered
on D8 resulted in lower dose intensity and more dose
delays
25. Carboplatin/Gemcitabine in NSCLC
D1 or D8 Carboplatin?
Based on the results of the Crombie study, eviQ
superseded their Carbo/Gem (D8 Carbo) protocol in July
2013, and left only the Carbo/Gem (D1 Carbo) protocol
approved
At ISCaHN, we had also made a similar change in practice
From January 2011 to March 2013 patients were
prescribed the Carboplatin/Gemcitabine protocol with D8
Carboplatin
From March 2013 the majority of patients were prescribed
Carboplatin/Gemcitabine, with carboplatin being delivered
on D1
27. Demographics
Carbo D1 Carbo D8
Carbo
D1
Carbo
D8
Sex Male % 70 45
Sex Female % 30 55
Stage III % 15 30
Stage IV % 85 70
Crombie et al
28. Results
Progressive disease on treatment comparison is possible
Average patients delayed per cycle comparison is possible
Crombie et al ISCaHN
D1 Carbo
n = 20 (%)
D8 Carbo
n = 20 (%)
D1 Carbo
n = 83 (%)
D8 Carbo
n = 97 (%)
7 (35) 8 (40) 33 (40) 38 (39%)
Crombie et al ISCaHN
D1 Carbo
n = 20 (%)
D8 Carbo
n = 20 (%)
D1 Carbo
n = 83 (%)
D8 Carbo
n = 97 (%)
2 (10) 4.75 (24) 23 (27) 40.7 (42)
29. Results
Toxicity comparison will be possible, still working on bugs
in the report and display of these
Unable to provide comparison for response rates as this is
currently poorly and/or not uniformly documented in day to
day clinical practice
Survival rates/time currently not calculated, but will be
possible in future
30. Dashboard Difficulties
Similar to large scale difficulties
– Incomplete data entry
– Inconsistent data entry
– Incorrect data entry
Survival outcomes, particularly for positive prognostic early
stage dx (breast, colon etc), requires lengthy time for
measurement of PFS rates and OS rates
31. Conclusion
Able to extract, aggregate and analyse data generated at
point of care to inform and optimise patient care
Ability to identify and measure patterns and trends in real
time
Visualisation of data enables rapid hypothesis generation
Possible to quickly compare treatment data with that from
published clinical trials
32. Conclusion
There are holes in the data – requires continued audit and
QA
Engage with staff, make the data presentable and
actionable, giving a reason for complete and correct data
entry
34. References
Abernethy AP, Etheredge LM, Ganz PA et al. Rapid-Learning System for Cancer Care.
Journal of Clinical Oncology. 2010;28(27): 4268-4274
Crombie C, Burns WI, Karapetis C, Lowenthal RM et al. Randomized phase II trial of
gemcitabine and either day 1 or day 8 carboplatin for advanced non-small-cell lung cancer: Is
thrombocytopenia predictable? Asia-Pacific Journal of Clinical Oncology 2009;5: 24-31
Hinweis der Redaktion
1
Treatment dashboard still at development stage – but feel it provides a good example of rapid learning, and hopefully gives people food for thought on some of the possibilities that this rapid learning/data visualisation may provide.
In conjunction with clinical trials and published literature
Today I’ll look to briefly discuss the foundation for the development of the dashboard
Then discuss development of the dashboard
Finally we’ll view the dashboard and view a few examples of how it can be utilised
So, at the foundation we have the framework of a rapid learning system, and the source of our data, where it’s all stored, the OIS
Etheredge defined….
So, uploading point of care data from multiple locations in large numbers to a central knowledge base
Aggregating data from multiple health records with clinical trials and published guidelines
Whilst this model has been developed around the concept of “big data”, it is also possible to apply it at a localised level to achieve similar outcomes
7
Whilst this shows great potential, I believe there are still real challenges for big data
Data correctness
Completeness
Consistency - Clinical trials have the ability to report strictly on tight definitions with additional staff and funding resources – this is not the norm in day to day clinical practice
Data storage - different locations in the same software at different sites, making multilateral data extraction time consuming and challenging
It is possible to mitigate these issues using computer intelligence, but issues remain
There are also other challenges for big data..
Like how he is going to step over this building without crushing innocent bystanders.
Here we have a Schema based on an ASCO CancerLinq slide on a RLS, and today I’d like to structure the development and production section of the talk around this
Firstly, I’d like to speak to the development or transformation and aggregation of data, and in particular, the visualisation of data
Then observe how the data visualisation facilitates the analysis of data
So what I’ll show you are basically pivot charts viewed in a dashboard. Perhaps they’re not that innovative. The novel step here is not in the presentation of the data. The novel step is the data extraction…..
Requires a broad skill set – an analyst/programmer with knowledge of the db and reporting nous; plus a clinician with extensive knowledge of clinical practice/process (click) and how this practice is recorded in the eMR software
So, with rapid learning as a foundation and an extract developed to provide us with the required data, what are some of the issues around development of a locally based RLS?
Age old adage garbage in, garbage out.
How do we ensure quality of data entered?
Manual and automated QA’s
Regular staff training
Most importantly, the data needs to accessible and actionable for the clinicians so that they can see the outcomes of their data entry
As Data from the Goonies proved, size in itself doesn’t matter – what matters is having the data, of whatever size, to assist in solving a problem or addressing a question we have
Combining our framework with our data intake, ISLHD cancer service have looked to use our data, routinely generated in delivery of care, to inform clinical care and service development
This has culminated in the development of the treatment dashboard
Whilst in it’s early stages, I believe it truly exhibits the potential that RLS holds at a local level
Chemotherapy is delivered in a set protocol….
Example of our clinical dashboard
Currently at proof of concept stage
Example of a relatively new drug for melanoma – Ipilimumab, and I think provides a good example of how the dashboard can be utilised
Drug showed promise for pts with stage 3 and 4 melanoma where treatment options were limited
(click) On the left hand side we have the number of patients (as a percentage) on the y axis – 39 patients delivered 1st cycle (with 4 unknown), and number of cycles on the x.
(click) In the middle with have toxicity grading
(click) On the right here we have demographics including Dx stage, Pt Age, Pt Gender and ECOG performance status
Ipilumumab is given for a total 4 cycles, every 3 weeks;
Ipilimumab costs $30,000 each cycle – at $157 per mg – To put that in perspective, that’s approximately 4,000 times the cost of gold….
In the real world PALLIATIVE setting (not the strict clinical trial setting, with strict guidelines on who can receive the drug), we felt anecdotally that there were:
High rates of toxicity and cessation of treatment due to toxicity;
Significant disease progression on treatment (with a high mortality rate);
The dashboard validated this (click). If we follow “delivered” treatment colours, we see the dramatic attrition rate, with a decreasing trend in the number of patients receiving treatment (39 pts in C1 to 18 pts in C4), and (click) an increasing number of discontinued treatments (just over 25%), with an unfortunately high mortality rate (just over 25%)
Exploring the dashboard further…
(click) On the left we can see in more detail the toxicity grading 0-3, with each column representing the grade of toxicity each cycle (this is still a work in progress)
(click) Can see N/A column which is to some extent, the level of incomplete data
(click) We can see some of the data aggregation where demographic data is used to filter to a specific patient, and this filters on both the assessments and treatment details
We also aim to introduce Patient Reported Side Effects/Outcomes - having different views dependent upon the individual accessing the information
Here is an example where we’ve aggregated the data to see only stage 4, male patents, aged between 60-69
So there are a few aspects of this data visualisation that I believe can play a role in clinical decision support.
1. We have treatment data from patients treated in the real life setting that can assist in guiding clinician judgement on perhaps who to treat and when to treat
2. Can we display the results differently for patient’s to view? Ask patients what they want to see, what would assist them in making a treatment decision.
3. Is there a trend to the subset of patients deceased during treatment that warrants further examination? Can we identify these patients earlier and perhaps palliate them more appropriately?
I’ve just mentioned data analysis. Now that we’ve seen the transformation and aggregation of data, let’s observe how the dashboard assists in data analysis
The next care plan I want to discuss in the dashboard setting is Carboplatin and Gemcitabine in Non Small Cell Lung Cancer.
The change in practice not only gave us an ability to compare the two treatment care plans, it also gave us the opportunity to compare with the published literature
The dashboard gives us an ability to quickly assess our own treatment details in comparison to published literature
In comparison to the literature (20 patients in each arm), here we have 83 patients receiving D1 Carboplatin and 97 patients receiving D8 carboplatin
You can also see that we have some data entry issues with unknown patients representing approximately 15% of total entries
There are also the issues around patients not being randomized and the analysis being retrospective, however, we have a much larger population of patients to analyse
Here we are able to compare the Crombie article demographics with our own
Genders are more evenly represented in both arms
Staging is different, particularly as a result of incomplete diagnosis data entry (i.e. the diagnosis staging was not completed)
We are able to quickly compare results between the literature and our own data
We have similar rates of progressive disease treatment
However, our average number of patients delayed is significantly higher in both arms – could be that the study had a much higher use of blood products to address neutropenia and thrombocytopenia
Contributing factors: staff workload, unintuitive software, lack of clinician buy in, data entry errors
Rapid learning requires choosing the correct outcomes to learn from (shorter outcomes – not long term)
Optimisation of patient care includes time, resources, and clinical decisions (e.g. number of staff required for clinical trials)
Anecdotal theories can quickly be visualised to determine if there is a trend worth investigating
Will never replace clinical trials, but can be used to support treatment decisions alongside clinical trial
Holes in data, but I believe we’ve proved that it can still be done despite weaknesses
Dashboard can be plugged into other sites with MOSAIQ OIS