1. Our story: How we got to an innovative model for
interprofessional longitudinal learning focused on
the care of older adults and the role for
developmental evaluation in practice improvement
Marion Briggs, BScPT, MA, DMan
Assistant Professor Clinical Sciences
Director, Health Sciences and Interprofessional Education
Northern Ontario School of Medicine (Sudbury)
Janet McElhaney, MD, FRCPC
HSN Volunteer Association Chair in Healthy Aging
Advanced Medical Research Institute of Canada,
Professor, Northern Ontario School of Medicine (Sudbury)
Maurianne Reade, MD
Assistant Professor, Clinical Sciences
Northern Ontario School of Medicine (Manitoulin Island)
MUSTER Conference, Uluru, Australia
October 30, 1115-1145
How to Drive the MoH Crazy!

2. Faculty/Presenter Disclosure
• Marion Briggs and Maurianne Reade
have no commercial relationships
• Janet McElhaney has commercial
relationships (honoraria and travel
reimbursement) with Sanofi, GSK,
Merck, MedImmune, Valiant – there are
no conflicts of interest related to this
project.

3. The “ask” in a Ministry-funded health
education policy initiative ...
• Review best practices in Canada and Internationally in
collaborative care models for older adults that integrate
student and provider learning
• Identify gaps in collaborative care models for older
Ontarians in the acute care context
• Recommend opportunities for further development of
collaborative care models
• Identify Health and Education policy implications ...

4. The “ask” in a Ministry-funded health
education policy initiative ...

5. The search for certainty and standardization ...
No go ... The innovation
committee said it doesn’t meet
utility specs. They want
something linear, stable,
controllable, and targeted to
reach a pre-set destination. They
couldn’t see any use for this.
Patton, 2011

6. When policy tries to direct WHAT
practitioners do and that translation of
“best” into practice fails...
...we respond by improving either the evidence (getting it “more”
right) or the translation strategy (improving compliance) ... it’s a bit
like doing more of the same, expecting different results
Stacey, R. (2005). Experiencing Emergence in Organizations: Local Interaction and the Emergence of Global Patterns. Routledge.

7. Our preliminary literature scan
showed, as expected, that ...
• Models of IPE/IPC claiming to represent
“best practice”:
– Tended to be descriptive
– Rarely linked “best practice” to patient outcomes
– Often did not thoroughly describe the context, the
team, and/or patient characteristics
– Used multiple methodologies and methods
– Attempts to replicate studies with positive
outcomes were rarely successful

8. Our team argued that researchers, practitioners
and policy makers face a choice between two
competing kinds of practice knowledge …
• Evidence-based Practice:
– Based upon technical know how
– Standardization is valued
– Equated with a ‘search for certainty’ and accountability
– Policy tells people what to do and how to do it
• Critical Practice:
– Concerned with critical exploration and development;
promotes innovative, locally relevant practice and education
– Makes creative use of uncertainty and considers context
vital – relevance is valued over standardization
– Policy tells people what outcomes are needed (why, not
what or how)
Stepney, P., Rostila, I., (2011). Towards an integrated model of practice evaluation balancing accountability,
critical knowledge and developmental perspectives. Health Sociology Review 20(2). 133-146.
How to Drive the MoH Crazy!

9. Meta-narrative Synthesis
Make sense of different kinds of evidence about complex
interventions applied in diverse contexts; used to inform policy
• What disciplines have researched in this area?
• How did they conceptualize the problem?
– a problem to be solved; a process to be described; a
process to be evaluated ...
• What theories and methods did they define as
“rigorous” and “valid”?
– RCT’s? Qualitative statistical data? Ethnography?
Narrative?
• Difference is of most interest in M-N methodology

10. Our recommendations ...

11. Shift AWAY from policy grounded in assumptions of
homogeneity or universality (“best practice for all”),
instead ...
• Develop policies that take diversity, complexity,
and local context seriously
• Emphasize local continuous improvement
through strategies like development evaluation
Stand up and wave if you think the Ministry
understood and was excited by this recommendation!

12. Stand up and
wave if you could
get excited by this
recommendation!
Policy-makers need to STOP telling practitioners
what to do, or “HOW” to practice ... instead
• Have communities of patients develop and
policy makers/funders support accountability for
key patient-oriented outcomes such as:
– Preventing loss of independence during acute illness
– Preventing complications of care within and between
care environments (including home)
– Healthcare decision-making that supports enhanced
patient-determined quality of life
The “why” excites people ... it speaks to why we decided
to become HCP’s ... if people remain committed to “why”,
the “how” will emerge in relevant, meaningful ways ...

13. Shift away from sector-specific policy,
instead:
• Support integrated cross-sector policies that
take into account a broad range of genetic,
biological and social determinants of health
• Draw the links to health in many policy sectors
… not just health, but in education across the
lifespan and in all sectors, transportation,
social care, law, libraries, public works, finance
…
Changes would be
needed in
classroom and
clinical education!
Do we know what
changes would be
needed?
Are we ready to
make them?
We confess here to a belief that what is good for the health of older
adults is good for everyone ...

14. Shift away from HP Educational approaches that rely
on individual competence acquired through organ-based,
short-term, single-sector, single discipline
clinical education ... Instead …
Develop HE approaches that:
• Support sector-integrated, “whole patient”,
interprofessional clinical education
• Expose learners to a broad range of health and
social care models across the continuum of care
• Emphasize longitudinal, interprofessional clinical
education models
• Focus on the acquisition of unique discipline-specific,
interpersonal (putting the “I” back in team) and
interprofessional team competencies

15. Finally, person-centred principles should
inform all policy development ...
• Includes attention to patient-centredness and to
relationships between healthcare providers, learners,
community leaders ...
• Takes into account the many complex, interdependent
relationships needed to support health …
• Minimizes the impact of differences in power and position
• Understands expert knowledge as an “improvable idea”*
(to be shared within a knowledge-building community for
the purpose of collective and collaborative reflection and
improvement)
• Commits to relational ethics (inclusive of principle-based
ethics) as the foundational ethic for healthcare practices
*Mylopolous, 2012; Briggs, 2012

16. A word about developmental evaluation
• Supports real-time, adaptive learning
• Informs ongoing decision-making and thus
actively continuously shapes practice
• Mechanisms used to measure/monitor evolve
thus support improved understanding and
enhanced adaptation
• May be a precursor to more traditional
formative and summative evaluation strategies
Patton, 2011; Dozios, 2010; Gamble, 2006

17. Traditional and DE Evaluation
A Comparison of Traditional and Developmental Evaluation Approaches
Evaluation Component Traditional Evaluation Developmental Evaluation
Purpose Validate a model or hypothesis; accountability Development, adaptation
Situation Stable, goal oriented, predictable Complex, dynamic, changing
Mind-set Effectiveness, impact, compliance Innovation, learning
Measurement Based on predetermined indicators Based on emergent indicators
Unexpected consequences Paid token attention (explained away) Paid serious attention (thoroughly
explored)
Evaluation design By evaluator Collaborative with program staff
Evaluation methods Emphasis on inputs (RCTs) Emphasis on how outcomes change
Evaluation results (ideal) Best practices Best principles
Evaluator qualities Strong methodological skills, credibility with
external authorities and funders
Strong methodological skills, credibility
with organizational and program staff

18. Questions for further exploration
• If “what” and “how” are not policy matters, how would
“success stories” be shared and adapted?
• In shifting from the ideal of “best” to a balance of best
and contextually relevant practices, how would the
emphasis in HPE curricula need to change (consider
UME/PGME, pre/post licensure; CEPD/FD ...)
• What would it take to develop IP CTUs in acute and
community care? IP longitudinal clinical education
models?
• How would longitudinal IP clinical learning influence the
formation of professional identity?
• How should patient-oriented outcomes be developed,
measured, reported, interpreted, and used?

19. Thank you!
The final report of our project is available through:
http://cou.on.ca/policy-advocacy/health-education/
pdfs/health-workforce-educational-needs-for-seniors-
car