A database of patients using interactive tools to allow for online case studies

1. Assessment of the use of the Virtual Orthodontic
Case Studies in an Undergraduate Teaching
Program
Dr Jean –Marc Retrouvey
VOCS: An Interactive Teaching Tool in Orthodontics

2. What is CAL?
Computer Assisted Learning
 Any type of pedagogical material that is delivered
through a computer software.
 Can be very simple (PPTs) or very complex (Flight
simulators)
 Most effective is the interactive model where students
participate to the learning process

3. Validity of CAL
 Rosenberg study demonstrated that CAL usage was at
least as effective in orthodontic undergraduate
education (diagnosis) as standard classroom lectures
 One of the few orthodontically oriented pedagogical tools
build with instructional designers
 Rosenberg, H., et al., Evaluation of computer-aided learning in orthodontics.
American Journal of Orthodontics and Dentofacial Orthopedics, 2010. 138(4): p.
410-419.

4. Interactive Seminars are coming to Orthodontics….
 Interactive Distance Learning in Orthodontic
Residency Programs: Problems and Potential
Solutions
 Katherine P. Klein, D.M.D., M.S.; Wallace H. Hannum, Ph.D.; Henry W. Fields, D.D.S., M.S.,
M.S.D.; William R. Proffit, D.D.S., Ph.D. (2012)
 Sharing resources through distance education has been
proposed as one way to deal with a lack of full-time
faculty members and maintain high-quality content in
orthodontic residency programs. To keep distance
education for orthodontic residents cost-effective while
retaining interaction, a blended approach was
developed that combines observation of web-based
seminars with live post-seminar discussions.

5. Rationale for VOCS
 Database of carefully selected patients used for
pedagogical reasons (searchable)
 Study and present cases anywhere (
home, school, lectures)
 Avoid model storage and manipulations (paperless
records)
 Fast loading time on any computer (No 3D )
 Capability to test students electronically

6. Introduction to VOCS

7. Objectives of this study
 First objective:
To evaluate students’ perceptions concerning a series of
aspects of the VOCS learning tool including quality and
quantity of information provided and simplicity of use.

8. Objectives of the study
Second Objective:
To determine if there was a difference in the results of
data collected from “conventional “ examination and
measurements on study casts and the VOCS software.
 Question : Is VOCS a reliable tool to study orthodontic
cases?

9. Study Design: part 1
 21 students
 Each student studied one case using the VOCS online system
 questionnaire was filled by same student to assess his/her
perception about the online method’s quality and likability.
 evaluating ‘general material’, ‘patient history’, ‘facial
images’, ‘intra-oral images’, ‘dental cast’, ‘panoramic’,
‘cephalometrics’, ‘quiz’, and ‘overall feedback’ were
presented.
 Each item had a 4 category scale ranging from ‘strongly
disagree’ to ‘strongly agree’.

10. – Virtual Orthodontic Case Studies (VOCS)
Date: ___________________
Instructions: Evaluations provide us with important feedback regarding our work. Please take a couple of minutes to answer the following questions
candidly. We thank you for your time and input which will assist in preparing for future work developed for the VOCS. If you require additional room for
your comments, please feel free to add an extra sheet of paper.
Strongly Disagree Agree Strongly
Disagree Agree
Interactive Materials/Activities
1. Module objectives were clear.    
2. Module materials were well written.   

3. Module materials were well organized.    
4. Materials/activities were helpful in learning new skills    
5. Materials/activities were relevant and interesting    
6. The module provided new information and/or    
sufficient depth to challenge me.
Program Outcomes
7. The outlined module objectives were met.    
8. I learned something I can immediately apply.    
9. My time was well invested in this module.    
10. I would recommend this module to other individuals.    

11. Evaluation – VOCS
Strongly Disagree Agree Strongly
Disagree Agree
Overall Usability of Website
11. The website was easy to navigate.    
12. It was simple to find information.    
13. The radiographs were clear.    
14. The photographs were clear.    
15. The website models were sufficient for diagnosis.    
Website as a Teaching Method
16. The website is a good teaching method.    
17. The cases on the website are accessible anywhere/time.    
18. The cases on the website can be reviewed easily.    
19. The case material is easy to refer to during the quiz.    
Website Compared to other Teaching Methods
20. The website is a better teaching method than the    
student presentations.
21. The website is a better teaching method than the    
hands-on teacher-to-class technique.

12. Evaluation – VOCS
You Tell Us
22. In your opinion, what were the advantages of the website over other teaching methods?
23. In your opinion, what were the disadvantages of the website over other teaching methods?
24. Additional comments (please feel free to add further remarks on another sheet).

14. History

15. TABS

16. Radiographs

17. Boley Gage Assessment
Boley Gage
instructions easy to Boley Gage is simple to use
follow
100 100
90
80
80
60 70
60
40 50
20 40
30
0 20
Strongly Disagree Agree Strongly 10
Disagree Agree 0
Strongly Disagree Disagree Agree Strongly Agree

18. Quizz

19. Diagnostic Capability
The website models were sufficient for
diagnosis
All students either
100

90
80
agreed that this 2 D
70
method was sufficient to
60 diagnose simple to
50 moderately complex
40 malocclusions.
30
20
10
0
Strongly Agree Agree Disagree Strongly
Disagree

20. Overall Feedback
My time was well invested in this module I would recommend this module to other
individuals
100 100
90 90
80 80
70 70
60 60
50 50
40 40
30 30
20 20
10 10
0 0
Strongly Agree Agree Disagree Strongly Strongly Agree Agree Disagree Strongly
Disagree Disagree

21. Comments for part 1
 95% of the students responded ‘agree’ or ‘strongly
agree’ to the questionnaire’s items (36 and
59%, respectively). Twenty of the 46 items had 100%
of the students responding either ‘agree’ or ‘strongly
agree’.
 The only negatively phrased item (‘some questions are
redundant’) had 65% of responses at ‘agree’ or
‘strongly agree’ and the remaining 35% at ‘disagree’ or
‘strongly disagree’.(quality of the quizz)

22.  Biggest problem: The item concerning ‘intra-oral
images being clear’ had 2 people (10%) responding
‘strongly disagree’ .
 Midline determination is challenging

23. Part 2: Reliability measurements

24. Reliability of the VOCS measurements in relation
to conventional measurements
 Duplicated the methodology used in dental schools for
an initial orthodontic case study (undergraduate level)
 Assess if the students measured the same things and
obtain the same diagnosis ( correctly or incorrectly).

25. Methods
 21 students participated
 13 completed the exercise correctly
 Analyzed conventional records and measured study
casts (test reliability and accuracy of the virtual tools)
 Teachers (3) corrections or evaluations were deemed
be worth 100% of the score (total agreement on data
analysis)

26. Table 1: Individual students percentage score in the
orthodontic analysis evaluation using either models or the
VOCS.
Students Percentage score
Model VOCS
1 50,0 59,1
2 68,2 50,0
3 72,7 72,7
4 45,5 68,2
5 45,5 50,0
6 81,8 31,8
7 81,8 81,8
8 45,5 63,6
9 77,3 68,2
10 63,6 68,2
11 86,4 81,8
12 77,3 72,7
13 59,1 54,5

27. Table 2: Test statistics on the differences in average scores
between groups using model or VOCS to complete the
orthodontics analysis evaluation.
n Mean score Standard Deviation p
Model 13 65,7 15,2 0,63
VOCS 13 63,2 14,1

28. Results of the analyses from students
 The results show that their was no significant
difference between the analyses of the students if
they used either the standard study casts, boley gages
and rulers or if they used the virtual dental casts and
tools (P=0.63)
 The students who got the right or wrong diagnosis
using either methods (except for one student 80 -30)

29. Discussion and Conclusions
 Case studies can be made available 24/7 on any LMS
(learning management system)
 Quizzes and cases studies can be packaged by the
instructor or the administrator
 Repository of searchable cases
 Easy to load on any computer system
 Reliable and test at the same level any case analysis

30.  Self paced teaching tool used now at McGill
 Canadian National Dental Board has used 4 of the VOCS
cases for its examination process
 Very positive response to the Virtual environment.
 All students from any group can be tested on the same
case

31. Better teaching tool?
More than 50% still preferred the one on one hands-on teaching technique
(gold standard). Perfectly normal as feed back and assistance are not readily
available with the on line method.
It was preferred to the students presentations .
100
The website is a better teaching method 100
90 The website is a better teaching
than the student presentations 90
method than the hands-on
80
80 teacher-to-class technique
70
70
60
60
50
50
40
40
30
30
20
20
10
10
0
0
Strongly Agree Disagree Strongly
Strongly Agree Disagree Strongly
Agree Disagree
Agree Disagree

32. Thank you