This document outlines the research phases and studies conducted to develop and evaluate a system called SoPhy, which uses socks with sensors to capture lower body movement data and visualize it to support physiotherapists in video consultations. The research included observations of current video consultations, prototyping SoPhy, laboratory evaluations to assess SoPhy's diagnostic efficacy, and field deployments at a hospital. Key findings were that SoPhy allowed physiotherapists to assess patients with fewer repetitions, increased confidence in assessments, and guided more specific treatments over video. The research contributes to understanding bodily communication in physiotherapy and enhancing clinical video consultations.

1. Supporting
Bodily Communication
in Video Consultations
of Physiotherapy
Deepti Aggarwal
Wednesday, 16 August 17

2. Video Consultations of Physiotherapy
van Dyk 2014; Yellowlees 2005
Wednesday, 16 August 17

3. Bodily Communication in Physiotherapy
Argyle 2013; Heath 1986, 2002
Wednesday, 16 August 17

4. Existing Literature
Physiotherapy
Video
Consultation
Bodily
Communication
Wednesday, 16 August 17

5. Prior works on Video Consultations
Focus on validation & viability for
different clinical domains.
Physiotherapy
Video
Consultation
Bodily
Communication
Ekeland et al., 2010 ; van Dyk, 2014
Only audio-visual medium is
explored.
Wednesday, 16 August 17

6. Prior works on Physiotherapy
Physiotherapy
Video
Consultation
Bodily
Communication
Technologies are mainly
designed for autonomous
home rehabilitation.
Focus is on patients’
motivation.
Lam et al., 2016; O’Hara et al., 2016; Tang et al., 2015
Wednesday, 16 August 17

7. Prior works on Bodily Communication
Physiotherapy
Video
Consultation
Bodily
Communication
Limitation of video mediated bodily
communication is known.
Bodily communication is primal
in face-to-face consultation.
Olson & Olson, 2013; Heath 1986, 2002
Wednesday, 16 August 17

8. Research Gap
Physiotherapy
Video
Consultation
Bodily
Communication
Wednesday, 16 August 17

9. Main Research Question
How can interactive technologies
support physiotherapists in
understanding bodily information
during video consultations?
Wednesday, 16 August 17

10. Research Phases
#1 Observations
#2 Prototyping
#3 Laboratory Evaluation
#4 Field Deployments
Wednesday, 16 August 17

11. Research Phases
#2 Prototyping
#3 Laboratory Evaluation
#4 Field Deployments
#1 Observations
Wednesday, 16 August 17

12. Study 1 - Research Question
How do physiotherapists
interpret bodily information in
current practices of video
consultations?
Wednesday, 16 August 17

13. #1 Observations
Face-to-Face (3) Video (7)
2 physiotherapists & 5 patients with chronic pain
Prescribed exercises: tip-toes, squats
Wednesday, 16 August 17

14. Face-to-Face Video
1. Opening
2. History taking
3. Examination
4. Diagnosis
5. Treatment
6. Closing
Byrne & Long 1976
Wednesday, 16 August 17

15. Lower Body
Movements
Wednesday, 16 August 17

16. #1 Limited Visual Acuity of Video Technology
Weight Distribution Range of Movement
Wednesday, 16 August 17

17. #2 Alignment with upper torso
Wednesday, 16 August 17

18. Focusing Camera Removes the Context
Wednesday, 16 August 17

19. Reduced confidence in
assessment
Less effective treatment
Wednesday, 16 August 17

20. ACM DIS 2016 (full paper)
Wednesday, 16 August 17

21. Research Phases
#1 Observations
#3 Laboratory Evaluation
#4 Field Deployments
#2 Prototyping
Wednesday, 16 August 17

22. Study 2 - Research Question
How can we collect and visualize
lower body movements to support
physiotherapists in assessing
patients during video consultations?
Wednesday, 16 August 17

23. Key aspects of lower body movements
Weight
Distribution
Range of
Movements
Foot
Orientation
Wednesday, 16 August 17

24. How to capture?
Wednesday, 16 August 17

25. SoPhy
SOcks for PHYsiotherapy
Socks with sensors Web interface
Wednesday, 16 August 17

26. SoPhy socks
IMU
Pressure
Sensors
Bluetooth
Shield
Arduino
Pro Mini
Wednesday, 16 August 17

27. SoPhy web-interface
Wednesday, 16 August 17

28. Wednesday, 16 August 17

29. Wednesday, 16 August 17

30. Research Phases
#1 Observations
#2 Prototyping
#4 Field Deployments
#3 Laboratory Evaluation
Wednesday, 16 August 17

31. Study 3 - Research Question
What is the diagnostic efficacy
of SoPhy for assessing lower
limb movements in video
consultations?
Wednesday, 16 August 17

32. Study Setup
Clinician’s room
10 physiotherapists & 1 actor (patient)
Patient’s room
Wednesday, 16 August 17

33. Study Design (2x2)
Extreme pain Low pain
Factors: Consultation Technology & Pain Levels
Wednesday, 16 August 17

34. Study Design (2X2)
Independent
Variables
Consultation Technology - with
SoPhy & without SoPhy
Pain Levels - extreme and low
Dependent
Variables
Weight Distribution, Range of
Movement, Foot Orientation,
Confidence, #Repetitions
Data
Collection
Interviews, Questionnaire, Video
Recording
Wednesday, 16 August 17

35. Tasks for
Participants
Assess the patient’s exercises
Organize 4 video sessions
Report confidence in a
questionnaire
Wednesday, 16 August 17

36. Tasks for Actor
Dorsiflexion &
Plantarflexion
1-leg tip-toes &
2-leg tip-toes
Squats
Wednesday, 16 August 17

37. Key Findings
Fewer repetitions of exercises
were required for assessment
Increased the confidence in
assessment
Insights on weight distribution
was found invaluable
Wednesday, 16 August 17

38. ACM CHI 2017 (full paper)
Wednesday, 16 August 17

39. Revised SoPhy Design
Wednesday, 16 August 17

40. Revised SoPhy Interface
Wednesday, 16 August 17

41. Research Phases
#1 Observations
#2 Prototyping
#3 Laboratory Evaluation
#4 Field Deployments
Wednesday, 16 August 17

42. Study 4 - Research Question
How does SoPhy support
physiotherapists in assessing
and treating patients with lower
limb issues during naturally
occurring video consultations?
Wednesday, 16 August 17

43. Study Design
1 physiotherapist, 3 patients with lower limb issues
Simulated video sessions at Royal Children’s Hospital
Hospital Room #1 Hospital Room #2
Wednesday, 16 August 17

44. Study Design
Face-to-Face
(60 mins)
Video
Consultation
(45 - 50 mins)
Face-to-Face
follow up
(15 - 10 mins)
First session Second session
Observations: 4 face-to-face and 2 video consultations
Interviews: 10
Wednesday, 16 August 17

45. SoPhy as an Assessment Tool
Wednesday, 16 August 17

46. SoPhy as a Therapeutic Tool
Wednesday, 16 August 17

47. Without SoPhy With SoPhy
(Study 4)(Study 1)
#5. Treatment
#1. Opening
#2. History Taking
#3 & 4. Examination & Diagnosis
#6. Closing
Wednesday, 16 August 17

48. Without SoPhy
Limited incidental cues Increased incidental cues
With SoPhy
(Study 4)(Study 1)
#1. Opening
Limited small talk Opportunities for small talk
Wednesday, 16 August 17

49. Without SoPhy
Reliance on verbal
explanation
Visualisation aided the
verbal explanation
With SoPhy
(Study 4)(Study 1)
#2. History Taking
Difficulty in observing
subtle differences in
movements
Subtle differences in
movements were
distinguishable
Wednesday, 16 August 17

50. Without SoPhy
Lack of clarity in patient’s
condition
Better understanding of
patient’s condition
With SoPhy
(Study 4)(Study 1)
#3 & 4. Examination & Diagnosis
Cheating or tricking was
difficult to detect
Contradictory behaviors
were easy to notice
Wednesday, 16 August 17

51. Without SoPhy
Hesitation in changing
the treatment
Confidence in changing the
treatment
With SoPhy
(Study 4)(Study 1)
#5. Treatment
Wednesday, 16 August 17

52. Without SoPhy
Limited small talk Opportunities for small talk
With SoPhy
(Study 4)(Study 1)
#6. Closing
Wednesday, 16 August 17

53. Key Findings
Guided more specific treatment
Increased the confidence in
assessment
Made the therapy goals intuitive
for patients
Wednesday, 16 August 17

54. Thesis Contributions
To theory -
First conceptual
understanding of
bodily communication
in physiotherapy
To clinical practice -
First exploration on
how to enhance
diagnostic ability of
clinicians over video
To methodology -
The study design of
Study 3 & 4 could
guide future
interventions
To HCI practice -
Practical insights on
the design of wearable
system around bodily
communication
Wednesday, 16 August 17

55. Dissertation Status
Time Tasks
July - Aug.
2017
Literature Review (Ch 2)
Discussion (Ch 8)
Methodology (Ch 3)
Sept. 2017
Revision of complete thesis
Thesis Submission
Wednesday, 16 August 17

56. Some memorable moments
Wednesday, 16 August 17

57. OzCHI 2015
OzCHI 2015
CHI 2017
Wednesday, 16 August 17

58. Foundation of ‘Social’NUI, 2015
Wednesday, 16 August 17

59. Past CHI Chair :D
Wednesday, 16 August 17

60. Space for the
convocation
photograph :)
A long awaited
award!
Wednesday, 16 August 17

61. Thank you!
Wednesday, 16 August 17