Ted Drake from Intuit and this year's general chair of Web4All Conference 2020 is organising a series of lunch and learn session to take advantage of people working from home! The W4A keynotes of Vivienne Conway, Director (Web Key IT Pty Ltd) and Ricardo Baeza-Yates (NTENT & Khoury College of Computer Sciences Northeastern University at Silicon Valley) have been already shared. Now, they ask me to give an overview of research and technologies around dyslexia which is next Thursday on the 14th of May !
Incoming and Outgoing Shipments in 3 STEPS Using Odoo 17
Technologies for Dyslexia
1. Technologies for Dyslexia
Maria Rauschenberger
Max Planck Institute for Software Systems, Saarland, Germany.
14th May 2020, at Intuit
Rauschenberger (MPI) Twitter: Rauschii 1 / 1 14th May 2020, at Intuit 1 / 28
2. Outline
1 Common Characteristics of Dyslexia
2 Intervention and Assisted Technology
3 Screening of Dyslexia
Early and playful Screening of Dyslexia
Methodology
Game and Content Design
Results
Conclusion
4 Take Away
Rauschenberger (MPI) Twitter: Rauschii 2 / 2 14th May 2020, at Intuit 2 / 28
3. Common Characteristics of Dyslexia
Common Characteristics of Dyslexia
Rauschenberger (MPI) Twitter: Rauschii 3 / 3 14th May 2020, at Intuit 3 / 28
4. Common Characteristics of Dyslexia
Definition of Dyslexia
What is dyslexia?
(American Psychiatric Association, 2013)
Rauschenberger (MPI) Twitter: Rauschii 4 / 4 14th May 2020, at Intuit 4 / 28
5. Common Characteristics of Dyslexia
Dyslexia is ...
Definition of Dyslexia
A specific learning disorder/difficulty
(American Psychiatric Association, 2013)
Reading and writing impairments
Wrongly associated to reduced intelligent
Important cause of school failure
Rauschenberger (MPI) Twitter: Rauschii 5 / 5 14th May 2020, at Intuit 5 / 28
6. Common Characteristics of Dyslexia
Dyslexia is ...
Definition of Dyslexia
A specific learning disorder/difficulty
(American Psychiatric Association, 2013)
Reading and writing impairments
Wrongly associated to reduced intelligent
Important cause of school failure
Affects 5–15% of the population
Multiple factors investigated to discover its cause
(Catts et al., 2017; De Zubicaray and Schiller, 2018)
Dyslexia is a combination of characteristics
(De Zubicaray and Schiller, 2018)
Rauschenberger (MPI) Twitter: Rauschii 5 / 5 14th May 2020, at Intuit 5 / 28
7. Common Characteristics of Dyslexia
What kind of Errors do People with Dyslexia make?
Rauschenberger (MPI) Twitter: Rauschii 6 / 6 14th May 2020, at Intuit 6 / 28
8. Common Characteristics of Dyslexia
What kind of Errors do People with Dyslexia make?
Error Classification: Substitution, Insertion, Omission, ...
Depending on the Orthography of the language
(Rauschenberger et al., 2016)
Rauschenberger (MPI) Twitter: Rauschii 6 / 6 14th May 2020, at Intuit 6 / 28
9. Common Characteristics of Dyslexia
Language-Independent Indicators
Evidence from literature & strongly related to dyslexia
Similarities between languages, German, English and Spanish
(Rello et al., 2016; Rauschenberger et al., 2016, 2017)
— Error category Substitution: exchanging a letter for another one
(Rello et al., 2016; Rauschenberger et al., 2017)
Examples:
— b, d, p, q
— versbrochen (versprochen, ‘promise’)
— Walt (Wald, ‘forest’)
(Rello et al., 2016; Rauschenberger et al., 2016)
What happens if the error produces a new word?
Rauschenberger (MPI) Twitter: Rauschii 7 / 7 14th May 2020, at Intuit 7 / 28
10. Intervention and Assisted Technology
Real World Errors
(Rauschenberger et al., 2015)
Rauschenberger (MPI) Twitter: Rauschii 8 / 8 14th May 2020, at Intuit 8 / 28
11. Intervention and Assisted Technology
Intervention with Errors
(Rauschenberger et al., 2015)
https://itunes.apple.com/de/app/dyseggxia/id534986729?mt=8
Rauschenberger (MPI) Twitter: Rauschii 9 / 9 14th May 2020, at Intuit 9 / 28
12. Intervention and Assisted Technology
Assisted Reading for Dyslexia
Text Customization
e.g., Font size (>16pt)
Font Type such as Arial
(italics and serif should be avoided (Rello and Baeza-Yates, 2016,
2017; British Dyslexia Association, 2018).)
Text Simplification
“Since dyslexia is a learning disorder and not a cognitive disability, the
simplification depends more on the typographical errors and not on
the complexity of the content.” (Rauschenberger et al., 2019b)
Text to Speech
Rauschenberger (MPI) Twitter: Rauschii 10 / 10 14th May 2020, at Intuit 10 / 28
13. Intervention and Assisted Technology
Assisted Writing for Dyslexia
“[...] people with dyslexia experience more often negative feedback on the
writing which can trigger or increase their stress and anxiety.”
(Reynolds and Wu, 2018)
Spelling Correction
Text Suggestions
Dictation
Rauschenberger (MPI) Twitter: Rauschii 11 / 11 14th May 2020, at Intuit 11 / 28
14. Screening of Dyslexia
Screening
How to detect if you have dyslexia?
Rauschenberger (MPI) Twitter: Rauschii 12 / 12 14th May 2020, at Intuit 12 / 28
15. Screening of Dyslexia
Examples of Dyslexia Screening
Digital
Readers
Dytective
GraphoGame
Lexercise Screener
Nessy
Pre-Readers
AGTB 5–12
DYSL–X
Game–Collection
Lexa
Others
Diagnostischer Rechtschreibtest
Hamburger Leseprobe
Special hardware, i.e.,
— MRI or fMRI Scans
(Paulesu et al., 2014; Tamboer et al., 2016;
Paz-Alonso et al., 2018)
— Eye-tracking
(Asvestopoulou et al., 2019)
Specialist staff, i.e.,
— learning therapists
(Rauschenberger et al., 2019b)
Rauschenberger (MPI) Twitter: Rauschii 13 / 13 14th May 2020, at Intuit 13 / 28
16. Screening of Dyslexia
To sum up...
All reading and spelling tests need a ...
— minimum knowledge of phonological awareness,
— grammar,
— and vocabulary
... to predict risk of dyslexia is a specific language task.
Rauschenberger (MPI) Twitter: Rauschii 14 / 14 14th May 2020, at Intuit 14 / 28
17. Screening of Dyslexia Early and playful Screening of Dyslexia
What we have done!
Screening a person without using linguistic knowledge
while having fun!
Rauschenberger (MPI) Twitter: Rauschii 15 / 15 14th May 2020, at Intuit 15 / 28
18. Screening of Dyslexia Methodology
Screening Risk of Dyslexia through a Web-Game
using Language-Independent Content and Machine Learning.
Challenge → Approach
Hidden disorder → Indicators
Different languages → Language-independent
Content → Iterative design
Design to measure → Human-centered design
No global standards → Data science
External factors → Transparency
Collecting health data → Ethical standards
Small data → Avoid over-fitting
Rauschenberger et al. (2019a)
Rauschenberger (MPI) Twitter: Rauschii 16 / 16 14th May 2020, at Intuit 16 / 28
19. Screening of Dyslexia Methodology
Steps
1 Designing a language-independent content
game
— Visual and auditory cues
2 Measuring the interaction (e.g., score, duration)
— Between children with and without dyslexia
— Age 7-12 year’s old
3 Analysing the interaction measures
— Traditional statistical methods
— Existing machine learning classification
Child playing.
Rauschenberger et al. (2019a)
Rauschenberger (MPI) Twitter: Rauschii 17 / 17 14th May 2020, at Intuit 17 / 28
20. Screening of Dyslexia Game and Content Design
Game and Content Design
Visual part with the priming of the
target cue symbol (left) and then the
(right) nine-squared design including the
distractors for each symbol.
Auditory part for the first two clicks on
two sound cards (left) and then when a
pair of equal sounds is found (right).
Rauschenberger et al. (2019a)
Rauschenberger (MPI) Twitter: Rauschii 18 / 18 14th May 2020, at Intuit 18 / 28
21. Screening of Dyslexia Game and Content Design
Prototype MusVis — Visual Part
Rauschenberger (MPI) Twitter: Rauschii 19 / 19 14th May 2020, at Intuit 19 / 28
22. Screening of Dyslexia Game and Content Design
Prototype MusVis — Auditory Part
Rauschenberger (MPI) Twitter: Rauschii 20 / 20 14th May 2020, at Intuit 20 / 28
23. Screening of Dyslexia Game and Content Design
Related Language-Independent Content
Related with auditory and visual content that refers mainly to one
single acoustic or visual indicator
e.g., frequency or horizontal similarity
Rauschenberger (MPI) Twitter: Rauschii 21 / 21 14th May 2020, at Intuit 21 / 28
24. Screening of Dyslexia Results
There are significant statistical differences
between children with and without dyslexia
when playing a game with auditory and visual content!
Data set Visual Auditory
ES (n =153) total clicks, first click, hits,
and efficiency (4)
4th click, duration, and
average (3)
DE (n =149) / /
ALL (n = 313) total clicks (1) /
Overview of the significant dependent variables of MusVis. Since all variables
were not normally distributed (Shapiro-Wilk test), we applied the
Mann-Whitney U Test.
⇒ Results let us assume, we can measure over different languages
with the same content.
Rauschenberger et al. (2018)
Rauschenberger (MPI) Twitter: Rauschii 22 / 22 14th May 2020, at Intuit 22 / 28
25. Screening of Dyslexia Results
There are significant statistical differences
between children with and without dyslexia
when playing a game with auditory and visual content!
Data set Visual Auditory
ES (n =153) total clicks, first click, hits,
and efficiency (4)
4th click, duration, and
average (3)
DE (n =149) / /
ALL (n = 313) total clicks (1) /
Overview of the significant dependent variables of MusVis. Since all variables
were not normally distributed (Shapiro-Wilk test), we applied the
Mann-Whitney U Test.
⇒ Results let us assume, we can measure over different languages
with the same content.
Rauschenberger et al. (2018)
Rauschenberger (MPI) Twitter: Rauschii 22 / 22 14th May 2020, at Intuit 22 / 28
26. Screening of Dyslexia Results
Is it possible to predict risk of dyslexia
based on language-independent
auditory and visual content
using a game and machine learning
for different languages?
Rauschenberger (MPI) Twitter: Rauschii 23 / 23 14th May 2020, at Intuit 23 / 28
27. Screening of Dyslexia Results
Prediction Results
Clas. Data set Feat. Recall Precision F1 Accuracy
RF DE 5 0.77 0.78 0.75 0.74
RFW DE 5 0.75 0.75 0.74 0.73
Baseline DE 0.60 0.37 0.46 0.50
ETC ES 20 0.76 0.76 0.75 0.69
RF ES 5 0.74 0.73 0.72 0.65
Baseline ES 0.68 0.46 0.55 0.50
GB ALL 20 0.66 0.65 0.65 0.61
GB ALL 5 0.64 0.64 0.63 0.59
Baseline ALL 0.63 0.40 0.49 0.50
Best results of the different classifiers, features and data sets. Results are ordered
by the best F1-score and accuracy.
Rauschenberger (MPI) Twitter: Rauschii 24 / 24 14th May 2020, at Intuit 24 / 28
28. Screening of Dyslexia Results
Prediction Discussion
Why is ALL NOT performing better than DE?
Caused by difference in the informative features
Features canceling each other out
⇒ Expected due to very different languages.
Is the F1-Score and Accuracy high enough?
Indicators are probably not as strong as reading and writing
mistakes
Early indication can equal earlier intervention.
⇒ Improve with content related to more characteristics
⇒ Therefore, we aim to optimize the recall and F1-score by
finding as many participants with dyslexia as possible.
Rauschenberger (MPI) Twitter: Rauschii 25 / 25 14th May 2020, at Intuit 25 / 28
29. Screening of Dyslexia Results
Prediction Discussion
Why is ALL NOT performing better than DE?
Caused by difference in the informative features
Features canceling each other out
⇒ Expected due to very different languages.
Is the F1-Score and Accuracy high enough?
Indicators are probably not as strong as reading and writing
mistakes
Early indication can equal earlier intervention.
⇒ Improve with content related to more characteristics
⇒ Therefore, we aim to optimize the recall and F1-score by
finding as many participants with dyslexia as possible.
Rauschenberger (MPI) Twitter: Rauschii 25 / 25 14th May 2020, at Intuit 25 / 28
30. Screening of Dyslexia Results
Prediction Discussion
Why is ALL NOT performing better than DE?
Caused by difference in the informative features
Features canceling each other out
⇒ Expected due to very different languages.
Is the F1-Score and Accuracy high enough?
Indicators are probably not as strong as reading and writing
mistakes
Early indication can equal earlier intervention.
⇒ Improve with content related to more characteristics
⇒ Therefore, we aim to optimize the recall and F1-score by
finding as many participants with dyslexia as possible.
Rauschenberger (MPI) Twitter: Rauschii 25 / 25 14th May 2020, at Intuit 25 / 28
31. Screening of Dyslexia Results
Prediction Discussion
Why is ALL NOT performing better than DE?
Caused by difference in the informative features
Features canceling each other out
⇒ Expected due to very different languages.
Is the F1-Score and Accuracy high enough?
Indicators are probably not as strong as reading and writing
mistakes
Early indication can equal earlier intervention.
⇒ Improve with content related to more characteristics
⇒ Therefore, we aim to optimize the recall and F1-score by
finding as many participants with dyslexia as possible.
Rauschenberger (MPI) Twitter: Rauschii 25 / 25 14th May 2020, at Intuit 25 / 28
32. Screening of Dyslexia Conclusion
Conclusions
Screening Risk of Dyslexia through a Web-Game using Language-Independent Content
and Machine Learning.
1 To the best of our knowledge this is the first time that risk of
dyslexia is screened using a language-independent content
web-based game and machine-learning.
2 Our method yields an accuracy of 0.74 for German and 0.69 for
Spanish as well as a F1-score of 0.75 for German and 0.75 for
Spanish, using Random Forests and Extra Trees, respectively.
3 However, different models are needed for the prediction in
different languages, something that in retrospect made sense.
Rauschenberger (MPI) Twitter: Rauschii 26 / 26 14th May 2020, at Intuit 26 / 28
33. Screening of Dyslexia Conclusion
Conclusions
Screening Risk of Dyslexia through a Web-Game using Language-Independent Content
and Machine Learning.
1 To the best of our knowledge this is the first time that risk of
dyslexia is screened using a language-independent content
web-based game and machine-learning.
2 Our method yields an accuracy of 0.74 for German and 0.69 for
Spanish as well as a F1-score of 0.75 for German and 0.75 for
Spanish, using Random Forests and Extra Trees, respectively.
3 However, different models are needed for the prediction in
different languages, something that in retrospect made sense.
Rauschenberger (MPI) Twitter: Rauschii 26 / 26 14th May 2020, at Intuit 26 / 28
34. Screening of Dyslexia Conclusion
Conclusions
Screening Risk of Dyslexia through a Web-Game using Language-Independent Content
and Machine Learning.
1 To the best of our knowledge this is the first time that risk of
dyslexia is screened using a language-independent content
web-based game and machine-learning.
2 Our method yields an accuracy of 0.74 for German and 0.69 for
Spanish as well as a F1-score of 0.75 for German and 0.75 for
Spanish, using Random Forests and Extra Trees, respectively.
3 However, different models are needed for the prediction in
different languages, something that in retrospect made sense.
Rauschenberger (MPI) Twitter: Rauschii 26 / 26 14th May 2020, at Intuit 26 / 28
35. Take Away
Web Accessibility - Technologies for Dyslexia
There is a lot of Technology ...
Assisted Reading for Dyslexia
Assisted Writing for Dyslexia
Dyslexia Screening
Dyslexia Intervention
Rauschenberger et al. (2019b)
https://link.springer.com/chapter/10.1007/978-1-4471-7440-0_31
Rauschenberger (MPI) Twitter: Rauschii 27 / 27 14th May 2020, at Intuit 27 / 28
37. References
American Psychiatric Association (2013). Diagnostic and Statistical Manual of Mental
Disorders. American Psychiatric Association, London, England.
Asvestopoulou, T., Manousaki, V., Psistakis, A., Smyrnakis, I., Andreadakis, V.,
Aslanides, I. M., and Papadopouli, M. (2019). Dyslexml: Screening tool for dyslexia
using machine learning.
British Dyslexia Association (2018). Dyslexia Style Guide 2018.
Catts, H. W., McIlraith, A., Bridges, M. S., and Nielsen, D. C. (2017). Viewing a
phonological deficit within a multifactorial model of dyslexia. Reading and Writing,
30(3):613–629.
De Zubicaray, G. and Schiller, N. O. (2018). The Oxford handbook of neurolinguistics.
Oxford University Press, New York, NY.
Paulesu, E., Danelli, L., and Berlingeri, M. (2014). Reading the dyslexic brain: multiple
dysfunctional routes revealed by a new meta-analysis of PET and fMRI activation
studies. Frontiers in human neuroscience, 8:830.
Paz-Alonso, P. M., Oliver, M., Lerma-Usabiaga, G., Caballero-Gaudes, C., Quiñones, I.,
Suárez-Coalla, P., Duñabeitia, J. A., Cuetos, F., and Carreiras, M. (2018). Neural
correlates of phonological, orthographic and semantic reading processing in dyslexia.
NeuroImage. Clinical, 20:433–447.
Rauschenberger, M., Füchsel, S., Rello, L., Bayarri, C., and Thomaschewski, J. (2015).
Exercises for German-speaking children with dyslexia. In Human-Computer
Interaction–INTERACT 2015, pages 445–452, Bamberg, Germany.
Rauschenberger (MPI) Twitter: Rauschii 28 / 28 14th May 2020, at Intuit 28 / 28
38. References
Rauschenberger, M., Füchsel, S., Rello, L., and Thomaschewski, J. (2017). DysList
German resource: A language resource of German errors written by children with
dyslexia. https://zenodo.org/record/809801#.XOVWRFMzYWo. [Online, accessed
06-June-2019].
Rauschenberger, M., Rello, L., and Baeza-Yates, R. (2019a). Predicting Dyslexia
through a Web-Game using Language-Independent Content (in progress. In in
progress, N.N. in progress.
Rauschenberger, M., Rello, L., and Baeza-Yates, R. (2019b). Technologies for Dyslexia.
In Yesilada, Y. and Harper, S., editors, Web Accessibility Book, volume 1, pages
603–627. Springer-Verlag London, London, 2 edition.
Rauschenberger, M., Rello, L., Baeza-Yates, R., and Bigham, J. P. (2018). Towards
language independent detection of dyslexia with a web-based game. In W4A ’18: The
Internet of Accessible Things, pages 4–6, Lyon, France. ACM.
Rauschenberger, M., Rello, L., Füchsel, S., and Thomaschewski, J. (2016). A language
resource of German errors written by children with dyslexia. In Proceedings of the
Tenth International Conference on Language Resources and Evaluation (LREC 2016),
Paris, France. European Language Resources Association (ELRA).
Rello, L. and Baeza-Yates, R. (2016). The Effect of Font Type on Screen Readability by
People with Dyslexia. ACM Transactions on Accessible Computing, 8(4):1–33.
Rello, L. and Baeza-Yates, R. (2017). How to present more readable text for people with
dyslexia. Universal Access in the Information Society, 16(1):29–49.
Rauschenberger (MPI) Twitter: Rauschii 28 / 28 14th May 2020, at Intuit 28 / 28
39. References
Rello, L., Baeza-Yates, R., and Llisterri, J. (2016). A resource of errors written in
Spanish by people with dyslexia and its linguistic, phonetic and visual analysis.
Language Resources and Evaluation, 51(2):1–30.
Reynolds, L. and Wu, S. (2018). "I’m Never Happy with What I Write": Challenges and
Strategies of People with Dyslexia on Social Media. In Proceedings of the 12th
International Conferenceon Web and Social Media, page 280. The AAAI Press, Palo
Alto, California USA.
Tamboer, P., Vorst, H. C. M., Ghebreab, S., and Scholte, H. S. (2016). Machine
learning and dyslexia: Classification of individual structural neuro-imaging scans of
students with and without dyslexia. NeuroImage. Clinical, 11:508–514.
Rauschenberger (MPI) Twitter: Rauschii 28 / 28 14th May 2020, at Intuit 28 / 28