Presentation at eAccess-12 (#eAccess12) on uKinect and sign recognition systems by Prof Jonathan Hassell, co-lead of uKinect project (www.ukinect.co.uk)

1. The Signed Internet – Gesture Control
@jonhassell #eAccess12

2. Speech recognition is cool

3. Speech translation is cooler

4. The challenge we were set
“Communication is a basic human right.
Communication is the means through which we control our existence.
It is the way we make friends and build relationships.
It is the way we become independent and make choices.
It is the way we learn” (BILD, 2007).
• many scenarios exist where learners with disabilities lack independence due
to an inability to communicate by speech or due to lack of motor control
• communication may involve having to write down or manually input
information into a device
• may be manageable for communicating simple concrete things to people
who are physically close
• for more sophisticated communication it is slow, inadequate and ultimately
disabling
TechDis, BIS, TSB SBRI ‘Making Waves’ competition

5. Vision
“If signs and gestures can be
easily learned, recognized
and converted to digital data,
a whole new world of
opportunity opens up.”

6. The ideal

7. The “breakthrough”
• http://www.fastcoexist.com/1680081/a-device-
to-translate-sign-language-into-speech and
http://www.uh.edu/news-
events/Fulbright/2012/may/0529MyVoice.php

8. Some“breakthroughs”
The progress in Aberdeen
• The PSLT team apologised for not being able to make it today…
• Check out: http://www.pslt.org/ for further details

9. It takes time… – cf. Siri
Speech recognition was able to The intelligence has been worked on
understand digits in the 50s… for at least 10 years

10. The problem
We had
6 months…

11. Our approach
Great Technology
User needs innovations
possibilities
User-focused innovation

12. The way ahead
Find a breakthrough
technology enabler

13. The way ahead
Find people
who can
benefit from
each of your
steps…

14. Our target audiences
Young people who use Makaton signs to help
them with their communication

15. Why choose Makaton?
BSL Makaton
Thousands of signs (> 21,000) Hundreds of signs
Individual sign vocab (> 5,000) Individual sign vocab < 200
Long sequences of signs 1 or 2 sign sequences
Sign quality fairly uniform Sign quality very variable, plus
personal (idiosyncratic) signs
Many competing teams Very few competing teams

16. User-Centred Dev approach
Engagement
through a
character and
humour…
Inspiration:
Talking Tom

17. Proof of concept solution
Play the video at: http://vimeo.com/32022176

18. Phase Two

19. Users and contexts of use
Users with comms
difficulties Supporters of these users:
through LDs, Autism, colleagues, teachers,
stroke carers, parents
Signing
e-Learning game
Education Employment Independent
Living

21. The potential of reverse inclusion: OXO Good Grips
• Well-known pioneer of Inclusive Design
in the USA
• Sam Farber’s wife, a keen cook,
suffered from arthritis
“Why do ordinary kitchen tools hurt
your hands?”
• First 15 products launched in 1990
• Sales growth over 35% per year from
1991 to 2002
• The line has now grown to over 500
products
• Over 100 design awards received
Centre for Business Innovation
“Connected Communities, helping you Do more with - 21 - Engineering Design Centre
Less”
© 2011 Centre for Business Innovation Ltd

22. Which brings us back to Siri
again… :-)

23. Find out more at:
http://www.ukinect.co.uk

24. Innovation
Training &
support for
BS8878
Standards
Strategy &
research
www.hassellinclusion.com

25. Contact us
e: jonathan@hassellinclusion.com
t: @jonhassell
w: www.hassellinclusion.com
With thanks to: TechDis, BIS
and the Technology Strategy Board