Video and slides synchronized, mp3 and slide download available at URL https://bit.ly/2EaO0Iv. Adam Berenzweig talks about brain-computer interfaces, neuromuscular interfaces, and other biosensing techniques that can eliminate the need for physical controllers. Berenzweig also approaches the emerging field of neural interaction design. Filmed at qconsf.com. Adam Berenzweig is the Director of R&D at CTRL-Labs, building the world's first practical, non-invasive neural interface. Previously he was the founding CTO of Clarifai, an early pioneer in deep learning and image recognition. From 2003 to 2013 he was a software engineer at Google, where he built the music recommender for Google Play Music, and worked on Photo Search and Google News.

1. Rethinking HCI
with Neural Interfaces
@CTRLlabsco
Adam Berenzweig, Director of R&D, CTRL-labs
@madadam | adam@ctrl-labs.com

2. InfoQ.com: News & Community Site
• Over 1,000,000 software developers, architects and CTOs read the site world-
wide every month
• 250,000 senior developers subscribe to our weekly newsletter
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• 2 dedicated podcast channels: The InfoQ Podcast, with a focus on
Architecture and The Engineering Culture Podcast, with a focus on building
• 96 deep dives on innovative topics packed as downloadable emags and
minibooks
• Over 40 new content items per week
Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations/
hci-ctrllabsco

3. Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide
Presented at QCon San Francisco
www.qconsf.com

6. 1. A Brief History of User Interface Paradigm Shifts
2. Introduction to Neural Interfaces
3. Designing for Neural Interfaces

16. 1. A Brief History of User Interface Paradigm Shifts
2. Introduction to Neural Interfaces
3. Designing for Neural Interfaces

17. Human
output
30 bits/s
Human input
10,000,000 bits/s

19. Cortical BMI is hard

21. Surface EMG neural interface.
Biophysical signal to control signal.
Neural activity EMG Control

22. In the future you will connect your
nervous system to everything.

24. 1. A Brief History of User Interface Paradigm Shifts
2. Introduction to Neural Interfaces
3. Designing for Neural Interfaces

25. Neural Interaction Design Challenges
1. Robustness & Reliability
2. Novel Interactions
3. Ergonomics
a. Touchless Control vs. Spatial Computing
4. Learnability & Customization
a. Fun to learn, and it learns you.
5. Fit to Environment
6. Props

27. Neural Interaction Design Challenges
1. 3- and 6-DOF navigation
2. Activation, “wake-word” & mode switching
3. Simultaneous continuous & discrete control
4. Hand-object interaction in XR.
5. Text Input!!

28. Ergonomics & Comfort
1. Smaller is better -- motionless is best?
2. Anatomically informed
3. Variation is essential for comfort.
4. Force: brief or light.

30. Hand-Object Interaction in XR
1. Grasping, manipulating.
2. Spatial Computing vs.
Touchless Control, i.e. direct
vs indirect.
3. Haptics & feedback.
Use your arm, or use The Force?

31. Novel Interactions
Co-evolution of software and
controllers.
Control these with an xbox
controller… or your hand?

32. Novel Interactions
Co-evolution of software and
controllers.
Control these with an xbox
controller… or your hand?

33. Text Input!!
1. One-handed vs two
2. QWERTY-or-not
3. Visual, audio feedback
4. Swipe, chords, …?
We can do better than this...

34. Props
1. Constrain hand shape &
muscle activation
2. Something to push against
3. Feels cool
4. Optional scaffolding?

36. https://www.reddit.com/r/gaming/comments/84wjsk/inverted_mouse/

40. 2019 2020
…

41. Hardware
SDK & API
CTRL-kit developer platform.
@CTRLlabsco
ctrl-labs.com

42. Questions?
@CTRLlabsco
@madadam
ctrl-labs.com
adam@ctrl-labs.com

43. [Extra slides]

44. Neurocontrol: EMG → motor neuron activity

45. Tool / object control
Spoken language
The richest signals are at the forearm.

46. Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations/
hci-ctrllabsco