Presentation of Tom Gurion's master thesis project from the Music and Technology program, Bar-Ilan University.

1. Audio-Only Augmented
Reality System for Social
Interaction
TOM GURION1 AND NORI JACOBY1,2
1BAR-ILAN UNIVERSITY, RAMAT-GAN
2HEBREW UNIVERSITY, JERUSALEM

2. Augmented Reality
Google Glasses
www.google.com/glass/
Decolabs
http://www.decolabs.com/
Wikitude
http://www.wikitude.com/

3. Augmented reality in
the audio domain?

4. Introduction and framework:
Interactive music systems
ADA: The Intelligent Space (2002)
http://specs.upf.edu/installation/547
Reactable (2007)
http://www.reactable.com/

5. Introduction and framework:
Interactive music systems
AutoRap by Smule (2012)
http://www.smule.com/autorap
RjDj – Sonic Experiences (2008)
http://www.smule.com/autorap

6. Introduction and
framework:
Social context
“A flash mob is a group of
people who assemble
suddenly in a public place,
perform an unusual and
seemingly pointless act for a
brief time, then quickly
disperse, often for the
purposes of entertainment,
satire, and artistic
expression. Flash mobs are
organized via
telecommunications, social
media, or viral emails.”
(Wikipedia)

7. Introduction and
framework:
Social context
“Silent disco has become a
common name for
a disco where people dance
to music listened to on
wireless headphones.
Rather than using a speaker
system, music is broadcast
via an FM-transmitter with
the signal being picked up
by wireless headphone
receivers worn by the
participants.”
(Wikipedia)

8. Research targets
Propose and implement an audio-only augmented reality system for
social interaction.
Evaluate the system in the context of silent disco party.
More generally, explore the potential of new ways of music consumption
through mobile devices.

9. System Description

10. Overview
The six balloons bundles Bluetooth beacon

11. Overview
App Screenshot The partyApplication page in Google Play

12. Demonstration
http://www.youtube.com/watch?v=2kJoeD2iWBA

13. Our positioning requirements
What we need:
Accuracy of about one meter
Mobile
No infrastructure
What we don’t need:
Exact positioning
The solution:
Bluetooth Based Relative Positioning system

14. Bluetooth discovery
routine
Repeatedly search for
Bluetooth beacons.
Extract received signal
strength indicator (RSSI)
from the Bluetooth device
discovery and use it as an
estimation of the distance
between the user and the
beacon.
Send the device address
and RSSI value to Pure
Data patch using libpd API
(Brinkmann, 2011).

15. Player patch
Each one of the six beacons
correspond to pre defined
sound zone player.
The patch receives RSSI
values from the app and
route them to sound zone
player according to
Bluetooth address.
Each sound zone uses the
RSSI value differently to
manipulate the music in
real-time (E.g. volume,
music filtration, etc.)

16. Evaluation: preliminary results

17. Experiment design
Participants were randomly
assigned to two groups.
Both groups started the
experiment together.
In the control blocks
participants heard pre
composed music based on
the music of the interactive
system.
They were informed that the
experiment consists of
interactive and control
segments.
Group A consists of 8 participants (4 females and 4 males) with mean age of 36.7
(s.d = 12.3); group B consists of 10 participants (3 females and 7 males) with
mean age of 29.6 (s.d = 10.2). Participants had a diverse musical background with
4.7 mean years of musical training (s.d = 5.2).

18. Measurements
Surveys
Each participant filled pre/post party surveys that included questions
regarding their musical background and preferences as well as system
evaluation feedback.
Bluetooth discoveries count
Counting the number of Bluetooth device discoveries made by the
participants’ phones during the interactive and the control blocks. In
order to eliminate edge effects, we analyzed only the two middle blocks
of the experiment.
Positioning tracking
Video capturing of the party from above the dance floor. Analyzed offline.

19. Results: Surveys
We found that using the
interactive system
participants tends to dance
less with people they knew
in advance.
Participants self-reported
significantly higher levels of
movement using the system.
We didn’t find supportive
results to show that
participants dance more with
strangers.
Dancing with known people: paired t-test, t(14) = -2.5, p_value = 0.01
Changing location in space: paired t-test, t(15) = 3.9, p_value < 0.01

20. Results: Bluetooth
discoveries count
We found higher counts
during the interactive blocks
of the party compared with
the control blocks.
paired t-test, t(16) = 1.7, p_value = 0.06, non-significant

21. Video tracking
Tracking participants
manually in Pure Data to
generate positioning tables.

22. Results:
Positioning tracking
We didn’t find significant
difference between the
position in the interactive
and control blocks. However
position was indicative to the
social relations between
participants!
We asked participants for
their social relationships to
other participants.
We compared the resulted
map with video correlation
index.
The results shows that the p_value < 0.001

23. Conclusions
Our system demonstrate a simple way to use Bluetooth technology for
relative indoor positioning.
Our preliminary results demonstrate the potential for audio-only
augmented reality to significantly enrich the experience of music
consumption and social interaction.
We show that the social implications of the system can be validated in a
controlled experiment.
We show that positioning tracking can be used to obtain social
relationships within large group of people.

24. Future research
We will use the devices accelerometers to asses entrainment.
Larger and more comprehensive experiments with a different design.

25. Thank you!