SlideShare a Scribd company logo
1 of 64
Daniel Roggen
2011
Wearable Computing
Part II
Sensors
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Physical Social
Mental
Mental
ā€¢ Emotion awareness
ā€“ Own/Others
ā€“ Sadness, joy
ā€“ Depression
ā€¢ Cognitive awareness
ā€“ Cognitive load
ā€“ Attention, concentration
ā€“ Stress
ā€¢ ā€¦
Social
ā€¢ Social interactions
ā€“ Detect known people
ā€¢ Social network
ā€“ Information exchange
ā€“ Optimization of organizations
ā€¢ Crowd / collective behavior
ā€¢ ā€¦
Dimensions of context
Physical
ā€¢ User location
ā€“ Absolute, relative
ā€¢ User activity
ā€“ Manipulative gestures,
pointing movements, modes
of locomotion, posture,
composite and hierarchical
activities
ā€¢ ā€¦
Environment
ā€¢ Map of surrounding services
ā€¢ Environment characteristics
ā€“ Temperature, light, humidity
ā€¢ Radio fingerprints
ā€¢ ā€¦
Environment
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Sensing context
ā€¢ Sensors are needed to infer the userā€™s context
ā€¢ Ā« Software sensors Ā»
ā€“ SMS on mobile phones: e.g. used to infer friend network
ā€“ Email reception: e.g. work / home presence
ā€“ Calendar: e.g. location, work activities
ā€“ ā€¦
ā€¢ Hardware sensors
ā€“ (the typical definition of sensors in EE ļŠ)
ā€“ Accelerometers: e.g. gesture recognition
ā€“ RFID tags: e.g. object detection
ā€“ Reed switch e.g. door/window open/closed
ā€“ GPS: e.g. outdoor location
ā€“ WIFI fingerprints: e.g. indoor location
ā€“ ā€¦
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
There is no Ā« Drink Sensor Ā» ļŠ
ā€¢ Simple sensors (e.g. RFID) can provide a "binary" information
ā€“ Presence (e.g. RFID, Proximity infrared sensors)
ā€“ Movement (e.g. ADXL345 accelerometer ā€˜activity/inactivity pinā€™)
ā€“ Fall (e.g. ADXL345 accelerometer ā€˜freefall pinā€™)
ā€¢ But in general Ā« activity-X sensor Ā» does not exist
ā€“ Sensor data must be interpreted
ā€“ Multiple sensors must be correlated (data fusion)
ā€“ Several factors influence the sensor data
ā€¢ Drinking while standing: the arm reaches the object then the mouth
ā€¢ Drinking while walking: the arm moves, and also the whole body
ā€¢ Context is inferring from the sensor data with
ā€“ Signal processing
ā€“ Machine learning
ā€“ Reasoning
ā€¢ Can be integrated into a Ā« sensor node Ā» or Ā« smart sensor Ā»
ā€“ Sensor chip + data processing in a device
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.comRoggen et al., Collecting complex activity datasets in highly rich networked sensor environments, Proc INSS, 2010
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Accelerometer
ā€¢ The most common sensor in activity recognition
ā€¢ Used in mobile phones to rotate screen
ā€¢ Nintendoo Wii
ā€¢ Highly miniaturized with MEMS technology
[1] ADXL335 datasheet, Analog devices
[2] http://www.silicondesigns.com/tech.html
[2]
ā€¢ ADXL335 (analog output)
ā€“ 3D accelerometer
ā€“ 4x4 mm
ā€“ Vin: 1.8-3.6V
ā€“ I: 350 uA
ā€“ Analog output: 300mV/g
ā€“ <4$/unit
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Accelerometer
ā€¢ Information about static postures and movements
ā€¢ Static acceleration
ā€“ Without user movement (or very slow)
ā€“ Earth gravity projected onto accelerometer coordinates
ā€“ Senses limb angle
Ī±
Ax
r
Az
Ī±
1g
ā€¢ Dynamic acceleration
ā€“ Two components: limb acceleration and earth gravity
Ax
Ay
Az
1g
Gesture in x-z plane Gesture in x-y plane
ā€¢ Challenges:
ā€“ On-body rotation
ā€¢ use acceleration magnitude!
ā€“ Plane of gesture
ā€¢ Different signals!
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Walking, running, jumping
Figo, Diniz, Ferreira, Cardoso. Preprocessing techniques for context recognition from accelerometer data, Pers Ubiquit Comput, 14:645ā€“662, 2010
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Car manufacturing activities
Data from Zappi et al, Activity recognition from on-body sensors: accuracy-power trade-off by dynamic sensor selection, EWSN, 2008
Dataset available at: http://www.wearable.ethz.ch/resources/Dataset
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Swim styles
Delphin
Kraul
Brust
RĆ¼cken
(jeweils zwei Bahnen)
BƤchlin, Fƶrster, Trƶster, SwimMaster: A wearable assistant for swimmer, Ubicomp, 2009
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Swim styles
BƤchlin, Fƶrster, Trƶster, SwimMaster: A wearable assistant for swimmer, Ubicomp, 2009
Armstrokes
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
OPPORTUNITY Dataset
ā€¢ Appartment flat, early morning activities
"Activity of daily living" run
ā€¢ Loose high level instructions
ā€¢ Primitives to high-level activities
ā€¢ 5 repetitions
ā€¢ 12 subjects
ā€¢ ~20mn / run
Drill run
ā€¢ Scripted sequence
ā€¢ Only activity primitives
ā€¢ 20 repetitions
ā€¢ 12 subjects
ā€¢ ~30mn / run
http://vimeo.com/8704668
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Gyroscope
ā€¢ Measures rate of turn (degrees / sec)
ā€“ Recently introduced in mobile phones (iPhone 4)
ā€“ Camera image stabilization
ā€“ Augmented reality (with accelerometer and compass as IMU)
ā€¢ Principle:
ā€“ MEMS vibrating structure
ā€“ Coriolis force displaces masses
ā€“ Change in capacitance
ā€¢ Suited to
ā€“ measure limb rotation
ā€“ heading change
ā€“ step length
[1] http://www.findmems.com/wikimems-learn/introduction-to-mems-gyroscopes
[1]
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Detection of snowboard turns with a gyro [1]
[1] Holleczek et al., Recognizing Turns and Other Snowboarding Activities with a Gyroscope, ISWC, 2010
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Inertial measurement unit
ā€¢ Combination of
ā€“ 3D accelerometer
ā€“ 3D compass
ā€“ 3D gyroscope
ā€¢ Provides device orientation in earth reference frame
ā€“ Euler angles
ā€“ Quaternions
[1] Xsens MT9 orientation sensor
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Inertial measurement unit: principle
ā€¢ [1]: Kalman filter to estimate 3DoF orientation
ā€“ Accelerometer and compass compensate drift in integration of gyroscope
ā€“ "Attitude and heading reference system"
ā€“ Accelerometer stabilizes attitude: dynamic acceleration is zero on average (earth gravity)
ā€“ Compass stabilizes heading (careful with magnetic disturbances, metallic objects!)
[1] MTi and MTx User Manual and Technical Documentation, Xsens, 2009
[2] Torres, Flynn, Angove, Murphy, Mathuna, Motion Tracking Algorithms for Inertial Measurement, BodyNets, 2007
[2]
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Inertial measurement unit
ā€¢ One IMU per body segment for body-model reconstruction
[1] Xsens MVN - Inertial Motion Capture
[2] Stiefmeier et al, Wearable Activity Tracking in Car Manufacturing, PCM, 2008
[1]
[2]
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Location awareness in wearables
Augmented Reality Annotations
Uratani et al., Wearable Augmented Reality System
with Annotation Visualization, CREST, 2004
Virtual object in physical world
Tinmith AR, University South Australia
Modeling transportation, likely routes, significant places...
Patterson, Inferring High-Level Behavior from Low-Level Sensors, Ubicomp 2003
Ashbrook, Using GPS to Learn Significant Location, Personal and Ubiquitous Computing 7(5)
Location-based activity
segmentation
Stiefmeier, Ogris, ETHZ, Uni Passau
Location-Aware Computing
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Location awareness
Hightower, Borriello, Location systems for ubiquitous computing. IEEE Computer (2001) 57ā€“
Localization beacons
IR, Bluetooth
Existing radios
Wifi, GSM
Indoor App. Specific
UWB (Ubisense)
Video tracking
+ Large area coverage
+ ā€œSymbolicā€ localization (room#) + Indoor, deployment
+ High accuracy + Unobtrusive
- Urban canyons, indoor - Wireless map
- Occlusion, privacy
- Environment instrumentation
- Instrumentation, short range
Ideally, for wearable computing:
ā€¢ Indoor and outdoor use
ā€¢ No instrumented environment
ā€¢ No a-priori knowledge / map
ā€¢ Anytime, anywhere
ā€¢ Low-power
ā€¢ Small
Satellite-systems
GPS
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
GPS revisited
ā€¢ Simultaneous measurement of time-of
flight (phase) from 4 satellites
ā€¢ Weak signal: -160dBW, 1575MHz L1
ā€¢ For outdoors....
ā€¢ ... but some indoor capability [1]
ā€¢ Uses lots of energy
[1] KjƦrgaard et al., Indoor Positioning Using GPS Revisited, Pervasive, 2010
[2] http://en.wikipedia.org/wiki/GNSS_positioning_calculation
[1]
[2]
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Radio fingerprints
ā€¢ Radio fingerprint quite unique for any location:
ā€“ Visible radios (Wifi MAC address, GSM towers)
ā€“ Signal strength
ā€¢ APIs for absolute localization from radio fingerprints:
ā€“ http://www.opencellid.org/ (cellphone towers)
ā€“ Google geolocation API
ā€¢ In principle not limited to radio:
ā€“ information also in sound, visual patterns, etc...
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Dead reckoning
ā€¢ Historically used by sailors: speed estimation and compass
ā€¢ With wearable sensors [1]:
ā€“ Compass to estimate heading
ā€“ Count steps (pedometer, accelerometer); integrate acceleration over single steps
[1] Randell, Djiallis, Muller, Personal Position Measurement Using Dead Reckoning, ISWC, 2003
ā€¢ Quite accurate (~10%) with a correct motion model
ā€“ (e.g. compass must be parallel to ground)
ā€¢ Integration of errors lead to "closing-the-loop" problem
ā€“ Identify already visited points with additional sensors and "close the loop"
ā€“ Multiple sensors and Kalman filter
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Dead reckoning with vision (optical flow)
Forward motion Rotation Side step
ā€¢ Typical motion pattern result in characteristic optical flows
Dead-reckoning from optical flow:
1. Compute optical flow
2. Derive corresponding camera movement (egomotion)
3. Integrate egomotion in a map
[1] Roggen et al, Mapping by seeing: wearable vision-based dead-reckoning, and closing the loop, EuroSSC 2007
Slow Fast
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
ā€¢ Straight walk: 5.5m
ā€¢ Estimated distance: 5.17m
ā€¢ Relative error: 6% Speed profile
Path integration (outdoor)
ā€¢ Square walk (10m sides)
ā€¢ Distance: 38.8m
ā€¢ Estimated distance: 43.4m
ā€¢ Relative error: 12%
ā€¢ Challenge: high rotation speeds (30Ā°/s-90Ā°/s)
ā€¢ h=145cm, Ī±=48Ā°
ā€¢ Chest-placed camera
Footsteps
[1] Roggen et al, Mapping by seeing: wearable vision-based dead-reckoning, and closing the loop, EuroSSC 2007
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Closing the loop
ā€¢ Errors accumulate during path integration
ā€¢ Coinciding start-end point of trajectory do not map on
same location
A-posteriori correction of motion vectors
1. Recognize identical locations A, B
2. Correct motion vectors so A matches B
ā†’ Idea: closing the loop
ā„¦cl=f(ā„¦)ā„¦
[1] Roggen et al, Mapping by seeing: wearable vision-based dead-reckoning, and closing the loop, EuroSSC 2007
ā€¢ 6-12% distance error
ā€¢ Underestimation of rotation
ā€“ Note: camera only! With a gyro can be improved
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Estimating proximity from passive fingerprints: sound
ā€¢ Proximity information is important for many applications!
[1] Wirz et al., A wearable, ambient sound-based approach for infrastructureless fuzzy proximity estimation, ISWC 2010
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Estimating proximity from passive fingerprints: Wifi
[1] Cugia, Wirz, ETH Tech Rep, 2011
ā€¢ Devices A, B scan Wifi (MAC addresses)
ā€¢ Compute fingerprint similarity
ā€“ E.g. Jaccard index: normalized number
MAC address seen by both A and B
Sensor number Hears WIFI signals
S0 W2 W6 W7
S1 W1 W3 W4 W5
S2 W2 W3 W4 W5 W6 W7
S3 W0 W1 W2 W3 W4 W5
S4 W0 W1 W2 W3 W4 W5
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
From proximity to topology
ā€¢ Proximity can be used to find node topology
ā€“ Binary case: "sense"/"not sense" (e.g. communication in/out-of range)
ā€“ Or distance estimate
ā€¢ Convert proximity to topology
ā€“ Multidimensional scaling [1,2]
ā€¢ "Anchor nodes" (e.g. with GPS) can "ground" the topology
[1] Koo, Cha, Autonomous Construction of a WiFi Access Point Map Using Multidimensional Scaling, Pervasive, 2011
[2] Chan&So, Efficient Weighted Multidimensional Scaling for Wireless Sensor Network Localization, IEEE Tr Sig Proc, 57(11), 2009
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Active capacitive sensing [1,2]
ā€¢ Challenge: sensing inner body function
ā€“ E.g. drinking, heart rate, respiration, etc.
ā€¢ Principle:
ā€“ Capacitor with human body as dielectric
ā€“ Body function affect dielectric: muscle contractions, joint motions, air
entering the lungs, drinking
ā€“ Readout using a Colpitts (LC) oscillator
ā€“ Suitable for textile integration!
[1] Cheng et al., Active Capacitive Sensing: Exploring a New Wearable Sensing Modality for Activity Recognition, Pervasive, 2011
[2] Lukowicz et al., On-Body Sensing: From Gesture-Based Input to Activity-Driven Interaction, IEEE Computer Magazine
Dielectric
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Active capacitive sensing [1,2]
ā€¢ Characteristic signal for swallowing
ā€¢ Also reacts to movements (head direction, body movement)
[1] Cheng et al., Active Capacitive Sensing: Exploring a New Wearable Sensing Modality for Activity Recognition, Pervasive, 2011
[2] Lukowicz et al., On-Body Sensing: From Gesture-Based Input to Activity-Driven Interaction, IEEE Computer Magazine
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Electrooculography [1]
ā€¢ Measures E potential around the eye
ā€¢ Can identify saccades, fixation, blinks, movement patterns
ā€¢ Alternative technology: camera-based
[1] Bulling et al., Whatā€™s in the Eyes for Context-Awareness? Pervasive computing magazine, 2011
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Video on the emotional computer
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Sensing intense fear (startle event)
ā€¢ Skin conductance: Galvanic Skin Response = Electrodermal Activity
ā€“ Affected by Ā« internal states Ā»: emotions, phobias, arousal, stress
ā€¢ Controlled by the sympathetic nervous system (SNS) [1]
ā€“ Low frequency Ā« tonic Ā» part (0-0.05 Hz)
ā€“ Fast changing phasic component (0.05 Hz-1.5 Hz)
ā€¢ Quickly reacts to the Ā« fight or flight Ā» reflex (e.g. fear / startle event)
[1] Fuller GD. GSR History, & Physiology. San Francisco: Biofeedback Institute San Francisco; 1977.
[2] Schumm et al., Effect of Movements on the Electrodermal Response after a Startle Event, Methods of Information in Biomedicine, 2008
Wearable EDA sensors from [2]
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Experiment
ā€¢ Walking on a treadmill
ā€¢ Listening to music
ā€¢ Loud Ā« bang Ā» at unexpected moments
Schumm et al., Effect of Movements on the Electrodermal Response after a Startle Event, Methods of Information in Biomedicine, 2008
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Experiment
ā€¢ Visualization with cross-correlogram
Schumm et al., Effect of Movements on the Electrodermal Response after a Startle Event, Methods of Information in Biomedicine, 2008
EDA activation after starleā€¦
ā€¦but some challenges:
ā€¢ Physical activity
ā€¢ Background EDA activation
ā€¢ No unique response to startle
Startle events
aligned at t=0
Multimodal approach?
ā€¢ Sensing movement and EDA
ā€¢ ā€¦
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Sensing cognitive states from eye movements
[1] Hayhoe, Ballard Eye movements in natural behavior, TRENDS in Cognitive Sciences Vol.9 No.4 April 2005
[2] Heisz, Shore, More efficient scanning for familiar faces, Journal of Vision (2008) 8(1):9, 1ā€“10
[3] Bulling et al., Whatā€™s in the Eyes for Context-Awareness? Pervasive computing magazine, 2011
[1]
[1]
Familiar v.s. unfamiliar faces [2]
Gaze patterns are consciously and unconsciously
controlled. Can eye movements reveal something
about cognitive states?
ā€¢ "Eventually, analyzing the link between unconscious eye movements and cognition
might even pave the way for a new genre of pervasive computing systems that can
sense and adapt to a personā€™s cognitive context. [....] A computing system is
cognition-aware if it can sense and adapt to a personā€™s cognitive context." [3]
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Sensing cognitive states from eye movements [1]
[1] Bulling et al., Whatā€™s in the Eyes for Context-Awareness? Pervasive computing magazine, 2011
Applications:
ā€¢ Memory assistants
ā€¢ Smart computer interfaces
ā€¢ Significant change in fixation count
ā€¢ Ongoing: classify in "seen" / "unseen"
ā€¢ Challenge: outside of the lab!
ā€¢ Cognition: Memory, Stress, Reasoning, ...
ā€¢ Can a computer detect whether a picture has already been seen before?
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Group near exit
Clogged
Dense
Queuing
Group split
Lane formation
ā€¢ Capability and properties of mobile system...
ā€¢ for machine recognition of crowd behavior
ā€¢ Supports situation awareness
Social context: sensing crowd behaviors
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Social context: sensing crowd behaviors
Wirz, M.; Roggen, D.; Trƶster, G.: Decentralized Detection of Group Formations from Wearable Acceleration Sensors
Wirz et al. A Methodology towards the Detection of Collective Behavior Patterns by Means of Body-Worn Sensors, Workshop at Pervasive, 2010
Roggen et al., Recognition of crowd behavior from mobile sensors with pattern analysis and graph clustering methods, Submitted to NHM, 2011
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Integrating sensors into clothing
ā€¢ Clothing integration is the "dream" of wearable computing
ā€¢ "Normal" clothes are extremely ruggedized!
ā€¢ Challenges for smart-textiles / sensorized garments:
ā€“ Washing
ā€“ Bending
ā€“ Friction
ā€“ Integration in standard manufacturing process
ā€“ Requires sophisticated equipment!
ā€¢ Example technology:
ā€“ Kapton substrate, thin film transistor
ā€“ Thin straps woven with standard machine
Our smart textile fabrication process, Cherenack et al,
IEEE Electr. Dev. Lett, vol. no. 7, pp. 740-742, July
2010
A woven temperature sensor,
Kinkeldei et al, Proc. of the IEEE
Sensors Conference, pp 1-4 2009
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Smart-textiles
System on Textiles
ā€¢ Development of conductive fabrics for
signal transmission in wearables
ā€¢ Investigation of the electrical
performance: measuring and modeling
high-frequency properties
ā€¢ Simulation and optimization of different
fabrics and configurations
ā€¢ Interconnections
Woven fabric with
conductive fibers
Unobtrusive context recognition, healthcare (e.g. rehabilitation), sports
Routing Methods Adapted to e-Textiles. I. Locher, T. Kirstein and G. Trƶster, Proc. 37th International Symposium on
Microelectronics (IMAPS 2004), Long Beach CA, Nov. 14-18, 2004
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Smart-textiles
Textile Antennas
ā€¢ Magnetic coupled antenna
ā€¢ Signal transmission between pieces of clothing.
Textile Body Area Network
Trans-
mitter
Receiver
H
ā€¢ Bluetooth antenna
ā€¢ Signal transmission
along body
ā€¢ Communication with
near infrastructure
ā€¢ Modeling and measuring of electrical properties
ā€¢ Modeling and measuring of textile-specific properties
Design and Characterization of Purely Textile Patch Antennas. I. Locher, M. Klemm, T. Kirstein and G. Trƶster, IEEE
Transactions on Advanced Packaging, Vol. 29, No. 4, Nov. 2006, pp. 777-788
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Index
Finger
Temperature Profile Temperature Sensor
ā€¢ 74 g/m2
ā€¢ Polyester Interlining
ā€¢ Insulated copper wires
Temperature Profile Estimation with Smart Textiles. I. Locher, T. Kirstein and G. Trƶster, Proc. 1st International Scientific
Conference Ambience 05, Tampere, Finland, Sept. 19-20, 2005
Smart-textiles
Textile Temperature Sensor
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
C o n d u c tiv e t e x tile
S p a c e r ( f o a m , t e x t ile )
N o n c o n d u c t iv e t e x t ile
Capacitive textile Sensor
Electrodes embroidered with
conductive yarn on both sides
of compressible spacer
Applications:
ā€¢ Medicine (Decubitus prevention)
ā€¢ Weight measurement in car seats
Smart-textiles
Capacitive Textile Pressure Sensor
Textile Pressure Sensor for Muscle Activity and Motion Detection. J. Meyer, P. Lukowicz, G. Trƶster, ISWC 2006: Proceedings
of the 10th IEEE International Symposium on Wearable Computers, Montreux, Switzerland, 11.-14. October 2006
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Integrating sensors into clothing
ā€¢ A rapid prototyping technology [1]
[1] Harms et al., Rapid prototyping of smart garments for activity-aware applications, JAISE, 2009
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
lower body
upper body
VDRG: Velocity-Damped Resonant Generator
CDRG: Coulomb-Damped Resonant Generator
CFPG: Coulomb-Force Parameteric Generator
Optimization of Inertial Micropower Generators for Human Walking Motion. Thomas von BĆ¼ren, P. D. Mitcheson, T. C. Green, E.
M. Yeatman, A. S. Holmes, and G. Trƶster. IEEE Sensors Journal, 2005
Energy Harvesting
Mechanical Āµāˆ’Generator for On-Body Sensor Networks
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Maximum-Power-Point Tracking for On-Body Context Systems. N.B. Bharatula, J.A. Ward, P. Lukowicz, G. Trƶster, ISWC
2006: Proceedings of the 10th IEEE International Symposium on Wearable Computers
Energy Harvesting
Hybrid solar-battery power supply
Solar cell
Li-Ion battery
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Existing datasets
PlaceLab dataset
TU Munchen kitchen
Many others
Homesetting
(van Kasteren)
Long-term activity
(van Laerhoeven)
+ Real behaviors in house
+ Marker free motion capture
+ Application specific
+ Real behaviors in house
+ Free-living activities
- Small number of instances
- No wearable sensors
- Often not open
- Binary sensors
- Annotations
http://www.wearable.ethz.ch/resources/Dataset
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Collecting large scale datasets: Lessons learned
ā€¢ Appartment flat, early morning activities
"Activity of daily living" run
ā€¢ Loose high level instructions
ā€¢ Primitives to high-level activities
ā€¢ 5 repetitions
ā€¢ 12 subjects
ā€¢ ~20mn / run
Drill run
ā€¢ Scripted sequence
ā€¢ Only activity primitives
ā€¢ 20 repetitions
ā€¢ 12 subjects
ā€¢ ~30mn / run
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Wearable sensors: sense body movement (and sound)
ā€¢ Nodes: 24 (mostly IMUs, accel.)
ā€¢ Modalities: 4
ā€¢ Networks: 6
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Instrumented objects: Sense object use
ā€¢ Sensor nodes: 12 objects
ā€¢ Modalities: 2 (acceleration, gyroscope)
ā€¢ Networks: 1 (Bluetooth)
Accelerometer + gyroscope
FSRs
XSense
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Ambient sensors: sense environment interactions
ā€¢ Sensor nodes: 36
ā€¢ Modalities: 9
ā€¢ Networks: 8
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
The OPPORTUNITY dataset: key facts
Sensor rich
ā€¢ 72 sensors (28 sensors in 2.4GHz band)
ā€¢ 10 modalities
ā€¢ 15 wired and wireless systems (USB, Bluetooth, 802.15.4, custom)
ā€¢ 2.5% data loss on wireless sensors
Activity rich
ā€¢ 12 subjects, ~25 hours of total data
ā€¢ > 30'000 interaction primitives (object, environment)
ā€¢ Annotation length = 230% of dataset length
Highly collaborative effort
ā€¢ 11 days recording session, 10 experimenters
ā€¢ Subjects instrumented for 5-7 hours
ā€¢ ~10 hours of annotation effort for 30mn of data
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Challenge 1: Data recording with heterogeneous sensor networks
Integration at system level
+ Central control & monitoring
+ Synchronized data acquisition
- Internals of sensor systems
- Fixed real-time merge
Integration at data level
+ Independent data recorders
+ Robustness, flexibility
- Complex control & monitoring
- Offline synchronization
Obtain synchronized data streams for further processing
ā€¢ 7 computers recording sensor data
ā€“ Store data and data reception time
ā€“ Coarse NTP synchronisation
ā€“ Fine synchronisation with specific gestures (ā€œjump and clapā€)
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Challenge 2: data handling after recording
Burst equalization w/ streaming sensors Missing data represented as NaNs
Stream alignment to video footage
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Challenge 3: flexible activity annotations, at all levels
~60
instances
~30'000
instances
Solution: annotation on multiple tracks, hand-action-object representation
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Some lessons learned and best practices
ā€¢ Complex recordings will see heterogeneous WSN
ā€“ System level integration is desiredā€¦
ā€“ ā€¦ however data level integration may be sufficient
ā€¢ Developing custom tools can be beneficial
ā€“ Signal/video alignment, annotation
ā€“ Now: students can do the work
ā€¢ Plan for a repository for raw data
ā€“ Required already before data can be stored in a database
ā€“ Can be huge!
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Some lessons learned and best practices
ā€¢ Test infrastructure locally
ā€“ Placement of antennas
ā€“ Integration & optimization of the wireless links
ā€¢ Expect, plan for, and accept data loss or failures
ā€“ Independent systems
ā€“ Strategy: ignore problem, fix&restart, fix&continue
ā€¢ Data integrity checks
ā€“ Tradeoff: frequency / time of test-subjects
ā€¢ Develop a rapid on-body sensor deployment solution
ā€“ Clothing attached/integrated sensors
ā€“ Reproducible placement
ā€“ More convenient
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Some lessons learned and best practices
ā€¢ Avoid wireless links altogether :)
ā€“ Use local data storage
ā€“ Wireless only for synchronization
ā€¢ Do not underestimate logistics
ā€“ 10 people during 11 days
ā€“ Room rented for 11 days, 3 days setup
ā€“ Setup: 3 days
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
ā€¢ http://www.intersense.com
ā€¢ Nintendo WiiMote
ā€¢ Open source BlueSense
acceleration/gyro sensors
(http://www.wearable.ethz.
ch/resources/educationkit)
Some readily available sensors
ā€¢ http://www.phidgets.com/
ā€¢ http://www.sparkfun.com/
ā€¢ LilyPad
(http://hlt.media.mit.edu/?
p=34)
ā€¢ http://www.xsens.com/
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
Summary
ā€¢ User requirements
ā€“ Small / unobtrusive
ā€“ Invisible to the outside
ā€“ Comfortable
ā€“ Privacy issues
ā€¢ Context-recognition requirements
ā€“ Must be discriminative of the context to recognize
ā€“ Minimize subsequent processing (simpler modalities favored)
ā€¢ Hardware requirements
ā€“ Low-power
ā€“ Low-computational complexity
ā€¢ Usually multimodal approaches
ā€“ Multiple activities affect a sensor
ā€“ Several sensors can disambiguate the activity
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
For further reading
Context
ā€¢ Dey, Understanding and Using Context, Personal and Ubiquitous Computing,2001
Activity-aware computing
ā€¢ Lukowicz et al., On-Body Sensing: From Gesture-Based Input to Activity-Driven Interaction, IEEE Computer, 43(10), pp. 92-96, 2010
ā€¢ N. Davies, D. Siewiorek, R. Sukthankar, Special Issue: Activity-Based Computing, IEEE Pervasive Computing, 7(2), pp. 20-21, 2008
ā€¢ Stiefmeier et al, Wearable Activity Tracking in Car Manufacturing, PCM, 2008
Cognitive-affective computing
ā€¢ R. Picard, Affective computing, MIT Press, 1997
ā€¢ R. Picard, E. Vyzas, J. Healey, Toward Machine Emotional Intelligence: Analysis of Affective Physiological State, IEEE Transactions on
Pattern Analysis and Machine Intelligence, 23(10), pp. 1175-1191, 2001
ā€¢ Bulling et al., Whatā€™s in the Eyes for Context-Awareness? Pervasive computing magazine, 2011
Location-awareness
ā€¢ Hightower, Borriello, Location systems for ubiquitous computing, IEEE Computer, pp. 57-66, 2001
ā€¢ Kjaergaard, Blunck, Godsk, Toftkjaer, Christensen, Gro}nb{ae}k, Indoor Positioning Using GPS Revisited, Pervasive, 2010
ā€¢ Koo, Cha, Autonomous Construction of a WiFi Access Point Map Using Multidimensional Scaling, Pervasive, 2011
Social context
ā€¢ Wirz, Roggen, Trƶster, Decentralized Detection of Group Formations from Wearable Acceleration Sensors, Int. Conf. Social Computing,
2009
Datasets
ā€¢ Roggen et al., Collecting complex activity data sets in highly rich networked sensor environments, INSS, 2010
Context and Sensing frameworks
ā€¢ Roggen et al., Titan: An Enabling Framework for Activity-Aware ``PervasiveApps'' in Opportunistic Personal Area Networks, EURASIP
Journal on Wireless Communications and Networking, 2011
ā€¢ Kukkonen, Lagerspetz, Nurmi, Andersson, BeTelGeuse: A Platform for Gathering and Processing Situational Data, IEEE Pervasive
Computing Magazine, 8(2): 49-56, 2009
ā€¢ Fortino, Guerrieri, Bellifemine, Giannantonio, SPINE2: Developing BSN Applications on Heterogeneous Sensor Nodes, Proc. IEEE
Symposium on Industrial Embedded Systems SIES2009, 2009
ā€¢ Bannach et al., Rapid Prototyping of Activity Recognition Applications, IEEE Pervasive Computing Magazine, 7(2):22-31, 2008
ā€¢ Kurz et al., The OPPORTUNITY Framework and Data Processing Ecosystem for Opportunistic Activity and Context Recognition,
International Journal of Sensors, Wireless Communications and Control, 2011
Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com

More Related Content

What's hot

Biohackers Summit 2015 - Lifelogging, a new era of Personal Data
Biohackers Summit 2015 - Lifelogging, a new era of Personal DataBiohackers Summit 2015 - Lifelogging, a new era of Personal Data
Biohackers Summit 2015 - Lifelogging, a new era of Personal DataCathal Gurrin
Ā 
Lifelogging & Personal Data Analytics
Lifelogging & Personal Data AnalyticsLifelogging & Personal Data Analytics
Lifelogging & Personal Data AnalyticsCathal Gurrin
Ā 
Personal Big Data
Personal Big DataPersonal Big Data
Personal Big DataCathal Gurrin
Ā 
Sixth sense technology
Sixth sense technology Sixth sense technology
Sixth sense technology Pulkit Singhal
Ā 
Play, Pause, Rewind - The Era of Archived Lifetimes
Play, Pause, Rewind - The Era of Archived LifetimesPlay, Pause, Rewind - The Era of Archived Lifetimes
Play, Pause, Rewind - The Era of Archived LifetimesCathal Gurrin
Ā 
Cps innovation lab kolkata iiest
Cps innovation lab kolkata iiestCps innovation lab kolkata iiest
Cps innovation lab kolkata iiestArpan Pal
Ā 
Footstep Sensors Identify People by Gait
Footstep Sensors Identify People by GaitFootstep Sensors Identify People by Gait
Footstep Sensors Identify People by GaitAbaram Network Solutions
Ā 
[EUC2014] cODA: An Open-Source Framework to Easily Design Context-Aware Andro...
[EUC2014] cODA: An Open-Source Framework to Easily Design Context-Aware Andro...[EUC2014] cODA: An Open-Source Framework to Easily Design Context-Aware Andro...
[EUC2014] cODA: An Open-Source Framework to Easily Design Context-Aware Andro...Matteo Ferroni
Ā 
Assessment Test Framework for Collecting and Evaluating Fall - Related Data u...
Assessment Test Framework for Collecting and Evaluating Fall - Related Data u...Assessment Test Framework for Collecting and Evaluating Fall - Related Data u...
Assessment Test Framework for Collecting and Evaluating Fall - Related Data u...Martin Ebner
Ā 
BackHome: Assisting and Telemonitoring People with Disabilities
BackHome: Assisting and Telemonitoring People with DisabilitiesBackHome: Assisting and Telemonitoring People with Disabilities
BackHome: Assisting and Telemonitoring People with DisabilitiesEloisa Vargiu
Ā 
Internet Of Things
Internet Of ThingsInternet Of Things
Internet Of ThingsPiTechnologies
Ā 
Real time hand gesture recognition system for dynamic applications
Real time hand gesture recognition system for dynamic applicationsReal time hand gesture recognition system for dynamic applications
Real time hand gesture recognition system for dynamic applicationsijujournal
Ā 
Real time hand gesture recognition system for dynamic applications
Real time hand gesture recognition system for dynamic applicationsReal time hand gesture recognition system for dynamic applications
Real time hand gesture recognition system for dynamic applicationsijujournal
Ā 
Introduction to Wearable Technology
Introduction to Wearable TechnologyIntroduction to Wearable Technology
Introduction to Wearable TechnologyRain Ashford
Ā 
Sixth sense tecnology.
Sixth sense tecnology.Sixth sense tecnology.
Sixth sense tecnology.Appam Sushma
Ā 
Design of Image Projection Using Combined Approach for Tracking
Design of Image Projection Using Combined Approach for  TrackingDesign of Image Projection Using Combined Approach for  Tracking
Design of Image Projection Using Combined Approach for TrackingIJMER
Ā 
Gesture Based Interface Using Motion and Image Comparison
Gesture Based Interface Using Motion and Image ComparisonGesture Based Interface Using Motion and Image Comparison
Gesture Based Interface Using Motion and Image Comparisonijait
Ā 

What's hot (20)

Biohackers Summit 2015 - Lifelogging, a new era of Personal Data
Biohackers Summit 2015 - Lifelogging, a new era of Personal DataBiohackers Summit 2015 - Lifelogging, a new era of Personal Data
Biohackers Summit 2015 - Lifelogging, a new era of Personal Data
Ā 
Lifelogging & Personal Data Analytics
Lifelogging & Personal Data AnalyticsLifelogging & Personal Data Analytics
Lifelogging & Personal Data Analytics
Ā 
Personal Big Data
Personal Big DataPersonal Big Data
Personal Big Data
Ā 
Qs2015
Qs2015Qs2015
Qs2015
Ā 
Sixth sense technology
Sixth sense technology Sixth sense technology
Sixth sense technology
Ā 
Play, Pause, Rewind - The Era of Archived Lifetimes
Play, Pause, Rewind - The Era of Archived LifetimesPlay, Pause, Rewind - The Era of Archived Lifetimes
Play, Pause, Rewind - The Era of Archived Lifetimes
Ā 
Cps innovation lab kolkata iiest
Cps innovation lab kolkata iiestCps innovation lab kolkata iiest
Cps innovation lab kolkata iiest
Ā 
Footstep Sensors Identify People by Gait
Footstep Sensors Identify People by GaitFootstep Sensors Identify People by Gait
Footstep Sensors Identify People by Gait
Ā 
[EUC2014] cODA: An Open-Source Framework to Easily Design Context-Aware Andro...
[EUC2014] cODA: An Open-Source Framework to Easily Design Context-Aware Andro...[EUC2014] cODA: An Open-Source Framework to Easily Design Context-Aware Andro...
[EUC2014] cODA: An Open-Source Framework to Easily Design Context-Aware Andro...
Ā 
Assessment Test Framework for Collecting and Evaluating Fall - Related Data u...
Assessment Test Framework for Collecting and Evaluating Fall - Related Data u...Assessment Test Framework for Collecting and Evaluating Fall - Related Data u...
Assessment Test Framework for Collecting and Evaluating Fall - Related Data u...
Ā 
BackHome: Assisting and Telemonitoring People with Disabilities
BackHome: Assisting and Telemonitoring People with DisabilitiesBackHome: Assisting and Telemonitoring People with Disabilities
BackHome: Assisting and Telemonitoring People with Disabilities
Ā 
Posture Patch
Posture PatchPosture Patch
Posture Patch
Ā 
Sixth sense
Sixth senseSixth sense
Sixth sense
Ā 
Internet Of Things
Internet Of ThingsInternet Of Things
Internet Of Things
Ā 
Real time hand gesture recognition system for dynamic applications
Real time hand gesture recognition system for dynamic applicationsReal time hand gesture recognition system for dynamic applications
Real time hand gesture recognition system for dynamic applications
Ā 
Real time hand gesture recognition system for dynamic applications
Real time hand gesture recognition system for dynamic applicationsReal time hand gesture recognition system for dynamic applications
Real time hand gesture recognition system for dynamic applications
Ā 
Introduction to Wearable Technology
Introduction to Wearable TechnologyIntroduction to Wearable Technology
Introduction to Wearable Technology
Ā 
Sixth sense tecnology.
Sixth sense tecnology.Sixth sense tecnology.
Sixth sense tecnology.
Ā 
Design of Image Projection Using Combined Approach for Tracking
Design of Image Projection Using Combined Approach for  TrackingDesign of Image Projection Using Combined Approach for  Tracking
Design of Image Projection Using Combined Approach for Tracking
Ā 
Gesture Based Interface Using Motion and Image Comparison
Gesture Based Interface Using Motion and Image ComparisonGesture Based Interface Using Motion and Image Comparison
Gesture Based Interface Using Motion and Image Comparison
Ā 

Viewers also liked

Wearable Computing and Human Computer Interfaces
Wearable Computing and Human Computer InterfacesWearable Computing and Human Computer Interfaces
Wearable Computing and Human Computer InterfacesJeffrey Funk
Ā 
Wearable Sensors and UX Development
Wearable Sensors and UX DevelopmentWearable Sensors and UX Development
Wearable Sensors and UX DevelopmentMark Melnykowycz
Ā 
Wearable sensors and mobile applications
Wearable sensors and mobile applicationsWearable sensors and mobile applications
Wearable sensors and mobile applicationsGene Leybzon
Ā 
Trends in Sensors, Wearable Devices and IoT
Trends in Sensors, Wearable Devices and IoTTrends in Sensors, Wearable Devices and IoT
Trends in Sensors, Wearable Devices and IoTWalt Maclay
Ā 
virtual reality Barkha manral seminar on augmented reality.ppt
virtual reality Barkha manral seminar on augmented reality.pptvirtual reality Barkha manral seminar on augmented reality.ppt
virtual reality Barkha manral seminar on augmented reality.pptBarkha Manral
Ā 
DIA 2010 Presentation - Time To Make It Work
DIA 2010 Presentation -  Time To Make It WorkDIA 2010 Presentation -  Time To Make It Work
DIA 2010 Presentation - Time To Make It WorkMichael Myers
Ā 
Modify elm327 to add the hs/ms can switch
Modify elm327 to add the hs/ms can switchModify elm327 to add the hs/ms can switch
Modify elm327 to add the hs/ms can switchamysmyy
Ā 
Catalogue jeep compass 2011 autoprestige-accessoires-4x4
Catalogue jeep compass 2011 autoprestige-accessoires-4x4Catalogue jeep compass 2011 autoprestige-accessoires-4x4
Catalogue jeep compass 2011 autoprestige-accessoires-4x4autoprestige
Ā 
2010 jeep patriot installation with quicktime control
2010 jeep patriot installation with quicktime control2010 jeep patriot installation with quicktime control
2010 jeep patriot installation with quicktime controlbuttenob
Ā 
Responsive design - College Hogeschool Utrecht
Responsive design - College Hogeschool UtrechtResponsive design - College Hogeschool Utrecht
Responsive design - College Hogeschool UtrechtFloor van Riet
Ā 
Design Principles & Guidelines
Design Principles & GuidelinesDesign Principles & Guidelines
Design Principles & GuidelinesBob de Graaf
Ā 
Emotion sensitive robots
Emotion sensitive robotsEmotion sensitive robots
Emotion sensitive robotscharan malyala
Ā 
Securty Issues from 1999
Securty Issues from 1999Securty Issues from 1999
Securty Issues from 1999TomParker
Ā 
Wearable computing
Wearable computing Wearable computing
Wearable computing kdore
Ā 
Manual reparacion Jeep Compass - Patriot Limited 2007-2009_Wiper washer
Manual reparacion Jeep Compass - Patriot Limited 2007-2009_Wiper washerManual reparacion Jeep Compass - Patriot Limited 2007-2009_Wiper washer
Manual reparacion Jeep Compass - Patriot Limited 2007-2009_Wiper washerGustavo Castro
Ā 
Manual reparacion Jeep Compass - Patriot Limited 2007-2009_Sunroof
Manual reparacion Jeep Compass - Patriot Limited 2007-2009_SunroofManual reparacion Jeep Compass - Patriot Limited 2007-2009_Sunroof
Manual reparacion Jeep Compass - Patriot Limited 2007-2009_SunroofGustavo Castro
Ā 
Manual reparacion Jeep Compass - Patriot Limited 2007-2009_Cabin air filter
Manual reparacion Jeep Compass - Patriot Limited 2007-2009_Cabin air filterManual reparacion Jeep Compass - Patriot Limited 2007-2009_Cabin air filter
Manual reparacion Jeep Compass - Patriot Limited 2007-2009_Cabin air filterGustavo Castro
Ā 
Manual reparacion Jeep Compass - Patriot Limited 2007-2009_Transfer case
Manual reparacion Jeep Compass - Patriot Limited 2007-2009_Transfer caseManual reparacion Jeep Compass - Patriot Limited 2007-2009_Transfer case
Manual reparacion Jeep Compass - Patriot Limited 2007-2009_Transfer caseGustavo Castro
Ā 
Conditional Access Systems
Conditional Access SystemsConditional Access Systems
Conditional Access SystemsNamith CM
Ā 

Viewers also liked (20)

Wearable Computing and Human Computer Interfaces
Wearable Computing and Human Computer InterfacesWearable Computing and Human Computer Interfaces
Wearable Computing and Human Computer Interfaces
Ā 
Wearable Sensors and UX Development
Wearable Sensors and UX DevelopmentWearable Sensors and UX Development
Wearable Sensors and UX Development
Ā 
Wearable sensors and mobile applications
Wearable sensors and mobile applicationsWearable sensors and mobile applications
Wearable sensors and mobile applications
Ā 
Trends in Sensors, Wearable Devices and IoT
Trends in Sensors, Wearable Devices and IoTTrends in Sensors, Wearable Devices and IoT
Trends in Sensors, Wearable Devices and IoT
Ā 
virtual reality Barkha manral seminar on augmented reality.ppt
virtual reality Barkha manral seminar on augmented reality.pptvirtual reality Barkha manral seminar on augmented reality.ppt
virtual reality Barkha manral seminar on augmented reality.ppt
Ā 
DIA 2010 Presentation - Time To Make It Work
DIA 2010 Presentation -  Time To Make It WorkDIA 2010 Presentation -  Time To Make It Work
DIA 2010 Presentation - Time To Make It Work
Ā 
Modify elm327 to add the hs/ms can switch
Modify elm327 to add the hs/ms can switchModify elm327 to add the hs/ms can switch
Modify elm327 to add the hs/ms can switch
Ā 
Catalogue jeep compass 2011 autoprestige-accessoires-4x4
Catalogue jeep compass 2011 autoprestige-accessoires-4x4Catalogue jeep compass 2011 autoprestige-accessoires-4x4
Catalogue jeep compass 2011 autoprestige-accessoires-4x4
Ā 
2010 jeep patriot installation with quicktime control
2010 jeep patriot installation with quicktime control2010 jeep patriot installation with quicktime control
2010 jeep patriot installation with quicktime control
Ā 
Responsive design - College Hogeschool Utrecht
Responsive design - College Hogeschool UtrechtResponsive design - College Hogeschool Utrecht
Responsive design - College Hogeschool Utrecht
Ā 
Elm327
Elm327Elm327
Elm327
Ā 
Design Principles & Guidelines
Design Principles & GuidelinesDesign Principles & Guidelines
Design Principles & Guidelines
Ā 
Emotion sensitive robots
Emotion sensitive robotsEmotion sensitive robots
Emotion sensitive robots
Ā 
Securty Issues from 1999
Securty Issues from 1999Securty Issues from 1999
Securty Issues from 1999
Ā 
Wearable computing
Wearable computing Wearable computing
Wearable computing
Ā 
Manual reparacion Jeep Compass - Patriot Limited 2007-2009_Wiper washer
Manual reparacion Jeep Compass - Patriot Limited 2007-2009_Wiper washerManual reparacion Jeep Compass - Patriot Limited 2007-2009_Wiper washer
Manual reparacion Jeep Compass - Patriot Limited 2007-2009_Wiper washer
Ā 
Manual reparacion Jeep Compass - Patriot Limited 2007-2009_Sunroof
Manual reparacion Jeep Compass - Patriot Limited 2007-2009_SunroofManual reparacion Jeep Compass - Patriot Limited 2007-2009_Sunroof
Manual reparacion Jeep Compass - Patriot Limited 2007-2009_Sunroof
Ā 
Manual reparacion Jeep Compass - Patriot Limited 2007-2009_Cabin air filter
Manual reparacion Jeep Compass - Patriot Limited 2007-2009_Cabin air filterManual reparacion Jeep Compass - Patriot Limited 2007-2009_Cabin air filter
Manual reparacion Jeep Compass - Patriot Limited 2007-2009_Cabin air filter
Ā 
Manual reparacion Jeep Compass - Patriot Limited 2007-2009_Transfer case
Manual reparacion Jeep Compass - Patriot Limited 2007-2009_Transfer caseManual reparacion Jeep Compass - Patriot Limited 2007-2009_Transfer case
Manual reparacion Jeep Compass - Patriot Limited 2007-2009_Transfer case
Ā 
Conditional Access Systems
Conditional Access SystemsConditional Access Systems
Conditional Access Systems
Ā 

Similar to Wearable Computing - Part II: Sensors

Mobile sensing kolkata lab tac_tics2014
Mobile sensing  kolkata lab tac_tics2014Mobile sensing  kolkata lab tac_tics2014
Mobile sensing kolkata lab tac_tics2014Arpan Pal
Ā 
lecture5-wearables-and-motion-sening.pptx
lecture5-wearables-and-motion-sening.pptxlecture5-wearables-and-motion-sening.pptx
lecture5-wearables-and-motion-sening.pptxVishwajeetGupta11
Ā 
Behaviometrics: Behavior Modeling from Heterogeneous Sensory Time-Series
Behaviometrics: Behavior Modeling from Heterogeneous Sensory Time-SeriesBehaviometrics: Behavior Modeling from Heterogeneous Sensory Time-Series
Behaviometrics: Behavior Modeling from Heterogeneous Sensory Time-SeriesJiang Zhu
Ā 
Behavioral Analytics with Smartphone Data. Talk at Strata + Hadoop World 2014...
Behavioral Analytics with Smartphone Data. Talk at Strata + Hadoop World 2014...Behavioral Analytics with Smartphone Data. Talk at Strata + Hadoop World 2014...
Behavioral Analytics with Smartphone Data. Talk at Strata + Hadoop World 2014...Joerg Blumtritt
Ā 
Smartphone sensor and gesture
Smartphone sensor and gestureSmartphone sensor and gesture
Smartphone sensor and gestureBhavya5800
Ā 
From Context-awareness to Human Behavior Patterns
From Context-awareness to Human Behavior PatternsFrom Context-awareness to Human Behavior Patterns
From Context-awareness to Human Behavior PatternsVille Antila
Ā 
Data, Big Data and real time analytics for Connected Devices
Data, Big Data and real time analytics for Connected DevicesData, Big Data and real time analytics for Connected Devices
Data, Big Data and real time analytics for Connected DevicesSrinath Perera
Ā 
Sherlock: Monitoring sensor broadcasted data to optimize mobile environment
Sherlock: Monitoring sensor broadcasted data to optimize mobile environmentSherlock: Monitoring sensor broadcasted data to optimize mobile environment
Sherlock: Monitoring sensor broadcasted data to optimize mobile environmentijsrd.com
Ā 
Total station and its application to civil engineering
Total station and its application to civil engineeringTotal station and its application to civil engineering
Total station and its application to civil engineeringTushar Dholakia
Ā 
2013 Lecture3: AR Tracking
2013 Lecture3: AR Tracking 2013 Lecture3: AR Tracking
2013 Lecture3: AR Tracking Mark Billinghurst
Ā 
Tango[VivekKumar_CS-C_6Sem_MIT]
Tango[VivekKumar_CS-C_6Sem_MIT]Tango[VivekKumar_CS-C_6Sem_MIT]
Tango[VivekKumar_CS-C_6Sem_MIT]Vivek Kumar
Ā 
Mobile user experience conference 2009 - The rise of the mobile context
Mobile user experience conference 2009 - The rise of the mobile contextMobile user experience conference 2009 - The rise of the mobile context
Mobile user experience conference 2009 - The rise of the mobile contextFlorent Stroppa
Ā 
Google Project Soli
Google Project SoliGoogle Project Soli
Google Project SoliAkshay Mohokar
Ā 

Similar to Wearable Computing - Part II: Sensors (20)

Project soli
Project soliProject soli
Project soli
Ā 
Making sense
Making senseMaking sense
Making sense
Ā 
Mobile sensing kolkata lab tac_tics2014
Mobile sensing  kolkata lab tac_tics2014Mobile sensing  kolkata lab tac_tics2014
Mobile sensing kolkata lab tac_tics2014
Ā 
lecture5-wearables-and-motion-sening.pptx
lecture5-wearables-and-motion-sening.pptxlecture5-wearables-and-motion-sening.pptx
lecture5-wearables-and-motion-sening.pptx
Ā 
Sensor's inside
Sensor's insideSensor's inside
Sensor's inside
Ā 
Behaviometrics: Behavior Modeling from Heterogeneous Sensory Time-Series
Behaviometrics: Behavior Modeling from Heterogeneous Sensory Time-SeriesBehaviometrics: Behavior Modeling from Heterogeneous Sensory Time-Series
Behaviometrics: Behavior Modeling from Heterogeneous Sensory Time-Series
Ā 
Behavioral Analytics with Smartphone Data. Talk at Strata + Hadoop World 2014...
Behavioral Analytics with Smartphone Data. Talk at Strata + Hadoop World 2014...Behavioral Analytics with Smartphone Data. Talk at Strata + Hadoop World 2014...
Behavioral Analytics with Smartphone Data. Talk at Strata + Hadoop World 2014...
Ā 
Smartphone sensor and gesture
Smartphone sensor and gestureSmartphone sensor and gesture
Smartphone sensor and gesture
Ā 
From Context-awareness to Human Behavior Patterns
From Context-awareness to Human Behavior PatternsFrom Context-awareness to Human Behavior Patterns
From Context-awareness to Human Behavior Patterns
Ā 
Data, Big Data and real time analytics for Connected Devices
Data, Big Data and real time analytics for Connected DevicesData, Big Data and real time analytics for Connected Devices
Data, Big Data and real time analytics for Connected Devices
Ā 
Sherlock: Monitoring sensor broadcasted data to optimize mobile environment
Sherlock: Monitoring sensor broadcasted data to optimize mobile environmentSherlock: Monitoring sensor broadcasted data to optimize mobile environment
Sherlock: Monitoring sensor broadcasted data to optimize mobile environment
Ā 
Total station and its application to civil engineering
Total station and its application to civil engineeringTotal station and its application to civil engineering
Total station and its application to civil engineering
Ā 
2013 Lecture3: AR Tracking
2013 Lecture3: AR Tracking 2013 Lecture3: AR Tracking
2013 Lecture3: AR Tracking
Ā 
Sixth sense
Sixth senseSixth sense
Sixth sense
Ā 
Gesture control
Gesture controlGesture control
Gesture control
Ā 
Tango[VivekKumar_CS-C_6Sem_MIT]
Tango[VivekKumar_CS-C_6Sem_MIT]Tango[VivekKumar_CS-C_6Sem_MIT]
Tango[VivekKumar_CS-C_6Sem_MIT]
Ā 
Smart car
Smart carSmart car
Smart car
Ā 
Networks of wearables and augmented reality for vulnerable user protection
Networks of wearables and augmented reality for vulnerable user protectionNetworks of wearables and augmented reality for vulnerable user protection
Networks of wearables and augmented reality for vulnerable user protection
Ā 
Mobile user experience conference 2009 - The rise of the mobile context
Mobile user experience conference 2009 - The rise of the mobile contextMobile user experience conference 2009 - The rise of the mobile context
Mobile user experience conference 2009 - The rise of the mobile context
Ā 
Google Project Soli
Google Project SoliGoogle Project Soli
Google Project Soli
Ā 

More from Daniel Roggen

W10: Laboratory
W10: LaboratoryW10: Laboratory
W10: LaboratoryDaniel Roggen
Ā 
W10: Interrupts
W10: InterruptsW10: Interrupts
W10: InterruptsDaniel Roggen
Ā 
W9: Laboratory 2
W9: Laboratory 2W9: Laboratory 2
W9: Laboratory 2Daniel Roggen
Ā 
W9_3: UoS Educational Processor: Assembler Memory Access
W9_3: UoS Educational Processor: Assembler Memory AccessW9_3: UoS Educational Processor: Assembler Memory Access
W9_3: UoS Educational Processor: Assembler Memory AccessDaniel Roggen
Ā 
W9_2: Jumps and loops
W9_2: Jumps and loopsW9_2: Jumps and loops
W9_2: Jumps and loopsDaniel Roggen
Ā 
W8: Laboratory 1
W8: Laboratory 1W8: Laboratory 1
W8: Laboratory 1Daniel Roggen
Ā 
W8_2: Inside the UoS Educational Processor
W8_2: Inside the UoS Educational ProcessorW8_2: Inside the UoS Educational Processor
W8_2: Inside the UoS Educational ProcessorDaniel Roggen
Ā 
W8_1: Intro to UoS Educational Processor
W8_1: Intro to UoS Educational ProcessorW8_1: Intro to UoS Educational Processor
W8_1: Intro to UoS Educational ProcessorDaniel Roggen
Ā 
Wearable Computing - Part IV: Ensemble classifiers & Insight into ongoing res...
Wearable Computing - Part IV: Ensemble classifiers & Insight into ongoing res...Wearable Computing - Part IV: Ensemble classifiers & Insight into ongoing res...
Wearable Computing - Part IV: Ensemble classifiers & Insight into ongoing res...Daniel Roggen
Ā 

More from Daniel Roggen (9)

W10: Laboratory
W10: LaboratoryW10: Laboratory
W10: Laboratory
Ā 
W10: Interrupts
W10: InterruptsW10: Interrupts
W10: Interrupts
Ā 
W9: Laboratory 2
W9: Laboratory 2W9: Laboratory 2
W9: Laboratory 2
Ā 
W9_3: UoS Educational Processor: Assembler Memory Access
W9_3: UoS Educational Processor: Assembler Memory AccessW9_3: UoS Educational Processor: Assembler Memory Access
W9_3: UoS Educational Processor: Assembler Memory Access
Ā 
W9_2: Jumps and loops
W9_2: Jumps and loopsW9_2: Jumps and loops
W9_2: Jumps and loops
Ā 
W8: Laboratory 1
W8: Laboratory 1W8: Laboratory 1
W8: Laboratory 1
Ā 
W8_2: Inside the UoS Educational Processor
W8_2: Inside the UoS Educational ProcessorW8_2: Inside the UoS Educational Processor
W8_2: Inside the UoS Educational Processor
Ā 
W8_1: Intro to UoS Educational Processor
W8_1: Intro to UoS Educational ProcessorW8_1: Intro to UoS Educational Processor
W8_1: Intro to UoS Educational Processor
Ā 
Wearable Computing - Part IV: Ensemble classifiers & Insight into ongoing res...
Wearable Computing - Part IV: Ensemble classifiers & Insight into ongoing res...Wearable Computing - Part IV: Ensemble classifiers & Insight into ongoing res...
Wearable Computing - Part IV: Ensemble classifiers & Insight into ongoing res...
Ā 

Recently uploaded

Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j
Ā 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationRidwan Fadjar
Ā 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...shyamraj55
Ā 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdfhans926745
Ā 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersThousandEyes
Ā 
Google AI Hackathon: LLM based Evaluator for RAG
Google AI Hackathon: LLM based Evaluator for RAGGoogle AI Hackathon: LLM based Evaluator for RAG
Google AI Hackathon: LLM based Evaluator for RAGSujit Pal
Ā 
FULL ENJOY šŸ” 8264348440 šŸ” Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY šŸ” 8264348440 šŸ” Call Girls in Diplomatic Enclave | DelhiFULL ENJOY šŸ” 8264348440 šŸ” Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY šŸ” 8264348440 šŸ” Call Girls in Diplomatic Enclave | Delhisoniya singh
Ā 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking MenDelhi Call girls
Ā 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking MenDelhi Call girls
Ā 
šŸ¬ The future of MySQL is Postgres šŸ˜
šŸ¬  The future of MySQL is Postgres   šŸ˜šŸ¬  The future of MySQL is Postgres   šŸ˜
šŸ¬ The future of MySQL is Postgres šŸ˜RTylerCroy
Ā 
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 3652toLead Limited
Ā 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Servicegiselly40
Ā 
Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101Paola De la Torre
Ā 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Igalia
Ā 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking MenDelhi Call girls
Ā 
Transcript: #StandardsGoals for 2024: Whatā€™s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: Whatā€™s new for BISAC - Tech Forum 2024Transcript: #StandardsGoals for 2024: Whatā€™s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: Whatā€™s new for BISAC - Tech Forum 2024BookNet Canada
Ā 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfEnterprise Knowledge
Ā 
Swan(sea) Song ā€“ personal research during my six years at Swansea ... and bey...
Swan(sea) Song ā€“ personal research during my six years at Swansea ... and bey...Swan(sea) Song ā€“ personal research during my six years at Swansea ... and bey...
Swan(sea) Song ā€“ personal research during my six years at Swansea ... and bey...Alan Dix
Ā 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitecturePixlogix Infotech
Ā 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationMichael W. Hawkins
Ā 

Recently uploaded (20)

Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Ā 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 Presentation
Ā 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Ā 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
Ā 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Ā 
Google AI Hackathon: LLM based Evaluator for RAG
Google AI Hackathon: LLM based Evaluator for RAGGoogle AI Hackathon: LLM based Evaluator for RAG
Google AI Hackathon: LLM based Evaluator for RAG
Ā 
FULL ENJOY šŸ” 8264348440 šŸ” Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY šŸ” 8264348440 šŸ” Call Girls in Diplomatic Enclave | DelhiFULL ENJOY šŸ” 8264348440 šŸ” Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY šŸ” 8264348440 šŸ” Call Girls in Diplomatic Enclave | Delhi
Ā 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
Ā 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
Ā 
šŸ¬ The future of MySQL is Postgres šŸ˜
šŸ¬  The future of MySQL is Postgres   šŸ˜šŸ¬  The future of MySQL is Postgres   šŸ˜
šŸ¬ The future of MySQL is Postgres šŸ˜
Ā 
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Ā 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
Ā 
Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101
Ā 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Ā 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
Ā 
Transcript: #StandardsGoals for 2024: Whatā€™s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: Whatā€™s new for BISAC - Tech Forum 2024Transcript: #StandardsGoals for 2024: Whatā€™s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: Whatā€™s new for BISAC - Tech Forum 2024
Ā 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
Ā 
Swan(sea) Song ā€“ personal research during my six years at Swansea ... and bey...
Swan(sea) Song ā€“ personal research during my six years at Swansea ... and bey...Swan(sea) Song ā€“ personal research during my six years at Swansea ... and bey...
Swan(sea) Song ā€“ personal research during my six years at Swansea ... and bey...
Ā 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC Architecture
Ā 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
Ā 

Wearable Computing - Part II: Sensors

  • 2. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Physical Social Mental Mental ā€¢ Emotion awareness ā€“ Own/Others ā€“ Sadness, joy ā€“ Depression ā€¢ Cognitive awareness ā€“ Cognitive load ā€“ Attention, concentration ā€“ Stress ā€¢ ā€¦ Social ā€¢ Social interactions ā€“ Detect known people ā€¢ Social network ā€“ Information exchange ā€“ Optimization of organizations ā€¢ Crowd / collective behavior ā€¢ ā€¦ Dimensions of context Physical ā€¢ User location ā€“ Absolute, relative ā€¢ User activity ā€“ Manipulative gestures, pointing movements, modes of locomotion, posture, composite and hierarchical activities ā€¢ ā€¦ Environment ā€¢ Map of surrounding services ā€¢ Environment characteristics ā€“ Temperature, light, humidity ā€¢ Radio fingerprints ā€¢ ā€¦ Environment
  • 3. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Sensing context ā€¢ Sensors are needed to infer the userā€™s context ā€¢ Ā« Software sensors Ā» ā€“ SMS on mobile phones: e.g. used to infer friend network ā€“ Email reception: e.g. work / home presence ā€“ Calendar: e.g. location, work activities ā€“ ā€¦ ā€¢ Hardware sensors ā€“ (the typical definition of sensors in EE ļŠ) ā€“ Accelerometers: e.g. gesture recognition ā€“ RFID tags: e.g. object detection ā€“ Reed switch e.g. door/window open/closed ā€“ GPS: e.g. outdoor location ā€“ WIFI fingerprints: e.g. indoor location ā€“ ā€¦
  • 4. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com There is no Ā« Drink Sensor Ā» ļŠ ā€¢ Simple sensors (e.g. RFID) can provide a "binary" information ā€“ Presence (e.g. RFID, Proximity infrared sensors) ā€“ Movement (e.g. ADXL345 accelerometer ā€˜activity/inactivity pinā€™) ā€“ Fall (e.g. ADXL345 accelerometer ā€˜freefall pinā€™) ā€¢ But in general Ā« activity-X sensor Ā» does not exist ā€“ Sensor data must be interpreted ā€“ Multiple sensors must be correlated (data fusion) ā€“ Several factors influence the sensor data ā€¢ Drinking while standing: the arm reaches the object then the mouth ā€¢ Drinking while walking: the arm moves, and also the whole body ā€¢ Context is inferring from the sensor data with ā€“ Signal processing ā€“ Machine learning ā€“ Reasoning ā€¢ Can be integrated into a Ā« sensor node Ā» or Ā« smart sensor Ā» ā€“ Sensor chip + data processing in a device
  • 5. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.comRoggen et al., Collecting complex activity datasets in highly rich networked sensor environments, Proc INSS, 2010
  • 6. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Accelerometer ā€¢ The most common sensor in activity recognition ā€¢ Used in mobile phones to rotate screen ā€¢ Nintendoo Wii ā€¢ Highly miniaturized with MEMS technology [1] ADXL335 datasheet, Analog devices [2] http://www.silicondesigns.com/tech.html [2] ā€¢ ADXL335 (analog output) ā€“ 3D accelerometer ā€“ 4x4 mm ā€“ Vin: 1.8-3.6V ā€“ I: 350 uA ā€“ Analog output: 300mV/g ā€“ <4$/unit
  • 7. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Accelerometer ā€¢ Information about static postures and movements ā€¢ Static acceleration ā€“ Without user movement (or very slow) ā€“ Earth gravity projected onto accelerometer coordinates ā€“ Senses limb angle Ī± Ax r Az Ī± 1g ā€¢ Dynamic acceleration ā€“ Two components: limb acceleration and earth gravity Ax Ay Az 1g Gesture in x-z plane Gesture in x-y plane ā€¢ Challenges: ā€“ On-body rotation ā€¢ use acceleration magnitude! ā€“ Plane of gesture ā€¢ Different signals!
  • 8. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Walking, running, jumping Figo, Diniz, Ferreira, Cardoso. Preprocessing techniques for context recognition from accelerometer data, Pers Ubiquit Comput, 14:645ā€“662, 2010
  • 9. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Car manufacturing activities Data from Zappi et al, Activity recognition from on-body sensors: accuracy-power trade-off by dynamic sensor selection, EWSN, 2008 Dataset available at: http://www.wearable.ethz.ch/resources/Dataset
  • 10. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Swim styles Delphin Kraul Brust RĆ¼cken (jeweils zwei Bahnen) BƤchlin, Fƶrster, Trƶster, SwimMaster: A wearable assistant for swimmer, Ubicomp, 2009
  • 11. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Swim styles BƤchlin, Fƶrster, Trƶster, SwimMaster: A wearable assistant for swimmer, Ubicomp, 2009 Armstrokes
  • 12. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com OPPORTUNITY Dataset ā€¢ Appartment flat, early morning activities "Activity of daily living" run ā€¢ Loose high level instructions ā€¢ Primitives to high-level activities ā€¢ 5 repetitions ā€¢ 12 subjects ā€¢ ~20mn / run Drill run ā€¢ Scripted sequence ā€¢ Only activity primitives ā€¢ 20 repetitions ā€¢ 12 subjects ā€¢ ~30mn / run http://vimeo.com/8704668
  • 13. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Gyroscope ā€¢ Measures rate of turn (degrees / sec) ā€“ Recently introduced in mobile phones (iPhone 4) ā€“ Camera image stabilization ā€“ Augmented reality (with accelerometer and compass as IMU) ā€¢ Principle: ā€“ MEMS vibrating structure ā€“ Coriolis force displaces masses ā€“ Change in capacitance ā€¢ Suited to ā€“ measure limb rotation ā€“ heading change ā€“ step length [1] http://www.findmems.com/wikimems-learn/introduction-to-mems-gyroscopes [1]
  • 14. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Detection of snowboard turns with a gyro [1] [1] Holleczek et al., Recognizing Turns and Other Snowboarding Activities with a Gyroscope, ISWC, 2010
  • 15. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Inertial measurement unit ā€¢ Combination of ā€“ 3D accelerometer ā€“ 3D compass ā€“ 3D gyroscope ā€¢ Provides device orientation in earth reference frame ā€“ Euler angles ā€“ Quaternions [1] Xsens MT9 orientation sensor
  • 16. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Inertial measurement unit: principle ā€¢ [1]: Kalman filter to estimate 3DoF orientation ā€“ Accelerometer and compass compensate drift in integration of gyroscope ā€“ "Attitude and heading reference system" ā€“ Accelerometer stabilizes attitude: dynamic acceleration is zero on average (earth gravity) ā€“ Compass stabilizes heading (careful with magnetic disturbances, metallic objects!) [1] MTi and MTx User Manual and Technical Documentation, Xsens, 2009 [2] Torres, Flynn, Angove, Murphy, Mathuna, Motion Tracking Algorithms for Inertial Measurement, BodyNets, 2007 [2]
  • 17. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Inertial measurement unit ā€¢ One IMU per body segment for body-model reconstruction [1] Xsens MVN - Inertial Motion Capture [2] Stiefmeier et al, Wearable Activity Tracking in Car Manufacturing, PCM, 2008 [1] [2]
  • 18. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Location awareness in wearables Augmented Reality Annotations Uratani et al., Wearable Augmented Reality System with Annotation Visualization, CREST, 2004 Virtual object in physical world Tinmith AR, University South Australia Modeling transportation, likely routes, significant places... Patterson, Inferring High-Level Behavior from Low-Level Sensors, Ubicomp 2003 Ashbrook, Using GPS to Learn Significant Location, Personal and Ubiquitous Computing 7(5) Location-based activity segmentation Stiefmeier, Ogris, ETHZ, Uni Passau Location-Aware Computing
  • 19. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Location awareness Hightower, Borriello, Location systems for ubiquitous computing. IEEE Computer (2001) 57ā€“ Localization beacons IR, Bluetooth Existing radios Wifi, GSM Indoor App. Specific UWB (Ubisense) Video tracking + Large area coverage + ā€œSymbolicā€ localization (room#) + Indoor, deployment + High accuracy + Unobtrusive - Urban canyons, indoor - Wireless map - Occlusion, privacy - Environment instrumentation - Instrumentation, short range Ideally, for wearable computing: ā€¢ Indoor and outdoor use ā€¢ No instrumented environment ā€¢ No a-priori knowledge / map ā€¢ Anytime, anywhere ā€¢ Low-power ā€¢ Small Satellite-systems GPS
  • 20. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com GPS revisited ā€¢ Simultaneous measurement of time-of flight (phase) from 4 satellites ā€¢ Weak signal: -160dBW, 1575MHz L1 ā€¢ For outdoors.... ā€¢ ... but some indoor capability [1] ā€¢ Uses lots of energy [1] KjƦrgaard et al., Indoor Positioning Using GPS Revisited, Pervasive, 2010 [2] http://en.wikipedia.org/wiki/GNSS_positioning_calculation [1] [2]
  • 21. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Radio fingerprints ā€¢ Radio fingerprint quite unique for any location: ā€“ Visible radios (Wifi MAC address, GSM towers) ā€“ Signal strength ā€¢ APIs for absolute localization from radio fingerprints: ā€“ http://www.opencellid.org/ (cellphone towers) ā€“ Google geolocation API ā€¢ In principle not limited to radio: ā€“ information also in sound, visual patterns, etc...
  • 22. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Dead reckoning ā€¢ Historically used by sailors: speed estimation and compass ā€¢ With wearable sensors [1]: ā€“ Compass to estimate heading ā€“ Count steps (pedometer, accelerometer); integrate acceleration over single steps [1] Randell, Djiallis, Muller, Personal Position Measurement Using Dead Reckoning, ISWC, 2003 ā€¢ Quite accurate (~10%) with a correct motion model ā€“ (e.g. compass must be parallel to ground) ā€¢ Integration of errors lead to "closing-the-loop" problem ā€“ Identify already visited points with additional sensors and "close the loop" ā€“ Multiple sensors and Kalman filter
  • 23. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Dead reckoning with vision (optical flow) Forward motion Rotation Side step ā€¢ Typical motion pattern result in characteristic optical flows Dead-reckoning from optical flow: 1. Compute optical flow 2. Derive corresponding camera movement (egomotion) 3. Integrate egomotion in a map [1] Roggen et al, Mapping by seeing: wearable vision-based dead-reckoning, and closing the loop, EuroSSC 2007 Slow Fast
  • 24. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com ā€¢ Straight walk: 5.5m ā€¢ Estimated distance: 5.17m ā€¢ Relative error: 6% Speed profile Path integration (outdoor) ā€¢ Square walk (10m sides) ā€¢ Distance: 38.8m ā€¢ Estimated distance: 43.4m ā€¢ Relative error: 12% ā€¢ Challenge: high rotation speeds (30Ā°/s-90Ā°/s) ā€¢ h=145cm, Ī±=48Ā° ā€¢ Chest-placed camera Footsteps [1] Roggen et al, Mapping by seeing: wearable vision-based dead-reckoning, and closing the loop, EuroSSC 2007
  • 25. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Closing the loop ā€¢ Errors accumulate during path integration ā€¢ Coinciding start-end point of trajectory do not map on same location A-posteriori correction of motion vectors 1. Recognize identical locations A, B 2. Correct motion vectors so A matches B ā†’ Idea: closing the loop ā„¦cl=f(ā„¦)ā„¦ [1] Roggen et al, Mapping by seeing: wearable vision-based dead-reckoning, and closing the loop, EuroSSC 2007 ā€¢ 6-12% distance error ā€¢ Underestimation of rotation ā€“ Note: camera only! With a gyro can be improved
  • 26. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Estimating proximity from passive fingerprints: sound ā€¢ Proximity information is important for many applications! [1] Wirz et al., A wearable, ambient sound-based approach for infrastructureless fuzzy proximity estimation, ISWC 2010
  • 27. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Estimating proximity from passive fingerprints: Wifi [1] Cugia, Wirz, ETH Tech Rep, 2011 ā€¢ Devices A, B scan Wifi (MAC addresses) ā€¢ Compute fingerprint similarity ā€“ E.g. Jaccard index: normalized number MAC address seen by both A and B Sensor number Hears WIFI signals S0 W2 W6 W7 S1 W1 W3 W4 W5 S2 W2 W3 W4 W5 W6 W7 S3 W0 W1 W2 W3 W4 W5 S4 W0 W1 W2 W3 W4 W5
  • 28. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com From proximity to topology ā€¢ Proximity can be used to find node topology ā€“ Binary case: "sense"/"not sense" (e.g. communication in/out-of range) ā€“ Or distance estimate ā€¢ Convert proximity to topology ā€“ Multidimensional scaling [1,2] ā€¢ "Anchor nodes" (e.g. with GPS) can "ground" the topology [1] Koo, Cha, Autonomous Construction of a WiFi Access Point Map Using Multidimensional Scaling, Pervasive, 2011 [2] Chan&So, Efficient Weighted Multidimensional Scaling for Wireless Sensor Network Localization, IEEE Tr Sig Proc, 57(11), 2009
  • 29. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Active capacitive sensing [1,2] ā€¢ Challenge: sensing inner body function ā€“ E.g. drinking, heart rate, respiration, etc. ā€¢ Principle: ā€“ Capacitor with human body as dielectric ā€“ Body function affect dielectric: muscle contractions, joint motions, air entering the lungs, drinking ā€“ Readout using a Colpitts (LC) oscillator ā€“ Suitable for textile integration! [1] Cheng et al., Active Capacitive Sensing: Exploring a New Wearable Sensing Modality for Activity Recognition, Pervasive, 2011 [2] Lukowicz et al., On-Body Sensing: From Gesture-Based Input to Activity-Driven Interaction, IEEE Computer Magazine Dielectric
  • 30. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Active capacitive sensing [1,2] ā€¢ Characteristic signal for swallowing ā€¢ Also reacts to movements (head direction, body movement) [1] Cheng et al., Active Capacitive Sensing: Exploring a New Wearable Sensing Modality for Activity Recognition, Pervasive, 2011 [2] Lukowicz et al., On-Body Sensing: From Gesture-Based Input to Activity-Driven Interaction, IEEE Computer Magazine
  • 31. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Electrooculography [1] ā€¢ Measures E potential around the eye ā€¢ Can identify saccades, fixation, blinks, movement patterns ā€¢ Alternative technology: camera-based [1] Bulling et al., Whatā€™s in the Eyes for Context-Awareness? Pervasive computing magazine, 2011
  • 32. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com
  • 33. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Video on the emotional computer
  • 34. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Sensing intense fear (startle event) ā€¢ Skin conductance: Galvanic Skin Response = Electrodermal Activity ā€“ Affected by Ā« internal states Ā»: emotions, phobias, arousal, stress ā€¢ Controlled by the sympathetic nervous system (SNS) [1] ā€“ Low frequency Ā« tonic Ā» part (0-0.05 Hz) ā€“ Fast changing phasic component (0.05 Hz-1.5 Hz) ā€¢ Quickly reacts to the Ā« fight or flight Ā» reflex (e.g. fear / startle event) [1] Fuller GD. GSR History, & Physiology. San Francisco: Biofeedback Institute San Francisco; 1977. [2] Schumm et al., Effect of Movements on the Electrodermal Response after a Startle Event, Methods of Information in Biomedicine, 2008 Wearable EDA sensors from [2]
  • 35. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Experiment ā€¢ Walking on a treadmill ā€¢ Listening to music ā€¢ Loud Ā« bang Ā» at unexpected moments Schumm et al., Effect of Movements on the Electrodermal Response after a Startle Event, Methods of Information in Biomedicine, 2008
  • 36. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Experiment ā€¢ Visualization with cross-correlogram Schumm et al., Effect of Movements on the Electrodermal Response after a Startle Event, Methods of Information in Biomedicine, 2008 EDA activation after starleā€¦ ā€¦but some challenges: ā€¢ Physical activity ā€¢ Background EDA activation ā€¢ No unique response to startle Startle events aligned at t=0 Multimodal approach? ā€¢ Sensing movement and EDA ā€¢ ā€¦
  • 37. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Sensing cognitive states from eye movements [1] Hayhoe, Ballard Eye movements in natural behavior, TRENDS in Cognitive Sciences Vol.9 No.4 April 2005 [2] Heisz, Shore, More efficient scanning for familiar faces, Journal of Vision (2008) 8(1):9, 1ā€“10 [3] Bulling et al., Whatā€™s in the Eyes for Context-Awareness? Pervasive computing magazine, 2011 [1] [1] Familiar v.s. unfamiliar faces [2] Gaze patterns are consciously and unconsciously controlled. Can eye movements reveal something about cognitive states? ā€¢ "Eventually, analyzing the link between unconscious eye movements and cognition might even pave the way for a new genre of pervasive computing systems that can sense and adapt to a personā€™s cognitive context. [....] A computing system is cognition-aware if it can sense and adapt to a personā€™s cognitive context." [3]
  • 38. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Sensing cognitive states from eye movements [1] [1] Bulling et al., Whatā€™s in the Eyes for Context-Awareness? Pervasive computing magazine, 2011 Applications: ā€¢ Memory assistants ā€¢ Smart computer interfaces ā€¢ Significant change in fixation count ā€¢ Ongoing: classify in "seen" / "unseen" ā€¢ Challenge: outside of the lab! ā€¢ Cognition: Memory, Stress, Reasoning, ... ā€¢ Can a computer detect whether a picture has already been seen before?
  • 39. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Group near exit Clogged Dense Queuing Group split Lane formation ā€¢ Capability and properties of mobile system... ā€¢ for machine recognition of crowd behavior ā€¢ Supports situation awareness Social context: sensing crowd behaviors
  • 40. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Social context: sensing crowd behaviors Wirz, M.; Roggen, D.; Trƶster, G.: Decentralized Detection of Group Formations from Wearable Acceleration Sensors Wirz et al. A Methodology towards the Detection of Collective Behavior Patterns by Means of Body-Worn Sensors, Workshop at Pervasive, 2010 Roggen et al., Recognition of crowd behavior from mobile sensors with pattern analysis and graph clustering methods, Submitted to NHM, 2011
  • 41. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Integrating sensors into clothing ā€¢ Clothing integration is the "dream" of wearable computing ā€¢ "Normal" clothes are extremely ruggedized! ā€¢ Challenges for smart-textiles / sensorized garments: ā€“ Washing ā€“ Bending ā€“ Friction ā€“ Integration in standard manufacturing process ā€“ Requires sophisticated equipment! ā€¢ Example technology: ā€“ Kapton substrate, thin film transistor ā€“ Thin straps woven with standard machine Our smart textile fabrication process, Cherenack et al, IEEE Electr. Dev. Lett, vol. no. 7, pp. 740-742, July 2010 A woven temperature sensor, Kinkeldei et al, Proc. of the IEEE Sensors Conference, pp 1-4 2009
  • 42. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Smart-textiles System on Textiles ā€¢ Development of conductive fabrics for signal transmission in wearables ā€¢ Investigation of the electrical performance: measuring and modeling high-frequency properties ā€¢ Simulation and optimization of different fabrics and configurations ā€¢ Interconnections Woven fabric with conductive fibers Unobtrusive context recognition, healthcare (e.g. rehabilitation), sports Routing Methods Adapted to e-Textiles. I. Locher, T. Kirstein and G. Trƶster, Proc. 37th International Symposium on Microelectronics (IMAPS 2004), Long Beach CA, Nov. 14-18, 2004
  • 43. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Smart-textiles Textile Antennas ā€¢ Magnetic coupled antenna ā€¢ Signal transmission between pieces of clothing. Textile Body Area Network Trans- mitter Receiver H ā€¢ Bluetooth antenna ā€¢ Signal transmission along body ā€¢ Communication with near infrastructure ā€¢ Modeling and measuring of electrical properties ā€¢ Modeling and measuring of textile-specific properties Design and Characterization of Purely Textile Patch Antennas. I. Locher, M. Klemm, T. Kirstein and G. Trƶster, IEEE Transactions on Advanced Packaging, Vol. 29, No. 4, Nov. 2006, pp. 777-788
  • 44. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Index Finger Temperature Profile Temperature Sensor ā€¢ 74 g/m2 ā€¢ Polyester Interlining ā€¢ Insulated copper wires Temperature Profile Estimation with Smart Textiles. I. Locher, T. Kirstein and G. Trƶster, Proc. 1st International Scientific Conference Ambience 05, Tampere, Finland, Sept. 19-20, 2005 Smart-textiles Textile Temperature Sensor
  • 45. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com C o n d u c tiv e t e x tile S p a c e r ( f o a m , t e x t ile ) N o n c o n d u c t iv e t e x t ile Capacitive textile Sensor Electrodes embroidered with conductive yarn on both sides of compressible spacer Applications: ā€¢ Medicine (Decubitus prevention) ā€¢ Weight measurement in car seats Smart-textiles Capacitive Textile Pressure Sensor Textile Pressure Sensor for Muscle Activity and Motion Detection. J. Meyer, P. Lukowicz, G. Trƶster, ISWC 2006: Proceedings of the 10th IEEE International Symposium on Wearable Computers, Montreux, Switzerland, 11.-14. October 2006
  • 46. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Integrating sensors into clothing ā€¢ A rapid prototyping technology [1] [1] Harms et al., Rapid prototyping of smart garments for activity-aware applications, JAISE, 2009
  • 47. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com lower body upper body VDRG: Velocity-Damped Resonant Generator CDRG: Coulomb-Damped Resonant Generator CFPG: Coulomb-Force Parameteric Generator Optimization of Inertial Micropower Generators for Human Walking Motion. Thomas von BĆ¼ren, P. D. Mitcheson, T. C. Green, E. M. Yeatman, A. S. Holmes, and G. Trƶster. IEEE Sensors Journal, 2005 Energy Harvesting Mechanical Āµāˆ’Generator for On-Body Sensor Networks
  • 48. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Maximum-Power-Point Tracking for On-Body Context Systems. N.B. Bharatula, J.A. Ward, P. Lukowicz, G. Trƶster, ISWC 2006: Proceedings of the 10th IEEE International Symposium on Wearable Computers Energy Harvesting Hybrid solar-battery power supply Solar cell Li-Ion battery
  • 49. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Existing datasets PlaceLab dataset TU Munchen kitchen Many others Homesetting (van Kasteren) Long-term activity (van Laerhoeven) + Real behaviors in house + Marker free motion capture + Application specific + Real behaviors in house + Free-living activities - Small number of instances - No wearable sensors - Often not open - Binary sensors - Annotations http://www.wearable.ethz.ch/resources/Dataset
  • 50. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Collecting large scale datasets: Lessons learned ā€¢ Appartment flat, early morning activities "Activity of daily living" run ā€¢ Loose high level instructions ā€¢ Primitives to high-level activities ā€¢ 5 repetitions ā€¢ 12 subjects ā€¢ ~20mn / run Drill run ā€¢ Scripted sequence ā€¢ Only activity primitives ā€¢ 20 repetitions ā€¢ 12 subjects ā€¢ ~30mn / run
  • 51. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Wearable sensors: sense body movement (and sound) ā€¢ Nodes: 24 (mostly IMUs, accel.) ā€¢ Modalities: 4 ā€¢ Networks: 6
  • 52. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Instrumented objects: Sense object use ā€¢ Sensor nodes: 12 objects ā€¢ Modalities: 2 (acceleration, gyroscope) ā€¢ Networks: 1 (Bluetooth) Accelerometer + gyroscope FSRs XSense
  • 53. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Ambient sensors: sense environment interactions ā€¢ Sensor nodes: 36 ā€¢ Modalities: 9 ā€¢ Networks: 8
  • 54. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com The OPPORTUNITY dataset: key facts Sensor rich ā€¢ 72 sensors (28 sensors in 2.4GHz band) ā€¢ 10 modalities ā€¢ 15 wired and wireless systems (USB, Bluetooth, 802.15.4, custom) ā€¢ 2.5% data loss on wireless sensors Activity rich ā€¢ 12 subjects, ~25 hours of total data ā€¢ > 30'000 interaction primitives (object, environment) ā€¢ Annotation length = 230% of dataset length Highly collaborative effort ā€¢ 11 days recording session, 10 experimenters ā€¢ Subjects instrumented for 5-7 hours ā€¢ ~10 hours of annotation effort for 30mn of data
  • 55. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Challenge 1: Data recording with heterogeneous sensor networks Integration at system level + Central control & monitoring + Synchronized data acquisition - Internals of sensor systems - Fixed real-time merge Integration at data level + Independent data recorders + Robustness, flexibility - Complex control & monitoring - Offline synchronization Obtain synchronized data streams for further processing ā€¢ 7 computers recording sensor data ā€“ Store data and data reception time ā€“ Coarse NTP synchronisation ā€“ Fine synchronisation with specific gestures (ā€œjump and clapā€)
  • 56. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Challenge 2: data handling after recording Burst equalization w/ streaming sensors Missing data represented as NaNs Stream alignment to video footage
  • 57. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Challenge 3: flexible activity annotations, at all levels ~60 instances ~30'000 instances Solution: annotation on multiple tracks, hand-action-object representation
  • 58. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Some lessons learned and best practices ā€¢ Complex recordings will see heterogeneous WSN ā€“ System level integration is desiredā€¦ ā€“ ā€¦ however data level integration may be sufficient ā€¢ Developing custom tools can be beneficial ā€“ Signal/video alignment, annotation ā€“ Now: students can do the work ā€¢ Plan for a repository for raw data ā€“ Required already before data can be stored in a database ā€“ Can be huge!
  • 59. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Some lessons learned and best practices ā€¢ Test infrastructure locally ā€“ Placement of antennas ā€“ Integration & optimization of the wireless links ā€¢ Expect, plan for, and accept data loss or failures ā€“ Independent systems ā€“ Strategy: ignore problem, fix&restart, fix&continue ā€¢ Data integrity checks ā€“ Tradeoff: frequency / time of test-subjects ā€¢ Develop a rapid on-body sensor deployment solution ā€“ Clothing attached/integrated sensors ā€“ Reproducible placement ā€“ More convenient
  • 60. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Some lessons learned and best practices ā€¢ Avoid wireless links altogether :) ā€“ Use local data storage ā€“ Wireless only for synchronization ā€¢ Do not underestimate logistics ā€“ 10 people during 11 days ā€“ Room rented for 11 days, 3 days setup ā€“ Setup: 3 days
  • 61. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com ā€¢ http://www.intersense.com ā€¢ Nintendo WiiMote ā€¢ Open source BlueSense acceleration/gyro sensors (http://www.wearable.ethz. ch/resources/educationkit) Some readily available sensors ā€¢ http://www.phidgets.com/ ā€¢ http://www.sparkfun.com/ ā€¢ LilyPad (http://hlt.media.mit.edu/? p=34) ā€¢ http://www.xsens.com/
  • 62. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com Summary ā€¢ User requirements ā€“ Small / unobtrusive ā€“ Invisible to the outside ā€“ Comfortable ā€“ Privacy issues ā€¢ Context-recognition requirements ā€“ Must be discriminative of the context to recognize ā€“ Minimize subsequent processing (simpler modalities favored) ā€¢ Hardware requirements ā€“ Low-power ā€“ Low-computational complexity ā€¢ Usually multimodal approaches ā€“ Multiple activities affect a sensor ā€“ Several sensors can disambiguate the activity
  • 63. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com For further reading Context ā€¢ Dey, Understanding and Using Context, Personal and Ubiquitous Computing,2001 Activity-aware computing ā€¢ Lukowicz et al., On-Body Sensing: From Gesture-Based Input to Activity-Driven Interaction, IEEE Computer, 43(10), pp. 92-96, 2010 ā€¢ N. Davies, D. Siewiorek, R. Sukthankar, Special Issue: Activity-Based Computing, IEEE Pervasive Computing, 7(2), pp. 20-21, 2008 ā€¢ Stiefmeier et al, Wearable Activity Tracking in Car Manufacturing, PCM, 2008 Cognitive-affective computing ā€¢ R. Picard, Affective computing, MIT Press, 1997 ā€¢ R. Picard, E. Vyzas, J. Healey, Toward Machine Emotional Intelligence: Analysis of Affective Physiological State, IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(10), pp. 1175-1191, 2001 ā€¢ Bulling et al., Whatā€™s in the Eyes for Context-Awareness? Pervasive computing magazine, 2011 Location-awareness ā€¢ Hightower, Borriello, Location systems for ubiquitous computing, IEEE Computer, pp. 57-66, 2001 ā€¢ Kjaergaard, Blunck, Godsk, Toftkjaer, Christensen, Gro}nb{ae}k, Indoor Positioning Using GPS Revisited, Pervasive, 2010 ā€¢ Koo, Cha, Autonomous Construction of a WiFi Access Point Map Using Multidimensional Scaling, Pervasive, 2011 Social context ā€¢ Wirz, Roggen, Trƶster, Decentralized Detection of Group Formations from Wearable Acceleration Sensors, Int. Conf. Social Computing, 2009 Datasets ā€¢ Roggen et al., Collecting complex activity data sets in highly rich networked sensor environments, INSS, 2010 Context and Sensing frameworks ā€¢ Roggen et al., Titan: An Enabling Framework for Activity-Aware ``PervasiveApps'' in Opportunistic Personal Area Networks, EURASIP Journal on Wireless Communications and Networking, 2011 ā€¢ Kukkonen, Lagerspetz, Nurmi, Andersson, BeTelGeuse: A Platform for Gathering and Processing Situational Data, IEEE Pervasive Computing Magazine, 8(2): 49-56, 2009 ā€¢ Fortino, Guerrieri, Bellifemine, Giannantonio, SPINE2: Developing BSN Applications on Heterogeneous Sensor Nodes, Proc. IEEE Symposium on Industrial Embedded Systems SIES2009, 2009 ā€¢ Bannach et al., Rapid Prototyping of Activity Recognition Applications, IEEE Pervasive Computing Magazine, 7(2):22-31, 2008 ā€¢ Kurz et al., The OPPORTUNITY Framework and Data Processing Ecosystem for Opportunistic Activity and Context Recognition, International Journal of Sensors, Wireless Communications and Control, 2011
  • 64. Ā© Daniel Roggen www.danielroggen.net droggen@gmail.com

Editor's Notes

  1. Now, letā€˜s have a look how the incorporation of technology that is sensitive and responsive to the presence of people can help in real life scenarios. Letā€˜s assume thereā€˜s a fire in a bulding and people are trying to escape. One person walks towards an exit and sees that the corridor is blocked. Why? Whatā€˜s ahead? What should he do? In such a situation, a mobile device could provide relevant information to come to a decsion. Another person can be guided efficiently out of the building. In this example, it might be faster to take the much longer escape route since it is not jammed.