Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Andere mochten auch
Ähnlich wie Mobile Sensing Data Sensemaking with Middleware
Ähnlich wie Mobile Sensing Data Sensemaking with Middleware (20)
Mobile Sensing Data Sensemaking with Middleware
- 1. Sensemaking from Distributed and Mobile Sensing Data: A Middleware Perspec;ve S.Sarma, N. Venkatasubramanian, N. DuA 1
- 2. Overview • Introduc0on to Crowdsensing and Sensemaking • A Middleware Perspec0ve • Example Middleware Pla?orms and techniques • Research Direc0ons 2
- 3. Mobile Phone Trends • Mobile subscrip;on 5.96 billion 2011 es;mate • Smartphones (487.7 million) exceeding PCs (414.6 million) • More Mobile Internet Users Than Wireline Users in the U.S. by 2015 • Smartphone and bandwidth cost reduces • Smart devices contribute to more than 90% of mobile data traffic 3
- 4. Sensors In Mobile Phones • MEMS & sensors for cell phones, expanding from $ 3.5 bn in 2009 to $7.9 bn in 2015 [Yole Developpement] • Smartphone sensors to be $ 6 bn business by 2016 [Juniper Research] • 44 % of the mobile phones will be smartphones in 2015 • 7x increase in mobile health apps from 2010 to 2011 • mo;on sensor in smartphones and tablets will expand to $ US 2.1 billion in 2015 with a 25.3 % CAGR, up from $1.19 billion in 2011 (IHS iSuppli) 4
- 5. Mobile Sensors Trends Source: IHS Consumer & Mobile MEMS Market Tracker, April 2014. 5
- 6. Mobile Data Delivery Everywhere 6 Smart devices contribute to more than 90% of mobile data traffic The exploding number of apps is driven by a huge up;ck in the number of smart devices ~55% Cisco’s report 2014
- 7. Crowdsourcing and CrowdSensing 7 Pushing toward more interven0on
- 8. Power of the Crowd • Using mobile crowdsensing to – Leverage already deployed smartphones – Extend the ranges of exis0ng in-‐situ sensors – Send mobile users to specific loca0ons • Crowdsensing broad use cases – Disaster and emergency response – Personal health monitoring and wellness – Smart spaces and their effec0ve u0liza0on 8 [YKL11] M. Yuen, I. King, and K. Leung. A survey of crowdsourcing systems. In Proc. of IEEE Interna0onal Conference on Social Compu0ng (SocialCom’11), pages 766–773, Boston, MA,
- 9. • Earthquakes • Hurricanes • Tornadoes • Energy/utility outages • Fire hazards • Hazardous materials releases • Terrorism/ Emergency Use Cases 9
- 10. Emergency Response During Fire accidents can cause electric power failure. Mobile broadcast can be used to provide direc;ons to the users about rescue opera;ons. 10
- 11. Emergency situa;on Automa;c Altering can be used to inform family, rescue teams, or nearby cars / passengers in case of accidents. Emergency Response 11
- 12. Sensing -‐> Sensemaking Alert System Severity Personal Sensing to indicate Fall detec0ons, injury severity, alerts in old age people to provide scalable health care 12
- 13. Sensing -‐> Sensemaking Radia0on field near Fukushima Crisis Map Showing Latest Informa0on Hazardous gas in campus Spa0al Field Sensing With Mobile Sensors 13
- 14. Sensing -‐> Sensemaking • Avoiding congested streets in a city • Finding the most popular booth in a fair • Searching for the ride with shortest lineup in an amusement park 14
- 15. SenseMaking : Purpose & Goals u Simple and Easy-‐to-‐Use Framework for Sensing, Actua0on and Collabora0on using mobile phone u Powerful addi0onal sensing abili0es and features for community of users by community of users u Understand user and group context efficiently u Building energy-‐efficient collabora0on apps over exis0ng mobile pla?orms u Supported and empowered by community of users for community of user 15
- 16. The Problem – A cross layer, end to end issue § Several barriers and huge investment of 0me to build collabora0ve smart applica0ons § Lack of a framework to ease and speed the development of applica0ons § Non-‐Scalable, Ad-‐hoc, non-‐standardized API § Unsupported network infrastructure, and configura0ons 16
- 17. Solu0on to the Problem – Middleware Approach, Hierarchy for Scale… • Design and Develop and Open source distributed middleware framework suppor0ng collabora0ve mobile sensing • Provide API and libraries to perform: – Collabora0on – Virtual Sensing and Compressive Context Determina0on – Computa0onal Offloading – Cloud interface for scalability 17
- 18. Middleware Pla?orms and Techniques for Sensemaking • On phone, on broker (SenseDroid, SATWARE) • Techniques implemented in middleware – Compressive and Collabora0ve Sensing – Virtual Sensing for Sensemaking – Seman0cs Driven Sensing and Actua0on • Combining In-‐situ Sensors with Mobile Crowdsensing 18
- 19. Combining In-‐situ Sensors with Mobile Crowdsensing Pushing toward more interven0on • For sensing tasks not covered by any in-‐situ sensors – Try opportunis0c and par0cipatory sensing using nearby mobile users • What if there are no nearby mobile users • Pushing toward even more interven0on à Crowdsourcing 19
- 20. Explosion of Contextual Data Delivery 20 Emergency response Transporta0o n ~2.5 M mobile apps Entertainment Mobile social networks Healthcare Shopping Apps have various performance needs (reliability, ;meliness, quality…)
- 21. Explosion of Contextual Data Delivery 21
- 22. Explosion of Contextual Data Delivery 22
- 23. SenseDroid Architecture … Mobile Users … … Internet /Public Cloud Middleware Broker Wi-‐Fi AP 3G AP Query/ Response Cloud Users • Use compressive sensing with computa0onal offloading for energy-‐ efficiency • Use collabora0on for addi0onal and efficient sensing abili0es • Leverage reconstruc0on abili0es of compressive sensing to improve robustness and reliability 23
- 25. SenseDROID Distributed Middleware APPS$1$ Communica.on$ Sensing$&$ Sampling$ Context$ Processing$ &$Fusion$$ Query$+$ Storage$ Manager$ Privacy$&$ se>ngs$ Communica.on$ Sensing$&$ Sampling$ Context$ Processing$ &$Fusion$ Query$+$ Storage$ Manager$ Privacy$&$ se>ngs$ Query$&$$ Response$ Analysis$&$Processing$ Query$+$ Storage$ Communica.on$ Collabora.on$ Data$Collec.on&$ Comp.$Sampling$ Infrastructure$Sensing$$ Manager$ S1$ S2$ Sm$…….$ Query$ Response$ …….$ Query$&$$ Response$ Infrastructure$Sensors$ Mobile$Node$ Broker$ Mobile$Node$ APPS$2$ APPS$N$ Cloud AP S1$ Sn$ S1$ Sn$ 25
- 26. Sensemaking Using Compressed Sensing • A random sampling technique that can represent Sparse signal with few random measurements • Represents a Sparse Signal with few salient coefficients in a transformed domain • Integrates sensing, compression, processing based on new uncertainty principles 26
- 27. Collabora0ve Compressive Sensing Sink Node(Broker) Mobile NodeSampled Mobile SensorLegend No#of#Measurements## Reconstruction##Error#(MSE)# Number of Measurement Accuracy of Sensemaking Number of Measurement Energy Consumed in Sensing Accuracy of Sensemaking Scalability and Coverage Traded-‐off 27
- 28. Sensemaking using Virtual Sensing Ambient Light 3D Magnetometer 3D Accelerometer Barometer Processing ( Compressed Sensing and Calibration) Sensor Fusion 3D Gyroscope Ambient Light Barometer Thermometer Accelerometer Gyrometer Inclinometer Orientation Compass Physical Devices IsDriving IsRunning IsWalking IsSitting AtHome InOffice IsIndoor IsAlone hasFallen IsHappy Virtual SensingProcessing Sampling & Data Collection (Compressive Sampling, Adaptive Sampling) Location Contexts Activity Contexts Context Processing Social Contexts Emotional Contexts Environmental Contexts Health Contexts 28
- 29. Research Direc0ons • Energy Efficiency – Exploit collabora0ve & compressive sensing for energy efficiency • Incen0ve Mechanisms – Device incep0ves for par0cipa0on and collabora0on • Privacy Regula0on – Facilitate privacy preserving incen0ves • Heterogeneity in Mobile Cloud – Use and exploit heterogeneity of sensors and devices 29
- 30. 30 RELATED WORK REVIEW • Energy-Efficient Smart Spaces - Smartphone Augmented Infrastructure Sensing • Optimizing Event Detection on Smartphones • Spatial-temporal Information Gathering using Smartphones
- 31. Smart Spaces • Difference scales of intelligent systems: such as ci0es, stadiums, airports, building, and roads • Ci0zens of a smart space are not observers but ac0vely help the officials to make the space berer, e.g., – Safer – More entertaining – More energy efficient – More situa0on-‐aware • Similar to smart home, but across mul0ple users 31
- 32. Pla?orm for Public Smart Spaces • Goal: develop a pla?orm to provide safety with sustainability for smart spaces • Detec0ng many events in an energy-‐efficient way – Security related events: fights riots, protests, and demonstra0ons – Hazardous events: fires, chemical leaks, and stampedes – High crowd levels for poten0al conflicts London School of Economics’ app that monitors crowd safety at events 32
- 33. Limita0on of Current Approach State-‐of-‐the-‐art: Infrastructure sensing using in-‐situ sensors – High installa0on and maintenance cost – Insufficient node coverage ß limited budget – Does not scale! ß for crowded events 33
- 34. Usage Scenario #1 • Task: Sensing temperature at CS building • What if there is no working thermometer at the CS building? – Infer the temperature by nearby buildings – Infer the temperature provided by 3G/4G smartphone users walking by the CS building 34
- 35. Usage Scenario #2 • Task: Traffic surveillance for safety applica;ons • What if the fixed surveillance videos are insufficient ? – Leverage videos from nearby in-‐situ cameras – Leverage videos captured by police officers, fire fighters, and EMTs – Leverage large volume of user-‐generated, geo-‐tagged videos captured by ci0zens 35
- 39. Challenges • How to efficiently carry out the sensing requests? • How does the broker assign the requests to workers? • How to guide workers to the correct sensing loca0on? • How to efficiently process the raw sensory data? • Where to process the raw sensory data? • Can we leverage mul0ple close-‐by sensors for higher accuracy? 39
- 40. 40 RELATED WORK REVIEW • Energy-Efficient Smart Spaces - Smartphone Augmented Infrastructure Sensing • Optimizing Event Detection on Smartphones • Spatial-temporal Information Gathering using Smartphones
- 41. Event Detec0on on Smartphones • Each event may be detected by mul0ple subsets of sensors ß subop0mal sensor subsets? – E.g., traffic jam may be detected by GPS, accelerometer, or GPS + accelerometer • Mul0ple events may be (par0ally) detected by the same sensors ß uncoordinated sensor usage leads to redundant sensor ac0va0on – E.g., earthquake may also be detected by accelerometer • Problem: how to select efficient sensing strategies 41
- 42. Context-‐aware Mobile Applica0ons • Increasingly more context-‐aware apps leverage the smartphone sensors for berer user experience • What is context-‐aware? – Essen0ally inferred from sensor readings! 42
- 43. An Equivalent Research Problem • Context-‐aware apps may – Infer the same context using various combina0ons (sets) of sensors – Impose diverse accuracy requirements • How to select efficient sensing strategy? – Sa0sfy all apps’ requirements – Minimize energy consump0on • Proposal: OSM (Op0mal Sensor Management) middleware 43 OSM Middleware
- 44. OSM Middleware • It sits between apps and hardware • Apps may register or unregister requests through an API at any 0me. • Our middleware is response to – Maintain a database of ac0ve requests – Determine what sensors to ac0vate at what 0me 44
- 45. System Architecture 45 API: 1. Register()/Unregister() 2. Feedback() Request Manager 1. Manages a Request Queue 2. Preprocess the contexts Context Analyzer 1. Context Updater 2. Model Trainer Resource Manager 1. Barery Monitor 2. Scheduling Algorithm System Model • Combina0on/Accuracy/ Energy • Coordinated and efficient sensor usage! • Avoid redundant energy waste!
- 46. How to Op0mally Schedule Sensor Ac0va0ons? • Tradeoff between accuracy and energy consump0on • Our scheduling algorithms have to pick the best combina0on for all requests • The already-‐on sensors have to be considered 46 What if WiFi is already on?
- 47. Our Proposed Scheduling Problems Two op0miza0on criteria: – Energy Minimiza;on (EM) Schedule with the lowest energy to sa0sfy all the apps’ requirements – Accuracy Maximiza;on (AM) Schedule with the highest overall accuracy under an energy budget 47
- 48. Energy Minimiza0on (EM) Formula0on 48 Minimize energy Sa0sfy accuracy requirements
- 49. Within energy budget Maximize accuracy Accuracy Maximiza0on (AM) Formula0on 49
- 50. Proposed Scheduling Algorithms • Energy Minimiza;on Algorithm (EMA) Accuracy Maximiza;on Algorithm (AMA) • Good performance • Suitable for smaller problems due to high complexity • Efficient Energy Minimiza;on Algorithm (EEMA) Efficient Accuracy Maximiza;on Algorithm (EAMA) • Shorter running 0me • More suitable for smartphones • Inspired by two approxima0on algorithms for the weighted set cover and 0/1 knapsack problems ß But the approxima0on factor proofs do not work in our problems 50
- 51. Our Simulator • We developed an event-‐driven simulator in Java • Baseline algorithm – Selects the sensors for the highest accuracy of each context • We compare the scheduling algorithms: – Op0mal : EMA/AMA – Heuris0c : EEMA/EAMA – Baseline • Collect running apps in Android ac0vity stack from 5 users for three weeks • Measure power consump0on on a Samsung Galaxy S 8
- 52. Energy Saving • Save at least 40%, compared to the baseline • EEMA achieves a small gap of ∼2% than EMA • EMA terminates in 50ms and EEMA terminates in 1ms 9
- 53. Accuracy Improvement • Increase accuracy by up to 39.06% than the baseline • EAMA achieves a gap of ~1% than AMA • AMA terminates in 5000ms and EAMA terminates in 1ms 53
- 54. More Restricted Environments Lead to Higher Gains 54 Lower Accuracy Requirement Less Energy Budget Save More Energy Higher Accuracy Boost
- 55. Larger Problems Result in Higher Gains 55 Save More Energy Higher Accuracy Boost
- 56. Real Prototype System • Implement two heuris0c algorithms and the proposed OSM on Android • EEMA – Prolongs barery life two 0mes – Achieves accuracy : 93.94% • EAMA – Prolongs barery life 1.5 0me – Achieves accuracy : 94.85% 56
- 57. Summary • We propose an Op0mal Sensor Management middleware • Four algorithms with different op0mal criteria and complexity levels for sensor scheduling • EEMA (EAMA) saves energy (boost accuracy) in real-‐0me • Real implementa0on on smartphone • Designed for a single smartphone, but the same sensor management mechanisms may be used for event detec0on in smart spaces 57
- 58. 58 RELATED WORK REVIEW • Energy-Efficient Smart Spaces - Smartphone Augmented Infrastructure Sensing • Optimizing Event Detection on Smartphones • Spatial-temporal Information Gathering using Smartphones
- 59. Geospa0al Informa0on Gathering • A new class of crowdsourcing systems • Requesters: companies and organiza0ons • Submit geospa0al and temporal-‐dependent tasks (specific 0me and loca0on) • Task: capturing videos/pictures or collec0ng sensor readings • Workers: smartphone users • Report their des0na0on and deadline • They wouldn’t mind to take some detour routes for small rewards 59
- 60. Detour Planning Problem • Sample scenario: A smartphone user who needs to get to the Chia-‐Yi HSR Sta,on at 7 p.m. may have a few hours to spare. Why not making some money? – But it’s hard for a person to come up with the detour path • Our problem: How to find the best detour path for each worker – to maximize the profit (= rewards – costs) – while guaranteeing on-‐0me arrival at the des0na0on 60
- 63. Problem Formula0on Maximize overall profits Start and end points No rep. feasible spots Arrive des0na0on in 0me Visit each request once Start 0me of each request Finish 0me of each request 63
- 64. Orienteering Problem with Time Window • A similar problem – Goal: maximize the score – Game: players go to specific spots, and finish the predetermined job for a reward – Not exactly the same: (1) mul0ple feasible spots and (2) travel cost (gas and car deprecia0on) • We enhanced a dynamic programming based OPTW algorithm [GS09] for an op0mal Detour Planning (DP) algorithm – Runs in polynomial 0me: O( N3Z3 ) 64 [RS09] Decremental state space relaxa0on strategies and ini0aliza0on heuris0cs for solving the orienteering problem with 0me windows with dynamic programming. Computers and Opera0ons Research, 36(4):1191–1203, April 2009.
- 65. Collec0ng Feasible Spots • Find 25 landmarks in Taipei (hrp://taipeitravel.net) and Vancouver (hrp://hotels.com) • Use Flickr API to download the pictures tagged with each landmark, and retrieve the longitude/ la0tude • Use hierarchical clustering algorithm to group these photos at the granularity of blocks (~100 m) ß gives us the feasible spots • Employ Google map to compute the distance between any two feasible spots 65
- 66. Simulator Implementa0on • We implement a trace-‐driven simulator in C++ • It supports five algorithms – The proposed DP algorithm – Four heuris0c algorithms • Highest-‐Reward (HR) ß mimic human behavior • Closest-‐Request (CR) ß mimic human behavior • Highest-‐Reward with On0me (HROT) • Closest-‐Request with On0me (CROT) 66
- 67. Simula0on Design • Parameters – N: number of requests: {5, 10, 15, 20, 25} – T: deadline: {1, 2, 4, 8, 16} (hr) – C: travel cost: {0, 0.06, 0.12, 0.24, 0.48} ($/km) • Metrics – Total rewards – Running-‐0me – On0me-‐ra0o 67
- 68. On0me Ra0o HR and CR (mimicing humans) à low on0me ra0os! 68
- 69. Total Profits • Although HROT and CROT guarantee on0me arrival, they suffer from low profits • Compared to HROT and CROT, DP doubles the rewards with 25 requests – More requests à larger gap! 69
- 70. DP is Efficient • Terminates in less than 60 ms • Slower for Vancouver (right) ß more feasible spots 70
- 71. Implica0on of Travel Cost • Higher profits when per-‐km cost is lower 71
- 72. Summary • Studies a new class of crowdsourcing problems – Geospa0al informa0on gathering • Proposes an op0mal detour planning algorithm based on an OPTW algorithm • Simula0on results are encouraging • Poten0al Extensions – Implemen0ng a working prototype – Guide the workers to shoot photos using augmented reality – Quality assurance and cheat detec0on mechanisms • Designed for collec0ng spa0al-‐temporal mul0media informa0on, but can be extended for event detec0on 72
- 73. Ques0ons? 73 Challenges to realize smart spaces • How to efficiently carry out the sensing requests? • How does the broker assign the requests to workers? • How to guide workers to the correct sensing loca0on? • How to efficiently process the raw sensory data? • Where to process the raw sensory data? • Can we leverage mul0ple close-‐by sensors for higher accuracy?