This document discusses IBM's development of a first-of-a-kind big data solution for telecommunications companies. It describes IBM Research activities related to advanced analytics using telco data. The solution includes an advanced analytics platform that derives insights from telco data through predictive analytics, behavioral analysis, and other techniques. It then discusses two use cases: using aggregated anonymous location data from telcos for city-scale transit optimization, and developing enriched consumer profiles through individual-level mobility analytics.
2. Please note
IBM’s statements regarding its plans, directions, and intent are subject to
change or withdrawal without notice at IBM’s sole discretion.
Information regarding potential future products is intended to outline our general
product direction and it should not be relied on in making a purchasing decision.
The information mentioned regarding potential future products is not a
commitment, promise, or legal obligation to deliver any material, code or
functionality. Information about potential future products may not be
incorporated into any contract. The development, release, and timing of any
future features or functionality described for our products remains at our sole
discretion.
Performance is based on measurements and projections using standard IBM
benchmarks in a controlled environment. The actual throughput or performance
that any user will experience will vary depending upon many factors, including
considerations such as the amount of multiprogramming in the user’s job
stream, the I/O configuration, the storage configuration, and the workload
processed. Therefore, no assurance can be given that an individual user will
achieve results similar to those stated here.
2
4. Agenda
• Motivation and Background
• IBM Research activities
• Advanced Analytics Platform
• Life Style Analytics
4
5. Telco and cross industry data to create a unified view of the
customer. Mobility information is becoming increasingly
valuable…..
Structured
Repeatable
Linear
Monthly sales reports
Profitability analysis
Customer surveys
Other
Industries
Other
Data
Industry
Reports
Retail
Social
Media
Data
Customer
• Segment
• Social
Network
• Demographics
• Sex,
Age
Group,
etc
• Tenure
• Rate
plan
• Credit
RaBng,
ARPU
Group
Device
• Class
• Manufacturer
• Model
• OS
• Media
Capability
• Keyboard
Type
TransacBons
• Voice,
SMS,
MMS
• Data
&
Web
Sessions
• Click
Streams
• Purchases
• Downloads
• Signaling,
AuthenBcaBon
• Probe/DPI
Network
• Availability
• Throughput/Speed
• Latency
• LocaBon
• FaciliBes
Interface
• Discovery
• NavigaBon
• RecommendaBons
Product/Service
• SubscripBons
• Rate
Plans
• Media
Type
• Category/ClassificaBon
• Price
Starts,
Stops
Success
Rates
Errors
Throughput
Setup
Time
ConnecBon
Time
Usage
Recency
Frequency
Monetary
Latency
Telco Data Cross Industry Data
5
6. Building Context and Intent from Location data
• Deriving location: location information may be derived using multi-
modal information
– CDR data, tower data, device data, Wi-fi etc.
– Accuracy of location information depends on data fidelity etc.
• Building context: making sense of the location information
– Correlate location information with business data
– Various other correlation rules may be used to build a rich context
• Inferring intent: infer consumer level intents by leveraging location
and mobility patterns
Deriving Location Inferring IntentBuilding Context
6
7. 7
Data
Cell tower locations
Wi-fi locations
Device locations
Device usage data – apps, web
sites
Customer data – demographics
Refined locations
Mobility Patterns
Hang outs
Hang outs correlated with
business locations
Mode of transportation
Traveling buddies
Analytics
7
8. 8
Type of Location Analytics…..
Habitual Journey
Patterns
Demographic
customer profiles
Common origins
and destinations
Direction of
travel
Level of Mobility +
Segmentation
Aggregated
Mode of transport
Average journey
times
Travel pattern
anomalies
Accurate
Location
Congestion
Real time traffic
incident flags
Optimal route
planning
Foot traffic Customer wait times
Individual
mode of transport
Possible with event data More detailed data required
VCC Board Morph Update, June 2011
9. First-of-a-Kind Program
§ Experimental technology-based
solutions engagements
§ Testing tomorrow’s innovations on
today’s business problems
§ Yielding prototype solutions across a
range of industries
§ Creating valuable intellectual capital
for IBM’s portfolio
§ Value to IBM Clients
– Early market advantage
– Access to world class researchers
9
10. FOAK Deliverables
• Early thought leadership and experiences
with new technologies
• Working prototype of an innovative
solution not yet available in the
marketplace
• The know-how to improve a business
process or solve a problem
• Software components, methodologies
and tools
• Press & media coverage
10
11. Agenda
• Motivation and Background
• IBM Research activities
• Advanced Analytics Platform
• Life Style Analytics
11
12. Two Scenarios: Aggregate and Individual
• Aggregate Anonymized Analytics: Sensing City-scale
people movement from Telco data and leveraging for Transit
Optimization
• Enriched Consumer Profile: Customer Analytics with
Mobility Profiles from Telco Data
12
13. Enriched Consumer Profiles for Enabling Telco
Data Monetization
• We develop enriched consumer profiles by deriving insights
about consumer preferences, life style, and intent from location,
mobility and call data joined with use case appropriate data
sources.
• Enriched consumer profiles are utilized to enable new services
and effective campaign through targeted segmentation.
13
14. Two Scenarios: Aggregate and Individual
• Aggregate Anonymized Analytics: Sensing City-scale
people movement from Telco data and leveraging for Transit
Optimization
• Enriched Consumer Profile: Customer Analytics with
Mobility Profiles from Telco Data
14
15. Sensing City Scale People Movement from Telco Data
Cities Demonstrated: Istanbul (Turkey), Dubuque (USA) for Transit
Optimization and a series of subsequent client pipeline
Challenge Cities have very little real understanding of where citizens, goods and
transportation move during the day. Without this information it is
difficult to accurately plan and manage the usage of roads and
infrastructure.
Solution Using a variety of real time data from “smart phones”, GPS devices,
terminals, traffic cameras, public transportation schedules and transit
data, develop models of zonal density, flow of goods and origin /
destination pairs. From these models, drive processes to manage this
flow against a specific objective.
Benefits Evaluates the efficacy of existing transit system and transportation
infrastructure; provides the structure for design incentive strategies to
win new riders – information, incentives, services; optimize fleet
operations in situations where demand outpaces supply; manage
revenue through better zoning and permits. comprehensive solution
that will address the management of congestion, fleet management,
people attending events, and multimodal transit
1515
17. Identifying Meaningful Locations
Where People Live Where People Work
Istanbul Movement Analysis
- 4.7 million phones w. 3B+ events/week
- Accurate detection of home, work & meaningful locations
17
18. Traffic Monitoring
Uses basic analytics building blocks already seen to display time based
traffic flow levels mapped to city road system. A snapshot at 8:30am:
18
20. Feeder Bus Route Optimization for M4 Metro
Line on Anatolian side of Istanbul
Feeder bus routes based on demand to 4 metro stations on Kadikoy-Kartal metro line
20
21. Optimal Bus Stop Location Design
• Stops are added by
considering the greatest
potential demand for transit
and accessibility at origin and
destination
• Some stops are added to far
places in which demand to the
area already served by
existing stops is potentially
large
21
22. Two Scenarios: Aggregate and Individual
• Aggregate Anonymized Analytics: Sensing City-scale
people movement from Telco data and leveraging for Transit
Optimization
• Enriched Consumer Profile: Customer Analytics with
Mobility Profiles from Telco Data
22
23. Consumer Analytics with Enhanced Consumer
Profiles
• Derive advanced location/mobility attributes and patterns from Telco data to
enrich consumer profiles with mobility context
• Derive predictive model about consumers location and mobility patterns
• Leverage enriched consumer profiles for data monetization opportunities by
correlating and joining other data sources
• Build an operational asset on IBM Big Data platform to enable Telco to
extract mobility attributes and patterns efficiently
23
24. Set of example mobility attributes
• Base set of example mobility attributes
– Home and work location
– Weekday top locations
– Weekend top locations
– Meaningful location detection
– Classification of where and when time spent
– Detecting tourism pattern
– Detecting specified habits related to mobility
– Trip purpose
– Anomaly in mobility from baseline patterns
– Detecting who’s who in the household based on mobility pattern
• Advanced predictive models (Next Best Location)
– Likely place a person would be at a future time
– Likelihood of a person going to a Mall during this weekend
– When this person is likely to be a tourist
24
25. 25
Enhanced Micro-segmentation with Mobility Model
Mobility
Patterns
Buying
Patterns
Social
Patterns
Demographics
• Gender
• Age group
• Address
• Income
Historical buying patterns
Social network
influencers
Mobility Model
• Location and movement pattern (space,
time)
• Meaningful location detection
• Meaningful location classification
• Trip purpose
• Estimated Duration of stay
• Estimated Duration of travel
• Mode of travel
• Calling patterns
• Detecting tourist patterns
• Detecting student patterns
• Estimated demographic profile of user of
phone
• Anomalies in regular patterns
Enhanced Attributes for Customer Segmentation
26. Retailer Customer Profile
Real Time Targeted Advertisement for IPTV
AAP
(Advanced
Analytics
Platform)
3 - AAP catches the
new football interest
flag, his frequent
sports shopping, and
in realtime matches
Tom’s profile with an
offer for 20% off
coupon to an Nike
store.
4 - Tom is also an
existing SMS Opt-
In mobile cust.
5 – Tom receives
targeted IPTV
advertisements based
on his IPTV, mobility
and social profiles
2 - Tom is channel surfing,
mostly sports channels,
primarily football games where
Nike advertises a lot (AAP enhances
his customer profile, after 10 football
games viewed in 1st month,
with an interest flag as a “football fan”)
Enhanced Cust. Profile
Interest / Mobile # / Email
1- Tom activates IPTV service
with the America 50 package and
adds the ESPN sports ala carte
option (we have an initial
customer profile with his fixed #
and a mobile#)
A la carte option
Sports Packages
tom@gmail.com
212-‐201-‐1234
Language
Package
26
27. Location Based Real Time Offering on Mobile Phone
Lisa
4 - AAP catches that
Lisa is entering a mall,
and matches her
“Fashion” interest flag
and “Perfume”
preference, sends in
realtime an offer for
20% off coupon for
Byonce fragrance at
Sephora in that mall.
5 - Lisa receives
an SMS/email/App
notification that
her mobile app
account contains a
new offer for
Beyonce perfume.
Beyonce Fan Page
2 - She follows a
friend’s post on FB and
clicks the Like button on
the Beyonce Fan Page.
3 - Lisa’s IPTV viewing
& mobile clickstream
behaviors set her Interest
flag to “Fashion” and one
preference to “Perfume”.
6 - Lisa uses
the mWallet
app on her
smartphone to
purchase some
perfume at POS
via NFC.
1- Lisa is a mobile subscriber
with Telco and downloads the
mobile app and agrees to receive
offers related to her interests.
AAP
(Advanced
Analytics
Platform)
Retailer Customer Profile
Enhanced Cust. Profile
Interest & Preference
IPTV a la carte option &
Mobile Features/Apps
IPTV Lang
Pkg &
Mobile Pkg
27
28. Agenda
• Motivation and Background
• IBM Research activities
• Advanced Analytics Platform
• Life Style Analytics
28
29. 1
2
3
Advanced Analytics Platform
End-use
Applications
Analytics
Visualization
Big Data Analytics
Warehouse
Predictive Analytics
Sens
e
Analyze Act
Search / Explore
KPIs
Dashboards
Drill-Downs
Reports
Marketing
Campaigns
Rules Engine
Behavioral
Analysis
Outcome
Optimization
Propensity
Scoring
Model
Creation
Structured /
Unstructured
Data
Data Governance
Data Integration
ETL/ELT
ChangeCapture
DataQuality/Validity/Security-Privacy
Format/UnitConversion
Consolidation/De-duplication
DataRepositories
Network
Data
Customer
Behavior Data
Customer
Data
ProductDataNetworkTopology
Data
ContinuousFeed
Sources
Usage Data
Reference
Data
Historical
Analysis Data
Demographics
Segmentation
Location
Past Actions
Propensity
Scores
Behaviors
Predictive Model
Deployment
Actionable
Insight
Stream Processing
Streaming Data
Operational
Systems
4
5
AAP Capabilities
High Performance Historical analysis (Big Data Platform)
Model Based Analytics - behavioral scoring, micro segmentation,
correlation detection analysis
Real-time scoring, classification, detection and action
Visualize, explore, investigate, search and report
Take action on analytics
IBM’s Advanced Analytics
Platform (AAP) Supports Use
Cases across the business
with New Era Capabilities
Create new Services
and Business Models Transform Operations
Build Smarter
Networks
Personalize Customer
Engagements
1
1
2
3
4
5
5
29
30.
Social
Informa-on
Loca-on
Informa-on
Customer
database
informa-on
InfoSphere Streams
Low Latency Analytics for streaming data
InfoSphere BigInsights
Hadoop-based low latency analytics for
variety and volume
IBM Netezza
BI and Ad Hoc Analytics
Structured Data
Customer database
Coremetrics
Low Latency Analytics for streaming data
Data sources…..
30
31. The carmel
frappuccino
in starbucks is
just heavenly.
IBM
BigInsights
Text
Analy-cs
Accelerators
BigInsights
Custom
analyBcs
First
Name:
Joe
Last
name:
Smith
Address:
1234
Anyroad
….
[
X
]
Likes
coffee
[
X
]
Likes
frappuccino
[
]
Likes
cappuccino
….
[
posiBve]
SenBment
coffee
Social Media Profile Creation
31
32. URL Analysis- Extract Implicit User Profile
analysis"
URL Analysis: for each user,
report the most meaningful
interests to describe her profile.
Large scale analysis
Update users
profiles"
Consume"
Adaptive user
segmentations: create
new users segmentation by
clustering similar interests
Data Cleansing
32
34. Collecting and analyzing in real-time millions of events from multiple sources to detect the
right time to respond to the event
CDRs
Billing
CRM
Location
Account Mgt
Internet
Network
Millions of events
per second
Microsecond Latency
Dropped Calls
Outgoing International Calls
Call Duration
Extra Call
Contract Expiration
Entered new cell
New Top-Up
5 minutes left on pre-paid
EDW
Invoice Issued
Predictive Models
3 dropped calls in 10 minutes
Customer is close to a store
Customer entered a shopping area
Invoice paid + called competitor
Smart phone browsing pattern
Customer is watching a video
Congested Cells
Invoice Paid
Acquired new products
Change contracts
Brand Reputation
Customer Sentiment from Social network
Customer is roaming
Customer is at home
Campaign
ManagementInvoke appropriate campaign
Score
Real-time Stream Analytics
34
37. Agenda
• Motivation and Background
• IBM Research activities
• Advanced Analytics Platform
• Life Style Analytics
37
38. Determining Buddies, Hangouts, Life Style
Example Lifestyle Attributes for marketing demonstration
§ Subscriber Lifestyles
§ Popular Locations
§ Subscriber Pairings
Who Are You?
Homebody
Daily Grinder
Delivering the Goods
Globetrotter
Nomad
10 Top Hangouts
Best Buddies
Next Steps
• Given the lifestyles, popular locations, and best buddy data => predict where
individuals or groups of similar individuals will be and when.
• Use time series modeling and clustering we can create time/location based marketing
campaigns targeted at homogenous groups in specific locales.
38
40. What are Profiles
• Lifestyle Profiles are defined by marketing analysts for specific
use cases or marketing programs
• Usage Profiles are created using data mining algorithms and
define how a person uses services during the day
• Location Affinity is created with algorithms and determines
preferred locations for individuals throughout the day and week
• Together these uniquely define a person with relation to how
the retailer or marketer might want to market to them
40
41. Creating Groups of Mobility Profiles Enables
Better Prediction for Certain Groups
l profiles breakdown like this
l Homebody, doesn't visit too many unique locations
l Daily Grinder, back and forth to work, quiet weekends, makes
stops along the way
l Norm Peterson, inside the lines, no deviations
l Delivering the goods, no predictable patterns, many different
locales during the day
l Globe Trotter, either not in town, or keeps their phone turned off
l Rover Wanderer, spends evenings at various location (sofa
surfers www.couchsurfing.org)
l “Other”, is a group hard to categorize
41
42. By Profile, when is it easy or difficult to predict
where they will be?
Profile Day Time Predictability
Daily Grinder Thursday Dinner Highest
Daily Grinder Friday Afternoon Lowest
Homebody Saturday Night Highest
Homebody Wednesday Morning Lowest
These are the 2 most predictable profiles, yet there is diversity in their predictability.
To best communicate with Daily Grinders, contact them on Thursday Afternoons just before dinner
42
43. Preferred Locations of by profile type at
Lunchtime Weekdays (Central Stockholm)
Delivering
the Goods
Night
Shifters
Daily
Grinders
43
44. What analysis is available (Anonymous
Data)
From the mobility profiles, summarized, anonymous analysis is
available
l Summarized to ensure anonymity, analysis of popular locations
by time of day and profile of subscribers is possible
l For retailers this information can help understand what types
of people are nearby at lunch time
l What types of people prefer which areas. Some obvious
results are Globe Trotters go to airports, Daily Grinders go to
office buildings. Other non-obvious results show up also.
l Are there predictable patterns that we can use to target
certain groups in the future?
44
45. What Makes this Possible?
l Using the power of Netezza and modeling capabilities of SPSS we
can literally throw all the data at data mining algorithms and create
discrete clusters of subscribers by activity, mobility
l Apply the data mining outputs to the entire subscriber base by
creating detailed specific analyses for each subscriber refined by the
mobility profiles
45
46. Enriched Consumer Profile Hub
Customer
Profile
Hub
IPTV
-‐
SubscripBon
Billing
-‐ VOD
Billing
&
viewed
-‐
channel
viewing
history
-‐ -‐
contents
purchased
-‐ Logs
&
Tuning
Events
-‐
package
subscripBon
Mobile
-‐
LocaBon
-‐
URL+App
Transac-ons
-‐
xDRs
and
inb.
roaming
-‐
RAN
(incl.
HLR/VLR)
-‐
Top
Up
-‐
Pkgs
-‐
Billing
-‐
SMS,
browing
URLs
Other:
-‐
Devices
-‐
Dealer
Network
-‐
Contact
Center
-‐
Call
Recordings
-‐
Trouble
Tickeing
-‐
Campaign
Results
(Imagine)
-‐
Loyalty
-‐
CompeBBon
Website
-‐
Retail
Transac-ons
Fixed
-‐
CDR
-‐
URL
(IP)
-‐ Radius
(IP-‐Cust)
-‐
Pkgs
-‐
Billing
Historical
TransacBons/
Events
Partners/Retailers
AdverBsers
Other/Internal
GIS
-‐
Business
map
and
numbers
-‐
Point
of
Interest
maps
Consumers
of
new
Insights
Feedback
Social
Media
Data
46
47. Agenda
• Motivation and Background
• IBM Research activities
• Advanced Analytics Platform
• Life Style Analytics
47
48. Thank You
Your feedback is important!
• Access the Conference Agenda Builder to
complete your session surveys
o Any web or mobile browser at
http://iod13surveys.com/surveys.html
o Any Agenda Builder kiosk onsite
48