1. Recommender Systems for
Analysis Applications
Roger Bradford
Agilex Technologies
14 April 2014
International Information Conference on
Search, Data Mining and Visualisation

2. 2
• Customers who Shopped for ' A Tale of two
Cities' also Shopped for ….
• Customers Who Bought Items in Your Recent
History Also Bought ….
• Users who Enjoyed Titanic also Enjoyed ….
Recommender Systems in
Internet Commerce

3. 3
Provider Items Recommended
Amazon Items to Buy
FastWeb Scholarships
LeShop Groceries to Buy
Netflix Movies to Rent
Pandora Music to Listen to
Tripadvisor Places to Visit
Twitter People to Follow
YaHoo Movies to Watch
Popular Commercial
Recommender Applications

4. 4
• Business Strategy Development
• Investment Analysis
• Risk Analysis
• IP Analysis
• Fraud Detection
• Event Monitoring
• Technology Monitoring
Example Analysis Applications
In Analytic Applications, Recommender
Systems Primarily Function as
Knowledge Discovery Tools

5. 5
Value of Recommender Systems for Analysis
Automatically Identify Important Information in
Large Quantities of Incoming Data
Reduce the Cognitive Load on Analysts
Aid in Discovery of New Relevant Information
- that the User didn’t Know to Search for
Produce Alerts about Entities of Importance –
not just more Documents to Read

6. 6
Typical Commercial
Applications
Typical Analytic
Applications
# of Users >> # of Items # of Items >> # of Users
User Interests are Fairly
Stable
User Interests are
Dynamic
Unambiguous Indicators
are Available
Indicators are Mostly
Subtle
Missing a
Recommendation
Typically has Small Impact
Missing a
Recommendation may
have a Large Impact
Recommender Application Differences

7. 7
Approach Recommendations
Based on
• Collaborative Filtering Actions of Other People
• Content-based Characteristics of Items
• Demographic User Characteristics
• Knowledge-based Example Cases or
Constraints
• Community-based Social Networks
• Hybrid Combinations of the
Above
Implementation Approaches

8. 8
Incoming
Reporting Stream
Recommender
Engine
User-provided
Exemplars
Xxxxxxxxx
Xxxxxxxxx
.criminal
Xxxxxxxxx
...crime..
Recommended
Documents Recommended
Entities
User
Action
Artifacts
Jason Brown
Robert Fisher
Walter Williams
Analytic Recommendation Process

9. 9
Example User Interface
Example
Documents
used to
Define
Interests
Recommended Items in Relevance Order
Confidence
Indictors
A 2011 report issued by the US
Geological Survey and US
Department of the Interior,
"China's Rare-Earth Industry,"
outlines industry trends within
China and examines national
policies that may guide the future
of the country's production. The
report notes that China's lead in
the production of rare-earth
minerals has accelerated over the
past two decades. In 1990, China
accounted for only 27% of such
minerals. In 2009, world
production was 132,000 metric
tons; China produced 129,000 of
those tons. According to the
report, recent patterns suggest
that China will slow the export of
such materials to the world:
"Owing to the increase in
domestic demand, the
Government has gradually
reduced the export quota during
the past several years." I
User Feedback Mechanism
Exemplar
Management
Console

10. 10
Key Requirements for
Analytic Recommenders
Quickly Identify and Present Desirable
Information to the User without Overwhelming
the User with Irrelevant Information.
Be Flexible Enough to Deal with Variability in
Individuals and Activities
Evaluate Complex Associations Based on
Multiple Attributes (Including Metadata)
Incorporate Data from Multiple Sources.
Begin Making Recommendations Based on
Small Amounts of Data

11. 11
Accommodate Data Volumes that can be
Expected to be Very Large
Deal with Data that is Sparse, Incomplete,
and Noisy.
Make Explanations of the Reasoning Used in
Reaching the Recommendations Available to
the User.
Work with Data from Existing Corporate or
Government Data Repositories.
Key Requirements for
Analytic Recommenders (Cont’d)

12. 12
• # of Items >> # of Users
• Dynamic Items & User
Interests
• High Accuracy & Low Miss
Rate Requirements
Requirements Drive Implementation Approach
Primary
Recommendation
Technique must be
Content-based
Matrix Factorization is the best Available
Content-based Approach

13. 13
100 Million Ratings of 17,770 Movies by >
480,000 Users
$1Million (US) Prize for 10% Improvement
44,000 Entries, From Over 41,000 Teams
Won by Koren and Bell using a Combination
of Techniques, Featuring Matrix Factorization
The Netflix Challenge

14. 14
Matrix Factorization Advantages*
Prediction Accuracy Superior to Other Techniques.
Use of a Memory-efficient, Compact Model.
Simple Training.
Natural Ability to Integrate Multiple Forms of User Feedback.
Ability to Incorporate Temporal Dynamics of User Interests
and Item Attributes.
No Reliance on Arbitrary or Heuristic Similarities.
Inherent Protection against Overfitting.
Ability to Capture the Totality of Weak Signals in the Data.
Ability to Incorporate Confidence Levels.
High Scalability.
*Koren & Bell, Recommender Systems Handbook, Springer, 2011

15. 15
RecommendationAccuracy
ComparedtoBaseline
Degree of Text Corruption
Noise Resilience

16. 16
Search Terms
Viewing an Item
Time Spent Viewing
an Item
Saving an Item
Printing an Item
Refining User Interests
Explicit Input Implicit Indicators
Exploit both Positive
and Negative Indicators

17. 17
• Accuracy
• Confidence Indicators for Recommendations
• User Control
• Explanation
Contributors to User Confidence

18. 18
Explainability - Documents

19. 19
Explainability - Entities

20. 20
Lists
Tables
Text
Analyst’s Notes:
Identified
Relevant
Documents
Documents
In
Novelty
Order
Previously Seen Information
Published
Reports
Previously
Reviewed
Documents
Novelty in Recommendations

21. 21
Crosslingual Recommendations
Documents in
Multiple Languages
Farsi
Arabic
English
Recommendations
in Relevance Order
Recommended Items

22. 22
Accuracy+Completeness
ofCategorization
Number of Simultaneous Languages
English Documents &
English Examples
Documents in Latin Languages
& English Examples
Range of
Human
Performance
High-Accuracy Multilingual
Recommendations

23. 23
Multimedia Recommendations
Integrated
Semantic Analysis
Structured Data
Images
Text Audio
8/18/02500 lbPicric
Acid
Saif al
Adel
Zaid
Khayr
DateAmountMaterialSellerBuyer
Sensor Data
Video
Geospatial Data
Biometrics

24. 24
High Performance with Modest HardwareTimeinHours
Number of Documents
K K KK K
Minimum Latency –
Single Processor
Maximum Throughput –
16-node Hadoop Cluster

25. 25
Algorithm Scalability
Conversational Recommender Systems
Context-aware Recommenders
Explanations and Evidence
Preference Elicitation
Privacy and Security
Semantic Web Technologies for
Recommendation
Trust and Reputation
Recommender Topics of Current High Interest

26. 26
The ACM Recommender System Conference
(RecSys 2014), Foster City, California, USA, 6-10 October 2014
Recommender Systems Handbook, F. Ricci, L. Rokach,
B. Shapira, and P. Kantor, Springer Publishing, 2011 118€
Recommender Systems, P. Melville and V. Sindhwani, In
Encyclopedia of Machine Learning, Springer, 2010. Available at:
http://www.prem-melville.com/publications/recommender-systems-
eml2010.pdf
Matrix Factorization Techniques for Recommender
Systems, Y. Koren, Y., R. Bell, and C. Volinsky, IEEE
Computer, August 2009, pp. 42-49. Available at:
http://www2.research.att.com/~volinsky/papers/ieeecomputer.pdf
Resources

27. 27
Questions or Comments
Roger Bradford
Agilex Technologies Inc
r.bradford@agilex.com