2. Course Structure
Slide 2 www.edureka.in/mahout
 Module 1:
Introduction to Machine Learning
and Apache Mahout
 Module 2:
Mahout and Hadoop
 Module 3:
Recommendation Engine
 Module 4:
Implementing a Recommender and
Recommendation Platform
 Module 5:
Clustering
 Module 6:
Classification
 Module 7:
Mahout and Amazon EMR
 Module 8:
Project Discussion

3. How it Works?
Slide 3 www.edureka.in/mahout
 Live Classes
 Class Recordings
 Module wise Quizzes, Coding Assignments
 24x7 on-demand Technical Support
 Sample Application and Live Project
 Online Certification Exam
 Lifetime access to the Learning Management System

4. Module 1
Slide 4 www.edureka.in/mahout
 Mahout Overview
 ML Common Use Cases
 Algorithms in Mahout
 Mahout Commercial Use
 Mahout Summary
 Supervised and Unsupervised Learning
 Introduction of Clustering and Classification
 Similarity Metrics
 Similarity by correlation
 Similarity by distance
 Distance Measure Types

5. Mahout Overview
 Mahout began life in 2008 as a subproject of Apache’s Lucene project, which provides the well-known
open source search engine of the same name.
 Lucene provides advanced implementations of search, text mining, and information-retrieval
techniques.
 In the universe of computer science, these concepts are adjacent to machine learning techniques like
clustering and, to an extent, classification.
 As a result, some of the work of the Lucene
committers that fell more into these machine
learning areas was spun off into its own subproject.
 Soon after, Mahout absorbed the Taste open source
collaborative filtering project.
Slide 5 www.edureka.in/mahout

6. Mahout Overview
Apache Mahout and its related projects within the Apache Software Foundation
Apache
Slide 6 www.edureka.in/mahout

7. What are we going to learn today?
What is machine learning
Can a Machine learn
How to do it
Slide 7 www.edureka.in/mahout

8. Mahout : Scalable Machine learning Library
Machine Learning is a Programming Computers to optimize a Performance Criterion
using Example Data or Past experience
 Machine learning – what does it mean?
 A branch of artificial intelligence
 Systems that learn from data
 Classify data after learning
 Learn on test data sets
 Generalisation – the ability to classify unseen data sets
Slide 8 www.edureka.in/mahout

9. What is Mahout?
Collaborative Filtering
Clustering
Classification
Slide 9 www.edureka.in/mahout
 Apache Mahout is an Apache project to produce open source
implementations of distributed or otherwise scalable machine learning
algorithms focused primarily in the areas of collaborative filtering,
clustering and classification, often leveraging, but not limited to, the
Hadoop platform.
 The Apache Mahout project aims to make building intelligent
applications easier and faster. Mahout co-founder Grant Ingersoll
introduces the basic concepts of machine learning and then
demonstrates how to use Mahout to cluster documents, make
recommendations, and organize content.
Three specific machine-learning tasks that Mahout currently
implements are:
 Collaborative Filtering
 Clustering
 Classification

10.  Machine Learning is a class of algorithms which is data-driven, i.e.
unlike "normal" algorithms it is the data that "tells" what the "good
answer" is.
 Example:
An hypothetical non-machine learning algorithm for face recognition in
images would try to define what a face is (round skin-like-colored disk,
with dark area where you expect the eyes etc).
A machine learning algorithm would not have such coded definition,
but will
"learn-by-examples": you'll show several images of faces and not-
faces and a good algorithm will eventually learn and be able to predict
whether or not an unseen image is a face.
What is Mahout (Cont’d).
Slide 10 www.edureka.in/mahout

11. Mahout – How does it work?
Uses Hadoop
MapReduce
Supports four Use
Cases
Has many supplied
algorithms
Apache Mahout
Recommendation mining Clustering Classification Fixing Item Set Mining
Slide 11 www.edureka.in/mahout

12. Mahout Applications
Genetic
Freq. Pattern
Mining
Classification Clustering
Recommend
ers
UtilitiesLuce
ne/Vectorizer
Math
Vectors/Matri
ces/
SVD
Collections
(primitives)
Apache
Hadoop
Slide 12 www.edureka.in/mahout
Application

13. Mahout Overview
ML is all over the web today
Mahout has functionality for many
of today’s common machine
learning tasks
MapReduce magic in action
Mahout is about scalable
machine learning
Slide 13 www.edureka.in/mahout

14. Hello There!!
My name is Annie.
I love quizzes and
puzzles and I am here to
make you guys think and
answer my questions.
Slide 14 www.edureka.in/mahout
Annie’s Introduction

15. Annie’s Question
Q: What is Machine Learning?
Slide 15 www.edureka.in/mahout

16. Annie’s answer
A: Machine Learning is a branch of Artificial Intelligence.
Training the machine through data in such a way that the
machines can simulate human like decisions - TRUE
Slide 16 www.edureka.in/mahout

17. Q: Is Statistical Modeling equivalent to Machine Learning?
Slide 17 www.edureka.in/mahout
Annie’s Question

18. Answer: NO
Reason:- Statistical modeling is a way to identify the
relationships between variables through mathematical
equations. In statistical modeling the relation between the
variables is NOT deterministic rather Stochastic.
Whereas Machine learning uses Statistical modeling to train
the system and generate the model/functions
Slide 18 www.edureka.in/mahout
Annie’s answer

19. utilizes recommendation
systems to bring videos to a user
that it believes the user will be
interested in.
They are designed to:
 Increase the numbers of
videos the user will watch
 Increase the length of time he
spends on the site, and
 Maximize the enjoyment of his
YouTube experience.
Machine Learning Use Cases – You Tube
Slide 19 www.edureka.in/mahout

20. User Activity:
In order to obtain personalized recommendations, YouTube's recommendation system combines the
related videos association rules with the user's personal activity on the site.
This includes several factors:
 There are the videos that were watched - along with a certain threshold, say by a certain date.
After all, you don't want to count videos watched from 2 years ago if the user has watched enough
videos, most likely.
 Also, YouTube factors in with emphasis any videos that were explicitly "liked", added to favourites,
given a rating, added to a playlist. The union of these videos is known as the seed set.
 Then, to compute the candidate recommendations for a seed set, YouTube expands it along the
related videos.
Use Case – You Tube (Contd.)
Slide 20 www.edureka.in/mahout

21. Use Case – Wine Recommendation
Slide 21 www.edureka.in/mahout

22. Use Case – Wine Recommendation (Contd)
What wine will I enjoy? More than 2 million consumers turn
to the Internet for the answer to this question every day
Recommend
ation
Wine
Enthusiast
Robert
Parket
Wine
Spectator
Problem
• Mysterious ratings and adjective-based reviews do little
to help consumers decide which wine to buy
• They can't even agree amongst themselves
Solution
• Next Glass solves this problem by removing subjectivity
and applying science to deliver recommendations
based on your previous ratings
Slide 22 www.edureka.in/mahout

23. Use Case - Biometrics
Biometrics : The Science of establishing the identity
of an individual based on the physical, chemical or
behavioral attributes of the person.
Why is it Important?
 Identify Individual credentials
 Identify and prevent banking fraud
 Enforcement of law and security
Face
Voice
Vein
Retinal
Iris
Writing
Hand
Geometry
Finger Print
Slide 23 www.edureka.in/mahout

24. How Does a Fingerprint Optical Scanner Work?
A fingerprint scanner system has two basic jobs
 Get an image of your finger
 Determine whether the pattern of ridges and valleys in this image matches the
pattern of ridges and valleys in pre-scanned images
Process
 Only specific characteristics, which are unique to every fingerprint, are filtered are
saved as an encrypted biometric key or mathematical representation.
 No image of a fingerprint is ever saved, only a series of numbers (a binary code),
which is used for verification. The algorithm cannot be reconverted to an image, so
no one can duplicate your fingerprints
Slide 24 www.edureka.in/mahout

25. Use Case – Aadhaar
India is reportedly creating a biometric database to hold the fingerprints and face images for each
of 1.2 Billion citizens as part of its Unique Identification Project.
Slide 25 www.edureka.in/mahout

26. Use Case – Paycheck Secure System
All Trust Network Paycheck Secure System has enrolled over 6 Million users and over 70 Million
Transactions.
Slide 26 www.edureka.in/mahout

27. Computer Vision
Stimulate reality: Generate
complex, physically realistic
stimuli, while
maintaining precise control over
stimulus variables
Rigorous Theory:
Apply rigorous computational
principles to develop theories of
human visual perception
Multisensory
Perception
Develop Heuristics:
Create perceptually inspired “short
cuts” to increase efficiency, or
achieve advanced effects
Analysis for synthesis:
Application of segmentation,
shape-from-shading, machine
learning, etc. to rendering and
animation
Computer
Graphics
Biological Inspiration:
Imitate design principles of
biological systems to solve under-
constrained vision problems
Ground Truth:
Test vision algorithms on
computer generated images for
which all scene parameters are
known precisely.
Computer
Vision
Slide 27 www.edureka.in/mahout

28. Learning Techniques
Attain knowledge by study, experience, or by being taught.
Supervised
Learning
Unsupervised
Learning
Types of Learning
Slide 28 www.edureka.in/mahout

29. Supervised Learning
Slide 29 www.edureka.in/mahout
Supervised learning : Training data includes both the input and the desired results.
 For some examples, the correct results (targets) are known and are given in input to the model
during the learning process.
 The construction of a proper training, validation and test set (Bok) is crucial.
 These methods are usually fast and accurate.
 Have to be able to generalize: give the correct results when new data are given in input without
knowing a priori the target.

30. Example – Supervised Learning Model
Training
Text,
Documents,
Images, etc.
Machine
Learning
Algorithm
New Text,
Document,
Image, etc.
Expected Label
Feature
Vectors
Feature
Vector
Predictive
Model
LLaabbeelslsLabels
Slide 30 www.edureka.in/mahout

31. Example(Supervised Learning)
Slide 31 www.edureka.in/mahout

32. Unsupervised learning
Unsupervised Learning:
 The model is not provided with the correct results during the training.
 Can be used to cluster the input data in classes on the basis of their statistical properties only
Cluster significance and labeling.
 The labeling can be carried out even if the labels are only available for a small number of objects
representative of the desired classes.
Slide 32 www.edureka.in/mahout

33. Example – Unsupervised Learning Model
Training
Text,
Documents,
Images, etc.
Machine
Learning
Algorithm
New Text,
Document,
Image, etc.
Likelihood or
Cluster ID or
better
representation
Feature
Vectors
Feature
Vector
Predictive
Model
Slide 33 www.edureka.in/mahout

34. Example(UnSupervised Learning)
Slide 34 www.edureka.in/mahout

35. Annie’s Question
Q: What is true about Supervised learning and Unsupervised Learning
techniques:
A)Supervised learning is creating model/function through labeled training data
B)Unsupervised learning is a way to find unknown groups in un-labeled
training data
C)In Supervised learning the input observations contain the input vector and
the target variable (also called as label)
D) Unsupervised learning the input observations contain only input vector
E) All of the above
Slide 35 www.edureka.in/mahout

36. Annie’s Answer
A: All of the above
Slide 36 www.edureka.in/mahout

37. Q: K-means clustering algorithm fall under supervised
learning or un-supervised learning techniques
Slide 37 www.edureka.in/mahout
Annie’s Question

38. A: Unsupervised learning technique, as the input dataset
will NOT have the labels (target variable) and allow users to
infer hidden groups with in the input datasets
Slide 38 www.edureka.in/mahout
Annie’s Answer

39. Mahout Use Cases
Recommendation
Clustering Classification
Frequent Item
set Mining
Use Cases
supported by
Mahout
Slide 39 www.edureka.in/mahout

40. Top-level packages define the Mahout interfaces to these key abstractions:
DataModel
UserSimilarity
ItemSimilarity
UserNeighborhood
Recommender
Mahout Packages
Slide 40 www.edureka.in/mahout

41. Vector
A vector is a quantity or phenomenon that has two indepen
properties: magnitude and direction.
The term also denotes the mathematical or geometrical
representation of such a quantity.
dent
Vector R A
B
R2= A2 + B2
tan ϴ = A/B
ϴ
A
Slide 41 www.edureka.in/mahout
B
C

42. Visualizing Vectors
In two dimensions, vectors are represented as
an ordered list of values, one for eachdimension,
like (4, 3). Both representations are illustrated in
this figure.
We often name the first dimension x and the
second y when dealing with two dimensions,
but this won’t matter for our purposes in
Mahout.
As far as we’re concerned, a vector
can have 2, 3, or 10,000 dimensions. The first is
dimension 0, the next is dimension 1,
and so on.
Slide 42 www.edureka.in/mahout

43. Vectors implementation in Mahout
Dense Vector Sequential Access
Sparse Vector
Random Access
Sparse Vector
Vectors
implementation in
Mahout
It can be thought of as an
array of doubles, whose
size is the number of
features in the data.
Because all the entries in
the array are preallocated
regardless of whether the
value is 0 or not, we call it
dense.
It is implemented as a
HashMap between an
integer and a double,
where only nonzero valued
features are allocated.
Hence, they’re called as
SparseVectors.
It is implemented as two
parallel arrays, one of
integers and the other of
doubles. Only nonzero valued
entries are kept in it.
UnliketheRandomAccessSpar
se Vector, which is optimized
for random access , this one
is optimized for linear
reading.
Slide 43 www.edureka.in/mahout

44. Similarity Measurement
Similarity measurement definition
Similarity by Correlation Similarity by Distance
Slide 44 www.edureka.in/mahout

45. Similarity Measurement
Similarity by distance
Euclidean
distance
measure
Manhatten
distance measure
Cosine distance
measure
Tanimoto
distance measure
Squared
Euclidean
distance measure
Slide 45 www.edureka.in/mahout

46. Euclidean distance measure
The Euclidean distance is the
simplest of all distance
measures.
It’s the most intuitive and
matches our normal idea of
distance.
For example, given two
points on a plane, the
Euclidean distance measure
could be calculated by using
a ruler to measure the
distance between them.
Mathematically, Euclidean
distance between two n-
dimensional vectors (a1, a2,
... , an) and (b1,b2, ... , bn)
is:
The Mahout class that
implements this measure is
Euclidean Distance Measure.
Slide 46 www.edureka.in/mahout

47. Squared Euclidean distance measure
Slide 47 www.edureka.in/mahout
 Just as the name suggests, this distance measure’s value is the square of the value
 Returned by the Euclidean distance measure.
 For n-dimensional vectors (a1, a2, ... , an) and (b1, b2, ... ,bn) the distance becomes
d = (a1 – b1)2 + (a2 – b2)2 + ... + (an – bn)2
 The Mahout class that implements this measure is Squared Euclidean Distance Measure

48. Manhatten distance measure
Slide 48 www.edureka.in/mahout
 The distance between any two points is the sum of the absolute differences of their coordinates
 Mathematically, the Manhattan distance between two n-dimensional vectors (a1, a2, ... , an)
and (b1, b2, ... , bn) is
d = |a1 – b1| + |a2 – b2| + ... + |an – bn|
 The Mahout class that implements this measure is ManhattanDistanceMeasure.

49. Difference between Euclidean and Manhattan
From this image we can say that, The Euclidean distance measure gives 5.65 as the distance
between (2, 2) and (6, 6) whereas the Manhattan distance is 8.0
Slide 49 www.edureka.in/mahout

50. Cosine distance measure
 The cosine distance measure requires us to again think of points as vectors from the origin to
those points.
 These vectors form an angle, θ, between them, When this angle is small, the vectors must be
pointing in somewhat the same direction, and so in some sense the points are close.
 The cosine of this angle is near 1 when the angle is small, and decreases as it gets larger. The
cosine distance equation subtracts the cosine value from 1 in order to give a proper distance,
which is 0 when close and larger otherwise.
 The formula for the cosine distance between n-dimensional vectors (a1, a2, ... , an) and
(b1, b2, ... ,bn) is
Slide 50 www.edureka.in/mahout

51. Cosine distance measure
Cosine angle between the
vectors (2, 3) and (4, 1), as
calculated from the origin
Slide 51 www.edureka.in/mahout
 This measure of distance doesn’t account for the
length of the two vectors; all that matters is that
the points are in the same direction from the
origin.
 The cosine distance measure ranges from 0.0
(two vectors along the same direction) to 2.0
(two vectors in opposite directions).
 The Mahout class that implements this measure
is CosineDistanceMeasure.
 The cosine distance measure disregards the
lengths of the vectors. This may work well for
some data sets, but it’ll lead to poor clustering in
others where the relative lengths
of the vectors contain valuable information.

52. Cosine distance measure
 The Tanimoto distance measure, also known as Jaccard’s distance measure, captures the
information about the angle and the relative distance between the points.
 The formula for the Tanimoto distance between two n-dimensional vectors (a1, a2, ... ,
an) and (b1, b2, ... , bn) is
Slide 52 www.edureka.in/mahout

53. Annie’s Question
Q: What is a Vector?
A) A vector has both magnitude and direction
B) A vector has only magnitude but not direction
C) A vector will NOT have magnitude but has only direction
D) None of the above
Slide 53 www.edureka.in/mahout

54. Annie’s Answer
A: A vector has both magnitude and direction
Slide 54 www.edureka.in/mahout

55. Q:Which one of the following vectors need more storage
space?
A) Dense Vector
B) Sparse Vector (Random Access Sparse Vector/
Sequential Access Sparse Vector)
Slide 55 www.edureka.in/mahout
Annie’s Question

56. A: Dense Vector as regardless of presence of value of a
variable, the variables will get pre-allocated in the array
Slide 56 www.edureka.in/mahout
Annie’s Answer

57. Q: What are the valid distance measures in the following
A)CustomDistanceMeasure implementing
org.apache.mahout.common.distance.DistanceMeasureInterface in Mahout
B) Tanimoto Distance Measure
C) Manhatten Distance Measure
D) Cosine Distance Measure
E) Euclidean Distance Measure
F) Squared Eucliden Distance Measure
G) All of the above
Slide 57 www.edureka.in/mahout
Annie’s Question

58. A: All of the above
Slide 58 www.edureka.in/mahout
Annie’s Answer

59. Assignments
Its Your task
list!!
1. Install and setup Hadoop in
the Cloudera VM
2. Go through Java Essentials
for Hadoop
3. Install and setup Myrrix
software
4. Install and setup Spark
Slide 59 www.edureka.in/mahout

60. References
Slide 60 www.edureka.in/mahout
Mahout API : https://builds.apache.org/job/Mahout-Quality/javadoc/
Apache Mahout : http://mahout.apache.org/
Mahout in Action : http://www.flipkart.com/mahout-in-action/p/itmdyndgwrbr7krj

61. Prework
Slide 61 www.edureka.in/mahout
1)Review Hadoop configuration files
a) Core-site.xml
b) Hdfs-site.xml
c) Mapred-site.xml
d) Masters and slaves
e) Hadoop-env.sh
2)Understand the differences between Mahout and Spark
http://spark.apache.org/
3) Prepare the basics of Spark and MLib
http://spark.apache.org/mllib/
4)Go through the basics of Myrrix Recommender Engine,
http://myrrix.com/myrrix-is/

62. Questions?
Slide 62 www.edureka.in/mahout

63. Thank You
See You in Class Next Module