SlideShare a Scribd company logo

SVD Explained: Singular Value Decomposition Factorizes Matrices

•
•
Isaac Yowetu
Isaac Yowetu

It is about a general understanding of SVD

Education
1 of 11
Download to read offline
Singular Value Decomposition (SVD) Isaac Amornortey Yowetu December 9, 2021
A factorization Let A be an m × n matrix. The factorization of A takes the form: A = USV T • U is an m × m orthogonal matrix. • V is an n × n orthogonal matrix. • S is an m × n matrix with nonzero element along the diagonal. • Assuming m ≥ n 2/11
Properties To factorize A, we consider: • AT A and AAT Some Properties • The matrices AT A and AAT are symmetric. • The eigenvalues of AT A and AAT are: • Real and Nonnegative • Nonzero 3/11
Proofs (i) The matrices AT A and AAT are symmetric AT A = (AT A)T = (A)T (AT )T (1) = AT A (2) So similar is the AAT 4/11
Proof: (ii)The eigenvalues of AT A and AAT are real & nonnegatives The matrix AT A is symmetric so its eigenvalues are real numbers. Suppose that v is an eigenvector of AT A with ||v||2 = 1 corresponding to the eigenvalue λ. Then 0 ≤ ||Av||2 = (Av)T (Av) = vT AT Av (3) = vT (AT Av) = vT (λv) (4) = λvT v (5) = λ||v||2 (6) = λ (7) 5/11
Proof: (iii)The eigenvalues of AT A and AAT are nonzero Let v be an eigenvector corresponding to nonzero eigenvalue λ of AT A. Then AT Av = λv It also implies (8) (AAT )Av = λ(Av) (9) If Av = 0, then AT Av = AT 0, this contradicts the assumption that λ 6= 0 Hence Av 6= 0 and Av is an eigenvector of AAT associated with λ. 6/11
Example 1 Determine the singular values of the 5 x 3 matrix A =          1 0 1 0 1 0 0 1 1 0 1 0 1 1 0          7/11
Solution A =          1 0 1 0 1 0 0 1 1 0 1 0 1 1 0          AT =     1 0 0 0 1 0 1 1 1 1 1 0 1 0 0     AT A =     2 1 1 1 4 1 1 1 2     8/11
Characteristics Polynomial P(AT A) = λ3 − 8λ2 + 17λ − 10 (10) = (λ − 5)(λ − 2)(λ − 1) (11) Eigenvalues λ1 = s2 1 = 5 (12) λ2 = s2 2 = 2 (13) λ3 = s2 3 = 1 (14) 9/11
Singular Values s1 = p λ1 = √ 5 (15) s2 = p λ2 = √ 2 (16) s3 = p λ3 = 1 (17) Singular Values Decompostion of A S =          √ 5 0 0 0 √ 2 0 0 0 1 0 0 0 0 0 0          10/11
Reference(s) Burden, R. L. and Faires, J. D. (2010). Numerical Analysis, Ninth Edition. Richard Stratton. 11/11

Recommended

5. Linear Algebra for Machine Learning: Singular Value Decomposition and Prin...
5. Linear Algebra for Machine Learning: Singular Value Decomposition and Prin...5. Linear Algebra for Machine Learning: Singular Value Decomposition and Prin...
5. Linear Algebra for Machine Learning: Singular Value Decomposition and Prin...Ceni Babaoglu, PhD
 
The Singular Value Decomposition theroy + example
The Singular Value Decomposition theroy + example The Singular Value Decomposition theroy + example
The Singular Value Decomposition theroy + example shivam choubey
 
Eigen values and eigenvectors
Eigen values and eigenvectorsEigen values and eigenvectors
Eigen values and eigenvectorsAmit Singh
 
Neural Networks: Least Mean Square (LSM) Algorithm
Neural Networks: Least Mean Square (LSM) AlgorithmNeural Networks: Least Mean Square (LSM) Algorithm
Neural Networks: Least Mean Square (LSM) AlgorithmMostafa G. M. Mostafa
 
Machine learning with ADA Boost
Machine learning with ADA BoostMachine learning with ADA Boost
Machine learning with ADA BoostAman Patel
 
1. Linear Algebra for Machine Learning: Linear Systems
1. Linear Algebra for Machine Learning: Linear Systems1. Linear Algebra for Machine Learning: Linear Systems
1. Linear Algebra for Machine Learning: Linear SystemsCeni Babaoglu, PhD
 
Dimensionality reduction: SVD and its applications
Dimensionality reduction: SVD and its applicationsDimensionality reduction: SVD and its applications
Dimensionality reduction: SVD and its applicationsViet-Trung TRAN
 
Eigenvalues and Eigenvectors
Eigenvalues and EigenvectorsEigenvalues and Eigenvectors
Eigenvalues and EigenvectorsVinod Srivastava
 

Recommended

5. Linear Algebra for Machine Learning: Singular Value Decomposition and Prin...
5. Linear Algebra for Machine Learning: Singular Value Decomposition and Prin...5. Linear Algebra for Machine Learning: Singular Value Decomposition and Prin...
5. Linear Algebra for Machine Learning: Singular Value Decomposition and Prin...Ceni Babaoglu, PhD
 
The Singular Value Decomposition theroy + example
The Singular Value Decomposition theroy + example The Singular Value Decomposition theroy + example
The Singular Value Decomposition theroy + example shivam choubey
 
Eigen values and eigenvectors
Eigen values and eigenvectorsEigen values and eigenvectors
Eigen values and eigenvectorsAmit Singh
 
Neural Networks: Least Mean Square (LSM) Algorithm
Neural Networks: Least Mean Square (LSM) AlgorithmNeural Networks: Least Mean Square (LSM) Algorithm
Neural Networks: Least Mean Square (LSM) AlgorithmMostafa G. M. Mostafa
 
Machine learning with ADA Boost
Machine learning with ADA BoostMachine learning with ADA Boost
Machine learning with ADA BoostAman Patel
 
1. Linear Algebra for Machine Learning: Linear Systems
1. Linear Algebra for Machine Learning: Linear Systems1. Linear Algebra for Machine Learning: Linear Systems
1. Linear Algebra for Machine Learning: Linear SystemsCeni Babaoglu, PhD
 
Dimensionality reduction: SVD and its applications
Dimensionality reduction: SVD and its applicationsDimensionality reduction: SVD and its applications
Dimensionality reduction: SVD and its applicationsViet-Trung TRAN
 
Eigenvalues and Eigenvectors
Eigenvalues and EigenvectorsEigenvalues and Eigenvectors
Eigenvalues and EigenvectorsVinod Srivastava
 
3. Linear Algebra for Machine Learning: Factorization and Linear Transformations
3. Linear Algebra for Machine Learning: Factorization and Linear Transformations3. Linear Algebra for Machine Learning: Factorization and Linear Transformations
3. Linear Algebra for Machine Learning: Factorization and Linear TransformationsCeni Babaoglu, PhD
 
Bayesian networks in AI
Bayesian networks in AIBayesian networks in AI
Bayesian networks in AIByoung-Hee Kim
 
Svd
SvdSvd
Svdssuser1163cd
 
Back propagation
Back propagationBack propagation
Back propagationNagarajan
 
Dimensionality Reduction
Dimensionality ReductionDimensionality Reduction
Dimensionality ReductionSaad Elbeleidy
 
HML: Historical View and Trends of Deep Learning
HML: Historical View and Trends of Deep LearningHML: Historical View and Trends of Deep Learning
HML: Historical View and Trends of Deep LearningYan Xu
 
Recurrent Neural Network (RNN) | RNN LSTM Tutorial | Deep Learning Course | S...
Recurrent Neural Network (RNN) | RNN LSTM Tutorial | Deep Learning Course | S...Recurrent Neural Network (RNN) | RNN LSTM Tutorial | Deep Learning Course | S...
Recurrent Neural Network (RNN) | RNN LSTM Tutorial | Deep Learning Course | S...Simplilearn
 
Pca ppt
Pca pptPca ppt
Pca pptDheeraj Dwivedi
 
Feature selection concepts and methods
Feature selection concepts and methodsFeature selection concepts and methods
Feature selection concepts and methodsReza Ramezani
 
AI Lecture 7 (uncertainty)
AI Lecture 7 (uncertainty)AI Lecture 7 (uncertainty)
AI Lecture 7 (uncertainty)Tajim Md. Niamat Ullah Akhund
 
Graph Theory
Graph TheoryGraph Theory
Graph Theorykailash shaw
 
Machine Learning: Generative and Discriminative Models
Machine Learning: Generative and Discriminative ModelsMachine Learning: Generative and Discriminative Models
Machine Learning: Generative and Discriminative Modelsbutest
 
Regularization
RegularizationRegularization
RegularizationDarren Yow-Bang Wang
 
Hebbian Learning
Hebbian LearningHebbian Learning
Hebbian LearningESCOM
 
03 Machine Learning Linear Algebra
03 Machine Learning Linear Algebra03 Machine Learning Linear Algebra
03 Machine Learning Linear AlgebraAndres Mendez-Vazquez
 
Maths-->>Eigenvalues and eigenvectors
Maths-->>Eigenvalues and eigenvectorsMaths-->>Eigenvalues and eigenvectors
Maths-->>Eigenvalues and eigenvectorsJaydev Kishnani
 
K - Nearest neighbor ( KNN )
K - Nearest neighbor ( KNN )K - Nearest neighbor ( KNN )
K - Nearest neighbor ( KNN )Mohammad Junaid Khan
 
Introduction to Graph Theory
Introduction to Graph TheoryIntroduction to Graph Theory
Introduction to Graph TheoryYosuke Mizutani
 
vector space and subspace
vector space and subspacevector space and subspace
vector space and subspace2461998
 
Support Vector Machine ppt presentation
Support Vector Machine ppt presentationSupport Vector Machine ppt presentation
Support Vector Machine ppt presentationAyanaRukasar
 
Positive
PositivePositive
PositivePhong Tran
 
Notes on eigenvalues
Notes on eigenvaluesNotes on eigenvalues
Notes on eigenvaluesAmanSaeed11
 

More Related Content

What's hot

3. Linear Algebra for Machine Learning: Factorization and Linear Transformations
3. Linear Algebra for Machine Learning: Factorization and Linear Transformations3. Linear Algebra for Machine Learning: Factorization and Linear Transformations
3. Linear Algebra for Machine Learning: Factorization and Linear TransformationsCeni Babaoglu, PhD
 
Bayesian networks in AI
Bayesian networks in AIBayesian networks in AI
Bayesian networks in AIByoung-Hee Kim
 
Back propagation
Back propagationBack propagation
Back propagationNagarajan
 
Dimensionality Reduction
Dimensionality ReductionDimensionality Reduction
Dimensionality ReductionSaad Elbeleidy
 
HML: Historical View and Trends of Deep Learning
HML: Historical View and Trends of Deep LearningHML: Historical View and Trends of Deep Learning
HML: Historical View and Trends of Deep LearningYan Xu
 
Recurrent Neural Network (RNN) | RNN LSTM Tutorial | Deep Learning Course | S...
Recurrent Neural Network (RNN) | RNN LSTM Tutorial | Deep Learning Course | S...Recurrent Neural Network (RNN) | RNN LSTM Tutorial | Deep Learning Course | S...
Recurrent Neural Network (RNN) | RNN LSTM Tutorial | Deep Learning Course | S...Simplilearn
 
Feature selection concepts and methods
Feature selection concepts and methodsFeature selection concepts and methods
Feature selection concepts and methodsReza Ramezani
 
Machine Learning: Generative and Discriminative Models
Machine Learning: Generative and Discriminative ModelsMachine Learning: Generative and Discriminative Models
Machine Learning: Generative and Discriminative Modelsbutest
 
Hebbian Learning
Hebbian LearningHebbian Learning
Hebbian LearningESCOM
 
Maths-->>Eigenvalues and eigenvectors
Maths-->>Eigenvalues and eigenvectorsMaths-->>Eigenvalues and eigenvectors
Maths-->>Eigenvalues and eigenvectorsJaydev Kishnani
 
Introduction to Graph Theory
Introduction to Graph TheoryIntroduction to Graph Theory
Introduction to Graph TheoryYosuke Mizutani
 
vector space and subspace
vector space and subspacevector space and subspace
vector space and subspace2461998
 
Support Vector Machine ppt presentation
Support Vector Machine ppt presentationSupport Vector Machine ppt presentation
Support Vector Machine ppt presentationAyanaRukasar
 

What's hot (20)

3. Linear Algebra for Machine Learning: Factorization and Linear Transformations
3. Linear Algebra for Machine Learning: Factorization and Linear Transformations3. Linear Algebra for Machine Learning: Factorization and Linear Transformations
3. Linear Algebra for Machine Learning: Factorization and Linear Transformations
 
Bayesian networks in AI
Bayesian networks in AIBayesian networks in AI
Bayesian networks in AI
 
Svd
SvdSvd
Svd
 
Back propagation
Back propagationBack propagation
Back propagation
 
Dimensionality Reduction
Dimensionality ReductionDimensionality Reduction
Dimensionality Reduction
 
HML: Historical View and Trends of Deep Learning
HML: Historical View and Trends of Deep LearningHML: Historical View and Trends of Deep Learning
HML: Historical View and Trends of Deep Learning
 
Recurrent Neural Network (RNN) | RNN LSTM Tutorial | Deep Learning Course | S...
Recurrent Neural Network (RNN) | RNN LSTM Tutorial | Deep Learning Course | S...Recurrent Neural Network (RNN) | RNN LSTM Tutorial | Deep Learning Course | S...
Recurrent Neural Network (RNN) | RNN LSTM Tutorial | Deep Learning Course | S...
 
Pca ppt
Pca pptPca ppt
Pca ppt
 
Feature selection concepts and methods
Feature selection concepts and methodsFeature selection concepts and methods
Feature selection concepts and methods
 
AI Lecture 7 (uncertainty)
AI Lecture 7 (uncertainty)AI Lecture 7 (uncertainty)
AI Lecture 7 (uncertainty)
 
Graph Theory
Graph TheoryGraph Theory
Graph Theory
 
Machine Learning: Generative and Discriminative Models
Machine Learning: Generative and Discriminative ModelsMachine Learning: Generative and Discriminative Models
Machine Learning: Generative and Discriminative Models
 
Regularization
RegularizationRegularization
Regularization
 
Hebbian Learning
Hebbian LearningHebbian Learning
Hebbian Learning
 
03 Machine Learning Linear Algebra
03 Machine Learning Linear Algebra03 Machine Learning Linear Algebra
03 Machine Learning Linear Algebra
 
Maths-->>Eigenvalues and eigenvectors
Maths-->>Eigenvalues and eigenvectorsMaths-->>Eigenvalues and eigenvectors
Maths-->>Eigenvalues and eigenvectors
 
K - Nearest neighbor ( KNN )
K - Nearest neighbor ( KNN )K - Nearest neighbor ( KNN )
K - Nearest neighbor ( KNN )
 
Introduction to Graph Theory
Introduction to Graph TheoryIntroduction to Graph Theory
Introduction to Graph Theory
 
vector space and subspace
vector space and subspacevector space and subspace
vector space and subspace
 
Support Vector Machine ppt presentation
Support Vector Machine ppt presentationSupport Vector Machine ppt presentation
Support Vector Machine ppt presentation
 

Similar to SVD Explained: Singular Value Decomposition Factorizes Matrices

Notes on eigenvalues
Notes on eigenvaluesNotes on eigenvalues
Notes on eigenvaluesAmanSaeed11
 
Positive matrix by sarmad baloch
Positive matrix by sarmad balochPositive matrix by sarmad baloch
Positive matrix by sarmad balochSarmad Baloch
 
orthogonal matrix and its properties
orthogonal matrix and its propertiesorthogonal matrix and its properties
orthogonal matrix and its propertiesGOVINDKUMAR689610
 
Eigen-Decomposition: Eigenvalues and Eigenvectors.pdf
Eigen-Decomposition: Eigenvalues and Eigenvectors.pdfEigen-Decomposition: Eigenvalues and Eigenvectors.pdf
Eigen-Decomposition: Eigenvalues and Eigenvectors.pdfNehaVerma933923
 
Partial midterm set7 soln linear algebra
Partial midterm set7 soln linear algebraPartial midterm set7 soln linear algebra
Partial midterm set7 soln linear algebrameezanchand
 
Linear Algebra Gauss Jordan elimination.pptx
Linear Algebra Gauss Jordan elimination.pptxLinear Algebra Gauss Jordan elimination.pptx
Linear Algebra Gauss Jordan elimination.pptxMaths Assignment Help
 
Tutorial on EM algorithm – Part 1
Tutorial on EM algorithm – Part 1Tutorial on EM algorithm – Part 1
Tutorial on EM algorithm – Part 1Loc Nguyen
 
Matrices & determinants
Matrices & determinantsMatrices & determinants
Matrices & determinantsindu thakur
 
Definitions matrices y determinantes fula 2010 english subir
Definitions matrices y determinantes fula 2010 english subirDefinitions matrices y determinantes fula 2010 english subir
Definitions matrices y determinantes fula 2010 english subirHernanFula
 
Problem_Session_Notes
Problem_Session_NotesProblem_Session_Notes
Problem_Session_NotesLu Mao
 
Chapter 3: Linear Systems and Matrices - Part 3/Slides
Chapter 3: Linear Systems and Matrices - Part 3/SlidesChapter 3: Linear Systems and Matrices - Part 3/Slides
Chapter 3: Linear Systems and Matrices - Part 3/SlidesChaimae Baroudi
 
Eighan values and diagonalization
Eighan values and diagonalization Eighan values and diagonalization
Eighan values and diagonalization gandhinagar
 

Similar to SVD Explained: Singular Value Decomposition Factorizes Matrices (20)

Positive
PositivePositive
Positive
 
Notes on eigenvalues
Notes on eigenvaluesNotes on eigenvalues
Notes on eigenvalues
 
Positive matrix by sarmad baloch
Positive matrix by sarmad balochPositive matrix by sarmad baloch
Positive matrix by sarmad baloch
 
eigenvalue
eigenvalueeigenvalue
eigenvalue
 
orthogonal matrix and its properties
orthogonal matrix and its propertiesorthogonal matrix and its properties
orthogonal matrix and its properties
 
Eigen-Decomposition: Eigenvalues and Eigenvectors.pdf
Eigen-Decomposition: Eigenvalues and Eigenvectors.pdfEigen-Decomposition: Eigenvalues and Eigenvectors.pdf
Eigen-Decomposition: Eigenvalues and Eigenvectors.pdf
 
Partial midterm set7 soln linear algebra
Partial midterm set7 soln linear algebraPartial midterm set7 soln linear algebra
Partial midterm set7 soln linear algebra
 
Special Square Matrices
Special Square MatricesSpecial Square Matrices
Special Square Matrices
 
Linear Algebra
Linear AlgebraLinear Algebra
Linear Algebra
 
Linear Algebra Gauss Jordan elimination.pptx
Linear Algebra Gauss Jordan elimination.pptxLinear Algebra Gauss Jordan elimination.pptx
Linear Algebra Gauss Jordan elimination.pptx
 
Senior_Project
Senior_ProjectSenior_Project
Senior_Project
 
Eigen value and vectors
Eigen value and vectorsEigen value and vectors
Eigen value and vectors
 
Tutorial on EM algorithm – Part 1
Tutorial on EM algorithm – Part 1Tutorial on EM algorithm – Part 1
Tutorial on EM algorithm – Part 1
 
Matrices & determinants
Matrices & determinantsMatrices & determinants
Matrices & determinants
 
Definitions matrices y determinantes fula 2010 english subir
Definitions matrices y determinantes fula 2010 english subirDefinitions matrices y determinantes fula 2010 english subir
Definitions matrices y determinantes fula 2010 english subir
 
Problem_Session_Notes
Problem_Session_NotesProblem_Session_Notes
Problem_Session_Notes
 
Chapter 3: Linear Systems and Matrices - Part 3/Slides
Chapter 3: Linear Systems and Matrices - Part 3/SlidesChapter 3: Linear Systems and Matrices - Part 3/Slides
Chapter 3: Linear Systems and Matrices - Part 3/Slides
 
Lecture_06_Part_1.ppt
Lecture_06_Part_1.pptLecture_06_Part_1.ppt
Lecture_06_Part_1.ppt
 
Eighan values and diagonalization
Eighan values and diagonalization Eighan values and diagonalization
Eighan values and diagonalization
 
DOC-20231230-WA0001..pdf
DOC-20231230-WA0001..pdfDOC-20231230-WA0001..pdf
DOC-20231230-WA0001..pdf
 

More from Isaac Yowetu

Inverse-power-method.pdf
Inverse-power-method.pdfInverse-power-method.pdf
Inverse-power-method.pdfIsaac Yowetu
 
Approximating Dominant Eivenvalue By The Power Method
Approximating Dominant Eivenvalue By The Power MethodApproximating Dominant Eivenvalue By The Power Method
Approximating Dominant Eivenvalue By The Power MethodIsaac Yowetu
 
Singular Value Decompostion (SVD): Worked example 3
Singular Value Decompostion (SVD): Worked example 3Singular Value Decompostion (SVD): Worked example 3
Singular Value Decompostion (SVD): Worked example 3Isaac Yowetu
 
Singular Value Decompostion (SVD): Worked example 2
Singular Value Decompostion (SVD): Worked example 2Singular Value Decompostion (SVD): Worked example 2
Singular Value Decompostion (SVD): Worked example 2Isaac Yowetu
 
Singular Value Decompostion (SVD): Worked example 1
Singular Value Decompostion (SVD): Worked example 1Singular Value Decompostion (SVD): Worked example 1
Singular Value Decompostion (SVD): Worked example 1Isaac Yowetu
 
Cayley-Hamilton Theorem, Eigenvalues, Eigenvectors and Eigenspace.
Cayley-Hamilton Theorem, Eigenvalues, Eigenvectors and Eigenspace.Cayley-Hamilton Theorem, Eigenvalues, Eigenvectors and Eigenspace.
Cayley-Hamilton Theorem, Eigenvalues, Eigenvectors and Eigenspace.Isaac Yowetu
 
Givens rotation method
Givens rotation methodGivens rotation method
Givens rotation methodIsaac Yowetu
 
Sherman-Morrison Formula Proof
Sherman-Morrison Formula ProofSherman-Morrison Formula Proof
Sherman-Morrison Formula ProofIsaac Yowetu
 
Householder transformation | Householder Reflection with QR Decomposition
Householder transformation | Householder Reflection with QR DecompositionHouseholder transformation | Householder Reflection with QR Decomposition
Householder transformation | Householder Reflection with QR DecompositionIsaac Yowetu
 
Gram-Schmidt and QR Decomposition (Factorization) of Matrices
Gram-Schmidt and QR Decomposition (Factorization) of MatricesGram-Schmidt and QR Decomposition (Factorization) of Matrices
Gram-Schmidt and QR Decomposition (Factorization) of MatricesIsaac Yowetu
 
Gram schmidt orthogonalization | Orthonormal Process
Gram schmidt orthogonalization | Orthonormal Process Gram schmidt orthogonalization | Orthonormal Process
Gram schmidt orthogonalization | Orthonormal Process Isaac Yowetu
 
Regula Falsi (False position) Method
Regula Falsi (False position) MethodRegula Falsi (False position) Method
Regula Falsi (False position) MethodIsaac Yowetu
 
Projectors and Projection Onto Subspaces
Projectors and Projection Onto SubspacesProjectors and Projection Onto Subspaces
Projectors and Projection Onto SubspacesIsaac Yowetu
 
Projectors and Projection Onto a Line
Projectors and Projection Onto a LineProjectors and Projection Onto a Line
Projectors and Projection Onto a LineIsaac Yowetu
 
Secant Iterative method
Secant Iterative methodSecant Iterative method
Secant Iterative methodIsaac Yowetu
 
Newton Raphson iterative Method
Newton Raphson iterative MethodNewton Raphson iterative Method
Newton Raphson iterative MethodIsaac Yowetu
 
Fixed point iteration
Fixed point iterationFixed point iteration
Fixed point iterationIsaac Yowetu
 

More from Isaac Yowetu (19)

Inverse-power-method.pdf
Inverse-power-method.pdfInverse-power-method.pdf
Inverse-power-method.pdf
 
Approximating Dominant Eivenvalue By The Power Method
Approximating Dominant Eivenvalue By The Power MethodApproximating Dominant Eivenvalue By The Power Method
Approximating Dominant Eivenvalue By The Power Method
 
Singular Value Decompostion (SVD): Worked example 3
Singular Value Decompostion (SVD): Worked example 3Singular Value Decompostion (SVD): Worked example 3
Singular Value Decompostion (SVD): Worked example 3
 
Singular Value Decompostion (SVD): Worked example 2
Singular Value Decompostion (SVD): Worked example 2Singular Value Decompostion (SVD): Worked example 2
Singular Value Decompostion (SVD): Worked example 2
 
Singular Value Decompostion (SVD): Worked example 1
Singular Value Decompostion (SVD): Worked example 1Singular Value Decompostion (SVD): Worked example 1
Singular Value Decompostion (SVD): Worked example 1
 
Cayley-Hamilton Theorem, Eigenvalues, Eigenvectors and Eigenspace.
Cayley-Hamilton Theorem, Eigenvalues, Eigenvectors and Eigenspace.Cayley-Hamilton Theorem, Eigenvalues, Eigenvectors and Eigenspace.
Cayley-Hamilton Theorem, Eigenvalues, Eigenvectors and Eigenspace.
 
Givens rotation method
Givens rotation methodGivens rotation method
Givens rotation method
 
Sherman-Morrison Formula Proof
Sherman-Morrison Formula ProofSherman-Morrison Formula Proof
Sherman-Morrison Formula Proof
 
Householder transformation | Householder Reflection with QR Decomposition
Householder transformation | Householder Reflection with QR DecompositionHouseholder transformation | Householder Reflection with QR Decomposition
Householder transformation | Householder Reflection with QR Decomposition
 
Gram-Schmidt and QR Decomposition (Factorization) of Matrices
Gram-Schmidt and QR Decomposition (Factorization) of MatricesGram-Schmidt and QR Decomposition (Factorization) of Matrices
Gram-Schmidt and QR Decomposition (Factorization) of Matrices
 
Gram schmidt orthogonalization | Orthonormal Process
Gram schmidt orthogonalization | Orthonormal Process Gram schmidt orthogonalization | Orthonormal Process
Gram schmidt orthogonalization | Orthonormal Process
 
Regula Falsi (False position) Method
Regula Falsi (False position) MethodRegula Falsi (False position) Method
Regula Falsi (False position) Method
 
Bisection method
Bisection methodBisection method
Bisection method
 
Projectors and Projection Onto Subspaces
Projectors and Projection Onto SubspacesProjectors and Projection Onto Subspaces
Projectors and Projection Onto Subspaces
 
Projectors and Projection Onto a Line
Projectors and Projection Onto a LineProjectors and Projection Onto a Line
Projectors and Projection Onto a Line
 
Secant Iterative method
Secant Iterative methodSecant Iterative method
Secant Iterative method
 
Aitken's Method
Aitken's MethodAitken's Method
Aitken's Method
 
Newton Raphson iterative Method
Newton Raphson iterative MethodNewton Raphson iterative Method
Newton Raphson iterative Method
 
Fixed point iteration
Fixed point iterationFixed point iteration
Fixed point iteration
 

Recently uploaded

Industrial Policy - 1948, 1956, 1973, 1977, 1980, 1991
Industrial Policy - 1948, 1956, 1973, 1977, 1980, 1991Industrial Policy - 1948, 1956, 1973, 1977, 1980, 1991
Industrial Policy - 1948, 1956, 1973, 1977, 1980, 1991RKavithamani
 
microwave assisted reaction. General introduction
microwave assisted reaction. General introductionmicrowave assisted reaction. General introduction
microwave assisted reaction. General introductionMaksud Ahmed
 
Nutritional Needs Presentation - HLTH 104
Nutritional Needs Presentation - HLTH 104Nutritional Needs Presentation - HLTH 104
Nutritional Needs Presentation - HLTH 104misteraugie
 
How to Make a Pirate ship Primary Education.pptx
How to Make a Pirate ship Primary Education.pptxHow to Make a Pirate ship Primary Education.pptx
How to Make a Pirate ship Primary Education.pptxmanuelaromero2013
 
Sanyam Choudhary Chemistry practical.pdf
Sanyam Choudhary Chemistry practical.pdfSanyam Choudhary Chemistry practical.pdf
Sanyam Choudhary Chemistry practical.pdfsanyamsingh5019
 
Contemporary philippine arts from the regions_PPT_Module_12 [Autosaved] (1).pptx
Contemporary philippine arts from the regions_PPT_Module_12 [Autosaved] (1).pptxContemporary philippine arts from the regions_PPT_Module_12 [Autosaved] (1).pptx
Contemporary philippine arts from the regions_PPT_Module_12 [Autosaved] (1).pptxRoyAbrique
 
Q4-W6-Restating Informational Text Grade 3
Q4-W6-Restating Informational Text Grade 3Q4-W6-Restating Informational Text Grade 3
Q4-W6-Restating Informational Text Grade 3JemimahLaneBuaron
 
Organic Name Reactions for the students and aspirants of Chemistry12th.pptx
Organic Name Reactions for the students and aspirants of Chemistry12th.pptxOrganic Name Reactions for the students and aspirants of Chemistry12th.pptx
Organic Name Reactions for the students and aspirants of Chemistry12th.pptxVS Mahajan Coaching Centre
 
Beyond the EU: DORA and NIS 2 Directive's Global Impact
Beyond the EU: DORA and NIS 2 Directive's Global ImpactBeyond the EU: DORA and NIS 2 Directive's Global Impact
Beyond the EU: DORA and NIS 2 Directive's Global ImpactPECB
 
Separation of Lanthanides/ Lanthanides and Actinides
Separation of Lanthanides/ Lanthanides and ActinidesSeparation of Lanthanides/ Lanthanides and Actinides
Separation of Lanthanides/ Lanthanides and ActinidesFatimaKhan178732
 
Z Score,T Score, Percential Rank and Box Plot Graph
Z Score,T Score, Percential Rank and Box Plot GraphZ Score,T Score, Percential Rank and Box Plot Graph
Z Score,T Score, Percential Rank and Box Plot GraphThiyagu K
 
CARE OF CHILD IN INCUBATOR..........pptx
CARE OF CHILD IN INCUBATOR..........pptxCARE OF CHILD IN INCUBATOR..........pptx
CARE OF CHILD IN INCUBATOR..........pptxGaneshChakor2
 
Measures of Central Tendency: Mean, Median and Mode
Measures of Central Tendency: Mean, Median and ModeMeasures of Central Tendency: Mean, Median and Mode
Measures of Central Tendency: Mean, Median and ModeThiyagu K
 
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...Krashi Coaching
 
Hybridoma Technology ( Production , Purification , and Application )
Hybridoma Technology ( Production , Purification , and Application ) Hybridoma Technology ( Production , Purification , and Application )
Hybridoma Technology ( Production , Purification , and Application ) Sakshi Ghasle
 
Web & Social Media Analytics Previous Year Question Paper.pdf
Web & Social Media Analytics Previous Year Question Paper.pdfWeb & Social Media Analytics Previous Year Question Paper.pdf
Web & Social Media Analytics Previous Year Question Paper.pdfJayanti Pande
 

Recently uploaded (20)

Staff of Color (SOC) Retention Efforts DDSD
Staff of Color (SOC) Retention Efforts DDSDStaff of Color (SOC) Retention Efforts DDSD
Staff of Color (SOC) Retention Efforts DDSD
 
TataKelola dan KamSiber Kecerdasan Buatan v022.pdf
TataKelola dan KamSiber Kecerdasan Buatan v022.pdfTataKelola dan KamSiber Kecerdasan Buatan v022.pdf
TataKelola dan KamSiber Kecerdasan Buatan v022.pdf
 
Industrial Policy - 1948, 1956, 1973, 1977, 1980, 1991
Industrial Policy - 1948, 1956, 1973, 1977, 1980, 1991Industrial Policy - 1948, 1956, 1973, 1977, 1980, 1991
Industrial Policy - 1948, 1956, 1973, 1977, 1980, 1991
 
microwave assisted reaction. General introduction
microwave assisted reaction. General introductionmicrowave assisted reaction. General introduction
microwave assisted reaction. General introduction
 
Nutritional Needs Presentation - HLTH 104
Nutritional Needs Presentation - HLTH 104Nutritional Needs Presentation - HLTH 104
Nutritional Needs Presentation - HLTH 104
 
INDIA QUIZ 2024 RLAC DELHI UNIVERSITY.pptx
INDIA QUIZ 2024 RLAC DELHI UNIVERSITY.pptxINDIA QUIZ 2024 RLAC DELHI UNIVERSITY.pptx
INDIA QUIZ 2024 RLAC DELHI UNIVERSITY.pptx
 
Mattingly "AI & Prompt Design: Structured Data, Assistants, & RAG"
Mattingly "AI & Prompt Design: Structured Data, Assistants, & RAG"Mattingly "AI & Prompt Design: Structured Data, Assistants, & RAG"
Mattingly "AI & Prompt Design: Structured Data, Assistants, & RAG"
 
How to Make a Pirate ship Primary Education.pptx
How to Make a Pirate ship Primary Education.pptxHow to Make a Pirate ship Primary Education.pptx
How to Make a Pirate ship Primary Education.pptx
 
Sanyam Choudhary Chemistry practical.pdf
Sanyam Choudhary Chemistry practical.pdfSanyam Choudhary Chemistry practical.pdf
Sanyam Choudhary Chemistry practical.pdf
 
Contemporary philippine arts from the regions_PPT_Module_12 [Autosaved] (1).pptx
Contemporary philippine arts from the regions_PPT_Module_12 [Autosaved] (1).pptxContemporary philippine arts from the regions_PPT_Module_12 [Autosaved] (1).pptx
Contemporary philippine arts from the regions_PPT_Module_12 [Autosaved] (1).pptx
 
Q4-W6-Restating Informational Text Grade 3
Q4-W6-Restating Informational Text Grade 3Q4-W6-Restating Informational Text Grade 3
Q4-W6-Restating Informational Text Grade 3
 
Organic Name Reactions for the students and aspirants of Chemistry12th.pptx
Organic Name Reactions for the students and aspirants of Chemistry12th.pptxOrganic Name Reactions for the students and aspirants of Chemistry12th.pptx
Organic Name Reactions for the students and aspirants of Chemistry12th.pptx
 
Beyond the EU: DORA and NIS 2 Directive's Global Impact
Beyond the EU: DORA and NIS 2 Directive's Global ImpactBeyond the EU: DORA and NIS 2 Directive's Global Impact
Beyond the EU: DORA and NIS 2 Directive's Global Impact
 
Separation of Lanthanides/ Lanthanides and Actinides
Separation of Lanthanides/ Lanthanides and ActinidesSeparation of Lanthanides/ Lanthanides and Actinides
Separation of Lanthanides/ Lanthanides and Actinides
 
Z Score,T Score, Percential Rank and Box Plot Graph
Z Score,T Score, Percential Rank and Box Plot GraphZ Score,T Score, Percential Rank and Box Plot Graph
Z Score,T Score, Percential Rank and Box Plot Graph
 
CARE OF CHILD IN INCUBATOR..........pptx
CARE OF CHILD IN INCUBATOR..........pptxCARE OF CHILD IN INCUBATOR..........pptx
CARE OF CHILD IN INCUBATOR..........pptx
 
Measures of Central Tendency: Mean, Median and Mode
Measures of Central Tendency: Mean, Median and ModeMeasures of Central Tendency: Mean, Median and Mode
Measures of Central Tendency: Mean, Median and Mode
 
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
 
Hybridoma Technology ( Production , Purification , and Application )
Hybridoma Technology ( Production , Purification , and Application ) Hybridoma Technology ( Production , Purification , and Application )
Hybridoma Technology ( Production , Purification , and Application )
 
Web & Social Media Analytics Previous Year Question Paper.pdf
Web & Social Media Analytics Previous Year Question Paper.pdfWeb & Social Media Analytics Previous Year Question Paper.pdf
Web & Social Media Analytics Previous Year Question Paper.pdf
 

SVD Explained: Singular Value Decomposition Factorizes Matrices

  • 1. Singular Value Decomposition (SVD) Isaac Amornortey Yowetu December 9, 2021
  • 2. A factorization Let A be an m × n matrix. The factorization of A takes the form: A = USV T • U is an m × m orthogonal matrix. • V is an n × n orthogonal matrix. • S is an m × n matrix with nonzero element along the diagonal. • Assuming m ≥ n 2/11
  • 3. Properties To factorize A, we consider: • AT A and AAT Some Properties • The matrices AT A and AAT are symmetric. • The eigenvalues of AT A and AAT are: • Real and Nonnegative • Nonzero 3/11
  • 4. Proofs (i) The matrices AT A and AAT are symmetric AT A = (AT A)T = (A)T (AT )T (1) = AT A (2) So similar is the AAT 4/11
  • 5. Proof: (ii)The eigenvalues of AT A and AAT are real & nonnegatives The matrix AT A is symmetric so its eigenvalues are real numbers. Suppose that v is an eigenvector of AT A with ||v||2 = 1 corresponding to the eigenvalue λ. Then 0 ≤ ||Av||2 = (Av)T (Av) = vT AT Av (3) = vT (AT Av) = vT (λv) (4) = λvT v (5) = λ||v||2 (6) = λ (7) 5/11
  • 6. Proof: (iii)The eigenvalues of AT A and AAT are nonzero Let v be an eigenvector corresponding to nonzero eigenvalue λ of AT A. Then AT Av = λv It also implies (8) (AAT )Av = λ(Av) (9) If Av = 0, then AT Av = AT 0, this contradicts the assumption that λ 6= 0 Hence Av 6= 0 and Av is an eigenvector of AAT associated with λ. 6/11
  • 7. Example 1 Determine the singular values of the 5 x 3 matrix A =          1 0 1 0 1 0 0 1 1 0 1 0 1 1 0          7/11
  • 8. Solution A =          1 0 1 0 1 0 0 1 1 0 1 0 1 1 0          AT =     1 0 0 0 1 0 1 1 1 1 1 0 1 0 0     AT A =     2 1 1 1 4 1 1 1 2     8/11
  • 9. Characteristics Polynomial P(AT A) = λ3 − 8λ2 + 17λ − 10 (10) = (λ − 5)(λ − 2)(λ − 1) (11) Eigenvalues λ1 = s2 1 = 5 (12) λ2 = s2 2 = 2 (13) λ3 = s2 3 = 1 (14) 9/11
  • 10. Singular Values s1 = p λ1 = √ 5 (15) s2 = p λ2 = √ 2 (16) s3 = p λ3 = 1 (17) Singular Values Decompostion of A S =          √ 5 0 0 0 √ 2 0 0 0 1 0 0 0 0 0 0          10/11
  • 11. Reference(s) Burden, R. L. and Faires, J. D. (2010). Numerical Analysis, Ninth Edition. Richard Stratton. 11/11