[Math Review] Linear Algebra for Singular Value Decomposition (SVD)

Matrix and Determinant

Let C be an M × N matrix with real-valued entries, i.e. C={cij}mxn

Determinant is a value that can be computed from the elements of a square matrix. The determinant of a matrix A is denoted det(A), det A, or |A|.

In the case of a 2 × 2 matrix the determinant may be defined as:

Similarly, for a 3 × 3 matrix A, its determinant is:

See more information about determinant here.

Rank of Matrix

The Rank of a matrix is the number of linearly independent rows (or columns) in it, so rank(C)≤min(m,n).

A common approach to finding the rank of a matrix is to reduce it to a simpler form, generally row echelon form, by elementary row operations. The rank equals to the number of non-zero rows of the final matrix (in row echelon form).

The reduce step can be found in this article.

Eigenvalues and Eigenvectors

For a square M × M matrix C and a vector x that is not all zeros, the values of λ satisfying

are called the eigenvalues of C . The N-vector ?x satisfying the equation above for an eigenvalue λ is the corresponding right eigenvector.

How to Calculate

The eigenvalues of C are then the solutions of

|(C ? λIM)| = 0,

where |S| denotes the determinant of a square matrix S.

For each value of  λ, we can calculate the corresponding eigenvector x through solving the following equation:

This article gives a specific example of the calculating process.

Matrix Decompositions

Matrix diagonalization theorem

Let S be a square real-valued M × M matrix with M linearly independent eigenvectors. Then there exists an eigen decomposition

where the columns of U are the eigenvectors of S and Λ is a diagonal matrix whose diagonal entries are the eigenvalues of S in decreasing order

If the eigenvalues are distinct, then this decomposition is unique.

Symmetric diagonalization theorem

Let S be a square, symmetric real-valued M × M matrix with M linearly independent eigenvectors. Then there exists a symmetric diagonal decomposition

S = QΛQT

where the columns of Q are the orthogonal and normalized (unit length, real) eigenvectors of S, and Λ is the diagonal matrix whose entries are the eigenvalues of S.

Further, all entries of Q are real and we have Q?1 = QT.

Singular value decompositions

Let r be the rank of the M × N matrix C. Then, there is a singular- value decomposition (SVD for short) of C of the form

where

1. U is the M × M matrix whose columns are the orthogonal eigenvectors of CCT.

2. V is the N × N matrix whose columns are the orthogonal eigenvectors of CTC.

3. 

The values σi are referred to as the singular values of C.

Here is the illustration of the singular-value decomposition.

原文地址:https://www.cnblogs.com/sherrydatascience/p/10293948.html

时间: 2024-10-07 12:17:14

[Math Review] Linear Algebra for Singular Value Decomposition (SVD)的相关文章

机器学习学习笔记 PRML Chapter 2.0 : Prerequisite 2 -Singular Value Decomposition (SVD)

Chapter 2.0 : Prerequisite 2 -Singular Value Decomposition (SVD) Chapter 2.0 : Prerequisite 2 -Singular Value Decomposition (SVD) Christopher M. Bishop, PRML, Chapter 2 Probability Distributions 1. Vector Terminology Orthogonality Two vectors and are

Machine Learning - III. Linear Algebra Review (Week 1, Optional)

机器学习Machine Learning - Andrew NG courses学习笔记 矩阵和向量及其表示介绍 what are matrices矩阵 matrix is just another way for saying, is a 2D or a two dimensional array. dimension of the matrix is going to be written as the number of row times the number of columns in

Coursera公开课机器学习:Linear Algebra Review(选修)

这节主要是回顾了下线性代数的一些简单知识. 矩阵与向量 矩阵 由$m\times n$个数$a _{ij}(i=1,2,...,m;j=1,2,...,n)$排成的$m$行$n$列的数表,称为$m$行$n$列的矩阵,简称$m\times n$矩阵,记作: $$ \matrix{A}= \begin{bmatrix} a _{11} & a _{12} & \cdots & a _{1n} \cr a _{21} & a _{22} & \cdots & a

线性代数导论 | Linear Algebra 课程

搞统计的线性代数和概率论必须精通,最好要能锻炼出直觉,再学机器学习才会事半功倍. 线性代数只推荐Prof. Gilbert Strang的MIT课程,有视频,有教材,有习题,有考试,一套学下来基本就入门了. 不多,一共10次课. 链接:https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/calendar/ SES # TOPICS KEY DATES 1 The geometry of linear e

We Recommend a Singular Value Decomposition

We Recommend a Singular Value Decomposition Introduction The topic of this article, the singular value decomposition, is one that should be a part of the standard mathematics undergraduate curriculum but all too often slips between the cracks. Beside

奇异值分解(We Recommend a Singular Value Decomposition)

奇异值分解(We Recommend a Singular Value Decomposition) 原文作者:David Austin原文链接: http://www.ams.org/samplings/feature-column/fcarc-svd译者:richardsun(孙振龙) 在这篇文章中,我们以几何的视角去观察矩阵奇异值分解的过程,并且列举一些奇异值分解的应用. 介绍 矩阵奇异值分解是本科数学课程中的必学部分,但往往被大家忽略.这个分解除了很直观,更重要的是非常具有实用价值.譬如

A Linear Algebra Problem(唯一性的判定)

A Linear Algebra Problem Time Limit: 3000/1000MS (Java/Others)     Memory Limit: 65535/65535KB (Java/Others) Submit Status God Kufeng is the God of Math. However, Kufeng is not so skilled with linear algebra, especially when dealing with matrixes. On

Here’s just a fraction of what you can do with linear algebra

Here’s just a fraction of what you can do with linear algebra The next time someone wonders what the point of linear algebra is, send them here. I write a blog on math and programming and I see linear algebra applied to computer science all the time.

cdoj793-A Linear Algebra Problem

http://acm.uestc.edu.cn/#/problem/show/793 A Linear Algebra Problem Time Limit: 3000/1000MS (Java/Others)     Memory Limit: 65535/65535KB (Java/Others) Submit Status God Kufeng is the God of Math. However, Kufeng is not so skilled with linear algebra