幂迭代法求特征值和特征向量

幂迭代法求第k大的特征值和特征向量

数学表述

设矩阵A
\[
A = \left[ \begin{matrix}
X_{11}&\ldots&X_{1n} \newline
X_{21} & \ldots& X_{2n} \newline
&\vdots& \newline
X_{n1}&\ldots &X_{nn} \newline
\end{matrix} \right]
\]
求其最大特征值对应的特征向量\(b_k = [v_1,v_2,\ldots,v_n]^T\):
\[b_{k+1} = \frac{Ab_k}{||Ab_k ||}..........(1)\]

证明：见英文wiki Power_iteration

代码：

import numpy as np
def power_iteration(A, num_simulations: int):
    # Ideally choose a random vector
    # To decrease the chance that our vector
    # Is orthogonal to the eigenvector
    b_k = np.random.rand(A.shape[1])

    for _ in range(num_simulations):
        # calculate the matrix-by-vector product Ab
        b_k1 = np.dot(A, b_k)

        # calculate the norm
        b_k1_norm = np.linalg.norm(b_k1)

        # re normalize the vector
        b_k = b_k1 / b_k1_norm

    return b_k

例子

\[
A = \left[ \begin{matrix}
2&1\newline
1&2\newline
\end{matrix} \right]
\]

求其特征值：
\[
|A-\lambda I| = \left| \begin{matrix}
2-\lambda &1\newline
1 &2-\lambda\newline
\end{matrix} \right| = 3 - 4\lambda + \lambda^2
\]
求得最大特征值\(\lambda_2 = 3\)，另一特征值\(\lambda_1 = 1\)
\(\lambda_2 = 3\) 时：
\[
|(A-\lambda I)V| = 0
\]

\[
(A-3I)V = \left[ \begin{array} {cccc}
-1 &1\newline
1 &-1\newline
\end{array} \right] \left[ \begin{array} {cccc}
v_1\newline
v_2\newline
\end{array} \right] = \left[ \begin{array} {cccc}
0\newline
0\newline
\end{array} \right]
\]

求得\(v_1 = v_2\),归一化后，\(V_{\lambda=3} = [0.707,0.707]^T\)

验证代码：

A = np.array([[2,1],[1,2]])
V = power_iteration(A, 100).reshape(1,-1)
V

array([[0.70710678, 0.70710678]])

求得特征向量后，特征值
\[
\lambda = VAV^T........(2)
\]
代码

lbd = np.dot(V,np.dot(A,V.T))
lbd 

array([[3.]])

扩展

求次大特征向量与特征值:
\[
B = A - \frac{\lambda}{||V||^2}V^TV ........(3)
\]

\[
b_{k+1}' = \frac{Bb_k'}{||Bb_k' ||} ...........(4)
\]

代码

B = A - lbd / np.linalg.norm(V)**2 * np.dot(V.T,V)
V_2 = power_iteration(B, 100)
lbd1 = np.dot(V_2.T,np.dot(B,V_2))
lbd1

1.0000000000000002

以此类推，我们可以得到任意第k大特征值。这在很多机器学习方法中，取前k重要特征有着重要的作用。
欢迎访问个人博客。转载请注明出处。

原文地址：https://www.cnblogs.com/fahaizhong/p/12240051.html