2D-2D:对极几何

对极约束

\[
\boldsymbol{x}_{2}^{T} \boldsymbol{F} \boldsymbol{x}_{1}=\boldsymbol{0} \quad \hat{\boldsymbol{x}}_{2}^{T} \boldsymbol{E} \hat{\boldsymbol{x}}_{1}=\mathbf{0}
\]

其中

\[
\boldsymbol{E}=\boldsymbol{K}_{2}^{-T} \boldsymbol{F K}_{1} \quad \hat{\boldsymbol{x}}_{1}=\boldsymbol{K}_{1}^{-1} \boldsymbol{x}_{1} \quad \hat{\boldsymbol{x}}_{2}=\boldsymbol{K}_{2}^{-1} \boldsymbol{x}_{2}
\]

公式推导

\[
\boldsymbol{P}_{1}=\boldsymbol{K}_{1}[\boldsymbol{I}, \quad \mathbf{0}] \quad \boldsymbol{P}_{2}=\boldsymbol{K}_{2} \left[ \begin{array}{ll}{\boldsymbol{R},} & {\boldsymbol{t}}\end{array}\right]
\]

\[
d_{1} x_{1}=K_{1} X \quad \Rightarrow \quad d_{1} K_{1}^{-1} x_{1}=X=d_{1} \hat{x}_{1}
\]

\[
d_{2} \boldsymbol{x}_{2}=\boldsymbol{K}_{2}(\boldsymbol{R} \boldsymbol{X}+\boldsymbol{t}) \Rightarrow \quad d_{2} \boldsymbol{K}_{2}^{-1} \boldsymbol{x}_{2}=\boldsymbol{R} \boldsymbol{X}+\boldsymbol{t}=d_{1} \boldsymbol{R} \hat{x}_{1}+\boldsymbol{t}
\]

\[
\quad d_{2}[t]_{\times} \hat{x}_{2}=d_{1}[t]_{\times} R \hat{x}_{1}+[t]_{\times} t
\]

\[
\quad d_{2} \hat{x}_{2}^{T}[t]_{\times} \hat{x}_{2}=d_{1} \hat{x}_{2}^{T}[t]_{\times} R \hat{x}_{1}=0
\]

\[
\hat{\boldsymbol{x}}_{2}^{T}[\boldsymbol{t}]_{ \times} \boldsymbol{R} \hat{x}_{1}=\hat{\boldsymbol{x}}_{2}^{T} \boldsymbol{E} \hat{x}_{1}=0
\]

\[
\boldsymbol{E}=[\boldsymbol{t}]_{ \times} \boldsymbol{R}
\]

\[
\boldsymbol{x}_{2}^{T} \boldsymbol{K}_{2}^{-T}[\boldsymbol{t}]_{\times} \boldsymbol{R} \boldsymbol{K}_{1}^{-1} \boldsymbol{x}_{1}=\boldsymbol{x}_{2}^{T} \boldsymbol{F} \boldsymbol{x}_{1}=0
\]

\[
\boldsymbol{F}=\boldsymbol{K}_{2}^{-T} \boldsymbol{E} \boldsymbol{K}_{1}^{-1}
\]

基础矩阵性质

• 3x3的矩阵，秩为2
• 具有7个自由度
• 奇异值为\(\left[ \begin{array}{lll}{\sigma_{1},} & {\sigma_{2},} & {0}\end{array}\right]^{T}\)
• 极线约束\(l_{1}=\boldsymbol{x}_{2}^{T} \boldsymbol{F}, \quad l_{2}=\boldsymbol{F} \boldsymbol{x}_{1}, \quad \boldsymbol{x}_{2}^{T} \boldsymbol{F} \boldsymbol{x}_{1}=0\)

基础矩阵求解方法

直接线性变换法 （8点法+最小二乘法）

对于一对匹配点
\[
\boldsymbol{x}_{1}=\left[ \begin{array}{ll}{u_{1},} & {v_{1},} & {1}\end{array}\right]^{\mathrm{T}}, \quad \boldsymbol{x}_{2}=\left[ \begin{array}{ll}{u_{2},} & {v_{2},} & {1}\end{array}\right]^{\mathrm{T}}
\]
根据对极约束
\[
\boldsymbol{x}_{2}^{T} \boldsymbol{F} \boldsymbol{x}_{1}=\boldsymbol{0}
\]
有
\[
\left( \begin{array}{lll}{u_{1}} & {v_{1}} & {1}\end{array}\right) \left[ \begin{array}{ccc}{F_{11}} & {F_{12}} & {F_{13}} \\ {F_{21}} & {F_{22}} & {F_{23}} \\ {F_{31}} & {F_{32}} & {F_{33}}\end{array}\right] \left( \begin{array}{c}{u_{2}} \\ {v_{2}} \\ {1}\end{array}\right)=0
\]
令
\[
\boldsymbol{f}=\left[ \begin{array}{lll}{F_{11},} & {F_{12},} & {F_{13}}, \quad \end{array}\right.F_{21}, \quad F_{22}, \quad F_{23}, \quad F_{31}, \quad F_{32}, \quad F_{33} ]^{T}
\]
有约束
\[
\left[ \begin{array}{llllll}{u_{1} u_{1},} & {u_{1} v_{2},} & {u_{1},} & {v_{2} u_{1},} & {v_{1} v_{2},} & {v_{1},} & {u_{2},} & {v_{2},} & {1}\end{array}\right] f=0
\]
当有n对匹配点时
\[
A=\left( \begin{array}{ccccccccc}
{\begin{array}{l}
{u_{1}^{(1)} u_{1}^{(1)}, \quad u_{1}^{(1)} v_{2}^{(1)}, \quad u_{1}^{(1)}, \quad v_{1}^{(1)} u_{2}^{(1)}, \quad v_{1}^{(1)} v_{2}^{(1)}, \quad v_{1}^{(1)}, \quad u_{2}^{(1)}, \quad v_{2}^{(1)}, \quad 1}
\\ {u_{1}^{(2)} u_{1}^{(2)}, \quad u_{1}^{(2)} v_{2}^{(2)}, \quad u_{1}^{(2)}, \quad v_{1}^{(2)} u_{2}^{(2)}, \quad v_{1}^{(2)} v_{2}^{(2)}, \quad v_{1}^{(2)}, \quad u_{2}^{(2)}, \quad v_{2}^{(2)}, \quad 1}
\\ \quad \vdots \quad\quad \quad \vdots \quad\quad \quad \vdots\quad \quad\quad \vdots \quad \quad\quad \vdots \quad\quad \vdots\quad \quad \vdots\quad\quad \quad \vdots\quad\quad \vdots
\\u_{1}^{(n)} u_{1}^{(n)}, \quad u_{1}^{(n)} v_{2}^{(n)}, \quad u_{1}^{(n)}, \quad v_{1}^{(n)} u_{2}^{(n)}, \quad v_{1}^{(n)} v_{2}^{(n)}, \quad v_{1}^{(n)}, \quad u_{2}^{(n)}, \quad v_{2}^{(n)}, \quad 1
\end{array}}
\end{array}\right).
\]

\[
A f=0
\]

• 要保证有唯一解至少需要8对匹配点
• n = 8时，若A非奇异，则有唯一解，称为8点法
• n >= 8时，可用最小二乘法求解( $ A^TA $ 的最小特征值对应的特征向量即为最优解)

奇异值约束

直接线性变化法无法保证基础矩阵的奇异值约束—有两个非0奇异值
根据奇异值约束对矩阵进行重构
\[
\min \|\boldsymbol{F}-\hat{\boldsymbol{F}}\|, \quad \text { wrt. } \operatorname{svd}(\boldsymbol{F})=\left[ \begin{array}{lll}{\sigma_{1},} & {\sigma_{2},} & {0}\end{array}\right]
\]
对得到的基础矩阵进行奇异值分解
\[
\hat{\boldsymbol{F}}=\boldsymbol{U S V}^{T} \quad \text { with } S=\operatorname{diag}\left(\sigma_{1}, \sigma_{2}, \sigma_{3}\right)
\]
利用奇异值约束对基础矩阵进行重构
\[
\boldsymbol{F}=\boldsymbol{U} \operatorname{diag}\left(\sigma_{1}, \quad \sigma_{2}, \quad 0\right) \boldsymbol{V}^{T}
\]

RANSAC-估计基础矩阵

算法流程

1. 随机采样8对匹配点
2. 8点法求解基础矩阵
3. 奇异值约束获取基础矩阵
4. 计算误差，并统计内点个数
5. 重复上述过程，选择内点数最多的结果
6. 对所有内点执行2,3，重新计算

内点判断标准-Sampson Distance

\[
\begin{array}{c}{d\left(\boldsymbol{x}_{1}, \boldsymbol{x}_{2}\right)=\frac{\left(\boldsymbol{x}_{2}^{T} \boldsymbol{F} \boldsymbol{x}_{1}\right)^{2}}{\left(\boldsymbol{F} x_{1}\right)_{x}^{2}+\left(\boldsymbol{F} x_{1}\right)_{x}^{2}+\left(\boldsymbol{x}_{2}^{T} \boldsymbol{F}\right)_{x}^{2}+\left(\boldsymbol{x}_{2}^{T} \boldsymbol{F}\right)_{y}^{2}}} \\ {d\left(\boldsymbol{x}_{1}, \boldsymbol{x}_{2}\right)<\tau}\end{array}
\]

RANSAC-采样次数的计算

N - 样本点个数
K - 求解模型需要最少的点的个数
p - 表示内点的概率
\[
K 个点都是内点概率 p^{K}
\]

\[
K个至少有一个外点(采样失败)的概率1-p^{K}
\]

\[
M次采样全部失败的概率\left(1-p^{K}\right)^{M}
\]

\[
M次采样至少有1次成功的概率 \quad z=1-\left(1-p^{K}\right)^{M}
\]

\[
采样次数 \quad M=\frac{\log (1-z)}{\log \left(1-p^{K}\right)}
\]

\[
计算p=0.9, \quad K=8 ， 时，想要采样
成功率达到 z \geq 0.99， 所需要的采样
次数 M
\]

本征矩阵E性质

1. 3x3的矩阵，秩为2
2. 具有5个自由度
3. 奇异值为\(\left[ \begin{array}{lll}{\sigma,} & {\sigma,} & {0}\end{array}\right]^{T}\)

求解本征矩阵

\[
先求解基础矩阵F
\]

\[
\widehat{\boldsymbol{E}}=\boldsymbol{K}_{2}^{T} \boldsymbol{F} \boldsymbol{K}_{1}
\]

\[
\widehat{\boldsymbol{E}}=\boldsymbol{U} \operatorname{diag}\left(\sigma_{1}, \quad \sigma_{2}, \quad 0\right) \boldsymbol{V}^{T}
\]

\[
\boldsymbol{E}=\boldsymbol{U} \operatorname{diag}\left(\frac{\sigma_{1}+\sigma_{2}}{2}, \quad \frac{\sigma_{1}+\sigma_{2}}{2}, \quad 0\right) \boldsymbol{V}^{T}
\]

相机姿态的恢复

求解Rt

\[
\boldsymbol{E}=\boldsymbol{U} \Sigma \boldsymbol{V}^{T}, \boldsymbol{\Sigma}=\operatorname{diag}(\sigma, \quad \sigma, \quad 0)
\]

\[
t_{1}=U( :, 2) \quad R_{1}=U R_{Z}\left(\frac{\pi}{2}\right) V^{T}
\]

\[
\boldsymbol{t}_{2}=-\boldsymbol{U}( :, \boldsymbol{2}) \quad \boldsymbol{R}_{2}=\boldsymbol{U} \boldsymbol{R}_{\mathrm{Z}}^{T}\left(\frac{\pi}{2}\right) \boldsymbol{V}^{T}
\]

\[
\boldsymbol{R}_{z}\left(\frac{\pi}{2}\right)=\left( \begin{array}{ccc}{0,} & {-1,} & {0} \\ {1,} & {0,} & {0} \\ {0,} & {0,} & {1}\end{array}\right), \boldsymbol{R}_{z}^{T}\left(\frac{\pi}{2}\right)=\left( \begin{array}{ccc}{0,} & {1,} & {0} \\ {-1,} & {0,} & {0} \\ {0,} & {0,} & {1}\end{array}\right)
\]

\[
共有4种情况 \left(\boldsymbol{R}_{1}, \boldsymbol{t}_{1}\right),\left(\boldsymbol{R}_{1}, \boldsymbol{t}_{2}\right),\left(\boldsymbol{R}_{2}, \boldsymbol{t}_{1}\right),\left(\boldsymbol{R}_{2}, \boldsymbol{t}_{2}\right)
\]

选择正确的Rt

相机的世界坐标
\[
\boldsymbol{O}_{1}=-\boldsymbol{R}^{T} \boldsymbol{t}=0, \quad \boldsymbol{O}_{2}=-\boldsymbol{R}^{T} \boldsymbol{t}
\]
相机的世界坐标中的朝向
\[
d_{1}=\left[ \begin{array}{lll}{0,} & {0} & {1}\end{array}\right]^{T}
\]

\[
\boldsymbol{d}_{2}=\left[ \begin{array}{cccc}{\boldsymbol{r}_{1}^{T}} & {\boldsymbol{r}_{2}^{T}} & {\boldsymbol{r}_{3}^{T}} & {-\boldsymbol{R}^{T} \boldsymbol{t}}\end{array}\right]\left[ \begin{array}{l}{0} \\ {0} \\ {1} \\ {0}\end{array}\right]=r_{3}^{T}
\]

\[
\boldsymbol{R}=\left[ \begin{array}{c}{\boldsymbol{r}_{1}} \\ {\boldsymbol{r}_{2}} \\ {\boldsymbol{r}_{3}}\end{array}\right] \quad \boldsymbol{R}^{T}=\left[ \begin{array}{ccc}{\boldsymbol{r}_{1}^{T}} & {\boldsymbol{r}_{2}^{T}} & {\boldsymbol{r}_{3}^{T}}\end{array}\right]
\]

利用相机姿态和匹配点进行三角量测得到三维点P

P需满足同时位于两个相机的前方：

方法1：
\[
\begin{array}{l}{\left(\boldsymbol{P}-\boldsymbol{O}_{1}\right)^{T} \boldsymbol{d}_{1}>0} \\ {\left(\boldsymbol{P}-\boldsymbol{O}_{2}\right)^{T} \boldsymbol{d}_{1}>0}\end{array}
\]
方法2：
\[
\left[ \begin{array}{l}{x_{c}} \\ {y_{c}} \\ {z_{c}}\end{array}\right]=\boldsymbol{R P}+\boldsymbol{t}, \quad Z_{c}>0 对两个相机成立
\]

单应矩阵H

假设条件

1. 空间中特征点位于一平面上
2. 纯旋转

原文地址：https://www.cnblogs.com/narjaja/p/10768179.html