文章目錄
- 1. 前置知識
- 1.1. 向量叉乘
- 1.2. 混合積
- 1.3. 引理證明
- 2. 本質矩陣
- 3. 基礎矩陣
- 4. 應用例子
1. 前置知識
1.1. 向量叉乘
假設 a = ( a x a y a z ) \mathbf{a} = \begin{pmatrix} a_x \\ a_y \\ a_z \end{pmatrix} a= ?ax?ay?az?? ? 以及 b = ( b x b y b z ) \mathbf{b} = \begin{pmatrix} b_x \\ b_y \\ b_z \end{pmatrix} b= ?bx?by?bz?? ?,叉乘 a × b \mathbf{a} \times \mathbf{b} a×b 的矩陣表示為:
a × b = ( 0 ? a z a y a z 0 ? a x ? a y a x 0 ) ( b x b y b z ) = [ a ] × b (1) \mathbf{a} \times \mathbf{b} = \begin{pmatrix} 0 & -a_z & a_y \\ a_z & 0 & -a_x \\ -a_y & a_x & 0 \end{pmatrix} \begin{pmatrix} b_x \\ b_y \\ b_z \end{pmatrix} = [\mathbf{a}]_{\times} \mathbf{b} \tag{1} a×b= ?0az??ay???az?0ax??ay??ax?0? ? ?bx?by?bz?? ?=[a]×?b(1)其中, [ a ] × = ( 0 ? a z a y a z 0 ? a x ? a y a x 0 ) [\mathbf{a}]_{\times} = \begin{pmatrix} 0 & -a_z & a_y \\ a_z & 0 & -a_x \\ -a_y & a_x & 0 \end{pmatrix} [a]×?= ?0az??ay???az?0ax??ay??ax?0? ? 為反對稱矩陣,且有 [ a ] × T = ? [ a ] × [\mathbf{a}]_{\times}^T = -[\mathbf{a}]_{\times} [a]×T?=?[a]×?。
1.2. 混合積
設 a \mathbf{a} a、 b \mathbf{b} b、 c \mathbf{c} c 是三個向量,則混合積的定義為: a ? ( b × c ) \mathbf{a} \cdot (\mathbf{b} \times \mathbf{c}) a?(b×c)。更進一步地,設 a = a 1 i + a 2 j + a 3 k \mathbf{a} = a_1 \mathbf{i} + a_2 \mathbf{j} + a_3 \mathbf{k} a=a1?i+a2?j+a3?k、 b = b 1 i + b 2 j + b 3 k \mathbf{b} = b_1 \mathbf{i} + b_2 \mathbf{j} + b_3 \mathbf{k} b=b1?i+b2?j+b3?k、 c = c 1 i + c 2 j + c 3 k \mathbf{c} = c_1 \mathbf{i} + c_2 \mathbf{j} + c_3 \mathbf{k} c=c1?i+c2?j+c3?k,則有:
a ? ( b × c ) = ∣ a 1 a 2 a 3 b 1 b 2 b 3 c 1 c 2 c 3 ∣ (2) \mathbf{a} \cdot (\mathbf{b} \times \mathbf{c}) = \begin{vmatrix} a_{1} & a_{2} & a_{3} \\ b_{1} & b_{2} & b_{3} \\ c_{1} & c_{2} & c_{3} \end{vmatrix} \tag{2} a?(b×c)= ?a1?b1?c1??a2?b2?c2??a3?b3?c3?? ?(2)證明:
a ? ( b × c ) = ( a 1 i + a 2 j + a 3 k ) ? ∣ i j k b 1 b 2 b 3 c 1 c 2 c 3 ∣ = ( a 1 i + a 2 j + a 3 k ) ? ( i ∣ b 2 b 3 c 2 c 3 ∣ ? j ∣ b 1 b 3 c 1 c 3 ∣ + k ∣ b 1 b 2 c 1 c 2 ∣ ) = a 1 ∣ b 2 b 3 c 2 c 3 ∣ ? a 2 ∣ b 1 b 3 c 1 c 3 ∣ + a 3 ∣ b 1 b 2 c 1 c 2 ∣ = ∣ a 1 a 2 a 3 b 1 b 2 b 3 c 1 c 2 c 3 ∣ \begin{align*} \mathbf{a}\cdot(\mathbf{b}\times\mathbf{c}) &= \left(a_{1}\mathbf{i}+a_{2}\mathbf{j}+a_{3}\mathbf{k}\right)\cdot \begin{vmatrix} \mathbf{i} & \mathbf{j} & \mathbf{k} \\ b_{1} & b_{2} & b_{3} \\ c_{1} & c_{2} & c_{3} \end{vmatrix} \\ &= \left(a_{1}\mathbf{i}+a_{2}\mathbf{j}+a_{3}\mathbf{k}\right)\cdot \left( \mathbf{i} \begin{vmatrix} b_{2} & b_{3} \\ c_{2} & c_{3} \end{vmatrix} - \mathbf{j} \begin{vmatrix} b_{1} & b_{3} \\ c_{1} & c_{3} \end{vmatrix} + \mathbf{k} \begin{vmatrix} b_{1} & b_{2} \\ c_{1} & c_{2} \end{vmatrix} \right) \\ &= a_{1}\begin{vmatrix} b_{2} & b_{3} \\ c_{2} & c_{3} \end{vmatrix} - a_{2}\begin{vmatrix} b_{1} & b_{3} \\ c_{1} & c_{3} \end{vmatrix} + a_{3}\begin{vmatrix} b_{1} & b_{2} \\ c_{1} & c_{2} \end{vmatrix} \\ &= \begin{vmatrix} a_{1} & a_{2} & a_{3} \\ b_{1} & b_{2} & b_{3} \\ c_{1} & c_{2} & c_{3} \end{vmatrix} \end{align*} a?(b×c)?=(a1?i+a2?j+a3?k)? ?ib1?c1??jb2?c2??kb3?c3?? ?=(a1?i+a2?j+a3?k)?(i ?b2?c2??b3?c3?? ??j ?b1?c1??b3?c3?? ?+k ?b1?c1??b2?c2?? ?)=a1? ?b2?c2??b3?c3?? ??a2? ?b1?c1??b3?c3?? ?+a3? ?b1?c1??b2?c2?? ?= ?a1?b1?c1??a2?b2?c2??a3?b3?c3?? ??證畢。
行列式具有性質:互換行列式的兩行或兩列,行列式的值變號。因此有:
a ? ( b × c ) = ∣ a 1 a 2 a 3 b 1 b 2 b 3 c 1 c 2 c 3 ∣ = ∣ b 1 b 2 b 3 c 1 c 2 c 3 a 1 a 2 a 3 ∣ = b ? ( c × a ) = ∣ c 1 c 2 c 3 a 1 a 2 a 3 b 1 b 2 b 3 ∣ = c ? ( a × b ) \begin{align*} \mathbf{a}\cdot(\mathbf{b}\times\mathbf{c}) &= \begin{vmatrix} a_{1} & a_{2} & a_{3} \\ b_{1} & b_{2} & b_{3} \\ c_{1} & c_{2} & c_{3} \end{vmatrix} \\ &= \begin{vmatrix} b_{1} & b_{2} & b_{3} \\ c_{1} & c_{2} & c_{3} \\ a_{1} & a_{2} & a_{3} \\ \end{vmatrix} = \mathbf{b}\cdot(\mathbf{c}\times\mathbf{a}) \\ &= \begin{vmatrix} c_{1} & c_{2} & c_{3} \\ a_{1} & a_{2} & a_{3} \\ b_{1} & b_{2} & b_{3} \\ \end{vmatrix} = \mathbf{c} \cdot (\mathbf{a} \times \mathbf{b}) \end{align*} a?(b×c)?= ?a1?b1?c1??a2?b2?c2??a3?b3?c3?? ?= ?b1?c1?a1??b2?c2?a2??b3?c3?a3?? ?=b?(c×a)= ?c1?a1?b1??c2?a2?b2??c3?a3?b3?? ?=c?(a×b)?所以:
a ? ( b × c ) = b ? ( c × a ) = c ? ( a × b ) (3) \mathbf{a}\cdot(\mathbf{b}\times\mathbf{c}) = \mathbf{b}\cdot(\mathbf{c}\times\mathbf{a}) = \mathbf{c} \cdot (\mathbf{a} \times \mathbf{b}) \tag{3} a?(b×c)=b?(c×a)=c?(a×b)(3)此外,如果 a \mathbf{a} a、 b \mathbf{b} b、 c \mathbf{c} c 中任意兩個向量相等,則混合積等于零。不失一般性,假設 a = b \mathbf{a} = \mathbf{b} a=b,則有:
a ? ( a × c ) = a ? ( c × a ) = c ? ( a × a ) = 0 \mathbf{a}\cdot(\mathbf{a}\times\mathbf{c}) = \mathbf{a}\cdot(\mathbf{c}\times\mathbf{a}) = \mathbf{c} \cdot (\mathbf{a} \times \mathbf{a}) = 0 a?(a×c)=a?(c×a)=c?(a×a)=0
1.3. 引理證明
引理一: R ( a × b ) = ( R a ) × ( R b ) \mathbf{R}(\mathbf{a} \times \mathbf{b}) = (\mathbf{R} \mathbf{a}) \times (\mathbf{R} \mathbf{b}) R(a×b)=(Ra)×(Rb)。
證明:
對于任意向量 v \mathbf{v} v:
- R ( a × b ) ? v = [ R ( a × b ) ] T v = ( a × b ) T R T v = ( a × b ) ? ( R T v ) \mathbf{R}(\mathbf{a} \times \mathbf{b}) \cdot \mathbf{v} = [\mathbf{R}(\mathbf{a} \times \mathbf{b})]^T \mathbf{v} = (\mathbf{a} \times \mathbf{b})^T \mathbf{R}^T \mathbf{v} = (\mathbf{a} \times \mathbf{b}) \cdot (\mathbf{R}^T\mathbf{v}) R(a×b)?v=[R(a×b)]Tv=(a×b)TRTv=(a×b)?(RTv)。
- 根據混合積、向量點積和旋轉矩陣 ( R R T = I , det ? ( R ) = 1 ) (\mathbf{R} \mathbf{R}^T = \mathbf{I}, \det(\mathbf{R}) = 1) (RRT=I,det(R)=1) 的性質有:
[ ( R a ) × ( R b ) ] ? v = v ? [ ( R a ) × ( R b ) ] = det ? ( [ v R a R b ] ) = det ? ( R [ R T v a b ] ) = det ? ( R ) det ? ( [ R T v a b ] ) = det ? ( [ R T v a b ] ) = ( R T v ) ? ( a × b ) = ( a × b ) ? ( R T v ) \begin{align*} [(\mathbf{R} \mathbf{a}) \times (\mathbf{R} \mathbf{b})] \cdot \mathbf{v} &= \mathbf{v} \cdot [(\mathbf{R} \mathbf{a}) \times (\mathbf{R} \mathbf{b})] = \det([\mathbf{v} \quad \mathbf{R} \mathbf{a} \quad \mathbf{R} \mathbf{b}]) = \det(\mathbf{R}[\mathbf{R}^T\mathbf{v} \quad \mathbf{a} \quad \mathbf{b}]) \\ &= \det(\mathbf{R}) \det([\mathbf{R}^T\mathbf{v} \quad \mathbf{a} \quad \mathbf{b}]) = \det([\mathbf{R}^T\mathbf{v} \quad \mathbf{a} \quad \mathbf{b}]) \\ &= (\mathbf{R}^T\mathbf{v}) \cdot (\mathbf{a} \times \mathbf{b}) = (\mathbf{a} \times \mathbf{b}) \cdot (\mathbf{R}^T\mathbf{v}) \end{align*} [(Ra)×(Rb)]?v?=v?[(Ra)×(Rb)]=det([vRaRb])=det(R[RTvab])=det(R)det([RTvab])=det([RTvab])=(RTv)?(a×b)=(a×b)?(RTv)?
綜上有,對于任意向量 v \mathbf{v} v 有:
R ( a × b ) ? v = ( R a × R b ) ? v ? R ( a × b ) = ( R a ) × ( R b ) \mathbf{R}(\mathbf{a} \times \mathbf{b}) \cdot \mathbf{v} = (\mathbf{R} \mathbf{a} \times \mathbf{R} \mathbf{b}) \cdot \mathbf{v} \Rightarrow \mathbf{R}(\mathbf{a} \times \mathbf{b}) = (\mathbf{R} \mathbf{a}) \times (\mathbf{R} \mathbf{b}) R(a×b)?v=(Ra×Rb)?v?R(a×b)=(Ra)×(Rb)證畢。
2. 本質矩陣
由上圖可知,本質矩陣 E \mathbf{E} E 在極線的計算中起著重要的作用。本質矩陣的計算方式如下: E = [ T ] × R = R [ R T T ] × (4) \mathbf{E} = [\mathbf{T}]_{\times} \mathbf{R} = \mathbf{R} [\mathbf{R}^T \mathbf{T}]_{\times} \tag{4} E=[T]×?R=R[RTT]×?(4)其中, R \mathbf{R} R 和 T \mathbf{T} T 表示從圖像 I 1 I_1 I1? 到圖像 I 2 I_2 I2? 的旋轉矩陣和平移向量。
現在我們來證明 [ T ] × R = R [ R T T ] × [\mathbf{T}]_{\times} \mathbf{R} = \mathbf{R} [\mathbf{R}^T \mathbf{T}]_{\times} [T]×?R=R[RTT]×?。證明如下:
對于任意向量 v \mathbf{v} v 有:
- [ T ] × R v = T × ( R v ) [\mathbf{T}]_{\times} \mathbf{R} \mathbf{v} = \mathbf{T} \times (\mathbf{R} \mathbf{v}) [T]×?Rv=T×(Rv)
- R [ R T T ] × v = R ( R T T × v ) \mathbf{R} [\mathbf{R}^T \mathbf{T}]_{\times} \mathbf{v} = \mathbf{R} (\mathbf{R}^T\mathbf{T} \times \mathbf{v}) R[RTT]×?v=R(RTT×v)
令 T ′ = R T T \mathbf{T}^{\prime} = \mathbf{R}^T\mathbf{T} T′=RTT,則 T = R T ′ \mathbf{T} = \mathbf{R} \mathbf{T}^{\prime} T=RT′,根據引理一有:
T × ( R v ) = ( R T ′ ) × ( R v ) = R ( T ′ × v ) = R ( R T T × v ) = R [ R T T ] × v \mathbf{T} \times (\mathbf{R} \mathbf{v}) = (\mathbf{R} \mathbf{T}^{\prime}) \times (\mathbf{R} \mathbf{v}) = \mathbf{R}(\mathbf{T}^{\prime} \times \mathbf{v}) = \mathbf{R}(\mathbf{R}^T\mathbf{T} \times \mathbf{v}) = \mathbf{R} [\mathbf{R}^T \mathbf{T}]_{\times} \mathbf{v} T×(Rv)=(RT′)×(Rv)=R(T′×v)=R(RTT×v)=R[RTT]×?v所以,對于任意向量 v \mathbf{v} v 有:
[ T ] × R v = R [ R T T ] × v ? [ T ] × R = R [ R T T ] × [\mathbf{T}]_{\times} \mathbf{R} \mathbf{v} = \mathbf{R} [\mathbf{R}^T \mathbf{T}]_{\times} \mathbf{v} \Rightarrow [\mathbf{T}]_{\times} \mathbf{R} = \mathbf{R} [\mathbf{R}^T \mathbf{T}]_{\times} [T]×?Rv=R[RTT]×?v?[T]×?R=R[RTT]×?證畢。
3. 基礎矩陣
基礎矩陣與本質矩陣的關系如下:
F = K ′ ? T E K ? 1 = K ′ ? T [ T ] × R K ? 1 (5) \mathbf{F} = \mathbf{K}^{\prime-T} \mathbf{E} \mathbf{K}^{-1} = \mathbf{K}^{\prime-T} [\mathbf{T}]_{\times} \mathbf{R} \mathbf{K}^{-1} \tag{5} F=K′?TEK?1=K′?T[T]×?RK?1(5)其中, K \mathbf{K} K 和 K ′ \mathbf{K}^{\prime} K′ 分別為圖像 I 1 I_1 I1? 和 I 2 I_2 I2? 對應的內參矩陣。
l 1 l_1 l1? 和 l 2 l_2 l2? 極線的方程為:
{ l 2 = F x 1 l 1 = F T x 2 (6) \begin{cases} l_2 = \mathbf{F} \mathbf{x}_1 \\ l_1 = \mathbf{F}^T \mathbf{x}_2 \end{cases} \tag{6} {l2?=Fx1?l1?=FTx2??(6)
4. 應用例子
假設圖像 I 1 I_1 I1? 和 I 2 I_2 I2? 到世界坐標系的變換分別為 R s \mathbf{R}_s Rs?、 T s \mathbf{T}_s Ts? 和 R t \mathbf{R}_t Rt?、 T t \mathbf{T}_t Tt?。已知圖像 I 1 I_1 I1? 存在點 x \mathbf{x} x,求對應的極線方程。
代碼如下:
R = R_t @ R_s.inverse()
T = R_s @ R_t.inverse() @ T_t - T_s
T = T.squeeze()S = torch.zeros((3, 3))
S[0, 1] = -T[2]
S[1, 0] = T[2]
S[0, 2] = T[1]
S[2, 0] = -T[1]
S[1, 2] = -T[0]
S[2, 1] = T[0]E = R @ S
F = K_t.inverse().transpose(0, 1) @ E @ K_s.inverse()epipolar_line = F @ x
a = epipolar_line[0]
b = epipolar_line[1]
c = epipolar_line[2]
上述代碼修改自:https://github.com/USMizuki/NexusGS/blob/main/utils/flow_utils.py。
設點 x \mathbf{x} x 對應的空間點為 X \mathbf{X} X,在圖像 I 2 I_2 I2? 的對應點為 x ′ x^{\prime} x′,現在我們來進行理論推導:
{ x = R s X + T s x ′ = R t X + T t \begin{cases} \mathbf{x} = \mathbf{R}_s \mathbf{X} + \mathbf{T}_s \\ \mathbf{x}^{\prime} = \mathbf{R}_t \mathbf{X} + \mathbf{T}_t \end{cases} {x=Rs?X+Ts?x′=Rt?X+Tt??將 X = R s ? 1 ( x ? T s ) \mathbf{X} = \mathbf{R}_s^{-1} (\mathbf{x} - \mathbf{T}_s) X=Rs?1?(x?Ts?) 帶入 x ′ = R t X + T t \mathbf{x}^{\prime} = \mathbf{R}_t \mathbf{X} + \mathbf{T}_t x′=Rt?X+Tt? 有:
x ′ = R t R s ? 1 ( x ? T s ) + T t = R t R s ? 1 x ? R t R s ? 1 T s + T t \mathbf{x}^{\prime} = \mathbf{R}_t \mathbf{R}_s^{-1} (\mathbf{x} - \mathbf{T}_s) + \mathbf{T}_t = \mathbf{R}_t \mathbf{R}_s^{-1} \mathbf{x} - \mathbf{R}_t \mathbf{R}_s^{-1} \mathbf{T}_s + \mathbf{T}_t x′=Rt?Rs?1?(x?Ts?)+Tt?=Rt?Rs?1?x?Rt?Rs?1?Ts?+Tt?所以,從圖像 I 1 I_1 I1? 到 I 2 I_2 I2? 的旋轉矩陣 R \mathbf{R} R 和平移向量 T \mathbf{T} T 的計算公式如下:
{ R = R t R s ? 1 T = ? R t R s ? 1 T s + T t (7) \begin{cases} \mathbf{R} = \mathbf{R}_t \mathbf{R}_s^{-1} \\ \mathbf{T} = - \mathbf{R}_t \mathbf{R}_s^{-1} \mathbf{T}_s + \mathbf{T}_t \end{cases} \tag{7} {R=Rt?Rs?1?T=?Rt?Rs?1?Ts?+Tt??(7) T \mathbf{T} T 的計算公式與代碼不符,不妨設代碼中對應 T ′ \mathbf{T}^{\prime} T′,則有:
T ′ = R s R t ? 1 T t ? T s \mathbf{T}^{\prime} = \mathbf{R}_s \mathbf{R}_t^{-1} \mathbf{T}_t - \mathbf{T}_s T′=Rs?Rt?1?Tt??Ts?不難發現 T \mathbf{T} T 和 T ′ \mathbf{T}^{\prime} T′ 存在如下關系:
R t R s ? 1 T ′ = R t R s ? 1 ( R s R t ? 1 T t ? T s ) = T t ? R t R s ? 1 T s = T \mathbf{R}_t \mathbf{R}_s^{-1} \mathbf{T}^{\prime} = \mathbf{R}_t \mathbf{R}_s^{-1} (\mathbf{R}_s \mathbf{R}_t^{-1} \mathbf{T}_t - \mathbf{T}_s) = \mathbf{T}_t - \mathbf{R}_t \mathbf{R}_s^{-1} \mathbf{T}_s = \mathbf{T} Rt?Rs?1?T′=Rt?Rs?1?(Rs?Rt?1?Tt??Ts?)=Tt??Rt?Rs?1?Ts?=T即有: R T ′ = T ? T ′ = R ? 1 T = R T T \mathbf{R} \mathbf{T}^{\prime} = \mathbf{T} \Rightarrow \mathbf{T}^{\prime} = \mathbf{R}^{-1} \mathbf{T} = \mathbf{R}^T \mathbf{T} RT′=T?T′=R?1T=RTT
則本質矩陣的計算方式為:
E = [ T ] × R = R [ R T T ] × = R [ T ′ ] × \mathbf{E} = [\mathbf{T}]_{\times} \mathbf{R} = \mathbf{R} [\mathbf{R}^T \mathbf{T}]_{\times} = \mathbf{R} [\mathbf{T}^{\prime}]_{\times} E=[T]×?R=R[RTT]×?=R[T′]×?可以看出代碼中的 E \mathbf{E} E 采取的計算方式是 R [ T ′ ] × \mathbf{R} [\mathbf{T}^{\prime}]_{\times} R[T′]×?。