Camera Model

Pinhole Camera Model

pinhole-img

Let P=[xyz]TP = \begin{bmatrix} x & y & z \end{bmatrix}^T be a 3D point in camera coordinates. By similar triangles with focal length ff:

P′=[x′y′]=[fxzfyz]P' = \begin{bmatrix} x' \\ y' \end{bmatrix} = \begin{bmatrix} f\frac{x}{z} \\ f\frac{y}{z} \end{bmatrix}

Paraxial Refraction Model

paraxial-img

Refraction model with effective image distance z′z':

P′=[z′xzz′yz]P' = \begin{bmatrix} z'\frac{x}{z} \\ z'\frac{y}{z}\end{bmatrix}

Camera Matrix Model

Accounting for pixel scaling factors (α=fkx,β=fky)(\alpha = f k_x, \beta = f k_y) and principal point offset (cx,cy)(c_x, c_y):

P′=[fkxz+cxflyz+cy]=[αxz+cxβyz+cy]P' = \begin{bmatrix} fk\frac{x}{z} + c_x \\ fl\frac{y}{z} + c_y \end{bmatrix}= \begin{bmatrix} \alpha\frac{x}{z} + c_x \\ \beta \frac{y}{z} + c_y\end{bmatrix}

Since the perspective projection is non-linear, homogeneous coordinates are used hereafter to matrix-formulate the transformations.

P′=[x′y′z′]=[α0cx00βcy00010][xyz1]=[α0cx0βcy001][I0]P=K[I0]P=MPP' = \begin{bmatrix} x'\\ y'\\ z' \end{bmatrix} = \begin{bmatrix} \alpha & 0 & c_x & 0 \\ 0 & \beta & c_y & 0 \\ 0 & 0 & 1 & 0 \end{bmatrix} \begin{bmatrix} x\\ y\\ z\\ 1 \end{bmatrix} = \begin{bmatrix} \alpha & 0 & c_x\\ 0 & \beta & c_y\\ 0 & 0 & 1 \end{bmatrix} \begin{bmatrix} I & 0 \end{bmatrix} P = K\begin{bmatrix}I & 0\end{bmatrix} P= MP

Accounting for potential sensor skewness between image axes, the intrinsic matrix is generalized as:

K=[α−αcot⁡θcx0βsin⁡θcy001]K = \begin{bmatrix} \alpha & -\alpha \cot{\theta} & c_x\\0 & \frac{\beta}{\sin{\theta}} & c_y\\0 & 0 & 1\end{bmatrix}

KK has 5 degrees of freedom, which also known as intrinsic parameters.

Extrinsic Parameters

Transforming 3D points from World Frame PwP_w to Camera Frame PcP_c using rotation R∈SO(3)R \in \mathrm{SO}(3) and translation T∈R3T \in \mathbb{R}^3:

Pc=[RT01]PwP_c = \begin{bmatrix} R & T \\ 0 & 1\end{bmatrix}P_w

Full Camera Projection Pipeline:

P′=K[RT]Pw=MPwP' = K\begin{bmatrix} R & T \end{bmatrix} P_w = MP_w

3D Rotation & Quaternion

Rodrigues’ Rotation Formula

Rotates vector v\mathbf{v} around unit axis u\mathbf{u} (∥u∥=1\|\mathbf{u}\|=1) by angle θ\theta:

v′=cos⁡θv+(1−cos⁡θ)(u⋅v)u+sin⁡θ(u×v)\mathbf{v'} = \cos{\theta}\mathbf{v} + (1-\cos{\theta})(\mathbf{u}\cdot \mathbf{v})\mathbf{u} + \sin{\theta}(\mathbf{u} \times \mathbf{v})

Quaternion

Definition

q=a+bi+cj+dk=[s,v](s=a∈R,  v=[b,c,d]T∈R3)q = a + bi + cj + dk = [s, \mathbf{v}] \quad (s = a \in \mathbb{R}, \; \mathbf{v} = [b, c, d]^T \in \mathbb{R}^3)

Hamilton Fundamental Formula: i2=j2=k2=ijk=−1i^2 = j^2 = k^2 = ijk = -1

×\times11iijjkk
1111iijjkk
iiii−1-1kk−j-j
jjjj−k-k−1-1ii
kkkkjj−i-i−1-1

Operations

  1. Left Multiplication Matrix Operator [q1]L[q_1]_L:
q1q2=[a−b−c−dba−dccda−bd−cba][efgh]q_1 q_2 = \begin{bmatrix}a & -b & -c & -d \\b & a & -d & c \\c & d & a & -b \\d & -c & b & a\end{bmatrix}\begin{bmatrix} e \\ f \\ g \\ h \end{bmatrix}
  1. Vector Form Multiplication:
    For q1=[s,v]q_1 = [s, \mathbf{v}], q2=[t,u]q_2 = [t, \mathbf{u}]:
q1q2=[st−v⋅u,  su+tv+v×u]q_1 q_2 = [st - \mathbf{v}\cdot\mathbf{u}, \; s\mathbf{u} + t\mathbf{v} + \mathbf{v}\times\mathbf{u}]
  1. Conjugate & Inverse:
(q1q2)∗=q2∗q1∗where q∗=[s,−v](q_1 q_2)^* = q_2^* q_1^* \quad \text{where } q^* = [s, -\mathbf{v}] q−1=q∗∥q∥2(if ∥q∥=1,  q−1=q∗)q^{-1} = \frac{q^*}{\|q\|^2} \quad (\text{if } \|q\|=1, \; q^{-1} = q^*)

3D Rotation via Quaternion

Let pure quaternion v=[0,v]v = [0, \mathbf{v}] and unit quaternion q=[cos⁡θ2,sin⁡θ2u]q = [\cos\frac{\theta}{2}, \sin\frac{\theta}{2}\mathbf{u}]:

v′=qvq∗=qvq−1v' = q v q^* = q v q^{-1}

Expanding v′=qvq∗v' = q v q^* into 3×33 \times 3 Rotation Matrix RqR_q (a=cos⁡θ2,[b,c,d]T=sin⁡θ2ua = \cos\frac{\theta}{2}, [b,c,d]^T = \sin\frac{\theta}{2}\mathbf{u}):

v′=Rqv=[1−2c2−2d22bc−2ad2bd+2ac2bc+2ad1−2b2−2d22cd−2ab2bd−2ac2cd+2ab1−2b2−2c2]v\mathbf{v'} = R_q \mathbf{v} = \begin{bmatrix}1 - 2c^2 - 2d^2 & 2bc - 2ad & 2bd + 2ac \\2bc + 2ad & 1 - 2b^2 - 2d^2 & 2cd - 2ab \\2bd - 2ac & 2cd + 2ab & 1 - 2b^2 - 2c^2\end{bmatrix} \mathbf{v}

References