Digital Image Processing (II) Image Registration.

This is an automatically translated post by LLM. The original post is in Chinese. If you find any translation errors, please leave a comment to help me improve the translation. Thanks!

Image registration refers to the process of transforming the coordinates of two or more images that have similar content but differ in aspects like angle, size, or geometric position. This transformation aims to align these images to the same standardized size, angle, and coordinates.

Image registration finds extensive applications in image stitching, creating panoramic images, and environmental recognition. Its principle is simple, utilizing matrix operations for fast transformation. With appropriately selected mapping points, registration yields excellent results. The specific transformation method is outlined as follows:

Registration Algorithm

Taking image A as the reference for registering image B.

Selecting n pixels in image A: \[ p_1(x_1,y_1,1),p_2(x_2,y_2,1),...,p_n(x_n,y_n,1) \] Forms a coordinate matrix: \[ P=\begin{bmatrix}x_1 &x_2 &...&x_n\\y_1&y_2&...&y_n\\1&1&...&1\end{bmatrix} \] Selecting n pixels corresponding to these n points in image B: \[ q_1(x_1,y_1),q_2(x_2,y_2),...,q_n(x_n,y_n) \] Forms another coordinate matrix: \[ Q=\begin{bmatrix}x_1 &x_2 &...&x_n\\y_1&y_2&...&y_n\\1&1&...&1\end{bmatrix} \]

If image B can be translated, rotated, and scaled to match image A, then there exists a transformation matrix H such that: \[ Q=HP \] Thus, if the transformation matrix H can be computed, we can use the following formula: \[ H^{-1}Q=P \] Mapping each pixel in image B to the corresponding position in image A, thereby obtaining image B registered with image A as the template.

How to compute the transformation matrix H?

Given the point sets P and Q already selected in images A and B, it's evident that \[ H=QP^{-1} \]

As P is not a square matrix, it cannot be inverted directly. Here, we find the pseudo-inverse.

At this point, we've outlined the basic process of image registration:

• Select corresponding point sets $ P $ and $ Q $ in the template image and the image to be registered.
• Compute the transformation matrix $ H $ using the formula $ H=QP^{-1} $.
• Map the image to be registered onto the template image using the transformation matrix.

Registration Example

For instance: using image A as the template to register image B:

Selecting seven points within them, distributed as follows:

Note: The coordinate system displayed by the drawing tool is opposite to the (x, y) coordinates. Pay attention when recording coordinates.

\[ P=\begin{bmatrix}1448&1694&1756&383&2290&2035&2150\\1308&1198&2744&2516&933&2693&1968\\1&1&1&1&1&1&1\end{bmatrix} \]

\[ Q=\begin{bmatrix}1042&1252&1708&323&1761&1966&1890\\1077&907&2387&2519&498&2265&1535\\1&1&1&1&1&1&1\end{bmatrix} \]

The computed transformation matrix: \[ H=QP^{-1}=\begin{bmatrix}0.9668&0.2565&693.0275\\-0.2570&0.9671&184.1373\\0.0000&0.0000&1.0000\end{bmatrix} \] Transforming image B using the transformation matrix yields the following result:

Source Code:

import cv2 as cv
import numpy as np
import math

imgA=cv.imread("Image A.jpg?x-oss-process=style/webp")
imgB=cv.imread("Image B.jpg?x-oss-process=style/webp")
point_A=np.array([[1448,1694,1756,383,2290,2035,2150],[1308,1198,2744,2516,933,2693,1968],[1,1,1,1,1,1,1]])
point_B=np.array([[1042,1252,1708,323,1761,1966,1890],[1077,907,2387,2519,498,2265,1535],[1,1,1,1,1,1,1]])
H=np.dot(point_B,np.linalg.pinv(point_A))
print(H)
shape_A=imgA.shape
shape_B=imgB.shape
result,mistake=np.zeros(shape_A),np.zeros(shape_A)
result=result.astype(np.uint8)
mistake=mistake.astype(np.uint8)
for i in range(shape_A[0]):
    for j in range(shape_A[1]):
        address=np.dot(H,np.array([i,j,1]))
        address=address.astype(np.int)
        if address[0]>0 and address[0]<shape_B[0] and address[1]>0 and address[1]<shape_B[1]:
            result[i][j]=imgB[address[0]][address[1]]
            mistake[i][j]=imgA[i][j]-result[i][j]
cv.namedWindow("test",cv.WINDOW_NORMAL)
cv.imshow("test",mistake)
cv.waitKey(0)
cv.imwrite("result.jpg?x-oss-process=style/webp",result)
cv.imwrite("mistake.jpg?x-oss-process=style/webp",mistake)

Conclusion

The image registration algorithm is relatively straightforward. With well-selected coordinate points, the computed registration results are commendable. However, manual point selection was employed in this instance. For large-scale or real-time image registration, automated point selection becomes essential. Thus, devising a method for the program to select appropriate and corresponding point sets poses a challenge in image registration.