Implementation of stereo-reconstruction algorithm. Reprojects points in relative 3D space using triangulation and stereo-matching from a pair of calibrated images.
helper.py: subset of helper functions courtesy 16-720 course staff. Some useful functionalities : refines F matrix using Modified Powell's minimization, calculating all possible M2s.
Given: Stereo-rectified pair of images, given camera intrinsics (K1, K2), point correspondences between left and right images, also given a set of 'identified features' (x1, y1) coordinate pairs corresponding to image1. A high-level overview of the algorithm is given below -
-
Estimate Fundamental Matrix (F) using given point correspondences.
-
Construct Essential Matrix (E) using F, K1, and K2.
-
Recover projection matrices M1 and M2 from E.
-
Implement stereo-matching to find features (x2, y2) in image2 that correspond to the same features in image1 by searching along the epi-line.
-
Use linear triangulation with a given 'M' to formulate a linear system and find optimal solution using SVD.
-
Implement DLT triangulation using (x1, y1) and (x2, y2) to return the set of points which has the least reprojective errors.
-
Constructing final camera matrices (which are used in triangulation) C1, C2 using intrinsics (K1, K2) and projective camera matrices (M1, M2).
- M1 and M2 are [I | 0] and [R | t] respectively. (obtained from E)
- C1 is fixed, since it is obtained using just M1 and K1
- Find all possible M2
- Find best M2 that gives the least triangulation error. (by reprojecting and taking the euclidean norm)
-
Triangulate points using best C2 and plot.
| Image1 | Image2 |
|---|---|
 |
 |
 |
 |
 |
 |