Skip to content

Add documentation for Two View Estimator #829

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: refactor/documentation
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 70 additions & 1 deletion assets/TWO_VIEW_ESTIMATOR.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,73 @@
- **Two View Estimator**
- [Multiview Optimizer](assets/MULTIVIEW_OPTIMIZER.md)

## What is a Two-View Estimator?
## What is a Two-View Estimator?


Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what information? (feature correspondences)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove: how one camera is positioned and oriented with respect to the other one

use 3D relative orientation and 3D unit translation (or translation direction) to avoid ambiguity.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The correspondences are generated in the correspondence_generator, which is outside the two-view estimator. the config mirrors this structure: https://github.com/borglab/gtsfm/blob/master/gtsfm/configs/unified.yaml

Two-View Estimator (TVE) takes information about two images and tries to determine their relative pose (how one camera is positioned and oriented with respect to the other one). It also generates correspondences between keypoint in the images.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor nitpick: this can be an unordered list, remove "3 things" as well. I think the exact number depends on how you break it down, and is not important.

As seen in the diagram above, TVE sits between [Correspondence Generator](assets/CORRESPONDENCE_GENERATOR.md) and [Multiview Optimizer](assets/MULTIVIEW_OPTIMIZER.md). TVE does three things in GTSfM:

1) It filters incorrect matches received from correspondence generator by removing outliers.

2) It provides initial "seed" relative pose estimates that are passed on to Multi View Optimizer (MVO) to start the global optimization process. A good initial estimate is important for MVO to be able to converge to a good solution.

3) It also provides data to be used by view graph estimator to improve robustness in a global coordinate frame.


## Relevant Files:

- [gtsfm/two_view_estimator.py](https://github.com/borglab/gtsfm/blob/master/gtsfm/two_view_estimator.py): Core implementation of the `TwoViewEstimator` class, contains logic for two view estimation, bundle adjustment and evaluation.
```python
class TwoViewEstimator:

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please indent the same way we indent code, helps readability.

"""Wrapper for running two-view relative pose estimation on image pairs in the dataset."""

def __init__(
self,
verifier: VerifierBase,
inlier_support_processor: InlierSupportProcessor,
bundle_adjust_2view: bool,
eval_threshold_px: float,
triangulation_options: TriangulationOptions,
bundle_adjust_2view_maxiters: int = 100,
ba_reproj_error_thresholds: List[Optional[float]] = [0.5],
allow_indeterminate_linear_system: bool = False,
) -> None:

"""Initializes the two-view estimator from verifier.



Args:

verifier: Verifier to use.

inlier_support_processor: Post-processor that uses information about RANSAC support to filter out pairs.

bundle_adjust_2view: Boolean flag indicating if bundle adjustment is to be run on the 2-view data.

eval_threshold_px: Distance threshold for marking a correspondence pair as inlier during evaluation

(not during estimation).

bundle_adjust_2view_maxiters (optional): Max number of iterations for 2-view BA. Defaults to 100.

ba_reproj_error_thresholds (optional): Reprojection thresholds used to filter features after each stage of

2-view BA. The length of this list decides the number of BA stages. Defaults to [0.5] (single stage).

allow_indeterminate_linear_system: Reject a two-view measurement if an indeterminate linear system is

encountered during marginal covariance computation after 2-view bundle adjustment.

"""
```

- [gtsfm/two_view_estimator_cacher.py](https://github.com/borglab/gtsfm/blob/master/gtsfm/two_view_estimator_cacher.py): Provides a caching implementation to speed up two view estimation process when the same image pairs is processed a second time.

- [gtsfm/common/two_view_estimation_report.py](https://github.com/borglab/gtsfm/blob/master/gtsfm/common/two_view_estimation_report.py): Defines the `TwoViewEstimationReport` dataclass used to store information and metrics about the two-view estimation result.

- [gtsfm/configs/* .yaml](https://github.com/borglab/gtsfm/blob/master/gtsfm/configs/): Configuration files that specify the parameters and components to be used for the two-view estimator.

- [gtsfm/frontend/verifier/](https://github.com/borglab/gtsfm/blob/master/gtsfm/frontend/verifier/): Verifiers (RANSAC, OpenCV, etc) used in `TwoViewEstimator` to improve the robustness of estimation.
Loading