Direct Bundle Adjustment of Video

Open Access
Morgan, John Paul
Graduate Program:
Electrical Engineering
Master of Science
Document Type:
Master Thesis
Date of Defense:
March 18, 2016
Committee Members:
  • Richard Laurence Tutwiler, Thesis Advisor
  • William Evan Higgins, Thesis Advisor
  • Bundle adjustment
  • monocular simultaneous localization and mapping
  • structure from motion
  • direct image alignment
Given a set of 3D points seen by multiple cameras, bundle adjustment is the process of jointly optimizing the location of the 3D points and the pose of each camera. Often the image coordinates of the points are known, usually via feature detection and matching, and the reprojection error can be minimized. Here the focus is instead on intensity-based methods that directly minimize the intensity residual between viewpoints. These methods require a good initialization, which is implicitly provided by video because the transformation between frames is small allowing each frame to initialize the next frame. The goal of this thesis is to analyze the feasibility of intensity-based bundle adjustment for dense reconstructions and compare this to recent work on multi view stereo. The proposed method optimizes the inverse depth of a reference frame rather than explicit 3D points, which leads to a significantly more efficient algorithm. Dense reconstructions are possible because the sparsity of the bundle adjustment problem can be exploited to achieve linear runtime with respect to the number of points. The greatest advantage over existing algorithms is that special initialization cases are not required. Each point is tracked through the video, and therefore the inverse depth and camera pose are always a feasible explanation for what was observed. It will be shown that very few frames are required to converge to an accurate estimate. Dense reconstructions do not run in real-time; however, the method is equally applicable with sparse sets of points for real-time pose estimation.