Stereo matching is used for finding corresponding pixels in a pair of images, which allows 3D reconstruction by triangulation, using the known intrinsic and extrinsic orientation of the cameras. Unfortunately, the problem is ill-posed, since images are locally very ambiguous. Many techniques have been proposed in photogrammetry and, later on, in computer vision. However, all of them are far away from being optimal. The Semi-Global Matching (SGM) technique offers a good trade-off between accuracy and runtime and is therefore well suited for many practical applications.
SGM Advantages
Semi-global matching allows high-density point clouds of the first reflective surface to be rapidly and automatically extracted from stereo-imagery. In practice, it is not able to produce a point cloud for every pixel in the source imagery but only about every third pixel, which is still amazing. SGM is now commonly implemented in a variety of desktop mapping applications such as SimActive, Hexagon, Pix4D, and many others, including open-source versions like OpenCV.
One advantage of SGM is that these point clouds look and feel much like lidar point clouds. SGM models can be colorized from the available imagery just like lidar and are generally available in LAS format. They are typically processed much like lidar data.
SGM Limitations
Semi-global matching suffers from two very important practical limitations. First, it can assign an elevation to only pixels in the image (that is, only those things photographed). Aerial photography is taken from above, so only the tops and sides of things are visible. The ground under a vegetative canopy or dark shadow is obscured. So our SGM DSM will contain only points on the surface of things, not the ground. This point cloud cannot depict the Earth’s surface unless the bare earth is photographed, like at a gravel pit.
The second limitation is that a coordinate cannot be assigned to a pixel unless that same pixel is “found” in a second or third overlapping image. Imagery that contains scenes with very uniform texture (think fields of grass or corn, dense forest canopies, blacktop parking lots, dark shadows, etc.) cannot be modeled. SGM finds it nearly impossible to reliably find a second pixel that “matches” the first one in such “texture-less” areas. In these areas, the SGM point cloud will contain holes, a void of points.
There is considerable work being done to improve SGM and better model these scenes, but it may never be possible until the resolution (GSD) of these areas improves. At much higher resolutions the canopy or parking lot that looks uniform green or black will have much more “texture,” and SGM will be able to better model those surfaces.
Unmanned aerial vehicles (UAV) are an important consideration here. Using a UAV we can fly much lower and produce better GSD. At these resolutions, the “texture” of many surfaces is much better, and SGM modeling improves. This is one reason Pix4d and other software packages developed for UAV mapping include SGM-produced point clouds as an output.
Point cloud with pronounced voids on monotonous surfaces, especially on the water surface
Original article: Obtaining economical and accurate point clouds from semi-global matching
