Fisheye Lenses for Photogrammetry

Most of the computer vision literature tends to concentrate on the use of linear pinhole projection models and simple lens distortion terms. This is advantageous in terms of simplicity and computational speed, but by using a more general projection model expressed as an image coordinate to ray function, it becomes surprisingly easy to make use of more exotic lens and camera types such as an ultra wide angle fisheye.

Why would you want to do this? Using photogrammetry to scan enclosed spaces can be rather tedious due the need to maintain good overlap between images while achieving sufficient parallax and covering a full 360 degree field of view. The wider the lens, the easier this gets, and a fisheye lens is as wide as you can get with a normal camera.

My choice of lens in this case is the Samyang 7.5mm fisheye for micro four thirds cameras. It covers the full sensor area (rather than being a circle-in-a-square type which wastes available resolution), is very cheap, and is very sharp in all but the extreme corners of the frame.

To express the projection of the Samyang fisheye, I used an equisolid angle model given by: 

Where r, f and theta are the radius from the centre of the sensor, focal length, and angle between a ray and the optical axis respectively.
It seemed unlikely that the lens would fit this perfectly so I added polynomial radial distortion and decentering terms to that.

Armed with my fisheye lens on a Panasonic G3 and a suitable mathematical model for the lens, I went down to the beach to capture some images for calibration.

Perhaps not the most beautiful photos ever taken of a New Zealand beach, but they fill the frame with some nice high texture sand for accurate feature matching.

Performing a bundle adjust on the calibration sequence yielded extremely low values for the decentering and extra distortion terms. The Samyang fisheye turns out to be very close to an ideal equisolid angle projection, closer than most standard wide angle lenses are to being rectilinear.

The resulting reconstruction has good detail, but includes rather a lot of geometry from the surroundings in this non-enclosed setting.

Statues in Berlin

To get things started, this is a post about a piece of photogrammetry / multi-view stereo reconstruction software that I have been gradually writing during spare time over the last few years. Think of something like Agisoft Photoscan, but without the user-friendliness. At the time, this was somewhat more trendy than it is now and I spent my weekends traveling around Berlin finding anything worth scanning.

Berlin is a fascinating place for 3D scanning, thanks to a mixture of older and more modern architecture, along with plenty of statues scattered around the many parks. In this case my target was a nicely detailed statue of a boy and horse located near the Schloss Bellevue in Tiergarten. It was a fairly easy choice thanks to the weathered surface, but there is just enough of a specular response to cause some concern, and the possibility of bad feature tracking on the leaves behind moving in the wind.







For the capture I used a 16 Megapixel Panasonic G3 micro four thirds camera with a 25mm lens. The smaller sensor can cause some noise issues in shadowed or poorly textured regions, but in this case most of the scene was fairly well illuminated.

I generally use Meshlab http://meshlab.sourceforge.net/ for viewing results and applying a bit of cleanup. It's free and easy to use, but a little slow with large meshes.