FlowStacker – A simple tool for stacking photos of moving people.

Photo stacking is used a lot to generate HDR images and increase depth of field in macro photography, but it is also a useful tool for reducing image noise, even without exposure bracketing. Stacking gives better results than traditional noise reduction because it can enhance detail, instead of smearing the image. Being a fan of small and light cameras, I find stacking to be a very useful tool for reducing grain and increasing colour depth when shooting in low light conditions.

Usually, trying to photo stack images of people results in 'ghosting' problems because they just can't keep still long enough so the images don't line up with each other in a simple way. To make things a bit easier, I've put together a simple tool to handle the deformations required to stack photos of people and other moving subjects.  It will handle a moderate amount of movement, but nothing excessive. It's also quite useful for hand-held macro photography without a flash because it can compensate for the parallax effect caused by small movements of the camera.

However, if you are hoping to use it for bracketed HDR, or macro focus stacking, it won't do that.

You will need a 64 bit version of Windows to run it, and you must also install OpenCL (which should come with recent graphics card drivers).

Download FlowStacker from here: FlowStacker

The general idea is to set your camera on burst mode, and high enough ISO to avoid motion blur and shoot a set of around 4 to 16 frames as quickly as possible, then use FlowStacker to combine them to reduce noise, and increase colour depth. It can handle small movements of people while you shoot the burst, and quite large distortions from electronic or rolling shutter. You will get best results from shooting raw, and converting to 16 bit png or tiff without using any noise reduction or compression. The option for extra smoothing can be enabled for high ISO images where the only changes comes from camera movement, such as star photography or static landscapes.

When you open FlowStacker, you will see the main window:

 

Press 'load images' to select the photos you want in the stack.  You can select which of the image is the master that all others will be aligned to.  Press delete on a selected image to remove it.  Set the output image for your stacked result, and hit 'Start Processing'.  Expect processing to take a few minutes, depending on how fast your processor and graphics card are.

Here is an example from a dimly lit restaurant. Even using a wide f/1.4 aperture, I was shooting at ISO 3200 and 1/40th second exposure to avoid camera shake. I was able to fire off 8 shots in burst mode before my subject's reluctant smile had faded.

Each individual raw converted image has clearly visible noise:

After combining the 8 shots in FlowStacker, the result has a much higher quality:

The next one was an attempt to capture a tiny spider hiding in the far reaches of my greenhouse. Getting the shot required holding the camera at arm's length while attempting to keep manual focus while the web was moving gently in the air current. Yes, a flash would be easier, but sometimes inspiration strikes when we are least prepared.

One of 16 original images taken at ISO 6400 and heavily cropped:

After stacking, the image noise is down to a more acceptable level:

Fisheye Lenses for Photogrammetry

Most of the computer vision literature tends to concentrate on the use of linear pinhole projection models and simple lens distortion terms. This is advantageous in terms of simplicity and computational speed, but by using a more general projection model expressed as an image coordinate to ray function, it becomes surprisingly easy to make use of more exotic lens and camera types such as an ultra wide angle fisheye.

Why would you want to do this? Using photogrammetry to scan enclosed spaces can be rather tedious due the need to maintain good overlap between images while achieving sufficient parallax and covering a full 360 degree field of view. The wider the lens, the easier this gets, and a fisheye lens is as wide as you can get with a normal camera.

My choice of lens in this case is the Samyang 7.5mm fisheye for micro four thirds cameras. It covers the full sensor area (rather than being a circle-in-a-square type which wastes available resolution), is very cheap, and is very sharp in all but the extreme corners of the frame.

To express the projection of the Samyang fisheye, I used an equisolid angle model given by: 

Where r, f and theta are the radius from the centre of the sensor, focal length, and angle between a ray and the optical axis respectively.
It seemed unlikely that the lens would fit this perfectly so I added polynomial radial distortion and decentering terms to that.

Armed with my fisheye lens on a Panasonic G3 and a suitable mathematical model for the lens, I went down to the beach to capture some images for calibration.

Perhaps not the most beautiful photos ever taken of a New Zealand beach, but they fill the frame with some nice high texture sand for accurate feature matching.

Performing a bundle adjust on the calibration sequence yielded extremely low values for the decentering and extra distortion terms. The Samyang fisheye turns out to be very close to an ideal equisolid angle projection, closer than most standard wide angle lenses are to being rectilinear.

The resulting reconstruction has good detail, but includes rather a lot of geometry from the surroundings in this non-enclosed setting.

Statues in Berlin

To get things started, this is a post about a piece of photogrammetry / multi-view stereo reconstruction software that I have been gradually writing during spare time over the last few years. Think of something like Agisoft Photoscan, but without the user-friendliness. At the time, this was somewhat more trendy than it is now and I spent my weekends traveling around Berlin finding anything worth scanning.

Berlin is a fascinating place for 3D scanning, thanks to a mixture of older and more modern architecture, along with plenty of statues scattered around the many parks. In this case my target was a nicely detailed statue of a boy and horse located near the Schloss Bellevue in Tiergarten. It was a fairly easy choice thanks to the weathered surface, but there is just enough of a specular response to cause some concern, and the possibility of bad feature tracking on the leaves behind moving in the wind.







For the capture I used a 16 Megapixel Panasonic G3 micro four thirds camera with a 25mm lens. The smaller sensor can cause some noise issues in shadowed or poorly textured regions, but in this case most of the scene was fairly well illuminated.

I generally use Meshlab http://meshlab.sourceforge.net/ for viewing results and applying a bit of cleanup. It's free and easy to use, but a little slow with large meshes.