Saturday, February 12, 2011

Stitching Tips

Not usual that I'll write tips or tutorials on here, in fact I can't remember doing one since I wrote about creating 3D photo-textured models in Google Earth in early 2009. Not many advice found here...unless it's my biased, synesthetic-based suggestions on songs or videos in my reviews.

The Background

Over the past year, I've dived into trying (with various levels of success) to create full equirectangular panoramas, to put on 360 Cities - and also to create stereographic projections of them, or, as every photographer or viewer likes to call them, little floating 'planets.'

The first 'planet' I ever tried to do was in autumn 2008, with this:
There's a full blog post on my creation of this.

It's not a proper 'planet,' though. For one thing, it's a normal 360 panorama taken with a point and shoot camera, that begins and ends. There's no center ground coverage, no sky coverage, and it's not a true equirectangular image.

Anyway, throughout 2010 I used the free panorama stitching program 'Hugin.' It worked alright and I learnt how to use it properly over time. To see how my planets evolved from errors to better refinement, check out my Flickr set of Planets. I desired the program PTGui though, largely because it has a cost, and my belief was that if you have to pay for it, it's a better program that's superior to Hugin.
That was my first false assumption.

Have you seen the beautiful results produced by people using PTGui though? It's perfect - I have a very hard time finding stitching errors. Unfortunately, when I finally bought the program a few weeks ago, I was so shocked over how crappy my panoramas turned out I barely used it anymore when I kept getting the same results, no matter how much I tinkered or tried to fix it.

The Tips (or The Secret, whichever you prefer)

A few days ago I realized that Hugin had since been upgraded and further advanced since the version I'd downloaded in early 2009 (which was version 0.08). It had gone up to version 4.0. Jesus.

But here is the plain truth: No panorama stitching program, paid or free, is going to give you your perfect result automatically, without your intervention.

My full secret, though, is this: Don't let Hugin, or PTGui, or whatever, do a thing until you are ready to stitch the final result.

A panorama is created, when using these programs, by implementing control points onto the images. There is always a set of control points  between two images. Each are placed on the exact same spot in the corresponding images. These are the points the program refers to when stitching it together - matching the points over each other on each image, therefore matching each corresponding point on each image, and warping it accordingly, so that it is properly stitched and aligned.

When you load your images into either Hugin or PTGui, you can press a button - 'Align' - and it will auto-generate control points on each loaded image. That is, each loaded image that in some way matches the other.

When you let the program do this, you have very varying levels of success in the result. But it is almost never properly aligned, or accurate, or stitched properly, to the point it's a perfectly stitched panorama.

The problem is this: When the program auto-generates its own control points, what it actually does is try its best in finding a point in an image that is exactly like the point in the several corresponding images around it. A pixel that is the exact same color and surrounded by similar pixels.
The result is often either several control points on top or within a pixel of each other, clusters where points are further from each other but still clustered around an area of image, or, worse, control points that are completely miss-aligned altogether. Usually it's all three scenarios - clusters, miss-alignments, and double-deckers (points directly on top or within a pixel of each other).

What's more, there will be still more images that the program cannot find a single similar point to add points to, at all, so you still have to manually add them. This is usually the case with stitching the floor, the ceiling, and any random shot even, especially if it's significantly brighter or darker than the rest.

Auto-generating control points may take half a minute to a minute for the program to do (PTGui is slightly faster at this than Hugin) but the time you take to tinker with and attempt to fix and replace and re-align the panorama can take hours, and even separate days if you give up in frustration and resolve to try it again later.

That was my case just about all the time, basically. Especially with PTGui.

You can get pretty lucky and have almost no trouble at all in auto-generating a stitched panorama. It's possible. There will still be (albeit, small) errors, but you can do it as long as you take the panorama in an empty, expansive space. Examples like a field, or empty parking lot, or anywhere there isn't a lot of, well, things or objects to stitch. To stitch a horizon in a field is nothing. When I stitched Moloughney Park here in Barrhaven, all I had to manually stitch was the sky. Otherwise, after hitting 'align,' I just did the sky, then hit 'Stitch Now!' and saved it to the folder it's in. Done.
I'd forgotten to stitch the ground in that one.

The first time I got a good result from manually stitching a panorama was my backyard. I'd been extremely surprised at how it turned out, if I'd done it all myself. The reason for the manual stitching was because the program on my computer couldn't for some reason stitch it automatically. Why hadn't I realized then that manually stitching it would easily give me better results? That was last April.

Doing it manually, you know the identical points within two images and where they are, so you can accurately place each corresponding point in the proper place. Furthermore - and this is where the program fails - you can place them all throughout the image instead of clustering them. For a better result, I'd recommend that you place them as far apart from each other as possible, particularly in places where the images would really need to be warped. Like in corners, or at the top, or edges, and the center as well. You don't have to place them virtually (and literally) right on top of each other.

The best part in all of this is this: You only need 4-8 control points for each image. I put 8. Eight well-placed, accurate, distanced control points essentially guarantee you the closest perfectly-stitched image possible.

The time it takes you to do this manually is ridiculously shorter than letting the program mess it up and fixing it over a long, frustrating time. For 26-30 images, it usually takes me about an hour to an hour and a half. Then I'm, really, done.

When you stitch the images, you need to do more than just stitch the image with the one next to it. You do one full rotation of stitching images next to each other. For me, I use a 10mm wide-angle lens. I take around 10-12 photos, vertical, looking largely downwards so that the horizon is in the upper half of the image. Then I take about 12 more looking essentially upwards closer to the sky, with with horizon in the lower half of the image (never removed). Then I take a shot of the ground and one of the sky (or floor and ceiling). That's about 25-26 images, more if you are cautious and your images have a lot of overlap. You do the bottom half of the panorama - the images pointed more towards the ground - first. One rotation. Then you do the upper half of the panorama - images pointed more towards the sky but not directly up - second. Another rotation. The next thing you do is stitch the upper and lower images together to put the halves together. A third and final rotation.
The last thing you do is stitch the ground and sky, which is a little harder but still perfectly possible. My recommendation is two to three control points for each lower-half image and the ground, placed as far as possible away from each other (preferably in each bottom corner of the image). It's the same for the sky.


Doing this all manually is a bit of work at first, because you're starting from scratch. But it pays off hugely. You really don't have to do much at all. After doing three full rotations of control points, lower, upper, and connecting the two, plus the ground and sky, all you have to do is press 'Align' - that's when you press it - and the program will stitch together a preview of the panorama that will look almost perfect instead of generating its own points. The last thing you do is fine-tweak it (there can still be some easily-noticeable miss-alignments) and then save the result and press 'Stitch Now!'

Program-wise, I would suggest Hugin since it's free and just as capable and PTGui, and as far as user-friendly goes between the programs, Hugin is better. When you place the points, it will zoom into the place in the image you clicked on for a perfect placement (while PTGui doesn't).

These are my tips and my secret if you want to think of it that way, and I hope they are helpful.

Justin C.


Anonymous said...

They are absolutely amazing! Great job! I can imagin the amount of work that goes into them.

Justin S. Campbell said...

Thanks. What I wrote came out of a full year's working with trial and error, arriving at my final instruction with a lot of effort.
By the way, PTGui doesn't let you manually add control points until after you let it align it for you first. Utterly useless...but that's my opinion. Anyway, thanks for your feedback, now I know my words are effective to some extent.