4.12.09

but which is procedural?

Comparison of a procedural city and a photo of a city: which is which? How long did you have to look?

The procedural image (of a reconstructed Pompeii) was from Procedural Inc (right), the photo was by mharrsch (left). Highlight this paragraph in your browser for the answer! Filter was a quickie "photocopy" in gimp.

The lack of real scientific method in computer science is quite apparent in fast moving and new fields, like procedural modelling. Evaluating procedural content has been an ignored subject in the journals, there is very little said about the quality or the believability of the output of a procedural system. In contrast lots is said about the number of processor cycles or the lack of artist involvement. This a sketch of a project that hopes to address this issue. We can the above idea turn this into a web-game (similar to the google image labeller game) that challenges people to spot the fake photograph in a certain time, and rewards them with points for being right.

Going even further we can build a competitive game that find the realistic and unrealistic areas of an image by asking you to attempting to fool your opposition. The computer invents a goal ("prove to your opposition that this is a real photo"), and you get points for doing it. The more of the image you reveal, the less points you get. When it's your turn, you see an image slowly revealed as your opponent reveals the bits he can see. You have to decide if it's a photo or a render (in this case). It could create a "believability" map of each image by examining the areas scratched off by the subjects. The following example just asks the user to distinguish between a render and a photograph but by using a filter that obscures the geometry (such as the photocopies in the above example) it would be possible to compare procedural & artist-defined geometry.

Images for this were stolen from this guy. Video (or on youtube):

You can play a simulated ("offline") version of this game by clicking here (Java 1.6 needed, binary file - run at your own risk). Your own moves are played back when it's your turn to guess. There is an option to add image noise to make the game harder to play.

This is one of the most promising avenues of exploration I've thought of when it comes to evaluating procedural modelling systems. Of course it is very hard to count the contributions from the artist and from the procedural mechanisms. To solve this some sort of test of artist generated geometry against procedurally generated content may be called for. In both cases we would record the quantity of programmer/artist time taken to create a scene, and then evaluate it. The hope would be that the procedural models show comparable realism, while absorbing many fewer man-hours.

Future ideas:
• Could do this for videos
• Could render examples from a procedural model in real time, get feedback on them, and update the model automatically. This kind of technology ("which parts of these models represent the door knobs that the guy scratched out?") is a few years away yet...
• Add in distractors (trees, people, lens flare, motion blur) to allow a fair test against a random selection of architecture photos (from flickr, say).
Let me know if you think this is an interesting project, and have some ideas for carrying the idea further. It should make a half decent paper when written up.

One of the biggest problems with this is that there aren't many examples of detailed procedural models. The Pompeii examples from Procedural is a couple of years old and is still one of the best/most complex examples. As the above image shows, it is only just detailed enough to make this test plausible. I had to hunt around on the web for a while to find a photo of Pompeii without too many people/clues etc... to make the above a reasonable comparison.

It's getting harder and harder to tell rendered images from photos, if you look at the top right of this image of houses in Glasgow (park circus), it's hard to tell it's just a photograph. The houses were being refurbished, so no people to leave distinguising features, and they had just been cleaned using sand-blasting.