#Tiny 3D worlds from street photos

While having a walk on a Saturday morning, I wanted to test how easy it is to generate a playful 3D model of a street corner from a couple of pictures taken on the fly with my phone.

The corner is in Paris 7th, where rue de Babylone meets rue Monsieur. On one side, La Pagode, a neighbourhood landmark from 1896 that became a cinema and has been under renovation for years. On the other, a Haussmann building whose top floors are wrapped in scaffolding and white tarpaulin.

Here are the three photos I used, all taken from the pavement.

##1 picture + 1 prompt + Opus 5.5

The Opus 5.5 version has the pagoda with two tiers of curved roofs, the building under its tarpaulin, the banner on the construction fence and the red barriers. It also added typical Parisian things, that are however not in the photo: a fountain, a "Morris" poster column, a bench, pigeons, a car driving past.

##1 picture + 1 prompt + Fable 5.2

Same photo, same prompt. Fable 5.2 kept closer to what the photo shows: the curved-gable porch of the pagoda, the scalloped cream wall, the timeline on the banner, the two people reading it. At night, the tarpaulin glows nicely like a paper lantern.

##3 pictures + 2 prompts + Opus 5.5 + Fable 5.2

This one started with the three photos and a first prompt to Opus 5.5:

Then a follow-up prompt handed the result to Fable 5.2:

Fable reworked the light (lower autumn sun, reflections on the zinc and the gold), gave the stone a texture, added bow windows and wrought-iron railings to the building, and rebuilt the roofs and the spire of the pagoda. Some details are invented, like the red and white striped awnings, which are not in the pictures.

The buttons at the bottom switch between afternoon and evening, and show or hide the scaffolding, which I didn't ask for.

##What I learned

All three renders are delightful. In one shot, with a single picture, Fable did a more precise job than Opus. The level of detail is better, as can be seen in the roofs of the pagoda and in the façade of the Haussmann building, especially its wrought-iron balconies.

The third version is better and more detailed. Three things could explain it:

• two extra pictures
• a follow-up prompt, handled by Fable
• an explicit request to bring the 3D rendering closer to the pictures

Still, architectural accuracy is far from perfect. For example, some doors are not in the right place, and the scaffolding doesn't really match the real one. Also, some details have been hallucinated. This could certainly be improved with more follow-up prompting, and additional pictures that give a better overall view of the scene.

Beyond the quick fun experiment, I see a small use case here, for real estate agents perhaps. From a few basic pictures, they could very easily get an approximate render of a neighbourhood, in a style people could find really appealing.

The three versions side by side: lassence.net/projects/pagode-3d