5 min read

Week 6: Our City in DALL-E

By Lang Qin, Mashiyat Zaman

Creating Our City in DALL-E

We invited our classmates to take the following three steps:

  1. Choose a style of art or medium that you feel represents New York City best.
  2. Use DALL-E to generate a scene of the city anywhere on the canvas in the style you’ve picked by creating your own prompt.
  3. Personalize the scene by adding something that you enjoy doing using DALL-E’s editing functions.
Prompts and Generated Images
Links to the prompts and images generated by our classmates in the making of Our City. 1. Bingrui https://labs.openai.com/s/7ey0Muu2VDRrEzAyFK6vs7oT https://labs.openai.com/s/StRqCvyJvXh1wzfOY8GiIhNw https://labs.openai.com/s/x6gQ7vPftbsAq3ZbvUog5SKb https://labs.openai.com/s/zuLnxXTbgqNsHeQXF0...
See all of the prompts by our participants through this link!

We had instinctively assumed that the “conflict” between participants’ versions of the city would require “resolution.” But when our second participant’s black and white street photography-inspired vision of the city was already clashing with the impressionist autumnal rendering above, it hardly seemed to cause them discomfort.

As we gathered others, we also observed our three steps shift from directions to suggestions, as participants offered their own ideas of what they wanted to depict and how. The distinction between a “scene of the city” and more personalized rendering became blurred as prompts featured activities, like enjoying a picnic with friends, and locations, like museums. One participant who couldn’t join us in person even provided a prompt of their own with a reference image.

Participants were adventurous about trying to append their creations to their surroundings, and seemed pleasantly surprised whenever they blended well, or understandably discouraged when the results didn’t match what they had written. It wasn’t always easy to predict whether certain styles would blend, so we encouraged participants to keep trying until they were satisfied. In one case, a participant tried multiple times to create an image inspired by the artist Jean-Michel Basquiat, but could only manage once generated in isolation from the rest of the piece.

How could we extend this activity further? As our process became more streamlined with each subsequent participant, it occurred to us that there doesn’t need to be an end – if there were a way to keep the DALL-E editor open to the ITP community, individuals could continue adding to the piece. Perhaps islands of similar styles would emerge, the beginnings of which we can already see. More significantly, these myriad visions of our city would simply coexist on screen.

View or download the final version of our piece here.

The final piece created on DALL-E using prompts by Bingrui, Zey, Lily, Michelle, Queenie, Sophie, Maya, Elyana, and Alfonssette. At least a dozen 1024 by 1024 frames are scattered about a canvas, some of them overlapping with other. See links of individual photos and their prompts through this link.

Working with the Swell Form Machine

We had office hours with Daniel Johnston to discuss how we might make our work more accessible using the swell form machine, which uses heat on special paper to puff up black ink and create a tactile image. Daniel explained that whatever we create with the machine would be just one part of a whole experience, so he encouraged us to think about exactly what we wanted to communicate through touch, and how we could take sections of our work to showcase it.

We decided we wanted to bring attention to the shape of the final product itself, and how participants branched off of existing images on the canvas. We outlined the individual frames generated in the canvas but removed the images themselves, leaving only shades of gray where they intersect. We also wanted to demonstrate how despite the stylistic differences between adjacent frames, these intersections often produced seamless results, and so we chose a cluster of three frames and posterized them to emphasize the gradient that links them together.

A rendering of our final piece showing only the outlines of the generated images, and a shade of gray where they intersect. An even darker shade is used to show where three frames intersect.
A black-and-white posterized rendering of a single section of our piece of just three frames. The left side is light gray, and shows a line of men wearing headphones lined up in an art nouveau-influenced style. The right side, a street photography scene of the city, is almost completely black save for light coming through the windows and a mural in the center.

Our prints with the Swell Form taught us a few things we ought to consider next time. First, it’s important to isolate the information we’d like to communicate, even more than we had. While posterizing the image reduced the number of colors, creating a clear shift between the left and right sides, it’s still hard to get a sense of either scene depicted – for example, you probably can’t distinguish by touch that the figures on the left are people, or that there are buildings on the right. Daniel suggested that we edit the image by isolating a single person with a black outline, for example.

Second, patterns are good alternatives to color. We noticed that the gray sections of our prints hadn’t puffed up at all, so we couldn't communicate as much as we’d hoped. Instead, we could have used stripes or dots. Of course, when using any kind of pattern or color scheme, a clear key is necessary.

Finally, for these examples, we wrote text, which should be in a different color so that it doesn’t react to the machine and potentially crowd the image. We could also print braille.

The Swell Form Machine is a cheaper alternative to other kinds of machines used to make tactile images, but is far from being an affordable option. The machine itself is well over $1,000, and the paper used is between $150–200 per 100 sheets. But Chansey Fleet at the NYPL is working on introducing this and other technologies to more members of the community.