Social-media platforms like FourSquare and Twitter have been a boon for sociologists and geographers who now have entirely new ways of tracking how we move through cities, where we go, who we are, and even what we think of the world around us. There is one set of social-media platforms, however, that has been tougher to crack for useful data than others: photo-sharing sites.
Their metadata can illustrate where people take photos, and how active they are. In at least one case, researchers have also been able to break down the architectural details contained in photos of urban streetscapes. But on the whole, how do you aggregate useful data about entire cities and the differences between them from the content of millions of photos on a site like Instagram?
Researchers Nadav Hochman and Lev Manovich have been working on this for the past year, and they've just posted some of the initial results from their Phototrails project here. They also describe their process of data-mining Instagram in a paper set to be published this month in the online journal First Monday. The project is less an exploration of a specific research question, and more a first foray into what we might learn by treating user-generated photography as another source of Big Data.
Collectively, Instagram captures some zoomed-in patterns in how people experience a given city. But these images, in their elemental forms, also reveal some curious comparisons between places: "It’s almost like each city has its kind of visual character," says Manovich, a professor at The Graduate Center at the City University of New York.
The above image shows 53,498 photos from Tokyo, taken between February 18-25, visualized according to upload date and time. Collectively, the photos reflect the rhythm of the city, between night and day, between light photos and ones that are darker on average. Here is a zoomed-in glimpse of the same collage:
And here is a subtly different illustration of 57,983 pictures from New York, taken during the same week, revealing a different rhythm in Instagram use:
Some of the most interesting parts of this project, however, further abstract all of these individual photos into measures of their median or mean brightness, hue, or saturation. Using their own open-source software, Manovich and Hochman, a PhD student in the history of art and architecture at the University of Pittsburgh, analyzed 2.3 million photos form 13 cities across the world, all taken between late 2011 and early 2012.
In this visualization of 50,000 randomly sampled images from Tokyo, the distance of a given photo form the center of the diagram illustrates the mean brightness of all the pixels in the picture. And the photos are organized around the center by mean hue.
Here you can compare San Francisco on the left with Tokyo on the right, using the same metrics:
Here is a slightly different glimpse of Bangkok, organized by brightness mean (radius) and hue median (perimeter):
In many ways these cities are fairly similar (where's all the green and blue?). But small differences emerge as well. Photos in Bangkok, for instance, tend to be slightly lighter than those in New York.
"There’s not one right way to interpret it, that’s what interesting about visualizations – you may be seeing something different from what I’m seeing," Manovich says. "I’m still thinking how to interpret them. It’s not like a bar graph."
The significance of this visualization may be a little easier to grasp. It's composed of 23,581 photos uploaded from Brooklyn in the 24 hours between the early morning on Oct. 29 of 2012, and the following day. In this case, the photo's distance from the center represents its mean hue, and its position along the perimeter its time stamp.
This was the day Superstorm Sandy reached New York. And that stark like at 10:23 p.m. shows when power went out for many people. Up until that point, the approaching storm was a densely photographed event. After the power went out, many people understandably put down their phones. Looking at that visualization, Manovich says, "You almost get this feeling of this emotional impact of the event."
Photography may be able to capture the "mood" of a city like this in ways that FourSquare check-ins or Twitter locations cannot. But this is also just a first stab at what photography might tell us. Manovich and Hochman, working with collaborator Jay Chow, chose some of the simplest attributes – hue, brightness, etc. – to make their techniques more accessible to the public.
"Whenever possible, we say 'why don’t we start with simple things and see what we can get,'" Manovich says. "And then we’ll move to more complex things."
Beyond brightness and hue, these academics or others could dig deeper into the content of such pictures. Are there, for instance, some neighborhoods in a city that produce more outdoor pictures with people in them? Or how about some cities that seem to have more nature in them than others?