A black-and-white illustration of a shore, hill, forests, and clouds lets users concentrate on the sounds
A new virtual reality collaboration maps the sounds of Manhattan in 1609 Courtesy of Calling Thunder: The Unsung History of Manhattan

A new virtual reality project reconstructs the city’s historic soundscape.

You’re standing on the High Line. The walkways are damp, and fellow strollers have zipped on windbreakers to ward off the drizzle. Look up, and you’ll find a sky the color of poured mercury. The sound of a revving motorcycle ricochets in the canyons between the buildings. Bend your gaze to the street and you’ll stop for a moment on windows glowing amber before settling on cabs, startlingly yellow against all the gray.

A few seconds later, 400 years melt away. A recent 360-degree video dissolves into a threadbare black-and-white sketch; traffic sounds give way to waves lapping against a shoreline and distant bird calls. The present disappears, and in its place is an aural recreation of a distant past in the same footprint.

The High Line’s slice of Manhattan’s western edge is one of four sites that comprise “Calling Thunder: The Unsung History of Manhattan,” a new virtual reality project that pulls New York City’s contemporary soundtrack into conversation with its predecessors. Bill McQuay, an audio producer at the Cornell Lab of Ornithology, collaborated with David Al-Ibrahim, a graduate student in the interaction design program at the School of Visual Arts, on the immersive audio landscape to capture what the island probably sounded like in 1609, just before the Dutch washed ashore on a coast inhabited by the Lenape people.

It seems logical to invite listeners into an immersive audio world—along with smell, hearing is a sense that we already use in 360 degrees. But making the worlds of Inwood Hill Park, Collect Pond Park, and the modern-day American Museum of Natural History historically accurate required a lot of legwork.

Before the team stitched together the component sounds—drawn from the archives of the Macaulay Library—they had to locate topographic features that would have affected them. The presence of hills or water, for instance, would alter how sound carried. To reconstruct the lay of the land, Al-Ibrahim and McQuay consulted Eric Sanderson, a landscape ecologist at the Wildlife Conservation Society and author of Manahatta: A Natural History of New York City, which compiled block-by-block annotations of the island’s historical ecology. Sanderson inferred species distribution by running contemporary models and consulting historical engravings and maps plotted by European arrivals. (Few archaeological traces of Lenape settlements persist, Sanderson says, and there are very limited extant written records or maps of these features that predate Dutch arrival.)

Hundreds of years ago, present-day Manhattan played host to black bears, wolves, and mountain lions; whales and porpoises schooled thick in coastal waters. When deciding which sounds to place in the landscape, Al-Ibrahim says, the team omitted extinct species such as passenger pigeons out of fear that they wouldn’t be able to replicate their sounds or placement with a high degree of fidelity. They also skipped any species that were infrequent visitors, since the aim was to illustrate “a typical day, not a rare moment,” Sanderson says.

Drawing from Sanderson’s research about species density and distribution, McQuay slotted the sounds where he thought they belonged: frogs by the pond, for instance, and birds vocalizing from trees either closer to water or deeper in the glens of trees. He then adjusted the echoes, frequencies, and reverberations to telegraph distances, and tapped an ornithologist to fact-check his work.

At first, the team imagined that the 15th-century installments would play in darkness, to completely immerse the listener in the acoustic elements of the landscape. But, in focus groups, “people kept thinking it was broken,” explains Al-Ibrahim. Now, the calls of the ring-billed gull, osprey, and common raven are coupled with visuals, but they’re bare bones. The scene consists of white features drawn against a black background: ghostly trees; the outlines of hills, like graceful brushstrokes; stippled waves; smudges of drifting clouds.

The moving clouds help create a sense of dynamism and motion as the viewer rotates the VR glasses without seeing the shape of the creatures whose sounds they’re hearing. It’s a surprise to hear a bird call, seeming to arc right over your head, and see no flash of wing or buoyant shadow. The creators describe the visuals as a “coloring book for sound,” adding perimeters and suggestions without filling in all the blanks.

The project’s larger purpose is more than romanticizing or lamenting bygone centuries. Accompanying narration celebrates local efforts to shore up pollinator populations and incubate native plants—strategies that cities around the globe are pursuing as grassroots efforts and official policy position green infrastructure as a crucial arm of environmental remediation.

The project’s co-creators anticipate that the project’s scope will expand as they earmark more funding and the landscape of VR technology continues to shift. Already, they’ve enlisted technology “that wouldn’t have been possible eight months ago,” Al-Ibrahim says. Next, they’ve set their sights on adapting the material to different user experiences, viewable both with and without VR goggles, and on both mobile and desktop, as well as through an app.

McQuay also hopes the project will help modern New Yorkers appreciate the contemporary urban soundscape, instead of blotting out the roar of the city with ever-more-sophisticated headphones. We’re missing out on something important, he says: “Our environment is as much about what we hear as what we see. We’re always listening.”

About the Author

Most Popular

  1. Transportation

    You Can’t Design Bike-Friendly Cities Without Considering Race and Class

    Bike equity is a powerful tool for reducing inequality. Too often, cycling infrastructure is tailored only to wealthy white cyclists.

  2. Amazon HQ2

    Amazon’s HQ2 Fiasco Will Cost the Company More Than It Costs New York

    The mega-company has bucked dealing reasonably with New York City, Seattle, and any community that asks them to pay for its freight.

  3. Children and adults sit on and around a deck with multi-colored chairs and giant LEGOs.

    If You Build It, They Might Not Come: Animating City Spaces

    Why do revamped areas remain barren after so much thought and money are put into redesigning them? A case study in Charlotte, North Carolina, offers clues.

  4. Charts

    The Evolution of Urban Planning in 10 Diagrams

    A new exhibit from the San Francisco Planning and Urban Research Association showcases the simple visualizations of complex ideas that have changed how we live.

  5. Villa 31, an informal settlement in Buenos Aires

    The Global Housing Crisis

    Scarce, unaffordable housing is not a local problem in a few places, but is baked into the 21st-century global city. It’s time for cities, nations, and global leaders to start acting like it.