Reuters

Be wary of self-selection bias when measuring engagement with digital platforms, a New York City official warns.

CityLab 2013
Exploring urban solutions to global challenges
See full coverage

Like businesses and organizations everywhere, city governments are getting on board with big data. In particular, the New York City mayor’s office has embraced analytical tools. According to The New York Times, the city's Office of Policy and Strategic Planning processes a terabyte of information (the equivalent of about 143 million pieces of paper) about its citizens every day, and the New York Police Department famously uses Compstat, an analytics software, to track and respond to crime. But cities have to be wary of how they use data, cautioned Chris Corcoran, the deputy in the Mayor’s Office of Analytics, in a panel discussion at The Atlantic’s CityLab summit.

I used to work for an energy efficiency company, and we had people input information about their energy use, and we had a participation of about 30 percent. There was a huge self-selection bias.

If you looked at Foursquare data for New York City, and you looked at day-to-day operations of where Foursquare operates, you would assume everyone lives in Midtown – you would never see anything in the outer boroughs.

As we talk about crowdsourcing information, at least from the cities perspective, we have to be cautious about the huge, silent majority of people who are not participating in all these platforms, and how we still make sure we’re using data to serve their needs.

As cities figure out new ways to use data to their advantage, this warning – from a data scientist, no less! – is worth remembering: Although social media and digital platforms can be powerful tools for connecting with residents, geography and income can easily bias who is able to use those tools.

About the Author

Most Popular

  1. Multicolored maps of Los Angeles, San Francisco, and Tampa, denoting neighborhood fragmentation
    Equity

    Urban Neighborhoods, Once Distinct by Race and Class, Are Blurring

    Yet in cities, affluent white neighborhoods and high-poverty black ones are outliers, resisting the fragmentation shown with other types of neighborhoods.

  2. A photo of a new subdivision under construction in South Jordan, Utah.
    Perspective

    A Red-State Take on a YIMBY Housing Bill

    Utah’s SB 34, aimed at increasing the state’s supply of affordable housing, may hold lessons for booming cities of the Mountain West, and beyond.

  3. Design

    A History of the American Public Library

    A visual exploration of how a critical piece of social infrastructure came to be.

  4. Design

    There’s a Tile Theft Epidemic in Lisbon

    With a single azulejo fetching hundreds of euros at the city’s more reputable antique stores, these tiles, sitting there out in the open, are easy pickings.

  5. Design

    The Curious Politics of a Montreal Mega-Mall

    The car-dependent suburb it’ll be built in wants to greenlight Royalmount against the city government’s wishes but it needs them to pay for the public infrastructure.