In the 1980s and early 1990s, New York City was a dangerous place to live. The city’s homicide rate peaked at more than 2,200 in 1990; meanwhile the risk of robbery and muggings was a daily reality for the city’s roughly 7 million residents. Time magazine’s September 17, 1990 cover story proclaimed “the rotting of the Big Apple,” citing a “surge of brutal killings” that had left New Yorkers feeling unsafe and uncertain whether to remain in the city. [1] The New York Times declared in December that the city’s streets resembled “a New Calcutta, bristling with beggars and sad schizophrenics tuned to inner voices.” ^[2]

In 1994, William J. Bratton took over as commissioner of the New York City Police Department (NYPD) with an ambitious goal: bring the crime rate down, fast. Key to his efforts was a management tool known as Compstat (“computerized comparison crime statistics”), which allowed Bratton to reorient the department toward proactive crime prevention by analyzing crime data and directing more resources to higher-crime areas. Using up-to-date data to guide decision-making was a radical departure for the department, and it paid off. By the end of the decade, New York City’s crime rate had dropped by half. ^[3]

The Police Department’s success using data analysis to improve service delivery drew the notice of other city departments—in particular, the New York City Fire Department (FDNY). After a 2007 fire that killed two firefighters, due in part to inadequate safety inspections, FDNY’s leadership decided it needed a better way to prioritize building inspections. In 2013, the department instituted a computerized inspection system based on sophisticated and up-to-date measures of a building’s fire risk and, like the Police Department, began directing scarce resources to the highest-risk areas.

Meanwhile, Michael Bloomberg took office as New York City’s mayor in 2002. A financial data services mogul, he championed efficiency through data-driven decisionmaking. Data became a City Hall watchword. The mayor oversaw the implementation in 2003 of a 311 non-emergency hotline to streamline the delivery of city services through a single point of contact. In 2013, he created a new Mayor’s Office of Data Analytics (MODA)—a team of number-crunchers tasked with uncovering correlations and locating problems.

As Bloomberg’s three-term tenure drew to a close in late 2013, more city agencies had moved to data-driven decisionmaking. Still, information was often stovepiped in individual departments. While city agencies were collecting more and perhaps better data than ever before, the uncoordinated nature of department-level reforms made it hard to reconcile data sets to solve system-wide problems. MODA had managed this itself on an ad hoc basis by aggregating data to solve a specific problem, like identifying which pharmacies distributed painkillers illegally.

But it was unclear whether Bloomberg’s initiative would continue after his term expired. Should data-driven governance be standardized across all city agencies? What about privacy concerns? Was the New York model one to emulate across the country and around the globe? At what cost?