Can Big Data Save Lives?

As the challenges posed by urbanization, globalization, and climate change mount, resilience is becoming a top concern of city governments and citizens alike. While the pace of technical innovation suggests data-driven solutions to many problems, the ability to generate vast amounts of data often outpaces the ability to make sense of it.

At Palantir, we’ve found that many analysis problems are really data integration problems – before you can take action, you need all your facts in one place. Building a unified yet evolving knowledge base – one that seamlessly accommodates new data and user types – enables public and private sector leadership to make more effective decisions. Below are some ways we’re using data integration to build resilience.

Natural disasters: Responding effectively to natural disasters requires immediate situational awareness and rapid prioritization and decision-making amid many unknowns. Following Typhoon Haiyan, we demonstrated how big data could highlight humanitarian needs, identify gaps in response efforts, and save lives:

  • Within hours of Haiyan’s landfall, Palantir engineers integrated all publicly available OCHA data, assessment data, satellite imagery, weather data, geospatial data on key infrastructure and relief resources, and reports from news agencies and governments into a single decision-making platform. Since communications in the Philippines were severely degraded, a tactical cloud was deployed from Singapore to host this platform, and to support instant synchronization of data from a variety of mobile devices including laptops, phones, and GPS.
  • In Leyte province, Direct Relief International used lightweight GPS devices with configurable web forms to submit real-time reports on health facilities in the hardest-hit, areas, then allocated medical supplies accordingly.
  • Analysts viewed pre- and post-disaster satellite imagery overlaid on critical infrastructure locations, allowing them to make initial damage estimates before deploying resources.
  • Palantir partners including UNOCHA used configurable dashboards to identify gaps in service by comparing Philippine government reports on affected people against the work being done by hundreds of NGOs.

Homicide: Well before Hurricane Katrina, the City of New Orleans suffered one of the United States’ highest per-capita murder rates. By integrating numerous public safety, open source, and proprietary data systems, city analysts and researchers have developed comprehensive views and rich insights into gang activity.  This has helped to inform investigations of past crimes and operations to disrupt and dismantle violent networks, as well as development of long-term youth intervention strategies, under the auspices of the NOLA for Life initiative.

Urbanization: As millions of people move into cities, city governments must answer many important questions: How can they expand sustainably? Are new constructions compliant with building and environmental codes? Can municipal services be streamlined to better serve the city’s needs? How should emergency services and operational agencies share data to form a common operating picture? The NYC Mayor’s Office of Data Analytics helps address each of these questions, and more, through DataBridge, a data warehouse linking such departments as Buildings, Finance, City Planning, Consumer Affairs, Fire, and Health. Key use cases include:

  • The Mayor’s Office of Special Enforcement uses DataBridge to manage complaints and prioritize field inspections, as well as using mobile devices to access intelligence in the field and directly submit digital inspection reports. This allows agents to target pockets of illegal activity before they degrade a neighborhood’s quality of life.
  • The Department of Finance Sheriff’s Division investigates fraud against the city by drawing connections across DataBridge sources, resulting in decisive action against repeat offenders and ensuring small businesses can compete on a level playing field.
  • The Department of Sanitation analyzes historical complaint and response data to improve planning for such events as an upcoming snow season or the rollout of a new recycling program.
  • The Fire Department of New York is training users to investigate prior incident locations. Using previously siloed data from multiple agencies, analysts can explore the confluence of factors that indicate increased likelihood of deadly fires, including complaints, violations, and previous inspection results.

Finally, it’s worth acknowledging that each of these approaches is constantly evolving. Some of our best work has occurred in response to live emergencies, but we also understand that while response is the present, resilience is the future. Getting there will require everyone to incorporate lessons learned, operational insights, and new data approaches into long-term planning and prevention. We couldn’t ask for better partners than the Rockefeller Foundation and the cities leading the way.