Machine learning for population health and disease surveillance

Undergraduate Seminar Talk Seminar

Daniel Neill (Courant University, Computer Science/Wagner School of Public Service / Center for Urban Science & Progress)

Thursday, September 13, 2018 -
4:30pm to 5:30pm
Location: 
330 Gross Hall, Ahmadieh Family Grand Hall

Over the past decade, we have developed a variety of new machine learning approaches for early and accurate detection of emerging outbreaks of disease. This talk will describe our work in addressing three distinct public health challenges: syndromic surveillance using small-area count data, drug overdose surveillance using multidimensional case data, and pre-syndromic surveillance using free-text emergency department chief complaints. In the first problem setting, we monitor a set of known syndrome types (e.g., gastrointestinal illness) and identify space-time clusters of disease. In the second problem setting, we use the multiple dimensions of each case (age, race, gender, location, and drug types) to identify emerging patterns of fatal accidental overdoses affecting specific subpopulations. In the third problem setting, we identify clusters of cases that are of interest to public health but do not correspond to existing syndrome categories, such as "novel" disease outbreaks with previously unseen patterns of symptoms. Across all three problem settings, we develop new "fast subset scan" approaches to deal with the size and complexity of real-world data. Subset scanning is a novel pattern detection approach which treats the detection problem as a search over subsets of data records and attribute values, finding those subsets which maximize an expectation-based scan statistic. One key insight is that this search over subsets can be performed very efficiently, reducing run times from years to milliseconds, using the "linear-time subset scanning" property of many commonly used likelihood ratio scan statistics. These fast subset scanning approaches enable accurate, precise, and computationally efficient detection of emerging public health threats, providing state and local health departments with the situational awareness needed for early and targeted interventions.

Last updated: 2019/12/10 - 10:28pm