Visual Analytics for Multi-Level Triage and Investigation of Incident Reports

Kakar, Tabassum

Etd

Visual Analytics for Multi-Level Triage and Investigation of Incident Reports

Public

Regulatory agencies in domains from health-care, finance, and aviation receive a large number of incident reports related to the products or services being overseen. Analysts at these agencies must conduct triage and investigation tasks to uncover critical incidents of concern and take appropriate action to stop such incidents from happening in the future. The number and complexity of these incident reports pose challenges for the analysts who need to make critical yet timely decisions by combing through these reports. Yet computational techniques to analyze these reports alone are not sufficient for the effective triage and investigation of incident reports due to requiring human judgment and thus involvement in the process. In this dissertation, we design visual analytics techniques based on the analysts’ workflows to augment their capabilities to examine incident reports. We approach this goal through three primary research tasks: (1) Exploratory triage of incidents at an overview level by leveraging machine-learning and Natural Language Processing (NLP) generated meta-data to help in hypothesis formation about potential incidents of concern. Specifically, we design interactive visualizations guided by domain-workflows that help analysts explore, screen, and prioritize critical incidents. We evaluate these systems via user studies and case studies. (2) Confirmatory triage of individual reports associated with a screened incident to decide if it warrants further investigation. We design a domain-informed glanceable visual summary that assists analysts in getting the gist of all the information crucial for decision making. We conduct a user study with 20 domain experts to evaluate the effectiveness of our visual summary on triage performance and experience as compared to using a tabular baseline used by the FDA. (3) Investigatory analysis of incident reports to build cases supporting particular concerning incidents to take regulatory action. We design user-driven visualizations and interactions that support analysts in the evidence identification, collection, and management for alarming incidents. We evaluate these designs via case studies and user interviews with the domain experts from the FDA. Successful results in this research may have a significant impact on the current practices of incident analysis in regulatory agencies, particularly Pharmacovigilance activities performed in the majority of western countries. Our research forms the basis for designing visual analytics systems that facilitate the seamless analysis of the huge influx of incoming critical incident reports requiring immediate attention. We discuss the implications of using visual analytics for triage in critical workflows. This lays the groundwork for further research targeting transparency and uncertainty in text-based visual designs for triage.

Creator