Series
The 5th ELSI Congress - ELSIcon2022
ELSIcon2022 Paper: Facilitators and barriers to ethical machine learning in healthcare: A qualitative study on developer perspectives of potential harms
Type
Conference

ELSIcon2022 • Paper • June 3, 2022

Ariadne Nichol, Meghan Halley, Carole Federico, Pamela Sankar, Mildred Cho

Machine learning predictive analytics (MLPA) applications have attracted significant investment in recent years. Examples include tools marketed for predicting disease onset and healthcare costs, as well as assistive technology for making treatment recommendations for patient care. MLPA-based products are not the typical medical device; the tools are dynamic as they constantly learn and iterate. Thus, the FDA is testing novel process-based regulatory approaches for these unique products. For this process-based approach to be successful, it will have to rely on developers’ awareness of potential harms and their perception of whose responsibility it is to mitigate such harms. To better understand what developers see as key safety concerns and potential unintended consequences of their products, we interviewed 40 developers currently working on MLPA-based products for healthcare. We found that developers, in the aggregate, described potential harms to individuals, groups and health systems. We could distinguish harms that were inherent to machine learning and those that arose from the health care or industry contexts in which the applications were used or developed. Developers also described factors that could contribute to or mitigate those harms. Our findings suggest that developers could be receptive to process-based regulatory frameworks but that much education and guidance would be needed.

Tags

Keywords
Live
Paper
Emerging technologies - public opinion and policy
AI and machine learning

Videos in Series

The 5th ELSI Congress - ELSIcon2022 - ELSIcon2022: Paper Session 12

The 5th ELSI Congress - ELSIcon2022 - ELSIcon2022: Flash Session 2

The 5th ELSI Congress - ELSIcon2022 - ELSIcon2022: Paper Session 14

The 5th ELSI Congress - ELSIcon2022 - ELSIcon2022: Moderated Plenary Panel

The 5th ELSI Congress - ELSIcon2022 - ELSIcon2022: Paper Session 13

The 5th ELSI Congress - ELSIcon2022 - ELSIcon2022: Paper Session 17

The 5th ELSI Congress - ELSIcon2022 - ELSIcon2022 Paper: Fresh Air for Genetic Data Sharing

The 5th ELSI Congress - ELSIcon2022 - ELSIcon2022: Paper Session 15

The 5th ELSI Congress - ELSIcon2022 - ELSIcon2022: Paper Session 10

The 5th ELSI Congress - ELSIcon2022 - ELSIcon2022: Paper Session 9

The 5th ELSI Congress - ELSIcon2022 - ELSIcon2022: Paper Session 8

The 5th ELSI Congress - ELSIcon2022 - ELSIcon2022: Paper Session 7

The 5th ELSI Congress - ELSIcon2022 - ELSIcon2022 Trainee Networking Session: Working with NHGRI

The 5th ELSI Congress - ELSIcon2022 - ELSIcon2022 Paper: Diversity’s Pandemic Distractions

The 5th ELSI Congress - ELSIcon2022 - ELSIcon2022: Paper Session 6

The 5th ELSI Congress - ELSIcon2022 - ELSIcon2022: Paper Session 5

The 5th ELSI Congress - ELSIcon2022 - ELSIcon2022: Paper Session 4

The 5th ELSI Congress - ELSIcon2022 - ELSIcon2022: Paper Session 3

The 5th ELSI Congress - ELSIcon2022 - ELSIcon2022: Paper Session 2

The 5th ELSI Congress - ELSIcon2022 - ELSIcon2022: Paper Session 1

The 5th ELSI Congress - ELSIcon2022 - ELSIcon2022: Flash Session 1

The 5th ELSI Congress - ELSIcon2022 - ELSIcon2022 Paper: ELSI and the New Precision Stewards?

The 5th ELSI Congress - ELSIcon2022 - ELSIcon2022 Paper: Toward an anthropology of rare disease

The 5th ELSI Congress - ELSIcon2022 - ELSIcon2022 Paper: In the Shadows of Virtual Family Trees

The 5th ELSI Congress - ELSIcon2022 - ELSIcon2022 Paper: The ELSI of Pandemic Biobanking

The 5th ELSI Congress - ELSIcon2022 - ELSIcon2022 Paper: Whose future should we worry about?

Related Videos