Skip to main content

Ethical and Legal Challenges Posed by the Implementation of AI Applications in the Healthcare Setting

Publication Date:
Updated:

Collection Editor(s):

Collection Editor(s)
Name & Degree
Sara Gerke, Dipl-Jur Univ, MA
Work Title/Institution
Assistant Professor of Law, Penn State Dickinson Law
  • Introduction

    Healthcare institutions are implementing artificial intelligence (AI) at a rapid pace. The hope is that AI will improve the quality of care and reduce costs in the long run. However, the deployment of AI in healthcare settings also presents new ethical and legal challenges. For example, AI can reproduce health disparities and pose a risk to patients if human factors, like implicit or explicit bias, are present in the training data set or it lacks representation from population subgroups. AI also raises challenges for regulators like the U.S. Food and Drug Administration (FDA). How should regulators like the FDA deal with so-called “adaptive” AI algorithms that continuously learn or opaque (“black-box”) algorithms? Regulators need to update the regulatory framework for AI-based medical devices as soon as possible to ensure that AI-based medical devices are reasonably safe and effective when introduced on the market and stay safe and effective throughout their entire life cycle. A new regulatory framework is particularly important given that the FDA has already permitted the marketing of more than 340 AI-based medical devices, and shortcomings in the framework can compromise…

    Healthcare institutions are implementing artificial intelligence (AI) at a rapid pace. The hope is that AI will improve the quality of care and reduce costs in the long run. However, the deployment of AI in healthcare settings also presents new ethical and legal challenges. For example, AI can reproduce health disparities and pose a risk to patients if human factors, like implicit or explicit bias, are present in the training data set or it lacks representation from population subgroups. AI also raises challenges for regulators like the U.S. Food and Drug Administration (FDA). How should regulators like the FDA deal with so-called “adaptive” AI algorithms that continuously learn or opaque (“black-box”) algorithms? Regulators need to update the regulatory framework for AI-based medical devices as soon as possible to ensure that AI-based medical devices are reasonably safe and effective when introduced on the market and stay safe and effective throughout their entire life cycle. A new regulatory framework is particularly important given that the FDA has already permitted the marketing of more than 340 AI-based medical devices, and shortcomings in the framework can compromise patient safety and shake public trust. There are also several questions about data privacy and security. AI relies on a large amount of data, and its use and sharing can, in some cases, compromise patient privacy. The Dinerstein v. Google lawsuit shows the likelihood that patients will struggle to successfully sue hospitals for sharing their data with technology companies like Google because they cannot demonstrate damages. This case also raises the issue of “data triangulation,” emphasizing the risk that even a de-identified data set may be re-identified if technology companies use other data sets at hand. Lastly, AI raises liability questions. For example, who is liable when a physician follows an inaccurate AI recommendation and the patient is injured as a result? This collection is a starting point for readers who wish to become familiar with bias, privacy, liability, and other ethical and legal challenges of implementing AI applications in the healthcare setting.

Collection Header
Ethical Issues
Body
Collection Header
Bias
Body
Collection Header
Data Privacy and Security
Body
Collection Header
Liability
Body
Tags
Artificial intelligence
Healthcare
health disparities
Regulation
FDA
data privacy
data security
bias
Machine Learning

Suggested Citation

Gerke, S. (2022). Ethical and legal challenges posed by the implementation of AI applications in the healthcare setting. In ELSIhub Collections. Center for ELSI Resources and Analysis (CERA). https://doi.org/10.25936/cv1z-0f06

Share

About ELSIhub Collections

  • ELSIhub Collections are essential reading lists on fundamental or emerging topics in ELSI, curated and explained by expert Collection Editors, often paired with ELSI trainees. This series assembles materials from cross-disciplinary literatures to enable quick access to key information.

ELSIhub Collections