AMA Journal of Ethics^®

Abstract

In June 2018, the American Medical Association adopted new policy to provide a broad framework for the evolution of artificial intelligence (AI) in health care that is designed to help ensure that AI realizes the benefits it promises for patients, physicians, and the health care community.

Responding to Industry Activity on Artificial Intelligence

In June 2018, the American Medical Association (AMA) adopted a new policy, H-480.940, “Augmented Intelligence in Health Care,”¹ to provide a broad framework for the evolution of artificial intelligence (AI) in health care that is designed to help ensure that AI realizes the benefits it promises for patients, physicians, and the health care community. In parallel, a wave of scientific and investment activity is cresting, focused on AI and its applications in health care and medicine. Advances in computing power, storage, sensors, and multidisciplinary research have laid the groundwork for the increased development and use of AI techniques in health care.

AI is the ability of a computer to function appropriately and with foresight in its environment, that is, to complete tasks in a manner typically associated with a rational human being. Augmented intelligence is an alternative conceptualization that focuses on AI’s assistive role, emphasizing a design approach and implementation that enhances human intelligence rather than replaces it. Collectively, these areas of scientific research and health care industry activity represent thousands of peer-reviewed studies and, by one estimate, over $2.7 billion in investment across 121 digital health companies and 206 funding deals between 2011 and 2017, just within the United States.²

The background report that informs this new policy focused on 2 fundamental normative conditions for appropriate integration of AI into health care.³ First, health care AI should be understood as a tool to augment professional clinical judgment, not a technology to replace or override it. Second, the development of health care AI tools should attend carefully to the design and evaluation of individual applications, to issues of patient privacy, and to thoughtful clinical implementation.

Ethical Dimensions of AI in Health Care Practice

Design challenges loom large. An AI-derived algorithm is only as good as the data with which it works.

The research, patient care, and insurance records available as training data sets for health care AI can be highly variable.… Clinical trials systematically include or exclude participants with certain characteristics; patient charts and insurance records capture information only from those individuals who have access to the health care system.³

Rarely do such records contain information about social determinants of health, for example. AI systems can, invisibly and unintentionally, reproduce or magnify the biases of their human designers or training data sets in ways that risk exacerbating existing health disparities—such as when data reflect only the conditions of individuals who have access to health care to begin with.⁴

Conversely, algorithms properly designed and deployed can help compensate for or minimize human bias. AI algorithms must also be evaluated using criteria that are “clinically relevant and evaluation should be representative of how the algorithm will be applied in practice.”³ Predictive algorithms, for example, should be able to predict events sufficiently in advance to meaningfully influence care decisions and patient outcomes.

Addressing concerns about privacy and security are 2 further challenges for the evolution of health care AI. Existing practices of notifying patients and obtaining consent for data use are not adequate, nor are strategies for de-identifying data effective in the context of large, complex data sets when machine learning algorithms can re-identify a record from as few as 3 data points.⁵ Algorithms can also be vulnerable to cyberattack.⁶ Evolving technical responses to the challenge of ensuring data security and integrity, such as blockchain-style technologies,⁷ are promising, but traditional expectations for health care privacy might no longer be attainable. Rigorous oversight of data use and transfer will be critical to protecting patients’ interests.

To realize its promise, health care AI must be deployed in ways that promote quality of care and minimize potentially disruptive effects. Physicians will need to learn to work effectively with AI systems,³ just as medical students and residents are now trained to work effectively with electronic health records.⁸ If physicians are to base clinical recommendations on AI, “they will need to understand AI methods and systems sufficiently to be able to trust an algorithm’s predictions.”³ Even as technical solutions to the problem of trust evolve, such as algorithms that “explain” to users why a particular prediction has been made, the health care organizations that implement AI systems should vigilantly monitor the operation of those systems to identify and address adverse consequences.

Legal experts and commercial developers of AI tools that aid in diagnosis must also begin to address questions of liability when incorrect diagnoses are made either by humans using augmented intelligence tools or by AI tools directly. Questions also remain about the evolving role of the patient-physician relationship and fiduciary compact in an algorithm-enabled health care environment.⁹

The AMA’s adoption of H-480.940 suggests the ethical importance of these questions in calling for development of thoughtfully designed, high-quality, clinically validated health care AI that does the following¹:

a) is designed and evaluated in keeping with best practices in user-centered design, particularly for physicians and other members of the health care team;
b) is transparent;
c) conforms to leading standards for reproducibility;
d) identifies and takes steps to address bias and avoids introducing or exacerbating health care disparities including when testing or deploying new AI tools on vulnerable populations; and
e) safeguards patients’ and other individuals’ privacy interests and preserves the security and integrity of personal information.

Values of ethical relevance considered in this policy include professionalism, transparency, justice, safety, and privacy.