Schedule

This panel assembles key representatives from federal and state regulatory stakeholders to offer their perspectives on the evolving regulatory landscape for development and use of AI applications in health care. The FDA and ONC play a vital role in advancing safe, effective, interoperable and transparent health technology across the health ecosystem, leveraging their authorities, investments, and programs to guide industry. The FTC uses its authorities to support AI innovation that is pro-competitive and protects consumers. State medical boards play an important role in regulating the use of AI in care delivery, particularly when it comes to the practice of medicine by licensed physicians and other health care professionals. Hear firsthand how regulators evaluate and approach managing risk related to emerging AI use cases in health care using their existing authorities, and where they see gaps in existing regulatory frameworks.

Good governance plays a critical role in developing and using AI in ways that create fair, accurate, legally compliant, and socially beneficial outcomes that patients and the health care industry can trust. This panel will delve into the critical issues surrounding governance in the development and use of AI in health care. A cross-functional panel of experts will share their insights on the following key areas:

• Ethical Frameworks: Developing ethical principles and decision-making processes to guide AI use cases

• Stakeholder Engagement: Fostering collaboration and dialogue among health care providers, technologists, and patients

• Transparency and consent: Supporting transparency and determining whether consent is required or feasible

• Regulatory requirements: Keeping up with evolving federal and state requirements related to privacy and other regulatory requirements

• Data Quality: Assessing quality of data after it is generated

Bias, a natural part of human nature, has traditionally impacted health care to create and maintain profound inequities. How we recognize, confront, and adjust to this reality has powerful implications with the dawn of artificial intelligence. Algorithms themselves, step-by-step procedures designed to solve specific problems and/or perform specific tasks are neither good nor bad, biased nor unbiased, but when infused with biased information or without recognition of existing bias, may further entrench health care inequities. This panel will address how to navigate bias in AI applications as well as various approaches to identify, mitigate, and monitor the impact of bias using various solutions throughout the AI lifecycle.

Companies are in the early stages of developing, contracting for and using AI tools. The industry is evolving so quickly that lawyers are trying to catch up to determine how to apply existing contractual protections to this new technology and evaluate new protections. This session will discuss key contracting considerations for health AI tools, including:

• IP property rights
• AI algorithms
• Training data
• Generated content
• Assignment, licensing, etc.
• Data ownership
• AI Governance, transparency, and audit rights
• Performance warranties, including representations as to training data)/protections and the ability to evolve with changing laws
• Ongoing monitoring/evaluation, including proactively assessing potential bias
• Insurance
• Liability
• Termination and exit strategies
• Other contractual considerations within a continuously shifting regulatory environment

This session will explore how to navigate HIPAA, the Confidentiality of Substance Use Disorder Patient Records Rule at 42 C.F.R. Part 2, state general privacy laws, new state consumer health data laws (like Washington's My Health My Data Act), and state laws governing specific conditions and treatments when seeking to use and disclose health data to develop AI. Topics will include:

• Whether the development of AI potentially constitutes "research" for purposes of HIPAA
• The extent that business associates can use protected health information (PHI) to develop AI under typical business associate agreements;
• HIPAA's prohibition on the sale of PHI and its potential application to transactions between health care entities and technology companies to develop AI;
• Regulatory requirements on de-identified data that is sold or licensed for purposes of AI development; and
• Categories of health data that raise additional privacy concerns if included in data sets for AI development (such as behavioral health data).

• What are real-world practical use cases of AI in health care? We begin our presentation with a demonstration of Microsoft Generative AI products useful to attorneys and to health care industry players

• AI use cases raise a host of legal considerations. Our presentation will provide an overview of some key legal considerations, including privacy and security, copyright, data use and ownership, liability, and monitoring bias

• Hearing from the front lines, we’ll discuss practical challenges and considerations implementing AI use cases, including what issues providers face when contracting with AI vendors

• To manage certain legal and practical challenges, companies create internal AI policies. We’ll discuss best practices for preparing company AI policies and real-world examples

AI is increasingly used in health care and life sciences in a variety of clinical and non-clinical ways. With regulations often lagging behind technological advancements, it is imperative for legal practitioners to stay abreast of case law and enforcement trends in this domain. This presentation aims to evaluate relevant case law and enforcement actions pertaining to AI in health care as a means to identify key areas of risk and possible regulation moving forward. We will examine antitrust implications of market consolidation implicating vast amounts of patient data as well as use of patient data related to AI-based algorithmic pricing. We will also examine recent government enforcement actions related to use of AI in ways that could interfere with clinical decision-making to maximize profit. We will also examine government enforcement in the false claims arena related to non-clinical applications of AI in the context of automation of medical claims submission.

• AI in health care can perpetuate discrimination if not implemented carefully. Providers should be aware of potential biases in data and algorithms. Current anti-discrimination laws and recent actions by the Biden administration aim to provide safeguards.
• The Human-Centered AI (HCAI) framework emphasizes incorporating human values and considerations into AI development and use. While beneficial, it also has limitations. Commercial AI health technologies show varying alignment with HCAI principles.
• AI "black box" models can be difficult to interpret and current metrics may not accurately reflect trustworthiness. Centering human considerations is proposed as a better approach for establishing appropriate reliance on AI models.
• Key considerations include: 1) Proactively identifying and mitigating sources of bias 2) Evaluating AI health tools through an HCAI lens 3) Establishing performance in the specific context of use as a touchstone measure for model trustworthiness 4) Ongoing monitoring and adjustment as real-world impacts become apparent.

Trustworthy AI must be safe, secure, and resilient. As healthcare organizations and other stakeholders grapple with implementing an appropriate AI governance framework, they must consider how best to identify and address safety risks and other harms to patients, clinicians, and the organization. This session will explore:

• How health care delivery organizations are evaluating AI use cases and solutions

• The risks to safety and other harms that can arise from health AI, including to patients, clinicians and other health care workers, and the operations of the health care organization

• The role that patient safety organizations (PSOs) can play in supporting safety in health AI, where we will discuss:

  • The basics of PSOs, including what they are, what they protect, and how providers and other stakeholders can work with PSOs;

  • Emerging legal and regulatory requirements for addressing AI safety, including in the White House Executive Order on AI (October 2023) and other frameworks; and

  • Ways PSOs can support AI safety and how existing PSO and patient safety frameworks can be modernized to address AI’s unique challenges

• How health care delivery organizations can balance safety with innovation when implementing AI, including implementation considerations for today’s evolving AI landscape