California’s SB 1120 Regulates AI in Health Plan Utilization Review and Management Activities Starting in January

On September 28, California Gov. Gavin Newsom signed Senate Bill 1120 Health Care Coverage: Utilization Review into law, amending § 1367.01 of the Health and Safety Code and § 10123.135 of the state’s Insurance Code. Effective January 1, the new law regulates how health care service plans and disability insurers, including specialized health care service plans and specialized health insurers, use artificial intelligence (AI), algorithms, and other software tools to ensure compliance with specified requirements—including fairness and non-discrimination standards—for utilization review and management processes. SB 1120 also applies to vendors contracted with covered health plans for services that include utilization review or utilization management functions. SB 1120 is just one of several wide-ranging AI-related laws California produced this legislative session.

Overview of SB 1120 Health Care Coverage: Utilization Review

SB 1120 aims to address the growing use of AI and machine learning in health care, particularly in reviewing, approving, modifying, or denying requests for covered health care services based on medical necessity in utilization review or management activities. It establishes guidelines for the application of these technologies to ensure that they complement, rather than replace, human clinical decision-making.

Key Provisions of SB 1120

1. Individualized Decision-Making: Health care plans and insurers must ensure that their use of AI or algorithmic systems for the purposes of utilization review or management functions rely on specific clinical data from an enrollee's medical history, individual clinical circumstances as presented by a requesting provider, and other relevant clinical information contained in a medical or other clinical record, rather than using generalized datasets. This is intended to prevent decisions being made based solely on aggregated data that might overlook individual medical needs.
   
   
2. Human Oversight and Control: AI tools cannot replace healthcare decision-making and autonomously deny, delay, or modify care. Final determinations regarding medical necessity must be made by licensed physicians or health care professionals competent to evaluate the clinical issues. The bill stresses that AI tools cannot supplant professional medical judgment and decision-making, and that AI tools cannot directly or indirectly cause harm to enrollees, ensuring patient safety remains paramount.
   
   
3. Transparency and Disclosure: Health care service plans and disability insurers must clearly disclose how AI tools are being used in utilization reviews. This includes disclosing compliance records, written policies and a description of the process by which the plan reviews and approves, modifies, delays, or denies requests in the provision of health care services. Plans must file these policies and procedures—and the description of the process by which the plan reviews and approves requests with the appropriate authorities—with the appropriate authorities, and disclose them to providers, enrollees, and the public upon request.
   
   
4. Non-Discrimination: AI systems must be applied fairly and equitably, without discrimination against any enrollees. The bill emphasizes that AI tools cannot directly or indirectly discriminate against enrollees in violation of state or federal anti-discrimination laws.
   
   
5. Audits and Compliance Reviews: AI systems used by health plans are subject to regular audits and compliance reviews by the California Department of Managed Health Care (DMHC) and the Department of Insurance (DOI). These audits are intended to ensure the technology's accuracy, reliability, and adherence to legal requirements, such as protecting patient data from unauthorized use or disclosure under the state’s Confidentiality of Medical Information Act and the Health Insurance Portability and Accountability. Health plans must also periodically review the outcomes of AI-driven decisions to improve system performance and reliability.
   
   
6. Penalties for Non-Compliance: Willful violations of SB 1120's requirements by health care service plans may result in significant administrative penalties imposed by the DMHC or the Insurance Commissioner. The penalties aim to ensure strict adherence to the new AI regulations and protect patients from the risks of inappropriate denials of care.

Implications for Health Plans and Insurers

Health care providers and insurers that rely on AI tools for utilization review and management activities need to evaluate their systems carefully to ensure compliance with SB 1120. This may involve updates to AI models, increased transparency in their application, and ensuring that human medical professionals remain the final decision-makers in determinations of medical necessity.

SB 1120 provides important safeguards for patients, ensuring that decisions about their care are made with proper oversight and individualized consideration. The law aims to strike a balance between technological innovation in health care and the need for human judgment in critical medical decisions.

Additionally, the Centers for Medicare & Medicaid Services clarified earlier this year in an FAQ memo that Medicare Advantage Organizations (MAOs) can use an algorithm or AI system in making coverage determinations, but MAOs must ensure that the algorithms comply with existing laws and regulations for how those coverage determinations are made. For determinations of medical necessity, MAOs must remain compliant with the rules of 42 CFR § 422.101(c), including consideration of a patient’s medical history, physician recommendations, and clinical notes. Therefore, AI systems cannot make determinations of medical necessity without considering factors and circumstances specific to each patient.

Next Steps

Health care service plans and insurers should promptly evaluate their utilization review and management practices (as well as those of their vendors that provide such services) that involve AI tools and algorithms, to ensure they meet the requirements set forth in SB 1120, which appear to apply to prospective, retrospective, or concurrent reviews of requests for covered health care services. Given these broad and somewhat vague requirements, it will be important to monitor how the DMHC and DOI will implement the law and issue further guidance. Legal and compliance teams should also prepare for potential audits and ensure that disclosures about AI tool use are readily available and transparent to both patients and regulators.