California Lawmakers Pass Bill AB 1018 to Improve Accountability in Automated Decision Making Processes

California State Assembly Considers Comprehensive Bill on Automated Decision Systems for Consumer Safety

On 20 February 2025, a Bill on Automated Decision Systems (AB 1018) was introduced to the California State Assembly.

This proposed regulation sets out a comprehensive framework for managing high-risk automated decision systems (ADS) powered by machine learning and artificial intelligence.

These systems, which impact consequential decisions in areas such as employment, education, housing, and healthcare, will now be subject to a series of guidelines designed to ensure accountability, transparency, and the protection of individual rights.

Bill Text - AB-1018 Automated decision systems.

AB 1018 Automated decision systems.

AB-1018 Automated decision systems.

Robust Guidelines for Accountability and Transparency

Under AB 1018, both developers and deployers of ADS are required to adhere to strict performance and documentation standards. Key provisions include:

• Performance Evaluations:
  • Developers must conduct regular performance evaluations of their ADS.
  • Any updates or fine tuning that materially change the system’s output trigger a new evaluation.
• Independent Audits and Documentation:
  • Both developers and deployers must submit their systems to third-party audits.
  • Comprehensive records and evaluations must be maintained and made available on request to enforcement authorities.
• Disclosure and Consumer Rights:
  • Beginning 1 January 2027, deployers must clearly disclose the use of ADS to individuals affected by consequential decisions.
  • They are required to offer opt-out options and provide an opportunity to appeal decisions made or facilitated by these systems.

These measures are intended to create a transparent operational environment, ensuring that any automated decision that has a material impact on people is carried out responsibly.

Impact Assessments and Consumer Protections

For deployers using ADS to make decisions that affect more than 6,000 people, the Bill mandates regular impact assessments every three years.

This requirement is aimed at verifying that the automated systems operate within acceptable limits and do not produce adverse effects such as unfair bias or discriminatory outcomes.

The impact assessments will examine:

• The accuracy and reliability of the system’s outputs.
• Any unforeseen disparate impacts on groups of individuals.
• The overall fairness of the decision-making process facilitated by the ADS.

Additional safeguards include the obligation to maintain detailed documentation and provide clear, plain language disclosures to affected individuals.