On February 14, 2020, California State Assembly Member Ed Chau introduced the Automated Decision Systems Accountability Act of 2020, which would require any business in California that provides a person with a program or device that uses an “automated decision system” (“ADS”) to establish processes to “continually test for biases during the development and usage of the ADS” and to conduct an impact assessment on that program or device.

ADS is defined broadly as “a computational process, including one derived from machine learning, statistics, or other data processing or artificial intelligence techniques, that makes a decision or facilitates human decision making, that impacts persons.”  The required ADS impact assessments would study the various aspects of the ADS and its development process, “including, but not limited to, the design and training data of the ADS, for impacts on accuracy, fairness, bias, discrimination, privacy, and security.”  At minimum, the assessments must include “[a] detailed description of the ADS, its design, training provided on its use, its data, and its purpose” and “[a]n assessment of the relative benefits and costs of the ADS in light of its purpose,” with certain factors such as data minimization and risk mitigation required in the cost-benefit analysis.

The provider of the ADS also must determine whether the ADS system “has a disproportionate adverse impact on a protected class,” examine whether it serves “reasonable objectives and furthers a legitimate interest,” and consider alternatives or reasonable modifications that could be incorporated “to limit adverse consequences on protected classes.”
Continue Reading California Introduces Bill to Regulate Automated Decision Systems