As Attention on AI Increases, California Ramps Up Oversight

California policymakers are taking a crack this year at regulating the use of artificial intelligence given its growing prominence in everyday life, from teenagers using chatbots to help with homework to employers filtering prospective job seekers.

Advocates for greater AI safeguards in the Golden State say they believe they have momentum given the increased public attention with ChatGPT and other new technologies, as the effort follows on the heels of President Joe Biden’s call for more scrutiny of the industry.

“California usually moves faster than the federal government, but aside from the (California Privacy Rights Act), on AI, it has been quite slow,” said Vinhcent Le, an attorney at the Greenlining Institute focused on technology equity. “It does seem like there’s more interest this year. It seems like a consequential year for automated decision systems and the regulation of it.”

As federal agencies float out guidance amid few existing laws around AI, California is poised to pick up its pace. State regulations around AI’s usage in privacy and employment are beginning to take shape, while lawmakers are proposing bills that could lay groundwork for future AI laws.

Hiring and AI

While technologies like ChatGPT are relatively novel, almost a quarter of organizations already use AI in the hiring process, according to a study last year by the Society for Human Resource Management. Such tools, though, could unfairly filter out certain applicants and lead to discrimination.

The California Civil Rights Department has led in efforts to tackle automation’s hiring bias, and on Tuesday, it formally teed up those proposed regulations up for a public comment period, beginning the rulemaking process.

The proposed rules don’t tread new ground but simply adapt existing discrimination laws around automated technology, according to department council members. The regulations would ban an AI system’s use of a protected characteristic —such as age, race, sexual orientation—or data that could be affiliated with it. Records and data around AI would have to be maintained for four years, and use of personality tests or puzzles driven by AI would be banned for pre-job offers.

More notably, a business would be liable for its own automated decision systems as well as those used by its third-party vendors.

“There’s a focus on accountability; of those who are developing the algorithms,” said Gary Friedman, senior partner at Weil, Gotshal & Manges LLP. “California is trying to find a path to merge accountability with employers and by focusing in on assessing the actual algorithm that you use.”

But there are compliance concerns, given vendors may not want to turn over proprietary or confidential software, wrote Nathan Jackson, a California-based employment attorney with Liebert Cassidy Whitmore, in a blog post. There will need to be more guidance on what exactly constitutes algorithmic bias if one wants to conduct an audit, he added.

Enforcement could also be an issue. Someone discriminated against by an automated system may not be aware such technology is being used in the first place, a concern also raised on the federal level.

Other Agency Work

Such oversight could be addressed in upcoming work by the California Privacy Protection Agency, which is turning its attention toward privacy issues around AI.

Recent updates to the state’s comprehensive privacy law give both consumers and employees certain rights when a business uses automated decision-making technology to collect personal data, such as allowing them to opt out or to be able to request information on how the technology is being used.

The agency is collecting public input on what regulations should be issued in this area. Other subjects it is exploring, such as risk assessments and cybersecurity, could also ensure any AI systems are safe and effective, said Le, who also is a member of the privacy agency’s board.

Both the privacy agency’s and the civil rights department’s efforts are still in the very early stages, observers noted, and it could take months or longer before anything is finalized.

State of Legislation

California bills focused on artificial intelligence didn’t emerge until around 2018. There were a handful of attempts at legislation soon after, but such measures have been sparse or largely unsuccessful.

At least three pieces of legislation have been filed this year aimed at putting more safeguards around AI and potentially filling in any gaps not covered by the regulatory process.

“Absolutely,” said state Sen. Bill Dodd (D) on whether there’s a better shot at passage with all the buzz around AI than in years past. “It’s [AI] got the great ability to make decisions based on data that it has, but nobody knows what’s in that black box. We need to have our eyes wide open about the power of this, and make sure that it’s here to help people and not inadvertently injure consumers.”

Dodd’s bill, SB 313, would create an Office of Artificial Intelligence that would oversee the use of AI among state agencies. Another bill by Assemblymember Rebecca Bauer-Kahan (D), AB 331, could also potentially deal with how state government uses AI.

Both measures have preliminary text. Dodd said he expects a fuller version around April; Bauer-Kahan’s office had no comment.

The third bill, AB 302, would simply ask the state government to conduct an inventory of high-risk automated decision systems.

All three pieces of California legislation will at least address AI usage in state government, setting an example for the private sector. Past bills tasked agencies to figure out guidelines and principles around automation. But the White House’s recent AI Bill of Rightsalready contains a framework for such parameters, providing a road map in the debate.

Dodd said his revised bill would contain a disclosure requirement that alerts a user of when they are interfacing with an AI-driven government application. That standard, if applied to businesses, could beef up enforcement of regulations when people know when they’re dealing with AI.

Though Dodd’s bill only applies to state government, he said there’s a possibility it could be expanded to the private sector, though such inclusion would face an uphill battle. The US Chamber of Commerce has expressed concerns about the Biden administration’s AI framework and California business groups are likely to have similar sentiments.

“We’re going to be mindful also in this bill not to do anything that will decrease innovation,” Dodd said. “The last thing we want to do is disrupt that.”