What Will California’s New AI Auditors Actually Test?

Washington policy shops have a durable job description. They keep half-formed ideas on life support until a scare, a hearing, or a headline makes the previously impossible look inevitable. That is not cynicism. It is how most technology law actually gets written. The ideas sit in draft bills, white papers, and coalition letters for years. Then something breaks, and the nearest binder becomes the statute.

The shock has not arrived. No frontier model has yet produced the kind of public catastrophe that forces Congress to legislate in a weekend. What has arrived is a change in the weather. Federal talking points on AI have hardened in a matter of weeks. Company talking points have hardened with them. OpenAI’s chief global affairs officer, Chris Lehane, published a note this week arguing that “the AI policy window is open” and that the country needs mandatory national safety requirements. The same post endorsed four California bills sitting on Governor Gavin Newsom’s desk, two of which he signed on September 9.

That is not a conversion so much as a repositioning. OpenAI has spent the last year arguing for a federal floor and against a patchwork of state experiments. Supporting Sacramento while Congress stalls is consistent with that strategy if you accept the company’s own label for it: reverse federalism. Build a critical mass of state rules the labs can live with, then dare Washington to codify the result. Anthropic had already backed the two auditor bills while the legislature raced to adjourn. When the two largest frontier labs bless the same pair of statutes in the same month, the market should pay attention. Not because the laws are radical. Because they are not.

Two statutes, one marketplace, zero mandate

The bills are Assembly Bill 1405, authored by Assemblymember Rebecca Bauer-Kahan, and Senate Bill 813, authored by Senator Jerry McNerney. Together they do something American AI law has avoided until now. They start building the plumbing for a regulated industry of outside assessors. They do not require a developer to hire one.

Read that again, because the press releases will not linger on it. Newsom’s office called the package a first-in-the-nation framework for independent third-party evaluation. Bauer-Kahan said California cannot expect industry to grade its own homework. Both statements are directionally true and operationally incomplete. The homework is still optional. The state has licensed the graders.

California already has a frontier-model statute on the books. Senate Bill 53, the Transparency in Frontier Artificial Intelligence Act signed in 2025, forces large frontier developers to publish a safety framework, report certain critical incidents, and protect whistleblowers. That framework must disclose whether the developer used third-party assessments. Disclosure is not the same thing as a requirement to obtain one. A company can publish a polished safety narrative, note that it declined outside review, and remain inside the statute. Headlines that treat this week’s signing ceremony as the birth of mandatory AI audits are doing marketing work, not statutory work.

AB 1405 is the first layer. By January 1, 2029, the Government Operations Agency must stand up a public AI Auditor Registry, set annual fees, and open a channel for misconduct complaints. From that date, an unregistered person may not offer, sell, or conduct a “covered AI audit” in California. Registrants have to list relevant certifications, identify the state laws under which they audit, and file a standard operating procedure that explains the basis for any claim that their methods are accurate, reliable, or valid. They must keep records for ten years. They must meet independence rules that will look familiar to anyone who has sat through a financial-statement audit: no job-hunting with the auditee mid-engagement, no audit if a financial or employment relationship would reasonably impair objectivity, no auditing a system the firm previously designed.

Those are serious professional constraints. They are also constraints that bite only if someone is actually buying the service. TechNet, which counts OpenAI among its members, opposed an earlier draft of 1405 as premature and deficient, in part because the bill built an auditor regime without a corresponding duty to be audited. That objection was self-serving. It was also analytically correct. A registry without a mandate is a professional-licensing statute dressed up as safety policy. California has been on the bleeding edge of privacy and AI but as they built an AI auditor marketplace. It’s Currently Not a Requirement to  Use It…

The second layer arrives a year earlier, and it is still a sketch

SB 813 is the companion and, in some ways, the more ambitious instrument. It directs GovOps to deliver, by January 1, 2028, a regulatory framework for independent verification organizations. IVOs are the entities the state will designate as having demonstrated expertise in assessing the risks of an AI system or model and in identifying the metrics behind that assessment. The statute already gestures at transparency (annual reports) and independence. The substance — qualification criteria, testing methods, suspension and termination procedures, the standards against which a model is judged — is left for the agency to write.

That sequencing is not an accident. California is constructing a two-tier market. Ordinary registered AI auditors will handle a broader set of covered audits. IVOs will sit above them as the designated verifiers for higher-stakes assessments. Not every registered auditor will be an IVO. Every IVO will almost certainly have to register as an auditor. The architecture is neat. The payload is not specified.

Both bills punt the hardest question into future rulemaking. SB 813 tells GovOps to “identify and consider” standards, frameworks, guidelines, and best practices. AB 1405 requires widely recognized standards appropriate to the system only “to the extent appropriate standards are available.” In plain English: if NIST, ISO, the frontier labs, or a future industry consortium have not produced a test the state is willing to bless, the auditor’s job description remains a blank. Twenty-seven months until the registry deadline sounds like a long runway in legislative time. In model-release time it is several generations. Asking what these future auditors will test is not a rhetorical flourish. It is the entire statute.

Illinois already answered the question California ducked

Other states have been circling the same problem. Qualification regimes for outside assessors have been moving through legislatures all year. California is the first to enact a standalone auditor-and-IVO package. It is not the first to require an audit.

Illinois is. Governor J.B. Pritzker signed the Artificial Intelligence Safety Measures Act in July. Beginning in 2028, large frontier developers — those above a compute threshold and a $500 million revenue line — must retain an independent third party every year to audit compliance with the Act and assess internal controls. The auditor needs demonstrated competence in frontier-model safety, must work to generally accepted auditing standards, cannot have a financial interest in the developer beyond the engagement fee, and must receive reasonably necessary materials, including unredacted documents subject to security protocols. The report goes to the developer, then in redacted form to the public, the Illinois Emergency Management Agency, and the Attorney General.

That is a different animal. California built the storefront and left the door unlocked. Illinois told covered labs they have to walk through it. If you care about whether an audit regime changes behavior rather than letterhead, watch Springfield, not just Sacramento. The California package will still matter, because the largest developers already treat California as a de facto national standard-setter and because OpenAI has said out loud that it wants state laws to converge. But convergence around a voluntary assessment culture is not the same as convergence around verified compliance.

The EU comparison everyone will get wrong

The other comparison coming in every briefing deck is the EU AI Act. Treat it carefully. California and Brussels are not building the same machine. They are building mirror images of different machines.

The Act requires conformity assessments for high-risk systems. Most of that work is internal. The third-party path is narrow and conditional — biometrics and a handful of Annex III cases, not a general audit market. The standards themselves are more prescriptive than anything in AB 1405 or SB 813, because the EU is grafting AI onto a product-safety tradition that already knows how to designate notified bodies. Member states name national notifying authorities. Those authorities must meet EU-level tests for impartiality, competence, and transparency. The paperwork looks similar to California’s independence rules. The legal logic does not. Europe is saying: if you place a high-risk system on the market, you must demonstrate conformity against specified requirements, usually by yourself, sometimes with a designated body. California is saying: if you want to sell an audit in this state after 2028 and 2029, you must register and behave. Whether anyone must buy that audit is a question for a later legislature.

There is a respectable argument that California’s approach is more honest about the current state of the science. We do not yet have stable, widely accepted tests for “this frontier model will not help a competent actor build a biological weapon” or “this hiring model does not launder protected-class bias through a proxy.” Building a marketplace of assessors before locking the scoring rubric can look like prudence. It can also look like a decision to let the first movers write the rubric. Labs that already employ evaluation teams, fund academic red-teaming, and sit on standards bodies will have a structural advantage when GovOps starts “identifying and considering” best practices. That is not a conspiracy. It is how professional fields form. It is also why independence rules on paper will not save the regime if the underlying tests are the ones the auditee helped design.

What an AI audit is even for

Strip away the signing statements and the question is crude. What is the product? Financial audits exist because securities law, listing rules, and lender covenants make an unqualified opinion a condition of doing business. The Public Company Accounting Oversight Board exists because Congress decided that the profession could not police itself after Enron. Nobody serious pretends those audits catch every fraud. They still change the cost of lying.

AI audits do not yet have that economic function. A covered audit under AB 1405 might test compliance with a California statute, conformity with a benchmark, robustness against a red-team protocol, or some cocktail of all three. The bills leave that menu open. Until a regulator, a plaintiff, a procurement office, or a downstream enterprise customer treats an adverse finding as a real cost, the audit is a reputation accessory. Frontier labs will buy the good ones because buyers, insurers, and European counterparties will start asking. Mid-market deployers of automated decision systems — the hiring screeners, the underwriting models, the benefit-eligibility tools — will buy them when a contract or a state attorney general makes the alternative more expensive. That demand signal is not in either statute.

This is the part of the conversation the beltway class should not be allowed to skip. Registering auditors and designating IVOs is useful infrastructure. It is not accountability. Accountability starts when a developer that skips the audit, shops for a friendly one, or ignores a qualified opinion faces a consequence that is not a blog post. California has given itself two to three years to decide whether it wants that consequence. Industry has given itself the same window to make sure the standards that emerge are ones it can pass.

The consultants will get paid either way

None of this is an argument against the bills. Fly-by-night assurance shops were already circling the AI governance budget. A public registry, a misconduct channel, a ten-year retention rule, and a ban on auditing your own prior design work are better than the alternative, which was an unregulated market of PDF mills. Bauer-Kahan is right that self-grading is a joke. McNerney is right that Washington has not shown up. Newsom is right that a national rule would be cleaner than fifty state experiments. All of that can be true at once, and the package can still be what it is: a market-structure law that postpones the substance.

Compliance teams should treat the next twenty-four months as the actual legislative session. GovOps will write the IVO criteria. Trade associations will flood the docket. Frontier labs will offer their internal eval suites as the reasonable standard. Civil-society groups will argue that any test the lab prefers is contaminated. Procurement officials in California agencies will quietly decide whether an IVO letter becomes a bidding requirement even if the statute never says so. That last path — turning a voluntary regime into a de facto mandate through the state’s own buying power — is how a lot of “soft” technology law hardens. It is also how a lot of it gets captured.

If you work in privacy, AI governance, or vendor risk, the operational takeaway is narrower than the rhetoric. Map which of your systems could become a “covered AI audit” once GovOps defines the term. Decide now whether you want a registered auditor relationship before the 2029 cliff, when the serious firms will be booked. Watch Illinois’s 2028 audit cycle as a preview of what a mandatory report looks like when it has to be filed with an attorney general. Do not confuse a signed bill with a finished standard. And do not let anyone tell you that building the auditor industry is the same project as deciding what the auditor is allowed to fail you for.

The ideas were lying around. California picked them up. The crisis that would force the next step — a duty to submit, a duty to publish the ugly findings, a duty to remediate — has not happened yet. Until it does, the state has constructed a professional class with a rulebook and no compulsory clients. That is a beginning. It is not an audit.