Rooftop lidar sensor housing on a dark vehicle, captured mid-rotation

Dataset

Autonomous vehicle disengagement reports: reading the California data properly

The longest-running public dataset on automated driving is also the most widely misread. Here is what it measures, and what it does not.

Series begins: 2015
Jurisdiction: California
Filed: Annually
Comparable: No

The metric

Since 2015, every company testing an automated driving system on California public roads has been required to file annual autonomous vehicle disengagement reports stating its autonomous mileage and the number of times the system was disengaged. The result is the longest continuous public dataset the industry has produced, and for most of the last decade it has been used as a league table: divide miles by disengagements, sort descending, publish the ranking.

That ranking is not meaningful, and the reason is worth understanding rather than asserting. The autonomous vehicle disengagement reports were never designed to support it. The numerator and the denominator are both under the reporting company's control. An operator chooses which roads to test on, and the difficulty range between a divided highway and an unprotected left turn across four lanes of traffic is enormous. An operator also decides what counts as a reportable disengagement, because the regulation describes a category rather than a test procedure. Two companies with identical driving systems, testing in different places under different internal review standards, will file numbers that differ by a factor of ten.

This does not make the disengagement reports worthless. It makes them a record of activity rather than a measure of quality. Read as activity, they show something the press releases obscure: which companies were actually operating at scale in a given year, which ones quietly stopped, and how the population of permit holders churned. That is a genuinely useful longitudinal record, and it is available nowhere else.

Why the California disengagement reports do not compare

Sources of non-comparability in the annual disengagement filings
Variable	Standardised?	Effect on the ratio
Testing domain	Not standardised	Operators choose their own routes. Highway, suburban arterial and dense urban miles differ by more than an order of magnitude in difficulty, and the report does not weight for that.
Definition of a disengagement	Operator-defined	The regulation describes a category, not a test procedure. Some operators log every safety-driver intervention; others log only those judged necessary on review.
Fleet composition	Not disclosed per mile	A programme running a few heavily supervised research vehicles reports differently from one running a large fleet on repeatable routes.
Coverage of driverless miles	Partial	Designed for supervised testing. As operators move into genuine driverless service, the most relevant activity moves to other filings.
Jurisdiction	California only	Arizona, Texas, Nevada and Florida host substantial programmes with no equivalent public reporting requirement, so the series is a partial view of any national programme.

How to use the autonomous vehicle data anyway

Three readings of the disengagement reports survive the comparability problem. The first is within-company across years: an operator's own ratio, on a roughly stable test domain, does carry signal about whether its system is improving. The second is population-level: the number of active permit holders, and the total autonomous mileage across all of them, tracks the sector's real activity better than funding announcements do. The third is exit detection — a company whose mileage collapses between filings is usually winding down long before it says so publicly.

None of those readings produce a ranking, which is precisely why they are rarely the ones published. For the deployment picture the filings cannot give, the robotaxi deployment tracker records which operators carry public passengers with nobody in the vehicle. For the research literature on how mixed automated and conventional traffic behaves, the report library holds the two UMTRI papers that address it directly.

Reference

About the disengagement data

What is a disengagement report?

Every company holding an autonomous vehicle testing permit in California files an annual report with the Department of Motor Vehicles stating how many autonomous miles it drove on public roads and how many times the automated system was deactivated — either by the safety driver taking over, or by the system handing control back. The filings are public and have been collected since 2015.

Why is miles per disengagement a misleading metric?

Because the denominator is not standardised. Companies choose where they test, and a mile of empty suburban arterial is not a mile of dense urban intersection. They also define a reportable disengagement differently: some log every safety-driver touch, others only those where the vehicle would otherwise have made an error. A company can improve its ratio by testing somewhere easier or by narrowing its internal definition, without its driving system changing at all.

Do the reports cover driverless operation?

Only partially. The requirement was designed around testing with a safety driver. Once an operator moves to genuine driverless deployment, most of the activity that matters moves into permit categories and collision reports that are filed separately, so the disengagement series becomes progressively less representative of the leading programmes over time.

Where are the original filings published?

The California Department of Motor Vehicles publishes the annual reports on its autonomous vehicles programme pages, alongside the collision reports and the list of permit holders. Those filings are the primary source; anything derived from them, including summaries on this page, should be checked against them before being quoted.