The Engineering Problem · Editorial

The engineering problem

You tune against a model. Then reality disagrees. Closing that gap — and knowing where the model can't be trusted — is the job.

model

the gap

reality

The data

Comma.ai openpilot logs. This is what the car's own controllers saw: the same signals openpilot drives on. Production hardware, real roads.

On disk

data/raw/segments/
  PLATFORM/
    device/
      route/
        segment/
          rlog.zst

Four cars

Tesla Model 3 wheelbase 2.875 m mass 2,035 kg

Ford Mustang Mach-E wheelbase 2.984 m mass 2,336 kg

Hyundai Ioniq 5 wheelbase 2.970 m mass 2,084 kg

Ford F-150 Lightning wheelbase 3.70 m mass 3,084 kg

A 3-ton truck and a compact hatch. The baseline model is mass-independent — it predicts identical cornering for both.

The virtual model

A rigid rod of wheelbase L, no tyre, no slip — the car goes exactly where the wheels point. Yaw rate falls straight out of geometry.

ψ̇ = (v / L) · tan(δ)

a_y = v · ψ̇

state x = (x, y, ψ, v, δ) input u = (δ̇, a)

The scope

Input — measured

v, δ

speed · steering angle

Predicted

x, y, ψ

position · heading

Measured v and δ are clamped at every integration step, so the longitudinal channel is an input, not a prediction. What's left is purely lateral — and the model's lies are all lateral, so this isolates exactly the residual we want to measure.

How V0 is built · the file

Inputs

delta_road_rad v_mps a_long_mps2 accel_pedal_pct

Truth

yaw_rate_meas_rads a_lat_meas_mps2

Ford / Hyundai only

Prediction

yaw_rate_pred_rads a_y_pred_mps2 x_m y_m psi_rad

Residual

yaw_rate_resid_rads a_y_resid_mps2

= pred − truth

The truth columns exist in sim.csv — but not in what your model is given at inference.

The task

Yaw-rate RMSErad / s

Instantaneous fidelity — how close the predicted yaw rate is to measured, every sample.

Cross-track-error RMSEmetres

Where the integrated trajectory actually ends up, resampled at uniform distance.

Not redundant: a tiny persistent yaw bias is nearly invisible per-sample but compounds into hundreds of metres of drift.

The start line

V0, scored on 534 held-out segments. Everything from here is measured against them.

0.0163

Yaw-rate RMSE · rad/s

254

Cross-track-error RMSE · m

The 254 m is the compounding-bias problem made concrete — integrate a slightly biased yaw rate over a minute of driving and the trajectory drifts off the map.