The TTC has just published its headway reliability results for the third quarter of 2013. These numbers purport to show the percentage of service that operates within 3 minutes, give or take, of the scheduled headway on each route. The goal is that bus service does this 65% of the time and streetcar service 70% of the time.
On a daily basis, these numbers are rolled up to the system level, but this hides wide variations by route and time of day. Weekends are not reported on at all.
The system barely manages to achieve its goal on good days, and has little headroom to absorb events such as bad weather.
To simplify browsing the route-by-route data, I have consolidated the three quarterly reports into one table. The information is listed both by route, and ranked by the reliability index.
There are many problems with these numbers:
- On routes with short headways, it is easy to be within 3 minutes of target. Indeed, it is difficult to get beyond that target, and even a parade of buses or streetcars may count as one “off target” and several (the parade itself) “on target”.
- There is no measure of bunching, nor is there any indication of whether all or only part of the scheduled service actually operated over most or all of a route.
- There is no definition of what part(s) and directions of the route are measured, or how this might skew reported values. Performance at locations beyond common short-turn points may not be reported, or may be masked by data from central parts of a route.
- There is no time-of-day reporting. From service analyses presented on this site, it is clear that across the system, service at evenings and weekends is much less well-managed (assuming it is managed at all).
- On routes with wide headways, on-time operation is more relevant to riders than headway because they must plan journeys based on the schedule. This is particularly important where connections between infrequent services are part of a trip.
The TTC acknowledges that the headway adherence measurements are inadequate, and they are working on “Journey Time Metrics” based on the scheme used in London, UK. This approach looks at typical trips and the time required including access, waiting, in vehicle and transfer times to better reflect service as seen by a rider. For example, a frequent service with well-regulated headways is useless if the buses are full. An advertised headway is meaningless if half of the service is randomly short-turned and wide gaps are a common experience. The effect of a big delay in someone’s trip is much more severe than a short one because this adds to the unpredictability of journey times.
How, exactly, this will be boiled down into representative journeys while still preserving a granular view into system operations will be interesting to see. I believe that a combination of metrics will be needed, and the managerial penchant for a single index to report the behaviour of a large and complex system is dangerous because of what it hides. (I say this also from personal, professional experience in another field.) Without the details, the organizational goal becomes one of “gaming” the system to ensure a lovely column of green tick marks on a scorecard that mask pervasive problems.