November 1, 2014: Results for third quarter of 2014 have been consolidated into a new table below.
Third Quarter 2014 Update:
The statistics have not changed much from the second quarter. One issue with many routes operating on wide headways (night services and express routes) is that they have consistently low performance values. Such routes should, of course, be measured for on-time performance, not headway adherence, because missed vehicles have a far graver effect on would be riders than on a route that operates every 5 minutes. Express-to-downtown routes (the 140 series) should be measured for on time performance in their catchment areas. Their headway once they are on the express leg of their journey is of no consequence to riders.
Second Quarter 2014 Update:
There is little change in the route performance statistics for the second quarter despite our having emerged from a bitter winter. The change from Q1 to Q2 is less than 10% for most routes with some improving and others falling behind. Those that are beyond the 10% mark can, in some cases, be explained by route-specific issues such as construction, but not all of them.
Two new routes appear for the first time, 172 Cherry and 195 Jane Rocket. It is mildly amusing that the Cherry bus, which must fight its way through construction downtown, manages a 69% reliability score while the Jane express service manages only 58%.
In this quarter, the 58 Malton and 52 Lawrence routes were combined. Their former scores in the mid-50% range have astoundingly improved to 81% on the consolidated route. I will follow this up with the TTC to see what magic they have wrought here.
First Quarter 2014 Update:
The reported reliability stats continue to be dismal. Although it is tempting to say “ah, yes, but Toronto had an appallingly bad winter”, there is a basic problem here: the statistics reported by the TTC didn’t change very much and many routes actually improved relative to the end of 2013.
I will not rehash my critiques of this method of reporting service quality (see the original article below) beyond noting the the TTC’s targets show that irregular service will be the norm — 1 in 3 trips can exceed the target, but service remains acceptable. This means that in a typical day, a rider can expect to encounter at least one “off target” service in their travels.
Finally, a long-standing issue has been the inability to maintain reliable service on the Queen car due to its length and the mixture of Humber and Long Branch services. Although April 2014 is not included in these statistics, the CEO’s report for June 2014 notes an improvement in that month’s streetcar average:
The increase in performance was attributable to the turnback of the 501 Queen route at Humber Loop for the Gardiner bridge work. This shortened the route and promoted a more reliable eastbound service. [Page 10]
The original article from October 24, 2013, follows below.
The TTC has just published its headway reliability results for the third quarter of 2013. These numbers purport to show the percentage of service that operates within 3 minutes, give or take, of the scheduled headway on each route. The goal is that bus service does this 65% of the time and streetcar service 70% of the time.
On a daily basis, these numbers are rolled up to the system level, but this hides wide variations by route and time of day. Weekends are not reported on at all.
The system barely manages to achieve its goal on good days, and has little headroom to absorb events such as bad weather.
To simplify browsing the route-by-route data, I have consolidated the three quarterly reports into one table. The information is listed both by route, and ranked by the reliability index.
[The table originally linked here has been replaced with an updated version at the start of the article.]
There are many problems with these numbers:
- On routes with short headways, it is easy to be within 3 minutes of target. Indeed, it is difficult to get beyond that target, and even a parade of buses or streetcars may count as one “off target” and several (the parade itself) “on target”.
- There is no measure of bunching, nor is there any indication of whether all or only part of the scheduled service actually operated over most or all of a route.
- There is no definition of what part(s) and directions of the route are measured, or how this might skew reported values. Performance at locations beyond common short-turn points may not be reported, or may be masked by data from central parts of a route.
- There is no time-of-day reporting. From service analyses presented on this site, it is clear that across the system, service at evenings and weekends is much less well-managed (assuming it is managed at all).
- On routes with wide headways, on-time operation is more relevant to riders than headway because they must plan journeys based on the schedule. This is particularly important where connections between infrequent services are part of a trip.
The TTC acknowledges that the headway adherence measurements are inadequate, and they are working on “Journey Time Metrics” based on the scheme used in London, UK. This approach looks at typical trips and the time required including access, waiting, in vehicle and transfer times to better reflect service as seen by a rider. For example, a frequent service with well-regulated headways is useless if the buses are full. An advertised headway is meaningless if half of the service is randomly short-turned and wide gaps are a common experience. The effect of a big delay in someone’s trip is much more severe than a short one because this adds to the unpredictability of journey times.
How, exactly, this will be boiled down into representative journeys while still preserving a granular view into system operations will be interesting to see. I believe that a combination of metrics will be needed, and the managerial penchant for a single index to report the behaviour of a large and complex system is dangerous because of what it hides. (I say this also from personal, professional experience in another field.) Without the details, the organizational goal becomes one of “gaming” the system to ensure a lovely column of green tick marks on a scorecard that masks pervasive problems.