The monthly CEO’s report includes a number of “Key Performance Indicators” (KPIs) intended to track various aspects of the transit system. However, the methodology behind some of the KPIs, notably those related to service quality, leaves a lot to be desired. Moreover, information that could track basic issues such as vehicle reliability is not included. This begs the question of whether the indicators exist more as a security blanket (“we have KPIs therefore we are good managers”) than as meaningful management tools, not to mention as reports to the politicians and public.
A telling chart on page 6 of the presentation shows how badly the TTC has drifted from transit industry norms:
The TTC aims to have almost enough vehicles available for service relative to actual needs, and operates with a lower spare ratio than the industry overall. This has two effects.
- When unusual demands for service arise, there is no cushion to roll out extras.
- Vehicles are not maintained often enough to prevent in service breakdowns. This shows up in a mean distance between failures that is very much lower than the industry average.
The situation is actually compounded by an internal measure of service delivery: a garage counts a bus as “entering service” if it makes it across the property line onto the street. Whether the bus runs for an entire day or breaks down a block from the garage, it counts toward service provided. This is complete nonsense, but shows how the construction of a metric can induce behaviour that is counterproductive. Actually keeping the bus in the garage could allow it to be repaired and improve reliability, but that’s not what the garage is measured for.
Moving to a higher spare ratio and more frequent routine maintenance on vehicles is expected to yield better service with fewer in service breakdowns. Late in 2014, the TTC began this shift by slightly increasing spare ratios at each garage, and the MDBF for the bus fleet has risen to 7,000km. This will have to be tracked over a longer time, however, to ensure that the improvement is permanent and can be linked to further increases in spares and maintenance work.
This has a non-trivial cost for the TTC. With a total scheduled service of about 1,500 buses, a 6% increase in spares represents 90 vehicles, or a substantial portion of a typical yearly bus purchase, not to mention a fair amount of garage space.
On the operations side, the TTC aims at roughly 2/3 “punctuality”, defined as a vehicle being withing 3 minutes of scheduled headway. There is some debate about whether they are actually reporting relative to scheduled time, and that for less frequent services, the scheduled time is more important to riders. The problem is compounded by the mechanics of this measurement.
The measurement points do not include the termini of routes, and so the scheme rewards short-turning to preserve something vaguely like the scheduled headway in the middle of the route. It can be argued that there are more riders in the central section, but this is not valid for all routes. Moreover, ragged service on the ends of routes is echoed across the entire line thanks to irregular dispatching and bunching.
Missing from this chart is another important factor: the time of day. During periods when service is less frequent, reliability becomes even more important because scheduled headways are wider, and the time spent waiting for vehicles (including transfers enroute) can substantially add to total trip time.
The TTC proposes to change its measurements to look at departures and arrivals at terminals, and to count the number of trips actually operated versus scheduled service. The intention is to measure and thereby focus on a different view, a different philosophy of what “good service” should look like that is. with luck, better attuned to a rider’s view of the system. The four new metrics are:
- Number of short turns
- On time departures and arrivals at terminals
- Missed trips (a side effect of both short turns and of missing vehicles)
The premise of “on time” measurements is that if the line starts on time from a terminal and aims to be on time at the other end of the line, there is a good chance it will provide better service over the route. This partly addresses a common behaviour seen in every service analysis I have published on this site (and much more unpublished data) showing that service is irregular at the outset of trips even under ideal conditions. However, without attention to service along routes, especially long routes with branches or scheduled turnbacks, quality can deteriorate for many riders.
The effect will also vary depending on the riding profile of a route. In some cases, one end of a line is a peak point (a subway station, say) with most travel oriented to and from it. Other routes have one or more peaks distributed along them (the downtown streetcar lines are good examples). Locations where service reliability will affect many riders will vary over the course of a day, and weekend patterns will not match weekdays. Managing the ends of the line is a start, but attention to mid points is important too.
One issue with reliability is the question of whether scheduled running times are adequate for typical traffic and passenger demand. In some, but not all, cases there are shortfalls, and service cannot stay on time. The inevitable result is short turns and a focus on keeping operators on time (to avoid overtime costs at the end of their shifts). In theory, keeping service on time with short turns should provide reliability at least on part of a route, but in practice late running, short turns, and bunching (some caused by the short turns) are an unending situation that frustrates riders.
If running times are extended, the problem becomes a choice of a reasonable amount of time for typical conditions rather than scheduling for worst case traffic and weather. A related problem is the issue of “recovery time” at terminals for operators to have a break, particularly after long trips. The TTC does not want to make “recovery” a contracted requirement, and yet it includes this provision in many schedules. In practice, however, that time exists more to pad out timings to make schedules work, and the “recovery” time can vanish when conditions change. A simple example of this occurs on branching routes where the round trip time for each branch must be a multiple of the common headway, or of two headways if vehicles always stay on the same branch. On 501 Queen, recovery times have nothing to do with conditions throughout the day, but only exist to make the blended Humber and Long Branch services work (at least on paper).
Finally, there is limit to terminal capacity for vehicles that arrive early and/or have scheduled recovery times.
Experiments on 504 King and 512 St. Clair have included various tactics:
- Extended running times to allow for actual conditions
- Use “run as directed” extras to fill gaps as an alternative to short turns
- Increased supervision to manage on-time operation and minimize short turns
The changes in stats at the point where new schedules and supervisory tactics were implemented are evident in the charts above, but King and St. Clair show how two very different routes can have different results. The 512 St. Clair line is relatively short (7 km) and is entirely on a private right-of-way. Opportunities to “reset” service occur frequently with a one-way trip of about 30 minutes. Generous recovery times have been included in the schedules so that there is little excuse for late departures. By contrast, 504 King is a longer route (12.8 km), runs in mixed traffic through areas where congestion is common, and can be affected by spillover effects from events on parallel roads such as Queen Street or the Gardiner/Lakeshore. Moreover, it has multiple peak points and demand is not conveniently concentrated by location or direction. A one-way trip can take 65-75 minutes.
The contrast shows up in the on time arrival stats where the new schedules have little visible effect on King, although the short turn and missed trip stats both go down reflecting less need to cut trips short to keep operators on time.
On 29 Dufferin, the TTC took a different approach dispatching on headways at terminals in place of schedule changes.
The big improvement here is in short turn stats. By definition, dispatching on headways will provide regular service, but these vehicles will not necessarily be “on time”. Riders don’t care, but the stats look terrible. Missed trip stats can improve with fewer short turns, but dispatching on headways may well result in fewer trips because the available vehicles are being operated to match actual conditions. (If the dispatch headway is 5′ in place of a scheduled 4′ value, the result could actually be better service if the 4′ schedule was not actually practical for on-the-street conditions.)
This is an example of the problems of taking stats out of context. It is important to understand what the goals were for a specific route and time rather than simply making route-to-route comparisons where the management strategies could differ.
On March 29, 2015, the schedules for 29 Dufferin were changed to give extra running time with the following projected effect:
In a few months we will see whether the predicted changes actually materialize.
An important point here is that not all routes require extensive supervision and schedule revisions. The three routes above were chosen because they are busy and they had known problems with service quality. Meanwhile, on a much simpler route, 44 Kipling South, existing service is not too bad.
This is a short route (5.2km) with a one way trip time of 15-18 minutes. Note that its scheduled speed is somewhat higher than the 512 St. Clair car reflecting the fact that this route has much different loading patterns, and is able to run for much of its length in free flowing traffic with few stops. On time departures are easy to achieve, and the relatively low proportion of on time arrivals indicates that the route has plenty of running time.
There are short turns during fall 2014, although their actual numbers are low. This is an example of the problem of using raw counts rather than percentages because we have no way of knowing what proportion of the service is affected. Intriguingly the missed trip percentages are relatively high for such a small, simple route.
All of these stats are presented as weekly averages, and the real story from a rider’s perspective requires drilling down to daily or hourly data, possibly even to location-specific data for special cases. A relatively high on time performance can mask wide variations with very good off-peak and counter-peak service masking poor peak service. Much relatively good service during weekday daytime operations with a high level of supervision can mask poor service at evenings and weekends.
At the very least, the weekly stats need to have high, low and standard deviation values included to flag situations for detailed review and improvement.
In a coming article, I will review the 512 St. Clair route in detail including data from before the right-of-way was built west from Bathurst to Keele and data from the period when the new schedules and supervision came into play.
A big challenge for the TTC is that “good service” should not require continuous, hands-on supervision for the basics such as leaving on time (or on headway) and avoiding bunching. Some of this must happen because “that’s the way we do it” – a phrase rationalizing past practices – becomes a badge of honour for the quality of service every day.