How Unreliable Is My Service? (Updated July 18, 2014)

July 18, 2014: Results for second quarter of 2014 have now been posted on the TTC’s website, and they are consolidated into a new table below.

Route_Performance_Summary_2014Q2

Second Quarter 2014 Update:

There is little change in the route performance statistics for the second quarter despite our having emerged from a bitter winter. The change from Q1 to Q2 is less than 10% for most routes with some improving and others falling behind. Those that are beyond the 10% mark can, in some cases, be explained by route-specific issues such as construction, but not all of them.

Two new routes appear for the first time, 172 Cherry and 195 Jane Rocket. It is mildly amusing that the Cherry bus, which must fight its way through construction downtown, manages a 69% reliability score while the Jane express service manages only 58%.

In this quarter, the 58 Malton and 52 Lawrence routes were combined. Their former scores in the mid-50% range have astoundingly improved to 81% on the consolidated route. I will follow this up with the TTC to see what magic they have wrought here.

First Quarter 2014 Update:

Route_Performance_Summary_2014Q1

The reported reliability stats continue to be dismal. Although it is tempting to say “ah, yes, but Toronto had an appallingly bad winter”, there is a basic problem here: the statistics reported by the TTC didn’t change very much and many routes actually improved relative to the end of 2013.

I will not rehash my critiques of this method of reporting service quality (see the original article below) beyond noting the the TTC’s targets show that irregular service will be the norm — 1 in 3 trips can exceed the target, but service remains acceptable. This means that in a typical day, a rider can expect to encounter at least one “off target” service in their travels.

Finally, a long-standing issue has been the inability to maintain reliable service on the Queen car due to its length and the mixture of Humber and Long Branch services. Although April 2014 is not included in these statistics, the CEO’s report for June 2014 notes an improvement in that month’s streetcar average:

The increase in performance was attributable to the turnback of the 501 Queen route at Humber Loop for the Gardiner bridge work. This shortened the route and promoted a more reliable eastbound service. [Page 10]

The original article from October 24, 2013, follows below.

The TTC has just published its headway reliability results for the third quarter of 2013.  These numbers purport to show the percentage of service that operates within 3 minutes, give or take, of the scheduled headway on each route.  The goal is that bus service does this 65% of the time and streetcar service 70% of the time.

On a daily basis, these numbers are rolled up to the system level, but this hides wide variations by route and time of day.  Weekends are not reported on at all.

The system barely manages to achieve its goal on good days, and has little headroom to absorb events such as bad weather.

To simplify browsing the route-by-route data, I have consolidated the three quarterly reports into one table.  The information is listed both by route, and ranked by the reliability index.

[The table originally linked here has been replaced with an updated version at the start of the article.]

There are many problems with these numbers:

  • On routes with short headways, it is easy to be within 3 minutes of target.  Indeed, it is difficult to get beyond that target, and even a parade of buses or streetcars may count as one “off target” and several (the parade itself) “on target”.
  • There is no measure of bunching, nor is there any indication of whether all or only part of the scheduled service actually operated over most or all of a route.
  • There is no definition of what part(s) and directions of the route are measured, or how this might skew reported values.  Performance at locations beyond common short-turn points may not be reported, or may be masked by data from central parts of a route.
  • There is no time-of-day reporting.  From service analyses presented on this site, it is clear that across the system, service at evenings and weekends is much less well-managed (assuming it is managed at all).
  • On routes with wide headways, on-time operation is more relevant to riders than headway because they must plan journeys based on the schedule.  This is particularly important where connections between infrequent services are part of a trip.

The TTC acknowledges that the headway adherence measurements are inadequate, and they are working on “Journey Time Metrics” based on the scheme used in London, UK.  This approach looks at typical trips and the time required including access, waiting, in vehicle and transfer times to better reflect service as seen by a rider.  For example, a frequent service with well-regulated headways is useless if the buses are full.  An advertised headway is meaningless if half of the service is randomly short-turned and wide gaps are a common experience.  The effect of a big delay in someone’s trip is much more severe than a short one because this adds to the unpredictability of journey times.

How, exactly, this will be boiled down into representative journeys while still preserving a granular view into system operations will be interesting to see.  I believe that a combination of metrics will be needed, and the managerial penchant for a single index to report the behaviour of a large and complex system is dangerous because of what it hides.  (I say this also from personal, professional experience in another field.)  Without the details, the organizational goal becomes one of “gaming” the system to ensure a lovely column of green tick marks on a scorecard that masks pervasive problems.

About Steve

Steve thanks you for reading this article, even if you don't agree with it.
This entry was posted in Service Analysis, Service Cost and Quality, Transit. Bookmark the permalink.
RSS for Comments

57 Responses to How Unreliable Is My Service? (Updated July 18, 2014)

  1. OgtheDim says:

    I take it that despite the continued lack of improvement that Byford’s performance metrics are all ticketee-boo?

    Steve: I am still waiting to see the new Byford-era metrics published. The ones we have now are the same ones the TTC has used for years.

  2. Raymond Kennedy says:

    I don’t know how to describe the recent article (July 17th) in the National Post about Queen streetcar service. Sad, is one conclusion. It features a rush hour ride on the Queen Car westbound from Yonge to Spadina and return with a reporter. Rick Leary Chief Service Officer (new position?) ex MBTA 25 years and more recently YRT. John Morrison “head of streetcar operation” is the way he is identified. Is that his title? And, is this another new position? Never heard of either person.

    In any event after this short ride no doubt everyone has a handle on this service! Leary’s one observation is that sometimes your trip will take 20 minutes and sometimes 25 minutes. You have to allow for that. Very astute observation. I am betting regular riders allow a lot more variance if they want to get anywhere on time. In fact the car they rode was obviously late as a second car was right behind it. There are many more comments about the Queen Car but ALL are in the future, WAY in the future. I’ll bet riders would really like to see something done NOW! One ride from Yonge to Spadina and back won’t do it. Being out there DAILY might, just might do something. Its called _management_ and it requires on the spot observation and _action_ to get things done. Crack heads, kick ass, do whatever is necessary to get peoples attention that the status quo is not satisfactory. Like I said: sad.

    Steve: Leary’s comment about a five minute variation is the sort of out-of-touch remark that tells every reader how little he knows about the Queen car (or other routes) where long gaps are commonplace, and a variation of only 5 minutes in travel time would probably cause riders to fall to their knees in prayers of thanks to the transit gods.

  3. nfitz says:

    Steve:

    Bathurst is just recovering from a long-running water main replacement that blocked the southbound curb lane from last fall through to late spring.

    That means that St. Clair was less reliable than Bathurst! But how can that be the case? This is TTC’s showcase line that suffers from neither the undercapacity that kills Spadina, or the perennial construction of Harbourfront. It should have the best numbers going! It certainly shouldn’t be deteriorating!

  4. J. Routh says:

    I was walking by Queen and Kingston Rd on Saturday and noticed two 501 Humber cars going west in a convoy. Then a third 501 Long Branch pulled out of the loop right behind them. Must be magnets that makes them pull together like that.

    Steve: Maybe an early test of wider headways with longer cars. The TTC talks a good line on service management, but on the street, we all know what really happens. Saturday evenings are particularly bad because of the combined effect of entertainment-related congestion and far fewer supervisory staff “minding the store”.

  5. Malcolm N says:

    Steve said:

    “Without the details, the organizational goal becomes one of “gaming” the system to ensure a lovely column of green tick marks on a scorecard that masks pervasive problems.”

    Steve I would hope that there would be some reasonable attempt to approach this as a service metric for themselves. The on-time notion with a 3 minute variation and a low target achievement rate, however, does sound very much like creating a pat on the own back opportunity. I would hope that they would publish distribution information at the start, centre and end points on a line for each major line, and the number of vehicles not completing the route.

    This should be used highlight short turns, as these are very disruptive to service. This sort of metric should be a focus for the TTC, along with the types of numbers you have produced for the Bathurst bus, as you cannot fix what you cannot or will not see.

    However, I hate to say it, on some routes, the lack of support they receive in terms of budget for vehicles will compound the issues they have in basic line management. They need to be have the loading data able to identify regular surge loads (as you have pointed out in the past), schools, factory shift starts and ends, and the vehicles in service to meet them.

    The political decision to use high average loading standards as targets for fleet size and availability, begs for service disruptions, no load conditions and poor adherence to schedule, even if there was active management. Finer grained data is really required to make these decisions, and the city needs to make sure that it is included in the planned staff requirement and capital spending decisions.

  6. Raymond Kennedy says:

    195 Jane Rocket is a BIG improvement over the 35E Jane Express. When will the TTC introduce other Rocket express service on heavy routes such as 41 Keele?

    Steve: I know they are looking at express routes, but a lot depends on the balance of demand and the scheduled frequency. There is already an express branch on Keele, but obviously you want something more.

  7. L. Wall says:

    Tonight at 9pm (September 16, 2014) I took notice of a NVAS sign on the 506. It said the next vehicle was due in 55 minutes.

    The NextBus data shows 7 CLRV’s on the route that are supposed to be (in theory) providing “frequent service” (scheduled for 7.5 minute headway).

    Instead I saw several 20 minute gaps, a 30 minute gap, and a portion of the route where NextBus says there were no predictions available.

    I pray this was an anomaly with vehicles missing from the data but it’s sad that I’d almost believe it. According to the TTC twitter account there was a stalled streetcar on the route but that was over 90 minutes ago.

    Steve: I passed through Broadview Station just before 11 pm, and the NVAS sign showed immense waits for the King and Dundas cars. The online maps matched the displayed times. As I write this at 11:15pm, the NextBus map shows only 4 cars on the entire King route, 5 on Dundas, 5 on Carlton and 2 on St. Clair. This sort of thing happened once before when the schedules that are fed by TTC to Nextbus were out of sync with actual operations, and Nextbus did not recognize most of the vehicles on the street as having valid run numbers.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>