Ben Spurr in the Toronto Star published an article on September 30 about the mis-reporting of vehicle reliability for the fleet of Bombardier Flexity streetcars.
In brief, the TTC reports defects for the new cars on a different basis than for the old ones (the CLRVs and the recently retired ALRVs), and this has two effects:
- The reliability of the old cars looks worse by comparison to the new ones, and this supports the argument that the old cars should be retired as soon as possible.
- The new cars have recently crested the performance specification from the Bombardier contract, but this is based on the way the failure rate is calculated.
The September 2019 CEO’s Report contains Mean Distance Between Failure (MDBF) charts for both types of streetcars still in active service. August 2019 saw the new fleet’s reliability go about the 35,000 km MDBF, and CEO Rick Leary reported at the TTC Board meeting of September 24 that the current number was running above 50,000 km.
By contrast, the CLRVs have failed roughly every 4,000 to 6,000 km for much of the past three years with problems more common during the cold months. Note that the scales of the charts below are not the same.
However, according to Spurr’s article, the basis of calculation is different for the two fleets. In the case of the new cars, only failures chargeable against Bombardier’s contracted reliability level are counted while for the old cars any failure counts. This makes a big difference when one considers how many of the in service failures were not included in the calculation for the new fleet.
Spurr writes:
The vehicle contract the TTC and Bombardier signed in 2009 set a MDBF target of 35,000 km. The cars were supposed to reach that figure by the time the 60th vehicle was delivered. That car arrived in January 2018, but the new fleet failed to hit the target then or in subsequent months.
That changed this summer. As Bombardier edged closer to completing its delivery of the 204-car fleet, and the TTC weighed the option of placing an order for additional streetcars with the company, the publicly reported reliability figures shot up.
They showed the cars had an MDBF of 36,500 km in July, and 51,500 km in August, the best the fleet has recorded since the early days of the order. CEO Leary cited that most recent figure at last Tuesday’s meeting as evidence the cars are “performing exceptionally well.”
However, over the same period the unpublished reliability figures didn’t improve. The “legacy” numbers showed an MDBF of just 16,400 km in August, which while much better than the early months of the year, was virtually unchanged from the mark set in May.
The unpublished “legacy” figures are consistently significantly worse than those the contractual numbers.
He goes on to write:
Internal TTC documents reviewed by the Star show that in [August] the new streetcars experienced dozens of delays related to faulty brakes, malfunctioning doors, broken HVAC units, and short circuit warnings. The agency tabulated 43 significant delays during that period, but only 15 were deemed Bombardier’s responsibility and included in the version of the stats that are made public.
That lower delay figure led to the contractual number of the cars running more than 51,500 km without a failure.
Readers can judge for themselves the type of delay that is omitted from the TTC’s reliability numbers from the following table which is compiled from TTC delay reports.
There are 56 items in the list and several patterns are immediately obvious:
- Problems with the power collection system are common including pantograph failures, lost trolley poles or defective shoes, and dewirements snagging poles and/or damaging the overhead.
- Brake problems
- Mobility ramp problems
- Failures early in the morning on cars that are probably just entering service
Many of the problems have nothing to do with Bombardier reliability stats and are not included in the calculation included in the CEO’s Report. If they were, the numbers would not look anywhere near as good.
Something that is evident in reported reliability stats is that there can be large variations in MDBF values from month to month. The TTC does not make huge changes in the mileage operated by its fleet each month, and so the large swings must be due to a relatively low number of incidents. For example, if there were typically 100 incidents per month and this swings up or down by 10%, then the MDBF would not change much. However, if there were typically only 20 incidents per month, a small change in the month-to-month numbers would produce a big swing in the MDBF. This is evident in the Flexity reliability values and in those cited for the subway fleet, notably the newer TR trains on Lines 1 and 4.
Even if all types of failure were counted, the service delay it causes must be five minutes or more. This is a standard adopted from the NOVA group of rapid transit operators and really is more appropriate for rapid transit lines.
An important distinction is that vehicles that run in trains have the capability of “getting home” even if one unit is disabled under most circumstances, and reliability stats for this type of operation will be higher than for single vehicles on a streetcar system. Also, rapid transit lines operate at higher average speeds, and failures that are affected more by hours of service than by mileage are spread over a larger distance operated. This is quite evident in TTC subway stats where the MDBF is much higher than for streetcars.
By contrast, it is difficult to imagine how a bus breakdown can cause a significant service delay except in comparatively rare circumstances, and the five minute delay screen for a chargeable delay makes no sense for the bus fleet.
The question of just how reliable various vehicle types might be is part of a larger issue with the selective, and possibly misleading reporting of statistics by TTC management.
Delays to service, especially on the subway, are caused not just by equipment failures, but by a raft of other subsystems and problems such as signals, track, power supply, fires, passenger assistance alarms and track level incidents. The TTC tracks the various types of delay, but reports on them only rarely in public. This means that sources of service delay that might be under the TTC’s control are not tracked in a report that is routinely seen by the TTC Board, nor is there any tracking of the effects of preventative maintenance or capital works to reduce this type of delay. One obvious example is the new Automatic Train Control system which is now operating on about half of Line 1 YUS, but we know nothing of service reliability on that section, Vaughan to St. Patrick, compared to the old signals still in use from St. Patrick to Finch.
Bus reliability is reported in the aggregate for a fleet that ranges in age from brand new to over twelve years old. The TTC used to keep buses for at least 18 years, but now chooses to replace rather than rebuild old vehicles. Retiring a large tranche of 12-20 year old buses in recent years has had three effects:
- The average age of the fleet is now quite low, and it will continue to drop. Half of the fleet is less than five years old, but as the “bulge” of new buses ages, the fleet reliability will fall.
- With many new buses coming on stream, the TTC can keep old buses in service and maintain a high ratio of spares to service requirements. The situation is very different for the streetcar fleet where with the retirement of old cars, the fleet is too small to provide service on all routes with an adequate number of spares for maintenance.
- The large order of buses soaked up the then-available funding for transit infrastructure as it was the only way Toronto could spend its allocation within the short timeframe dictated by the federal government.
For reasons best known to the TTC, the chart above is clipped at 20,000 km rather than showing the actual variation, and this has been the case since early 2018. It is unclear whether the actual numbers are rising or falling over the past two years. Moreover, the values average the reliability for the entire fleet rather than showing subsets such as diesel and hybrid buses, or buses of varying ages or manufacture. This type of breakdown is vital in understanding fleet planning, not to mention tracking the benefits (or not) of technology changes such as the move to an all-electric fleet which is only just beginning.
The TTC fleet of buses is much larger than its requirement for service. In total, as of the September 2019 vehicle list (taken from the Scheduled Service Summary, last page) shows a total of 2,076 vehicles as compared to a peak service requirement of 1,626 (p. 63 of the same document). This is a generous spare factor of over 27%, or one spare bus for every four in service. It is easy to get very good performance from your fleet with such a high ratio, but this also means that, in effect, the TTC operates one garage worth of spares for every four garages worth of regular service. This is far higher than the target spare ratio for rail vehicles.
In a separate post, I will turn to the question of service reliability, scheduling and the way in which service quality is presented by management to the TTC Board. This is another area where there has been a lot of work to make the numbers “look good” but with detrimental effects on the system.
Questions for the TTC:
I have posed a series of questions to the TTC and await answers from them. This article will be updated when they reply.
1. Has the Flexity reliability number always been quoted on the basis of failures chargeable to Bombardier, or was there a change in the methodology somewhere along the way? To put it another way, was there a change in what counted as a failure that created an artificial improvement in the reported numbers?
2. What is the situation with subway delays and MDBF numbers? Are all failures counted (at least those producing a 5 minute or greater delay) or only those considered to be the manufacturer’s fault? Is the calculation done the same way for the TR and T1 fleets?
3. The NOVA metric which the TTC uses is based on the idea of a failure that causes a delay to service. This only makes sense in the case of rail modes where a car/train failure can block the line. For buses, only a rare and well-positioned failure could actually block service. How is a chargeable failure calculated for the bus fleet?
4. Are numbers available for subsets of the bus fleet (e.g. all buses from the same order, age, technology) so that reliability figures can be compared as they have been with rail modes?
5. The CEO’s Report includes only stats for delays caused by vehicle faults, not from other sources such as infrastructure failure. Why is this info not also tracked in the report so that the effects are clear on a proportionate basis? In particular, there is no tracking of signal failures on various parts of the subway with older and newer technologies.
Summary:
TTC management should report vehicle reliability numbers on a consistent basis for all types of vehicles.
The calculation of service interruption rates should reflect what riders experience, not simply numbers to establish contract performance for suppliers or to artificially enhance the reported performance of some vehicle types.
The reliability statistics for the bus fleet should be broken down by major vehicle groups (manufacturer, propulsion technology, age) to allow meaningful comparisons and to ensure tracking of maintenance/reliability as parts of the fleet age.
The very large spare ratio for the bus fleet should be reviewed to determine whether this size of fleet is actually required, or if more service could be operated if only the TTC would budget for the cost of its operation.
Delays caused by infrastructure issues and other interruptions should be tracked and reported so that their effect on service quality can be seen in comparison to vehicle related problems.
Steve Munro 