Toronto’s Auditor General Exposes Many Presto Problems

At the TTC Board meeting of October 24, 2019, Toronto’s Auditor General, Beverly Romeo-Beehler presented a detailed report on the problems faced by the TTC with the Presto farecard system.

For clarity: In this article, where text is directly quoted with an indented block and a page citation, these are the words of the Auditor General’s report. Where there is emphasis within a quoted section, this is taken from the original report, not added by me. Otherwise, comments here are my own with, in some cases, paraphrases of the report. Page numbers cited are within the linked PDF of the report, not necessarily the page numbers shown in the document because numbering restarts within it.

This is a long article, but nowhere as long as the 123 page report on which it draws. It begins with an overview and then continues with detailed reviews of various aspects of Presto and of the relationship between Metrolinx and the TTC. The sequence is slightly different from the Auditor General’s report in order to group related sections and comments together.

After the Introduction, the Audit Findings section delves into the detail. There is a lot of detail because there is a lot wrong with Presto. The Auditor General’s report is an indictment of a failed project and bad management primarily from Metrolinx, but the TTC does not go unscathed.

If anything is missing, it is a historical review of how the system came to be in its present state. However, that is beyond the remit of an investigation into whether the TTC is getting what it pays for from Presto, and whether shortcomings affect the revenue the TTC should receive through that system.

TTC management has accepted all of the Auditor General’s findings, and their remarks can be found in Appendix 1 beginning on page 110 of the report pdf.

Summary: TTC accepts all of the Auditor General’s recommendations. Most of the recommendations are consistent with TTC management and actions to date and ongoing efforts to address and resolve the issues as identified in the recommendations.

Metrolinx’ response to the findings are generally positive and acknowledge many shortcomings including missing or broken processes that should be in place by both parties. How well this will be sorted out remains to be seen. Probably the most important part of the letter is the view forward to the next iteration of the Presto system, and the recognition that what we have now can be improved for everyone’s benefit.

As you know, PRESTO is embarking on modernization plans that are enabling us to focus even more on delivering exceptional services. This work will result in new offerings that will remove customer pain points and give them new ways to pay, while offering transit agencies valuable features to drive ridership and fare revenue. Consultations with the transit agencies on new payment forms and other improvements included in the roadmap have been underway throughout the summer, including with TTC, and we look forward to their participation in shaping the future of PRESTO.

We are also preparing to retender our current supplier agreement with a view to opening PRESTO’s ecosystem to more market participants, which we will leverage to improve our overall performance. Recommendations stemming from your audit of the TTC will inform our future approach as we go to market for these services.

As with all reports from the Auditor General, there will be a follow-up in a year’s time to report on the progress achieved on her 34 recommendations.

Introduction

This is “Phase Two” of a detailed study of TTC fare collection that began with a review of fare evasion. I reported on this in February 2019. See Fare Evasion on the TTC: The Auditor General’s Report. That review raised questions about the unreliability of Presto equipment as a source of revenue loss and frustration for riders who could not always pay their fare properly.

A central issue here is the length of time Presto has been in place and the number of unresolved or even unacknowledged issues surrounding the system.

In November 2012, TTC contracted with Metrolinx to integrate and operate the PRESTO fare card on its transit network for 15 years, plus options for renewal. As of the end of 2016, PRESTO could be accepted for fare payment across the entire TTC network.

The Master E-Fare Collection Outsourcing Agreement (Master Agreement) stipulates that Metrolinx manage all PRESTO equipment on TTC’s properties, on board TTC vehicles, and where necessary on the street. This includes the design of the hardware and software, and the installation and maintenance of the machines. In return, Metrolinx is compensated with 5.25 per cent, inclusive of HST, of the gross revenue collected through the PRESTO system on TTC. [pp 33-34]

The Auditor General’s office conducted a field study of Presto equipment, looking at real machines on real buses.

In reviewing the functionality of PRESTO fare equipment, we conducted a bus device audit for two days in June 2019. Over 100 TTC operators representing all seven bus garages drove 168 buses and noted any PRESTO issues during their shift. Of the 330 PRESTO issues noted, nearly 300 (91 per cent) of them were frozen PRESTO card readers.

A frozen PRESTO card reader is when a passenger taps their PRESTO card but the reader is stuck and does not always accept the tap. Not only does this result in revenue loss if the passenger doesn’t tap on another reader successfully, but the device may be captured as “in-service” rather than “out-of-service” in the availability calculation and the availability rate could be overstated for this issue. [p 24]

They also dove into the murky world of contracts between the TTC, Presto and vendors who provide services within the Presto system. The results were not pretty, certainly not with the rosy confidence we often hear from Metrolinx/Presto spokespeople making claims of a robust, near-perfect system. Moreover, there is a wide gap between the services actually provided by Presto and those for which the TTC has contracted and is paying through its service fees.

There is a very long list of findings from the audit, and it reveals a complex web of ownership and responsibilities for provision, operation, monitoring and maintenance of the Presto system. One cannot help feeling that these arrangements grew like an unweeded garden where functions were cobbled together on an ad hoc basis to provide new or extended features. If Metrolinx has any desire to consolidate all of this into a new iteration of Presto (we already have “Presto Next Generation”, and so a new name is really required here), this will not be a simple process. Moreover, as noted by the Auditor General, both Metrolinx and its client transit systems must have a clear set of requirements for any new system rather than taking whatever Metrolinx management thinks they will accept willingly or not.

The appalling status of the entire contract is summarized in one paragraph in which we learn:

PRESTO card readers need to be available greater than 99.99 per cent as per the service level identified in the Master Agreement between TTC and Metrolinx. Given SLAs [Service Level Agreements] have not been set up, what goes into the calculation of the service level has not yet been defined or agreed upon. Metrolinx staff have advised us that “they do not have an agreement with TTC on the device service level commitment calculations, nor the consequences for non-performance”. [p 25]

One might reasonably ask why an SLA is not in place after all these years including a penalty mechanism, but given that it is Metrolinx who benefits from this situation it is hard not to assume that they simply do not want to take the responsibility and potential, publicly identified cost that might go with it.

There is more than a little irony that a report exposing bad project management and a difficult relationship between the TTC and Metrolinx should appear just before City Council was asked to approve a much more complex arrangement for the funding, expansion and operation of the rapid transit system.

The greatest problem between the two parties is the political aversion at the City to “rocking the boat”. The Provincial attitude is that they can more or less act as they please because they have the upper hand on funding and political control. The Liberal government threatened to withdraw Gas Tax funding from the TTC if it did not implement Presto, and the Conservative government was on the verge of stripping responsibility for at least the subway network, if not the entire transit system, from the City. This does not make for a level playing field nor honest, transparent public debates.

Quite bluntly, if Metrolinx were not a government agency with the power to do as it pleases, but instead a private vendor, it would have lost this contract years ago and faced claims for non-performance.

The first recommendation is the need for “Foresight” by both the TTC and Metrolinx. The contract has many unfulfilled requirements, but what does the TTC really want its fare system to look like? What are Metrolinx’ goals for Presto as it evolves? Are the various suppliers of parts of the Presto system delivering what Metrolinx and its clients really need?

Overall, and in our view, there must be a strategic refocussing at the top by both TTC and Metrolinx to tackle what matters most – the shared outcomes of customer experience and maximizing revenue.

For this to work, both parties also need to:

  • Define clear, agreed upon, and formalized outcomes and Service Level Agreement (SLA) targets
  • Seek a win/win for both parties, but acknowledge individual and shared accountabilities and responsibilities in this arrangement – several examples of which are outlined in this report. [p 2]

The second recommendation is the need for “Insight”.

To solve problems you need insight into the root cause. To gain such insight, the right level of information must be analyzed using the right data. Without that, you are solving what you think might be wrong without the evidentiary support to confirm you are addressing the true cause(s) and actual issue(s). [p 2]

One might cynically observe that this statement applies to a lot of what passes for “analysis” and “planning” for transit in Toronto, but that is entirely another article.

The Auditor General noted “fundamental” information gaps:

  • Service level agreements are not in place setting expectations and responsibilities for each party seven years after the contract was signed.
  • Key information that the TTC needs is either encrypted or purged in a short time-frame contrary to the Master Agreement.
  • The current analysis was limited by
    • problems with Presto readers including root cause analysis for frozen machines, problems with the device monitoring tool;
    • accurate monitoring of vending machines on streetcars and regular collection of coins from them to prevent out of service conditions;
    • manual practices for identification of out of service fare gates. [adapted and condensed from pp 2-3]

The third recommendation is the need for “Oversight”. Astoundingly, although the agreement provides for a Joint Executive Committee and an Expert Panel, these either do not exist or have not met for an extended time. One cannot help feeling that the TTC, in the face of an intransigent service provider, has simply given up and treats Presto as “broken as designed”. However, this could also mask resentment of a system forced onto the TTC compounded by little municipal political support to hold a provincial agency publicly to account.

Moreover …

… there needs to be the following to improve oversight of the system as a whole:

  • TTC, PRESTO and all vendors working together as one, sharing information, and diagnosing and solving problems together (e.g. coin collection issue on new streetcar vending machines needs to be resolved together despite all vendors staying within defined responsibilities)
  • Focusing, measuring and monitoring the desired outcomes
  • Controls over PRESTO revenue and assurance provided by PRESTO needs strengthening, including retailer network controls [p 3]

There are 34 recommendations in the report, and I leave it to dedicated readers to peruse the full list.

Looking back to Phase 1 of the study and the ongoing dispute between the TTC and Metrolinx over revenue losses due to Presto, the Auditor General concludes:

We prepared calculations to estimate a range for the overstatement of the PRESTO card reader availability rate and annual revenue loss. Based on the work performed with the information we could obtain, it is our view that TTC’s estimate of $3.4 million in revenue loss for 2018 due to malfunctioning PRESTO fare equipment does not appear to be overstated. TTC’s availability estimates may even be understated given the issues we identified in this report with availability of PRESTO card readers. We are not including revenue loss calculations in this report. It is our view that the information and data gaps identified at this time make it difficult to provide these important numbers with the required level of audit assurance. [p 9]

The two agencies are now going to arbitration over this and other issues, and it will be intriguing to see what effect the Auditor General’s report has on that process. A major problem for quite some time has been that there has been little field work to document how the system is actually behaving or to explore reasons behind the perceived unreliability of equipment. That neither the TTC nor Metrolinx undertook a definitive review says a lot about both organizations, but at least we now have the Audtor General’s work to show the kind of information that is available if only one makes the effort to collect it.

The high level summary of the audit occupies one page [click to enlarge].

Continue reading