How reliable is reliable enough?
How clean is clean enough?
How safe is safe enough?
To answer these kinds of questions, transit agencies will often compare Key Performance Indicators (KPI's) with peer transit agencies.
There are, however, a variety of pitfalls to be avoided. For example, interagency comparisons will not be valid if the contexts are different. If one agency operates in exclusive rights of way, and another agency operates on the street, is it fair to compare on-time performance of the two systems? Is a subway system performing well just because it compares favorably with buses that operate on congested streets, or should the subway be compared only with other subway systems?
Another pitfall is to cherry-pick a set of peer agencies to make your agency look good or bad, depending on your agenda. This is unfortunately easy to do by just excluding the highest or lowest performers. Peer agencies should be selected dispassionately based on appropriate criteria without considering whether their performance is high or low.
Another pitfall is exclusions. For example, do your on-time performance KPI's exclude scheduled runs that never dispatch due to labor or vehicle shortages? From a customer perspective, if the bus doesn't come then it is certainly late! So make sure missed runs are included in calculations of on-time performance, at least for external audiences. Otherwise, the data won't be credible. Similarly, for KPI's like escalator down-time at a rail or bus station, if the calculation excludes down-time due to planned maintenance, then there's a discrepancy with what the customer is experiencing.
Another question is: what is the appropriate industry to benchmark against? Usually, North American transit agencies benchmark against other North American agencies, but should they also compare themselves to international agencies that provide high quality customer experiences?
And are fellow transit agencies even the best benchmark to use? For example, for rail station cleanliness, should the benchmark be rail stations in other transit systems, or should we instead emulate other large public facilities like airports or malls? If we compare to other transit systems, are we setting the bar too low and accepting less cleanliness for transit riders than airport travelers or mall visitors?
And where does the customer fit into benchmarking? If, for example, analysts establish that your agency complies with industry norms for level of crowdedness on transit vehicles, that doesn't mean those norms meet customer expectations. Customer surveys and focus groups are essential. It is common that transit agencies use a maximum load factor of say 1.3 when planning service levels (load factor essentially means the average number of passengers per seat). That means that as many as 30% of customers will have to stand for at least part of their trip. If the goal though is to design services that compete with private autos or Uber and Lyft, is 1.3 an appropriate standard? How often do people have to stand when riding in an automobile? With transit ridership down, now is a great time to rethink load factor standards.
Unfortunately for transit, transit customer ratings lag behind most other industries. Some of you may know about a standard measure called the Net Promoter Score (NPS) that compares customer likeliness to recommend across a range of services. A recent survey by the providers of Transit App found that transit overall had lower NPS results than 21 of 23 consumer products and services. Transit only surpassed cable TV and Internet Service Providers. To raise the bar for transit, perhaps transit should only benchmark against the few transit agencies with decent NPS scores, or even benchmark to services outside the transit industry, such as department stores, brokerages, online shopping, or auto insurance companies? If transit continues to only benchmark against itself, how will it climb up the Net Promoter chart to become a respected service?
In closing, a key role for transit CX is to take a fresh look at KPI's to sync them with a customer point of view. Absent that, transit will not succeed in meeting the needs of current and potential customers. And if the customer experience is not right, all the marketing in the world won’t get people to ride.