Does a five-star rating say it all? Rita Samiolo ranks the ranks that pervade modern living
Almost every aspect of our existence, from the mundane details of our shopping to life-defining issues like education, healthcare and the strength of our democracies seems subject to a relentless urge to rate and rank. Our experiences and attainments in everything from being consumers to being citizens are increasingly turned into scores to be fed into indices and rankings that are meant to help us choose – and be chosen.
One’s life chances – whether an Uber driver, a university student, or a mortgage applicant – increasingly depend on how we and the organisations we belong to score in a variety of ratings and rankings. Why is this happening?
Rankings embody the growing aspiration to govern through competition, even in domains where competition has long been absent and where knowledge bases other than economics – such as law – might have prevailed in the past.
This is well illustrated by the growth in the use of rankings in Corporate Social Responsibility (CSR), where they are deployed as forms of soft regulation to entice competition around social goals such as human rights, mining conditions, sustainability, or access to medicine. All these issues, traditionally seen as market failures, are increasingly addressed through quasi-market mechanisms like rankings.
The language of performance and incentives is here supplementing or replacing that of compliance and sanctions. Companies are expected to care about CSR rankings and engage in a race to the top because of the reputational incentives rankings can trigger. The same race, in turn, is assumed to catalyse innovation and generate improvements in performance.
So competition has become the pivotal point for all the various planks of neoliberal economic thinking.1 Competition is expected to generate knowledge in the guise of innovations and solutions which other forms of social organisation, such as old-fashioned bureaucracies, are no longer trusted to produce. And this knowledge – that only competition is seen to be able to deliver – has become equated with, or reduced to, information.
Philip Mirowski and Edward Nik-Khah, in their book, The Knowledge We Have Lost in Information, lay out how the language of information economics has progressively spread from academia to policy-making and into popular culture. Knowledge has become identified with information processing and markets are cast as the ultimate information processors.2
Rankings resonate with these ideas. They consist of manufactured competitive spaces that are expected to generate information value in the guise of, sometimes immaterial, differences in rank. This information serves the alleged decision needs of various, not always well specified, users – including the ranked themselves. Rankings seek to induce competition and turn the improvements that competition allegedly generates into easy-to-process bits of information which others can then consume thereby keeping the game on.
“Rankings seldom live up to the abstract ideals of competition and information value they embody.”
But rankings seldom live up to the abstract ideals of competition and information value they embody because:
- first they are highly contrived;
- second they can be self serving;
- third they have unintended consequences; and
- fourth they fail to cultivate innovation.
The competitiveness in rankings is not as lively and dynamic as much economic theory would have it. This emerged, for example, from research by Afshin Mehrpouya and I into rankings in CSR. We found frequently that companies find it hard to move towards greater social responsibility by means of a ranking. They are often reluctant to supply the relevant data to the ranker. Some companies will seek to reap the reputational benefits of being a top-ranking company, but others will be less interested and less equipped to play the game. Companies operating in local markets or with idiosyncratic business models that are harder to match to the ranking criteria are particularly unmotivated to play the ranking game.
To engage companies in the ranking process – especially those at the bottom – rankers often need to create sub-rankings for areas of performance where the reluctant companies tend to do well. That hands the backmarkers a positive message to send to their stakeholders which creates a tailor-made incentive to remain in the game.
So rankers are akin to market designers by defining the attributes on which companies are judged and which inform the choices of their stakeholders. The contrived nature of the competitive processes that rankings induce and mediate is often effaced, so whether stakeholders believe in a ranking or not, they can seldom see how it was produced.
Information generated by rankings tends to be biased towards ensuring that the game remains on. For example, when two organisations are very similar and hard to differentiate, rankers often move the goalposts.
They refine their scoring criteria so as to identify subtler variations in performance, even when these may be marginal and not very informative. The rankers have to proceed as if such differences were material, so that information value can be substantiated as a difference in rank. Here the difference in rank represents an amplification of the underlying difference in performance. This exposes that primary mission of a ranking is to maintain a competition, or the impression thereof, rather than to provide an objective measurement of performance.
Research has shown that differences in rank, even where they correspond to marginal differences in performance, can have profound and unanticipated outcomes. The Law School rankings studied by Wendy Espeland and Michael Sauder provide a case in point.
Once two almost identical law schools were ranked based on immaterial differences, the lower-ranked school saw applications and funding decrease. As applications and funding were used as performance measures in the ranking exercise, the position of the lower-ranked school progressively deteriorated, making the difference between the two schools more material over time. The widening gap was not the result of an actual change in performance, but a consequence of the ranking itself. So the ranking produced the reality it set out to monitor passively.3
Rankings can bring on behaviour that hardly lives up to the ideal whereby competition generates innovation. They are notably short-term exercises – typically every one or two years – so they invite short-term adjustments in behaviour. These often amount to little more than gaming.
This has been well documented in the case of university rankings, in which easy fixes such as investing in student facilities or marketing have replaced investment in long-term knowledge creation and other social missions that are much harder to assess. Such short-term fixes are hard to avoid, because doing nothing would probably lead to a drop in measured performance relative to rival universities, regardless of whether actual performance had changed in any substantive way.
Ranking and rankers are born with the best intentions. They try to generate competition and change in areas where individuals and organisations have perhaps become too complacent or unaccountable. But such is our belief in the link between competition and improvement that when rankings fail to generate the change we hoped for, we are more likely to try and fix the ranking as opposed to finding other means of improving things.
“When rankings fail to generate the change we hoped for we are more likely to try and fix the ranking.”
Rankings are ostensibly there to guide us in a world we are arguably less and less able to navigate and experience first-hand. Hence, we seek to orient ourselves based on someone else’s judgment (whether the ranker is the expert analyst, the algorithm or the crowd). While our appetite for rankings grows, we must remain alert to the fact that rankings can hardly be true representations of the world we seek to navigate, and that they change that world in ways that are often unimagined.
The comfort of having rankings at our disposal may well distract us from finding things out by other means, in ways that entail more than putting people and organisations on a numerical scale which tends to trigger in us semi-automatic responses, potentially making us dumber and more conformist.
1. Davies, Will. (2014). The Limits of Neoliberalism: Authority, Sovereignty & The Logic of Competition. London: SAGE.
2. Mirowski, P. & Nik-Khah, E. (2017). The knowledge we have lost in information. New York, NY: Oxford University Press.
3. Espeland, W., & Sauder, M. (2007). Rankings and Reactivity: How Public Measures Recreate Social Worlds. American Journal of Sociology, 113(1), 1-40.