References

Barabási, A.-L. (2002) Linked, the new science of networks, Perseus Publishing, Cambridge, Mass.
Butterwegge, C. (2004) Globalisierung, Neoliberalismus und (Elite-)Bildung:
Drewe, P. (2006) How to assess urban competitiveness in the ICT age, in D.G. Tremblay & R, Tremblay (eds) La competitivité urbaine á l'ère de la nouvelle économie, enjeux et défis, Presses de l'Université du Québec, Québec: 255-267.
Drewe, P. (2007) Thinking cities or city regions - two paradigms, in P. Jakubowska, A. Kuklinski & P. Zuber (eds) The future of European regions, Ministry of Regional Development, Warsaw: 265-277.
Drewe, P. (2010) The innovative university (unpublished). See also Nieuwsbrief Science Guide, March 2010, number 8.
Drewe, P. & Rosenboom, H. (1993) Limites á la croissance urbaine en Europe, Revue d'Economie Régionale et Urbaine, No 4: 615-631.
Drewe, P. & Rosenboom, H. (1994) Ville et qualité de vie, 'qui vit ou le mieux en France. Paper presented at the Table Ronde 'Penser la ville' Festival International de  Géographie, Saint-Dié.
Pehrke, J. (2007) Rankings gehoeren zu den wirkmaechtigsten Mythen des neoliberalen Alltags:
Van Vught, F. & Ziegele, F., U-Multirank, design and testing the feasibility of a multi-dimensional global university ranking, CHERPA - Network (powerpoint undated).
Zarka, Y. C. (2009) L'idéologie de l'évaluation, la grande imposture, Cités, 37, Presses Universitaires de France, Paris.

Some warnings
What lessons can one draw from our short course on ranking, lessons to be formulated as warnings.

Firstly, don't take rankings for granted. Subject them to secondary analysis, using multivariate techniques to check whether they measure what their producers pretend to measure and to check for the dimensions of the measurement (data matrix). Check for the sensitivity of indicators and weights. Beware of manipulation.

Secondly, don't take rankings too seriously. Don't waste time, energy and money on either embracing favorable rankings or on fighting unfavorable ones. Different degrees of fitness for cities, regions, universities and so on do exist. Identify yours and settle for it.

Finally, don't use rankings for benchmarking purposes. Don't waste your money wanting to play in the Champions League, competing with the top players. According to Bain & Company, over the past years benchmarking has become the most popular management technique replacing strategic planning. One should rather search for strategic visions and strategic projects (as in mixed scanning) instead of playing with rankings and benchmarking.

Ranking, instead of being a useful tool, may just be another example of the so-called bikini theory of statistics: it attracts attention, but hides the essential.

Focus on university ranking
How do the foregoing reflections relate to university  rankings. very much in vogue at present?
Van Vught & Ziegele have proposed to construct a U-Map and a Multi-Rank:
U-Map is a European classification of Higher Education Institutions, designed to map the diversity of European universities
U-Multirank is designed to produce an international ranking of university performance, in dimensions analogous to those used to map diversity

A multivariate analysis could be used to check on the proposed dimensions: education profile, student profile, research involvement, knowledge exchange, international orientation, regional engagement. In a article on the innovative university I have listed three core missions: teaching, research, contributing to innovation plus 'something else'. The latter refers to the role of the university as a transmitter of culture, having wider social and cultural responsibilities or even being a 'finishing school' (Drewe, 2010). How to capture these less tangible aspects?

Still being at the feasibility stage, Van Vught's & Ziegele's project has already met with massive criticism from LERU, the League of European Research Universities (see ). According to this lobby of three Dutch and two English universities, the risk of the project is 'the potential for game playing when reputations for excellence are at stake'. Excellence, which reads competition, is the very essence of ranking. Who wants to be a loser when priority funding, attractiveness for foreign students and researchers or lucrative research contracts with the private sectors are at stake? If excellence entails specialization, one might oppose the underlying philosophy of the survival of the fittest to 'the survival of the least specialized' (Ruffié). Diversity is a different matter. It coulkd lead to the identification of different degree of fitness, albeit this may contradict European policy: some find that the top-down Bologna process embraces the stance that 'One size fits all'.

Ranking as a symptom

Ranking is regarded as a symptom attributed by some authors to neoliberalism. The central theme of ranking is (market) competition. Social Darwinism looms: from Mathew's 'Many are called, but few are chosen' to Spencer's 'survival of the fittest'. For an analysis of the wider societal context see Butterwegge (2004). Ranking shares the societal features of evaluation (Zarka, 2009). These kind of analyses, however, is beyond the scope of the present article.
One has to accept that 'not everybody can play in the Champions League' so to speak. But one also must acknowledge the existence of different degrees of fitness. Take e.g. cities. One can choose between two different paradigms: 'Best cities for business' or 'The sustainable city'. The latter can do without ranking (see Drewe, 2007).
Different degrees of fitness still raise a decisive question beyond neoliberalism: 'How do latecomers make it in a world where only the rich get richer?' (Barabási, 2002).

Use

There are 'winners' (those on top) and 'losers' (those that end up at the bottom). which attract the attention. Middle-of-the-road positions seem to be less exciting. 
Those on top feel proud. They can use the ranking for marketing or (re)branding purposes. They may also decide to go for more. But a top position, too, may hide major weaknesses.
More dramatic is the outcome for losers. They usually tend to attack the ranking exercise through casting doubt on the method applied. But at the back of their mind a feeling of inferiority may loom, prompting investments of various kinds: meant to catch up with the winners who serve as benchmarks. By doing so - more often than not - losers ignore that the situation is dynamic. Others do not cherish the status quo. Hence, at the end of the day, The relative position of all competitors may even remain unchanged.

Rankings depend on two choices:
- the choice of indicators and 
- the choice of weights attached to them

The second choice is vital as one usually measures 'apples and oranges' and the essence of ranking is to produce final, overall scores. If, for example, the regional quality of life is a matter of socio-economic benefits and costs, the final score may well depend on the relative weight of costs and benefits. 

The two choices provide an opportunity for potential manipulation. A sensitivity analysis of alternative sets of indicators and weights may reveal such manipulations. Drewe & Rosenboom (1994) have demonstrated the sensitivity of rankings with regard to the choice of weights.

In 2006, I have attended a conference in Montreal on the copmpetitivess of cities (Drewe, 2006). In 2004, Canadian Business had ranked 'the best places in Canada to set up shop'. Montreal/Laval ended up last: number 41 out of 41 major urban centers. As a joke, I have offered my services to produce another ranking with Montreal/Laval as number one...

Issues of ranking are twofold. They do not only relate to the method applied, but also to the way they are used.

Ranking - useful tool or wrong medicine?

Ranking is extremely popular today. The imagination of ranking producers is without limits. They just rank everything: from cities, regions and nations via artists and intellectuals to schools and universities (Pehrke, 2007). Ranking first became fashionable in the United States (visits to or similar sites  testify to its popularity). But meanwhile ranking has conquered the world. One of the latest hypes are university rankings, for example, pushed by the European Union. Down to earth, one may ask whether ranking is a useful tool or a wrong medicine. Some don'ts or warnings seem in place.

Method

Over the years we have done some research on ranking, focusing on cities and regions in different countries. Cities and regions have, for example, been rated in terms of environmental quality (Germany), quality of life (France) or locational quality for businesses (Germany and the US). See Drewe & Rosenboom (1993). Research in our case meant a secondary analysis  of the existing data, using a multivariate statistical technique. This allows to determine the dimensionality of the indicators, whether they measure what the ranking builders pretend to measure. As most ranking topics are complex, multidimensionality is the rule.