Sunday Essay - a bad new ERA for Australia's universities

In Universities, "standards" and diversity I expressed my personal disquiet at the Australian Government's way of measuring research performance in Australia's universities. Then yesterday I finished The beauty of Armidale & UNE with this comment.

Walking back to the Arts building for the paper, I took one last shot back towards the graduation site. No-one can deny its a pretty campus. Pity the Commonwealth Government creates so many difficulties. It really is hard today for anyone who want to run a real university. But that's a story for another post. 

This post extends that thought, looking specifically at ERA, Excellence in Research for Australia. I will explain what ERA  is in a moment. First, I want to set a broader context.

Problems with measurement.

We live in an age where we like to measure things. There is nothing wrong with that. However, we also live in an age were we like to keep things simple and quantifiable. This is also an age that has taken concepts such as "quality", " standards" and "benchmarks" far outside their original framework into every aspect of life.

For a number of years now I have been trying to make the simple point that we get what we measure. But what happens if we measure the wrong things? Let me give you a few examples without, hopefully, boring you too rigid. I am not giving links to my own writing. For those who are interested, I will give you links in later posts.

One of the big concerns in the management of professional services firms lies in finding the best way to maintain or, better, increase profitability. Profitability can be measured in different ways, but is often expressed in terms of profit per equity partner (PEP). To maximise PEP, you have to get the best fee earnings that you can from staff.  To that end, new computer based practice management systems emerged that tracked individual performance: staff were set time and fee based targets that were tracked on a monthly basis.

When I first started advising in this area, it was pretty easy to help a firm increase performance. All you had to do was to do was to introduce proper time recording systems and then focus on getting better results across the production and billing cycle. However, problems then emerged. These included:

• Increased pressure on staff increased problems such as depression and staff loss.
• Scandals about over billing that badly tarnished the reputation of certain firms and sectors leading to increased regulatory controls.
• Unwillingness of staff to become involved in anything that did not directly contribute to the achievement of their performance targets.

Most of these problems were questions of balance. For example, if you want staff to treat clients ethically, then praise those who do and give them rewards. If you want staff to market the firm, then include time spent on marketing in the measures of staff performance. In practice, balance was hard to achieve.

In public policy, another of my areas of professional interest, I have written a fair bit about the way in which the setting of simplistic, measurable, policy outcomes has led to policy failure. I have also suggested that part of the problem here lies in the way that cascading performance measures and agreements within organisations reduce flexibility and lock the organisation into specific approaches that clearly don't work in a broader sense.

Again, there is a balance question. As in professional services, once processes and targets are set, variation becomes very difficult.

What is ERA, Excellence in Research for Australia?

The Australian Government has been greatly concerned about the standard of Australian higher education. It also believes that Australian higher education must be benchmarked against international standards. But what standards, and for what purposes?  ERA is one outcome.

Before going on, Wikipedia has quite a good summary of ERA. The Government web site is here.

The aim of ERA seems to be to increase the international standing of Australia's universities against a single measure, the number of citations in international journals recognised of being of high standard. To manage this, Government and officials have to define fields of study one one side because they need a way of grouping results. On the other side, they have to define just what journals to include, how much weighting to give. To manage all this, they have defined fields of study on one-side, a journal ranking system on the other.

Results of ERA

It already seems clear that ERA is likely to be a monumental disaster. I wrote Universities, "standards" and diversity because I could not quite see how I as an adjunct with my own interests might fit in. My chats with staff members at UNE while I was there confirmed my fears.

Talking to staff, the on-ground effects of ERA can be summarised this way:

1. Government and media treatment of ERA has already made it very important from both a funding and pecking order perspective. If you don't rank high on ERA, then you are in trouble. 
2. ERA favours the big. You have to get to a certain number of publications in any field to be ranked. Then ranking depends upon volume.  Even a very bad large university is likely to rank higher than a small, very good, university.
3. Since money and student numbers are affected by this type of ranking, effort has to be devoted to getting the best ERA ranking you can regardless of its affects on other activities.

Some of the practical effects of can be summarised this way:

1. If you are smaller, you need to group or argue for grouping of articles to get maximum concentration. Those like me or the group that I belong to at UNE - the Heritage Futures Research Centre - are in trouble because as a multidisciplinary group we just don't fit with the standard statistical classifications. I was going to give you a link here to the HFRC, but the entire UNE web site appears to be down!
2. In publishing, you have to focus on higher ranking journals even though your research would better fit in a specialist publication. The Australian approach actually interferes with the global marketplace for ideas because it redirects publication traffic.
3. It makes it very difficult for a university to devote research resources to anything that doesn't fit with the ERA classifications no matter what the community payback.'

Conclusion

As part of my historical research, I have been looking at the contribution of UNE to its students, region, Australia and ideas since it began as a college in 1938. It's really been quite remarkable.

Yet none of that would happened had we had ERA.

I accept that I am UNE biased. However, the same arguments apply to other universities. I am using UNE as an example because it is my university.

When, and I am putting this crudely, are we going to tell the Feds to piss off!