Mar 102014
 

18466820_sThis post sets out some advice for academics on how often you should submit research grant applications. The advice I give is not what most people expect to hear from a dean, so I will start by stating what I think is an important principle and contradicting a common idea about what deans think.

The principle is that a good university strategy can only work if it promotes strategies that are good for individuals within the university. So whatever a university strategy requires academics to do in terms of submitting grant applications has to be beneficial to those academics. It follows from this that, contrary to a widely voiced complaint, no sensible university wants academics to waste valuable time writing grant applications that have a very high probability of failure.

I don’t want to dwell here on cases where the probability of failure is high because of inexperience or lack of skill. Last week I argued, on the ResearchFundingToolkit blog, that nobody should start writing a grant application unless they have the skill to write a good description of the research project. I will also be happy to discuss ways in which people can improve their skills if they need to do so in order to pursue a sensible strategy, but not right now. Right now I want to concentrate on explaining what is a sensible strategy for an individual to adopt. In line with the principle I outlined above, I will also argue that university strategies should support individuals and encourage them to adopt good individual strategies.

A sensible grant application strategy has to start by asking whether you yourself will need a new grant around the time that you would get the result of the application. We can consider how you answer that question elsewhere but the important point is that if the answer is no, you shouldn’t write any grant applications at all. On the other hand, if the answer is yes, you need a strategy that will get you a grant quickly. To get a grant quickly you will need to submit several applications in quick succession.

The failure rate makes it necessary to submit several applications to be reasonably certain of securing a grant. Even the best-written grant applications from the strongest applicants have a reasonably high chance of failure – maybe as high as 50%. This means it would be foolish to risk too much on a single application – or to be too disappointed by a single failure. I think that the best strategy is to submit four or five grant applications in quick succession, all based on the same set of ideas. Then, if you get five straight rejections, you can be reasonably sure that it is time to change your approach.

I have seen many departmental research strategies recommend that academic staff should write one grant application every year. This is a very poor strategy for individuals. Writing one grant application per year is a recipe for misery.

It’s not hard to understand why this should be. As I pointed out above, most grant applications get rejected. The decision process takes about 6 months. As I have said many times, grant rejections are utterly demoralising. It takes months to recover. Submitting another grant application within six months of a rejection would be a superhuman effort of will. With rejection rates approaching 90%, a strategy of submitting one grant application every year gives an excellent chance of spending several years alternating between demoralisation over each new rejection and anxiety about the next potential rejection.

Another important point is that, if you only submit one grant application per year, it takes too long to get evidence that you need to change your approach. You cannot tell on the basis of a single rejection that a particular set of ideas is unlikely to get funded. You really need four or five straight rejections. Then you can be reasonably sure. If your strategy is to make annual applications it could take five or 6 years to discover that you need to change your ideas. On the other hand, if you follow the strategy that I recommend, you will know within a year.

So I think the best strategy for an individual is straightforward. You shouldn’t write any grant applications until you need a new grant. As soon as you do need a new grant, you need to write several grant applications very quickly. The best strategy for a university is harder to define but one thing is clear. Your university should have a strategy that supports you to make and implement the decisions you need for your individual strategy.

This article has also been posted at Parker Derrington Ltd. The discussion will be continued in the blog on the Parker Derrington website.

May 202013
 
Boundless Campaign Posters in Toronto

‘Boundless’ campaign posters hang from the lamp posts in the University of Toronto.

I feel I should start this post with an optimism warning: the robust scepticism that is my trademark has been completely dissolved by a cascade of generosity, good humour and potentially priceless information delivered by some of the most professional, friendly and good-humoured colleagues I have ever met. The sensation is almost like a mild hangover after the most astonishingly fabulous party, the CASE fundraising study tour of Canadian universities.

The tour is an annual event, led by Joanna Motion and organised this year by the indefatigable Emma Darwin. Groups of academic leaders and fundraising professionals from Europe visit Canadian universities to learn how they approach fundraising. Canada is a good place to learn about university fundraising because the culture is fairly similar to ours but the practice tends to be about 20 years ahead. We visited McMaster, Wilfrid Laurier and Waterloo Universities, and the University of Toronto Mississauga. Everywhere we went we were given lavish hospitality while fundraising professionals, university leaders and, in some cases, multimillion-dollar donors shared with us stories of past difficulties and present successes. On day one, at McMaster we learned about the motivations of the three main actors, the fundraising professionals, the donors and the academic leaders.

Lorna Somers, vice president of McMaster University Foundation, told us about the transition from her early days at McMaster, in the late 1980s, when academics treated her ‘like someone with advanced leprosy’ and alumni would tell her that universities should look to the government for any money they wanted. Now academics are eager – sometimes too eager – to be involved. McMaster’s annual fundraising target is $21 million and their last campaign raised $470 million in 4 years.

Donors have changed too, as we discovered over lunch, when we met three of McMaster’s most generous donors. All were alumni who had lost touch with the university and had been attracted back, either to study or because they wanted to get involved in the running of the university. For each of them, giving was prompted by the realisation that they had the means to help the university do something exciting and extraordinary. This realisation is not the result of a fevered sales pitch but rather the culmination of a long relationship. Most of McMaster’s top 20 donors started small, with less than $1000 and more than a decade elapsed between their first gift and their major gift.

McMaster was not the only place where we heard how important it is to manage, or steward, relationships with donors. Donors give because they want to make a difference. Once they have given, they need to be told what their gift has achieved. Then they may know that they can achieve more by giving again. Professionals, who take the lead in donor stewardship, involve academic leaders to generate and sustain both the excitement, that may result in a gift, and the satisfaction, that the gift has been put to good use.

Lorna told us that not all academic leaders are good at this. My inference is that we tend to get too hung up on the issue of asking for money, ‘the ask’. Some of us are terrified of the idea, so we treat Lorna and her colleagues like lepers. Others become too eager to ask; they become ‘askaholics’.

Fortunately, Lorna had arranged for John Kelton, Dean of Health Sciences at McMaster, who has raised over $200 million in the last ten years, to talk to us about his relationships with major donors. Significantly, our meeting took place in the ‘floating board room’ which floats in space in the atrium of the Michael G. De Groote School of Medicine, a building funded by a donation of $105 million, the largest single donation to a Canadian University.

John’s account of his work with donors made it clear that listening is at least as important as talking. John listens to donors and remembers what they say. He wants to understand what a prospective donor is interested in, so that he doesn’t waste their time talking to them about projects they will never care about. One of the reasons Lorna likes working with John is that he listens well and is good at crafting a version of what the university wants that will excite a prospective donor: ‘he always has a dream in his pocket’ she says. But he also takes a university-wide perspective and shares prospective donors whose interests are outside his area.

Our day at McMaster ended with a dinner party at Lorna’s house. The house looks ordinary from the outside but it contains the most extraordinary collection of collections of clothes, books, shoes, oil cans, semi-precious stones, fossils, china among others. The list is not endless but it is not short either. I could not possibly do justice to it here but if you Google Lorna you can find a hint of what may be there. And the food was great too!

After our day at McMaster, I felt that there was no more to learn. I was wrong. Although our visits to the other three universities confirmed everything we had learned at McMaster, in each of them we learned something distinctive and new.

Max Blouw, President of Wilfrid Laurier University,  talked to us about the fact that a successful campaign must be rooted in reality. One of the first tasks he had set himself as President was to lead the university to develop a story about itself based on the facts of the present rather than the myths of the past. As a result the Wilfrid Laurier mission ‘Inspiring Lives of Leadership and Purpose’ contains a strong component of business excellence which adds distinctiveness to the slightly bland image that had been based on their origins as Waterloo Lutheran University. A significant component of their current campaign is a $103 million ‘Global Innovation Exchange’ building, which will house their School of Business and Economics and their School of Mathematics.

Waterloo University, just next door to Wilfrid Laurier, has a story of themselves as producers of scientific and technological innovations that fuel a local high-technology economy. They encourage their staff, their students and their alumni to generate wealth, substantial chunks of which get fed back as donations. Waterloo alumnus and inventor of the BlackBerry, Mike Lazaridis, has given hundreds of millions to support research to develop technologies of the future – Nanoscience and Quantum Computing. A Waterloo student donated the million dollars profit he made from selling his first business. Despite these successes, only a small fraction of the alumni who could make substantial donations do so. Many alumni are out of reach; they are disconnected from the university. Jason Coolman, Director of Alumni Affairs, told us about his research on how to use ‘elite’ alumni to engage these disconnected alumni and gradually get them engaged.

UT Mississauga’s President, Deep Saini, has had his car licence plate changed to ‘Boundless’ – truncated by the 8 letter limitation. Colleagues call it the ‘boundlesmobile’.

On our last day, at University of Toronto Mississauga (UTM), Gillian Morrison, Assistant Vice-President, Divisional Relations and Campaigns, from the University’s central Campaigns office, talked us through the stages in developing UT’s extraordinary $2 billion ‘Boundless’ campaign. The biggest in Canadian university history, the campaign is completely coherent with the academic mission, and, as illustrated by UT Mississauga President Deep Saini’s car (see left), has the complete commitment of academic leaders. This shows that smaller universities can have no excuse for lack of coherence in their campaigns.

As my euphoria begins to clear, the serious question I think we should consider is this. Could we emulate what the Canadians have done? The more I think about it the more that I think that we could. Moreover, if we learn from their mistakes rather than repeating them, we could do it faster. I think that an average sized, middle-ranking UK university should be able to get to a point where they are raising £5-£10 million per year from philanthropy within 5-10 years.

Postscript: I’d like to record the immense gratitude of all the tour delegates to Joanna Motion and   Emma Darwin.  Joanna was our guide; she prepared us for each of the very different visits, checking with each of us what we wanted to discover and making sure  that we had the opportunity to discover it. Emma arranged everything, effortlessly resolving the minor crises and making sure that lost delegates and lost suitcases all returned to the fold.
A shorter version of this post is available at The Guardian
May 052013
 

I posted last week on the Guardian HE website under the headline University managers are not malicious, we are misunderstood. The post provoked a lively discussion that made me realise that I want to write more on the topic, so by way of introduction, here’s my latest version of the Guardian HE article.

Although it was over 30 years ago, I still remember the shock of discovering that I was no-longer a student.  In the middle of my first tutorial, it hit me that there was a fundamental difference in outlook between me and my students. I wanted them to understand the complexity and beauty of the brain. They wanted me to tell them how to get a good mark with minimum effort.

Many academics feel that a similar difference in outlook separates them from their senior managers. Academic values, dedication to the development of discipline and the education of students, all are under threat from managers like me. We are drowning them in a cascade of initiatives, processes and performance indicators. We put academics under pressure do things that are manifestly stupid. We want to turn first-rate universities into second-rate businesses. We are incompetent and stupid.

This accusation is wrong. Managers are not malicious. We are not stupid. We are misunderstood. Before I became a manager I thought I knew why. It is fiendishly difficult for a manager to write a message that cannot be read as a some kind of cynical attempt to undermine academic values. Well-meaning but badly-expressed messages are seen as sinister, or stupid.

After a decade in management I know that I was wrong. It’s not fiendishly difficult. It’s impossible. Someone who feels threatened – whether or not the feeling is justified – makes what I call an assumption of unreasonableness. They assume that the message is a threat and they find evidence to support that assumption.

And it is a mistake to think that, because academics are clever, they are less likely to misunderstand a well-intentioned message. The opposite is true. Clever people are better at searching for evidence. And if it is hard to find the evidence, that makes the threat more powerful because it makes it look as if the evidence was concealed. The concealment implies that management intended to deceive as well as to threaten.  Once the assumption of unreasonableness has taken root, sinister interpretations can always drive out more plausible benign ones.

The assumption of unreasonableness has a helpful twin that you can use to protect yourself from perceived threats. Before I discuss it I’d like to despatch a perennial misconception that cropped up in the live chat about grant applications that I participated in a couple of weeks ago. It occurs in several variants but the general idea is that research, particularly winning grants and producing 4*-rated publications counts for everything and someone who delivers on the research front can neglect their teaching, shirk departmental duties and still win promotion.

I do not subscribe to this idea. The strength of our universities depends on the fact that they combine teaching and research. Neglect of teaching duties should put promotion out of the question and it could lead to dismissal. I once gave a senior colleague a formal disciplinary warning for deliberately neglecting to mark a piece of student work. A further disciplinary offence would have led to dismissal. The colleague found another job within a few months.

The fact is that as a senior manager I find that my concern for academic values has increased and my focus has broadened. I spend a lot of time helping academics work out how to succeed, both in teaching and in research, and win promotion. This is the fun part of the job. Occasionally I have to help them drive out the assumption of unreasonableness and replace it with its helpful twin, from which it was separated at birth, the assumption of reasonableness.

I recommend that you try this at at work the next time that you receive an apparently  hostile pronouncement from management. I have also found it helpful for dealing with threatening emails from vindictive colleagues. You have to make the assumption that the message is, in fact, intended to be helpful, that the writer of the message does in fact share your values, but that they have written in haste, expressed themselves clumsily, and maybe made a few mistakes. You must find the evidence, however slim, that supports this assumption.

Then, act as if the assumption of reasonableness were proven. My experience from doing this is that often it is possible to get the assumption of reasonableness to replace its evil twin and to become a self-fulfilling prophecy. This may seem like foolishness, but it really works. It happened to me at work this week, so I want to post about it soon.

An uncomfortable question arises if the assumption of reasonableness doesn’t work. If there really is a fundamental difference in outlook between you and the senior management of your university, then you can be sure that someone is not doing a very good job. The important question to ask is this. Can you be sure it isn’t you?

And please don’t all write to tell me. I have worked it out. When I ask the same question about that tutorial 30 years ago, the answer isn’t as comfortable as I would like!

Jan 262013
 

In 2012, tens of thousands fewer students accepted university places in England that had done in 2011.  It had been widely expected that highly ranked universities would take students from those in the middle of the league tables (the ‘squeezed middle’). This post looks at how the increases and decreases in student intake in 2012 varied with league table position. It also looks at the pattern of increase and decrease in three different groups, the  Russell Group, other pre-1992 Universities and post-1992 Universities.

Although the figures show a tendency for a decline in intake numbers at lower ranked universities, there are huge variations between universities of very similar rank. Across the groups, the greatest average reductions in intake numbers occurred in post-1992 Universities. There was much less decline in Scotland, Wales and Northern Ireland than in England..

The decline and the pattern of variation give a first indication of the way that the changes in fees and in the rules by which university admission operates have affected intakes. They raise interesting questions both for those whose student numbers have increased and for those whose student numbers have decreased.

The UCAS figures on applications and acceptances in 2012  were published on January 18th. Figures for previous years are available on the same web site. A comparison table for 2011 and 2012 is available on the Times Higher website. In English universities there was a decrease of over 26000 student acceptances from 2011 to 2012, with the following differences between groups:-

  • The Russell Group in England kept overall intake constant.
  • Other English pre-1992 universities showed a decline of about 5% in their intake on average.
  • English post-1992 universities showed a decline of nearly 10% in their intake on average.

In Wales, Scotland and Northern Ireland there was very little decline:

  • the Russell Group increased intake by over 1600 students, about 8% of their total intake.
  • the pre 1992 group showed negligible change
  • the post’1992 group showed a decline in intake of about 2000 students, about 8% of their total intake.

This summary ignores huge variations within each group, which are shown in the graphs below. There is a story on the THE website commenting on which institutions gained and lost the most students.

The graph below shows how the increase or decrease in student intake varies with league table position.

Numbers of students gained or lost for Universities in different positions in the Complete University Guide League table

 

The most striking feature of the data is the big variations between universities with very similar league table positions. It would be impossible to predict from its league-table position whether a university increased or decreased its intake. It is true that there is a general tendency for increases to be more common in the high-ranking universities and for decreases to be more prominent at low-ranking universities but there are universities in the top twenty that have decreased intake by hundreds of students and universities in the bottom twenty that have increased by hundreds. Note that this is not a comprehensive chart – it is based on the overlap between the Complete University Guide league table  and the UCAS data published last week.

To get a better sense of the pattern of increases and decreases we can group similar universities together. An easy starting point is to separate the data into three groups, Russell Group, other pre-1992 Universities and post-1992 universities. We get the following table of gains and losses for the different groups.

Change 2012 total Average Increase Percentage increase
Russell Group, England 240 87,620 12 0.3%
Russell Group, Scotland, Wales, N.Ireland 1,633 19,599 408 8.3%
Other Pre ’92 England -3,077 58,647 -147 -5.2%
Other Pre ’92, Scotland, Wales, N.Ireland -172 26,894 -17 -0.6%
Post ’92 England -22,750 213,862 -379 -10.6%
Post ’92, Scotland, Wales, N.Ireland -2,189 28,206 -168 -7.8%
Others -1,181 29,312 -8 -4.0%
Totals -27496 464140 -98 -5.9%

The first column of data, headed ‘Change’, shows the difference between the 2012 and 2011 figures for that group. The next column shows the total number of students admitted in 2012. The next column shows the average change, per institution, and the right-hand column shows the increase in numbers for the group expressed as a percentage.

However, the pattern of gains and losses within the different groups is highly variable, as shown in the set of graphs below:-

UCAS Acceptances in 2012 vs in 2011 for (a) Russell Group Universities, (b) Other Pre-1992 Universities and (c) Post-1992 Universities

UCAS Acceptances in 2012 vs in 2011 for (a) Russell Group Universities, (b) Other Pre-1992 Universities and (c) Post-1992 Universities

Each point represents one university. The 2012 acceptance numbers are on the vertical axis and the 2011 acceptance numbers on the horizontal axis. Red squares represent English universities. Blue squares represent Scottish, Welsh or Northern Irish universities. On the blue line, 2012 is the same as 2011. Points above the line represent universities that  increased intake in 2012. Points below the line represent universities that decreased intake in 2012.

The top left graph (a) shows results for the Russell Group. The red points are clustered evenly either side of the line, confirming that English Russell Group universities maintained their numbers, on average. However, many individual universities lie above or below the line, showing increases and decreases of up to 1000. All the blue points lie above the line, confirming that every one of the four Scottish, Welsh and Northern Irish Russell Group universities gained hundreds of students.

The top, right graph (b) shows a pattern of decreasing intakes for English pre-1992 universities. Almost all the points lie on or below the line. Institutions from other parts of the UK are scattered evenly about the line, with some of them showing increases or decreases of several hundred students.

The bottom graph (c) shows an overall pattern of decrease for all post-1992 Universities. There are big differences between individual universities and overall a slightly greater average decrease for English universities (about 10%) than for those in Wales, Scotland and Northern Ireland (about 8%).

The factors driving the changes in English university admissions have been widely aired.

  • Quotas were reduced both to create a pool of quota places available to low-cost providers and to create a free market of  students with qualifications equivalent to or better than AAB at A-level.
  • It was assumed that an appetite for growth in highly ranked universities would lead them to entice well-qualified students away from middle-ranked universities.
  • Fewer than expected numbers of students achieved qualifications equivalent to AAB.
  • Some tens of thousands of students decided not to complete the application process.
  • We can also expect that a proportion of the accepted students (these figures) will decide not to take up their places.

Before drawing any conclusion, it is important to raise two caveats about the data.

  • The numbers include some overseas students and exclude some home students, so they should be taken as a rough guide to intake numbers.
  • Several universities have stated that they are deliberately reducing numbers because the increased fee provides an incentive and an opportunity to raise both the quality of the educational offer and entry standards.

These caveats suggest that the patterns of increase and decrease are likely to vary again next year and in subsequent years. For me rather than conclusions, I have a number of questions.

  • The numbers of home and EU students accepted in English Universities fell by more than 5% in 2012. Early signs are that applications have fallen further this year. How long will that decline continue?
  • How long will it take universities who lost students this year to win numbers back by emulating the successful approaches of those who gained extra entrants this year?
  • Will competitive pressures drive innovations in the kinds of course and modes of study offered by universities?
  • Will competition between universities drive a recovery in total numbers of entrants.
  • Will the students who declined places this year enter university at some time in the future?
  • The parameters of admissions changed again in 2013, with a further reduction in quota and an extension of the free market down to ABB. Will this allow some of the Universities that lost students this year to regain them? Will it cause new decreases elsewhere?
Jun 292012
 

 

Dux Vist Day Students

Dux Visit Day Students in a Mock Graduation Ceremony at Liverpool University

What would you tell a group of very bright 14-year olds about University?

Last week I had the task of welcoming students and teachers supported by the Department for Education’s  Dux Awards Scheme, which pays for a year-9 student and a teacher from each secondary school to visit a Russell Group university together. I had 30 minutes, so I thought I would try and tell them something useful. I focused on things that I wish I had understood at their age.

I picked four things.

  • They should be confident and ambitious.
  • Our emphasis on research enables universities like Liverpool to educate students differently and, we think, better.
  • University education is active, not passive: the quality of the labs and the library are more important than the amount of contact time.
  • If they want to live up to their potential, they need to make choices that will stretch them, not choices that will bring easy success.

‘Anything I can do, you can do better’ was the title of a slide on which I talked them through my education and career as an example of how not to make choices. The title was prompted by my reflections on the choices I had made. Mostly they were either bad choices, or good choices made for bad reasons. I really do think that if these students use my wisdom to guide their choices, they will do a lot better than I have.

I explained to the students that I had tended to make safe, unambitious choices, partly because I was encouraged to feel insecure by those who advised me and partly because I was looking for easy options. For example, I chose to study medicine because, under constant pressure from my  school to choose a career, I chose a university course that leads straight into a profession. Except that in my case it led straight to my changing courses after 18 months at Oxford.

Oxford was a good choice of university. I’m proud that I made the choice because my school discouraged me: they thought I was overreaching myself. I’m less proud of the fact that I chose Oxford because I had seen a film of Charlie’s aunt on tv and I liked the vision of undergraduate life it portrayed.

I changed course to Psychology and Physiology, which was better for me than medicine, but the reason I made the change was simply to be able to spend an extra year as an undergraduate. I chose a good University (Cambridge), and one of the best labs in the world to do my PhD, but actually I made the choice because my girlfriend couldn’t get a job near any of the other places where I had offers. There is more, but I think you probably get the picture. The students certainly did!

My other points were made with more conventional arguments but not all of them are universally accepted so I shall repeat them here:-

  • The reason that research improves university education is, I think, because the true value of university education lies in the skills that the students develop and not in the knowledge they acquire. Involvement in research at a high level trains a range of useful analytical and creative problem-solving skills.
  • In the fields that I know best, we expect undergraduates to begin by being independent in their search for knowledge. As they develop we expect them to be able to explain the basis of knowledge, to relate knowledge in different spheres and ultimately to synthesise new knowledge.
  • Finally I stressed that they should pick the best University they can and in order to get there and profit from the experience they need to choose the traditionally ‘hard’ A-level subjects. The slides for the talk are downloadable.

 

 

 

 

 

 

 

 Posted by at 12:56 pm
May 092012
 

I am going to write in praise of the Research Excellence Framework (REF), the successor to the Research Assessment Exercise (RAE), which currently determines the distribution of university research funding grants in the UK. I am going to argue that the RAE is good value and good at what it does. I will speculate that its replacement, the Research Excellence Framework will likely be even better.

The main points I want to make are:-

  • The RAE is an extremely efficient way of giving out a large sum of money to support university research in the UK.
  • The RAE appears to be much cheaper than obvious alternatives.
  • The RAE is highly discriminatory but is generally perceived as fair.
  • The bad consequences of the RAE are not inevitable: they are usually consequences of decisions by institutions.

Before I try to develop these arguments I feel an urge, or possibly a duty, to explain how I come to be making them, not least because I have always been openly sceptical about the value and robustness of the RAE and more aware of bad consequences than good.

It all started a couple of months ago when I decided that I fancied writing a blog. Since then I have been avidly following some of the wonderful HE blogs out there and wondering what on earth I could write that would match the quality of what I was reading. I particularly commend the blogs by Dorothy Bishop and Athene Donald, which are both interesting and provocative – in the best sense.  If you are interested in looking at HE blogs you will find a good list of them on Phil  Ward’s blogroll.

Both Athene and Dorothy have made me think and have given me ideas for what I would like to write about in the future. It is to Dorothy that I owe my interest in today’s subject. Let me reiterate that I think Dorothy’s blog is terrific because I am going to disagree with almost everything that she said about the REF in a post in March, in which she described the REF as ‘a monster that sucks time and money from academic institutions’.

My first thought on reading Dorothy’s post was that I should perhaps see if the REF can be defended on the basis of available data on what it gives to Universities and what it costs them. Before I do that, I should declare an interest: my own career has benefited significantly from the invigoration of the UK academic job market caused by the RAE. Without the RAE I would probably still be a lecturer in Physiology in Newcastle. I can’t prove that my mobility is a consequence of the RAE but I can’t remember a job interview in which my likely contribution to the RAE hasn’t figured.

According to data in the public domain, the RAE is an extremely efficient way of giving out a large sum of money to support university research in the UK. The 2008 RAE exercise cost the Higher Education funding Council for England (HEFCE) about £12 million to run. The indirect costs to Universities were estimated at £47 million, according to  a report commissioned by HEFCE.

£60 million sounds like a lot of money but it goes a long way: over the RAE cycle about £10 billion will be allocated to universities using the 2008 RAE results. So the review costs about 0.6% of the funds given out. This is about a tenth of the proportional cost of research council peer review, which was estimated as  about £196 million per year, split about 95:5 between universities and the research councils, to allocate an annual spend of around £3 billion. Not only is research grant peer-review more expensive but a bigger proportion of the cost is borne by the universities. In comparison the RAE looks more like a midget that pumps money into universities and less like a monster that sucks money out.

I don’t want to get hung up on the cost of these processes, because they are not exactly comparable and anyway, the quality of the decision making is extremely important. We don’t have direct measures of the quality of either the RAE scoring process or of Research Council peer review but both processes seem to have earned a high degree of trust.

Trust in the assessment process is crucially important because the RAE delivers very uneven outcomes.

The graph here shows data from the HEFCE website: the distribution of HEFCE Research grant to institutions in England in 2012, based on the outcome of the 2008 RAE. Each datapoint represents the grant to an institution. I have ranked the grants by size and plotted them cumulatively to show how the total grant 0f £1.56 billion is divided between institutions. The increase in the slope of the graph from left to right shows how unequal is the distribution of funds. At the left, the slope is zero because the bottom few institutions get nothing. Yes, nothing. At the right, the increasing separation between the institutions is measured in tens of millions of pounds. The top institution, Oxford University, gets over £130 million, about 8% of the total. The top 12 institutions share half of all the money.

This very unequal distribution is widely accepted as fair because it is based on a robust evaluation process, one that places most weight on the quality of the research outputs, and one that is carried out by nominated representatives of the research communities that are being judged. The money is seen to go to the institutions that deserve it. Any radical simplification of the evaluation process, such as putting it to a vote, rather than basing it on measures of the quality of the research outputs, would risk breaking that trust.

Restricting research grants to a limited number of ‘research intensive’ universities, as has been proposed from time to time would also be counterproductive. It is an important feature of the current system that excellent performance can be recognised and rewarded, wherever it occurs. And importantly, in every RAE the process throws up surprises. Departments in weak institutions achieve excellent results. Formerly excellent departments in which everybody smart has left town, or stopped producing are uncovered. All this contributes to the sense that the process is fair. And the cost of inclusiveness is not high. The 65 institutions in the lower half of the distribution share less than 5% of the money. But they get the money that they earn.

A new feature of the REF is that 20% of the evaluation score will be based on a measure of the social or economic value of research, referred to as impact. This has led to widespread concern about the appropriateness and practicality of such measures, although many, including me, think that they will be very helpful in justifying the spending of public funds on university research.

Dorothy complains that the introduction of impact into the REF has also led to time-consuming meetings in Oxford and to the creation of jobs at University College London for people who will do nothing but help prepare the associated impact statements. I agree that these are diversions of time and money from the mainstream activities of the Universities but change always has a cost and the resources ‘wasted’ in this way are small in relation to the hundreds of millions of pounds that will be brought into each of these institutions by the REF.  And, of course, these wasteful activities are undertaken freely by the institutions concerned. They are not required by the REF. I think that there are both beneficial and damaging responses to the RAE and the REF that I would like to discuss in future blogs.

To finish on a positive note, I think that the long term effect of including impact as part of the routine assessment of research will be immensely positive. I think it will reinforce the view – inside and outside our universities – that we work for the good of society. In short, I think the REF will be a good thing.