Reputational Risks of Research Grant Lotteries Inhibit Funders
Awarding research grants using lotteries will only lead to “negligible” savings but could expose funders to considerable reputational risk, have admitted those funders who have developed such grants. systems.
While several large European funders are expected to announce experiments with grant lotteries next year, there is growing interest in how partial grant selection randomization works – in which funding requests that exceed a certain amount. quality threshold are chosen at random.
A potential benefit of these modified lotteries, which have been tested in New Zealand, Germany, Austria and Switzerland in recent years, is that they can eliminate the need for lengthy discussions on the merits of shortlisted entries, resulting in payouts. efficiency.
But a Research on Research Institute study, which interviewed the executives of six funders who had piloted grant selection lotteries, found that “there was little certainty about its money-saving properties. and the participants who had implemented the method considered that the time saved was negligible “.
“It doesn’t save a lot of time,” explained one interviewee who said it “could shorten panel meetings because it relieves reviewers [of the need] to assess very marginal or no differences between the top candidates ”, but this was a“ small advantage ”.
One research official claimed that setting up randomization was more expensive because peer reviewers took longer to identify projects to be placed in the lottery rather than simply selecting the best applicants. Evaluators also had the tedious task of identifying projects that were “so bad that it would actually damage the reputation of [our organisation] if they were drawn at random, ”he added.
This reputational risk of funding a terrible project through the new lottery system was one of the biggest concerns of funders who feared the haphazard nature of the selection would expose them to allegations of incompetence, the study continues. .
A research official explained that he was concerned that negative newspaper headlines could give the impression that “our organization is not capable of doing what it is supposed to do.”
“We don’t want the papers to say that the organization is playing around with research funding now,” another interviewee said of this “nightmare” scenario.
James Wilsdon, professor of research policy at the University of Sheffield, who conducted the study with Helen Buckley Woods, associate researcher at Sheffield, said donors feared that lotteries would implicitly recognize “the enormous amount of coincidence “that the selection of grants via them often means involved.
“The fear is that the lotteries will give way to the expert – where the power of funders to allocate money comes from – and admit that there is a tremendous amount of chance in who gets money. ‘money,’ said Professor Wilsdon, adding that the introduction of lotteries was required. an “epistemic humility about claims to excellence within the system” that some funders have found difficult.
For most funders, the ability of lotteries to increase fairness in decision-making by removing the possibility of bias, which can also improve the diversity of chosen researchers, was a major asset; while others felt that riskier and potentially more profitable projects were also more likely to be funded.
“I am convinced that this increases fairness,” explained a research official. “It adapts to whatever random or random element that we know exists in these panel processes. “