Dr Sara Shinton
Photo by Green Chameleon on Unsplash
This blog summarises some of the content from our Resilience, Response & Recycling session run by Sara Shinton, FLF Director and John Morison, Professor of Jurisprudence at Queen’s University Belfast. It’s aimed at anyone who has received a negative peer review response and wants some advice on how to construct an effective response.
The slides and handouts from the session are linked to below. I’ve written them to be reasonably self-explanatory so hope they help you to identify personal resilience strategies and to help you decide what is the best response to any proposal rejections. This blog will focus on the discussions we had in the workshop with John Morison, in response to four questions about responding to peer review. We were mostly talking about the grant processes, but there are parallels with publications too.
- What makes a convincing response?
- What pitfalls have you seen people fall into?
- How do YOU write an effective response?
- What are YOUR responses to reviews?
John started by saying that he’d applied for many grants and received many rejections. He echoed the early emphasis on resilience, saying that moving on from sadness and anger are vital, but not easy.
He started by reassuring our attendees about the fairness and quality of panel decisions. When he first sat on a panel he thought he had over prepared by reading everything but quickly realise that everyone else was as prepared and familiar with the proposals they were deciding on. He pointed out that panels often cover a wide range of topics, going right to the edges of each individual’s comfort zone, so they rely on the reports from the expert reviewers and the applicant response to these.
He recalled occasions where he’d seen disciplinary squabbles and what he called discipline “sectarianism” evident in reviews and that this was dismissed by the panels he had sat on. He also explained that the panel always recognises the effort each applicant has made and they are determined to safeguard the process so they are treated fairly.
What makes a convincing response?
John is impressed by responses that build on the application, but also go beyond this in their responses. They don’t take on reviewers in combative terms, but where good feedback has found a weakness will give ground graciously and admit that they haven’t presented enough evidence. Good responses are written with a confidence about their research plans and their ability to deliver on their ambitions, reassuring that they are working on the scale they have set out, have access to the resources they will need and quickly clear up any technical issues. John commented that in his field (Law) the reviewers typically question the “doability” of a programme of work rather than having issues with theoretical frameworks. In short, your response must reassure the panel that you can do what you have set out in your proposal.
What pitfalls have you seen people fall into?
John has seen problems for applicants who have indicated their proposal is interdisciplinary, when they largely sit in a single discipline with expertise and track record in that area. Their proposal will be sent to experts in the other fields and they will often identify omissions and assumptions that weaken it in their view. So he advises applicants to think carefully about who will review and to be careful about positioning work as IDR unless they are confident it is convincing to these other disciplines.
He explained that as a panel member, you draw up a checklist of the reviewer comments and review the response to ensure all have been addressed. This is a fundamental and simple piece of advice for applicants to follow. He also encourages you to state what seems obvious! Your panel is very unlikely to include people in your specific area of research and even if they are, many panels are under clear instruction NOT to re-review (FLFs who observe the sift panels in July will see this policed by chairs and conveners if it happens). So put everything in, even if you feel it’s obvious and evident in the proposal.
We also discussed situations where we had seen applicants blind-sided by an unfair review which had distracted them from comments from more reasonable reviewers. If the response only address critical challenges, but overlooks points of clarification, the panel cannot confirm the reviewers’ concerns are met and usually the proposal won’t progress.
How do YOU write an effective response?
John reflected on times where he wished he had requested an unfair review be struck out by the funder. It’s always worth asking for it to be removed before your proposal reaches the panel if you can demonstrate it isn’t a fair or valid critique, but get advice from a mentor or close colleague to help judge objectively if this is the case.
Assuming all reviews are fair, John starts by ensuring that the originality of the proposal was clear to the reviewers. He had some experience of funders who were nervous of very novel approaches, and explained that sometimes you need to prepare the ground in the year or so before you plan to apply by raising the importance of the question you want to address through media engagement.
Although the panels he has sat on do not consider track record as a criteria, so as not to disadvantage earlier career applicants, he admitted that when a proposal isn’t convincing, that sometimes you may look to see if the researcher has done something similar previously which might explain why they have taken the “doability” for granted. Again, if the reviewers question this, set out your evidence against their concerns.
What are YOUR responses to reviews and decisions?
Finally, I asked John how he felt about the reviews and funding decisions he gets. It helps him to remember that the process is a competition and not everyone can win. His response after a decision is influenced by whether his work is classified as fundable or not fundable. He’s been on panels where they’ve hoped that a particular proposal will be above the funding threshold, but seen them lose out to results from other panels. If your research is above the funding threshold then take heart and think about how to strengthen it. He warned against submitting to another funder without a fundamental reworking in line with the second funders strategy and approach – funders are used to see each other’s proposals appearing with only cosmetic changes and they rarely land well.
Finally, John encouraged our attendees to really listen to people in their field when they talk about your work. What do THEY find interesting? What are THEY not convinced by? What do THEY suggest you might do differently? These external perspectives give you an insight into the reviewer mindset, so make sure you engage with their reactions.
A huge thank you to John for sharing his experiences, including some difficult ones with such honesty. He is likely to be typical of other senior researchers around you – don’t be afraid to ask your local leaders about how they respond to review and funding decisions.
Please click to access the handouts from the session. And click here to download the session slides. Click here to see University of Edinburgh Resilience guide it’s one of many available. (Sadly the IOP one seems to have left their website.)
