Blog
Respond, don’t Rebut
This blog is written to support applicants to the FLF scheme at the point they receive their reviewer comments. Whilst written for this group, it will have wider relevance to anyone facing reviewer opinions and managing both the emotions of critical review and the process of addressing them. This was previously published on the former FLFDN Director’s personal blog.
This blog is structured around answers to questions from previous FLF applicants and draws on my experiences at FLF sift panels where I have a Roving Panel Member role. As such, this advice is pitched broadly without any disciplinary or field perspective. I’d advice you seek guidance from experts in your area as disciplines have their own cultures and the norm in one field might land very differently in another.
Before I jump into the Q&As, I want to emphasise how important it is to approach this as a RESPONSE. I push back on the word rebuttal in this context – whatever reviews include, they are an opinion provided by someone who has been invited to scrutinise your plans and is doing it as an act of collegiality. You may not agree with everything they say, but you need to respond to it rather than push it away.
Having said this, if the review is abusive, you should report it to the funder. Small teams collate reviews under huge time pressure, so these will slip through the net, but the funder has to be made aware of problematic behaviours and this won’t prejudice your application.)
- Do the panel read the application? Or just the reviews and response?
Introducing members (IMs) read the whole proposal, other panel members (who aren’t introducing your proposal to the panel) may only focus on some aspects along with the reviews and response. But the key people (the IMs) will read it all. Depending on your subject area, panels might be really broad (e.g. often for Arts and Humanities although the more applications that come in from these areas, the panels with a narrower scope will be convened). Therefore, you may have panel members that are quite far away from your expertise, and they will get the disciplinary input from the reviews. This is why it’s so important you fully address any valid points as well as shortcomings in the reviews in the best possible way.
- When reviewers have a lot of thoughts on some criteria and sections of the form, but haven’t commented at all on others, how does the panel handle this?
The IM closest to your field (IM1) will review this and decide whether the lack of comments is because these section of the proposal are fine or because the reviewer hasn’t engaged fully. It might be worth getting one of your supporters to look over any “missed” sections and check they don’t spot anything obvious. Reviewers with disciplinary expertise are likely to feel most confident in commenting about the Vision & Approach section which might dominate their comments. It is likely that IM2 and IM3 (who are usually a little further from your field) will pick up on other aspects such as your leadership journey and how you will support your team. So, reviews may focus on the research, but panels will cover everything.
- How do panels react to very positive comments with average scores? Should I draw attention to the discrepancy?
This is very common and the panel ALWAYS looks at both and are well aware that one person’s 4 is another’s 2 or 6. Don’t obsess about this – focus on the comments because the panel is specifically instructed to look for consistency between these two aspects.
- Should I highlight positive comments from reviewers?
Only if they help you to counter a negative from another reviewer. If you have space, you could have an opening line which summarises your thanks for the reviewers and notes the things that that you are pleased they’ve support.
- What approach should I take with a review which is irrelevant and goes into detail about a different area?
First, take a breath and double check that they have. It’s very easy to mistake a valid query with irrelevant perspectives. Get someone to sense check and if it is out of scope, reiterate what area you are working in. Be positive about their interest, but firmly state what you’re doing and why it’s NOT what they are suggesting. In terms of the precise wording, talk to someone senior in your discipline, who knows what phrasing will be interpreted as firm rather than petulant. Remember that panel members have a lot of experience and they will ignore irrelevant comments when they appear.
- Can you introduce new information in the response?
- Things there wasn’t room for in the proposal
- Things that have happened since submission
You’ll get the chance to do this in the interview, so should only raise in your response IF IT ADDRESSES A COMMENT. For example, if there is a new development that demonstrates your capability that was questioned in the review, put that in. If a review ask for elaboration on x, you can bring that additional information in. With skill you can probably weave in unsolicited good news skilfully if you are careful about positioning it. Don’t forget that you’ll want some headlines for the interview, as one of the questions will be “what’s happened since you put in this application?”
- I’ve mentioned applying for additional funding but a reviewer says I can’t because I’m 100% committed to the FLF. How do I address this?
They are right that your time can’t be costed twice and that the FLF covers your whole salary in the early stages of the award. To respond be clear that you wouldn’t be costed on these other proposals and they would be supporting/advancing the core aims of your FLF. This creates sustainability for your work and your team.
- Should I structure reviewer by reviewer or is there a better way?
This will depend on whether there is common ground between them. If not, then by reviewer might work, but generally it’s better to go with themes. For themed responses, referencing the reviewer and paragraph numbers (although not essential if you don’t have space) can be really helpful for helping the IMs to see how each concern has been addressed – ease of navigation will make IMs much happier!
- A reviewer has queried the time and costs allocated to one of my partners and suggested increasing it. What should I do?
Check with the partner – if they are happy to make the contribution as costed your response can emphasise their commitment to your work includes in-kind support. However, if it emerges that they would need more to fulfil commitments, this will have to be found within your budget. (Don’t panic at this – moving money within the permissible areas of your budget is common in project management, so you can explain that if the work they are involved in grows that resources would be found through budget adaptations.)
- All reviewers have made comments are about a lack of detail which is because of the constraints of the form, so I feel like I’m repeating myself.
The structure of the form may have been a difficult fit for you, but the response is additional space and you should take advantage of it. Don’t repeat again and again– put a general comment at the top about welcoming the opportunity to go into more detail on specific points and save that precious space for this extra information.
- My scores are 4545 and people have said this means I’ve got no chance of progressing. Are they right?
As noted above, it’s also about the comments. (Also, these aren’t bad scores!) Those scores are subjective so don’t worry and approach the response with determination to point out why your research is outstanding and so are you. It would be great to have 6666 but that doesn’t mean you’ll get through – the sift panel will weigh up the evidence in the proposal, the reviews and your response. Additionally, if you get one low outlier, that won’t determine your progression unless they make compelling points that the panel agree with. Make sure you address everything raised and explain clearly why you are disregarding anything that you feel is unfair or doesn’t apply.
- Reviewers are asking for more detail about years 5-7 but this isn’t clear until the work begins. Can I just say “we’ll find out”?
You can, but you could talk about what the best case scenario might be. Use this as a chance to be positive and ambitious about where it might lead and what you’re doing to safeguard and enable this fantastic possibility.
- There’s a concern about how generic the support is for training and development from the host rather than being tailored to my field. How do I address this?
Training programmes will also be generic to some extent, but there’s a lot of additional personalised things you can mention. Mentoring, developing skills, visits or secondments are going to be more closely tailored to your needs and that the coaching and mentoring for the FLFDN will help you to identify what you need as your project progresses. Reiterate that your host school are committed to providing this as it becomes clear what you’ll need – perhaps mention regular reviews so they can respond to emerging needs. Take this seriously – it’s a PERSONAL award with an emphasis on your leadership so your host must be committed to supporting your leadership development.
- Can I leave some comments unaddressed?
No, applications fail to progress if the response has gaps. You must acknowledge even with “this isn’t relevant because XYZ”.
- I don’t need all the available space for my response. Should I add more or keep it short?
If you REALLY don’t need it then a concise and precise response will win you friends on the panel. But do an extra check of the comments with your supporters to make sure you’ve not missed anything because a short response which misses things is worse than one that could be deemed to have run out of space. Also, be mindful of the culture of your discipline – if you are expected to fill the space and it will look odd if you don’t, add more detail.
- My reviews reflect a tension in my field with different “camps”. How do I challenge a reviewer who holds a belief about my field that I don’t share?
Point out that these factions exist (some of the panel may already be aware) and show evidence of the tensions/disagreement in terms of opposing views. Talk about why you are approaching the work in the way you are and why the belief mentioned isn’t held by you. Is it possible to connect with this other camp through your work? Remain neutral though – you have no idea whether one of these camps may be on your panel!
- Can I respond to one reviewer with a comment from another?
You can group these comments to highlight the differences of opinion, but craft your response with your own evidence addressing concerns from the less convinced reviewer.
- The reviewers seems not to have had access to part of the application as they say something is missing that should be there. How do I respond to this?
It’s possible that they’ve missed something so just state the facts – point out where the information is and hopefully you’ve provided what they’ve asked for. Sometimes the systems which present this information to people aren’t easy to navigate so resist the urge to make a negative comment. Just summarise what is in the application. As always, check that you haven’t made assumptions that something they can’t find is there – it’s possible they aren’t satisfied with what you’ve included and you risk disregarding a query.
You are likely to have more questions. Panels are made up of experienced researchers and innovators, so look around your network to see who has similar experience, ideally of the FLF scheme and seek their advice on your response.