By Paul Monahan, Peer Review Lead, Future Leaders Fellows Development Network.

Paul Monahan: the FLF Development Network Peer Review Lead and Institutional contact for Queen’s University Belfast.  I have been a member of the FLFDN Team from the beginning of the Network, initially as the contact for Queens University’s partnership with the Network.  Over the past year, I have also lead on developing resources on peer review for the Fellows.  As an FLFDN coach, I’ve meet with many Fellows and gained a real insight into their experience of undertaking a fellowship.  Beyond the Network I work as a Learning and Development Manager at Queens Belfast, focusing on researcher development, mentoring and coordinating the Queen’s Fellowship Academy, a core element in the university’s ambitions to attract leading researchers to the University. 

Being a Panel Rover 

During February, UKRI assessed the most recent applications from researchers seeking to gain a Future Leader Fellows Award.  Along with other FLFDN colleagues, I was asked for the first time to participate as a Panel Rover.  This provided a unique insight into the process and a better understanding of how applications are reviewed and assessed, across a number of panel meetings.  I have summarised what I have learnt below and this is provided to Fellows and those supporting Fellows, to inform your understanding of peer review processes. 

Process 

The FLF Sift Panels are moderating panels, with members of the panels having access to the reviewer comments and the applicant’s response to these comments.  During the Sift Panel it is expected that there will be no further reviewing of the application, including from a scientific perspective.  In my experience the panel chairs did strongly emphasis this point if a panel member strayed into potentially re-reviewing and if the chair missed this, the panel rover reported it during our daily check-in with the UKRI team. 

The Sift Panels are one element in the overall FLF peer review process, with its various elements guided by very clear scheme objectives and criteria on how applications will be assessed: 

  • Research & Innovation Excellence 
  • Applicant and their development 
  • Impact and Strategic Relevance 
  • Research & innovation Environment & Cost. 

During the Sift Panels members did look to ensure an equal balance in the application of these criteria. Panel Chairs encouraged and at times insisted that panel members consider each criteria, applying equal weight to each of these factors. 

The Chairs role is key to ensuring fairness and transparency.  A colleague recently described panels as having a ‘personality’, which the chair also influence.  From my limited observations, chairs took time at start of the two days of meetings, to outline their expectations of the panel and to encourage objective and active participation.  Many chairs that I observed have been participating in FLF reviews from the early rounds and have a deep understanding of the purpose of the scheme. 

The panels are fast paced, with literally minutes for the consideration of each application.  However when the panel needs time to consider an individual application in more detail or to reflect on the process, chair’s will facilitate this to ensure the applicant is given sufficient consideration, which further ensures the fairness and transparency of the overall process. 

There are multiple roles on the Sift Panel, many of whom are also involved in any subsequent interviews.  Information on the process and panels etc.is available from UKRI at https://www.ukri.org/opportunity/future-leaders-fellowships-round-8/ . 

Scoring 

Applications are initially scored by the Introducing Members and following the panel discussion, the overall panel arrives at a final score.  Information on the assessment process and panels etc. is available from UKRI at https://www.ukri.org/opportunity/future-leaders-fellowships-round-8/ . At the end of the panel meeting, all the scores are banded into categories A-D, which go onto the tensioning meeting, which checks the consistency of scoring across all the panels and finalises agreement with chairs and rovers on the banding. Based on the banding approx. 160 applications were selected for interview in FLF Round 8. 

As all these meeting require people to review and share their views, it is inevitable that the scoring and banding can fluctuate between the panels but the tensioning meetings and the banding approach really do bring consistency to overall process. 

 

What did I learn about applying for an FLF? 

I only participated in two panels as a panel rover, but this along with observation of the Round 7 panels, has given me a better understanding of how panels work and what is important to members in assessing applications.  The three key points, that I would highlight, which panel members focused were: 

  1. Value Added to applicants 
  2. Maturity of idea and project 
  3. PI Response to reviewers’ comments. 

Value Added 

1.1 Value added of the fellowship for the applicant is a core consideration for the panels.  How will an FLF fit with and boost the applicant’s career trajectory?  This includes the applicant’s career but also the career development of people in your team. The applicant must make the case for being “early career” especially if they have secured an open-ended contract or senior lectureship. 

1.2 Panels will consider if the proposed research and project is fundable by a more standard grant.  So ensure that you talk about how the fellowship and the associated project takes you forward into independence, which would not be possible through a standard grant approach. 

1.3 It is likely that your university’s internal demand management process will address this at the initial stage.  If you are a postdoc a standard grant route may not be available but to those applicants who already have a permanent academic post, it will be considered.  A fellowship is about the person as much as the project, so be very clear how the FLF will enable you to take a sizeable step forward in terms of your career and leadership potential.  This includes demonstrating that you are a ‘thought’ leader in your subject area, how will you during the fellowship move the dial on the thinking in your research area and possibility across the broader research environment? 

1.4 Time since you completed your PhD is not important, so long as the candidate meets the broader eligibility criteria as an early career researcher then it will be reviewed. 

1.5 Also, do not be concerned about having had a career break.  From what I observed the panel chair will highlight that one of scheme objectives is to support researchers at different career stages, including re-entering academia. In making this appraisal, review panels take into account time spent outside the active research / innovation environment, whether through career breaks or flexible working.  

Maturity of Idea and Project  

2.1  Panels will consider the maturity of the proposal, including how advanced the development of your research idea is and how well you have planned your project. This includes the development of partnership arrangements with collaborators so make sure they are explicit about what they are offering and why in their letters of support. If these are recent connections, make sure they set out why they are interested in you and your work. During panel discussions, they will consider if the ideas informing the proposed research and the overall project plans are fully rounded and account for other perspectives and thinking in your field. If you have only been guided by your previous experience and avoided seeking input from others in your area, including those in research support roles, you may create weaknesses in your proposal so it is really important to share your applications with “critical friends”. 

2.2 Ensure you maximise the support available from your university when applying and highlight what support will be available to you and your team when you are successful.  Including detail on host support will enhance the proposal.  Highlight specific support for fellowship holders but also what is available to support the postdocs and students on your team. 

Those applicants who are invited to interview will be asked about how their development plans and how they will engage with the FLF Development Network, https://www.flfdevnet.com/ .  Therefore, applicants should outline their own leadership and people management development plans, not just the technical skills you will develop, this is a core criteria.  For example, you should expect a question at interview about the benefits you would gain from the FLF cohort approach and Development Network. 

PI Response 

3. From my observations, the PI response to reviewer comments was key to successful proposals. It is evident that those applicants who have drafted a precise and considered response are much more likely to be successful.  Take time to draft your response, do not reply in haste and draw upon your entire network across your current university and colleagues in other places to review and input to your response.  As was said in one of the panels I observed, “They missed an open goal by not sufficiently answering the reviewers’ questions”. 

If invited to interview you will face questions that the sift panels have generated for the interview panels, which are informed by the reviewer comments.  So, re-read reviewer comments and your response and show them to other research leaders.  Ask yourself, what else could you have added? What comments might warrant further discussion? 

Best wishes with your applications. 

Additional Resources 

Expert Review overview: https://www.ukri.org/apply-for-funding/guidance-for-future-leaders-fellowships-applicants/expert-review/ (Includes links to the reviewer guidance : https://www.ukri.org/publications/reviewers-guidance-flf/ ) 

The latest guidance for each round will be on a dedicated page on the UKRI website. At the time of writing, this is Round Nine: 

https://www.ukri.org/opportunity/future-leaders-fellowships-round-9/ (scroll down to the bottom of the page for specific documents including UKRI Guidance for Academic-hosted Applicants) 

Business and non-academic applicants apply through the Innovation Funding Service: https://apply-for-innovation-funding.service.gov.uk/competition/1861/overview/eb6d0e1c-24bf-4269-b82f-854cbb21dcf6  

 

FLFDN Resources 

Watch the Participating in FLF Sift Panels workshop recordings 

2024 workshop 

https://www.flfdevnet.com/video_post/participating-in-ukri-future-leader-fellowships-sift-panels-2024/  

2023 workshop 

https://www.flfdevnet.com/video_post/participating-in-ukri-future-leader-fellowships-sift-panels/  

Insights into Peer Review 

https://www.flfdevnet.com/video_post/insight-series-how-do-peer-review-panels-work/  

Preparing for Peer Review Interviews 

https://vimeo.com/924320836?share=copy 

Attend upcoming Peer Review focused workshops 

Understanding Interdisciplinary Peer Review from a UKRI perspective (On-line) 19th July 1000-1130 

https://www.flfdevnet.com/course_event/understanding-interdisciplinary-peer-review-from-a-ukri-perspective/ 

Coping with Peer Review processes through Coaching: a positive psychology approach (In person London) 6th June 2024 1000-1300 

https://www.flfdevnet.com/course_event/coping-with-peer-review-processes-through-coaching-a-positive-psychology-approach/