Blog
Interdisciplinary Research (IDR) Peer Review
By Sara Shinton, Director, Future Leaders Fellows Development Network
Interdisciplinary research (IDR) runs through the UK’s research and innovation strategy, but close to a century after the term was first used we are still struggling as a sector to evaluate it. This blog summarises the key points from a recent FLFDN event on Interdisciplinary Peer Review and points to some helpful additional resources.
Although our session was focused on IDR Peer Review, we started with scene setting. Professor Barry Smith gave a whistle-stop tour of the policy landscape highlighting the “downward push” from BEIS for “a continued expansion in the conception of what research is and does” and how this underpins the UKRI Strategy. The highlights from the latest Budget show that research and innovation is a Government priority, with interdisciplinary research reflected in much of their language (see Barry’s slides for details).
Much interdisciplinary work is done by researchers with strong disciplinary bases and a clear sense of the value their IDR brings to their home discipline. However, this isn’t always an easy fit for traditional career paths and the impacts on interdisciplinary researchers have been widely reported (one example being the British Academy’s 2016 Crossing Paths project.) The Stern Review was prompted by concerns about the disadvantaging of IDR in REF2014 leading to changes to the 2021 REF with an Interdisciplinary Research Advisory Panel (IDAP), which Barry was a member of, providing expertise to ensure that “that IDR should be neither advantaged nor disadvantaged for assessment in the REF”.
To help promote better practice, REF 2021 included a definition of IDR (although it’s worth noting that IDR shouldn’t be considered to be “one thing” as it’s a term that covers many different approaches and activities). Institutional submissions also included a statement on how they were supporting IDR and you may learn something about your host by reading theirs! (The REF website includes a summary of their approach to IDR and a protocol for assessment.)
Our next speaker, Professor Patrick Haggard shared experience a panel chair and reviewer, giving us insights into the way interdisciplinarity is conveyed by effective proposal writers. He looks for
- Evidence that the PI understands the added value of working with a broad team AND that it brings extra challenges (reflected in the project structure, communication and team cohesion activities)
- Clear statements of added value of an IDR approach, including the value to the disciplines involved and what the partnerships will stimulate
- The use of efficient signposting language to help home in on benefits (“added value” and “synergy” were two examples)
- A deeper than “text book” knowledge of the disciplines in the partnership, providing evidence that the relationships are effective and have been built through genuine dialogue, listening and reframing
- A compelling narrative (whilst being aware that as an IDR reviewer, you mustn’t get carried away by good writing)
- Clear research questions which explain the need for IDR approaches, using schematics effectively to convey interrelationships succinctly
Underpinning much of this is respectful interaction between researchers with strong disciplinary identities. Patrick sees IDR strengthened when researchers act as guides for their collaborators, helping them to understand their expertise, being open to their perspectives and providing their own. A good IDR collaboration involves welcoming the “trespassers” whilst ensuring that there are benefits for all disciplines and partners.
As a reviewer, he warns that it takes more time to evaluate IDR as it will involve wider reading and reference checking. He has to be receptive to ideas outside his area of expertise and to invest in understanding them. He also had advice on the challenge many IDR reviewers face – feeling the limits of their knowledge don’t cover the work they evaluate. Funding bodies should ask reviewers for a statement about this and he is transparent, one example being finding it difficult to judge the degree of novelty in some fields.
Professor Catherine Lyall is an expert in IDR and brought insights from across the sector. Her publications include “Interdisciplinary Research Journeys” (2011, available on open access) and “Being an Interdisciplinary Academic” (2019). She spoke about what we mean by disciplines and how they are characterised and how this creates challenges for IDR as reviewers are often prone to pointing out disciplinary “weaknesses” rather than ID strengths. Added to this is that ID research goals are often different to disciplinary ones, requiring different research design and methods, but then they are subject to evaluation based on disciplinary models.
Catherine sees very little IDR peer review training from funders, but a motivated researcher can find a wealth of resources to help inform their approach. (Volkswagen Stiftung’s Freigest scheme and the Swiss National Science Foundation were applauded for good practice.) The EU-funded SHAPE-ID project includes Guided Pathways to help navigate resources by goal or role. There are lively communities of ID researchers and reviewers who share challenges and good practice, including:
ITD-Alliance – Global Alliance for Inter- and Transdisciplinarity
International Network for the Science of Team Science (INSciTS)
Catherine’s key points were
- Review of IDR can be biased against novelty (when judged from a disciplinary perspective)
- Panels work most effectively with “integration experts” who can bring their own experiences of working across disciplinary boundaries and with different people
- Funders who understand IDR will explain to reviewers what they mean by this term and how they want reviewers to approach it (as seen with the approach for REF 2021)
- There is a HUGE body of work on IDR and it’s frustrating to see a lack of awareness of this leading to so much reinvention. This was a major motivator and driver for the SHAPE-ID project
- The FLF community has a strong IDR base and she urged them to step forward to be evaluators and to look for opportunities to contribute to discussions about research evaluation
- Most institutions will have a centre or institute which promote and foster IDR – look for this in your home institution
Finally, with more opportunities and invitations for peer review expected from funders, participants were urged to speak up for IDR and to ask “how will we ensure we don’t penalise IDR?” and to see themselves as the IDR champion in the room. The links dotted throughout this blog should equip reviewers for this role.