Skip to main content
Download PDF
Download ePub

The academic impact of paediatric research agendas: a descriptive analysis

Research Involvement and Engagement volume 10, Article number: 97 (2024) Cite this article

Abstract

Background

Increasingly, researchers are involving children and young people in designing paediatric research agendas, but as far as we were able to determine, only one report exists on the academic impact of such an agenda. In our opinion, the importance of insight into the impact of research agendas designed together with children and young people cannot be overstated. The first aim of our study was therefore to develop a method to describe the academic impact of paediatric research agendas. Our second aim was to describe the academic impact of research agendas developed by involving children and young people.

Methods

We based our method on aspects of the Research Impact Framework developed by Kuruvilla and colleagues and the Payback Framework developed by Donovan and Hanney. We named it Descriptive Academic Impact Analysis of Paediatric Research Agendas, consisting of five steps: [1] Identification of paediatric research agendas, [2] Citation analysis, [3] Impact analysis, [4] Author assessment, and [5] Classification of the ease of determining traceability.

Results

We included 31 paediatric research agendas that were designed by involving children and young people. These agendas were cited 517 times, ranging from 0 to 71 citations. A total of 131 new studies (25%) were published, ranging from 0 to 23 per paediatric research agenda, based on at least one of the research priorities from the agenda. Sixty studies (46%) were developed by at least one of the first, second, or last authors of the paediatric research agenda on which the studies were based. Based on their accessibility and the ease with which we could identify the studies as being agenda-based, we categorised 44 studies (34%) as easy, 62 studies (47%) as medium, and 25 studies (19%) as difficult to identify.

Conclusion

This study reports on the development of a method to describe the academic impact of paediatric research agendas and it offers insight into the impact of 31 such agendas. We recommend that our results be used as a guide for designing future paediatric research agendas, especially by including ways of tracing the academic impact of new studies concerning the agendas’ research priorities.

Plain English Summary

Increasingly, researchers are involving children and young people in designing paediatric research agendas. However, few researchers have described the impact of these agendas on the research undertaken. We strongly believe that it is important to know how such agendas affect research, what their impact is. One of the reasons paediatric research agendas are being designed is to create a clear overview of what the research questions are that need to be investigated - if this question is left unanswered, why bother designing the agendas at all? Therefore, we developed a 5-step tool to identify these agendas and to describe their impact. We tested our tool on 31 paediatric research agendas that were designed together with children and young people. These agendas were mentioned 517 times, 131 new studies were based on these agendas, and 60 studies were performed by the same authors who had designed the agendas. Of the new studies, we found 44 that were easy to identify, 62 that were fairly easy, and 25 that were difficult to identify as being based on paediatric research agendas. We hope that our results will serve as a useful guide for future researchers who aim to involve children and young people in designing research agendas. Especially, if ways are included to trace the impact of new studies in relation to the most important questions stated in the original research agendas.

Peer Review reports

Background

The aim of designing research agendas

Research agendas list research questions addressing knowledge gaps that require further investigation. They serve as a guide for scientists and funders to coordinate and focus research on areas deemed most relevant and impactful. In the past, they were designed primarily by researchers. Currently, the expertise of paediatric patients, their parents, or carers is recognised increasingly as a critical component in paediatric research [1]. In 1995, Chalmers stated that increased involvement of patients and the public in designing research agendas would likely result in a more open-minded approach about which research questions are worth addressing [2]. Later, he found a mismatch between the research questions considered as important by patients and the public and those addressed by researchers – a mismatch that resulted in research waste [3]. Increasingly, funding agencies value and researchers, journal editors, and policymakers demand that waste in research be reduced [3,4,5,6]. To address mismatches and reduce research waste, Chalmers and colleagues developed the James Lind Alliance (JLA) Research Priority Setting Partnerships (PSPs) [7]. The PSPs ensure that research questions of patients, carers, and healthcare professionals are prioritised.

Involving children in designing paediatric research agendas

Unfortunately, children and young people (CYP) are still rarely involved in designing research agendas on paediatric topics, so-called paediatric research agendas (PRAs). A systematic review published in 2017 showed that CYP were involved in only four PRAs [8]. Additionally, our recently published review showed that CYP are now involved in 22 additional PRAs since 2017 [9]. One of the reasons CYP are infrequently involved is because there is still a tendency to underestimate the value of their voices rooted in a belief that they are not competent to speak about health issues related to their bodies [10]. Partnering with CYP to design PRAs is crucial for understanding what is important to them. Research teams sometimes spend years developing agendas in partnership with patients, parents, and carers, but we found that little is known about whether the research priorities are elaborated upon [9, 11]. The question whether the agenda had an impact on what research is undertaken remains [12]; this is what we refer to as the academic impact.

Describing the academic impact of paediatric research agendas

Describing the academic impact of PRAs holds significant importance. First, the academic impact should be described to evaluate whether the aim of reducing research waste is met. Second, describing the academic impact can help identify areas where progress has been made and where additional research efforts are needed. Third, when evaluating the academic impact, it can be determined whether funding agencies use the priorities. Describing the academic impact of the PRAs in which CYP are involved is particularly crucial. Pediatric research agendas that involve CYP require a significant investment of both time and resources [13,14,15]. Researchers need to be flexible when scheduling research meetings with CYP because they have multiple commitments, such as school and sports, during the typical working hours of researchers. This flexibility may involve conducting research activities during evenings, weekends, or school holidays when CYP are more readily available. Furthermore, researchers must overcome ethical considerations, such as power dynamics and facilitating environments, which helps to make CYP feel secure and confident to express themselves freely. Moreover, researchers should respect the authenticity of CYP’s voices [15]. Finally, one of the participants of the JLA PSP on Juvenile Idiopathic Arthritis considered the PSP a waste of time and money should the project end with the publication of the top 10 research priorities [16] and no attention was paid to whether the priorities were being used. Regrettably, little or no attention is given to reporting the academic impact of PRAs [9].

Example of a research team describing the academic impact of their paediatric research agenda

To the best of our knowledge, Geldof and colleagues stand alone in evaluating the academic impact of their PRA [12]. Staley and colleagues performed a qualitative evaluation of what happens after JLA PSPs, however, they did not conduct a systematic search or evaluation of whether the research priorities included in research agendas are elaborated on [17]. Geldof and colleagues evaluated the impact of their agenda six years after its initiation and three years after publishing the PRA [12]. Most of the studies based on their PRA were pharmaceutical-driven studies that focused on prioritising the development and validation of new medical treatments (71%). The authors concluded that the extent to which the current research landscape adequately represents the viewpoint of patients is debatable [12].

Methods for identifying the research impact of general studies and describing the academic impact of paediatric research agendas

Identifying the academic impact of PRAs differs from identifying the research impact of other studies. Identifying the research impact of general research aims to demonstrate “the contribution that excellent research makes to society,” as defined by the Economic and Social Research Council. [18]. Nevertheless, the impact of such research is difficult to identify, partly because impacts originating from consecutive activities may accumulate in the longer term. Given this, it becomes difficult, sometimes even impossible, to ascertain which activity ultimately contributes to impact [19]. Several approaches, such as the Payback Framework developed by Donovan and Hanney [20] and the Research Impact Framework (RIF) developed by Kuruvilla and colleagues, [21] have proven robust and useful for describing research impact. The RIF is divided into four broad areas: research-related impact, policy impacts, service impacts and societal impacts. The checklist was developed for academics interested in describing and monitoring the impact of their research. The Payback Framework was originally developed to examine the impact of health services research, but has been adapted to assess the impact of research in other areas such as the social sciences [14]. The Payback Framework consists of five categories: [1] Knowledge, [2] Benefits to future research and research use, [3] Benefits from informing policy and product development, [4] Health and health sector benefits, and [5] Broader economic benefits. Both approaches consist of almost identical categories. Each approach can be used for different circumstances for which researchers may seek to describe impact [19]. The limitation of these approaches is that they are not specifically developed to describe the impact of PRAs on what research is undertaken after publishing the agenda.

The aim of our study was two-fold. First, to devise a reliable method for describing the academic impact of PRAs. Second, to describe the academic impact of PRAs designed together with CYP. We chose to focus only on describing the academic impact of the PRA in which CYP had been involved because we sought to improve the quality of CYP involvement in designing PRAs. Therefore, we believe that describing the academic impact of these agendas is of utmost importance.

Methods

Design

We developed a method to describe the academic impact of PRAs based on the research-related impact of the RIF (Table 1) and the first two categories of the Payback Framework: Knowledge, and Benefits for Future Research and Research Use (Table 2). The categories of the RIF and the definitions of the Payback Framework that we used are highlighted in both tables. We used these categories because they resemble the academic impact of the PRAs we aimed to evaluate. The other three areas of the RIF and the Payback Framework are related to impacts other than academic impact (e.g., policy, services, and societal impact for the RIF, and policy, health, and economic impact for the Payback Framework). The authors of the RIF state that the themes can be adjusted, including removal, addition, grouping, or modification, to align with the research being described and relevant to assessment criteria [21]. In consultation with a medical information specialist and a methodologist, we portrayed our method as a Descriptive Academic Impact Analysis of Paediatric Research Agendas (DAIAPRA). The following section describes the development of the method.

Table 1 The broad areas and descriptive categories of the research impact framework
Full size table
Table 2 Example of the multidimensional categorisation of paybacks of the Payback Framework
Full size table

Developing the descriptive academic impact analysis of paediatric research agendas

In preparation for creating the impact tool, we defined the academic impact of PRAs using three identifiable factors: [1] The number of citations referencing the agenda, [2] The number of new studies based on the priorities, and [3] The variation in authorship between the original PRA and the subsequent studies. We opted for this approach because the data on citations, new studies, and research teams were readily accessible. Furthermore, we added an evaluative factor to the impact tool, which considered the ease of determining whether a study was based on one of the PRAs. It is important to note that impact encompasses various elements, but our study concentrated solely on those that could be quantified.

We defined the different steps of the impact tool, starting with Step 1: Identifying the PRAs. Next, we determined the data sources and metrics that would serve as the basis for describing impact. We based our PRA impact tool on three components of the section Research-Related Impacts: Publications and Papers, Type of Problem/Knowledge Addressed, and Research Networks and User Involvement (Table 1) [21]. Additionally, our tool drew upon two components of the Payback Framework: Journal Articles and Better Targeting of Future Research (Table 2).

In Step 2, we linked the component Publications and Papers of the RIF and the component Journal Articles of the Payback Framework to the number of citations generated by the PRA. Then, in Step 3, we linked the RIF component Type of Problem/Knowledge Addressed and the component Better Targeting of Future Research of the Payback Framework to new, PRA-based studies. Finally, in Step 4, we linked the components Research Networks and User Involvement to the difference in authorship between the PRA and the new studies. To make the impact analysis tool readily accessible, we included only publicly available, easily assessable, metrics or metrics. To determine whether a study should address the priority of the PRA, we included Step 5. We based Steps 1, 3, and 5 on a more subjective evaluation; hence, these steps should be performed independently by at least two people. We based Steps 2 and 4 on objective variables that could only be interpreted in one way. To use the method efficiently, we recommend Steps 3 and 5 to be performed simultaneously (Fig. 1). The steps of the DAIAPRA are explained in more detail in the section below.

Fig. 1
figure 1

Descriptive academic impact analysis of paediatric research agendas

Full size image

Step 1. Identification of paediatric research agendas

The research team, in partnership with an information librarian, developed the literature search strategy. The strategy utilised Medical Subject Headings and keywords for ‘children’, ‘priority setting partnerships’, and ‘paediatric research agenda’. The search terms from each category were combined using the “OR” operator, and the three categories were linked using the “AND” operator. The search was conducted on MEDLINE, EBSCOhost, Web of Science and Google Scholar. We utilised both forward and backwards citation chasing to ensure that we did not miss any important PRAs. This was done by checking the reference lists of the included studies. Furthermore, the James Lind Alliance page, which lists all JLA Priority Setting Partnerships, was reviewed to identify additional Priority Research Areas that were not captured by our search strategy. The resulting articles were then uploaded to the Rayyan screening tool, developed by Qatar Computing Research Institute (Doha, Qatar), and duplicate entries were eliminated. Several inclusion criteria were applicable in the process of identifying the research agendas (Table 3). The above described search strategy was utilized in our recently published review [9]. To include more pediatric research agenda in this study we added the PRAs identified by Odgers and colleagues in which CYP were included, and we repeated the same search after publication of our review.

Table 3 Inclusion criteria for step 1
Full size table

Step 2. Citation analysis

We uploaded the identified PRAs to Scopus, SciVal, and Altmetric. Scopus is an expertly curated abstract and citation base. It provides access to reliable data, metrics, and analytical tools. We extracted data on citations from the database. A medical information specialist from the University of Groningen helped us download the desired information from all the studies into a Microsoft Excel file format.

Step 3. Impact analysis

Next, we screened the studies that cited one of the PRAs and examined the context in which the PRA was cited. Two researchers (LP and SB) independently screened the citations and examined whether the PRA was referred to because the study addressed one of the priorities. When disagreements arose, the researchers engaged in discussion until they reached a consensus. The inclusion criteria for Step 3 can be found in Table 4.

Table 4 Inclusion criterion for step 3
Full size table

Step 4. Author assessment

We compared the authors of the studies included in Step 3 to the authors of the PRA on which the studies were based. We examined whether the first, second, or last author of the PRA was involved in the new studies that were based on a specific PRA.

Step 5. Classification of ease of tracing

Finally, we classified the studies that were included during the impact analysis into three categories based on the ease of tracing whether a study addresses a research priority of the PRA. We distinguished three categories for ease of tracing: easy - the research priority is explicitly stated in the publication, medium - the research priority is not explicitly stated but we could infer it from the text and difficult - the research priority is not stated and could not be inferred from the text.

Results

Step 1. Identification of paediatric research agendas

We included 31 PRAs in which CYP were involved [13, 14, 22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50]. Twenty-two PRAs were included because we had identified them in our recently published review [9], four were identified by Odgers and colleagues [8], and five were identified after the publication of our review. At the time of this study, the newest PRA was published in November 2022 and the oldest in June 2010. The themes of the included PRAs are shown in Table 5. Little variation was found in the methods used to develop the PRAs. The JLA method was most frequently used (n = 22, workshops (n = 2), focus groups (n = 2), Research Prioritisation by Affected Communities (RPAC) method (n = 1), online survey (n = 1), or a combination of the methods mentioned (n = 3).

Table 5 Themes of the included paediatric research agendas
Full size table

Step 2. Citation analysis

The 31 PRAs were cited 517 times, ranging from 0 to 71 citations per PRA, with a mean of 17 citations per PRA (Supplemental Material 1). The agenda of Batchelor and colleagues received the highest number of citations. Four PRAs have not yet been cited [22,23,24,25]. Figure 2 presents an overview of the citation count for each PRA.

Fig. 2
figure 2

Citations per paediatric research agenda

Full size image

Step 3. Impact analysis

Cumulatively, the 31 agendas were cited 517 times and 131 new studies (25%) were published based on at least one of the research priorities from the PRAs, ranging from 0 to 23 new studies per PRA, with a mean of 4 new studies per PRA. Hollis and colleagues’ PRA attracted most of the new studies [26]. Eight PRAs, however, did not yield new studies [13, 22,23,24,25, 27,28,29] as we show in Fig. 3.

Fig. 3
figure 3

New studies per paediatric research agenda

Full size image

Step 4. Author assessment

Sixty studies (46%) were developed by at least one of the first, second, or last authors of the PRA on which the study was based. Seventy-one studies (54%) were developed by other researchers who did not author the PRA (Fig. 4 and Supplemental Material 2). It is apparent from Fig. 3 that some research teams, such as Baldacchino [30], Lam [31], Medlow [32], Birnie [14], Peeks [33] Aldiss [34], Parsons [35], Layton [36] and Batchelor [37], developed most of the new studies themselves, based on their own PRAs. For Peeks and colleagues, 10 of the 12 new publications included the original author of the agenda. Batchelor and colleagues’ agenda resulted in 18 new studies, 16 of which included the original author of the agenda. The authors developed most of the new studies themselves (89%). The agendas of authors who had not elaborate on the research priorities themselves, as was the case for eight of the agendas, resulted in the publication of an average of two new studies. The PRA developed by Hollis and colleagues was most successful and resulted in 23 new studies; only one of which was published by these authors themselves (6%).

Fig. 4
figure 4

Paediatric research agendas authored by the same authors versus other authors

Full size image

Step 5. Classification of ease of tracing

Out of the 131 studies analysed (Supplementary Material 2), we could classify 44 studies (34%) as easy to trace, meaning that the article directly quoted the research priority from the agenda that they were addressing. Sixty-two studies (47%) we classified as medium, indicating that even though the research priority was not directly stated in the article, it was clear to us that the priority focused on was based on the aim and research question of the study. The remaining 25 studies (19%) we classified as difficult, meaning that although in the publication the authors claimed that following up on research is deemed as most important by the agendas, it was not clear to us which of the research priorities the studies focused on.

Discussion

To the best of our knowledge, this is the first study to evaluate the academic impact of multiple PRAs. To achieve this, we developed the DAIAPRA, a five-step tool. The first step identifies the PRAs, followed by three steps that evaluate the academic impact of PRAs. The last step classifies the ease of tracing whether a study addresses a research priority of the PRA. Using this tool, we found that the citations ranged from 0 to 71 per PRA. New studies based on a PRA ranged from 0 to 23. Furthermore, 46% of the new studies were developed by at least one of the first, second, or last authors of the PRA on which the study was based. Finally, only 34% of the new studies explicitly stated the research priority the study focused on, indicating that in these cases it was easy to trace it.

We found that the number of new studies based on a PRA varied between 0 and 23. A factor that might have influenced this wide range is that we included PRAs from 2010 up to and including 2022. Perhaps older PRAs already had more impact than newer ones. New agendas still need time to create impact. Another possible influence is that new agendas were published during the COVID-19 pandemic. Raynaud and colleagues showed a dramatic increase in COVID-19 publications and a substantial decrease in non-COVID-19 research during that time [51]. Another element that might play a significant role was whether the agenda received advance funding for elaborating on the research priorities. If that was the case, researchers could start elaborating on the research priorities immediately, instead of first finding appropriate research funders. Moreover, some funding programmes set their priorities for research and then advertise for research teams to conduct the research [52], which might have resulted in certain agendas being studied more frequently. Another aspect that caught our attention is that eight of the ten PRAs leading to the most new studies were developed in collaboration with the JLA, suggesting that partnerships with established organizations like the JLA can greatly amplify the impact and reach of PRAs. Another factor that could have played a role is that some researchers were unaware of, and thus paid no attention to, the dissemination and implementation of the PRAs [53]. We interviewed researchers and CYP, who had designed a PRA together, about the academic impact of their PRAs. Authors of the PRA could provide valuable information and examples of impact of their own work, which would otherwise have been unavailable to us. That is why we interviewed researchers and CYP, who had designed a PRA together, about the academic impact of their PRAs. We found that researchers were hardly aware of new studies based on their PRAs [11]. This lack of awareness is easily addressed by emphasising the importance of disseminating and implementing the PRAs. The awareness of and emphasis on the implementation of PRAs might enhance their academic impact. The JLA guidebook was updated in 2021, and Chaps. 9, 10, and 11 deal with the dissemination and publication of the research agenda, prioritising the research funders and long-term follow-up [52]. Concentrating on the phase following the PRAs’ design might already create the awareness that researchers need to take responsibility for encouraging the research and funding community to address the research priorities.

Another strong argument in favour of prioritising the implementation of the research agenda, is to ensure continuous and transparent communication with the CYP involved. Keeping CYP updated regarding the progress of research priorities is essential. It shows them that their input is valued and has contributed to meaningful changes [54]. Mawn and colleagues found that researchers can be criticised for failing to engage or update CYP as research progresses [55]. The result of doing nothing is that CYP may lose their trust. They may get the impression that what concerns them is unimportant, or that it may not be as important or valued by others in a position to fund research [53].

Interestingly, almost half of the new studies based on the PRAs were developed by the first, second, or last author of the PRAs. Researchers who design a PRA can use the agenda as a roadmap for their work, helping them to identify research questions and develop studies that are likely to contribute to the broader goals of their field [56]. Our study opens the door to discussion about whether the academic impact of the PRAs is achieved when a substantial portion of the new studies that are based on the PRAs, are authored by the same researchers as those who developed the agendas. The primary aim of a PRA is to change the broader context of research; it can be questioned whether this aim is achieved when nearly half of the new studies are published by the same researchers. We believe that an important distinction should be made when evaluating the academic impact of PRAs. The expected academic impact of a PRA depends on whether it is designed by an entire research field as opposed to designed by a specific research team. When all key researchers are involved in the design of a PRA, it is inherent that they are the ones who elaborate on the priorities together with their research teams. This approach is particularly fitting for a research agenda designed within a niche. However, when a research agenda is designed for a broad subject such as diabetes, it is practically impossible to include all key stakeholders. Therefore, one might expect that the research priorities of that agenda are elaborated by researchers other than those who designed the agenda. To date, it is impossible to determine whether the PRA was designed by an entire research field or by a specific research team. Consequently, placing the academic impact of a research agenda in context becomes more challenging.

If researchers, who were not involved in designing a PRA, continue to address research questions that they consider important instead of focusing on priorities of the agenda, we question whether the design of a PRA has the intended impact of changing the broader context of research.

Evaluation of the DAIAPRA

We developed the DAIAPRA because no reliable method was available that described the academic impact of PRAs. Our method focuses on quantifiable aspects of impact, such as citations, new studies based on the PRA, and the difference in authorship between the PRA and the new studies. We acknowledge that the method does not consider all potential factors that may contribute to the impact of a PRA, such as conference presentations on the PRAs, or receiving funding for the priorities. It should be noted that the method is still in its infancy. Additional metrics could be incorporated to provide a more complete understanding of academic impact.

Limitations of this evaluation

While we managed to evaluate the academic impact of 31 PRAs, our study has several limitations. The first issue was that the academic impact of the PRAs could not be compared one-on-one because they were published in different years and involved different research areas. Furthermore, it is important to recognise that our study offers an initial perspective on the academic impact of PRAs. With this first evaluation, we included quantifiable aspects of impact only. The qualitative forms (such as: improved collaboration, influence on policy and practice or increasing public understanding of research) of impact that are not measured with our method are nonetheless crucial for understanding the full impact of a research agenda. Therefore, we acknowledge that potentially unknown positive or negative impacts are missed. Our goal was not to classify the agendas according to their levels of impact, but rather to present a comparative analysis of their respective academic impacts. The original inclusion criterion was limited to PRAs with CYP below the age of eighteen. This initial search delivered a limited amount of results. Five more studies were then included, two PRAs in which the age of the CYP was not specified, and an additional three in which the age of the CYP was below 20 or 25 years. Currently, it is challenging to specifically examine research agendas involving only CYP under the age of 18, as the age of the CYP are not always clearly described in the PRA. Furthermore, we acknowledge that by evaluating the English literature only, we may have excluded valuable research published in languages other than English. However, given that English is the predominant language of academic communication and our focus was on studies published in recognised academic databases, it was necessary to prioritise English-language publications. Consequently, important work in non-English or non-academic sources may have been overlooked. The objective of this study was to initiate discussion and create awareness about the academic impact of research agendas. We did not aim to classify research agendas in terms of ‘good’ or ‘bad’ academic impact because the extent to which academic impact is achieved depends on many factors. All those factors must be considered when comparing the agendas, making it challenging to compare the PRAs.

Implications and future research

The DAIAPRA can be used by other researchers when evaluating the academic impact of research agendas. The findings of our study once again indicated the importance of the post-PRA phase. A research project does not end when the top 10 research priorities have been agreed upon. Researchers should disseminate the results of their PRAs to increase exposure to potential funders and researchers. In addition, our results showed that it is difficult to determine whether a research priority is elaborated on. This could argue in favour of establishing a system that enables us to trace a study to determine whether it is based on one of the priorities of a research agenda. For example, researchers could include a statement in the PRAs specifying how the priorities listed should be cited or referenced when researchers elaborate on one of the priorities. Providing an overview of which priorities are addressed by whom and in which country, for each research agenda accessible to everyone, could contribute to more transparency. This clarity also systematically highlights the remaining priorities that require further investigation. Our results indicated that it is challenging to contextualise the academic impact of a PRA because it is unclear who designed the PRA. Therefore, we suggest adding a statement indicating whether the agenda is designed by all key stakeholders or researchers, or whether it has been designed by a specific research team.

Future studies should focus on why some PRAs generated more new studies than others, to guide researchers in creating academic impact. We focused only on the academic impact of the PRAs. Future research should also focus on the policy and societal impact of the PRAs. This is important because, alarmingly, an estimated 85% of medical research evidence never finds its way into clinical practice [5].

Conclusion

Our study contributes to the development of a methodology to evaluate the academic impact of PRAs and we provide initial insight into the academic impact of 31 PRAs. Our findings could be used to inform future PRA design, especially by incorporating provisions for tracing the academic impact of new studies related to the research priorities outlined in the agenda. Overall, our study provides a valuable foundation for further research into the evaluation of academic impact in the field of paediatric research.

Data availability

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

CYP:

Children and young people

DAIAPRA:

Descriptive Academic Impact Analysis of Paediatric Research Agendas

JLA:

James Lind Alliance

PPI:

Patient and public involvement

PRA:

Pediatric research agenda

PSP:

Pediatric research agenda

RIF:

Research Impact Framework

References

  1. Tallon D, Chard J, Dieppe P. Relation between agendas of the research community and the research consumer. Lancet 2000 June 10,;355:2037–40.

  2. Chalmers I. What do I want from health research and researchers when I am a patient? BMJ. 1995;310:1315–8.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  3. Chalmers I, Glasziou P, Library JL, Lind J. Avoidable waste in the production and reporting of research evidence. Lancet. 2009;07–04:374:86–9.

    Article  Google Scholar 

  4. Zurynski Y, Smith CL, Knaggs G, Meulenbroeks I, Braithwaite J. Funding research translation: how we got here and what to do next. Australian New Z J Public Health 2021-07-12;45(5):420.

  5. ZonMW, Procedure voor A. 2019; https://www.zonmw.nl/fileadmin/zonmw/documenten/Corporate/ZonMw_A4_ProcedureAanvragers_201906_def_Projectnet_MijnZonMw_MP.pdf

  6. Hughes A, Martin B. Enhancing Impact: The Value of Public Sector R&D. 2012; https://www.jbs.cam.ac.uk/2012/enhancing-impact-the-value-of-public-sector-rd/. Accessed February, 2022.

  7. Chalmers I, Atkinson P, Fenton M, Firkins L, Crowe S, Cowan K. Tackling treatment uncertainties together: the evolution of the James Lind Initiative, 2003–2013. J R Soc Med 2013-07-03;106(12):482.

  8. Odgers HL, Tong A, Lopez-Vargas P, Davidson A, Jaffe A, McKenzie A, et al. Research priority setting in childhood chronic disease: a systematic review. Arch Dis Child. 2018;103(10):942–.

    Article  PubMed  Google Scholar 

  9. Postma L, Luchtenberg ML, Verhagen AAE, Maeckelberghe EL. Involving children and young people in paediatric research priority setting: a narrative review. Bmjpo 2022;6(1).

  10. Mohr WK, Suess S, THE CONUNDRUM OF CHILDREN IN THE US HEALTH CARE SYSTEM. Nurse Ethics 2001 May 8,;3:196–210.

  11. Postma L, Luchtenberg ML, Verhagen AAE, Maeckelberghe ELM. ‘It’s Powerful’ The impact of involving children and young people in developing paediatric research agendas: A qualitative interview study. Health Expect 2024 -04;27(2).

  12. Geldof J, Leblanc J, Lucaciu L, Segal J, Lees CW, Hart A. Are we addressing the top 10 research priorities in IBD? Frontline Gastroenterol 2021;12(7).

  13. Schilstra CE, Sansom-Daly UM, Schaffer M, Fardell JE, Anazodo AC, McCowage G et al. We have all this knowledge to give, so use us as a resource: partnering with adolescent and young adult Cancer survivors to Determine Consumer-Led Research priorities. J Adolesc Young Adult Oncol 2021:211–22.

  14. Birnie KA, Dib K, Ouellette C, Dib MA, Nelson K, Pahtayken D, et al. Partnering for Pain: a Priority setting Partnership to identify patient-oriented research priorities for pediatric chronic pain in Canada. CMAJ open. 2019;7(4):E654–64.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Montreuil M, Bogossian A, Laberge-Perrault E, Racine E. A review of approaches, strategies and ethical considerations in Participatory Research with Children. Int J Qualitative Methods. 2021;20:160940692098796.

    Article  Google Scholar 

  16. Aussems K, Schoemaker CG, Verwoerd A, Ambrust W, Cowan K, Dedding C. Research agenda setting with children with juvenile idiopathic arthritis: lessons learned. Child: Care Health Dev 2021:68–79.

  17. Staley K, Crowe S, Crocker JC, Madden M, Greenhalgh T. What happens after James Lind Alliance Priority setting partnerships? A qualitative study of contexts, processes and impacts. Res Involv Engagem 2020 Jan 01,;6(1):1–41.

  18. Defining impact. https://www.ukri.org/councils/esrc/impact-toolkit-for-economic-and-social-sciences/defining-impact/. Accessed November 12, 2022.

  19. Greenhalgh T, Raftery J, Hanney S, Glover M. Research impact: a narrative review. BMC Med 2016-05-23;14(1).

  20. Donovan C, Hanney S. The ‘Payback Framework’ explained. Res Evaluation. 2011;–09(3):181.

    Article  Google Scholar 

  21. Kuruvilla S, Mays N, Pleasant A, Walt G. Describing the impact of health research: a Research Impact Framework. BMC Health Serv Res. 2006;6(1):12.

    Article  Google Scholar 

  22. Vera San Juan N, Oram S, Pinfold V, Temple R, Foye U, Simpson A et al. Priorities for Future Research about screen use and adolescent Mental Health: a participatory prioritization study. Front Psychiatry 2022 -05;13.

  23. Smith EMD, Egbivwie N, Cowan K, Ramanan AV, Pain CE. Research priority setting for paediatric rheumatology in the UK. Lancet Rheumatol. 2022;4(7):e517–24.

    Article  CAS  PubMed  Google Scholar 

  24. Ismail D, Mcateer H, Majeed-ariss R, Mcphee M, Griffiths CEM, Young HS. Research priorities and identification of a health‐service delivery model for psoriasis from the UK Psoriasis Priority setting Partnership. Clin Exp Dermatol. 2020-10-06;46(2):276.

  25. von Scheven E, Nahal BK, Cohen IC, Kelekian R, Franck LS. Research questions that Matter to us: priorities of young people with chronic illnesses and their caregivers. Pediatr Res. 2021;89(7):1659–63.

    Article  Google Scholar 

  26. Hollis C, Sampson S, Simons L, Davies EB, Churchill R, Betton V et al. Identifying research priorities for digital technology in mental health care: results of the James Lind Alliance Priority setting Partnership. Health policy 1984.

  27. Schoemaker CG, Armbrust W, Swart JF, Vastert SJ, van Loosdregt J, Verwoerd A, et al. Dutch juvenile idiopathic arthritis patients, carers and clinicians create a research agenda together following the James Lind Alliance method: a study protocol. Pediatr Rheumatol Online J. 2018;16(1):57.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Knight SR, Metcalfe L, O’Donoghue K, Ball ST, Beale A, Beale W, et al. Defining priorities for Future Research: results of the UK kidney Transplant Priority setting Partnership. PLoS ONE. 2016;11(10):e0162136.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Manikam L, Shah R, Reed K, Santini G, Lakhanpaul M. Using a co-production prioritization exercise involving south Asian children, young people and their families to identify health priorities requiring further research and public awareness. Health Expectations: Int J Public Participation Health Care Health Policy. 2017;20(5):852.

    Article  Google Scholar 

  30. Vella-Baldacchino M, Perry DC, Roposch A, Nicolaou N, Cooke S, Ellis P, et al. Research priorities in children requiring elective surgery for conditions affecting the lower limbs: a James Lind Alliance Priority setting Partnership. BMJ open. 2019;9(12):e033233.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Lam JR, Liu B, Bhate R, Fenwick N, Reed K, Duffy JMN, et al. Research priorities for the future health of multiples and their families: The Global Twins and multiples Priority setting Partnership. Ultrasound Obstet Gynecology: Official J Int Soc Ultrasound Obstet Gynecol. 2019;54(6):715–21.

    Article  CAS  Google Scholar 

  32. Medlow S, Patterson P. Determining research priorities for adolescent and young adult cancer in Australia. Eur J Cancer Care. 2015;24(4):590–9.

    Article  CAS  Google Scholar 

  33. Peeks F, Boonstra WF, de Baere L, Carøe C, Casswall T, Cohen D, et al. Research priorities for liver glycogen storage disease: an international priority setting partnership with the James Lind Alliance. J Inherit Metab Dis. 2020;43(2):279–89.

    Article  PubMed  Google Scholar 

  34. Aldiss S, Fern LA, Phillips RS, Callaghan A, Dyker K, Gravestock H, et al. Research priorities for young people with cancer: a UK priority setting partnership with the James Lind Alliance. BMJ open. 2019;9(8):e028119.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Parsons S, Thomson W, Cresswell K, Starling B, McDonagh JE, Barbara Ansell Natl NA. What do young people with rheumatic disease believe to be important to research about their condition? A UK-wide study. Pediatr Rheumatol 2017;15.

  36. Layton A, Eady EA, Peat M, Whitehouse H, Levell N, Ridd M, et al. Identifying acne treatment uncertainties via a James Lind Alliance Priority Setting Partnership. BMJ open. 2015;5(7):e008085.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Batchelor JM, Ridd MJ, Clarke T, Ahmed A, Cox M, Crowe S, et al. The Eczema Priority setting Partnership: a collaboration between patients, carers, clinicians and researchers to identify and prioritize important research questions for the treatment of eczema. Br J Dermatology (1951). 2013;168(3):577–82.

  38. Pagnamenta E, Longhurst L, Breaks A, Chadd K, Kulkarni A, Bryant V, et al. Research priorities to improve the health of children and adults with dysphagia: a National Institute of Health Research and Royal College of Speech and Language Therapists research priority setting partnership. BMJ Open. 2022;12(1):e049459.

  39. Grant A, Crane M, Laupacis A, Griffiths A, Burnett D, Hood A, et al. Engaging patients and caregivers in research for pediatric inflammatory bowel disease: top 10 research priorities. J Pediatr Gastroenterol Nutr. 2019;69(3):317–23.

  40. Rankin G, Summers R, Cowan K, Barker K, Button K, Carroll SP, et al. Identifying priorities for physiotherapy research in the UK: the James Lind Alliance Physiotherapy Priority Setting Partnership. Physiotherapy 2020;107:161–8.

  41. Gill PJ, Bayliss A, Sozer A, Buchanan F, Breen-Reid K, De Castris-Garcia K, et al. Patient, caregiver, and clinician participation in prioritization of research questions in pediatric hospital medicine. JAMA Netw Open. 2022;5(4):e229085.

  42. Lopez-Vargas P, Tong A, Crowe S, Alexander SI, Caldwell PHY, Campbell DE, et al. Research priorities for childhood chronic conditions: a workshop report. Arch Dis Child 2019;104(3):237–45.

  43. Lim AK, Rhodes S, Cowan K, O’Hare A. Joint production of research priorities to improve the lives of those with childhood onset conditions that impair learning: the James Lind Alliance Priority Setting Partnership for ‘learning difficulties’. BMJ open 2019;9(10):e028780.

  44. Fackrell K, Stratmann L, Kennedy V, MacDonald C, Hodgson H, Wray N, et al. Identifying and prioritising unanswered research questions for people with hyperacusis: James Lind Alliance Hyperacusis Priority Setting Partnership. BMJ Open. 2019;9(11):e032178.

  45. Obeid N, McVey G, Seale E, Preskow W, Norris ML. Cocreating research priorities for anorexia nervosa: the Canadian Eating Disorder Priority Setting Partnership. Int J Eat Disord 2020;53(5):392–402.

  46. Shattuck PT, Lau L, Anderson KA, Kuo AA. A national research agenda for the transition of youth with autism. Pediatrics. 2018;141(Suppl 4):S355–61. https://doi.org/10.1542/peds.2016-4300M. PMID: 29610417.

  47. Tunnicliffe DJ, Singh-Grewal D, Craig JC, Howell M, Tugwell P, Mackie F, et al. Healthcare and research priorities of adolescents and young adults with systemic lupus erythematosus: a mixed-methods study. J Rheumatol. 2017;44:444–51.

  48. Morris RL, Stocks SJ, Alam R, Taylor S, Rolfe C, Glover SW, et al. Identifying primary care patient safety research priorities in the UK: a James Lind Alliance Priority Setting Partnership. BMJ open 2018;8(2):e020870.

  49. Finer S, Robb P, Cowan K, Daly A, Shah K, Farmer A. Setting the top 10 research priorities to improve the health of people with Type 2 diabetes: a Diabetes UK–James Lind Alliance Priority Setting Partnership. Diabet Med. 2018;35(7):862–70.

  50. Elwyn G, Crowe S, Fenton M, Firkins L, Versnel J, Walker S, et al. Identifying and prioritizing uncertainties: patient and clinician engagement in the identification of research questions. J Eval Clin Pract. 2018;16(3):627–31.

  51. Raynaud M, Goutaudier V, Louis K, Al-Awadhi S, Dubourg Q, Truchot A et al. Impact of the COVID-19 pandemic on publication dynamics and non-COVID-19 research production. BMC Med Res Methodol 2021-11-22;21(1).

  52. James Lind Alliance. The James Lind Alliance Guidebook. 2021; https://www.jla.nihr.ac.uk/jla-guidebook/. Accessed April, 2021.

  53. Gibson F, Aldiss S. What are the consequences of not responding to Research Priority setting exercises? Cancer care Res Online. 2023;3(1):e037.

    Article  Google Scholar 

  54. Jongsma K, van Seventer J, Verwoerd A, van Rensen A. Recommendations from a James Lind Alliance priority setting partnership - a qualitative interview study. Res Involv Engagem. 2020;6(1):68.

    Article  PubMed  PubMed Central  Google Scholar 

  55. Mawn L, Welsh P, Kirkpatrick L, Webster LAD, Stain HJ. Getting it right! Enhancing youth involvement in mental health research. Health Expect. 2015-07-22;19(4):908.

  56. Ertmer PA, Glazewski KD. Developing a research agenda: contributing new knowledge via intent and focus. J Comput High Educ. 2014;26(1):54–68.

    Article  Google Scholar 

Download references

Acknowledgements

We would like to express our sincere appreciation to Robin Ottjes, Medical Information Specialist, for his remarkable efforts in developing an Excel format that facilitated the extraction of all relevant information from studies that cited one of the included paediatric research agendas. Additionally, we would like to thank Selin Bogers for her valuable contributions as an independent researcher for steps three, four and five. Her contribution ensured an accurate assessment of the academic impact of the paediatric research agendas. We extend our gratitude to Titia van Wulfften Palthe, PhD, for correcting the English manuscript.

Funding

This study was conducted without external financial support.

Author information

Authors and Affiliations

  1. University of Groningen, University Medical Center Groningen, Beatrix Children’s Hospital, Hanzeplein 1, Groningen, 9713 GZ, The Netherlands

    L. Postma, M. L. Luchtenberg, A. A. E. Verhagen & E. L. M. Maeckelberghe

Authors
  1. L. Postma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. L. Luchtenberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. A. A. E. Verhagen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. E. L. M. Maeckelberghe
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

LP contributed to the conception, design, analysis, and interpretation of the data. She drafted the manuscript. SB contributed to Steps 3, 4, and 5 of the DAIAPRA analysis as an independent researcher (analysis of the data). ML, EV and EM contributed to the conception, design, and interpretation of the data and revised the manuscript. All the authors read and approved the final manuscript.

Corresponding author

Correspondence to L. Postma.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Postma, L., Luchtenberg, M.L., Verhagen, A.A.E. et al. The academic impact of paediatric research agendas: a descriptive analysis. Res Involv Engagem 10, 97 (2024). https://doi.org/10.1186/s40900-024-00630-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s40900-024-00630-x

Keywords

Research Involvement and Engagement

ISSN: 2056-7529

Contact us