- Research article
- Open Access
- Open Peer Review
The prevalence of patient engagement in published trials: a systematic review
Research Involvement and Engagement volume 4, Article number: 17 (2018)
Plain English summary
With the growing movement to engage patients in research, questions are being asked about who is engaging patients and how they are being engaged. Internationally, research groups are supporting and funding patient-oriented research studies that engage patients in the identification of research priorities and the design, conduct and uptake of research. As we move forward, we need to know what meaningful patient engagement looks like, how it benefits research and clinical practice, and what are the barriers to patient engagement?
We conducted a review of the published literature looking for trials that report engaging patients in the research. We included both randomized controlled trials and non-randomized comparative trials. We looked at these trials for important study characteristics, including how patients were engaged, to better understand the practices used in trials. Importantly, we also discuss the number of trials reporting patient engagement practices relative to all published trials. We found that very few trials report any patient engagement activities even though it is widely supported by many major funding organizations. The findings of our work will advance patient-oriented research by showing how patients can be engaged and by stressing that patient engagement practices need to be better reported.
Patient-Oriented Research (POR) is research informed by patients and is centred on what is of importance to them. A fundamental component of POR is that patients are included as an integral part of the research process from conception to dissemination and implementation, and by extension, across the research continuum from basic research to pragmatic trials [J Comp Eff Res 2012, 1:181–94, JAMA 2012, 307:1587–8]. Since POR’s inception, questions have been raised as to how best to achieve this goal.
We conducted a systematic review of randomized controlled trials and non-randomized comparative trials that report engaging patients in their research. Our main goal was to describe the characteristics of published trials engaging patients in research, and to identify the extent of patient engagement activities reported in these trials.
The MEDLINE®, EMBASE®, Cinahl, PsycINFO, Cochrane Methodology Registry, and Pubmed were searched from May 2011 to June 16th, 2016. Title, abstract and full text screening of all reports were conducted independently by two reviewers. Data were extracted from included trials by one reviewer and verified by a second. All trials that report patient engagement for the purposes of research were included.
Of the 9490 citations retrieved, 2777 were reviewed at full text, of which 23 trials were included. Out of the 23 trials, 17 were randomized control trials, and six were non-randomized comparative trials. The majority of these trials (83%, 19/23) originated in the United States and United Kingdom. The trials engaged a range of 2-24 patients/ community representatives per study. Engagement of children and minorities occurred in 13% (3/23) and 26% (6/23) of trials; respectively. Engagement was identified in the development of the research question, the selection of study outcomes, and the dissemination and implementation of results.
The prevalence of patient engagement in patient-oriented interventional research is very poor with 23 trials reporting activities engaging patients. Research dedicated to determining the best practice for meaningful engagement is still needed, but adequate reporting measures also need to be defined.
Patient-Oriented Research (POR) is research informed by patients and is centred on what is of importance to them. A fundamental component of POR is that patients are included as an integral part of the research process from conception to dissemination and implementation, and by extension, across the research continuum from basic research to pragmatic trials [1, 2]. This is commonly, but not solely, referred to as patient engagement [3, 4]. While limited evidence exists with regards to the benefits and potential difficulties of integrating patients into the research process [5,6,7,8], there is a clear movement towards doing so, especially in Europe and North America [9,10,11,12,13]. As is stated by the Canadian Strategy for Patient-Oriented Research, “patients bring the perspective as ‘experts’ from their unique experience and knowledge gained through living with a condition or illness” . Engaging patients in research therefore increases its quality and, as healthcare providers integrate this research into care, the quality of care will also increase .
In addition to the published literature, the concept of patient engagement in research is supported by established and emerging infrastructure at global, national, and institutional levels. Prominent national level organizations include INVOLVE (www.invo.org.uk), which was established in 1996. It is part of, and funded by, the National Institute for Health Research in the UK, to support active public involvement in the National Health Service (NHS), public health and social care research. INVOLVE acts as a national advisory group to bring together expertise, insight and experience in the field of public involvement in research. Their aim is to advance POR as an essential part of the process by which research is identified, prioritized, designed, conducted and disseminated. PCORI (Patient-Centered Outcomes Research Institute) (www.pcori.org) is a non-profit, non-governmental organization located in Washington, DC. Congress authorized the establishment of PCORI in the Patient Protection and Affordable Care Act of 2010. Financially, the organization is working with an estimated budget of $650 million per year (2014–2019). Canada initiated a national Strategy for Patient-Oriented Research (SPOR) (http://www.cihr-irsc.gc.ca/e/41204.html) in 2011.
While methodological guidance does exist on patient-engagement in research [3, 15], the launch of SPOR in Canada has led to questions raised by the research community: what does patient-oriented research look like?; how is research that engages patients being approached?; how should researchers identify and meaningfully engage with patients?; in which elements of the research process can patients contribute?; how do patients and investigators work together?; and what are the barriers and harms to patient engagement for Canadians?
Some of these questions were explored by PCORI in collaboration with the Mayo Clinic in 2011 [8, 16, 17]. Domecq et al.  identified that patient engagement in healthcare research occurs largely at the beginning of qualitative research (protocol and agenda setting), but significantly less so in quantitative research.
The objective of our systematic review was to identify randomized controlled trials and non-randomized comparative trials that report engaging patients in their research study. The goal was to determine the number of trials reporting patient engagement activities, and describe their study characteristics, the quantity, and extent of patient engagement activities within each trial. We aimed to identify key characteristics including research context, which patients or community representatives are engaged, and how patients are engaged.
This systematic review was conducted in partnership with an experienced patient advisor. She is a patient partner on the Canadian SPOR National Steering Committee. The patient partner was involved in reviewing and amending the protocol to ensure it was in line with SPOR priorities, and interpreting research results through a patient lens. The patient partner was involved in disseminating study results to appropriate patient communities.
This systematic review was conducted on the basis of an established protocol. Please see Additional file 1 for the search strategy.
As was stated in the protocol, the definitions of patient, patient engagement, and patient-oriented research are as follows. We defined patient as “individuals with personal experience of a health issue and informal caregivers, including family and friends” . Patient engagement was defined as the “meaningful and active collaboration in governance, priority setting, conducting research and knowledge translation” . Further, we defined patient-oriented research as “a continuum of research that engages patients as partners, focusses on patient-identified priorities and improves patient outcomes”. The research, conducted by multidisciplinary teams in partnership with relevant stakeholders, “aims to apply the knowledge generated to improve healthcare systems and practices” .
We have included all clinical trials that reported a patient perspective elicited for the purposes of research. We have included all reports where patients, family members, caregivers and community members provided input, guidance or consultation (or otherwise described contribution) on at least one element of the research process. Elements included topic generation, priority-setting, question refinement, defining outcomes, methods and study design, statistical analysis plan, interpretation of results, and dissemination and implementation of results.
We have excluded all systematic reviews, cross-sectional and non-comparative studies, studies that did not provide any details on patient perspectives, studies with no full text available, and those that were non-English or non-French language.
An experienced information specialist developed and conducted a search of MEDLINE®, EMBASE®, Cinahl, PsycINFO, Cochrane Methodology Registry, and PubMed were searched from May 2011 to March 2nd, 2015. This timeframe aligned with the search that was conducted by Domecq et al. . An updated search of MEDLINE EPUB ®, EMBASE®, Cinahl, PsycINFO, Cochrane Methodology Registry, and PubMed was conducted from January 1st, 2015 to June 16th, 2016. Potentially eligible titles and/or abstracts were identified using a combination of subject headings (e.g., “patient centered care”, “patient participation”, and “patient involvement”) and key words (e.g., “consumer”, “stakeholder”, “patient”) (See Additional file 1). Additional trials eligible for inclusion were identified by members of the research team prior to the start of the project and used as ‘seed’ articles when developing the electronic search strategy.
In order to determine the prevalence, the Cochrane Central Register of Controlled Trials (CENTRAL) was referenced. A full description of the creation of CENTRAL, including an explicit explanation of the methods used to conduct the search, is available on the Cochrane Library website .
Titles and abstracts were screened for potential inclusion using a liberal accelerated approach (i.e., one reviewer to include and two reviewers to exclude ) (MP, JM, MG, KP, ZM). Screening of full text reports was completed independently in duplicate by a team of reviewers. All screening disagreements were discussed, with any outstanding disagreements resolved by an independent third reviewer (BS). Data Management software, DistillerSR® , was used to manage retrieved records, screen reports, identify and track disagreements. An overview of the results from the screening process is reported using a PRISMA flow diagram (Fig. 1).
We developed forms for data extraction, according to our a priori protocol and items of interest from the included reports based on Domecq et al. . General characteristics and full data extraction was conducted by one reviewer and verified by a second; a 12% random sample of reports was assessed for accuracy. The following general characteristics were extracted: year of publication; title, aim, study design, country of conduct; topic the study focused on; single or multi-centered trial; ethnicity; number of patient/community representatives engaged in research; total sample size.
The engagement in research was categorized into four components: a) engagement in the development of the research question; b) engagement in selection of outcomes; c) engagement in any other way (e.g. input in design, intervention development, protocol review/approval, recruitment, intervention delivery, input in conduct, input in interpretation); d) dissemination and implementation.
Narrative descriptions were reported for all trials. No formal inferential statistical analyses were conducted. An assessment of the quality and risk of bias of included trials and investigation of meta-bias was not conducted as the purpose of our work focused on identifying the scope and types of patient engagement. Reports were categorized by report type (randomized controlled trial and non-randomized comparative trial).
Search and selection results
We retrieved 10,657 reports from electronic searching. These trials were assessed by the study inclusion criteria. Of the 9490 unique title and abstract records retrieved and screened, 2777 full text reports were reviewed for eligibility. A total of 23 reports met the eligibility criteria and were included (Table 1; Additional file 2: Table S2). The trials included 17 randomized-controlled trials and 6 non-randomized trials.
The Cochrane Central Register of Controlled Trials (CENTRAL) reported a total of 371,159 published and unpublished randomized and non-randomized controlled trials across several databases, dating back to 2011. Using this figure as a rough estimate, we have effectively found that across all 371,159 clinical trials, only 23 reported patient engagement practices.
The majority (82%; 14/17) of randomized controlled trials collected focused on different medical fields. The remaining 3 trials were centred on nutrition and exercise research [21,22,23]. All of the included non-randomized trials emerged from distinct medical fields including chronic obstructive pulmonary disease (COPD) , patient information aids , hypertension , influenza , prevention of suicide and alcohol abuse , and nutrition .
Country of conduct
Included trials originated predominantly in the United States (43%, 10/23). Trials originating in the United Kingdom were also included (39%, 9/23). Other countries (13%, 3/23) included the Netherlands, France, and Spain. Only one study (4%, 1/23) was included that reported patient engagement in research that was conducted in Canada .
Patient engagement setting
The included trials reported engaging a cohort rather than a single patient on the research team. The smallest group consisted of 2 patients , and the largest consisted of 24 engaged patients/community representatives . Patient representatives are reported as being patients, parents, caregivers, and members of community organizations.
Inclusion of ethnic minority, marginalized, and special populations
As reported in Additional file 2, almost half of trials (48%; 11/23) reported the inclusion of ethnic minority and marginalized populations in their study. Six trials (26%, 6/23) reported engaging patients of certain ethnic and racial denominations on their research team in some capacity [26, 28, 30,31,32,33].
Engagement with young patients was noted in three trials (13%, 3/23) [21, 28, 30]. In one study, the intervention was geared towards children. Their parents served as proxies and were engaged on the research team .
Engagement by research stage
Figure 2 displays the various types of patient engagement by research design. Our results suggest that research teams were able to engage patients in the development of the research question, the selection of study outcomes, as well as the dissemination and implementation of results. The exception is non-randomized trials where no study engaging patients in selecting outcomes was found. In addition to partnering with patients in developing the research questions, selecting outcomes, or dissemination and implementation of results, all trials reported engagement in a different form. Figure 3 displays “other” forms of patient engagement identified in the included trials. The majority of randomized controlled trials engaged patients in the development or refinement of the study intervention. This is also true of non-randomized comparative trials.
In this review, we identified randomized controlled trials and non-randomized comparative trials that report engagement of patients in research. The results of this review indicated that the prevalence of patient engagement across trials is extremely low in practice. With only 23 trials identified in this report, we estimate that far less than 1% (23/371,159) of clinical trials engage patients meaningfully and actively. The research captured in this review was conducted in various heterogeneous subject areas across the globe. There is not a single phase of research where engagement has not been documented. Generally, patients can contribute to the beginning, middle, and end phases of the research process. In comparison to an earlier systematic review by Domecq et al. , it is evident that the answers to basic questions about patient engagement and patient-oriented research have remained largely unchanged or unanswered.
Difficulties and methodological considerations
In the literature, it is reported that recruitment and retention of patients as partners is a barrier to patient engagement . This is particularly true of research in an acute disease or illness area. In a study on bronchiolitis, authors reported losing a link with patient representatives early in the study . They acknowledge that the short duration of the illness, and frequent misdiagnoses in primary care as potential reasons for this . The team was able to identify patient representatives through admission to the hospital for the remainder of the study . This could be seen as a potential barrier to understanding the perspective of patients in the primary care setting, however.
In several trials, community-based participatory research (CBPR) methods were incorporated into the study design. Benefits of CBPR methods include achieving a more meaningful relationship with the target population and a more effective dissemination and implementation of results . CBPR principles dictate involving community members as integral and equal partners in all research phases . In a trial, authors reported engaging community partners in the development of interventions, intervention testing, data interpretation, and dissemination . Further, in a randomized-controlled trial targeting obesity in the Pacific Islander community, the research team referred to their community partners as “co-researchers”. This team reported community engagement in designing and testing the intervention, delivering the intervention, collecting data, and dissemination and implementation back to the community . Development of the intervention was consistently seen in all trials that reported following a CBPR design [21, 23, 29, 31].
Of the trials where patient engagement activities are reported, our findings suggest that researchers strive to engage patients throughout the length of the research including conduct, implementation, and dissemination [21, 23, 31]. This successful and lasting engagement is potentially due to the advent of the CBPR approach in patient-oriented research. The CBPR approach is a means of working with the community to target their priority concerns. As such, while there exists a great deal of heterogeneity in the fields of research that engage patients, there is an underlying relationship between the trials identified in this review. All focus on prevalent medical conditions that significantly affect key patient-reported priorities.
Domecq and colleagues raised the concern about a lack of comparative effectiveness research in their 2014 review . Their report had not found any comparative effectiveness research on patient engagement methods. The present review has identified that this concern persists and in fact, has led to inconsistent and vague reporting of patient engagement research. Of our identified trials, the quality of reporting on the rationale for patient engagement and the type of engagement chosen is poor. This is potentially due to the absence of legitimate comparison between engagement types to determine best practice. At present, the literature is inconsistent and vague in its reporting.
Patient engagement is reported differently across trials. This irregularity is rooted in both the location of reporting within a research report and in the scope of information provided. Instances of engagement are not clearly distinguished and are often included under various headings in the report. For example, Halanych et al.  reported “Community Engagement” under a separate heading in the “Methods” section. This differs from Cunningham et al. , who reported “Patient and Public Involvement” under the “Discussion” section. This reporting tends to lack detail and explanation of the rationale behind choosing one method of engagement over another (e.g. focus group vs. community advisory board). A method of gauging the type and degree of engagement (i.e. from passive involvement or tokenism, to active collaboration) in research is therefore also lacking. Particularly in an emerging field, such as patient-oriented research and patient engagement in research, poor reporting leaves knowledge users with an insufficient understanding of how the work was conducted, thus limiting its reproducibility, applicability, and impact. Neglecting to report key details of patient engagement methods and impacts deprives the research community of knowledge that could advance the field. Poor reporting could also be responsible for the low prevalence of patient engagement across clinical trials. Without adequate reporting, we are unable to comment accurately on the number of trials practicing patient engagement and the impact this has on the research.
This concern for reporting is discussed by Staniszewska et al.  as well. Staniszewka et al. set out to conduct systematic reviews on the impact of patient and public involvement in research and on healthcare quality. Their findings were incomplete due to the poor quality of reporting methods and variability in reporting found in the literature. In response to this, the team created the Guidance for Reporting Involvement of Patients and Public (GRIPP) checklist . The intention of GRIPP is to standardize and enhance the quality of Patient and Public involvement in research reporting. The team indicated that there is a moral and ethical imperative to report research adequately. GRIPP is the first attempt to identify the most important aspects of good reporting in patient and public involvement work.
Strengths and limitations
Questions surrounding patient engagement are moving beyond the “why” to the “how”. Our review aimed to understand what patient engagement strategies are being used in trials. The paucity of trials reporting patient engagement highlights the gaps in patient-oriented research conduct and reporting. The narrative synthesis of the findings allows a critical assessment of the literature by identifying common themes in a diverse group of trials.
There are limitations to our study. While we followed systematic review methodology, our team decided not to conduct any quality assessment. PRISMA was not an appropriate framework to use given the type of review we conducted; we aimed to collect all examples of patient engagement reported across trials. The quality of the trial’s conduct and reporting was therefore irrelevant. As Domecq et al.  reported as well, conducting quality assessment relates to the outcome of the study, and is inessential to understanding patient engagement strategies used. Further, we are unable to verify the quality or potential for bias of the included trials because patient-oriented research is lacking a tool to assess validity.
A further limitation is in the reporting of patient engagement. Our work is limited by the information reported in the publications. It is possible that research teams did partner with patients, but did not report this in the publication as their focus would be on the results of the trial.
The number of trials reporting patient engagement remains extremely low despite the presence and promotion of patient-oriented research across the globe. In the trials where patient engagement is reported, we noted patients involved in an array of subject areas, and from the beginning of a study to its completion. In patient-oriented research, ethnic, minority, and marginalized populations are engaged is a means of meaningfully achieving outcomes that are important to these groups. Barriers still exist in patient engagement and comparative data determining the best practices of engagement are needed. We also suggest that a standardized manner of reporting patient engagement (e.g. GRIPP 2) is necessary to facilitate comparisons between patient engagement methods as well as conduct.
Community-Based Participatory Research
Patient-Centered Outcomes Research Institute
Strategy for Patient-Oriented Research
Deverka PA, Lavallee DC, Desai PJ, Esmail LC, Ramsey SD, Veenstra DL, et al. Stakeholder participation in comparative effectiveness research: defining a framework for effective engagement. J Comp Eff Res. 2012;1:181–94.
Mullins CD, Abdulhalim AM, Lavallee DC. Continuous patient engagement in comparative effectiveness research. JAMA. 2012;307:1587–8.
Patient-Centered Outcomes Research Institute The PCORI Methodology Report. [Internet]. https://www.pcori.org/research-results/about-our-research/research-methodology/pcori-methodology-report. Accessed 5 Jan 2018.
Canada Institutes for Health Research Strategy for Patient-Oriented Research. The Challenge: Health Research and Health Care. [Internet]. http://www.cihr-irsc.gc.ca/e/47473.html. Accessed 6 July 2016.
Shah SG, Robinson I. Benefits of and barriers to involving users in medical device technology development and evaluation. Int J Technol Assess Health Care. 2007;23:131–7.
Swartz LJ, Callahan KA, Butz AM, Rand CS, Kanchanaraksa S, Diette GB, et al. Methods and issues in conducting a community-based environmental randomized trial. Environ Res. 2004;95:156–65.
Edwards V, Wyatt K, Logan S, Britten N. Consulting parents about the design of a randomized controlled trial of osteopathy for children with cerebral palsy. Health Expect. 2011;14:429–38.
Domecq JP, Prutsky G, Elraiyah T, Wang Z, Nabhan M, Shippee N, et al. Patient engagement in research: a systematic review. BMC Health Serv Res. 2014;14:89.
Selby JV, Slutsky JR. Practicing partnered research. J Gen Intern Med. 2014;29(Suppl 4):814–6.
Hysong SJ, Woodard L, Garvin JH, Murawsky J, Petersen LA. Publishing protocols for partnered research. J Gen Intern Med. 2014;29(Suppl 4):820–4.
Costlow MR, Landsittel DP, James AE III, Kahn JM, Morton SC. Model for a patient-centered comparative effectiveness research center. Clin Transl Sci. 2015;8:155–9.
Tapp H, Dulin M. The science of primary health-care improvement: potential and use of community-based participatory research by practice-based research networks for translation of research into practice. Exp Biol Med (Maywood). 2010;235:290–9.
Harrison M, Palmer R. Exploring patient and public involvement in stroke research: a qualitative study. Disabil Rehabil. 2015;37:2174–83.
Canada Institutes for Health Research Strategy for Patient-Oriented Research. Strategy for Patient-Oriented Research- Patient Engagement Framework. [Internet]. http://www.cihr-irsc.gc.ca/e/48413.html#a5. Accessed 4 Apr 2018.
Briefing notes for researchers: involving the public in NHS, public health and social care research. [Internet]. http://www.invo.org.uk/wp-content/uploads/2012/04/INVOLVEBriefingNotesApr2012.pdf. Accessed 5 Jan 2018.
Shippee ND, Domecq Garces JP, Prutsky Lopez GJ, Wang Z, Elraiyah TA, Nabhan M, et al. Patient and service user engagement in research: a systematic review and synthesized framework. Health Expect. 2015;18:1151–66.
Forsythe LP, Szydlowski V, Murad MH, Ip S, Wang Z, Elraiyah TA, et al. A systematic review of approaches for engaging patients for research on rare diseases. J Gen Intern Med. 2014;29(Suppl 3):S788–800.
CENTRAL creation details. [Internet]. http://www.cochranelibrary.com/help/central-creation-details.html. Accessed 5 Jan 2018.
Khangura S, Konnyu K, Cushman R, Grimshaw J, Moher D. Evidence summaries: the evolution of a rapid review approach. Syst Rev. 2012;1:10.
Moher D, Hopewell S, Schulz KF, Montori V, Gotzsche PC, Devereaux PJ, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c869.
Bogart LM, Cowgill BO, Elliott MN, Klein DJ, Hawes-Dawson J, Uyeda K, et al. A randomized controlled trial of students for nutrition and eXercise: a community-based participatory research study. J Adolesc Health. 2014;55(3):415-22.
Bowrey DJ, Baker M, Halliday V, Thomas AL, Pulikottil-Jacob R, Smith K. A randomised controlled trial of six weeks of home enteral nutrition versus standard care after oesophagectomy or total gastrectomy for cancer: report on a pilot and feasibility study. Trials. 2015;16(1):531.
Kattelmann KK, Bredbenner CB, White AA, Greene GW, Hoerr SL, Kidd T, et al. The effects of Young Adults Eating and Active for Health (YEAH): a theory-based Web-delivered intervention. J Nutr Educ Behav. 2014;46(6):S27–41.
Mitchell KE, Apps LD, Harrison SL, Sewell L, Williams JE, Young HM, et al. The development and pilot testing of the self-management programme of activity, coping and education for chronic obstructive pulmonary disease (SPACE for COPD). Int J Chron Obstruct Pulmon Dis. 2013;8:317.
Man MS, Healthlines Study Group, Rick J, Bower P, MRC-START Group. Improving recruitment to a study of telehealth management for long-term conditions in primary care: two embedded, randomised controlled trials of optimised patient information materials. Trials [Electronic Resource]. 2015;16:309.
Tully M, Kos A, Eastwood D, Kusch J, Kotchen T. Implementation of an adjunct strategy to reduce blood pressure in blacks with uncontrolled hypertension: a pilot project. Ethnicity & Disease. 2015;25(2):168–74.
Goodacre S, Irving A, Wilson R, Beever D, Challen K. The PAndemic INfluenza Triage in the Emergency Department (PAINTED) pilot cohort study. Health Technol Assessment (Winchester, England). 2015;19(3):v-xxi.
Mohatt GV, Fok CC, Henry D. People awakening team, Allen J. Feasibility of a community intervention for the prevention of suicide and alcohol abuse with Yup'ik Alaska native youth: the Elluam Tungiinun and Yupiucimta Asvairtuumallerkaa studies. Am J Community Psychol. 2014;54(1–2):153–69.
Kolahdooz F, Pakseresht M, Mead E, Beck L, Corriveau A, Sharma S. Impact of the healthy foods north nutrition intervention program on Inuit and Inuvialuit food consumption and preparation methods in Canadian Arctic communities. Nutr J. 2014;13:68.
Bauermeister JA, Pingel ES, Jadwin-Cakmak L, Harper GW, Horvath K, Weiss G, et al. Acceptability and preliminary efficacy of a tailored online HIV/STI testing intervention for young men who have sex with men: the get connected! Program. AIDS & Behavior. 19:1860–74.
Pearson CR, Parker M, Fisher CB, Moreno C. Capacity building from the inside out: development and evaluation of a CITI ethics certification training module for American Indian and Alaska native community researchers. Journal of Empirical Research on Human Research Ethics. 2014;9(1):46.
O'Callaghan P, Branham L, Shannon C, Betancourt TS, Dempster M, McMullen J. A pilot study of a family focused, psychosocial intervention with war-exposed youth at risk of attack and abduction in north-eastern Democratic Republic of Congo. Child Abuse Negl. 2014;38(7):1197–207.
Kaholokula JK, Mau MK, Efird JT, et al. A Family and Community Focused Lifestyle Program Prevents Weight Regain in Pacific Islanders: A Pilot Randomized Controlled Trial. Health Educ Behav. 2012;39(4):386–95.
Cunningham S, Rodriguez A, Boyd KA, McIntosh E, Lewis SC, BIDS Collaborators Group. Bronchiolitis of Infancy Discharge Study (BIDS): a multicentre, parallel-group, double-blind, randomised controlled, equivalence trial with economic evaluation. Health Technology Assessment (Winchester, England). 19:i–xxiii.
Brett JB, Staniszewska S, Mockford C, Herron-Marx S, Hughes J, Tynsall C, et al. Mapping the impact of patient and public involvement on health and social care research: a systematic review. Health Expect. 2012;17(5):637–50.
Halanych JH, Andreae SJ, Cherrington A, Safford MM. Recruitment of a rural, southern, predominantly African-American population into a diabetes self-management trial. Contemporary Clinical Trials. 2012;33(3):499-506.
Staniszewska S, Brett J, Simera I, Seers K, Mockford C, Goodlad S, et al. GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research. Res Involv Engagem. 2017;3:13.
Zgibor JC, Schlenk EA, Vater L, Kola S, Vander BJ, Woody S, et al. Partnership building and implementation of an integrated healthy-aging program. Prog Community Health Partnersh. 2016;10(1):123–32.
Huppelschoten AG, Nelen WL, Westert GP, van Golde RJ, Adang EM, Kremer JA. Improving patient-centredness in partnership with female patients: a cluster RCT in fertility care. Hum Reprod. 2015;30(5):1137–45.
Ormerod AD, Thomas KS, Craig FE, Mitchell E, Greenlaw N, Norrie J, et al. Comparison of the two most commonly used treatments for pyoderma gangrenosum: results of the STOP GAP randomised controlled trial. BMJ. 2015;350:h2958.
Littlewood E, Duarte A, Hewitt C, Knowles S, Palmer S, Walker S, et al. A randomised controlled trial of computerised cognitive behaviour therapy for the treatment of depression in primary care: The Randomised Evaluation of the Effectiveness and Acceptability of Computerised Therapy (REEACT) trial. Health Technol Assessment. 2015;19:101,1.
McMillan A, Bratton DJ, Faria R, Laskawiec-Szkonter M, Griffin S, Davies RJ, et al. A multicentre randomised controlled trial and economic evaluation of continuous positive airway pressure for the treatment of obstructive sleep apnoea syndrome in older people: PREDICT. Health Technology Assessment (Winchester, England). 19:01.
Gaucher S, Boutron I, Marchand-Maillet F, Baron G, Douard R, Bethoux J-P. Assessment of a standardized pre-operative telephone checklist designed to avoid late cancellation of ambulatory surgery: the AMBUPROG multicenter randomized controlled trial. PLoS One. 2016;11(2):e0147194.
Fraser RT, Johnson EK, Lashley S, Barber J, Chaytor N, Miller J et al. The program for active consumer engagement in self-management (PACES) in epilepsy. Epilepsia conference: 31st international epilepsy congress Istanbul Turkey conference start: 2015,February.
Gimeno-Santos E, Raste Y, Demeyer H, Louvaris Z, de JC, Rabinovich RA, et al. The PROactive instruments to measure physical activity in patients with chronic obstructive pulmonary disease. European Respiratory Journal. 2015;46(4):988–1000.
We would like to thank Raymond Daniel for his support with running the search and identifying duplicates. We would also like to thank Lucy Turner for her support in the initial stages of this study. The Ontario SPOR SUPPORT Unit offered its advice and feedback during the development of this study, and we are very grateful for this support.
This project was completed on behalf of the Ottawa Methods Centre, funded by the Ontario SPOR SUPPORT Unit (OSSU), the Canadian Institutes of Health Research, and the Ontario Ministry of Health and Long-Term Care.
Availability of data and materials
All of the trials included in this systematic review are available from searchable databases.
Ethics approval and consent to participate
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.