Skip to content

Advertisement

  • Research article
  • Open Access
  • Open Peer Review

“Still learning and evolving in our approaches”: patient and stakeholder engagement among Canadian community-based primary health care researchers

Research Involvement and Engagement20184:47

https://doi.org/10.1186/s40900-018-0132-0

©  The Author(s). 2018

  • Received: 10 August 2018
  • Accepted: 14 November 2018
  • Published:
Open Peer Review reports

Plain English summary

Increasingly, health researchers are conducting their research in partnership with non-researchers such as patients and caregivers, advocacy groups, clinicians, and policymakers. The idea behind this partnership is to make research more relevant and appropriate. However, so far there is not much evidence about how this partnership or engagement actually affects research. We conducted an online survey of 12 teams in Canada that have engaged patients and other stakeholders in community based health research, partly as a requirement to obtain funding. We found that in many cases, the teams have engaged a wide variety and large number of stakeholders, and have involved them in many different stages of their research. Teams reported that their overall experience of this approach to research has been positive, but some challenges have been encountered along the way. Some teams found that it was difficult to communicate appropriately with all the stakeholders, and to keep them informed when research was going slowly. Other teams had trouble finding government representatives to work with. Several teams noted that engagement is time-consuming, and requires a lot of effort. Nevertheless, all teams reported that they had learned from the experience, and found it valuable. As a result, Canadian health care researchers are better positioned to engage with patients and other stakeholders in the future.

Abstract

Background

Patient and other stakeholder engagement in research is increasingly important, but there is limited evidence of its impact. In 2013, the Canadian Institutes of Health Research launched a five-year Community Based Primary Health Care (CBPHC) initiative that funded 12 teams for innovative approaches to primary health care involving engagement with patients, communities, decision-makers, and clinicians across jurisdictions in Canada. The present study examines the extent of engagement by these teams, and the factors that affected it, either as challenges or opportunities.

Methods

We conducted a cross-sectional web-based survey across the 12 CBPHC Innovation Teams, in which we were also participants. We used a data collection tool developed by the Patient Centered Outcomes Research Institute that included both closed and open-ended questions.

Results

The quantitative data showed that the CBPHC Innovation teams have engaged with diverse stakeholders at different levels and in different stages of research. Almost all teams surveyed engaged with policymakers, most with clinicians and health system representatives, and more than half with patients, mostly at the level of consultation or collaboration. There were very few instances of stakeholder-led research reported. There was a near universal recognition of the importance of communications processes/tools in facilitating engagement, whereas time was the most commonly identified challenge. In almost all cases, challenges encountered were partially if not fully resolved.

The qualitative findings showed that each team’s engagement was contextualized by factors such as the jurisdictions and geographic scope of the project, the number and type of stakeholders engaged and their level of involvement. These intersected with the researchers’ motivations for engagement, to give rise to diverse experiences, but ones that the CBPHC teams assessed positively as an approach to research.

Conclusions

Over the past five years, primary health care researchers in Canada have been actively engaging with patients and other stakeholders. The wide range, extent and nature of that engagement shows that these researchers have anticipated developments in this approach to research and are thus in a position to support and strengthen future efforts to understand the impact of this engagement on health care outcomes.

Keywords

  • Patient engagement
  • Stakeholder engagement
  • Community based
  • Primary care
  • Canada

Background

The role of patient and other stakeholder engagement or involvement in enhancing the relevance and impact of research findings has become an important criterion in health research funding decisions [1, 2]. (See Table 1 for how “patient” and “stakeholder” are used in this article).
Table 1

A note on terminology

The term “patient” is used in different ways in the literature, which seems to depend on region. In the North American context, it is often used in an inclusive sense, comprising patients and family members/caregivers (e.g., Canadian Institutes of Health Research (CIHR) [57]), and sometimes patient advocacy organizations (e.g., Patient-Centered Outcomes Research Institute (PCORI) [58]). In this paper, however, we use “patient” in the exclusive sense, as the survey instrument we used identifies patients, family members/caregivers and patient advocacy organizations separately. Throughout, we use “stakeholder” in CIHR’s sense (“An individual, group or organization having a “stake” in an issue and its outcome.” [59]) We use the phrase “patients and other stakeholders” to reflect these distinctions.

While stakeholders such as patients, caregivers, clinicians and policymakers have often been considered “passive audiences for research results” [3], researchers have increasingly sought their input across the range of research activities, from agenda setting to knowledge translation and dissemination [4–7]. The nature of such engagement varies from (sometimes tokenistic) co-option to collective action or full partnership in which stakeholders lead the research [8]. A number of research approaches have embraced an expanded stakeholder role, including participatory research [8–10], comparative effectiveness research [3, 11–13], implementation research [14], patient-oriented research [5, 15–17] and patient-centered outcomes research [3, 4, 11, 13, 18–20].

Over the past two decades health services, ministries and funding agencies have taken efforts to promote such engagement [2, 21–25]. As Domecq et al. point out, “there is a growing consensus about the crucial role of patient involvement in research, which may improve the value of healthcare research” [7]. Despite this growing interest, relatively few studies have evaluated the process and outcomes of patient and other stakeholder engagement [19, 26–30], particularly in the Canadian context [23], an issue attributed to poor reporting of engagement strategies in published research [31], particularly of researchers’ own expectations [32, 33], and the lack of a formal or systematic process to characterize or evaluate engagement [4, 7, 23, 29, 32, 34]. The difficulty in producing evidence of impact has also been attributed to the “focus on narrowly defined interventions” in the health literature that fails to take into account the more extended effects of engagement [35], and to the tendency to view engagement as an intervention in the research process itself [36, 37].

Some studies suggest it may be more productive to focus primarily on researchers’ expectations of engagement [22, 32]. However, many research studies report their engagement activities inadequately [38], making it difficult to determine their impact, which is complex because it involves contextual factors and “mechanisms” such as underlying beliefs, values, attitudes, etc. [39]. Of particular importance are the values that motivate engagement [2], which the literature categorizes as moral or normative (e.g., empowerment and rights), instrumental or substantive (e.g., improving research quality), and process (e.g., having to do with research conduct) [2, 3, 6, 29, 40–46].

Canadian Institutes of Health Research’s (CIHR) community based primary health care (CBPHC) innovation teams

In 2011, the Canadian Institutes of Health Research (CIHR) announced its Strategy for Patient Oriented Research (SPOR), subsequently releasing its Patient Engagement Framework in 2014 [5]. In the midst of this process, CIHR launched its Community Based Primary Health Care (CBPHC) initiative to develop and compare innovative CBPHC models across jurisdictions by supporting researchers “to conduct original research on innovative models of care delivery, build capacity for research excellence and translate evidence for uptake into practice and policy” [47]. One component of this program funded 12 CBPHC Innovation Teams with two research priorities: (i) access for vulnerable populations; and (ii) chronic disease prevention and management. The CBPHC Innovation Teams were geographically distributed across Canada and were highly diverse, ranging from one to eight principal investigators and up to 42 co-investigators for a total of 280 different researchers [48].

These teams were required to comprise investigators from at least two jurisdictions, at least two decision makers (“typically a health-system manager, policy-maker, community-based healthcare organization leader, or clinician leader” [49]) from two different jurisdictions, and at least one health professional. As part of the funding application, teams had to provide “evidence of a citizen and patient advisory process that enables the provision of feedback on proposed care delivery models, relevant targets for evaluation, community partnerships, and patient, family and community involvement” [49]. Teams were therefore required to collaborate with decision makers and health professionals, but only to consult with citizens and patients.

The present study adds to the evidence base on patient and other stakeholder engagement in the Canadian context by examining the extent of such engagement by the CBPHC Innovation Teams, and provides some insight into the way that engagement affects researchers’ understanding of their research projects, and how contextual factors and motivational values present challenges to and opportunities for successful engagement.

Methods

Study design

One of the 12 CBPHC Innovation Teams, in collaboration with principal investigators from other teams, conducted a cross-sectional web-based survey to assess stakeholder engagement across the 12 CBPHC Innovation Teams. We were therefore study participants as well as researchers.

Participants

The nominated principal investigator (NPI) of each CBPHC Innovation team was asked to identify the most appropriate member of their team to complete the survey, who was then emailed the informed consent form and a link to the survey on an encrypted web platform, Hosted in Canada Surveys. Follow-up emails were sent after two and four weeks to remind participants to complete the survey. If there was no response after four weeks, we followed up with the NPI to request assistance in facilitating a response.

Data collection

We used the Data Collection Tool developed by the Patient-Centered Outcomes Research Institute (PCORI) to assess patient and other stakeholder engagement [4], after modifying it for the Canadian primary health care context (Appendix 1). This tool includes both closed and open-ended questions. “Ethics processes” and “Research team governance” were added as response options for the stages in the research process in which stakeholders were involved, to reflect the core engagement areas outlined in CIHR’s SPOR framework [5]. Two questions under the “Challenges” section were merged to simplify the survey and take advantage of web-based delivery. Details of the tool and its development have been previously described by Forsythe et al. [4].

Data analysis

Data were imported into Excel for cleaning and analysis. Descriptive statistical analyses of responses to categorical survey questions were used to compare and characterize engagement across the CBPHC program. Responses to open-ended survey questions were iteratively analyzed for themes and exceptions [50]. These were initially coded and grouped into themes by one author through a line by line reading; the research team then reviewed the themes for disconfirming evidence, with any discordance being resolved through discussion. Because responses to the open-ended questions were not extensive (~ 8500 words), these data were unlikely to support extensive qualitative analysis [51]. Because of the small sample size and the heterogeneity of the quantitative data, we analyzed the narrative responses to the open-ended survey questions from each team (e.g., motivations, challenges and contributions) in tandem with the descriptive characteristics of team engagement (e.g., types of stakeholders engaged, level of engagement). This allowed us to explore how engagement proceeded in the context of descriptive characteristics found in the quantitative data.

Findings

Ten of the CBPHC teams responded to the survey; the two remaining teams declined to participate. Surveys were completed between July 2017 and January 2018, in the final year of the CBPHC funding initiative.

Quantitative survey responses

Stakeholders

As Fig. 1 shows, all respondents reported engaging patients or other stakeholders in their research projects, with teams engaging between two and six different types. Seven teams engaged patients, three of which also engaged caregivers/family members and another three patient advocacy organizations. The latter were also engaged by one team that engaged neither patients or caregivers. Other commonly engaged stakeholders included policy makers (9 teams), clinicians (8), and health system representatives (7). Two teams reported engaging stakeholders not listed: (i) a volunteer community organization; and (ii) international liaisons. Only one respondent reported engaging payers, explaining that “our policy makers were the same as payers – dual role” (T4), and one reported engaging industry representatives. Responses to the open-ended questions suggested that in some cases respondents may have included one type of stakeholder under another, for example, clinicians and policymakers under “Health System Representatives”.
Fig. 1
Fig. 1

Stakeholders engaged by CBPHC teams (n = 10)

In response to the question asking how many of each type of stakeholder the team had engaged, five respondents indicated they had engaged more than five patients, six that they had engaged more than five clinicians, six that they had engaged more than five health system representatives, and five that they had engaged more than five policymakers. The remaining types of stakeholder were engaged in far fewer numbers, the least being industry representatives, only one of whom was engaged by one team (data not shown).

Nature of involvement

As Fig. 2 shows, five teams reported consulting with patients (i.e., patients provide views on various aspects of the research), two of which also collaborated with patients (i.e., they were formally engaged on the project as ongoing partners). Another team reported engaging with patients only as collaborators. Lastly, only one team reported engaging with patients as stakeholders leading the research. This latter level of engagement, stakeholder-led research, was the least common and least frequent across all stakeholders, involving only five of the ten identified types; one team alone accounted for four of these instances. The majority of engagement was evenly divided between consultation and collaboration. Two respondents identified workshop participation as another type of involvement.
Fig. 2
Fig. 2

Nature of stakeholder involvement (n = 10)

Stages of research

As Fig. 3 shows, looking at the CBPHC teams as a whole, patients were engaged at all stages of research, the most common being results review/interpretation/translation (n = 6 teams), topic solicitation/agenda setting (n = 5) and question development/framing (n = 5). All teams that engaged patients did so in at least three stages of research, with one team engaging patients at ten research stages, encompassing all identified stages except for ethics processes and adding workshop participation (data not shown). An additional stage identified by another respondent was “training for the delivery of the intervention” (T5) (data not shown). Clinicians were the stakeholders most commonly engaged across the majority of research stages, although the largest number of teams engaged policymakers for results review/interpretation/translation. Payers and industry representatives were engaged by the fewest number of teams across all research stages. Involvement in research team governance was more common for patient advocacy organizations, clinicians and health system representatives than for other types of stakeholders.
Fig. 3
Fig. 3

Stakeholder engagement by stage of research (n = 10)

Frequency of engagement

Respondents were asked about the frequency of their email, spoken and in-person contact with stakeholders. As Fig. 4 shows, CBPHC teams engaged stakeholders most commonly on a quarterly basis. More than half of engagement with patients and all engagements with caregivers/family members were on this basis. Clinicians and health system representatives, in contrast, were engaged somewhat more frequently, with three teams engaging clinicians on a biweekly or greater basis. Almost all policymakers were engaged quarterly or less often.
Fig. 4
Fig. 4

Frequency of stakeholder engagement, by number of teams

Length of engagement

Because the question about how long the respondent’s team had been working with each type of stakeholder was open-ended, many of the responses were ranges, multiple lengths, or were vague, which made it difficult to do any precise analysis of the responses. Overall, the data include 11 indications of engagement pre-dating the grant, 17 where it was co-terminous, and 28 where it began sometime after (data not shown).

Facilitators of engagement

Respondents were asked a global question using a four-point Likert scale about the importance of each of a provided list of facilitators of engagement. As Fig. 5 shows, the most important facilitator was communications processes/tools, identified as “Critically important” by eight respondents and as “Important” by one. Remuneration was also identified as “Critically important” by a majority of respondents. Very few respondents identified any of the facilitators as “Not at all important”. Other facilitators respondents used were the “Research Manager,” “spending time in small group and individual interactions” (T4), “research governance of grant” (T8) and “having a structured approach” (T9).
Fig. 5
Fig. 5

Evaluation of facilitators to engagement (n = 10)

Challenges to engagement

Respondents were asked a global question using a three-point Likert scale about the extent to which each of a provided list of challenges was resolved. As Fig. 6 shows, the most commonly identified challenge was stakeholder time, which seven respondents indicated was “Partially resolved,” two that it was “Fully resolved,” and one “Not resolved.” Challenges were identified as “Partially resolved” in about two-thirds of the cases, and “Fully resolved” in about a quarter. One respondent identified “competing priorities” (T8) as an additional challenge.
Fig. 6
Fig. 6

Challenges to engagement (n = 10)

Summary

In summary, the quantitative data show that CBPHC Innovation teams have engaged with diverse stakeholders at different levels and in different research stages. Amongst the few common characteristics were a near universal engagement with policymakers, particularly in the Translation phase of research, and a near universal recognition of the importance of communications processes/tools in facilitating patient and stakeholder engagement.

Qualitative findings

Relationships

The qualitative data clearly indicated both the instrumental values (e.g., as a facilitator for finding appropriate representatives) and ethical values (e.g., “Nothing about us, without us” (T9)) that motivated teams’ engagement. Almost all teams indicated that they had an existing relationship with one type of stakeholder or another, and some referred to “existing contacts” (T5), “existing networks” (T9) or “other relationships” (T8) as ways in which they established relationships with other stakeholders.

Stakeholder engagement took place at the macro (organizations representing communities), meso (communities or groups of individuals) and micro (individual) levels. This was notably the case for one team that was morally motivated to engage with Indigenous communities. “Long standing relationships” (T1) with an Indigenous organization allowed these researchers to collaborate with appropriate health system representatives in those communities. In particular, this was facilitated by the Research Manager who was “housed” at the Indigenous organization: “This position was critically important in developing the initial relationship and facilitating communication between the academic and community research teams” (T1). The importance of this position and the prior relationship is evident from the challenge this team encountered with “establishing relationships and obtaining approval” (T1) in the non-Indigenous communities, where “knowing who to contact, was an issue” (T1) and there was a lack of “coordinated effort from leadership in the [jurisdiction’s] health system” (T1). For this team, “effective and clear communication between all stakeholders and team members” (T1) was essential in addressing such challenges.

Relationships themselves could also prove to be a challenge, particular with policymakers, due to the different lifecycles of policy-making and health care research (see below).

Communication

Consistent with the quantitative data, almost all respondents considered communication to be a critically important facilitator of engagement, but responses to the open-ended questions suggest that this was contextually specific. For example, a team that engaged with six different types of stakeholders indicated a difference between communication with patients and caregivers on the one hand, and health system representatives, payers and policymakers on the other. The patient and caregiver engagement was at both the consultant and collaborator levels, and was motivated moralistically “to ensure that patients and caregivers had a voice at the table” (T4). Engagement occurred on a quarterly basis, and did not include the Execution phase of research. Although the relationships pre-dated the CBPHC grant, and no particular challenges to this engagement were reported, the researchers noted that they needed to use “a different less technical and more approachable language” (T4). More significantly, they also suggested that communication needs run in both directions, as engagement with patients “may require extensive socialization and discussion with researcher[s], many of whom are not used to considering the advisory input of patients in agenda setting or decision-making within the research team” (T4). The contribution that resulted, however, was “to lead us to ask questions in ways that are more directly relevant to improving navigation in the health system from the patient’s perspective as opposed to from the provider perspective” (T4). Another team indicated that the change in the language they used was motivated by patients: “We agreed on term of references for the use of certain words during our meeting that were judged offending (patients with multi morbidity do not want to be called multi morbid!)” (T2).

The motivations for T4’s engagement with decision-makers were process (“First is the requirements of the competition to have these representatives” (T4)) and instrumental (“to have a shorter line to actual health system impact” (T4)). These relationships also pre-dated the CBPHC grant. Engagement occurred quarterly and did not include the Execution phase of research. The challenge of incorporating the interests of decision makers as collaborators and consultants involved paying attention to their own communication needs: “Extensive discussion and true listening to the ways that policy makers and decision-makers communicate with their audience to determine how we could and should present our key findings” (T4). From this, the researchers learned that “engaging with system decision-makers must appreciate the simplicity of the language that they need to communicate with their stakeholders and the lack of tolerance for extensive nuance” (T4) and identified as critically important the facilitator of “spending time in small group and individual interactions developing trust and relationships with key members of the team” (T4). In this context, one of the important contributions was relationship building: “[they] have lead us to spend time to work with them to listen to their questions and develop longer term relationships” (T4).

Diverse populations and communities

Relationship building and communication were also factors in a second theme evident in the qualitative data, namely, the challenge of engaging diverse communities. Another team that engaged six different types of stakeholders learned that engaging with patients and populations involves relationship building not only with patients themselves, but also with their communities:

“it is vital to reach out first to community members and workers who are well accustomed with these patients and these communities, if relevant; they are the ones who can support making contact, translating our different language (e.g., as researchers, we tend to use a terminology that is not intuitive to the general public, even less to vulnerable populations, and which can be a ‘turn off’ for them…). We needed to invest in building relationships with the patients and within the communities.” (T5)

This team engaged with more than five patients, some of whom they had a relationship with that pre-dated the grant, at the level of consultant in the preparatory phase of the research. The instrumental motivations for this engagement were “effectiveness of outcomes…it is necessary that we connect with these members, to understand well their needs, perspectives, values, etc.” (T5). Despite the pre-existing relationship, the team found that they “needed more clear guidelines of what was required for them. The patients also needed to trust those at the details, and understand well what their role as partner was, and how they were experts in their field” (T5).

This team also highlighted a procedurally important aspect of communication, namely consistency: “Developing and using consistency in communication mechanisms is key. Using different communication strategies as well” (T5). Despite these challenges, one of the contributions identified was that “participatory action research creates sustainable partnership, rich and innovative, diverse, communities of exchange and partners that survive and go beyond the particularity of our research programme” (T5).

Another team that involved all stakeholders in the majority of research stages including research team governance engaged with patients as collaborators out of moral motivations (“to ensure patient perspectives are represented” (T7)) and instrumental motivations (“that the research is relevant” (T7)), and recruited them “from our initial studies” (T7). The engagement began after the grant was awarded and occurred on a quarterly basis. This team encountered challenges with “role clarity and supporting previous research participants/subjects in understanding their new advisory/collaborator roles” (T7), which also pertained to engagement with caregivers/family members. Addressing these challenges meant that the researchers “needed…to learn how to tailor approaches to ensure they would align with our patients and family caregivers’ knowledge and interest” (T7), on the one hand, but also “to be aware of and acknowledge their changing health status” (T7). From these and the other stakeholders, however, researchers learned that “they each add a unique perspective to the research and add value in their lived experiences, knowledge and skills” (T7). Because of the high level of engagement of many stakeholders across research phases, the team also noted that “it is time consuming to keep all these stakeholders engaged in meaningful ways” (T7).

Time

Lack of time and changing schedules were a challenge noted by almost all teams, and did not seem to depend on the number of stakeholders engaged. For example, one team that engaged with advocacy organizations at the consultant level, clinicians as collaborators, and policymakers as collaborators and consultants for both instrumental reasons (“Their knowledge of the subject, The clinician’s primary health care experience and his research experience” (T6)) and process reasons (“their local networks” (T6)), noted challenges in “establishing a time for regular contact” (T6) with the advocacy organization and in “finding time to meet” (T6) with the policymakers. The initial learning in this case was “be flexible to meeting times, go where they are” (T6).

In contrast, time and scheduling challenges in engaging with clinicians were mentioned by several teams. For example, a team that engaged six different types of stakeholders found that in engaging clinicians at the levels of consultant and collaborator for instrumental reasons, “maintaining engagement was challenging due to clinicians’ time constraints” (T3), which this team did not encounter with any of the other stakeholders. Nevertheless, the learning reported was that “engagement takes time, but it’s worth it” (T3). A different team that engaged with three types of stakeholders found that the challenge of engagement with clinicians as consultants was that they are “busy people, difficult to meet with” (T2).

The only team to indicate that all of its engagement (with patients, advocacy organizations, clinicians and policymakers) was stakeholder-led also noted that scheduling was “the key challenge” (T9) in engaging with clinicians, even though “family physicians were the driving force behind this project, and as such, formed an integral part of the team” (T9) and the researchers “recruited specialists who have also been engaged since the beginning” (T9). The engagement was instrumentally motivated (“understanding the clinician needs and advancing knowledge to expand, enhance or change service delivery models” (T9)) and occurred more often than biweekly. For this team, however, engagement with policymakers “was the most challenging part of the grant in many ways”(T9). These stakeholders were engaged with from the beginning of the grant for instrumental reasons (“We anticipated that having engaged policy makers would have led to a higher chance of innovations arising from the grant succeeding” (T9)), although the frequency of engagement was low, being less often than quarterly. The researchers found it difficult to know “which policy makers to reach out to” (T9) and learned that they need “to find better ways to engage policy makers” (T9).

Bureaucratic context

For several teams, difficulty in finding the right policymakers to engage with appears to have been exacerbated by personnel turnover (“policymakers changed in each of our partner provinces within the first few years of funding” (T9), “dealing with change in leadership and change in position” (T5)) and bureaucratic structures (“We were challenged by the many layers and structures within government and had to take time to learn its organizational structure…Another challenge is the turnover of policy makers and identifying the right people to discuss our research with” (T8)). Commitment and accountability was another challenge in engaging with policymakers. For example, a team that only engaged with two stakeholders found that with policymakers, “time commitment, scheduling of meetings and follow-through with the recommendations” (T1) were all challenges, and that “often the decision makers at the table were those that had very little decision making power” (T1). Finally, the cross-jurisdictional nature of the CBPHC grant created challenges in “dealing with policy makers across Canada (had to determine who they were)” (T8).

Effects of engagement

Almost all teams indicated either having a plan to assess the influence of patients and other stakeholders on the research, or an interest in doing so. However, few provided any concrete details of such plans. Similarly, most teams had either collected or were planning to collect information regarding the research experiences of patients and other stakeholders through a variety of means, such as “interviews, self-assessment, and focus group data” (T5). One team had obtained funding “to survey and interview our stakeholders in their experiences” (T7), whereas two other teams (T4 and T9) had published papers on their patient and stakeholder experiences.

The teams’ “initial learnings” tended to focus on what the researchers themselves had learned about patient and other stakeholder engagement, and in large part overlapped with the engagement challenges they faced. However, several teams mentioned the effects of engagement on the organizational and governance aspects of their research: “Patients are important to reflect on exactly what is going on in primary care. We as clinician or researchers have a vision that is biased so we need their input to stay on track” (T2); “Being open to different ideas is important, even though it sometimes leads you to change the direction of the work” (T3); “governance structure (input of stakeholders built right in to the grant); stakeholder input in leadership committee” (T8). One team pointed out that this level of involvement also had an impact on stakeholders: “implementation is likely to be more successful if they…are a real collaborator with responsibilities” (T2).

The impact of engagement on the research itself was more evident in the context of “the most significant contributions” by patients and other stakeholders, predominantly in terms of relevance. The most detailed response contrasted the contributions of the different stakeholders. The “patient and caregiver” (T4) led the team “to ask questions in ways that are more directly relevant” (T4), whereas the contributions of decision-makers were an “opportunity to answer the questions that they are prepared to act upon” (T4) and led the team “to try to develop early advice from existing evidence” (T4). In each of these ways, then, the team ended up pursuing its research differently than would otherwise have been the case.

Discussion

This study examined how Canadian primary health care researchers have perceived their engagement with patients and other stakeholders in the course of CIHR’s five-year funding initiative for 12 CBPHC Innovation Teams. The findings show the complex and multi-faceted nature of patient and other stakeholder engagement in these projects. Respondents indicated that they encountered a number of challenges in the course of their engagement activities, such as communication, time, and even finding appropriate stakeholder representatives with whom to engage. Nevertheless, the responses also show that researchers have engaged patients and other stakeholders in a way that closely aligns with the goals articulated in CIHR’s SPOR Patient Engagement Framework [5]. The challenges respondents outlined regarding this engagement, and the efforts teams undertook to clarify roles and expectations, communicate effectively, and be flexible and adaptable certainly indicate that the interaction has been active, and suggest that it has also been meaningful, at least for the researchers. Although only infrequently mentioned by respondents, the funding requirement that CBPHC teams comprise both decision-makers and clinicians, and that they consult with patients, undoubtedly had an influence on the nature and level of their engagement. Interestingly, however, the majority of the teams that reported engaging patients went beyond the CIHR-mandated level, to actively collaborate with these stakeholders throughout the research process, rather than simply seeking feedback on aspects of the research.

This study adds to the evidence base for patient and other stakeholder engagement in the Canadian context, and particularly to our knowledge about the processes of engagement. For example, the motivations these researchers indicated for engaging different stakeholders seem to correspond well to the three types of values – moral or ethical, instrumental or substantive, and process – that have been reported in other studies [6, 26, 43, 46]. But the responses also show how these motivations cut across stakeholder groups, levels of engagement, and stage of research, which highlights the contextual nature of engagement. These findings also provide some insight into how the actual processes of engagement affect the progression of research projects. For example, finding appropriate representatives of different types of stakeholders with whom to engage had an effect on project timelines, which in turn had repercussions for engagement. The difficulty many of the CBPHC teams had in this regard with policymakers suggests that researchers need to take bureaucratic context(s) into account when developing projects incorporating such engagement.

The highly contextual nature of engagement can also be seen in the effect this had on communication, another key issue pointed to in earlier studies [23, 52–54]. Findings from the CBPHC teams provide further evidence of the centrality of communication in effective engagement, but this was realized differently for different stakeholders at different levels of involvement, and possibly at different stages of research. For example, with patients, the issue of communication revolved around the use of non-technical, lay language and sensitivity to vulnerability. With other stakeholders such as health care representatives and policymakers, the communication challenge concerned ways of aligning with their needs and interests. These findings suggest that stakeholders’ interests may also vary. For example, patients’ interests arise out of lived experience, whereas health care representatives’ interests could be process or instrumental. Interestingly, the findings also suggest that policymakers’ interests may be much less specific, possibly because of competing priorities. If this is so, it adds another challenging aspect to engaging such stakeholders.

In terms of the implications for further research, a significant finding was that very little of the engagement involved stakeholders leading the research. Although the reasons for this were not probed by the survey questions, the responses as a whole suggest that lack of time, support and mechanisms to build relationships could be factors. This suggests a possible need for more discussion in the Canadian primary health care research community about the optimal prevalence of stakeholder-led research. The findings also suggest that one of the ways that engagement with patients and other stakeholders works is by deepening researchers’ understanding of how their research is relevant to diverse audiences. Such findings support the idea that evaluations to date may be overlooking a key mechanism in the impact of engagement on research, namely researchers’ learning. Because patient and other stakeholder engagement is complex and “takes many forms and operates at many different levels” [32], more attention needs to be paid to the contexts and mechanisms of engagement to understand its wider impacts [22, 33, 55, 56].

Because the CBPHC projects are in their final stages, it is too early to evaluate the impact of engagement on primary health care outcomes, and their heterogeneity means that it is difficult to generalize from these researchers’ experiences. However, the formative role that patient and other stakeholder engagement has played in these projects can be seen in the survey responses, and is perhaps most evident in the way it has occasioned researchers’ re-evaluation and re-configuration of the various processes involved to take into account stakeholder needs and interests, for example, in communicating with vulnerable patients and accommodating policymakers’ practical requirements. Despite the novelty of this type of engagement for Canadian primary care researchers, the range and diversity of the CBPHC teams’ engagement experience evident in the survey responses demonstrate the extent to which these researchers have embraced this approach, and are now better positioned to actively collaborate with patients, decision-makers and other stakeholders in future research.

Limitations

Although the CBPHC Innovation teams that responded to the survey represent a significant proportion of primary health care researchers in Canada, it is not known to what extent the responses are reflective of all team members, as no data were collected on the process teams actually used to complete the survey. For example, it is unknown whether survey responses included direct feedback from patients and other stakeholders, particularly their views of the various stages of the research. The structure of the survey tool did not allow for response to all questions by type of stakeholder, which may have led to underreporting in some cases, and possible over-representation in cases where pre-formulated responses were provided. Furthermore, it should be noted that the priorities of the CBPHC project were access to care for vulnerable populations and chronic disease prevention and management, and therefore the findings of the present study may not be generalizable to all primary health care research in Canada. Finally, the PCORI study for which this survey tool was developed was aimed at a different context, namely 50 U.S. pilot projects on methods for patient-centered outcomes research [4] with the survey being conducted in the first year of a two year grant. Therefore, it did not include questions about certain factors of relevance to the CBPHC Innovation Teams, such as whether engagement that preceded the funding award had an impact on research design and execution, and the length of time required to build relationships capable of supporting engagement in research. Given these differences, and the small dataset in the present study, we felt that a comparative analysis would be problematic and therefore was not done.

Conclusion

Over the past five years, primary health care researchers in Canada have been actively pursuing patient and other stakeholder engagement. Although this was motivated in part by requirements stipulated by the funding agency, the wide range of stakeholders, extent and nature of that engagement shows that these researchers have anticipated developments in this area and are thus in a position to support and strengthen future efforts to understand the impact of patient and other stakeholder engagement on health care outcomes.

Abbreviations

CBPHC: 

Community-based primary health care

CIHR: 

Canadian Institutes of Health Research

NPI: 

Nominated principal investigator

PCORI: 

Patient-Centered Outcomes Research Institute

SPOR: 

Strategy for Patient Oriented Research

Declarations

Acknowledgements

The authors would like to thank Justin Joschko for editorial advice.

Funding

This paper presents independent research funded by the Canadian Institutes of Health Research.

Availability of data and materials

The dataset generated and analysed during the current study is not publicly available due to confidentiality requirements, but is available in anonymized form from the corresponding author on reasonable request.

Authors’ contributions

CK, CL, LC and RSK conceived of the research. All authors contributed to decision making, analysis and planning. RSK and MF analysed the data obtained from the surveys. CK, CL, LC, RSK and MF interpreted the findings. RK wrote and revised the first draft based on comments from all co-authors. All co-authors reviewed and commented on subsequent drafts. MF revised the manuscript. All authors read and approved the final manuscript.

Ethics approval and consent to participate

This study was approved by the Ottawa Health Sciences Network Research Ethics Board (protocol #20170516-01H) and the Bruyère Continuing Care Research Ethics Board (protocol #M16–17-049). Survey respondents were emailed a Participant Informed Consent Form along with a link to the online survey, which informed them that completion of the electronic questionnaire implied that they had consented to participate in the research study.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
C.T. Lamont Primary Health Care Research Centre, Bruyère Research Institute, Ottawa, ON, Canada
(2)
Department of Family Medicine, University of Ottawa, Ottawa, ON, Canada
(3)
Institute for Clinical Evaluative Sciences (IC/ES), Toronto, ON, Canada
(4)
Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Toronto, ON, Canada
(5)
Ottawa Hospital Research Institute, Ottawa, ON, Canada
(6)
School of Medicine, University of Ottawa, Ottawa, ON, Canada
(7)
School of Nursing, University of British Columbia, Vancouver, BC, Canada
(8)
Centre for Health Services and Policy Research, University of British Columbia, Vancouver, BC, Canada
(9)
Manitoba Centre for Health Policy, Rady Faculty of Health Sciences, University of Manitoba, Winnipeg, MB, Canada
(10)
Department of Family Medicine, Rady Faculty of Health Sciences, University of Manitoba, Winnipeg, MB, Canada
(11)
Department of Community Health Sciences, Rady Faculty of Health Sciences, University of Manitoba, Winnipeg, MB, Canada
(12)
Department of Family Medicine and Emergency Medicine, Université de Sherbrooke, Chicoutimi, QC, Canada
(13)
Centre Intégré Universitaire de Santé et de Services Sociaux du Saguenay-Lac St-Jean, Chicoutimi, QC, Canada
(14)
St. Mary’s Research Centre & Department of Family Medicine, McGill University, Montreal, QC, Canada
(15)
Bridgepoint Collaboratory of the Lunenfeld-Tanenbaum Research Institute, Sinai Health System, Toronto, ON, Canada
(16)
Institute of Health Policy, Management and Evaluation, Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada
(17)
Department of Family and Community Medicine, University of Toronto, Toronto, ON, Canada
(18)
School of Nursing, Faculty of Health Sciences, McMaster University, Hamilton, ON, Canada
(19)
Department of Health, Aging and Society, Faculty of Social Sciences, McMaster University, Hamilton, ON, Canada
(20)
McMaster Institute for Research on Aging, McMaster University, Hamilton, ON, Canada
(21)
Aging, Community and Health Research Unit, McMaster University, Hamilton, ON, Canada
(22)
Department of Family Medicine, Faculty of Medicine, University of Ottawa, Ottawa, ON, Canada

References

  1. Sieck CJ, Hefner JL, McAlearney AS. Finding the Patient Voice in Research Activities: Perspectives of Patient Advisors and Researchers. J Particip Med. 2017;9:2 Available from: https://participatorymedicine.org/journal/evidence/research/2017/04/01/finding-the-patient-voice-in-research-activities-perspectives-of-patient-advisors-and-researchers/.
  2. Wilson P, Mathie E, Keenan J, et al. Research with patient and public involvement: a Realist evaluation – the RAPPORT study. Heal Serv Deliv Res. 2015;3(38):1–176 Available from: https://www.journalslibrary.nihr.ac.uk/hsdr/hsdr03380/.View ArticleGoogle Scholar
  3. Deverka PA, Lavallee DC, Desai PJ, et al. Stakeholder participation in comparative effectiveness research: defining a framework for effective engagement. J Comp Eff Res, 2012 [cited 2018 Jun 11];1(2):181–94. Available from: https://www.futuremedicine.com/doi/10.2217/cer.12.7.View ArticleGoogle Scholar
  4. Forsythe LP, Ellis LE, Edmundson L, et al. Patient and stakeholder engagement in the PCORI pilot projects: description and lessons learned. J Gen Intern Med. 2016;31(1):13–21.View ArticleGoogle Scholar
  5. CIHR. Strategy for Patient-Oriented Research - Patient Engagement Framework - CIHR. 2014 [cited 2018 Apr 19]. Available from: http://www.cihr-irsc.gc.ca/e/48413.html.
  6. Shippee ND, Domecq Garces JP, Prutsky Lopez GJ, et al. Patient and service user engagement in research: a systematic review and synthesized framework. Heal Expect. 2015 [cited 2018 May 1];18(5):1151–1166. Available from: http://doi.wiley.com/10.1111/hex.12090.View ArticleGoogle Scholar
  7. Domecq JP, Prutsky G, Elraiyah T, et al. Patient engagement in research: a systematic review. BMC Health Serv Res. 2014;14(1):89 Available from: http://bmchealthservres.biomedcentral.com/articles/10.1186/1472-6963-14-89.View ArticleGoogle Scholar
  8. Cook T, Boote J, Buckley N, Vougioukalou S, Wright M. Accessing participatory research impact and legacy: developing the evidence base for participatory approaches in health research. Educ Action Res. 2017 [cited 2018 May 1];25(4):473–488. Available from: https://www.tandfonline.com/doi/full/10.1080/09650792.2017.1326964.View ArticleGoogle Scholar
  9. Macaulay AC. Participatory research: What is the history? Has the purpose changed? Fam Pract 2016 [cited 2018 May 3];34(3):cmw117. Available from: https://academic.oup.com/fampra/article-lookup/doi/10.1093/fampra/cmw117.
  10. Macaulay AC, Jagosh J, Seller R, et al. Assessing the benefits of participatory research: a rationale for a realist review. Glob Health Promot. 2011;18(2):45–8.View ArticleGoogle Scholar
  11. Burke JG, Jones J, Yonas M, et al. PCOR, CER, and CBPR: Alphabet Soup or Complementary Fields of Health Research? Clin Transl Sci 2013 [cited 2018 Jun 11];6(6):493–496. Available from: http://doi.wiley.com/10.1111/cts.12064.
  12. Hoffman A, Montgomery R, Aubry W, Tunis SR. How Best To Engage Patients, Doctors, And Other Stakeholders In Designing Comparative Effectiveness Studies. Health Aff, 2010 [cited 2018 Jun 11];29(10):1834–1841. Available from: http://www.healthaffairs.org/doi/10.1377/hlthaff.2010.0675.
  13. Concannon TW, Fuster M, Saunders T, et al. A systematic review of stakeholder engagement in comparative effectiveness and patient-centered outcomes research. J Gen Intern Med. 2014;29(12):1692–701.View ArticleGoogle Scholar
  14. Goodyear-Smith F. Collective enquiry and reflective action in research: towards a clarification of the terminology. Fam Pract, 2016 [cited 2018 Jun 11];34(3):cmw098. Available from: https://academic.oup.com/fampra/article-lookup/doi/10.1093/fampra/cmw098.
  15. Rouleau JL, Moody-Corbett P. Canadian Strategy on Patient-Oriented Research. In: Institute of Medicine, editor. Envisioning a transformed clinical trials enterprise in the United States: Establishing and agenda for 2020: Workshop summary. Washington: The National Academies Press; 2012.Google Scholar
  16. Strom BL, Norman S, Margolis DJ. Patient-oriented research: definitions and new paradigms. Am J Med 2000 [cited 2018 Jun 13];109(2):164–165. Available from: http://linkinghub.elsevier.com/retrieve/pii/S0002934300005027.View ArticleGoogle Scholar
  17. Shaywitz DA, Martin JB, Ausiello DA. Patient-oriented research: principles and new approaches to training. Am J Med 2000 [cited 2018 Jun 13];109(2):136–140. Available from: http://linkinghub.elsevier.com/retrieve/pii/S0002934300004526.View ArticleGoogle Scholar
  18. Sheridan S, Schrandt S, Forsythe L, Hilliard TS, Paez KA. The PCORI engagement rubric: promising practices for partnering in research. Ann Fam Med. 2017;15(2):165–70.View ArticleGoogle Scholar
  19. Forsythe L, Heckert A, Margolis MK, Schrandt S, Frank L. Methods and impact of engagement in research, from theory to practice and back again: early findings from the Patient-Centered Outcomes Research Institute. Qual Life Res. 2018 [cited 2018 Apr 19];27(1):17–31. Available from: http://link.springer.com/10.1007/s11136-017-1581-x.View ArticleGoogle Scholar
  20. Frank L, Forsythe L, Ellis L, et al. Conceptual and practical foundations of patient engagement in research at the patient-centered outcomes research institute. Qual Life Res. 2015;24(5):1033–41.View ArticleGoogle Scholar
  21. Pollock A, Campbell P, Struthers C, et al. Stakeholder involvement in systematic reviews: a protocol for a systematic review of methods, outcomes and effects. [cited 2018 Jun 11]; Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5611627/pdf/40900_2017_Article_60.pdf.
  22. Staley K, Buckland SA, Hayes H, Tarpey M. ‘The missing links’: understanding how context and mechanism influence the impact of public involvement in research. Heal Expect 2014 [cited 2018 Apr 30];17(6):755–764. Available from: http://doi.wiley.com/10.1111/hex.12017.View ArticleGoogle Scholar
  23. Black A, Strain K, Wallsworth C, et al. What constitutes meaningful engagement for patients and families as partners on research teams? J Health Serv Res Policy. 2018 [cited 2018 Apr 30];23(3):158-167. Available from: http://journals.sagepub.com/doi/10.1177/1355819618762960.View ArticleGoogle Scholar
  24. Boote J, Telford R, Cooper C. Consumer involvement in health research: a review and research agenda Health Policy. 2002 [cited 2018 may 30];61(2):213–236. Available from: http://www.ncbi.nlm.nih.gov/pubmed/12088893.
  25. Abelson J. Patient Engagement and Canada’s SPOR Initiative A Resource Guide for Research Teams and Networks, vol. 2015; 2015. p. 20. Available from: https://docs.wixstatic.com/ugd/882293_9545fca9a9c14ced9bd2db5894923783.pdf.Google Scholar
  26. Manafo E, Petermann L, Mason-Lai P, Vandall-Walker V. Patient engagement in Canada: a scoping review of the ‘how’ and ‘what’ of patient engagement in health research. Heal Res Policy Syst. 2018 [cited 2018 Apr 19];16(1):5. Available from: https://health-policy-systems.biomedcentral.com/articles/10.1186/s12961-018-0282-4.
  27. Brett J, Staniszewska S, Mockford C, et al. Mapping the impact of patient and public involvement on health and social care research: a systematic review. Health Expect. 2014;17(5):637–50.View ArticleGoogle Scholar
  28. Brett J, Staniszewska S, Mockford C, et al. A systematic review of the impact of patient and public involvement on service users. Researchers and Communities Patient. 2014;7(4):387–95.View ArticleGoogle Scholar
  29. Esmail L, Moore E, Rein A. Evaluating patient and stakeholder engagement in research: moving from theory to practice. J Comp Eff Res [Internet]. 2015;4(2):133–45 Available from: https://www.futuremedicine.com/doi/abs/10.2217/cer.14.79.View ArticleGoogle Scholar
  30. Lough S. Need to define patient engagement in research. CMAJ. 2015 [cited 2018 may 1];187(12):E385–E386. Available from: http://www.cmaj.ca/content/187/12/E385.long.View ArticleGoogle Scholar
  31. Fergusson D, Monfaredi Z, Pussegoda K, et al. The prevalence of patient engagement in published trials: a systematic review. Res Involv Engagem. 2018 [cited 2018 May 30];4(1):17. Available from: https://researchinvolvement.biomedcentral.com/articles/10.1186/s40900-018-0099-x.
  32. Staley K. ‘Is it worth doing?’ Measuring the impact of patient and public involvement in research. Res Involv Engagem. 2015 [cited 2018 Apr 30];1(1):6. Available from: http://researchinvolvement.biomedcentral.com/articles/10.1186/s40900-015-0008-5.
  33. Staley K. Changing what researchers “think and do”: Is this how involvement impacts on research? Res All. 2017 [cited 2018 May 1];1(1):158–167. Available from: http://www.ingentaconnect.com/content/10.18546/RFA.01.1.13.
  34. Staniszewska S, Brett J, Mockford C, Barber R. The GRIPP checklist: strengthening the quality of patient and public involvement reporting in research. Int J Technol Assess Health Care. 2011;27(4):391–9.View ArticleGoogle Scholar
  35. Springett J. Impact in participatory health research: what can we learn from research on participatory evaluation? Educ Action Res. 2017 [cited 2018 May 1];25(4):560–574. Available from: https://www.tandfonline.com/doi/full/10.1080/09650792.2017.1342554.View ArticleGoogle Scholar
  36. Staniszewska S, Adebajo A, Barber R, et al. Developing the evidence base of patient and public involvement in health and social care research: the case for measuring impact. Int J Consum Stud. 2011;35(6):628–32.View ArticleGoogle Scholar
  37. Sacristan JA, Aguaron A, Avendaño C, et al. Patient involvement in clinical research: why, when, and how. Patient Prefer Adherence. 2016 [cited 2018 Jun 12];631. Available from: https://www.dovepress.com/patient-involvement-in-clinical-research-why-when-and-how-peer-reviewed-article-PPA.
  38. Staniszewska S, Brett J, Simera I, et al. GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research. Res Involv Engagem. 2017 [cited 2018 May 2];3(1):13. Available from: http://researchinvolvement.biomedcentral.com/articles/10.1186/s40900-017-0062-2.
  39. Westhorp G. Realist Impact Evaluation. Res Policy Dev. 2014;September:1–12.Google Scholar
  40. Hamilton CB, Hoens AM, Backman CL, et al. An empirically based conceptual framework for fostering meaningful patient engagement in research. Heal Expect. 2018 [cited 2018 May 30];21(1):396–406. Available from: http://doi.wiley.com/10.1111/hex.12635.View ArticleGoogle Scholar
  41. Duffett L. Patient engagement: What partnering with patient in research is all about. Thrombosis Res 2017 [cited 2018 Apr 19]; 150:113. Available from: https://www.sciencedirect.com/science/article/pii/S0049384816306089.
  42. Edelman N, Barron D. Evaluation of public involvement in research: time for a major re-think? J Health Serv Res Policy. 2016 [cited 2018 May 2];21(3):209–211. Available from: http://journals.sagepub.com/doi/10.1177/1355819615612510.View ArticleGoogle Scholar
  43. Boote J, Wong R, Booth A. ‘Talking the talk or walking the walk?’ A bibliometric review of the literature on public involvement in health research published between 1995 and 2009. Heal Expect. 2015 [cited 2018 Apr 30];18(1):44–57. Available from: http://doi.wiley.com/10.1111/hex.12007.
  44. Bélisle-Pipon J-C, Rouleau G, Birko S. Early-career researchers’ views on ethical dimensions of patient engagement in research. BMC Med Ethics, 2018 [cited 2018 May 30];19(1):21. Available from: https://bmcmedethics.biomedcentral.com/articles/10.1186/s12910-018-0260-y.
  45. Ives J, Damery S, Redwod S. PPI, paradoxes and Plato: Who’s sailing the ship? J Med Ethics. 2013;39(3):181–5.View ArticleGoogle Scholar
  46. Gradinger F, Britten N, Wyatt K, et al. Values associated with public involvement in health and social care research: a narrative review. Heal Expect. 2015 [cited 2018 May 30];18(5):661–675. Available from: http://doi.wiley.com/10.1111/hex.12158.View ArticleGoogle Scholar
  47. CIHR. CBPHC overview - CIHR. 2015 [cited 2018 May 11]. Available from: http://www.cihr-irsc.gc.ca/e/44079.html.
  48. CIHR. Community-Based Primary Health Care - Research profiles - CIHR. 2017 [cited 2018 Apr 19]. Available from: http://www.cihr-irsc.gc.ca/e/50370.html.
  49. CIHR. Team Grant : Community-Based Primary Healthcare. 2012 [cited 2018 Apr 19]. Available from: https://www.researchnet-recherchenet.ca/rnr16/viewOpportunityDetails.do?progCd=10282&language=E&fodAgency=CIHR&view=browseArchive&browseArc=true&org=CIHR.
  50. Guest G, Namey EE, Mitchell ML. Collecting Qualitative Data: A Field Manual for Applied Research. 1 Oliver’s Yard, 55 City Road London EC1Y 1SP: SAGE Publications, Ltd; 2013 [cited 2018 Sep 24]. Available from: http://methods.sagepub.com/book/collecting-qualitative-data/.
  51. LaDonna KA, Taylor T, Lingard L. Why open-ended survey questions are unlikely to support rigorous qualitative insights. Acad Med. 2018;93(3):347–9.View ArticleGoogle Scholar
  52. Bellows M, Burns KK, Jackson K, Surgeoner B, Gallivan J. Meaningful and effective patient engagement: what matters most to stakeholders. Patient Exp J. 2015;2(1):18–28 Available from: http://pxjournal.org/journal/vol2/iss1/5.Google Scholar
  53. Barber R, Beresford P, Boote J, Cooper C, Faulkner A. Evaluating the impact of service user involvement on research: a prospective case study. Int J Consum Stud. 2011;35(6):609–15.View ArticleGoogle Scholar
  54. Witteman HO, Chipenda Dansokho S, Colquhoun H, et al. Twelve Lessons Learned for Effective Research Partnerships Between Patients, Caregivers, Clinicians, Academic Researchers, and Other Stakeholders. J Gen Intern Med. 2018 [cited 2018 May 28];33(4):558–562. Available from: http://link.springer.com/10.1007/s11606-017-4269-6.
  55. Jagosh J, Macaulay A, Pluye P, et al. Uncovering the benefits of participatory research: implications of a realist review for Health Research and practice. Milbank Q. 2012;90(2):311–46.View ArticleGoogle Scholar
  56. Jagosh J, Bush PL, Salsberg J, et al. A realist evaluation of community-based participatory research: partnership synergy, trust building and related ripple effects. BMC Public Health 2015 [cited 2018 May 3];15(1):725. Available from: http://bmcpublichealth.biomedcentral.com/articles/10.1186/s12889-015-1949-1.
  57. Canadian Institutes of Health Research. CIHR Jargon Buster - CIHR. 2015 [cited 2018 Oct 22]. Available from: http://www.cihr-irsc.gc.ca/e/48952.html#p.
  58. Patient-Centered Outcomes Research Institute. Glossary | PCORI. [cited 2018 Oct 25]. Available from: https://www.pcori.org/glossary.
  59. CIHR. CIHR’s Framework for Citizen Engagement - CIHR. 2012 [cited 2018 Apr 30]. Available from: http://www.cihr-irsc.gc.ca/e/41270.html.

Copyright

© The Author(s). 2018

Advertisement

Research Involvement and Engagement

ISSN: 2056-7529

Contact us