Ten of the CBPHC teams responded to the survey; the two remaining teams declined to participate. Surveys were completed between July 2017 and January 2018, in the final year of the CBPHC funding initiative.
Quantitative survey responses
Stakeholders
As Fig. 1 shows, all respondents reported engaging patients or other stakeholders in their research projects, with teams engaging between two and six different types. Seven teams engaged patients, three of which also engaged caregivers/family members and another three patient advocacy organizations. The latter were also engaged by one team that engaged neither patients or caregivers. Other commonly engaged stakeholders included policy makers (9 teams), clinicians (8), and health system representatives (7). Two teams reported engaging stakeholders not listed: (i) a volunteer community organization; and (ii) international liaisons. Only one respondent reported engaging payers, explaining that “our policy makers were the same as payers – dual role” (T4), and one reported engaging industry representatives. Responses to the open-ended questions suggested that in some cases respondents may have included one type of stakeholder under another, for example, clinicians and policymakers under “Health System Representatives”.
In response to the question asking how many of each type of stakeholder the team had engaged, five respondents indicated they had engaged more than five patients, six that they had engaged more than five clinicians, six that they had engaged more than five health system representatives, and five that they had engaged more than five policymakers. The remaining types of stakeholder were engaged in far fewer numbers, the least being industry representatives, only one of whom was engaged by one team (data not shown).
Nature of involvement
As Fig. 2 shows, five teams reported consulting with patients (i.e., patients provide views on various aspects of the research), two of which also collaborated with patients (i.e., they were formally engaged on the project as ongoing partners). Another team reported engaging with patients only as collaborators. Lastly, only one team reported engaging with patients as stakeholders leading the research. This latter level of engagement, stakeholder-led research, was the least common and least frequent across all stakeholders, involving only five of the ten identified types; one team alone accounted for four of these instances. The majority of engagement was evenly divided between consultation and collaboration. Two respondents identified workshop participation as another type of involvement.
Stages of research
As Fig. 3 shows, looking at the CBPHC teams as a whole, patients were engaged at all stages of research, the most common being results review/interpretation/translation (n = 6 teams), topic solicitation/agenda setting (n = 5) and question development/framing (n = 5). All teams that engaged patients did so in at least three stages of research, with one team engaging patients at ten research stages, encompassing all identified stages except for ethics processes and adding workshop participation (data not shown). An additional stage identified by another respondent was “training for the delivery of the intervention” (T5) (data not shown). Clinicians were the stakeholders most commonly engaged across the majority of research stages, although the largest number of teams engaged policymakers for results review/interpretation/translation. Payers and industry representatives were engaged by the fewest number of teams across all research stages. Involvement in research team governance was more common for patient advocacy organizations, clinicians and health system representatives than for other types of stakeholders.
Frequency of engagement
Respondents were asked about the frequency of their email, spoken and in-person contact with stakeholders. As Fig. 4 shows, CBPHC teams engaged stakeholders most commonly on a quarterly basis. More than half of engagement with patients and all engagements with caregivers/family members were on this basis. Clinicians and health system representatives, in contrast, were engaged somewhat more frequently, with three teams engaging clinicians on a biweekly or greater basis. Almost all policymakers were engaged quarterly or less often.
Length of engagement
Because the question about how long the respondent’s team had been working with each type of stakeholder was open-ended, many of the responses were ranges, multiple lengths, or were vague, which made it difficult to do any precise analysis of the responses. Overall, the data include 11 indications of engagement pre-dating the grant, 17 where it was co-terminous, and 28 where it began sometime after (data not shown).
Facilitators of engagement
Respondents were asked a global question using a four-point Likert scale about the importance of each of a provided list of facilitators of engagement. As Fig. 5 shows, the most important facilitator was communications processes/tools, identified as “Critically important” by eight respondents and as “Important” by one. Remuneration was also identified as “Critically important” by a majority of respondents. Very few respondents identified any of the facilitators as “Not at all important”. Other facilitators respondents used were the “Research Manager,” “spending time in small group and individual interactions” (T4), “research governance of grant” (T8) and “having a structured approach” (T9).
Challenges to engagement
Respondents were asked a global question using a three-point Likert scale about the extent to which each of a provided list of challenges was resolved. As Fig. 6 shows, the most commonly identified challenge was stakeholder time, which seven respondents indicated was “Partially resolved,” two that it was “Fully resolved,” and one “Not resolved.” Challenges were identified as “Partially resolved” in about two-thirds of the cases, and “Fully resolved” in about a quarter. One respondent identified “competing priorities” (T8) as an additional challenge.
Summary
In summary, the quantitative data show that CBPHC Innovation teams have engaged with diverse stakeholders at different levels and in different research stages. Amongst the few common characteristics were a near universal engagement with policymakers, particularly in the Translation phase of research, and a near universal recognition of the importance of communications processes/tools in facilitating patient and stakeholder engagement.
Qualitative findings
Relationships
The qualitative data clearly indicated both the instrumental values (e.g., as a facilitator for finding appropriate representatives) and ethical values (e.g., “Nothing about us, without us” (T9)) that motivated teams’ engagement. Almost all teams indicated that they had an existing relationship with one type of stakeholder or another, and some referred to “existing contacts” (T5), “existing networks” (T9) or “other relationships” (T8) as ways in which they established relationships with other stakeholders.
Stakeholder engagement took place at the macro (organizations representing communities), meso (communities or groups of individuals) and micro (individual) levels. This was notably the case for one team that was morally motivated to engage with Indigenous communities. “Long standing relationships” (T1) with an Indigenous organization allowed these researchers to collaborate with appropriate health system representatives in those communities. In particular, this was facilitated by the Research Manager who was “housed” at the Indigenous organization: “This position was critically important in developing the initial relationship and facilitating communication between the academic and community research teams” (T1). The importance of this position and the prior relationship is evident from the challenge this team encountered with “establishing relationships and obtaining approval” (T1) in the non-Indigenous communities, where “knowing who to contact, was an issue” (T1) and there was a lack of “coordinated effort from leadership in the [jurisdiction’s] health system” (T1). For this team, “effective and clear communication between all stakeholders and team members” (T1) was essential in addressing such challenges.
Relationships themselves could also prove to be a challenge, particular with policymakers, due to the different lifecycles of policy-making and health care research (see below).
Communication
Consistent with the quantitative data, almost all respondents considered communication to be a critically important facilitator of engagement, but responses to the open-ended questions suggest that this was contextually specific. For example, a team that engaged with six different types of stakeholders indicated a difference between communication with patients and caregivers on the one hand, and health system representatives, payers and policymakers on the other. The patient and caregiver engagement was at both the consultant and collaborator levels, and was motivated moralistically “to ensure that patients and caregivers had a voice at the table” (T4). Engagement occurred on a quarterly basis, and did not include the Execution phase of research. Although the relationships pre-dated the CBPHC grant, and no particular challenges to this engagement were reported, the researchers noted that they needed to use “a different less technical and more approachable language” (T4). More significantly, they also suggested that communication needs run in both directions, as engagement with patients “may require extensive socialization and discussion with researcher[s], many of whom are not used to considering the advisory input of patients in agenda setting or decision-making within the research team” (T4). The contribution that resulted, however, was “to lead us to ask questions in ways that are more directly relevant to improving navigation in the health system from the patient’s perspective as opposed to from the provider perspective” (T4). Another team indicated that the change in the language they used was motivated by patients: “We agreed on term of references for the use of certain words during our meeting that were judged offending (patients with multi morbidity do not want to be called multi morbid!)” (T2).
The motivations for T4’s engagement with decision-makers were process (“First is the requirements of the competition to have these representatives” (T4)) and instrumental (“to have a shorter line to actual health system impact” (T4)). These relationships also pre-dated the CBPHC grant. Engagement occurred quarterly and did not include the Execution phase of research. The challenge of incorporating the interests of decision makers as collaborators and consultants involved paying attention to their own communication needs: “Extensive discussion and true listening to the ways that policy makers and decision-makers communicate with their audience to determine how we could and should present our key findings” (T4). From this, the researchers learned that “engaging with system decision-makers must appreciate the simplicity of the language that they need to communicate with their stakeholders and the lack of tolerance for extensive nuance” (T4) and identified as critically important the facilitator of “spending time in small group and individual interactions developing trust and relationships with key members of the team” (T4). In this context, one of the important contributions was relationship building: “[they] have lead us to spend time to work with them to listen to their questions and develop longer term relationships” (T4).
Diverse populations and communities
Relationship building and communication were also factors in a second theme evident in the qualitative data, namely, the challenge of engaging diverse communities. Another team that engaged six different types of stakeholders learned that engaging with patients and populations involves relationship building not only with patients themselves, but also with their communities:
“it is vital to reach out first to community members and workers who are well accustomed with these patients and these communities, if relevant; they are the ones who can support making contact, translating our different language (e.g., as researchers, we tend to use a terminology that is not intuitive to the general public, even less to vulnerable populations, and which can be a ‘turn off’ for them…). We needed to invest in building relationships with the patients and within the communities.” (T5)
This team engaged with more than five patients, some of whom they had a relationship with that pre-dated the grant, at the level of consultant in the preparatory phase of the research. The instrumental motivations for this engagement were “effectiveness of outcomes…it is necessary that we connect with these members, to understand well their needs, perspectives, values, etc.” (T5). Despite the pre-existing relationship, the team found that they “needed more clear guidelines of what was required for them. The patients also needed to trust those at the details, and understand well what their role as partner was, and how they were experts in their field” (T5).
This team also highlighted a procedurally important aspect of communication, namely consistency: “Developing and using consistency in communication mechanisms is key. Using different communication strategies as well” (T5). Despite these challenges, one of the contributions identified was that “participatory action research creates sustainable partnership, rich and innovative, diverse, communities of exchange and partners that survive and go beyond the particularity of our research programme” (T5).
Another team that involved all stakeholders in the majority of research stages including research team governance engaged with patients as collaborators out of moral motivations (“to ensure patient perspectives are represented” (T7)) and instrumental motivations (“that the research is relevant” (T7)), and recruited them “from our initial studies” (T7). The engagement began after the grant was awarded and occurred on a quarterly basis. This team encountered challenges with “role clarity and supporting previous research participants/subjects in understanding their new advisory/collaborator roles” (T7), which also pertained to engagement with caregivers/family members. Addressing these challenges meant that the researchers “needed…to learn how to tailor approaches to ensure they would align with our patients and family caregivers’ knowledge and interest” (T7), on the one hand, but also “to be aware of and acknowledge their changing health status” (T7). From these and the other stakeholders, however, researchers learned that “they each add a unique perspective to the research and add value in their lived experiences, knowledge and skills” (T7). Because of the high level of engagement of many stakeholders across research phases, the team also noted that “it is time consuming to keep all these stakeholders engaged in meaningful ways” (T7).
Time
Lack of time and changing schedules were a challenge noted by almost all teams, and did not seem to depend on the number of stakeholders engaged. For example, one team that engaged with advocacy organizations at the consultant level, clinicians as collaborators, and policymakers as collaborators and consultants for both instrumental reasons (“Their knowledge of the subject, The clinician’s primary health care experience and his research experience” (T6)) and process reasons (“their local networks” (T6)), noted challenges in “establishing a time for regular contact” (T6) with the advocacy organization and in “finding time to meet” (T6) with the policymakers. The initial learning in this case was “be flexible to meeting times, go where they are” (T6).
In contrast, time and scheduling challenges in engaging with clinicians were mentioned by several teams. For example, a team that engaged six different types of stakeholders found that in engaging clinicians at the levels of consultant and collaborator for instrumental reasons, “maintaining engagement was challenging due to clinicians’ time constraints” (T3), which this team did not encounter with any of the other stakeholders. Nevertheless, the learning reported was that “engagement takes time, but it’s worth it” (T3). A different team that engaged with three types of stakeholders found that the challenge of engagement with clinicians as consultants was that they are “busy people, difficult to meet with” (T2).
The only team to indicate that all of its engagement (with patients, advocacy organizations, clinicians and policymakers) was stakeholder-led also noted that scheduling was “the key challenge” (T9) in engaging with clinicians, even though “family physicians were the driving force behind this project, and as such, formed an integral part of the team” (T9) and the researchers “recruited specialists who have also been engaged since the beginning” (T9). The engagement was instrumentally motivated (“understanding the clinician needs and advancing knowledge to expand, enhance or change service delivery models” (T9)) and occurred more often than biweekly. For this team, however, engagement with policymakers “was the most challenging part of the grant in many ways”(T9). These stakeholders were engaged with from the beginning of the grant for instrumental reasons (“We anticipated that having engaged policy makers would have led to a higher chance of innovations arising from the grant succeeding” (T9)), although the frequency of engagement was low, being less often than quarterly. The researchers found it difficult to know “which policy makers to reach out to” (T9) and learned that they need “to find better ways to engage policy makers” (T9).
Bureaucratic context
For several teams, difficulty in finding the right policymakers to engage with appears to have been exacerbated by personnel turnover (“policymakers changed in each of our partner provinces within the first few years of funding” (T9), “dealing with change in leadership and change in position” (T5)) and bureaucratic structures (“We were challenged by the many layers and structures within government and had to take time to learn its organizational structure…Another challenge is the turnover of policy makers and identifying the right people to discuss our research with” (T8)). Commitment and accountability was another challenge in engaging with policymakers. For example, a team that only engaged with two stakeholders found that with policymakers, “time commitment, scheduling of meetings and follow-through with the recommendations” (T1) were all challenges, and that “often the decision makers at the table were those that had very little decision making power” (T1). Finally, the cross-jurisdictional nature of the CBPHC grant created challenges in “dealing with policy makers across Canada (had to determine who they were)” (T8).
Effects of engagement
Almost all teams indicated either having a plan to assess the influence of patients and other stakeholders on the research, or an interest in doing so. However, few provided any concrete details of such plans. Similarly, most teams had either collected or were planning to collect information regarding the research experiences of patients and other stakeholders through a variety of means, such as “interviews, self-assessment, and focus group data” (T5). One team had obtained funding “to survey and interview our stakeholders in their experiences” (T7), whereas two other teams (T4 and T9) had published papers on their patient and stakeholder experiences.
The teams’ “initial learnings” tended to focus on what the researchers themselves had learned about patient and other stakeholder engagement, and in large part overlapped with the engagement challenges they faced. However, several teams mentioned the effects of engagement on the organizational and governance aspects of their research: “Patients are important to reflect on exactly what is going on in primary care. We as clinician or researchers have a vision that is biased so we need their input to stay on track” (T2); “Being open to different ideas is important, even though it sometimes leads you to change the direction of the work” (T3); “governance structure (input of stakeholders built right in to the grant); stakeholder input in leadership committee” (T8). One team pointed out that this level of involvement also had an impact on stakeholders: “implementation is likely to be more successful if they…are a real collaborator with responsibilities” (T2).
The impact of engagement on the research itself was more evident in the context of “the most significant contributions” by patients and other stakeholders, predominantly in terms of relevance. The most detailed response contrasted the contributions of the different stakeholders. The “patient and caregiver” (T4) led the team “to ask questions in ways that are more directly relevant” (T4), whereas the contributions of decision-makers were an “opportunity to answer the questions that they are prepared to act upon” (T4) and led the team “to try to develop early advice from existing evidence” (T4). In each of these ways, then, the team ended up pursuing its research differently than would otherwise have been the case.
