The impact of patient involvement in research: a case study of the planning, conduct and dissemination of a clinical, controlled trial

Abstract

Background

The interest in patient and public involvement (PPI) in health research is increasing. However, the experience and knowledge of PPI throughout the entire research process and especially in the analysis are limited. We explored ways to embrace the perspectives of patients in a research process, and the impact and challenges our collaboration has had on patients, researchers, and the research outcomes.

Methods

This is an explorative single case study of a Danish, clinical, controlled intervention trial and a nested intervention fidelity study included herein. Five patient representatives with metastatic melanoma were part of designing, undertaking and disseminating the trial where the effect of using patient-reported outcome (PRO)-measures as a dialogue tool in the patient-physician consultation was tested. In the fidelity study, audio-recorded consultations were analyzed after training in the Verona Coding Definitions of Emotional Sequences (VR-CoDES). Results were jointly disseminated at an international scientific conference. The outcomes, impact, and challenges were explored through a workshop.

Results

In the design phase, we selected PRO-measures and validated the dialogue tool. The information sheet was adjusted according to the patients’ suggestions. The analysis of the fidelity study showed that patients and researchers had a high consensus on the coding of emotional cues and concerns. The patients contributed with a new vocabulary and perspective on the dialogue, and they validated the results. PPI caused considerations related to emotional (sadness/sorrow and existential thoughts), administrative (e.g. arranging meetings, balancing work and small talk) and intellectual (e.g. avoiding information harm, continuing activities despite the death of patients) investments. A limitation of the study was the lack of use of a solid evaluation tool to determine the impact of PPI.

Conclusion

PPI throughout the process and co-creation in the analysis was feasible and beneficial. The case is unique in the degree of workable details, sustainability, and transparency. Moreover, the co-creation provides ideas of ways to operationalize PPI. An evaluation workshop revealed considerations about emotional, administrative and intellectual investments – best described as tacit, yet important ‘work’. This knowledge and experience can be applied to other studies where patients are partners in the research.

Trial registration

ClinicalTrials.gov ID: NCT03163433, registration date: 8th May 2017.

Plain English summary

Despite an increasing interest in patient and public involvement (PPI) in health research, the experience and knowledge of PPI throughout the entire research process and in particular in doing the analysis are limited. We involved five patients in designing, undertaking and disseminating a clinical trial. In the trial, the effect of using specific questionnaires as a dialogue tool in the consultation between clinicians and patients with metastatic melanoma was measured. In the trial’s design phase, patients were involved in selecting questionnaires in making the information sheet. Undertaking the trial, patients and researchers analyzed audiotaped consultations separately after receiving training together. We did this to see if the questionnaires were used as requested and to see if patients and researchers analyzed the consultations alike. We found agreement on the analysis, but the patients added a different perspective and new words to explain the dialogue than the researchers. A researcher and a patient presented the case of the PPI collaboration jointly at an international conference. At an evaluation workshop, some considerations emerged - like arranging meetings, avoiding information harm for patients and continuing activities despite the death of patients. These are emotional, administrative and intellectual investments and best described as tacit, yet important ‘work’. Patients had considerations about the sharing of sensitive topics, offering themselves in discussions, and how their work had contributed to the trial. This knowledge and experience can be applied to other studies where patients are partners in the research process.

Background

Patient and public involvement (PPI) in research refers to the inclusion and activation of patients or public laypeople as partners in the various stages of the research process, or as “research being carried out ‘with’ or ‘by’ members of the public rather than ‘to’, ‘about’ or ‘for’ them” [1]. There is increasing interest in PPI in health research [2, 3]. In particular, in North America, the UK, and Canada, PPI in health research is well established and supported by organizations and frameworks. These include Patient-Centered Outcomes Research Institute (PCORI) in the United States, INVOLVE in the UK, and SPOR in Canada [2,3,4]. In Denmark, PPI is still in its early stages, and hence not yet as well developed and incorporated in clinical research produced in Denmark as in the US, UK or Canada. Consequently, Danish researchers look at INVOLVE and others for guidance. However, several Danish patient associations as well as research funds in general have begun to require a statement of PPI in their call for research proposals. The Danish Cancer Society is one of the first patient organizations in Denmark to make specific requirements for PPI in applications for research grants; one study that received funding has since published an article on the subject [5].

Being a pioneer in the area of PPI, INVOLVE distinguishes between three PPI approaches: consultation, collaboration and patient-led research [1]. ‘Consultation’ refers to patients being asked about their opinion on specific areas chosen by the researchers; in ‘collaboration’ or ‘co-production’ patients and researchers work together in a more or less equal relationship, and in ‘patient-led research’ patients deliver and manage research themselves, with assistance from researchers [6]. Similarly, Health Canada divides PPI into five stages: inform or educate, gather information, discuss, engage and partner [7]. Methods and approaches for conducting PPI are applied with great variation and cover a broad range of areas, such as interviews, focus groups and surveys [1, 8].

Several different frameworks support PPI in research [9]. Especially three prevailing arguments supporting PPI exits: 1) patients have a right to have an input to research on their condition; 2) patients hold a lived-experience perspective that can improve the efficiency and value of research by increasing relevance, improving recruitment and retention rates, and improving dissemination of findings beyond academic audience; and 3) PPI increases the accountability and transparency of research and may be an effective way of attracting resources [9].

PPI in health research is believed to benefit quality, relevance and clinical results [1, 10], and can influence the choice of research topics and direction of research, project design and methods, recruitment and data collection, analysis, and dissemination [1, 11, 12]. Public involvement may also positively affect the researchers by making them reevaluate their thoughts on the research process [4, 11]. Studies show that, in clinical trials, patient involvement can lead to improvements in design, selection of relevant outcome measures [8] and effective recruitment of trial participants [13].

PPI in research can take place at every phase of the research project. INVOLVE has illustrated a research cycle in seven phases: identifying and prioritizing, commissioning, designing and managing, undertaking, disseminating, implementing, and evaluating impact [1]. Studies where PPI have been applied have mostly focused on early phases of the research cycle (e.g., prioritizing the research question, developing research design and planning a recruitment strategy) and on the dissemination of results [3, 8, 12]. However, since patients are rarely involved in the data collection or data analysis, little is known about how to engage patients in conducting a study [14]. Lack of PPI in this part of the research process, in which data is analyzed, may lead to the loss of valuable perspectives in the interpretation and validation of findings [6]. In addition, experience and knowledge of expanding PPI to several research phases or throughout the entire research process is limited [3, 8, 15].

An evaluation of the impact of PPI is warranted, not least in studies investigating the use of patient reported outcomes (PRO) in cancer consultations [3, 16, 17]. Furthermore, critical reflections, including both desirable and undesirable experiences of PPI, are lacking [3], and the reporting of PPI activities is often poor or omitted [12]. Unraveling some of the complexity of PPI impact can be done via the Public Involvement Impact Assessment Framework (PiiAF) [18] and the Guidance for Reporting Involvement of Patients and the Public (GRIPP) checklists (short and long format) [19, 20]. Each provide a comprehensive tool to improve the reporting of patient and public involvement in research.

Better descriptions are required on how, where and when engagement has been implemented, in order to improve understanding of PPI’s potential beneficial impacts [12] and to document how the activities in particular phases of the PPI process are done. In this article, we discuss the impact of engaging patients and researchers as partners in all phases of a clinical research project and, in particular, in performing the analysis. Furthermore, we examine: a) ways in which the perspectives of patients with metastatic melanoma can be embraced in health research, b) the impact of this collaboration on patients, researchers, and the research outcomes, and finally, c) the possible challenges and considerations that need to be addressed. The clinical research project is a Danish, clinical, controlled, intervention trial where patient representatives became patient research partners (PRPs) as members of a patient advisory board. We investigated the effect of using PRO-measures as a dialogue-based tool in consultations between clinicians and patients with metastatic melanoma. The intervention group completed PRO-measures before each consultation, and answers were fed back to their electronic medical records. Clinicians could then prepare for the consultation and focus the dialogue on what the patients found most important to discuss. The control group did not use PRO during consultations. Nested in the intervention, a pragmatic intervention fidelity study was included, see Fig. 1. Intervention fidelity refers to the degree to which an intervention is delivered as intended, and is a strategy to monitor and enhance the consistency and integrity of the intervention [21]. The intervention took place at a large cancer clinic at a university hospital while patients in the control group were recruited from another two Danish university hospitals.

Fig. 1

Overview of the clinical intervention trial

Methods

This is an explorative single case study, where the research cycle illustrated by INVOLVE served as a framework for engaging patients in every phase of the research project: “Feedback in the consultation – a dialogue-based tool in personalized care planning using systematic patient involvement”. During the research process, the level of involvement grew from consultant to collaboration and co-creation as described by INVOLVE [1]. We especially focused on the co-creation of the analysis, since this is one of the less explored phases of the research cycle. Thus, our study is “an empirical experiment in its natural setting without control over actual behavioural events by the investigator” [22, 23]. It thus embraces multiple sources of data and provides a highly detailed, contextualized analysis of “an instance in action” [22]. The GRIPP checklist (long format) was used as a guidance of reporting PPI in this paper, see additional file 1.

The case

The case (Table 1) represented the involvement of patients with metastatic melanoma throughout the research process. In the phases of “Identifying and prioritizing” and “Commissioning”, patients from the Danish Cancer Society were involved in prioritizing research topics and choosing which projects to fund. These phases are not described further in this case, because those patients did not engage in the advisory board of the clinical trial. Furthermore, the phases of “Implementing” and “Evaluating” are still at the planning stage, and will only briefly be covered.

Table 1 The case of PPI in the research project

Data material and analysis

The data material included multiple sources of data gathered from: email correspondences between PRPs and the PI; sticky notes about expectations and possible personal contributions; coding schemes; records of the proceedings, written by the PI and subsequently approved by all members; discussion-notes; and audio recordings from the evaluation workshop.

The coding and analysis of the pragmatic intervention fidelity study were made individually by all participants of the ad hoc sub-group, in accordance with the VR-CoDES, and were compared with Professor Finset’s codes of the same consultations. Consensus about the codes was reached after a discussion about our individual codes.

The analysis of the impact of PPI on PRPs, researchers and research outcome was qualitatively explored through all the above-mentioned sources of data, particularly from consensus on records of the proceedings and the workshop. At the workshop, fundamentals of the patient-researchers’ collaboration, challenges and outcomes of PPI were explored through dialogues moderated by an independent researcher who was not involved in the project, in order to illuminate any potential blind spots within the group.

Results

The involved patient research partners

Collaboration between patients with metastatic cancer and researchers has its immediate advances such as honest input from patients, inspiring, dynamic discussions, and new professional relationships. However, it also has its challenges. The patients’ cancer might progress, and they might get too ill to participate. Out of the five patients involved in this trial, only two patients are currently active partners in the advisory board. Patient characteristics of all PRPs at the time of joining the project can be seen in Table 2.

Table 2 Characteristics of patient representatives

It is apparent from Table 2 that the PRPs had various different motivations for joining the advisory board, and that none joined with a negative attitude or due to mistrust in the health system. They wanted to use their lived experiences in a meaningful way to help others and give something back to the system that had helped them.

PPI activities in the research process

In Fig. 2, an overview of the meetings in the advisory group and the ad-hoc sub-group is displayed in connection with the timeline and the research process.

Fig. 2

Overview of PPI-activities and time flow

In the “Designing and managing” phase, three meetings were held to choose PRO-measures for the dialogue tool and for the research project, to compose the patient information sheet, and to decide on a relevant design for the pragmatic intervention fidelity study. An additional meeting between the PI and the new PRPs took place to inform the latter about the advisory board and our common, future work within the group. The “Undertaking” phase comprised a joint training day for both researchers and PRPs, two consensus meetings, at which the codes were discussed, based on individual code-work done as homework, and an evaluation workshop. In the “Dissemination” phase, several meetings were held between the PI and one patient representative. This patient had been awarded a PRP scholarship to attend a conference together with the PI to present the work and thoughts on PPI. This PRP also engaged in the writing of the present article. During the research process, the PPI approach grew from consultation to co-creation. A detailed description of the activities in each of the phases, together with the levels of involvement, and the output and outcome, can be found in Table 3. PPI in the Undertaking phase will be further explored below, as an example of patient engagement in data analysis.

Table 3 Detailed overview of research PPI-activities, output and outcome

PPI in data analysis of the intervention fidelity study

The outputs by the ad hoc sub-group and the impact these had on the outcome, indicates that co-production in the analysis was feasible. The PRPs were in position to participate in a one-day training workshop and they subsequently individually applied the trained coding system to audio-recorded consultations of fellow patients. In interactive group discussions at two consensus meetings, we found that PRPs and researchers reached a high level of consensus about the clinical use of PRO and that PRO-data was used as intended in the audiotaped consultations. Most often, the physicians initiated the dialogue based on the PRO-data. The starting point of every dialogue were open-ended questions about the most important issues that the patient wanted to discuss. However, it was impossible to determine if PRO-data – such as patient answers regarding symptoms and functions – were used during the consultation. Additionally, we established a high consensus around the VR-CoDES. We generally coded unanimously and found no evidence of PRPs identifying emotional expressions that the researchers did not identify. The patients deemed the coded emotional cues and concerns to be identifiable and valid. This may indicate a robust coding system. However, PRPs contributed with new perspectives on how the physician-patient relationship and the patients’ position in their disease trajectories, determine the way emotional expressions and responses are articulated in the consultations. Furthermore, the PRP’ knowledge led the ad hoc sub-group to interpret words slightly differently to the system. For example, in the VR-CoDES, a patient’ use of swear words suggests an underlying unpleasant emotion, but in one of the audiotaped consultations, a patient cursed throughout the entire consultation including in sequences that the ad hoc sub-group deemed free of emotions (such as a discussion about the weather). This lead the PRPs to suggest that swearing was this patient’s way of speaking in general instead of an emotional cue. The same applied in another consultation, where the PRPs suggested that a patient’s use of humor in a description of symptoms could be linked to a geographical origin of the patient instead of linked to an unspecific way to describe an underlying unpleasant emotion. The experience with patient coding confirms the impression that coding according to VR-CoDES rules always should take context into consideration, and that input from patients regarding contextual factors may increase the validity of coding. Moreover, the PRPs broadened the views on the link between physicians’ responses to emotional expressions and their professional experiences of similar patients. By doing so, they used their lived experiences to reveal an understanding of the patient-clinician relationship that had been hidden to the researchers.

PPI-output, outcome, and impact

The above example shows how PPI impacted on how words and expressions were interpreted. The PRPs broadened our understanding of how emotional expressions are dependent on the patient-physician relationship and the disease trajectory. Besides these perspectives, the co-creation of the analysis also made sense to the PRPs. One of them said:

With this task, I felt my involvement in this project could make a difference – I was really engaged in and committed to identifying emotional concerns that the researchers and the coding system overlooked.

Even though the PRPs identified no additional emotional concerns, this statement indicated that they felt important throughout the process.

To capture the contribution of PPI on the research project, all decisions were written down in records of proceedings during the process. In addition, all participants in the evaluation workshop validated and extended the results. As illustrated in Table 3, contributions from PRPs were identified as outputs of PPI activities. For example, one output was the PRPs’ recommendation to include PRO-measures on psychological issues in the dialogue tool, and an endorsement of open-ended questions on the most important issues to discuss in the consultation. The PRPs deemed this to be extremely relevant. The output led to the selection of PRO-measures focusing on health-related quality of life, anxiety and depression, and included space for free text to fulfill the recommendations from the patients. Another output was the PRPs’ advice to arrange an information meeting to feedback results of the clinical trial to participants as a payback for their participation in the clinical trial. As a result, this output led to a notification in the written material report that all participants would be invited to an information meeting when the results of the clinical trial were available. These two examples of PRPs’ outputs and derived outcomes demonstrate new perspectives and the impact of PPI on the design and organization of the clinical trial.

The derived outputs and outcomes of PPI listed in Table 3 were all related to the research project and seen from a research perspective. However, in mapping the impact of PPI, the evaluation workshop revealed that outputs and outcomes did not make the same impact on everybody and that some outcomes of conducting PPI are not related to the trial study outcome but, rather, to the study process or the collaborating persons.

By coincidence, one of the PRPs received unwanted information on prognosis when listening to a fellow patient’s audiotaped consultation. This ethical dilemma was an outcome of co-creation that made an impact on all ad hoc sub-group members. We became concerned that other information could also cause harm (information harm). However, this had no impact on the clinical trial. Likewise, administrative elements, such as arranging meetings, determining salary and insurance and obtaining approval only impacted the researchers and had no impact on PRPs or the clinical project. The death of PRPs and the progression of the disease had an impact on both PRPs and researchers on a personal level. We were all affected and sad to realize that our fellow group members were no longer with us. In addition to the emotional and existential impact that the death of PRPs had on everybody, their deaths also had an impact on our common workload (introducing new members) and group dynamics (welcoming and getting to know new members), but none of these factors had an impact on the clinical project.

Challenges of doing PPI

PPI is a time consuming, emotional, cognitive, and practical effort. At the workshop, it became clear that all members faced challenges with PPI, even though there was variation in the challenges. Table 4 contains some of our challenges and some of the considerations these challenges call for – from both the PRPs’ and the researchers’ perspectives. From the researchers’ perspective, deciding which PRPs, how many and where from needed consideration in the recruitment phase. The balance between leaving the PRPs with a feeling of too little PPI leading to no significant impact or leaving them feeling overwhelmed with work and responsibilities involved much deliberation by the PI. Likewise, much time and effort were spent on keeping everybody informed so that they could make joint decisions in a genuine collaboration. Furthermore, it was challenging when PRPs progressed in their disease and died during the research process. The need to convey sad news to fellow advisory board members, acknowledge and process feelings of identification with the deceased patients’ disease trajectory, while, at the same time, maintaining and continuing the project activities and recruiting and informing new patients, meant additional considerations, time and effort.

Table 4 Examples of challenges and considerations of doing PPI

Moving from an involvement level of consultation to collaboration and co-creation meant an even bigger time investment and higher workload. Finding out about salary, insurance, and funding for the PRPs was not easy. As PPI is rather new in Denmark, there are no widespread guidelines, and what might be regarded as “good practice” regarding honoraria in some countries may not be applicable to a Danish setting where we have a long tradition of voluntariness in clinical research. The PRPs did not request or even expect honoraria, but we wanted to acknowledge their part in the analysis as equally important to the researchers. This is exceptional in Denmark, and since we were without much precedence in Denmark, we looked to INVOLVE for inspiration and offered salary according to the payment guidelines made by INVOLVE [27]. In conducting the analysis, ethical concerns arose around giving approval to PRPs to listen to tough consultations where fellow patients disclosed their fears. There was a risk of harm from being exposed to too much or unwanted information. Finally, additional time, energy and resource investments included coordinating the dissemination of results, applying for additional patient representative scholarships, preparing for joint conference participation and covering travel costs incurred.

From the patient’s perspectives, signing up for engagement in a research process for the first time is challenging in itself, as it involves entering a new work arena with potentially asymmetric relations. There were fears of not being able to fit in, understand and contribute to the research. Additionally, the balance between spending time and effort on a research project versus spending time with family also required consideration and the need for adaptation within the family.

As seen in Table 4, many of the considerations were related to administrative and intellectual investments. These represented investment in a meaningful PPI collaboration, but they were difficult to document, because the work was best described as tacit ‘work’. Despite the character of this tacit ‘work’, we found it to be just as essential as respect, sincerity, and openness in harnessing PPI. Maintaining authentic and sustainable collaborations between PRPs and researchers depends on respect, equality, and trust, which call for time and effort from everybody involved. Strong collaborations are not built overnight, they need nurturing. This includes meetings, ongoing correspondence, and constant follow-ups to maintain contact during periods in which no meetings are held.

Discussion

The case of PPI in the project “Feedback in the consultation – a dialogue-based tool in personalized care planning using systematic patient involvement” is an example of an authentic and sustainable operationalization of collaborations during the life cycle of a research project. Throughout the research process, the partnership was characterized by empathy and respect for each other, for our different backgrounds and contributions, and for our mutual project. The partnership lasted more than 3 years, despite the death and disease progression of some PRPs. To our knowledge, this is the first example of co-creation with metastatic cancer patients that report findings from such a long-lasting process.

Balance of PPI investment versus gains

PPI in health research requires huge investments (e.g., time, financial resources, and emotional and intellectual effort), not only by research organizations and teams, but also by the patients who get involved [28]. The sustainability of PPI in a project depends largely on the balance between investments in PPI and the gains, in terms of better research (relevance, feasibility, trustworthiness, credibility, adherence to protocol, etc.). In line with previous studies [12, 29], we found that PRPs’ contributions helped improve various aspects of the intervention, such as designing the questionnaires according to an estimated, patient-stated max time consumption of 15 min, selecting patient-requested PRO-domains, and producing user-friendly information. Additionally, the PRPs validated the findings in the pragmatic intervention fidelity study and led the ad hoc sub-group to interpret some words differently to the VR-CoDES. This could indicate that coding of audiotaped consultations can or should be interpreted in the light of its context, and that PRPs could contribute in that process. However, the tacit ‘work’ described earlier and the amount of planning that goes into the conducting of a research project with PPI (as illustrated in Fig. 2 ) indicate a need for a careful attention on the balance between investment and gain. The fact that no formal economic assessments were performed during the case confounds the picture of the balance.

PPI skills

Working collaboratively requires a range of skills, particularly various interpersonal skills and coordination skills – the latter which are required when handling the many administrative tasks displayed in Table 4. Interpersonal skills are necessary to establish and maintain a positive, conducive collaboration. Through the evaluation workshop, we found that respect, trust, honesty, and keenness towards PPI were fundamentals in a meaningful collaboration. This is in line with other studies [6, 30, 31]. Especially, when upgrading the level of PPI from consultation, which involves discrete, one-way interactions, to more relational approaches in co-creation, thought should be given to engaging in meaningful collaborations and establishing an atmosphere where sincere thoughts and feelings can be shared. This calls for a focus – beyond the project – on relationship building [32], such as allowing room for small talk and acknowledging and addressing well-intentioned, yet contradictory, ideas. Furthermore, it was found – in our case and previous studies – that the PI needed to frequently and clearly report back to PRPs about the ways in which their input was contributing to the study [11, 17].

Co-creation in the analysis

Considering the small number of studies on PPI in the analysis, and the absence of methodologies for collaborative data analysis [6], our pragmatic intervention fidelity study provides a meaningful and transparent contribution to the important knowledge gap on PPI in the data analysis. The involvement of patients in the analysis of transcripts has previously been done in a study on patients’ experiences of stroke and depression [33]. Among other aims of PPI in that study, one was to see if patients would identify themes (touchpoints) that were similar or different to the researchers. Like in our case, the patients and researchers identified very similar touchpoints. This may indicate that even though involving patients in data analysis is feasible, the added contribution of lived experience is hard to determine. The authors of that study concluded that even a small sample of data was too ambitious in terms of time available, suggesting that when involving patients in analysis, the focus should be on conversation and guidance in the further analysis, rather than data [33]. In our case, the solid training of the ad hoc sub-group contributed to a clear process, which enabled a deeper understanding of emotional expressions in clinical consultations based on PRO – and especially of open-ended questions as a facilitator of patient-centered consultations. Based on the codes and the PRPs’ lived experiences, elements such as patient-clinician relationship, disease trajectory, and professional expertise were highlighted by PRPs as determining elements of how emotions were expressed and responded to during consultations. These perspectives added a valuable layer to our understanding of the patient-clinician dialogue and were not solely a starting point or guidance for the analysis. In fact, this indicates that precious knowledge can be lost if PRPs are not engaged in data analysis. This is supported by another study on PPI in the analysis of data on service users’ discussions of Cognitive Behavioural Therapy (CBT) for psychosis [14]. The authors concluded that multiple coding is an important method for understanding and exploring multiple perspectives on data and building team consensus. We therefore recommend that the challenges of PPI in the analysis (e.g., information harm) should not prevent engagement in co-creation, however it must be articulated from the beginning as a possible condition of doing PPI.

Limitations

Our design was a single case study with a wide range of sources to inform our results and a high degree of detail. Generally, case studies are not perceived as strong evidence. There is nothing to which it can be compared, and we cannot rule out the possibility that the same results would have been found without PRPs.

Only five patients in total, and two patients in the analysis phase, participated as PRPs during the process. The fact that all PRPs had a higher-level educational status indicates that they possessed some level of methodological competencies in research, which they used. This could limit the transferability of our findings to other contexts where PRPs may have a lower level of education. Nevertheless, the small number of participants made possible a deeper relationship within the advisory board, thereby facilitating an open, honest and rich involvement.

Patients and clinicians in the pragmatic intervention fidelity study were aware of the audio recorder leading to a better compliance with the protocol than consultations not being recorded. Therefore, these consultations may not be representative of the actual use of PRO in the consultation. However, the audiotaped consultations could indicate an evaluation of clinicians’ adherence to and competence with the intervention, as well as the patients’ ability to understand how to use PRO and apply this in the consultation [34, 35]. The small number of audio recordings and the unblinded nature of the pragmatic intervention fidelity are considered limitations of the intervention, although they do not relate specifically to PPI.

Although a workshop had been proposed in an earlier study, to measure the impact of patient-engaged research [36], the robustness of such a workshop can be questioned, because no solid evaluation tool was used to determine the impact. At our workshop, the understanding of the impact was built through dialogues. The workshop revealed no potential negative impacts of PPI. Participants of the advisory board may have felt reluctant to speak critically about the case and PPI, given that they had developed it themselves and given that the evaluation was not anonymous.

Strengths and future perspectives

The use of INVOLVE’s definition of PPI and their research cycle provided an applicable and beneficial framework for the demonstration of PPI activity across the research process [1, 11], which is a strength of the study. This article and, in particular, the high level of detail in Table 3 can be used as inspiration when doing PPI in future studies. Lessons can be learned from our study. For example, we would consider mitigating the potential negative impacts of patients dying, suffering information harm, or dropping out during the PPI work by allowing a greater number of PRPs in future studies in order for the PPI work not to become vulnerable, and maybe designing studies where patients intentionally enter as PRPs at different time points during the process dependent on the tasks. Most of all, the importance of designing studies allowing enough time to focus on emotional well-being at every meeting, offering an open and honest dialogue, and having a plan for debriefing from the outset can be important lessons learned from this case. This goes for researchers who plan PPI, PRPs who may demand these elements, and patient associations or other funds that finance the projects.

Conclusion

Collaboration between researchers and PRPs throughout the research process was feasible and beneficial in this trial. Especially in the undertaking phase, the PRPs contributed substantially by co-creating parts of the analysis. An evaluation workshop revealed considerations about time and money investments that must be acknowledged. Particularly, the amount of tacit ‘work’ that PPI requires needs attention in order to balance the investment and the gains. Ethical discussions about the appropriate extent of involvement and potential harms to patients are also important considerations for future studies involving PPI. A small advisory board and patients with some level of methodological competencies in research contributed to a productive and constructive advisory board that shared sincere experiences. This case-study is somewhat unique, in that it had a high degree of workable details, sustainability, and transparency. Moreover, the co-creation through several phases of a clinical study provides an example of ways to incorporate patients’ perspectives in research. This knowledge and experience can be applied to other settings and studies where patients are partners in the research process.