Patient involvement in a qualitative meta-synthesis: lessons learnt
© Bayliss et al. 2016
Received: 28 October 2015
Accepted: 5 May 2016
Published: 12 May 2016
Plain English summary
Patients and researchers must work together to improve the relevance and quality of research. Qualitative systematic reviews synthesise findings from a range of published qualitative studies to identify common themes, and can make recommendations for practice or future research. The process of conducting a systemic review offers multiple opportunities for patient involvement.This paper explores the reflections of patient research partners involved in a qualitative systematic review. Patient partners were asked how their experience of the review process could be used to improve patient involvement in future qualitative systematic reviews. Following involvement in a systematic review an exploratory questionnaire was emailed to eight patient research partners. Open ended questions focussed on the training they had received, whether they had enjoyed taking part and how the process could be improved. Patients stated that they needed clear instructions and examples of how to take part in the systematic review. Face to face training was preferred, and it was important that patients were given enough time to complete the task. The study led to benefits for patients including gaining new skills and improved confidence. Each patient also wanted to know how their comments had influenced the paper and wanted feedback on whether they needed any further training. Through reflection with patient partners, recommendations for the involvement of patients in qualitative systematic reviews were developed to allow researchers to successfully involve patients in the review process.
Patient involvement in systematic reviews is seen as good practice, yet there is a lack of accessible standardised training for those involved. The aim of this paper is to inform the evidence base on effective ways of involving patients in a qualitative meta-synthesis. This process is evaluated and reflected by patient research partners (PRPs) who provided accounts of their experience.
An open ended questionnaire was emailed to eight PRPs who had participated in the analysis of a qualitative meta-synthesis. Questions focussed on the training they received, their experience of coding data and identifying themes, whether they enjoyed taking part in the project and how the process could be improved.
Our findings point to the importance of detailed training for PRPs, using plain English and clear examples of analysis techniques to improve confidence in engaging with meta-synthesis methods. Face to face training was preferred in order to discuss a PRP’s understanding of the task ahead. Time is an important consideration as PRPs often complete this work on top of their daily commitments and need the time and on-going support to be able to immerse themselves in the data. A focus group was a useful way to discuss the themes but it is important that PRPs understand how their comments have influenced the paper. PRPs reported benefits that included building new skills, improving confidence and gaining knowledge. They also asked for feedback on their contribution and any further training needs. All PRPs said they would take part in a meta-synthesis in the future as long as these considerations were addressed.
The recommendations for practice identified in this paper, and guidelines for training, can assist researchers in collaborating with PRPs when developing and conducting a qualitative meta-synthesis.
KeywordsPatient and public involvement Qualitative research Meta-synthesis Review Evaluation Co-researcher Co-production
The concept of patient and public involvement (PPI) in research is now well-accepted as a way to enhance the relevance, validity and quality of research . PPI roles often include patients drawing on their personal experience to suggest ways that could improve a project or service. However, some PPI roles, such as involvement in a systematic review, can also require specialist knowledge about the research process. This therefore goes beyond personal experience and asks the patient to invest time in training to develop their knowledge, skills and experience to facilitate effective input [2–7].
INVOLVE , which is the UK-based advisory body on PPI in health and social care research, encourages researchers to involve patients in systematic reviews by highlighting the benefits of this activity. For example, INVOLVE states that patients who know about the topic of the review and have first-hand experience of the disease and treatment can be involved in setting the scope of the review in terms of what is a priority to them; providing a different perspective on how to understand an idea; interpreting findings; and improving ‘readability’ of scientific language and jargon . Patients can be involved to different degrees from a consultation which includes a one off meeting to discuss the focus or content of a review , to a collaboration where patients are involved in the research design, data analysis and interpretation [9, 10]. However, this level of partnership is rare [9, 10].
Dudley and colleagues  report that very few patients are involved in systematic reviews and the evidence for how best to do it is still lacking. Most PPI training courses are developed within particular organisations or in the context of individual research projects , and there have been few empirical studies of research training needs for PPI contributors . This can mean that researchers often reinvent the wheel when involving patients in systematic reviews due to a lack of accessible standardised training [11, 12].
This paper aims to evaluate the patient experience of involvement in a qualitative meta-synthesis. This is important as the promotion of PPI activity in research, rather than its evaluation, has been the focus of much of the published literature . Previous projects have highlighted the value of listening to the accounts of PPI contributors in order to highlight good practice and areas in need of improvement . Furthermore, by evaluating the training developed for this study, the authors will help to ensure its relevance and therefore encourage uptake and a more consistent approach to PPI in systematic reviews.
In 2014, the authors collaborated with eight patient research partners (PRPs) to complete a meta-synthesis of qualitative studies to explore perceptions of predictive testing for those at risk of developing a chronic inflammatory disease . This study was part of the EuroTEAM project  and used meta-synthesis methods to investigate common themes and implications for future predictive testing programmes. The PRPs involved in this study were from a number of European countries (UK, Sweden, Estonia and Romania), and included PARE (People with Arthritis/Rheumatism in Europe) board members and founders and members of national rheumatic and RA patient organisations. The PRPs were initially recruited to give academic and industry partners feedback on research and provide an insight into the experiences of Rheumatoid Arthritis patients. They attended a PRP launch event as part of the wider EuroTEAM study, where they were made aware of what their role as a PRP would entail, and the nature of the research they were taking part in. This group were highly articulate, all female, and had previous experience of being a PRP as they had been involved in the larger EuroTEAM study, and contributed towards other research activities within that study. This meant that they had an understanding of the research process. An additional PRP was recruited from a Rheumatoid Arthritis service user group at the University of Manchester.
All PRPs were invited by email to take part in the meta-synthesis. The researchers did not collaborate with PRPs when developing the research question, although the PRPs were involved in the writing of the EuroTEAM grant application. PRPs also had some input into the search strategy as the researchers provided updates on the progress of the review via teleconference calls while the searches were completed and the papers selected. This helped PRPs to gain familiarity with the review process. A second email asked for three volunteers to help to develop a coding framework for the data in order to co-produce the themes with the researchers. Three qualitative papers were selected at random from the 11 papers included in the review and were sent to the first three volunteers.
In line with recommendations , we developed training that was tailored to the roles of patients, in the form of a set of written instructions on how to code the results sections (Appendix 1). It was necessary to develop written materials rather than face to face training as the PRPs were based across Europe. The training material was developed from the qualitative methods teaching material delivered to undergraduate students and trainee doctors with no qualitative experience. This included examples of qualitative research and meta-syntheses techniques. The material was shortened, and graphic representations were added. Although research experience varied within the group, the authors decided to prepare the materials with the view that all PRPs were new to qualitative research. This promoted consistency in the techniques that were used. The first draft of the materials was shown to a PRP, who highlighted any sections that were unclear or ambiguous. The authors then made the necessary changes before circulating. All PRPs were given three weeks to complete this task and were also given the option within the email of a phone call to talk through the guidance if required. Their feedback and recommendations supported the researchers’ coding framework and no substantial changes were made.
Do these themes and subthemes look logical? (For example, do the themes make sense to you as a person who has experience of a chronic inflammatory condition. Does the information flow?).
Based on your experience as a patient or relative do these themes reflect reality? (For example, do you think we have missed any themes in our analysis?).
Do you think there are any important concepts described within these subthemes?
Do you think that the overall themes or subthemes can be changed in anyway?
Do you think that our meaning/interpretation of the quotes within the papers can be changed?
What could be the implications of these findings for those at risk of developing a chronic illness in the future?
The aim of the session was to get patient feedback on whether they thought that the researchers had missed any important themes, or highlight whether they would have interpreted the data in a different way. The answers to the questions supported the themes presented by the researchers and highlighted the need to report clear recommendations for practice. PRPs helped to prioritise the data by choosing which quotes were included in the paper, organise themes and develop recommendation for practice, discussing the practicalities of how a new or improved intervention programme would work. Patients brought their own experience and helped the recommendations come to life, ensuring they were acceptable and valuable to patients. Considerations such as the emotional impact of any interventions were also discussed .
An open ended questionnaire (available from the authors) was developed to collect data about the PRP’s experience of collaborating with researchers to complete a qualitative meta-synthesis. Questions were developed from a review of the literature and discussions within the research team. The eight PRPs who worked on the meta-synthesis were sent this questionnaire by the researchers via email. Questions focused on the patients’ experience of training for this activity, what they gained from the experience and how the process could be improved. The email, which was sent in June 2015, also contained information on the process of the review to refresh their memory: what they had been asked to do and when, and the instructions they had received. All patients were given four weeks to complete the evaluation.
Analysis was inductive, using thematic analysis . Disconfirming evidence was actively sought and was used to modify emerging themes. The questionnaires were read, annotated and categorised independently by KB and RS. The researchers then met to discuss the themes. To support our interpretations, we present illustrative extracts from the data.
Feedback was received from six out of the eight PRPs involved in this research. Six themes were identified from the questionnaire data: understanding the research process; the need for face to face training; time provided for each task; the focus group to discuss themes; providing feedback and personal benefits for the patients.
Understanding the research process
The document is well laid out, is easy on the eye and encourages fluent reading. I particularly like the recommendations and considerations advice. The tips I suppose helps to prevent you from ‘wandering off’, the point. I find that both examples on how to summarise are explained very well, are simple and easy to follow (Patient research partner 3).
…the two different style examples of coding were very good (Patient research partner 1).
Yes. It was clearly laid out (Patient research partner 5).
I remember we were sent the explanatory materials and offered a skype or phone call talk with the researcher/the specialist, which was very helpful in letting me know what the researcher expects from me (Patient research partner 1).
Yes, some simplified explanations of the relevance and use of ‘themes’, ‘sub-themes’ and ‘coding’ would have been useful. An understanding of “a qualitative literature synthesis” is not a given (Patient research partner 4).
It was very difficult to comprehend fully research documents dealing with conditions other than RMDs [Rheumatic and Musculoskeletal Diseases] without an appropriate glossary etc. (Patient research partner 4).
The training consisted of reading the instructions ‘Patient involvement in the initial coding of data from selected qualitative papers’ which I found instructive and well written, but one thing confused me from the start and still do: the terminology. Coding is used in the title and summarise in the body text, but what I actually did I think of as writing comments on the text. English isn’t my native tongue so maybe I get more easily confused. I would like a short run through of the differences between these terms (to code, summarise, comment) and also an explanation to things like n = 9 that you immediately meet as you start reading the papers themselves (Patient research partner 2).
…I felt like I was doing an ordinary text analysis just as I do when translating drama, when you look for what is being said and what is really being said. I felt very unsure of whether that was what you wanted. Would have needed feedback (Patient research partner 2).
And I would have liked to receive on email (as a training material) an example of a paper coded as to answer exactly the researcher’s needs (Patient research partner 1).
The need for face to face training
Maybe a talk with the researcher and the other patient partners (so that we can raise questions). Also some training on how to read an article (for example, is it better to focus more on the Abstract, Discussion and Conclusion or is it better to read carefully the whole document in order not to miss important issues/ideas?) (Patient research partner 1).
Perhaps face to face (training) to explain the process and to answer questions [would be better] (Patient research partner 3).
I would have liked to go together with the researcher (face to face) through one page or so from the materials to be coded so that I am confident I’m doing the right thing (Patient research partner1).
Time provided for each task
I had to read it a number of times and come back to it quite often so about 3 hour’s ish (Patient research partner 3).
I remember that I didn’t do all the work at one sitting; it depended on the spare time I had (because of my part-time job and family commitments) (Patient research partner 1).
Instructions came very late together with the 11 papers. An immense amount of papers with too short a deadline. I’m sorry but I had no time to read or prepare before the meeting. To be able to do this well together with our ordinary jobs at least two months (11 papers!). Thinking needs that time I think (Patient research partner 2).
Sending 11 documents for reading before a meeting the following week was unrealistic, for me at least. In fact there was so much data that my inbox would not accept it (Patient research partner 4).
No, I didn’t [have enough time]. I remember I was working the whole duration of the flight. Taking into consideration that there were many articles, I would have liked to have two-three weeks at my disposal (this is also because of my part-time job and the fact that sometimes I get to work only at weekends) (Patient research partner 1).
The focus group to discuss themes
Yes, indeed. The researcher conducting the group was very attentive in offering everybody the opportunity to speak (Patient research partner 1).
Yes [I had an opportunity to share my views]. Relaxed group (Patient research partner 5).
Maybe working with a white board more and writing down patients’ ideas so that have a holistic view… A synthesis of the group discussion so that patients see if and what they were able to contribute to the draft paper, would have boosted patients’ confidence in their input and knowledge/value (Patient research partner 1).
I found this a rather confusing… I had no idea what coding was, how the themes had been chosen, nor how they might be used, or really what we were trying to achieve. I think the whole process was a bit opaque (Patient research partner 4).
If the articles had been sent [further] in advance, the researcher could have summarized everybody’s view/ideas in the “Draft Themes and Sub Themes” and when gathering together as a focus group we could have only rated and improved the document (Patient research partner 1).
A quick feedback after paper 1 to make sure I was doing what you wanted, with further requests from the researchers if needed. We PRP’s want to help the best we can (Patient research partner 2).
I would have liked some quick feedback… to get a notion of whether I could improve anything, whether I had understood the instructions (Patient research partner 2).
I would have liked to have another call with the researcher [to] explain to me what was good and what was wrong and give me examples/ideas on how to improve what was wrong (Patient research partner 1).
For the future it would be very helpful if we can fill in the evaluation forms in one or two months after completing the activity (Patient research partner 1).
Personal benefits for patient research partners
Taking into consideration that it was the first activity of that kind in my experience as a Patient Research Partner, I was very happy and thankful to gain experience in coding. I improved my skills in summarizing, finding key words and interpreting. I gained confidence in participating to future coding activities. It was also very interesting to see how other people think about RA and prevention – very educative (Patient research partner 1).
I feel I am contributing and being nosey; I get to find things out (Patient research partner 3).
[Benefits included the] sharing of ideas and experiences (Patient research partner 6).
Certainly, yes! [I would take part again] I liked it (Patient research partner 1).
To be honest, I’m not quite sure. I don’t know… if I was useful really, not in a general way but more specific. I think the idea of involving patients in this kind of work is potentially very good, but I had a constant lurking feeling of just being an amateur scientist (Patient Research partner 2).
This study provides an evidence base on effective ways to involve PRPs in a qualitative meta-synthesis. This follows the recommendations of previous research which highlights a need for quality standards and recommendations for good practice in the involvement of patients and carers in this area [18–21]. The importance of this work is also highlighted by Harris et al.  who states that a lack of involvement in systematic reviews may compromise the validity of the reviews as researchers may not always identify all of the important themes.
Patient research partners were asked to evaluate their experience of completing training to collaborate with researchers to develop a coding framework; complete a thematic analysis of qualitative data; and advise on the layout and focus of a paper. We have used the insights of PRPs to generate the recommendations below which may help researchers to involve patients in future systematic reviews. These recommendations should be considered alongside existing guidance on PPI , and may also assist funding bodies when reviewing whether plans for PPI in a qualitative meta-synthesis are feasible.
Lessons learnt: recommendations for PPI in a qualitative meta-synthesis
aEnsure that all training materials use plain English and include an appropriate glossary.
aEnsure that you add adequate time and resources to your project for meaningful PPI.
aIdentify individual support and training needs before the project starts. For example, the group may contain people with varying understanding of the research process or individuals with English as their second language. Providing clear instructions on what is expected from each patient research partner (PRP) avoids confusion and the feeling that they are acting as a ‘pseudo scientist’.
Always ask PRP’s how much time they have to give to the project. For example, if your literature search finds a large number of relevant papers, does each individual have the time to read these? Negotiate the level of involvement that is suitable for your PRPs and the specific project.
When asking PRPs to complete a thematic analysis, provide clear examples of the feedback that is required. This can include the level of detail and format of their response. An analysis template can be used to avoid PRPs spending unnecessary time on a task and improve the consistency of feedback so it is easier to collate these data.
aA phone call to each PRP after they have been sent any written instructions would be beneficial. Be proactive and do not wait for the patient to contact you with any questions.
aFace to face was seen as the preferred method of training in this study. If it is not possible to get all PRPs together then a video conference or live online training session are good alternatives.
aAssessed questions at the end of the training session can highlight whether PRPs have understood the training information and ensure that they have an understanding of the task ahead. Further training may be required on areas where they remain unsure.
Ask the PRPs what they believe a suitable time frame would be to complete each task considering existing commitments. The PRPs in the current study believe that three weeks was sufficient to code three papers, and two months should be given to analyse 11 papers.
It is beneficial to provide the themes identified by the researcher and the questions that patients will be asked before the focus group. This provides a template for patients to work from when completing their own analysis and preparing for the meeting. Patient research partners can then state whether they agree with the themes identified, if anything is missing, how to structure the results section of the paper and highlight priorities.
A focus group was useful and a good way to discuss the findings of the thematic analysis. Eight PRPs was a good number to allow a meaningful discussion around the themes identified.
Write up all feedback on a flip chart or white board during a focus session and circulate comments.
aThank PRPs for their involvement and highlight the value of their input. State how their contributions will affect the paper.
aAsk patients whether they would like feedback on their contribution, including any further training needs.
aEvaluate the project at each stage. This helps to identify support needs as the project progresses rather than waiting until the end.
aEvaluate the project as a whole as soon as possible following completion so the PRPs are able to clearly recall their experiences.
Comparisons with existing literature
This study developed a set of training materials for PRPs (Appendices 1 and 2) with the aim of supporting a collaboration between PRPs and researchers in a qualitative meta-synthesis. An evaluation of these materials found them to be clear and helpful, however, PRPs also highlighted that some amendments should be made to these documents if they are to be used as a template for future projects. For example, a glossary should be added to clearly explain all terminology, and a small assessment and evaluation at the end of the training could also identify any further training needs. An analysis template with a clear example of the level of feedback required would also be beneficial and should be developed with PRPs to add to the training. Furthermore, PRPs were recruited from across Europe therefore geographical distance and time constraints meant that training was done remotely by email. However, PRPs stated that face to face training would have been preferred. Buck et al.  found that face to face meetings were more conductive to forming relationships with PPI contributors than teleconferences. Therefore a video conference may be a preferred option when PRPs are in different countries. These recommendations are in line with Noe et al.  who state that a key consideration for any training is that it engages with the diversity of learners’ needs and is meaningful from their perspective.
Not all PRPs in this study were able to develop an understanding of the research methods due to the existing commitments and limited one to one support. These individuals therefore relied more on their personal experience to make comments and suggestions on the paper rather than analysing the data. This level of involvement is reflected in a systematic review completed by Rees and Oliver  where researchers produced a systematic map of the literature and a group of PRPs decided what to focus on and which outcomes should be prioritised. Although PRPs were able to use their personal experience to comment on the paper, they may have had greater input if they had had more time to develop meta-synthesis techniques and analyse the data.
As conducting a review can be very time consuming, PRPs may prefer to develop the focus of the review rather than being involved in the analysis of the data. It is therefore important to negotiate the level of involvement with PRPs before the study begins to ensure that involvement is meaningful and not simply tokenistic [6, 11, 16]. This is supported by Harris et al.  who found that although sustained and equal involvement of PRPs in the review process substantially improved the quality of the review, the level of participation should be flexible considering issues of time and cost, researcher skills and experience in PPI; and the circumstances in which the review is funded. This issue points to a ‘know-do’ gap, whereby researchers’ talk of the importance and value of PPI in the ‘ideal’ world in contrast to their experiences of ‘the reality’ of implementing PPI in practice where time restrictions can form barriers to PPI . By considering what is achievable in a given time frame the PRPs and researchers can work together to consider where PRP input is most valuable for individual projects [11, 13].
The authors had very little disagreement with PRPs on the analysis of data during this meta-synthesis. However, if more time had been available for the PRPs to complete the analysis before the focus session, more significant changes to the themes may have been suggested. Further guidance should be developed on what to do when PRPs and researchers have contrasting views about data. For example, Robinson et al.  found that it can be difficult for researchers to relinquish control and work with patients to develop themes. Training may be beneficial for researchers on how to meaningfully involve patients in a meta-synthesis/review and manage this distribution of power. Further research is therefore necessary with PRPs to develop training allowing contributors and researchers to learn from each other . Consideration might also be given to training PPI contributors and researchers side-by-side . Buck and colleagues  also state that PPI in systematic reviews could also be extended to involve patients in the distribution of research findings to relevant groups.
Strengths and limitations
To our knowledge, this is the first study to evaluate PRP training and involvement in a qualitative meta-synthesis. The insights of PRPs can be used to help researchers plan PPI in a qualitative meta-synthesis. This follows the recommendations of a recent study of the UK health and social care research community which informed the development of a Public Involvement Impact Assessment Framework (PiiAF). This emphasises the value of well thought-through planning before implementing PPI as well as the subsequent evaluation of its impact .
Previous research has found that it can be difficult to find and engage patients with an interest in and understanding of the research . However, the researchers in this study had access to articulate patients, who were already fully engaged in the topic as they had previously worked on other RA projects. This meant that they had an understanding of the research process and the constraints on researchers . Enany et al.  state that PPI contributors with higher levels of education may be prone to over identify with the perspectives of researchers rather than challenging them . However, patients do need certain skills to be able to contribute to systematic reviews which include the need to be able to articulate ideas and listen to other people’s ideas and work in a group. Resources must also be considered. The PRPs in this volunteered as part of the wider EuroTEAM project,  and therefore received no payment. If researchers need to recruit PRPs for specific projects, it is recommended that they follow the INVOLVE guidelines for payment .
PRPs in this study were involved in coding the data, and developing the themes. It was not possible to collaborate with PRPs in the earlier stages, although the patients were consulted about the search strategy and were involved in the writing of the EuroTEAM grant application. Recruiting a PPI co-applicant when writing the research proposal would allow PPI contributors to collaborate with researchers and guide the focus of the meta-synthesis. This is important as Fudge et al.  suggest that decisions made by professional researchers at the outset of a study have a cumulative and significant influence on the potential for PPI to impact on a study and that involvement is more difficult to achieve once studies are under way.
When reflecting on the level of control in the review process, it is important to recognise that the researchers trained the PRPs in coding so the initial phases were very much in the control of the researchers. However, as the PRPs in this study were allowed to code the data independently, they were in full control of the themes they developed. When the themes were incorporated into the researcher’s analysis, the direction, and therefore the level of control, was shared. Although the authors’ acknowledge that a PRP lead method may produce different results, this may not be as methodologically robust as PRPs are not fully trained qualitative researchers.
It is important to note that this evaluation was conducted by the researchers involved in the meta-synthesis. PRPs may have been more critical if they were asked to evaluate the study by a third party. Nevertheless, public contributors did provide some critical accounts so it seems unlikely that they struggled to articulate their own perspectives regarding training. This study also focuses on patient involvement in a qualitative meta-synthesis. Further research is needed to evaluate the involvement of PRPs in other types of systematic reviews including realist, effectiveness and quantitative meta-synthesis .
This study adds to the existing INVOLVE guidance by informing the beginning of an evidence base on best practice regarding PPI in systematic reviews. Patient research partner insights have generated recommendations which may help researchers to involve patients more effectively in future systematic reviews. Suggestions to improve training materials for PRPs were also made. Continued monitoring and evaluation of PPI during and after systematic reviews and meta-syntheses will help researchers to optimise PPI. However, the need for on-going flexibility in each project should also be recognised in order to accommodate the unexpected and respond to opportunities and difficulties as they arise.
Consent to publish
The authors have obtained consent to publish from the patient research partners to report individual data.
This work was supported within the FP7 HEALTH programme under the grant agreement FP7-HEALTH-F2-2012-305549 (Euro-TEAM) and by Riksbankens Jubileumsfond (The Swedish Foundation for Humanities and Social Sciences) under Grant M13-0260:1 ‘Mind the Risk’. The authors would like to thank the patient research partners for their involvement in this paper.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Rose D. Patient and public involvement in health research: ethical imperative and/or radical challenge? J Health Psychol. 2014;19:149–58.View ArticlePubMedGoogle Scholar
- INVOLVE. Developing Training and Support for Public Involvement in Research. Eastleigh: INVOLVE; 2012.Google Scholar
- INVOLVE: Involving the public in systematic reviews: What does the evidence tell us? Conference 2012. http://www.invo.org.uk/posttypeconference/involving-the-public-in-systematic-reviews-what-does-the-evidence-tell-us/. Accessed 29 Sept 2015.
- Snape D, Kirkham J, Preston J, Popay J, Birtten N, Collins M, et al. Exploring areas of consensus and conflict around values underpinning public involvement in health and social care research: a modified Delphi study. BMJ Open. 2014;4:e004217.View ArticlePubMedPubMed CentralGoogle Scholar
- Stewart R, Liabo K. Involvement in research without compromising research quality. J Health Serv Res Policy. 2012;17:248–51.View ArticlePubMedGoogle Scholar
- Staley K. Exploring Impact: Public Involvement in NHS, Public Health and Social Care Research. Eastleigh: INVOLVE; 2009.Google Scholar
- Boote J, Baird W, Beecroft C. Public involvement at the design stage of primary health research: a narrative review of case examples. Health Policy. 2010;95:10–23.View ArticlePubMedGoogle Scholar
- Rees R, Oliver K, Woodman J, Thomas J. The views of young children in the UK about obesity, body size, shape and weight: a systematic review. BMC Public Health. 2011;11:188.View ArticlePubMedPubMed CentralGoogle Scholar
- Boote J, Baird W, Sutton A. Public involvement in the systematic review process in health and social care: a narrative review of case examples. Health Policy. 2011;102(Suppl 2-3):105–16.View ArticlePubMedGoogle Scholar
- Liabo K, Grey K, Mulcahy D. A systematic review of interventions to support looked-after children in school. Child Fam Soc Work. 2013;18 Suppl 3:341–53.View ArticleGoogle Scholar
- Dudley L, Gamble C, Preston J, Buck D, EPIC Patient Advisory Group, Hanley B, Williamson P, Young B. What difference does patient and public involvement make and what are its pathways to impact? Qualitative study of patients and researchers from a cohort of randomised clinical trials. PLOS One. 2015. http://www.plosone.org/article/Metrics/info:doi/10.1371/journal.pone.0128817. Accessed 29 Sept 2015.
- Dudley L, Gamble C, Allam A, Bell P, Buck D, Goodare H, Hanley B, Preston J, Walker A, Williamson P, Young B. A little more conversation please? Qualitative study of researchers’ and patients’ accounts of training for patient and public involvement in clinical trials. Trials. 2015;16:190.View ArticlePubMedPubMed CentralGoogle Scholar
- Buck D, Gamble C, Dudley L, Preston J, EPIC Patient Advisory Group, Hanley B, Williamson PR, Young B. From plans to actions in patient and public involvement: qualitative study of documented plans and the accounts of researchers and patients sampled from a cohort of clinical trials. BMJ Open. 2014;4 Suppl 12:e006400.View ArticlePubMedPubMed CentralGoogle Scholar
- Bayliss K, Raza K, Simons G, Falahee M, Hansson M, Starling B, Stack R. Perceptions of predictive testing for those at risk of developing a chronic inflammatory disease: a meta-synthesis of qualitative studies. J Risk Res. In Press.
- EuroTEAM: http://www.team-arthritis.eu/. Accessed 29 Sept 2015.
- Oliver K, Rees R, Brady LM, Kavanagh J, Oliver S, Thomas J. Broadening public participation in systematic reviews: a case example involving young people in two configurative reviews. Res Synth Methods. 2015;6(2):206–17.View ArticlePubMedGoogle Scholar
- Huberman AM, Miles MB. Data management and analysis methods. In: Denzin NK, Lincoln YS, editors. Handbook of Qualitative Research. Thousand Oaks: Sage; 1994. p. 428–44.Google Scholar
- Braye S, Preston-Shoot M. Emerging from out of the shadows? Service user and carer involvement in systematic reviews. Evid Policy. 2005;1(2):173–94.View ArticleGoogle Scholar
- Boote J, Baird W, Sutton A. Involving the public in systematic reviews: a narrative review of organizational approaches and eight case examples. J Compr Eff Res. 2012;1(5):409–20.View ArticleGoogle Scholar
- Kreis J, Milo PA, Schünemann HJ, Dickersin K. Consumer involvement in systematic reviews of comparative effectiveness research. Health Expect. 2013;16(4):323–37.View ArticlePubMedGoogle Scholar
- Harris J, Croot L, Thompson J, Springett J. How stakeholder participation can contribute to systematic reviews of complex interventions. J Epidemiol Community Health. 2015;0:1–8.Google Scholar
- Noe RA, Tews MJ, McConnell Dachner A. Learner engagement: a new perspective for enhancing our understanding of learner motivation and workplace learning. Acad Manag Ann. 2010;4:279–315.View ArticleGoogle Scholar
- Rees R, Kavanagh J, Burchett H, Shepherd J, Brunton G, Harden A, Thomas J, Oliver S, Oakley A. HIV Health Promotion and Men who Have Sex With Men (MSM): A Systematic Review of Research Relevant to the Development and Implementation of Effective and Appropriate Interventions. London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London; 2004.Google Scholar
- Ward PR, Thompson J, Barber R, et al. Critical perspectives on ‘consumer involvement’ in health research: epistemological dissonance and the know-do gap. J Sociol. 2010;46:63–82.View ArticleGoogle Scholar
- Robinson L, Newton J, Dawson P. Professionals and the public: power or partnership in health research? J Eval Clin Pract. 2012;18:276–82.View ArticlePubMedGoogle Scholar
- PiiAF Study Group. The Public Involvement Impact Assessment Framework: executive summary. http://piiaf.org.uk/documents/exec-summary-0114.pdf Accessed 5 Mar 2014.
- Enany NE, Currie G, Lockett A. A paradox in healthcare service development: professionalization of service users. SocSci Med. 2013;80:24–30.View ArticleGoogle Scholar
- Fudge N, Wolfe C, Mc Kevitt C. Assessing the promise of user involvement in health service development: ethnographic study. BMJ. 2008;336:313.View ArticlePubMedPubMed CentralGoogle Scholar