Involving service users in the qualitative analysis of patient narratives to support healthcare quality improvement

User-centred narratives and quality improvement

Patient experience is defined as one of three components of quality in healthcare, alongside patient safety and clinical effectiveness. Improving people’s experience of healthcare has been highlighted as a priority for successive governments in the United Kingdom [1]. A number of specifically user-centred quality improvement approaches drawing on patient experience data have been developed, such as Experience-Based Co-Design (EBCD) and Patient and Family Centred Care [2–4].

Despite a substantial amount of evidence about what matters to people about their care, until recently organisational initiatives have commonly focused on collecting more patient experience data (typically through surveys) and measuring performance, rather than using it for quality improvement [5]. This situation is gradually changing, but clinical staff keen to use patient experience data for improvement may lack organisational support to do so. They need expert help sifting through such data to identify themes and priorities for action, and implementing quality improvement [6, 7]. This may be particularly true of narrative data. As Martin et al. [8] note, making sense of quantitative data is challenging enough; working out how to turn the ‘untamed richness’ of ‘soft’ forms of intelligence such as narrative and observational data into action for quality improvement may be even more daunting.

Bringing service user partners into the analysis process may be one way to help achieve this. We involved users in a secondary data analysis project which aimed to develop new resources to support EBCD.

Patient and public involvement and data analysis

Patient and public involvement (PPI) in health research is a well-established principle, meaning research is conducted with or by users, rather than to, for or about them. Service-user-researchers, who collect and sometimes analyse research data, are more common in some research fields than others, notably mental health. However, more typically PPI involvement has been limited to advising on research questions and research design, leaving professional researchers to complete data collection and analysis. PPI in data analysis is perhaps one of the most challenging and least well explored aspects of involvement. Analysing patient stories may appear more intuitive and approachable than, say, quantitative analysis such as logistic regression or mathematical modelling. At the same time, both the volume of the data generated and the need for the analyst to be alert to theory as well as patterns in the data can be a hurdle.

There are comparatively few articles documenting the process of and rationale for service user involvement in data analysis. A systematic review [9] found many examples of lay people becoming involved in the conduct, design or dissemination of research, but it was less common for them to be involved in execution or translation of the research. This echoes findings that ‘examples are few of participatory interpretation and analysis of data’ [10]. There are however some interesting examples of service user involvement in the analytic process, which focus both on the process of ‘doing qualitative analysis’ with lay partners, but also offer insights into the benefits and challenges. These challenges include how much training is required and to what level of detail; practical constraints and demands of reading through long transcripts; how to recruit user partners able to engage with the perspectives of the specific study population; and how to deal with the emotional impact of reading potentially difficult stories [11–13].

Whitmore [14] cautions against expecting lay partners to write for an academic audience (which could be ‘unfair and unrealistic’), and distinguishes this from involvement in interpretation and illuminating the meaning of participants’ accounts. The knowledge and understanding that lay partners bring to the table has been argued to offer clear benefits, and to counter the interests, assumptions and perspectives which researchers themselves bring to analysis.

The authors note that while studies using ‘consultation’ may appear to be the least democratic approach, they also ‘tended to have people with lived experience in their research team….[and] moved beyond the binary categorisation of researchers as academic or service users’ [15 p.4]. One example used ‘multiple coding’, with one clinical researcher, one psychologist and one service user researcher each independently coding focus group transcripts to identify outcomes for Cognitive Behavioural Therapy for psychosis [16]. The results showed a high degree or consensus between the analysts but also new themes, and ‘points of fracture or non-consensus’ (p.e96) which had to be resolved, in some cases by returning to individual research participants to clarify what they had meant.

An example of the gold standard ‘development and application’ approach is Cotterell [17]. To support his study of the needs of service users with life limiting conditions he held repeated interpretation and ‘theme generation’ sessions over the course of several months with a user panel. The researcher’s initial seven themes were revised to eight themes, only two of which (‘diagnosis’ and ‘relationships’) remained the same. Strikingly the revised themes included more emotional categories than the original set, including ‘fear’, ‘anger/frustration’, and ‘grief’. Garfield et al. [18] describe a lighter touch process of involving lay partners in analysis of qualitative data, by giving people a sub-set of transcripts and inviting them to come up with their own themes after brief training. These mapped closely onto the themes already identified in the researcher-led analysis, providing useful confirmation, but also identified one new theme. The final output of the research was presented as a ‘synergy of perspectives’ that were not easily visible or separated.

Best et al. [19] tested the Participatory Theme Elicitation or PTE method, with a youth advisory panel of 8 members. The research team selected 40 focus group data extracts, from a study of school-based physical activity, that ‘could be easily understood and interpreted as standalone statements’ (p.3). The young people involved were invited to sort these into thematic piles. These were then recorded and grouped by researchers, and the output was then compared with the researchers’ thematic analysis. Echoing Garfield [18], the authors found that ‘while PTE analysis was, for the most part, consistent with the researcher-led analysis, young people also identified new emerging thematic content’ (p.1). They note that approaches to involving people in analysis need to be accessible, and that some techniques (such as Byrne et al’s [10] experimentation with the Voice-Centred Relational or VCR Method) are too elaborate and time-consuming in terms of training and analysis time. Byrne et al. themselves identify that teenagers involved in the VCR method - which involves repeated attentive reading of transcripts and the development of an emerging narrative -found it ‘difficult, tedious and time consuming’ p.74).

Some researchers who have involved users in analysis have expressed concerns about lay partners potentially putting too much emphasis onto their own experiences rather than analysing what came out of the data [18]. Cotterell [17], however, records that his early concerns about ‘the ability of the group to remain ‘objective’ and to refrain from blurring interpretation of data with their own personal experience and concerns’ proved unfounded. He concludes that service users with similar concerns and experiences as those who are the focus of the study can bring a reflective lens to the process and a greater capacity for the analysis to truly ‘unpack the taken for granted’. One might add that researchers also bring their own biases about what they see in the data and what they regard as important.

Fisher [20] argues that involvement of lay partners in analysis can uncover nuances which may not be obvious to a researcher. Disabled user researchers drew his attention to a single passage in one transcript which for them gave an important sense of how users feel judged and oppressed by benefit assessment panels, but to which he might not otherwise have given much weight. This demonstrated how insider sensitivity to the nuance of language can bring a level of understanding to the process that researchers may not attain, even after long immersion within a community. Similarly, Gillard et al. [21] suggest that service user involvement in data analysis added ‘expertise by experience’ to the process, and had changed the conventions of ‘doing research’ to some degree by the inclusion of challenging voices, as well as adding a more nuanced set of themes that came from their differing points of reference.

What is experience-based co-design (EBCD)?

EBCD is a participatory action research approach to quality improvement based on active patient-staff partnership [2]. It has three phases: ‘discovery’, ‘co-design’, and implementation.

The discovery phase, which would normally last around 6 months, involves interviews with staff; observations of a particular care pathway; and video-recorded interviews with local service users and family members about their experiences of care. These interviews are analysed by a researcher (who may be an academic or a healthcare professional) to identify ‘touchpoints’ – these are key moments of interaction between person and service where something could have been done better, or which exemplify good experience. Video clips showing these touchpoints are selected and edited into a ‘trigger film’. Examples of touchpoints might include an account of insensitive communication; a memorable act of kindness or emotional care; disturbance from noise or lighting on the ward; particular efforts to ensure privacy and dignity.

In the co-design phase, the trigger film is shown first to a workshop for users and family members only, and then again to a joint workshop with healthcare staff. The aim of the film is to trigger discussion about local quality issues and agree a set of improvement priorities, which are then addressed through small co-design working groups (involving users and staff as equal partners). The whole EBCD process typically takes 12–18 months to complete [22].

Although evaluations have shown EBCD to be effective, they have also identified the time and resources it takes as barriers to widespread adoption [22]. In a previous study [23]we therefore tested an ‘accelerated’ version of EBCD in two lung cancer services and two intensive care units, in which we replaced much of the discovery phase with interviews from existing national interview collections, held by the Health Experiences Research Group, University of Oxford [2] and disseminated on the Healthtalk.org website through a series of lay-friendly ‘topic summaries’. Thus instead of trigger films made from local user interviews, we identified touchpoints from existing interviews which had been collected around the country, and made films which can be used in multiple locations. However, in common with other EBCD projects, this did not include direct user involvement in analysis.

Rationale for the new project

In this paper we focus particularly on this second area, and describe the work we undertook with user partners in analysis workshops. We conclude with recommendations developed with our workshop attendees for taking the process forward and our reflections on it.

As well as having a service user co-investigator on the team [GS], we organised two workshops to explore involvement in data analysis, with two service user panels; one panel member has also co-authored this paper [LB].

Initial researcher-led analysis

Secondary analysis [24] was undertaken on two existing HERG interview collections: ‘Experiences of stroke’, and ‘Experiences of young people with depression’. These topics were among several selected for re-analysis in consultation with our research partner at NHS England (NC) as national priorities. We focused on these two topics for user involvement because they offered diversity in terms of both the age groups affected and a mix of mental and physical conditions. Re-analysis of both the original transcripts for each study (n = approximately 40 for each study), and the summarised topic sections on the healthtalk website, was first undertaken by a researcher [SK]. This focused on identifying ‘touchpoints’ of care (described above), coding for narrative extracts that demonstrated ways in which service experiences impacted either positively or negatively on people’s overall experience of care, using a framework analysis [25, 26].

User analysis workshops

We then sought to sense-check the analytic themes identified by the researcher in two workshop sessions with users. The aim of the workshops was to help us understand more fully to what extent service users might become involved in the development of trigger films and to consider how this might bring new insights to the process of data analysis for trigger film development. We also anticipated that the workshops would help us to think more broadly about the practical feasibility of involvement at this level, gaining feedback from those involved on how they felt about taking part in such activities, and whether they might suggest other ways of working.

Researchers SK AND LL approached groups and organisations associated with people with lived experience of both health conditions (young people with depression, and stroke), using a flyer and poster. People interested were provided with details of how to contact the team and discuss any questions before deciding whether they would like to be involved. Arrangements for each workshop were made to fit with the needs of each group, involving a short day (approximately 10 am to 3 pm) of activities associated with qualitative data analysis, including refreshments, lunch and social time. Everyone was paid for their time and travelling expenses.

At the end of both workshops, we asked people for any verbal feedback on the process of being involved which could help us improve the process in future, and some people also sent us further feedback by email afterwards.

The format originally for the workshops included a brief training introduction to qualitative analytical methods; an explanation of EBCD, trigger films and the concept of touchpoint;, followed by participants reading interview transcript extracts and website summaries to identify touchpoints. Workshops were facilitated by the Principal Invesitgator (LL) and lead researcher (SK), with support from a research assistant. Each is described below.

Workshop one; young people’s experiences of mental health services

Workshop 2 – Experience of stroke, or caring for someone who had had a stroke

For this workshop SK and LL provided similar but shorter presentations to the group about qualitative research, healthtalk and EBCD. Next, instead of analysing interview transcripts, the group were asked to identify what they thought of as key touchpoints for stroke care, based on their own experiences. We then asked them to review a website ‘topic summary’ on stroke experiences, again looking for touchpoints. Towards the end of the session we asked the group to watch a draft half-hour film that had been pre-prepared by SK after completing her independent analysis. We asked them to consider how well it encapsulated the touchpoints they had identified earlier in the day.

Workshop 1 – Young people and depression

Our reflections on workshop 1 were that both the informative sessions in the morning and the subsequent analysis work in the afternoon took far longer than we had anticipated. Having prepared four transcript extracts and five topic summaries, we only managed to go through one transcript extract fully (working independently first and then comparing page by page as a group) and most of a topic summary (explored collectively page by page). However, given that this was a workshop designed specifically to explore the feasibility of these activities, it provided us with interesting insights into aspects of the process including whether or not the tasks appeal, or are practical to engage with, for people with differing interests and capabilities.

Analysing transcripts for quality improvement purposes involves reading through a lot of material that may not be relevant in order to identify touchpoints as well as deciding in each case whether a particular passage could be viewed as a touchpoint, and if so, why. In the group, a lot of time was spent reflecting on this, for each of the extracts that were identified. Some feedback suggested it was difficult to interpret a touchpoint because of this way they are interwoven with other aspects of the person’s narrative, making selecting and focusing a very difficult task. For example, a brief comment about access to services may be embedded within a longer narrative of early symptoms and inner feelings. Maybe researchers are so used to analysis, abstraction, summarising, and illustrating that we underestimate how complex this might be for others, despite the advantage of lived experience they bring.

The group were very focused on the importance of early intervention for depression. This was something SK had identified from her own analysis of the transcripts, and was therefore not an entirely new theme. However, the group’s input was powerful in terms of the emphasis and insight that came from their own lived experience, which in turn reinforced the importance of this aspect of service provision for the research team. This then influenced the selection of touchpoints on this topic for the trigger film.

Engaging with the material (both transcripts and summaries from the website) stimulated interesting discussions amongst the group about their own experiences, one key reason why we got through less material than originally planned. Although at first we viewed this as a failure on our part to ‘keep things on task’, it became clear that the value was in this very aspect – in giving space for the young people to discuss how their own experiences compared with the interview data. The impulse to discuss each touchpoint in detail also provided some personal benefit of sharing stories and mutual support – which one young person identified in her feedback as one of the main attractions of doing involvement activities.

Feedback suggested that although initially the idea of a touchpoint was difficult to grasp, it became clearer as they applied it to sections of text and agreeing examples. Group discussion and working in pairs helped shed more light than solo-working on what to look for and how to read the transcript in different ways. Facilitated discussion helped the young people refine and clarify their ideas, and gain confidence in sharing their own opinions and experiences. It also felt more enjoyable and less like an ‘exam scenario’, as one young person put it. There was general agreement that the research team’s expectations of how much material we might get through was unrealistic, and that we needed to reduce the volume considerably for any future analysis workshops.

Of course not everyone who gets involved in research as a user will end up training as a researcher, but this account resonates with a wider literature on the personal benefits of involvement, for example Thompson et al’s [27] suggestion that it might ‘offer spaces for the reconfiguration of identity’ (p.47).

Workshop 2 – Stroke

We then asked them to review a website ‘topic summary’ on stroke experiences, again looking for touchpoints. We found their assessment produced very similar results to the researcher perspective, though given varying degrees of fatigue as well as visual and cognitive impairment some found dealing with written material less easy or engaging than the verbal touchpoint discussion.

Overall, the group felt the film reflected well the themes relating to stroke services that they had brought up, but with some differences in both content and emphasis; for example, noise and light disturbance, sensory overload and disturbance from other patients were themes they identified which did not feature in the draft film. As well as the content of the film, the group commented that it was important for clips to be audible and succinct; they would have preferred a shorter film.

We invited feedback about the workshop process. Despite efforts to reduce the amount of information-giving about the analysis process in response to feedback from the first workshop, they felt that there had still been too much to get through in the early informative sessions so that after lunch they were starting to feel tired. Some also felt that the information had been too dense to engage with. One suggestion they gave to simplify the process was perhaps to ask people to come to the session with a short list of things that mattered to them the most about stroke care, and/or potentially sending out materials to them to work on at home, at a pace and time that they could control.

However, there was general support for the value of a group discussion to pool ideas about touchpoints as an accessible way to contribute. There was consensus that this made better and more efficient use of their time and knowledge than text-based work, and could be a way forward for collaborative analysis.

Everyone involved said they had enjoyed the day and that it felt good to be involved in something with the potential to make a difference. Two expressed a sense of frustration at no longer being able to work due to their strokes, and that taking part in this type of activity helped them to feel a sense of self-worth and being valued.

The trigger films they worked on, along with others produced by researcher-only secondary analysis, are now publicly available at http://www.healthtalk.org/peoples-experiences/improving-health-care/trigger-films-service-improvement/topics.