All three researchers and three of the four lay partners were interviewed; the fourth lay partner, who was one of the two not present at the open coding training session, declined to take part without giving a reason.
The lay partners and researchers had a range of previous experience in research and analysis. Two of the three lay partners involved in the open coding had previous experience of involvement in research projects, whereas the third had not. Two of the lay partners had experience of some form of data analysis as part of jobs they had held previously but the third had not been involved in any analysis before. All three researchers had experience of lay involvement in research projects but only one had previous experience in involving lay partners in data analysis and she indicated that this was still a somewhat ‘experimental’ rather than ‘established’ approach.
The way I like to work as a researcher. I always try and involve patients…. So I’ve worked a lot with people, both developing research proposals, but also then with getting them to comment on projects as they go along, writing patient-facing materials, or at least checking them if they’re not actually writing them. And then increasingly experimenting with getting lay involvement in both data collection and data analysis. (Researcher 2)
Lay partners and researchers expressed overall positive views on the training provided and were of the opinion that the informal setting of a café had worked well and that the training itself had been very interactive. The written instructions sent round after the meeting was also reported to be helpful and straightforward and sufficient for those unable to be present at the training session.
The view of the all the lay partners and researchers involved with open coding was that the training had been less detailed than the trainees had anticipated. The dominant view was that this reflected the fact that open coding was a simple and straightforward process, and that not a lot of training was required or would be helpful to achieve the objective of bringing out unique lay perspectives.
‘I think they almost wanted a template or a sort of proforma of this is how you do it. And I think as we suggested, it really is a matter of ‘read it through and write down what comes to mind’, and they sort of almost thought, ‘oh is that all? But I think that’s appropriate actually because I think if you do constrain people’s thinking you get what you ask for. Whereas if you allow them to be completely open then you’re preparing yourself to be surprised, which is great.’ (Researcher 2)
‘The open coding training was good, it was quite sort of sparse on the day and it was also quite, it seemed to me to be quite casual, but I think that's just the flexible nature of open coding.’ (Lay person 2)
However, one of the researchers expressed a concern that the scientific and systematic nature of qualitative analysis had not been communicated adequately, and another was of the view that it would have been helpful for the open coding group to have known more about how the open coding related to the framework analysis carried out by the lead researcher.
‘I kind of felt it was being presented in a less scientific way to the way I would do coding. … There was this talk about it is all very subjective, everyone will do this differently, whereas for me, when I am doing coding, there are all these reliability processes, which this was a part of. …. I think I would have wanted them to understand more the context. I think the fact what they were doing, like open coding, was really good, and the best way to do it, and I think it didn’t matter that we didn’t do it all so systematically, because [researcher] had done that and they were contributing their perspective to the theme. But I think it would have been helpful if they could understand a little bit more about how their analysis and [researcher’s] analysis would begin to gel together, and where the differences might be, and why the two were really important.’ (Researcher 3)
‘So, if I were starting again I would have … got some people to, offered the opportunity for them to code against [researcher’s] framework, because I think they could have really understood framework more. If you were going to add even more value it would be that, because then you'd get a cohort of people that really got the framework in the way they got the open coding and everything would be positive about that.’ (Researcher 1)
One of the lay partners was also of the view that a hands-on trial run of open coding may have been helpful and one of the researchers said that they had done that in lay open coding training sessions she had run previously.
All the lay partners reported an overall positive experience of carrying out the open analysis and that it had been an enjoyable experience.
‘I thought it was going to be a little bit of a chore … but actually I really enjoyed the analysis work.’ (Lay person 3)
In addition, one was of the view it had provided them with transferable skills for other research projects.
‘That was very good, first of all because I didn’t know about open coding at all, so this was a discovery. And the training was done extremely well by a member of the research team, and the whole process was really very enjoyable and very easy to do. And in fact one can wonder why it’s not done more often in [health] service research. Also because in fact open coding is actually quite simple and once one has done it once for a specific study then it becomes even easier for the next piece of work.’ (Lay person 1)
Both lay partners and researchers were of the view that lay involvement in the analysis had increased lay engagement with the project as a whole.
‘Well, I felt I enjoyed that and I felt very involved in the project and very involved with the other people that were doing similar things to me as well as the researchers, because you know, researchers had done the interviews. So, it increased my feeling of belonging if you see what I mean.’ (Lay person 2)
The researchers expressed the view that lay involvement in open coding had contributed greatly to the project and analysis by adding new perspectives to the findings and an enhanced understanding of the data as seen in the following example:
I think it definitely enriched our understanding of what was in that data… It kept us as researchers having a more open mind, being open to more possibilities. I think they did add a new perspective in terms of what they saw or the emphasis on what they saw from what maybe we had seen. So I think it was definitely additive and added a different dimension to some at least of the findings.’ (researcher 2)
It became apparent that lay and researcher contributions to the analysis had worked synergistically. For example the two researchers and lay partners in the first open coding session identified the theme of ‘contradictions’ where healthcare professionals said that they supported involvement of patients but then later expressed negative views about patient involvement. The lead researcher then did some extra analysis of the whole data set and found that these apparent contradictions reflected how healthcare professionals do support some involvement but that the extent of this is limited.
‘What was very important is that everyone’s contribution could be inserted in a very natural way; it wasn’t someone providing a contribution in an isolated – like in a tunnel, which sometimes can happen. It was basically a very rich mosaic of contribution and views all meshing together to provide a very, very rich outcome.’ (Lay person 1)
‘I did do a bit of extra analysis as a result of it, so I took their theme and made sure I had analysed it systematically for the whole data.’ (Researcher 3)
Researchers and lay partners expressed the view that lay involvement in analysis meant an additional time commitment from both lay partners and researchers in terms of the training, carrying out the analysis and then meeting together to discuss it. This was not necessarily thought of as something negative but as something that needed to be planned in.
‘I quite liked the fact that it was time-consuming and it made you concentrate for an hour on … [each] separate one as it were. So, I wouldn't want to alter the time that I put in, but what was useful to me as well was knowing when the stuff was going to come into my inbox, it was knowing that you know, they were expected on Tuesday next week sort of thing so that I could plan some time in the coming weeks in order to be able to do it at a quiet time when nothing else was going on in the house.’ (Lay person 2)
Another challenge, raised by the researcher who also had previous experience of lay analysis, was that lay partners may bring their own experiences rather than simply analysing what is in the data. While it was acknowledged that the purpose of lay involvement was to bring in new lay perspectives, the line may be crossed when perspectives from outside the data are brought in rather than using experiences to help interpret the data itself.
‘It’s very legitimate to interpret on the basis of personal experience, we all do that … I really don’t believe you want to sort of bracket your previous experience and I don’t think it’s desirable, I think usually. But I think … it’s a very fuzzy line actually between what enriches your analysis and what takes you completely off-message and off what’s actually in the data.’ (Researcher 2)
Yet another challenge identified was that of recruiting a diverse group of lay partners to conduct lay analysis who would be representative of the general population.
‘The main difficulty was actually the fact that we weren't able to recruit a broader range of people. But, I don't know that we could've done anything else to change that; we did everything we possibly could, I don't think we left a stone unturned.’ (Researcher 1)
Suggestions for future lay involvement
While lay partners and researchers were all in favour of future lay involvement in analysis, there were also opinions expressed that this would not be suitable for all projects. One researcher was of the view that this would be more suitable for qualitative rather than quantitative analysis, another suggested that it may not be suitable in some clinical areas such as palliative care or where data may be sensitive, confidential or distressing for lay partners. A lay partner thought it would be more suitable for health services research than for clinical research.
A range of views were also expressed as to who the lay partners involved in analysis should be. One of the researchers expressed the opinion that that if lay partners were continually involved in research projects, they may start ‘coming to the data with a researcher’s hat on.’ All the lay partners who were involved in the interview analysis had also been involved in the earlier observations conducted as part of the IMPRESS study. The lay partners thought there was a natural progression from the observations to the interview analysis and that the same themes emerged.
‘Well, sometimes I felt as though I could identify the patient and sometimes I felt as though I could identify the doctors and nurses and it was very similar to the observations that we did and it reflects the experience quite well I thought. So, in a sense there were no surprises in the interviews, because they followed the format that I had seen, I'd observed on the ward.’ (Lay person 2)
One of the researchers was of the opinion that less training had been needed because the lay partners already knew about the project. However, one of the lay partners expressed the opinion that although following on from the observations made the analysis easier, it would have been interesting to have had some lay partners conducting analysis who had not been involved in the observations.
‘One of the things I guess that you’re going to have – and I was aware as I was doing it, is you’re going to have your own biases from the observation work and you’re almost looking for those to come out in the analysis work, whereas if you came at it not having done the observations you might spot some different trends or some different issues. So I think it would probably be quite interesting to do a combination of people that have been involved in the observations and people that haven’t and see if there’s any fundamental difference.’ (Lay person 3)
Another question that arose in one of the researcher interviews was whether in the future it would be better to have a mixture of healthcare professionals and lay partners in the open coding group, as in the first open coding session, or to have lay partners only, as in the second session. The lead researcher was of the view that either would be of value and that it would depend on the desired outcome. She reported that she was able to integrate the findings into the coding structure more easily from the first but that she was able to incorporate lay perspectives into the final publication in both cases.
‘I think in terms of the output I found the first one easier to integrate more quickly with my analysis, so if you wanted the analysis to be totally integrated in, the first one would work better. I think if you just wanted to incorporate the lay analysis, but maybe you wouldn’t be able to do it in quite a systematic way as I have described … then you could use the second one. So I think you would need to think about what your overall objectives were, what was more important to you, to have it completely lay? Or to have it a bit more structured?.’ (researcher 3).
A fourth suggestion for the future was to train lay partners on how to do qualitative analysis electronically rather than on paper to avoid large amounts of printing.