Collaborative Group Analysis of Data

Janice McDrury PhD, R Comp N
Senior Lecturer, Otago Polytechnic
Dunedin, New Zealand

Qualitative research data provides us with rich and interesting information. This material is available for us to read, reflect upon and ponder. In the process of coming to know the data it is possible to identify key words, themes and even recognise patterns that emerge.

It is important to acknowledge two key factors which can impact on data analysis. The first is the enormous amount of data which requires analysis and secondly that the emergence of patterns frequently reflects the perception and awareness of one researcher. Equally valid and important themes could be overlooked and the themes that are presented could be more reasonably labelled as one researcher’s view than the view of various participants. While many researchers could argue that supervisors and peers have input, their comments usually focus on the researcher’s decisions and audit trail or the research process. It is not usual for such interest to extend to the level of reading and reflecting on the actual data.

In the interests of coming to a fuller, more complete and more manageable ways of understanding of data it can be useful to incorporate multiple perspectives in the analysis process. This can match the ".. multiple, sometimes conflicting mental constructions of every-day life that are situation and context dependent.." (Forde-Gilboe, Campbell & Berman, 1995:17), that are present in the data. Obviously using this type of analysis will not be suitable or appropriate for all types of data but is helpful where it is desirable to attempt to understand other ways of representing the response which is presented (Duffy, 1985). This can be place within a qualitative vs quantitative discussion (Duffy, 1986), with acknowledgment of the advantages of including the personal perspective (Scott, 1997), the importance of evaluating the researcher’s place in the interpretation (Ramos, 1989) and cognisant of the type of data collected (Carr, 1994; Cormick, 1991). However it is important to remember that it is not a ‘black and white’ issue (Hanson, 1994:942). There are other considerations that can be incorporated including the effective integration of students into the research process and the positive gains of collaborative interaction.

Collaborative work

Collaborative work has as an underlying principle that working together on an issue enables a result that cannot be attained alone. The process of talking through, discussing issues and exploring options enables those who are collaborating to reach conclusions that move the thinking forward to new insights. A central element of collaboration is the sharing of thinking processes with others. Thoughts that may, in an initial phase, appear vague or confused are expressed and through dialogue are sorted or clarified. The dialogue – especially if open-ended questions are used – can enable the further development of ideas. Such development can be in either depth or breadth.

The type of collaboration that is most suitable depends on the number of people involved and the material under consideration. If there are only two or three people it is frequently sufficient to engage in a free dialogue in which there is a genuine desire to explore the ‘other’s’ thinking. While open-ended questions are particularly useful, it is also possible to enrich understanding by attempting to express concepts or linkages in a diagrammatic way.

However, if a larger group is involved it may be more useful to use some of the established collaborative strategies to ensure that the most productive and creative thinking occurs. Use of ‘think, pair, share’ (Kagan, 1992) collaborative process can enable the expression and development of ideas within the group. This allows the individual time to think through what they want to say, try out the idea(s) with one other person, thus clarifying their thoughts, before sharing with the wider group. Through the sharing in pairs thoughts may be developed to incorporate more complex ideas and thus when the wider group discussion occurs the concepts being presented are more inclusive and frequently in a more complete form.

Application of Process in a Community Health Research Project

From its inception this research involved two key researchers. Ideas were shared and developed and in the stage of planning it was decided that a collaborative group of knowledgeable assistance would undertake the analysis of data. The main researchers would not be involved in the pencil and paper act of coding, but be available to coordinate and integrate the material under discussion and focus on ensuring that a sound and consistent approach was undertaken. Thus, when the funding was granted (from Health Research Council), it included money for payment of research assistants.

In this study a survey was the central tool that enabled us to explore the public’s perception of "The Role of the Nurse in the Community Setting". Each of the 490 questionnaires which were returned (32% return rate), consisted of both qualitative and quantitative data.

Approximately one quarter of the data was quantitative and was easily coded. It included transferring rankings or descriptive data related to the participants. However the other three-quarters involved qualitative data that needed to be analysed into some manageable form. There were six questions that asked for such data with a total of seven sections that could involve comment. Comments were generally short with 5 – 10 lines being provided. However respondents were informed that they could continue their response of a separate page if they wished. Only a small number (5) took up this option. All data that was presented was coded.

That coding was undertaken by a small group (11) of nursing students who had some experience with both community health and research methods. These knowledgeable research assistants worked with the primary researchers to assist in uncovering the meaning of the data. The collaborative process used individual reflection, pairs or small groups sharing ideas, and large group discussion in the ‘think, pair, sharing’ type of process.

The process of collaborative analysis

To begin the process, each assistant was allocated a number of questionnaires (43-44). To enable the assistants to become familiar with the process of coding, the quantitative data was dealt with in the initial session. From the second day, one section of the questionnaire was addressed in each session. The first task was to sort though the questionnaires to identify which respondents had answered that particular question. While the number varied considerably (40 – 97%), it was usually around 75%.

The first stage of the qualitative analysis involved reading the data and noting keywords and ideas that were presented. This enabled an initial familiarisation with the range of the ideas presented but also gave an opportunity for the noting down of key words that emerged from the data. There was no attempt at this point to collapse these into broader constructs. It was essentially a phase of familiarisation and capturing or identifying the issues. These key words were then recorded onto a white board so that everyone in the collaborative process was aware of the range of information.

The second stage involved each individual in the group spending time looking at all the words which had been presented and thinking through possible ways of developing themes which could encompass the data. This stage usually took 5 – 10 minutes as it was intended to only be a very initial process of thinking about the material. The individuals then shared their ideas with a partner and together attempted to come up with a framework through which the material could be structured.

At this stage it was emphasised that the number of categories should not be so big that each individual idea was separate or that the number of categories was so small that it did not demonstrate the range of ideas. It was acceptable to have an ‘other’ category but the number of items in this category should not be so many that an important range of data was hidden.

The establishing of boundaries to define each category which was being proposed was particularly important. This process helped to define the various categories but also made overt the interrelationship and difference in the categories. These boundaries needed to be established within the discussion between the ‘pair’, prior to the framework being presented to the group. The main focus was on capturing the meaning of the material that was presented. Items could be coded in more than one category and it was clear to the students that in reporting data, the codes would only provide reference to appropriate survey material and would not be used instead to the actual data.

The third phase involved the pairs sharing their ideas with the rest of the group. Each model that was proposed was written up on the white board as it was being presented so that everyone could think through the possibilities and problems. Particular attention was given to the boundaries by which each category was defined. There was a general discussion once all the models had been presented – which frequently involved further clarification of boundaries within the various frameworks. This stage, which usually took 10 – 20 minutes, was concluded with the selection of one particular framework that was to be implemented.

The fourth stage involved each assistant returning to their response data on the survey to have a quick review of responses to ensure that coding was possible within the chosen framework. This stage was an important work phase where individual research assistants worked through their own set of questionnaires to decide if the coding which was being proposed could accurately reflect the meaning of the material which was presented by the respondent.

The fifth stage involved all the research assistants coding copies of the same three questionnaires. Each worked individually without collaboration. If there were any problems these were recorded on a note-pad for later discussion. At the end of the process the codes were compared, to establish the level of reliability in the coding, and any questions which arose for the researchers were addressed. It was important at this stage that decisions were made by the researchers from their existing knowledge of the framework. This ensured that they clarified for themselves the boundaries with which they may find difficulty. Although usually this process went smoothly, on one occasion a whole framework as declared unworkable because their was overlapping between categories which could not be resolved and we returned to the discussion phase (2) to establish a new framework which was more suitable for the data. The steps from 3 – 5 were then repeated using the new framework.

The sixth stage was the application phase where individual research assistants coded their surveys and recorded the results as part of a cumulative database that provided ready access to the data in the writing-up phase. Each survey had been given a unique code number that made easy reference back to material from any given survey. During this phase the primary researchers were available to engage in discussion to clarify issues which arose. Where a new issue was being addressed, the whole group were brought into the discussion to both inform the decision but also to ensure appropriate implementation in subsequent coding.

The final stage involved all the research assistants completing three additional survey sheets. Again the codes were compared. While difficulty at this stage was rare, when it did occur it involved the assistants returning to any surveys where similar information was found and adjusting the coding involved. With each assistant having a relatively small number of surveys this reliability check did not cause any problems and resulted in the coding experience being more satisfying for the students involved.

The process that has been outlined is intensive and demanding for both the primary researchers and research assistants. Therefore a 5-hour day is most likely the optimum time frame. Any shorter than this does not provide enough time in the actual coding of data phase, but longer than this results in fatigue which can impact negatively on the coding process.

Advantages of the Collaborative Process

The collaborative process resulted in advantages for the students, the primary researchers and the quality of the data analysis.

Students

Students at the undergraduate level often struggle to come to terms with the reality of the research process. Reading about it, exploring methods and methodologies, preparing proposal and even undertaking small projects and dealing with the resulting data can be a two edged sword. It can clarify the process but still leave many questions unanswered. In particular they may be overwhelmed with the amount or complexity of the data.

The advantages for students included the:

• Reported enjoyment they experienced in working with the research project.
• Opportunity of have involvement in a research process that engaged them as active participants in the analysis and where they felt they could make a valuable contribution to structuring the data.
• Experience in research using real data: in particular becoming aware of the quantity and complexity of data.
• Opportunity to discuss the research processes particularly in terms of validity and reliability, informed consent and confidentiality and explore application of these concepts.
• Opportunity to learn about research while engaged in paid employment.
• Opportunity to explore the data in terms of practice. This frequently resulted in discussion about professional issues.

The primary researchers in a large research process such as this are often left feeling overwhelmed by the data. There is a real interest to know what is being said but attempting to grasp the range of responses is difficult to attain. In addition there is a real need to reflect on the implications of the ways in which a framework is established but when involved in coding the closeness to the data makes this a real challenge. However in this process the researchers were able to:

• Have an overview of all the data in the surveys. Key words were presented, data was discussed and difficult responses were addressed.
• Engage in discussion and have input into the framework that was established. In addition they had time to reflect on positive and negative implications of any given framework while the assistants were working independently.
• Have an overview of all the data within a very brief timeframe. Within 7 days all the data had been processed in its initial form.

The advantages in relation to the data were:

• That validity was attended to because there was a real attempt to be true to the meaning of the responses when coding the data.
• That reliability in the coding process was established by discussion of the boundaries between categories on the framework and then demonstrated by the reliability checks at the beginning and end of the coding.
• That multiple perspectives were incorporated into the analysis and establishing of frameworks for coding.
• That there was a clear audit trail that involved the recording of the framework and the boundaries that were involved with appropriate documentation of decisions made. These recorded the mutually acceptable themes in such a way that all research assistants could implement them and the primary researchers could be satisfied they has a record of the categories within the framework.

Conclusion

The use of collaborative processes provides an efficient, satisfying way to analyse large amounts of data in a way that incorporates multiple perspectives. In addition the attention to validity and reliability provides confidence in the analysis process. The positive gains for students cannot be underestimated. While all students commented on their increased understanding of the research process in general, they particularly appreciated an opportunity to explore and question aspects of the process and check their understanding or ability to apply knowledge within research. Perhaps the most positive gain was the enthusiasm they expressed in relation to the research process and the satisfaction they experienced being involved in finding ways to place a wide range of responses into a manageable framework to facilitate reporting of data.

References

Carr, L. (1994) The strengths and weaknesses of quantitative and qualitative research: what method for nursing? Journal of Advanced Nursing 20:716-721

Cormick DFS. (1991) The Research Process in Nursing (2^nd Ed.) Blackwell Scientific: Oxford.

Duffy, M.E. (1985) Designing nursing research: the qualitative – quantitative debate. Journal of Advanced Nursing 10(3)225-232.

Duffy, M.E. (1986) Quantitative and qualitative research: antagonistic or complementary? Nursing and Health Care 8(6)356-357.

Forde-Gilboe, M., Campbell, J., & Berman, H. (1995) Stories and numbers: co-existence without compromise. Advances in Nursing Science 18(1)14-26.

Hanson, E. (1994) Issues concerning the familiarity of researchers with the research setting. Journal of Advanced Nursing 20:940-942.

Kagan, S. (1992) Cooperative Learning. Kagan Cooperative Learning: San Juan CA.

Ramos, M. (1989) Some ethical implications of qualitative research. Research in Nursing and Health 12:57-63

Scott, D. (1997) The researcher’s personal responses as a source of insight in the research process. Nursing Inquiry 4:130-134