Suggested Guidelines for Assessing Qualitative Research at Biola University

The idea of establishing these guidelines emerged from discussions by the Biola University discussion group on qualitative research. The discussion group--composed of faculty from Talbot Theological Seminary (Christian education), the School if Intercultural Studies, Rosemead School of Psychology, and the School of Arts and Sciences (education)--asked Judith Lingenfelter and Donald Ratcliff to create the initial draft, from their breadth of experience and related study of qualitative research. Additional input was provided by Kevin Lawson (chair of the discussion group). The initial group discussion and the writing of these guidelines, were significantly influenced by criteria developed by Clive Seale (1999), which itself was an adaptation of a document by the British Sociological Association Medical Sociology Group (1996).

The following guidelines are suggested for use in evaluating qualitative research dissertations across the schools of Biola University. In no sense are they mandated criteria, but rather aspects to consider in the assessment of this variety of research.

1. How does the research problem fit into the overall knowledge base of the school or department in which it is being written? Do the hypotheses or sub-questions use criteria that involve sense experience and can in some way be recorded on paper and/or other media? Are discussions of changes and definitions of key terms included?

2. Does the author sufficiently disclose relevant personal background and his/her perspectives of participants and context?

3. To what degree is the role and/or level of participation by the researcher described? There should be an affirmation of openness to the data regardless of role, and consideration of differences between the offered role and the taken role.

4. Is the research literature cited relevant to the research problem?

5. Do the methods fit the context, participants, and questions? Discuss changes in methods if the initial plan is not appropriate, with an adequate rationale and evidence for revisions.

6. Is the selection of participants described and justified using accepted conventions, including possible modification/s of the initial plan?

7. Is the research setting described in detail, including physical location, surrounding community, relevant history, routines, and methods of initial and ongoing access by the researcher?

8. Is a systematic approach taken in data collection? Data and notes related to decision-making are to be available for external audit and verification.

9. Is a systematic approach taken in data analysis, including clear description of the genesis and meanings of categories, concepts and themes? Procedures used in data analysis should be related to formal analytic methods such as constant comparison.

10. Are decisions regarding which data to analyze and which data to ignore made on the basis of the research questions and priorities of participants, rather than researcher biases?

11. Are discrepant or conflicting results reported and supported, the result of an active search for negative cases?

12. Does the presentation include the full range of evidence, including the variation of perspectives of participants as well as views they hold in common?

13. Are validity-related procedures such as triangulation and participant feedback adequately utilized and described, with the goal of correcting, clarifying, and extending research analysis and conclusions?

14. Are conclusions in the presentation supported by primary evidence from the data? The data are to be clearly distinguished from interpretation.

15. Are conclusions and applications coherent and plausible?

16. Are the limitations of the study identified? Conclusions and applications should be topic-focused, and the potential for error recognized.

17. Have all relevant ethical guidelines—such as confidentiality, consequences to participants, etc.--been followed during the course of research? Procedures and relationships are to be consistent with a biblical/Christian world view.

18. To what extent does the research reflect a passion about the topic and a caring for the people studied? Details and conclusions should reflect participants’ words and ways of thinking, as understood and verified by a compassionate and open researcher.

This version of guidelines 6-27-02