What is “qualitative” research?

While quantitative research typically uses numerical data and seeks to generalize findings from a sample onto the population, qualitative research is, in many ways, a reversed mirror image. Qualitative research is focused on gaining a depth of understanding about a topic. Qualitative research rarely includes numerical data, rather most commonly relying on first-hand accounts (interviews, focus groups), visual records (pictures, videos), and/or primary source documents. While quantitative research might utilize some open-ended data (ex., from a survey question), that is often analyzed quantitatively, such as looking for a fixed set of possible answers and counting how frequently each appears. Qualitative analysis will often look quite different, such as coding, followed by categorizing, and finally themeing (Saldaña, 2016). This process will be explored in greater detail in the future chapter on qualitative data analysis. Thus, qualitative research is an approach to social research that focuses on the collection and analysis of non-numerical data about a phenomenon to explore its qualities.

I often discuss quantitative and qualitative research as complementary. Qualitative research is frequently (though not always) used in either an exploratory or explanatory way. When used in an exploratory fashion, a qualitative study might be done to consider something that has not specifically been studied before, such as how a certain immigrant population experiences their first year in primary education in the United States. This might help those working with that (or similar) population, help policy makers better understand the degree of difficulty faced, or generally create empathy. Then that study—by itself or in conjunction with other studies—might be used as the basis for a large-scale quantitative investigation to produce generalizable findings about, say, the experience of immigrant groups entering United States primary education. Qualitative research is also used in an explanatory fashion. If a quantitative survey had a generally consistent data set except for a small cluster of responses far from the trend line, those outliers might be an excellent population to do follow-up qualitative study with to explain how or why they are so far from the others.

Consider the visual in the diagram below for a qualitative-quantitative sequence of studies, wherein qualitative research is being used first in an exploratory fashion.

Presuppositions and Philosophical Commitments

As previously discussed in the parallel quantitative research chapter, the philosophical commitments one has about research and T/truth more broadly end up having significant practical impacts on how the research is done. Qualitative researchers tend to come from more varied philosophical commitments than quantitative. Though there could be many chosen to discuss here, I will focus on the two largest and most historically influential to qualitative research: post-positivism and constructivism.

Post-Positivism

In that parallel quantitative chapter, I discussed post-positivism and social research. However, it does have some differential impact on qualitative research as compared to quantitative. In a practical sense, post-positivist qualitative researchers tend to be very concerned with their qualitative research emulating quantitative methods (and thus natural science methods). It is common for post-positivists to utilize concepts like validity and reliability redefined for a qualitative context and then applied to their research. There is also often great focus given to formally following established rules, consistency, and rigor.

Constructivism

Cobern (1993) described constructivism as a “model of how learning takes place” (p. 105). Note that research is learning, so it is a reasonable application from pedagogy to research. Cobern further described that constructivism emerged from the recognition that scientific knowledge was limited to physical reality, which itself can only be perceived and described based on previously existing knowledge. Of course, social experiences exist as much in the human mind as they do in the physical realm, and they only have meaning in the mind. Only studying the physical aspects of a social experience will leave one with a woefully shallow understanding of those phenomena. Key aspects of constructivism for research then (based off Cobern, 1993) are that: the researcher is an active participant in the process of knowledge creation, and social knowledge generation involves interpretation, which is based on prior knowledge and discourse.

So, more succinctly stated, constructivism is the commitment that, whether objective truth exists or does not, it is only understood by humans as we construct it, which is driven by prior knowledge and social discourse. In the qualitative context, constructivists then tend to emphasize participant voices, participatory methods, methodological flexibility/creativity, and subjectivity.

Inductive Reasoning

Qualitative research generally relies upon inductive reasoning, which is often described as going from small → large in scale. In other words, one collects a variety of small-scale observations and comes to a tentative conclusion. In practice, this might look like:

• This qualitative study includes feedback from 11 elementary teachers who found <pedagogical practice> to be effective in their classes.
• <Pedagogical practice> is likely to be effective for elementary teachers in the same cultural setting.

A good practitioner might extend that even extend that a step further beyond reasoning and into their work:

• <Pedagogical practice> is likely to be effective in my elementary classes. I should try it and assess it through action research.

Saldaña and Omasta (2018) described the relationship between inductive reasoning and qualitative research in this way: “Induction is open-ended exploration of a problem, going into an inquiry to learn as you go, formulating answers as more information is compiled… Much of qualitative research is inductive inquiry or analytic induction, because researchers generally begin with open-ended questions for investigation rather than fixed hypotheses to test” (p. 9).

Researcher Positionality

Including a researcher positionality statement is a very common, perhaps even expected, component of a qualitative research paper, but, while it would likely be almost as important for quantitative studies, they are extremely uncommon there. A researcher positionality statement is usually approximately one paragraph and explains the relationship of the researcher to the topic. It commonly includes any relevant demographic relationships or non-relationships (ex., the author’s own racial identity and experience in a study that involves race) as well as direct experiences with the topic.

If, for example, one was studying the burnout of social workers contributing to leaving the profession, it would be quite relevant to know that the author had previously been a social worker but quit the profession after feeling burned out. Knowing this allows readers to attempt to account for potential biases in design, analysis, and interpretation. I have provided a sample positionality statement for myself in a fictional study on college faculty members’ tenure processes.

Methods of Qualitative Data Collection

Qualitative data, as discussed earlier in this chapter, are quite open-ended, as opposed to numerical/statistical representations. In quantitative social science, the methods of data collection generally stand on their own in place of a methodology (for better or worse), especially in regard to how that is written in a journal-article length manuscript. Qualitative research, however, uses methods for data collection and analysis, but those methods are selected and utilized within a methodology.

As the great grounded theorist Barney Glaser (2007) famously said with regard to qualitative research, “all is data” (para. 1). As such, the methods of collecting qualitative data are quite broad. Below are three of the most common examples, but this is only a skim at the surface of qualitative data collection methods.

Interviews

An interview occurs when a researcher directly and individually asks questions of a participant to gather data. The set of questions/interview plan is generally referred to as an “interview protocol.” Interviews can be used as a quantitative data collection tools, with scripted questions read verbatim that yield quantitative data.

However, qualitative interviews generally fall into the categories of structured, semi-structured, and unstructured. In a structured protocol, questions are asked verbatim, and the predetermined list of questions does not change. The questions are often designed to generate succinct answers, and there tend to be more questions than other types of qualitative interviews. In a semi-structured protocol, there is usually a smaller set of questions (say, 5-10) that are worded to generate longer answers. The interviewer will often adapt questions to the interviewee and ask follow-up/clarifying questions. In an unstructured protocol (commonly used in phenomenological interviews), there might just be a single, open-ended question posed to the participant. The interviewer then follows up in a more conversational style based on what the participant says in response to the first question.

I often think of structured interviews as the science of interviewing, while unstructured and semi-structured are more the art of interviewing. Structured interviews produce consistent responses and minimize researcher interjection; however, they can produce answers without meaning or inadequate to really answer qualitative research questions. However, all three types have value in qualitative research.

Focus Groups

Focus groups are very similar to interviews but with multiple participants simultaneously, and sometimes they are even referred to as group interviews. However, as with many things, the group is more than a sum of its members. Focus groups can be effective way to gather information from multiple participants in a short amount of time, also allowing the researcher to observe and listen to interaction among the group. Focus group participants are often a sub-set of a larger group that is carefully selected to give feedback.

Observations

Sometimes, qualitative researchers get great value from observing—whether from a distance or as a participant. Observations allow us to observe people and systems behaving naturally. Usually, the qualitative researcher doing observations will keep notes and memos, as well as possibly pictures or recordings.

A Healthy Dose of Skepticism

As referenced in the parallel quantitative research chapter, skepticism is essential to good research. Here are four of the most common issues associated with qualitative research:

• Data integrity & authenticity
• Researcher bias
• Errant generalization
• Difficulty defining “good” qualitative research

Data Integrity & Authenticity

How do we know the qualitative data are real? Because of protections for participants, it is very rare for qualitative research reports, such as journal articles, to include identifiable participants. If, for example, a participant mentions something that would get them fired, that might be very important data to report in findings, but making it identifiable would cause harm to the participant. As such, it is impossible for readers, peer reviewers, or editors to independently verify that the data came from actual participants (as opposed to the researcher making it up, using AI to generate simulated interviews, etc.).

This concern is more than hypothetical. Specifically, one should consider the case of Dr. Alice Goffman (Beuving, 2020; Neyfakh, 2015; Parry, 2015). Goffman’s sociological research on inner cities was initially lauded as groundbreaking, but shortly after the publication of her book, a firestorm erupted. Parts of her stories would implicate her in crimes and others would have been impossible, which upon being confronted with she acknowledged as having been fabricated. However, she defended that approach as protecting participant confidentiality and that her fictional data actually best represented what she saw as the truth. For those coming from a big-T Truth perspective, this was an incredible affront to the rigors of “good” research, and even most of those from a little-t truth perspective were taken aback. In one sense, Goffman was simply best representing her interpretation of the truth while anonymizing data to protect participants; from another, she was telling fictional stories and selling them as nonfiction.

This particular saga illustrates one of the greatest limitations inherent to qualitative research—the authenticity and integrity of the data is unverifiable.

Researcher Bias

Having some parallels to data integrity and authenticity (which is about its collection and the data themselves), it is also very easy for the qualitative researcher to insert their own bias into the analysis and interpretation of the findings. Indeed, it is impossible for a human to truly remove their biases from research design, data collection, analysis, and interpretation. While that is usually subtle in quantitative research, it is overt in qualitative, as rather than formulae to answer questions there are human decisions made to analyze and interpret. Some qualitative researchers even embrace bias to the extent of intentionally magnifying it.

To illustrate, consider an essay on “college football referees” that is written by a fan of a specific college football team that lost in the College Football Playoff due to an obvious referee error on the last play. How might their experience affect the evidence they select to support the points they choose to make? Does that, however, automatically make their point invalid? Could such an essay only be written well by someone who does not follow college football at all (i.e., unbiased), but if so, would that not introduce its own set of drawbacks to the points being made?

It is ultimately on the author to communicate their known biases (usually through a researcher positionality statement) and then on the readers to account for how that bias may have affected interpretation.

Errant Generalization

This problem tends to emerge more from readers than the authors of studies, but that is no less problematic. It is easy to read a compelling qualitative paper and consciously or subconsciously think that’s just the way things are… everywhere. While quantitative studies can also be over-generalized, qualitative studies should almost never be generalized (in a technical sense) at all. It is incumbent upon the readers to not infer true applicability beyond the setting of the study itself, but then those same readers must look for areas in which the study overlaps with their setting to find insights.

Difficulty Defining Quality

What makes a qualitative study “good?” There are a great number of opinions. Whereas quantitative studies utilize widely agreed upon metrics like validity and reliability, there is no direct parallel for qualitative research. Thus, perceptions of defining quality vary wildly among qualitative methodologists and others.

For an illustration, we will consider van Manen’s (2016) approach for a specific phenomenological sub-type (hermeneutic phenomenology), which is itself a type of qualitative research. He proposed four criteria to serve as proxy measures for validity: is the study based on a phenomenological question, are the data experientially descriptive accounts, is the study rooted in phenomenological literature rather than general methodological sources, and does it avoid using any non-phenomenological validation criteria (pp. 350-351)? On reliability, he noted that, “it is unlikely that a phenomenological study would be involved in measurement schemes involving interrater reliability by having different judges… The point is that phenomenological studies of the same ‘phenomenon’ or ‘event’ can be very different in their results” (p. 351). Finally, he concluded that, “empirical generalizations cannot be drawn from phenomenological studies” (p. 352).

Now, contrast van Manen’s approach within phenomenology to Yin’s (2014) approach to case studies (another type of qualitative research). Yin called for strict protocols to create construct, internal, and external validity that are an approximation of quantitative methods, and he defined reliability as the “consistency and repeatability of the research procedures used in a case study” (p. 240), which, if done well, would allow a “later investigator should arrive at the same findings and conclusions” (p. 48).

A major contributing factor to these differences is the philosophical commitments held by these two authors and by members of differing traditions (Yin, though claiming to be constructivist, is widely believed to be a post-positivist). Though such issues may seem irrelevant to a new qualitative researcher, there are significant implications, ranging from what does or does not get published to even getting basic approvals from local ethics boards to conduct research.

One concept that has gained a reasonable amount of traction is that of trustworthiness, which is an approach to evaluating the quality of qualitative research based on the “integrity of the data, balance between reflexivity and subjectivity, and clear communication of findings” (Williams & Morrow, 2009, p. 577). The integrity of the data is evaluated by whether the amount of data is adequate to draw conclusions from and the dependability of the the data itself. Then, the balance between reflexivity and subjectivity refers to the ability to balance researcher interpretation with the meaning used by participants. Finally, the clear communication of the findings indicates that qualitative studies should be readable and understandable by the original participants, scholars, and practitioners in the field of study.

Merriam and Tisdell (2016) also discussed the concept of trustworthiness for the rigor of a qualitative study, including eight tenets: triangulation, member checks/respondent validation, adequate engagement in data collection, research position or reflexivity, peer review/examination, audit trail, rich thick descriptions, and maximum variation (p. 259). That is not, however, to say that every study must address all of those fully to be “good” or that they are fully appropriate to every methodology (ex., a narrative study with a single participant would not have maximum variation in participants).

It is important to note, however, that all the tenets of Williams and Morrow’s (2009) and Merriam and Tisdell’s (2016) trustworthiness are qualitative themselves. It is entirely possible that different evaluators of a qualitative study come to completely different conclusions about whether the study met those metrics. For those conducting qualitative studies, it is then advisable that they consider each of these metrics carefully, make a reasoned decision on each, and communicate that process of decision making in the manuscript reporting findings.