Get complete information on Validity Threats of Qualitative Research

The causal analysis of qualitative data poses serious problems (Becker, 1958). Bernard (1988) points out that the qualitative researcher has become immersed in the setting and may well adopt the perspective of the key informants.

At the same time, the researcher must retain an outsider’s skepticism in interpreting the data. In practice, the analyst must switch back and forth between these perspectives, checking for consistencies and inconsistencies among the various informants and observations.

In this final section, we will review qualitative methodology from the standpoint of each of the four main types of validity, construct, internal, statistical inference, and external (also see Kidder, 1981; LeCompte & Goetz, 1982).

Construct Validity:

This text has defined different kinds of construct validity- measurement and experimental. Both face threats in qualitative research, but measurement construct validity is especially vulnerable. By definition, participant observation does not use standardized tests.

As a result, observations have a special proneness to random measurement error and, thus, unreliability. Measurement validity cannot exceed measurement reliability.

By their non-quantitative nature such observations do not lend themselves to reliability estimates. Because the participant observer often works in varying circumstances and with varying categories, we cannot assume she or he has good consistency over occasions.

Even with good reliability over time in the observations of a single researcher, qualitative data still face problems with interrater reliability and, in turn, measurement construct validity.

The measuring instrument in qualitative research consists of an individual without the support of standard instruments or baseline criteria. The observer must use his or her feelings, curiosity, hunches, and intuition to explore and understand the setting.

Consequently, two observers may arrive at results quite different from each other. As an example Derek Freeman (1983) came to a very different analysis of the Samoan culture than did Margaret Mead.

The qualitative observer also runs the risk of being biased by the feelings, loyalties, or antagonisms generated by the setting and the actors in it.

To achieve access to the most secret behaviors and perceptions of the actors, – the observer must seem trustworthy and likable. Most people would find it difficult to gain the necessary trust and friendship without returning some genuine affection.

As a result of these feelings, the observer may leave the neutral role of scientist and adopt the role of committed member in the setting, a role shift called going native. In practice, going native poses little threat to published research since it usually terminates the study unless the observer elects to publish a propaganda piece.

Experimental construct validity has less relevance in qualitative studies since the researcher seldom tries to make experimental manipulations in the natural setting.

On the other hand, the observer may welcome the chance to see the outcome of naturally occurring changes. However, the entry of observers at or near the time of a natural experiment may change the meaning of the natural event, as illustrated in When Prophecy Fails.

Near the time that the cult was expecting a major event, the researchers brought in several new observers. Some had cover stories that raised the cult’s confidence in their beliefs and social support resources.

Thus, not only can the setting bias the observer but also the observer can change the setting, thereby distorting the results. A good analyst must reflect on the ways that the observers have distorted the natural setting.

Internal Validity:

Qualitative research can at most approximate quasi-experimental designs, with all the threats to internal validity. More often, qualitative studies resemble correlation designs, with no manipulation of the independent variable.

In correlation designs, causal inferences depend on the association of two variables. However, an association requires that both variables take different values.

Qualitative research sometimes does not meet this requirement, for example, when it describes two aspects of behavior one of which does not vary. Observing that one level of one variable and one level of another variable occur together says nothing about the variables’ causal linkage.

For example, suppose that observer-finds that all ten members of a cult’s cell formerly belonged to the Catholic Church. Do these data imply any connection between a Catholic upbringing and cult membership? While we might make up some theory that fits such a link, these made-up data do not support such a causal claim.

Neither of our two variables, former religious experience and current cult membership shows any variability. We do not know how many no cult members come from a Catholic background. Perhaps the community had only Catholics until recently and all residents, cult and no cult alike, were reared Catholic.

Statistical Inference Validity:

More qualitative researchers are expressing their data in quantitative terms. A cult researcher could, for example, make a cross-tabulation between two variables-male/ female gender versus strong/weak belief.

With the data in this form, analysts can apply inferential statistics. However, most qualitative data do not lend themselves to inferential statistical analysis. As a result, we usually cannot assess the validity tested by such statistics.

This inability to make an inferential leap from a sample to a population does not trouble many qualitative researchers. Often, the qualitative researcher has little interest in generalizing to a larger population.

Researchers with an ethnographic perspective want only to describe in the deepest and most detailed possible way a unique group of people. Such researchers have no desire to make any claim about people outside of the studied setting.

External Validity:

For the same reason, qualitative researchers may have even less concern about external validity-generalizing to people in other populations, places, or times. All researchers have problems in assessing external validity.

We have no statistical procedure in quantitative research for checking generalizations beyond the studied samples. Thus qualitative research has no disadvantage compared to other methods in this respect.