The critical incident technique is a research process that invites respondents to identify events (incidents) they deem significant (critical) for a particular purpose, typically associated with job performance within an occupation. Researchers use data from participants’ accounts to form categories of behaviors that contribute to the success or failure of a given purpose.
History of the Critical Incident Technique
John C. Flanagan is credited with developing the critical incident technique, originally in connection with studies of World War II aviators (part of the Aviation Psychology Program, out of which emerged the American Institute for Research). Flanagan and his colleagues were interested in why pilots in training fail their programs, how pilots respond when they become disoriented, and what leads to failed bombing missions. His research technique permitted him to efficiently gather multiple instances of the behaviors in question and analyze those incidents for patterns. The research technique has since been applied to a wide variety of occupations, activities, and perspectives.
Critical Incident Technique Five-Stage Process
Most researchers using the critical incident technique employ some version of the five stages of the process that Flanagan articulated. In brief, the researcher (a) establishes the general aims of a particular activity; (b) specifies the precise conditions to be observed (or recalled); (c) collects data in the form of interviews or questionnaires that elicit relevant incidents; (d) analyzes the data; and (e) interprets and reports the findings.
An example may help to clarify the stages of the critical incident technique: Suppose that an executive in charge of a chain of movie theaters is interested in the best way to train managers. A common problem that such businesses face arises when unruly crowds of younger patrons create disruptions as they reach a critical mass. To better understand how managers might address this, a researcher asks theater managers to recount incidents in which they were effective or ineffective in minimizing such disruptions.
Stage 1: Establish Aims
In the first stage, the researcher defines the aims of the activity. This step lays the groundwork for determining what constitutes a critical incident. In general, an incident consists of an account, either from the direct experience or the observations of the participant, that relates closely to the aim of the activity.
Although a theater manager may have many aims (e.g., drawing in more young adolescents, efficiently moving crowds between showings, increasing concession sales, training employees), the aim in question focuses directly on one particular challenge: preventing or stopping young people from disrupting the theater experience of other patrons. The behavior that is critical in succeeding in this aim is not the same behavior that is critical for enticing more young adolescents to patronize a given theater. Flanagan indicated that it is often helpful at this stage to involve experts in defining the aim, particularly for activities that do not involve supervisors or when stakeholders define the aim differently.
Stage 2: Specify Plans and Conditions
The second stage in the process involves clearly specifying the conditions that address the general aim and the plans for collecting incidents. The researcher defines the situation to be observed, who is being observed, and who will make the observation. In our example, the young patrons are in the theater as opposed to waiting in line to purchase tickets; the theater has a mixed audience, not one composed exclusively of young adolescents; the behavior of one group of patrons is potentially disruptive to another; and the manager is the one who takes action. These conditions allow judgments to be made about the relevance of the behavior described.
In addition to the conditions, the researcher must determine who is best qualified to provide incidents. Generally, this requires familiarity with the conditions and behaviors. For the sample study, theater managers are uniquely qualified because they make choices about their behaviors in the context described. However, potential observers might also include patrons or other employees.
Stage 3: Collecting Critical Incidents
Next, the researcher must collect critical incidents. Many researchers prefer to gather oral accounts through the critical incident interview, but written accounts may also be collected. A sample protocol might include the following questions: “Think of a time when one of your theaters had an audience with a concentration of young adolescents whose behavior threatened or disrupted the viewing enjoyment of other patrons. Could you please describe the situation? What did you do (or what did you observe a manager do) to minimize the disruptions? Why was this particularly effective (or ineffective)?” The researcher continues this line of questioning, probing the interviewee for specific details. One interview may elicit numerous incidents because a manager may have encountered this particular situation often.
A key consideration in this technique is the number of incidents that must be collected. Generally, more complex activities require a greater number of incidents. The range of incidents in published studies is quite large (from fewer than 20 to more than 4,000). Flanagan indicated that one should continue collecting incidents until new incidents provide few or no additional critical behaviors.
Stage 4: Analyzing the Data
Data analysis occurs in conjunction with collecting incidents (because the final count of incidents is affected by the emerging analysis). Researchers create categories of behavior that are relevant to the purpose of the study or the way the data will be used. In the sample study, the researcher would develop categories of behavior that assist or impede the management of the problem in the theater situation. The analysis is intended to help design training programs for managers (or possibly selection criteria). A different aim would require a different frame of reference for the analysis. Given this general frame of reference, analysis then moves to the formation of categories and sub-categories of similar behaviors, a step that Flanagan acknowledged as subjective. Coding (categorizing) of data is an inductive process of comparing statements and grouping by patterns.
Stage 5: Interpreting and Reporting Findings
In the final stage, the researcher interprets and reports findings. Again, much of this report depends on the targeted use of the study. The report should clearly indicate the aim that has been studied, particularly if competing aims are present. Reports commonly include both the weight of the categories (i.e., percentage of comments) and the language used by respondents.
Critical Incident Technique Applications
The critical incident technique has been usefully employed to analyze behaviors, techniques, traits and characteristics, interactions, and even thought processes in a tremendous variety of occupations. The technique can form the basis for selection criteria, evaluation tools, and professional development, either as the foundation of training programs or as the actual mechanism for training if employees participate in the creation and analysis of critical incidents.
References:
- Anderson, L., & Wilson, S. (1997). Critical incident technique. In D. L. Whetzel & G. R. Wheaton (Eds.), Applied measurement methods in industrial psychology (pp. 89-112). Palo Alto, CA: Davies-Black.
- Andersson, B., & Nilsson, S. (1964). Studies in the reliability and validity of the critical incident technique. Journal of Applied Psychology, 48(6), 398-403.
- Flanagan, J. C. (1954). The critical incident technique. Psychological Bulletin, 51(4), 327-358.
- Kain, D. L. (2004). Owning significance: The critical incident technique in research. In K. deMarrais & S. D. Lapan (Eds.), Foundations for research: Methods of inquiry in education and the social sciences (pp. 69-85). Mahwah, NJ: Lawrence Erlbaum.
- Woolsey, L. K. (1986). The critical incident technique: An innovative qualitative method of research. Canadian Journal of Counselling, 20(1), 242-254.