Mixed Methodology Research

Mixed methodology research incorporates both qualitative and quantitative research methods. Qualitative research methods provide detailed descriptions about phenomena and may include interviews, observations, and analyses of documents, records, artifacts, photos, and film. Researchers choose this methodology when they are interested in a rich narrative description with an abundance of deep detail. Quantitative research methods, on the other hand, include randomized experimental and quasi-experimental designs, surveys, written or oral assessments, and other standardized instruments with which responses can be measured on a numerical scale. Statistical procedures are then used to analyze the numerical responses. In mixed methods research, both qualitative and quantitative methods are used in data collection or data analysis in the same study. Mixed methodologists believe that this developing paradigm will be the dominant form of research during the 21st century.

Paradigms

The type of methodology researchers use depends on their research perspective or paradigm. Scholarly conversations in this area have been highly developed for both the quantitative and qualitative paradigms. Over the last decade, mixed methodologists have begun to add to this debate.

Quantitative

Purely quantitative researchers generally work from the positivist-postpositivist paradigm. They believe that phenomena can best be measured and explained using the scientific method, which has been the dominant paradigm throughout the history of social science research. Quantitative researchers use experimental designs in which participants are randomly assigned to a treatment group, the group receiving the specific treatment, or the control group, which is the group not receiving the treatment. For example, in drug studies, some subjects will be randomly assigned to receive a drug and some subjects, assigned to the control group, will receive a placebo. The researchers then may administer a standardized instrument (an assessment that measures the results of the study) to both groups. Statistical analysis is conducted to compare the results.

A second type of design quantitative researchers use is the quasi-experimental design, which includes both treatment and control groups, but subjects are not randomly assigned to either group. Quasi-experimental designs are often used when dealing with intact groups where random assignment is not feasible, such as classrooms of students.

A third design quantitative researchers use is the causal comparative design. In causal comparative studies, the researcher does not impose a treatment. Instead research is done after the fact, or ex post facto. The researcher looks for cause and effect relationships based on group differences. For example, the researcher might investigate how familial support among new mothers is related to their postpartum depression levels.

Qualitative

Purely qualitative researchers work from the interpretive-constructivist paradigm. They believe that the way to understand phenomena is through exploring people’s interpretations. Qualitative researchers use designs such as case study, ethnography, phenomenology, grounded theory, and narrative inquiry. Data collection is carried out as naturalistically as possible in the research context through the use of observations, interviews, and collection of documents and records. Some examples of qualitative research include a case study of counseling techniques at a clinic, an ethnography of a school counselor’s day-to-day practice over an extended period of time, and an examination of counselors’ use of intuition in a phenomenological study. While there are many detailed approaches to qualitative analysis, qualitative research is analyzed categorically or thematically; the researcher reads the data to identify similarities and dissimilarities.

Mixed Methods

In mixed methods research, the researcher’s paradigm is often pragmatism. Pragmatists believe not only that it is acceptable to use multiple paradigms in the same research study but that qualitative and quantitative methods can be complementary. For a mixed methodologist, “what works” becomes the driving factor. Pragmatists value both the subjective and the objective; they believe that the research question is the most important issue. The research question, not the framework, should drive the method. By combining qualitative and quantitative methods, researchers are able to discover issues that might otherwise go undetected. However, critics of pragmatism have dismissed this paradigm as naive, simplistic, and overly applied. A mixed methodologist would contend that an undue focus on theory and paradigms has detracted from the need to focus on the point of research: the research question. The focus on the problem and not theory is one of the reasons mixed methodology has emerged as a field that is demanding respect.

History of Mixed Methodology Research

Mixed methodology research has been called the third methodological movement. Mixed methods research has not always been and is not currently accepted by all research methodologists. Though it is a “new” methodology, mixed methods research has been conducted throughout the 20th century. From the late 1950s through the late 1970s, mixed methods research was in a formative stage. During this time, researchers were beginning to combine surveys and interviews as well as using qualitative and quantitative data results to support each other. Some researchers viewed this mixing of methods as a mixing of mutually exclusive paradigms. The issue of paradigm conflicts was and remains difficult for some researchers to overcome. Therefore, the 1980s and early 1990s were characterized by this paradigm debate. Purists, those who believed strongly in either qualitative or quantitative methods, did not believe it was appropriate to combine the paradigms. For example, quantitative researchers have contested the use of qualitative methods in scientifically based experimental research designs on the grounds that the designs have a methodological hierarchy in which quantitative methods are preferred and qualitative methods are consigned to a supplementary role.

Though this debate continues for many researchers, by the end of the 20th century, some researchers began to identify their methodology as a mix or blend of qualitative and quantitative research. From the mid-1990s to the present, these researchers have been attempting to further develop mixed methods research procedures. They have worked on definitions, notation, and a classification system for the different combinations of mixed methods research. The millennium brought about a call to make mixed methods research its own separate paradigm.

Purpose of Mixed Methodology Research

The main purpose for using mixed methods research is to balance the weakness of single methodologies. Both qualitative and quantitative research methodologies have weaknesses when used alone. One weakness in qualitative research is that small sample sizes are used. Consequently, qualitative researchers are unable to statistically generalize the results to large groups of people. Mixing methods allows the researcher to study both small and large sample sizes in one study. Another perceived weakness of the qualitative approach is that researchers include their own interpretations and biases in the research.

Likewise, quantitative research has weaknesses. Usually the context of the study, the surroundings, and the environment are controlled and therefore not completely understood. Individual voices are not heard in a quantitative study. The researcher’s biases are not usually explicitly addressed in quantitative research. Mixing qualitative and quantitative research results in a more comprehensive and therefore stronger study.

Besides strengthening the design and interpretation, mixed methods research allows researchers to answer questions otherwise unanswerable using only one approach. For example, a counselor may want to find out which coping skills are used and how widely those skills are used within certain ethnic groups dealing with the emotional aspects of diabetes. The counselor may use interviews or focus groups to identify emotional themes and coping skills used by each ethnic group. The researcher may then use a survey, developed from those themes, to determine the extent of use of those coping skills discovered using interviews. Without the first qualitative approach, the researcher may not be sure that the relevant emotional themes and coping skills used by each ethnic group had been identified. Without the second quantitative approach, the researcher may not be sure that the relevant emotional themes and coping skills identified in the smaller sample would hold true in a larger group of subjects from each ethnic group.

Though it is not a purpose of mixed methods research, one advantage of this method is that it promotes collaboration between qualitative and quantitative researchers. Because it is not common that a single researcher is adept in both types of research, experts in each methodology often form a research team. This collaboration encourages multiple worldviews. This paradigm is practical in that all types of data collection and analysis techniques are available to mixed methods researchers. It allows words and numbers to be combined to help understand complex systems.

Procedures of Mixed Methodology Research

There are four common types of designs in mixed methods research: triangulation, embedded, explanatory, and exploratory. Each has a different purpose.

Triangulation Design

In the triangulation design, the purpose is to use the strengths of both qualitative and quantitative research to support each other with the goal of comparing and contrasting the results of the qualitative and quantitative approaches in order to identify where the results agree or converge. When the results agree, this strengthens the validity of the study. The qualitative and quantitative portions of the triangulation design are usually done in the same time frame and are given equal weight. For example, a counselor may study the effects of suicide using both a quantitative standardized instrument and an in-depth interview. The results of both would then be merged into one overall conclusion.

Embedded Design

In an embedded design, the primary focus is on either the qualitative or the quantitative piece with the other type having a secondary role. A typical use of this design is in an experimental study that is primarily quantitative, in which one group is given a treatment and then compared to a control group or a group with no treatment. Quantitative methods such as the use of a standardized instrument are used to determine whether the treatment worked. However, a qualitative portion can be embedded in the design, which may include an in-depth interview of the participants before, during, or after the treatment. In this way the qualitative methods inform the overall quantitative study.

Explanatory Design

The explanatory design is a sequential design in which one type of research is followed by the other for purposes of further explaining what was found in the first portion. For example, a quantitative study may reveal that certain mental health clinics are outperforming others. A qualitative follow-up study would be conducted to try to understand, or explain, what those high-performing clinics were doing. The qualitative piece would involve interviews and a site visit to observe the effective practices.

Exploratory Design

The exploratory design is also a sequential design in which one type of research is followed by the other. However, the purpose in an exploratory design is to build on the results from the first type of research to a second phase of the research. A commonly used exploratory design is to use qualitative methods to discover themes regarding an issue, and then use those themes to develop and administer an instrument that will generate data that will be analyzed quantitatively.

Data Analysis

Data analysis depends on whether the design is concurrent or sequential. In a concurrent design, the qualitative data are organized and then analyzed separately from the quantitative data. Then the two results are combined for another round of data analysis. In a sequential design, a first methodology informs the second. Therefore, the analysis of the first portion is completed before beginning the analysis of the second portion.

Data Analysis in Concurrent Design

Qualitative Data Analysis

Qualitative data analysis begins with preparing the data, which means organizing the data, documents, field notes, or other visual data. Taped interviews are transcribed. Exploring the qualitative data is next and involves making notes about early observations as well as developing codes. Codes are words, phrases, or even sentences that are assigned to chunks of qualitative data that allow researchers to categorize the data into themes. Data analysis then involves the researcher coding the data by reading the raw data (field notes, interview transcripts, researcher observations, etc.) and assigning codes to small portions of the text. Codes are then grouped together to form themes. Themes can be grouped together to form more themes. Software packages are available to help develop themes in qualitative data.

The findings from the qualitative portion are then represented using rich descriptive narratives, discussions of the themes, and tables, figures, diagrams, or visual models. The qualitative data analysis is then combined with the quantitative data analysis for another round of analysis.

Quantitative Data Analysis

In the quantitative portion of data analysis in the concurrent design, the quantitative data also must be prepared for analysis. Numeric values are assigned to the data using a predetermined coding scheme. The data are entered into a computer software package. The data are “cleaned,” which means that the researcher uses visual inspection and basic descriptive statistical procedures to identify and fix data entry errors. As in qualitative data analysis, exploring the quantitative data is next. This involves using descriptive statistics (e.g. measures of central tendency, measures of variation, and graphical representations) to check for patterns or trends in the data. The researcher chooses an appropriate statistical test to conduct the actual data analysis using inferential statistics.

Three types of inferential statistics techniques are commonly used; the choice of which will depend on the research question. The researcher may find and report a confidence interval, which is an estimate of a population parameter. The second type of inferential statistics commonly used is the hypothesis test. In hypothesis testing, a theory, belief, or hypothesis is tested using the data collected along with rules of probability. The third type of inferential statistics commonly used is regression. The goal in regression is to develop a mathematical model that describes the relationship between variables. For example, regression helps to describe what happens to one variable as another decreases or how much of the change in one variable can be attributed to the other variable changing.

Qualitative and Quantitative Analyses Combined

In a concurrent mixed methods design, the qualitative and quantitative data analysis portions are completed separately and independently of one another. Once completed, the two data sets are merged. Three techniques for merging are typically used.

One technique is to transform one type of data into the other type. For example, qualitative data can be converted to quantitative data by counting the number of times certain codes or themes appear. Quantitative data can be converted to qualitative data by conducting a factor analysis, which is a statistical procedure that allows researchers to categorize quantitative data into themes; these themes can be considered qualitative.

A second technique for merging the two data sets is to use a matrix, which is a table in which the qualitative themes are arrayed along the top of the table and the quantitative results appear on the side of the table. The cells within the table are the conclusions about each category of results.

A third technique for merging the two data sets is to write a narrative discussion. The discussion could be used in conjunction with the matrix or stand alone.

In the discussion, the researcher compares and contrasts the results of both types of analysis and discusses how results converge.

Data Analysis in Sequential Design

Sequential designs are used with explanatory or exploratory designs and some embedded designs. The idea of sequential designs is that one type of data collection and analysis is used to inform the other type of data collection and analysis. If the qualitative portion is first in the design, then the data are collected and analyzed qualitatively first before starting the quantitative portion. The themes that emerge from the qualitative portion are commonly used to inform the quantitative portion in the form of the development of a quantitative instrument. If the quantitative portion is first in the design, then the data are collected and analyzed quantitatively first before starting the qualitative portion. An example of this type of study would be to use the quantitative results to help identify specific cases that can be investigated further using qualitative techniques, such as in the scenario above of following up on high-performing clinics.

Validity and Reliability of Mixed Methodology Research

Efforts to maximize the validity and reliability of research are standard practice in all paradigms. Validity assumes that what is being studied is in fact being studied. Reliability refers to the idea of repeatability or replicability. Would the study, if conducted again, yield similar results? Mixed methodology researchers need to maximize validity and reliability by using established methods to enhance validity and reliability in both qualitative and quantitative research. The exact methods used will vary depending on the research design. With the qualitative data, the methods may include triangulation, member or peer checks, and rich, detailed descriptions and salient quotes regarding the data. With the quantitative data, the researcher may discuss the previously established validity and reliability of the instruments used and identify threats to internal validity.

Future

Mixed methodologists believe there are encouraging signs that this methodology is beginning to develop into a legitimate paradigm. For example, a journal has emerged recently that publishes only studies that use mixed methodology designs. Also, basic research textbooks are starting to include sections on mixed methodology. Mixed methodologists believe that this third paradigm will increase the ability of researchers to draw from the strengths of quantitative and qualitative methodologies without having to sacrifice the strengths of one methodology for the weaknesses of the other. The future of mixed methodology will be fueled by the advantages discovered through the more comprehensive picture created by combining qualitative and quantitative techniques.

References:

Creswell, J. W., & Plano Clark, V. L. (2007). Designing and conducting mixed methods research. Thousand Oaks, CA: Sage.
Greene, J. C., & Caracelli, V. J. (Eds.). (1997). Advances in mixed-method evaluation: The challenges and benefits of integrating diverse paradigms. New Directions for Evaluation, 74. San Francisco: Jossey-Bass.
Hanson, W. E., Creswell, J. W., Plano Clark, V. L., Petska, K. S., & Creswell, J. D. (2005). Mixed methods research designs in counseling psychology. Journal of Counseling Psychology, 52(2), 224-235.
Johnson, R. B., & Onwuegbuzie, A. J. (2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33(7), 14-26.
Morgan, D. L. (2007). Paradigms lost and pragmatism regained: Methodological implications of combining qualitative and quantitative methods. Journal of Mixed Methods Research, 1(1), 48-76.
Tashakkori, A., & Teddlie, C. (Eds.). (2003). Handbook of mixed methods in social and behavioral research. Thousand Oaks, CA: Sage.