KKV’s Designing Social Inquiry as a Common Ground
Djayadi Hanan

My position with regard to KKV’s Designing Social Inquiry is that I agree and support their main purpose and general advices for both quantitative and qualitative researchers to focus on the importance of scientific research design in making sure that their research will be valuable. I am, however, skeptical to several parts of their arguments and its implications to the practice of research and to the scientific community especially on the side of qualitative tradition.

The main argument of KKV is that the differences between qualitative and quantitative traditions are actually “only stylistic and are methodologically and substantively unimportant” (1994:4). Why this is so? Because “all good research can be understood—indeed is best understood—to derive from the same underlying logic of inference. Both quantitative and qualitative can be systematic and scientific” (ibid: 4-5). Statistical or quantitative technique is not always necessary. Instead, the importance is that the researcher pays a great attention to the “rules of scientific inference” (ibid: 6). Scientific research in social science according to KKV can be conducted when the research’s design can approximately meet four characteristics. The research is designed to reach inference, to use the explicit procedure in which public can see it and other researcher can replicate it, to be aware that the conclusion is uncertain, and to adhere to a set of rules of inference on which its validity depends (ibid: 7-9).

I would agree with KKV’s argument at this point because in fact, when the research is conducted the researcher usually combines the qualitative and quantitative data and analysis. Mahoney and Geertz (2006) for example, after examining ten criteria of qualitative and quantitative research, conclude that the labels quantitative and qualitative cannot explain the difference between the two traditions. All quantitative analysis also rely heavily on words for interpretation. On the other side, qualitative analysis quite frequently uses quantitative information or data. KKV themselves provide a number examples of research that rely both on quantitative and qualitative techniques such as Martin’s Coercive Cooperation and Putnam’s Making Democracy Works. Another recent example is from Bates et.al (1998) on The Politics of Interpretation: Rationality, Culture, and Transition which discusses the democratic transition in Zambia and ethnic violence in Yugoslavia by combining the quantitative methods of game theory and rational choice and interpretive approach.

An attempt to have or to propose a kind of what Laitin (1995) calls “common vocabulary and common standard” is very logical if we want to have a kind of common ground between quantitative and qualitative traditions. This common ground should be minimum or broad enough to include as many approaches as possible without losing its capability of disciplining the thought (ibid: 454). In line with this argument, the minimum standard of characteristic of scientific research proposed by KKV, in my opinion, is broad enough to include statistically minded scholars, formal modelers, comparativists, thick description proponents, and interpretivists. None of these research traditions will deny that they seek inference in their research, all of them use certain procedure in their endeavor, not only interpretivists will agree that their conclusion is uncertain but the statistically minded scholar also will say that their conclusion is probabilistic, and finally all approaches will agree as well that they use certain rules in making inference. Therefore, in a general sense, I think, KKV’s proposal for a common ground should be supported.

KKV also argues that there are two kinds of inference: descriptive and causal. Descriptive inference is “the process of understanding of unobserved phenomenon on the basis of a set of observations (KKV, 1994: 55). Causal inference is meant to meet the need for causal explanation of the phenomena that the researcher studies. Causality is defined in terms of causal effect: “the difference between the systematic component of observations made when the explanatory variable takes one value and the systematic component of comparable observations when the explanatory variable takes on another value” (ibid: 82). The differentiation between descriptive and causal inference enables KKV to bridge between the claim that social science research seeks for understanding particular event and that social science research seeks for generality. KKV believe, “where possible, social science research should be both general and specific” and “timeless and timebound at the same time” (ibid: 43). Descriptive inference can focus the researcher to understanding while causal inference to explanation. Depending on what the real subject of investigation is, the researcher can stop at descriptive inference or further pursue the causal inference.

Another point that KKV make clear is that social science is a collective enterprise. It is an endeavor that is conducted by many researchers across time, field, and method/tradition. Qualitative researchers are usually accused of working only with single observation such as a case study (small-n studies). In quantitative tradition, the problem of small-n is cured by increasing the number of observations then it will be able to be used for testing hypotheses or theories. This procedure cannot be applied to qualitative research because it deliberately deals with small-n or even single observation. Does it mean that qualitative research cannot make causal explanation? KKV’s answer to this is no.

According to KKV, because social science is a collective enterprise, “a single observation can be useful for evaluating causal explanations if it is part of a research program. If there are other single observations, perhaps gathered by other researchers, against which it can be compared, it is no longer a single observation” (ibid: 211). In their response to their critics KKV (1995) gave an example of this from the work of Lijphart on The Politics of Accomodation. Lijphart, in his research in 1968, intended to test the causal effect of the pluralist theory which says that the existence of cross cutting cleavages will increase the likelihood of social peace and legitimate and stable democratic government. Lijphart conducted his observation only in one country: Netherland. He found that this country had deep religious and class cleavages with very few cross cutting. He further found however that Netherland, by pluralist theory measurement, is socially peace with legitimate and stable democratic government. Because Lijphart based his study on previous scholars by measuring different variable (social peace instead of social conflict as previous researchers did), Lijphart observation cannot be considered as one observation. Therefore, his inference is valid and can be used to rule out the previous one. This is an example of collective enterprise. This is also the reason why qualitative research, which, for example focuses only descriptive inference, can be complementary to quantitative research which seeks causal explanation because any research with good scientific standard as proposed by KKV will contribute to this collective enterprise.

As many other critics of KKV, as a political science student, I am skeptical to several parts of KKV’s account. I will just point out some of them here.
Although KKV maintains that qualitative and quantitative approaches are not superior to each other, they clearly imply that causal inference is superior to descriptive inference. They for example states: “analysis is incomplete without causal inference… just as causal inference is impossible without good descriptive inference, descriptive inference alone is often unsatisfying and incomplete” (1994: 75). In their discussion on the chapter of descriptive inference (chapter 2), KKV tend to put most of the qualitative interpretivist works into descriptive. The definition of descriptive inference itself points out to “understanding,” the word which usually attached to the aim of qualitative works. Wendt (1998: 108) also concludes that KKV in their “Designing Social Inquiry, … accept that Understanding or constitutive theory is a distinct intellectual activity, but consider that activity to be descriptive inference.” The implication is that the qualitative works will still fall into second to quantitative ones. One of the proponents of KKV, Laitin (1995: 455) describes:

With a disciplinary division of labor, the search for valid causal inferences invites participation of scholars on both sides of our present disciplinary divide. …the discipline is open to pure describers. Historical and anthropological interpretation are potentially fundamental for us, just so long as researchers in this mode seek to distinguish what is systematic--and what, random-in the particular events they are interpreting.

Wendt (ibid) further argues that treating understanding or constitutive theory as descriptive will “inevitably have the effect of reinforceing the prejudice that it is second best, inferior, and not fully ‘science’”. This will render the main aim of KKV to provide common ground for both traditions.

Characterization of KKV to most of qualitative works as descriptive raises question. Is it true that understanding is only descriptive? According to Wendt (1998), although not in terms of causality, understanding or constitutive theory indeed gives explanation. He gives example in which when a child ask his father “what is a dog?” and the child’s father answer with “man’s best friend”, this answer is not only descriptive. It is also explanatory because it is explaining the role of dogs in human society and that dogs are not dangerous to human being (best friend). Besides that, according to Wendt, inferences, including descriptive inference, are always based on theories and the nature of theories is explaining.

Since causal inference is one most important goal in scientific research, KKV provide advices to qualitative researchers in order for them to achieve this goal. Although at the beginning of the book KKV clearly send signal that they “do not wish to encourage the exclusive use of quantitative techniques” (1994: 6), their advices to qualitative researchers to achieve causal inference are mainly statistical. This has invited heated criticism from qualitative researcher such as Alford (1995). He (1995: 425) writes: “The result is a set of inappropriate rules of causal inference for qualitative research, rules that cannot deal with dynamic and complex processes occurring in one or a few cases.” The problem, according to Alford is not on the researcher that includes too few cases or has no control at all on their explanatory variables, but the problem is on the view of the theory as a hypothesis stating that two or three variables are causally related.

This then raises another skepticism. Because KKV’s advices to qualitative researcher are mainly statistical: Can it overcome epistemological question that qualitative researcher is facing? Bartells (in Laitin, 1995) for example pointed out that the problem of context in area studies, which requires them not to seek generality, cannot be overcome merely by statistical convention such as increasing the number of cases. Context in this research tradition holds that it is “highly determinative of outcome yet itself not subject to variable analysis” (ibid: 454). Formal modeling and statistical method will be difficult also to be reconciled with Schwartz’s (1984) suggestion that comparative political scientist needs to study political participation from the perspective of subjectivity. Studying participation according to Schwartz must “avoid simply imposing the researcher’s theoretical assumption onto the data and begin to appreciate the diversity of self understanding that the people hold” (ibid: 1139). Statistical advice from KKV will also be difficult to be applied to the study of Rudolph and Rudolph (2003) on subjective knowledge which analyzes the diary of one person to understand and explain identity formation in British India. The main point here is that statistical method will not be able to accept that subjective knowledge as part of science. In the beginning of their book, KKV admitted that they are dealing with objective knowledge. They states: “we assume that it is possible to have some knowledge of external world but that such knowledge is always uncertain” (1994: 6).

Another skepticism to KKV’s common ground is about how far their “precepts and rules” can encompass multiple methods especially in qualitative tradition. McKeown (1999) has made an assessment that given their strong reliance on statistical method in framing scientific endeavor and their position that view the superiority of causal than descriptive inference, the status of several research techniques in the study of international relations remain unclear. Included in this category, according to McKeown are the constructions of decision or game trees, and computer language, or ordinary language representations of a decision making process (ibid: 175). According to Alford (1995: 426), interpretivist and historical works in KKV’s account are “downgraded to preparation for the genuine scientific work of causal inference.” Besides that, Alford added that KKV also ignore the development in rhetorical and cultural studies by emphasizing formal modeling and statistical criteria.

It seems to me that KKV, through this book, will reach only those researchers in the same tradition as theirs who see that their reliance on quantitative techniques is not enough in answering several or some part of their research questions. Putnam’s Making Democracy Works piece can be a good example of this. According to Tarrow (1995), Putnam, after spending about two decades conducting and administering his research by using quantitative design and techniques, faced difficulties in explaining big differences that he found between Italy’s north central and southern regions. His quantitative analysis only provided indirect evidence. As a way out, he turned to history from which he discovered that civic tradition that provides “social capital” was available in the north while lacking in the south.

Finally, this account, which is aimed to be the common ground between quantitative and qualitative tradition, was written by positivists, but the spirit is patronizing to qualitative researchers. This is not a good writing strategy since it sends the message that the authors still hold their superiority view over the qualitative tradition. Indeed in the substance of the account this superiority feeling is still implied, as I have mentioned above. This book, to be more productive, should have been written in another fashion: instead of the quantitative researcher patronizing and advising the qualitative researcher, it will be much better written in a fashion where quantitative researchers try to show how qualitative work can be fruitful for quantitative works. That way it will serve its aim of providing common ground. Or to be even more advanced: the book should have been written collaboratively by quantitative and qualitative researchers.

References:

Alford, Robert R. “Review: Designing Social Inquiry: Scientific Inference in Qualitative Research”. by Gary King; Robert O. Keohane; Sidney Verba. Contemporary Sociology, Vol. 24, No. 3. (May, 1995), pp. 424-427.

Bates, Robert H. et.al., “The Politics of Interpretation: Rationality, Culture, and Tradition.” Politics and Society, Vol. 26, No.4 (Dec. 1998), pp. 603-642.

King, Gary. et.al. 1994. Designing Social Inquiry: Scientific Inference in Qualitative Research. Princeton, New Jersey: Princeton University Press.

_________ “Review: The Importance of Research Design in Political Science.” The American Political Science Review, Vol. 89, No. 2. (Jun., 1995), pp. 475-481.

Laitin, David D. “The Qualitative-Quantitative Disputation: Gary King, Robert O. Keohane, and Sidney Verba’s Desiging Social Inquiry: Scientific Inference in Qualitative Research.” The American Political Science Review, Vol. 89, No. 2. (Jun., 1995), pp. 454-456.

McKeown, Timothy J. “Case Studies and the Statistical Worldview: Review of King, Keohane, and Verba’s Designing Social Inquiry: Scientific Inference in Qualitative Research.” International Organization 53, 1, (Winter 1999), pp. 161-190.

Rudolph, Lloyd I. and Susanne Hoeber Rudolph. “Engaging Subjective Knowledge: How Amar Singh’s Diary Narratives of and by the Self Explain Identity Formation.” Perspectives on Politics, Vol. 1, No. 4, (Dec. 2003), pp. 681-693.

Schwartz, Joel D. “Participation and Multisubjective Understanding: An Interpretivist Approach to the Study of Political Participation.” The Journal of Politics, Vol. 46. No. 4. (Nov., 1984), pp. 1117-1141.

Tarrow, Sidney. “Review: Bridging the Quantitative-Qualitative Divide in Political Science”. The American Political Science Review, Vol. 89, No. 2. (Jun., 1995), pp. 471-474.

Wendt, Alexander. “On Constitution and Causation in International Relations.” British International Studies Association, (1998). pp. 101-117.