3 Theory
By Salih O. Noor
3.1 Introduction
Most people may not know much of anything about theory. Theory is either so “esoteric and complicated as to be incomprehensible" or "so commonplace and obvious as to be platitudinous” (Shoemaker, Jr, and Lasorsa 2003, 5–6). Either way, to most people, theories seem to be of little use. In reality, however, people use theories every day about friendship, dating, success, and so on. Political scientists rely on theory to analyze public opinion or predict election results, and weather analysts apply theory to forecast weather conditions. Most people, however, misunderstand what a theory is and what a theory does.
In this chapter we will study the meaning, significance, and building blocks of theory as well as theory-building and theory testing procedures. In the second section, we will discuss what theory is and is not, and how empirical theory differs from other kinds of claims or theories in its application of the scientific method. In the third section, we will learn some characteristics that define a good theory, discussing four very important elements of a well-crafted theory. In the fourth section, we will try to understand literature review and its importance to theory. In the last section, we will study the relationship and differences between theory-building and theory-testing, in addition to inductive and deductive reasoning and procedures in theory-building. For elaboration, we will draw at all stages on various examples from the social (and when necessary the natural) sciences, including two check boxes on the scientific method and on examples in theory-building and testing.
3.2 What is a theory?
A scientific theory is a set of logically consistent statements that tell us why the empirical social and political phenomena we observe, or the relationships between them, occur in the way they occur. More formally, a theory “is a system of constructs (concepts) and propositions (relationships between those constructs) that collectively present a logical, systematic, and coherent explanation of a phenomenon of interest within some assumptions and boundary conditions” (Bacharach 1989, 496). In short, a theory is an interrelated set of propositions about empirical reality. These propositions are comprised of (1) concepts that introduce basic terms of the theory; (2) assumptions that relate the basic concepts to each other; and (3) generalizations that relate the statements to a set of observations or, simply, report the findings on observed relationships. It is important that these propositions are “logically consistent” in that they must all be true at the same time; the theoretical concepts, assumptions, statements should be coherent with each other. Concepts, variables, and hypothesis are the building blocks of theory.
For example, the “logic of collective action” is a theory that aims to explain the dilemma of collective action and public goods. Formulated by political scientist Mancur Olson, Jr. (Olson 1965), the theory explains when (and why) do some collective groups (such as trade unions, social movements, or college students) organize better to achieve public goods (like increased wage, policy change, or improved campus security) than other groups. Olson found that the interests of highly coherent minority groups can be overrepresented, and the interests of majorities get marginalized due to the “free-rider” problem. Collective action is difficult because individual members always have incentives to "free ride" on the efforts of others, because “public goods” — goods or services that are available to every member — are by definition non-excludable (i.e. one member cannot reasonably prevent another from consuming them) and non-rivalrous (i.e. one person’s consumption of the good does not affect the others’ chances). As a result, some members (e.g. workers) can expect to enjoy public goods, such as increased wages and improved workplace conditions without bearing the costs of participating in a strike (e.g. time, money, or physical harm). In particular, large groups face tremendous challenges for collective action than small groups, because individuals in large groups gain less per capita of a successful collective action due to diminishing returns. On the contrary, small groups can provide selective incentives to their members and a prospect of greater rewards for each a successful collective action due to small number of members. As a result, Olson concludes, it is highly possible that a minority group bound together by concentrated selective incentives can dominate a majority social group. In so observing, Olson refuted previous theories that held (a) individuals in a group (of any size) will act collectively to achieve their common interests, and (b) the greatest threat in a democracy is, due to the majority’s sheer numbers, “the tyranny of the majority”.
A theory should explain why things happen, rather than just describe or predict. It is entirely possible to predict events or behaviors using a set of predictors, without necessarily explaining why such events are taking place or why they take place together. For instance, stock market analysts predict fluctuations in the stock market based on market announcements, earnings reports of major companies, and/or new data from the Federal Reserve, based on previously observed correlations. In contrast, theoretical explanations require causation, or the understanding of cause-effect relationships. Establishing causation requires four conditions: (1) correlations between two concepts, (2) temporal sequence (the cause must precede the effect in time), (3) causal pathway (causal mechanism that link cause to effect), and (4) rejection of alternative hypotheses through testing (Bacharach 1989, 496–515)
Theoretical explanations can be idiographic or nomothetic that vary in their theoretical premise and explanatory scope. Idiographic explanations are those that explain a single situation or event, say unemployment in the state of Illinois, in idiosyncratic detail. The explanation is detailed, accurate, and valid, but it may not apply to other similar situations, say other states, and is hence not broadly generalizable. In contrast, nomothetic explanations seek to explain a class of situations or events, for example unemployment in several US state, rather than a specific situation or event. Because nomothetic explanations are designed to be generalizable across contexts (events, or people, countries), they nonetheless tend to be less precise, less complete, and less detailed. As such, idiographic and nomothetic explanations rely on different assumptions of causality, different analytical tools, and different approaches to theory-building. Methodologically speaking, therefore, the two approaches to social science theory often fall along the qualitative-quantitative divide; the first typically uses small-N methods of analysis (e.g. cross-case analysis, within-case analysis or process tracing, and set theory) for one or few number of cases, while the second applies large-N methods of quantitative analysis (e.g. large-scale surveys, statistical analysis, regression) to a large number of cases. Further on these methods, read the chapters on small-N and large-N analysis.
Theories are important in the social and political sciences. They help us, among other things, to understand the nature of political and social phenomena (such as political events, behavior, institutions, and processes), to explain observed regularities among these phenomena (i.e. causal relationships between events or processes), to make predictions about as yet unobserved relationships (e.g. the possible effect of immigration policy on the 2020 US presidential elections), and to take a particular policy action (e.g. universal healthcare to reduce high healthcare costs). Without theories it is hard to have valid knowledge of political events, behavior, and processes, or tools to understand the relationships between different political events and processes.
However, theories can also have their own share of (systematic or non-systematic) limitations. As simplified explanations of reality, theories may not always provide adequate explanations of the phenomenon of interest. While social reality is often more complex, theories are designed to be simple and parsimonious explanations based on a limited set of concepts/variables and concept/variable-relationships. Furthermore, theories may impose cognitive blinders or limit researchers’ “range of vision,” causing them to miss out on important concepts that are not identified by the theory (i.e. omitted variable). The nature of these limitations sharply vary between small-N and large-N theories, with the strengths of one being the limitations of the other.
For a better understanding of what theory is, it is good to think in terms of what theory is not. First and foremost, the theory – i.e. empirical theory – we are concerned with here, such as Olson’s “logic of collective action,” is epistemologically different from normative political theory in political or general philosophy. Empirical theory is concerned with the examination of empirical political and policy matters through the scientific assessment of empirical evidence rather than, as political theory, with the realm of political ideas, values, and norms from a normative perspective. The latter is typically concerned with questions of overtly normative nature, such as: What system of government best guarantees freedom, justice, and equality in society?; When is obedience to a ruling power justified, and when is disobedience not justified?; Or how citizens ought to behave towards their rulers or the state? Empirical social theory rather inquires, for example, how and why a particular political system (e.g. democracy, dictatorship, military regime) emerges, why citizens behave in a particular way towards their government or leaders, or what caused voters to support the Democratic Party over the Republican Party in the 2016 U.S. Presidential Elections. The latter also differs from normative theory in terms of the tools, methods, and techniques applied in answering questions about the social and political world around us.
Social science theories are generated through the application of the scientific method – or the principles and procedures of interpreting the empirical world through objective, value-neutral observation of facts. Put simply, the scientific method is a process of guessing and verifying to reach descriptive or causal explanations—i.e. making assumptions/ hypotheses about the real social/political world, examining evidence (data) gathered from that world, and confirming (or disconfirming) those hypotheses in view of the evidence. Even though there is no social scientific method clearly written down that is followed by all scientists, it is possible to identify five steps associated with the method:
Formulate a question after observing a social/political puzzle;
Develop a theoretical model/framework to explain it;
Propose a hypothesis/testable implication;
Test hypotheses against evidence; and
Confirm/reject the hypothesis after analyzing the evidence.
The scientific method stipulates clear and logical steps (Checkbox 1) that must be strictly followed in our search for explanations. Social scientists develop theories through the formulation of a question, proposing hypotheses about what they think the answers are, testing the hypothesis against evidence collected and examined in an objective and systematic manner, and drawing theoretical conclusions that are falsifiable through the iterative application of the scientific procedures. Therefore, empirical theory is different from normative political theory in that the latter relies on tools other than the scientific method to deal with normative and ethical questions. Normative questions ask for a normative response, seeking an indication of what is good or of what should be done; ultimately, the answers involve what someone likes or dislikes, values or rejects. The scientific method cannot provide the answers without regard for an individual’s personal values or preferences.
Checkbox 1: The Social Scientific Method
STEP1: Research Question,The first step in the scientific method is to observe the world and come up with a question. The very need for a theory begins when we observe something that is so puzzling that we ask “why did it occur?” or “what caused it to occur?” What makes the observation a puzzle worth exploring is that the observation does not fit with some prior expectation or theory that we held to be true about how the world works. Therefore, we always have a preexisting theory or expectation when we observe the world that leads to a new puzzle or question.
STEP 2: Theory or model The next step after observing something puzzling is to develop a theory (also sometimes called theoretical framework or model) to explain it. This is a set of logically consistent statements that tell us why the things that we observe occur in the way they do. The task here is to propose an explanation for the phenomenon the researcher is interested in understanding. Developing a theory requires imagination and creativity to fathom the social world, to impose some analytical order on an otherwise complex world. In short, the model will be a simplified picture of the world; it will be something that helps us understand some relationships between two or more empirical phenomena. A good model, therefore, contains only what is needed to explain the phenomenon that puzzles us and nothing else. At times, this step involves developing a theoretical framework or structure that can hold or support the theory. A theoretical framework consists of concepts, variables, and the theoretical assumptions of the theory that explains the problem under study. It is the conceptual basis for understanding, analyzing, and designing ways to investigate relationships within social systems.
STEP 3: Hypothesis (Implications) Once we have a model, the third step in the scientific method is to deduce implications from the model. Our model will presumably provide a logical explanation for the puzzling observation that we started with; after all that is what it was designed for. To actually test the model and allow for the possibility that it can be falsified, we will have to find other implications that can be deduced from it. We must ask “If the prior world that we created to explain the phenomena that we originally found puzzling really did exist, what else ought to exist? What else should we be able to observe?” Good models are those that produce many different implications because each prediction represents another opportunity for the model to fail and, thereof, makes the model easier to falsify. If the model fails to be falsified, we gain more confidence in its usefulness. Good models also produce small surprising implication –i.e. they tell us something we would no know without the model. Models are not particularly useful if they tell us only what we already know.
STEP 4: Test Hypotheses The fourth step is to examine whether the implications of the model are consistent with observation. We should not dogmatically uphold the implications of our model or defend them to prove they are right. On the contrary, we should try our best to falsify them because it is only after a theory has withstood these attempts that we can reasonably have confidence in it. Testing the implications that are most likely to be falsified is particularly important. Always subject a model to the harshest test that you can devise. It is also standard to ask if other (existing) models might also explain the phenomena of interest. In this case, the researcher should compare the implications of those other models with the implications of her own model. It is always the case that competing models have some of the same implications, yet they will differ in some other implications (otherwise they are not different models). The trick is to identify these points of conflict between the different models and the relevant observations in the real world that would help decide between them. This –called critical test – allows the analyst to use observation to distinguish between two or more competing explanations of the same phenomenon. After all there is only one world and only one of the models can be consistent with the real world.
STEP 5: Evaluation Confirmation or refutation of the theory is the last step in the scientific method. Our theory has been confirmed if we observe the implications deduced from our theory. Note that we cannot say our theory has been verified or proven because we can never prove or disprove a scientific explanation. Scientific method is a means to “provisionally” understand the world, and scientific theories serve as provisional explanations of the world contingent on better methods, better analytical tools, and better evidence. Our theory may or may not be true. All we can conclude, if the observations are consistent with our theoretical implications, is that our theory has not yet been falsified. We cannot rule out the possibility that it can be falsified the next time it is tested. (Clark, Golder, and Golder 2017)
Second, a theory is not the same as a model or paradigm. Theory and model are related terms and not infrequently confused. But the two are different from each other in their definition, purpose, and application. First, as defined above, theory is a conceptual framework or general explanation of an idea. A model (not the same as theoretical model) by contrast is a verbal or a visual representation of a concept in order to make the understanding of something easier and clearer. Second, the purpose of a theory is to explain things and is less practical, whereas a model is meant to simplify things and is more practical. The social and political world is immensely complex; models present a simplified picture of the world that puzzles us. Models present in simple and concise manner concepts, assumptions, and claims, which are the building blocks of theory. Models are commonly used in all political science, but game-theoretic models in rational-choice approaches represent the most popular forms of modelling the behavior and actions of rational actors like voters, politicians, special interest groups, and states. For example, in Olson’s theory of collective action individuals are modelled as rational, interest-maximizing actors who act only under circumstances that maximize their interests. This simple model illustrates an otherwise complex social and mental reality of actors interacting in large group contexts. Therefore, theory and model coexist in the same world of social science inquiry, yet they differ, and the failure to realize this difference can lead to confusion and perhaps in disillusionment. Theories should be understood as explanations or conclusions about certain situations or problems, while models as heuristic devices that help us understand, through concepts and theories, how some aspects of the world work and explain it to others. Models, therefore, can represent a theory but they cannot be a substitute for theory.
Read (Shoemaker, Jr, and Lasorsa 2003), chapter 7, for a greater discussion of theory versus model, and (Clark, Golder, and Golder 2013), pages 121-137, for examples of game-theoretic models.
Third, a theory is not a paradigm. A paradigm is a broad, general framework or approach that defines a particular scientific discipline. It is a distinct set of concepts and assumptions, including theories, research methods, postulates, and standards that guide scientific inquiry in a particular community of scholars. It determines the kind of questions supposed to be asked and their structure, the assumptions made, the methods used, and how the results should be interpreted (Kuhn 1996, 10). Scientific paradigms set the standards for studying the empirical world, while theories are explanations of some aspects of that world. In addition, unlike theory, a paradigm is not actually testable per se. Examples of paradigms in political science include systems theory, rational choice theory, comparative historical analysis, neo-liberal institutionalism, and constructivism.
Fourth, and last, social scientific theories are general explanations, and not “covering laws” of political and social behavior. It is possible to have law-like theories in the natural sciences with universal applicability to all natural phenomena; theories of electromagnetism, evolution, and relativity are some examples. This because natural phenomena display behavior and (causal) regularities that are uniform across time or space. For example, water boils at 100 degree centigrade almost always whereas, according to Albert Einstein, light travels at a speed of 186,000 miles/second, and is unchanging. As Max Weber argued, the laws that regulate social relations are quite different from the laws that govern nature; regularities in human behavior and the physical world are fundamentally different because the former display a great degree of irregularity, fluidity, and heterogeneity. Unlike natural events, political events and processes do not lend themselves to the same explanatory logic as is found in physics and the other hard sciences.
This is to neither say that human behavior is devoid of regularities nor law-like generalizations to explain it are entirely impossible. It is not rare that social scientists seek to identify such regularities and develop general explanations; examples include: Duverger’s law of plurality voting and two-party system (Duverger 1954), modernization theory on modernization and democracy (Lipset 1959); and Moore’s “No bourgeoisie no democracy” hypothesis on the middle class and democracy (Moore 1966). These theories validly explained a broad range of historical observations, but their applicability turned out to be limited to a particular context—i.e. mostly advanced Western democracies before mid-twentieth century—which signifies that the utility of social scientific theories is context- and time-specific because regularities in human behavior hinge on the given cultural, political, and economic context. Most social scientists aspire to produce generalizations about the world; in fact, a central goal of scientific analysis is to generate concepts, models, and theories that travel across time and space. However, social and political phenomena are characterized by complexity, randomness, and diversity to yield themselves to law-like, universal theories. Cause and effect greatly vary across countries, cultures, regions, and historical contexts. What obtains to observations in a specific context often does not apply to other observations in a different context. The demise of modernization theory after the 1960s was precisely because education, urbanization, and industrialization (i.e. modernization) in the Third World did not cause democracy but instability, revolutions and dictatorships. Moreover, the more general a theory is (i.e. it explains too many observations), the less is its explanatory power concerning each observation. In fact, a social science theory that explains everything does not explain anything. Due to the complexity of causality, therefore, social science theories are judged less by their universal applicability than by their validity and robustness in explaining a particular set of observations. Theoretical generality and specificity are two competing goals in theory-building, with large-N (quantitative) analysis associated with the former and small-N (qualitative) with the latter.
3.3 What is a good theory?
A good theory should explain previously puzzling facts, be logically consistent, and produce potentially falsifiable predictions. It builds on existing theories, has clearly specified concepts (valid conceptualization) codified as measurable variables (valid measurement), and clearly shows the relationship between the concepts (causal pathway). Even though the standards for a good theory are debatable, particularly among qualitative and quantitative traditions, social scientists agree on some basic elements of what makes a good theory. We will discuss here four major characteristics of a good theory.
Parsimony is the first such element. How simple is the explanation? The simplest theory (i.e. one that uses the smallest number of variable or makes the fewest assumptions) is considered the best. A theory is considered as parsimonious when it has the ability to explain often complex phenomena in relatively few terms and statements. A parsimonious theory can specify the causal relationship (X—>Y) in clear terms using a causal model (which might involve multiple variables and relationships) that reasonably simplifies a complex empirical reality in to something comprehensible.
The second feature is generalizability or theoretical coverage. A good theory is generalizable when it has the power to explain a broad range of similar cases or phenomena outside the context of that study. In other words, the conclusions of a scientific theory are applicable to other contexts not included in the study, which is also referred to as the external validity of a theory. In qualitative research, this criterion is less important because theory is generated from a small set of cases and is less applicable to other contexts. Qualitative analysis rather puts greater emphasis on the internal validity of a study or the extent to which the theoretical claims are based on valid methods of analysis and evidence about cause and effect. Theoretical claims or inferences possess internal validity if claims of a causal relationship between two variables demonstrate that the "cause" occurrence before the "effect" (temporal precedence), the "cause" and the "effect" tend to occur together (covariation), and there are no alternative channels or mechanisms that explain the observed variation (nonspuriousness).
Observable implications or the ability of a theory to help make more accurate predictions about new unobserved instances is the third quality of a good theory. Strong theories have strong observable implications or the things we would expect to observe in the real world if our theory is right. For example, the preference theory of judges states that judges want the law to reflect their ideological preferences; and, because they lack an electoral connection, they are free to vote in accord with their ideological preferences. If this theory is correct, we should observe judges generally voting in accord with their ideological preferences, such that conservative judges cast conservative votes and liberals, liberal votes.
The fourth and last criterion used to judge a social scientific theory is falsifiability or its refutability. A good theory must be falsifiable or liable to refutation when subjected to tests using new observations or new evidence; it must be possible to identify a possible outcome of test or observation that conflicts with predictions of a given theory. In fact, according to the philosopher of science Karl Popper who introduced the concept as the basic principle of scientific inquiry, statements and theories that are not falsifiable are unscientific or not based on the scientific method. The most common way in the social sciences to support falsifiability (or safeguard against invalid refutation of a theory) is to specify the scope conditions or assumptions under which a theory is applicable. Scope conditions are parameters or boundaries specified by the analyst that identify the types of empirical contexts or observations to which the theory applies. For example, we can state that the preference theory of judges is applicable under the condition that judges vote in accordance with their ideological preferences only in the absence of a liberal (i.e., a potential whistle-blower) on the panel. The theory may be falsified when we observe that, say, conservative judges fail to cast conservative votes even in the absence of a potential whistle-blower.
3.4 Literature Reviews and Theory
We noted in the first section that developing an explanation begins with a puzzle and a research question. The first major task in a research effort often is to find a puzzling topic and to translate a general interest in a topic into a manageable research question or series of questions. Framing an engaging and appropriate research question will get a research project off to a good start by defining, and limiting, the scope of the investigation while a poorly specified question inevitably leads to wasted time and energy. But most students, when confronting a research project for the first time, either do not have a well-formulated research question as their starting point or any specific interest or topic in mind at all. We may also not know whether explanations, that fully or partially address the puzzle we have observed, already exist. To address these challenges the first major task is to conduct a literature review; i.e. to examine systematically scholarly literature that is relevant to the puzzle. Why is this important? How does thoroughly studying extant literature contribute to theory?
A literature review is a survey of books, scholarly articles, and other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem at hand. It is designed to provide an overview of sources you have explored while surveying a particular topic and to demonstrate to your readers how your research fits within a larger field of study (Fink 2013, 5). Good research involves reviewing previous work to motivate and sharpen a research question. Reviewing relevant literature also contributes to theory development for several other reasons. Among these are: (1) to gauge what has and has not been studied, (2) to develop general explanations for observed variations in a behavior or a phenomenon, (3) to identify potential relationships between concepts and to find hypotheses, (4) to learn how others have defined and measured key concepts, (5) to identify data sources that other researchers have used, and (6) to develop alternative research designs. Lets further discuss some of the reasons that are more crucial to theory development.
Often times, a researcher or student will start off by expressing only a general interest in a topic, such as gun violence or the effects of campaign advertising, but the specific research question has yet to be formulated; for example, “What is the social background of individuals who engage in mass shooting?” or “Do negative TV campaign advertisements sway voters?” A review of previous research on these topics can help you carve a research topic by identifying research questions that others have addressed.
A researchers, on the other hand, may start with an overly specific research question such as "Do evangelicals have different views on abortion policy than non-evangelicals?" Reading the literature on public opinion on abortion will likely reveal that your specific research question is one of many aimed at answering the more general research question: What are the social attributes of people who are opposed to abortion, and do they differ from those who support abortion access? Compared to the former question, which is too narrow to sustain a research paper, the latter research question constitutes a topic that is likely to lead to theoretically crucial conclusions and more observable implications.
A literature review also can help you to identify gaps or analytical shortcomings in the literature. Here, you may find that, after reading the scholarly work in an area, previous research does not adequately answer the question for lack of effective research tools, sufficient data, and/or appropriate theoretical approach. You may design a new research project to answer an old question in a novel way using new data. A study may also replicate a previous study to confirm or challenge a hypothesis or expand our understanding of a concept. Replication is one of the cornerstones of scientific work; by testing the same hypothesis through different research design or confirming the results from previous research using the same data and methods, we can increase our confidence that the results are valid.
At other times, a researcher may begin with a hypothesis to develop an explanation for a relationship that has already been observed. Here, a literature review may reveal similar observations made by others previously and may also help you develop general explanations for the relationship by identifying theories that explain the phenomenon of interest. Your research will be more valuable if you can provide a general explanation of the observed or hypothesized relationship rather than simply a report of the empirical verification of a relationship.
A researcher, on the other hand, should be alert for competing or alternative hypotheses rather than just seeking theories that support the plausibility of own hypothesis. Here, you may start with a hypothesis specifying a simple relationship between two variables. Since it is rare for one political phenomenon to be related to or caused by just one other factor or variable (i.e. causal complexity), it is important to look for other possible causes or correlates of the dependent variable (i.e. omitted variable). Data collection should include measurement of these other relevant variables so that you may rule out competing hypotheses or at least specify more clearly the nature of the relationship between the variables (Johnson, Reynolds, and Mycoff 2016, 82–84).
A thorough understanding of existing scholarly work, therefore, is key to formulate an interesting question, test an existing hypothesis or craft new hypotheses, and the development of scientifically valid and useful explanations. Developing skills to understand key concepts and models in the subfield, to critically evaluate and synthesize expert knowledge, and to summarize complex arguments in often a large body of literature are essential for an excellent literature review. Furthermore, personal insight and non-scholarly sources (e.g. newspapers, broadcast media, internet) can be quite helpful in selecting a research topic, and a literature review can encompass virtually anything published on your topic. However, at the very least familiarity with the scholarly literature is strongly encouraged. Relying on scholarly rather than non-scholarly sources greatly improves the quality of a literature review. After all, a literature review is supposed to assess the knowledge about a topic that has been attained and communicated according to scientific principles. Finally, how many books and articles is one supposed to review depends on the purpose and scope of the project, as well as source availability. Obviously, a more complex research topic, or a subject with a larger literature, may require a more in-depth literature review than will a less complex topic or one with a smaller literature. Further readings on: the importance of literature review (Johnson, Reynolds, and Mycoff 2016; Fink 2013; Hart 1998; Ridley 2012; Knopf 2006; Jesson, Matheson, and Lacey 2011) and structure and writing techniques (Cook and Murowchick 2014; Fink 2013; Hart 1998; Jesson, Matheson, and Lacey 2011; Onwuegbuzie and Frels 2016; Ridley 2012; Booth, Sutton, and Papaioannou 2016).
3.5 Theory-building vs Theory testing
Social scientific research may involve many activities such as interpretation of constructs or concepts, describing a social phenomenon (descriptive inference), and identifying links between two or more related phenomenon (causal inference). But the two core activities and goals that underlie most activities (in causal inference in particular) are theory-building and theory testing. Both are interrelated scientific endeavors that apply the scientific method, but they vary in important respects that should be properly understood. As table 1 summarizes, they vary in terms of their epistemological approach, main goals and tasks, and end results. At the end of section, we will discuss three exemplary theory-building and theory testing works in the political science for elaboration; but in the meantime, we will use natural science examples to easily highlight – for the latter are relatively straightforwardness – the differences between the two.
| Theory building | Theory testing | |
|---|---|---|
| Main approach | Inductive reasoning | Deductive reasoning |
| Research goal | Estimating a relationship/ offer an explanation | Evaluating an explanation/ test existing hypothesis |
| Main task | Developing hypothesis; test hypothesis against evidence | Finding evidence to test existing hypothesis |
| Outcome | New or modified theory offering new explanation | Old theory confirmed or refuted |
Table 1: Theory-building and theory testing compared
Theory testing, as the phrase suggests, is the process of testing (verifying) whether a certain theory is a plausible explanation of a phenomenon you would like to investigate. Its goal is to test the validity of an explanation often, but not always, through a research design, new data, and/or data analysis tools. The main focus of theory testing is to discover whether there is evidence that supports (or does not support) a particular theory. Theory testing is relatively easier than theory building. While researchers (scholars and post-graduate students) undertake a much more challenging research task of theory building, students often do research primarily aimed towards theory testing. Still, though, it is critical to deeply understand the theory and how it is used to frame empirical research before you can adequately test it yourself.
To clarify theory testing, take the Anthropogenic Global Warming (AGW) Theory, which asserts that human-caused greenhouse gas emissions are the main cause for the rising global warming levels observed in recent years. Carbon dioxide comprises one of the greenhouse gasses. Carbon dioxide causes water on the surface of the earth to evaporate; increased water vapor in the atmosphere in turn can trap heat coming from the earth thus cause global warming. To test this theory, the first step is to look into the humidity levels associated with carbon dioxide emissions because the theory posits that carbon dioxide causes water to evaporate and trap heat. Greater carbon dioxide means greater water vapor in the atmosphere measured using, say, a wet and dry bulb thermometer. The next step is to find out if there is a correlation between surface humidity and temperature, which should be positive for the theory to be true. The main task of theory testing is thus to find evidence to confirm or refute a theory.If the evidence supports the theory, then no further action is required. If the evidence rejects the theory then you can conclude either the theory is incorrect or the data is inadequate.
Theory building by contrast is an attempt to explain something as yet obscure do novo or in different perspective than has previously been suggested. The goal of theory-building is to provide a framework for analysis to better understand puzzling empirical issues and to help address real world problems. As such, it requires knowledge of the plausible theories explaining the phenomenon currently are, and how they are used in empirical research. Theory building demands the application of higher-level thinking skills compared to theory testing. It requires the synthesis of a broad range of literature, concept formation, the formulation of testable hypotheses, the collection and systematic analysis of data, and evidence-based confirmation or refutation of the hypothesized relationships between cause and effect. To be sure, theory-building can also take place by extending or modifying existing theories to new contexts. Here, a researcher attempts to replicate and/or reexamine previously theorized relationships, identifies new causal mechanisms (or pathways), uncovers previously unexplored relationships between variables, and introduces a new concept (or significantly re-conceptualizes an existing one).
In general, there are four major ways of theory-building:
Grounded theory-building: building theory inductively based on observed patterns of events of behavior in one or few more cases.
Conceptual analysis: building theory inductively by conducting a bottom-up conceptual analysis to identify different sets of predictors relevant to phenomenon of interest using a predefined framework. In one such framework, a researcher looks for different categories of inputs (factors) related to the output (effect), and explain the underlying process that links the two categories or concepts.
Extend/modify existing theory: building theory deductively by extending or reformulating existing theories to explain a new context.
Apply existing theory in new context: building theory deductively by applying theories developed in one context to an entirely new context by drawing up on the structural similarities between the two contexts.
To further clarify the idea of theory building, let’s now consider another example from the hard sciences. To this day, scientists debate what caused the sudden extinction of dinosaurs in what is known as the Cretaceous-Tertiary extinction event, or the K-T event, at approximately 66 million years ago. The leading hypotheses predicted that a giant volcano, sudden cooling down of earth climate, and an asteroid strike was the cause. In the early 1980s, father-and-son scientists Luis and Walter Alvarez suddenly discovered (in Italy) a distinct thin layer of iridium–an element found in abundance only in space–that corresponds to the precise time the dinosaurs died. The researchers deduced that the thin layer of iridium at the K-T boundary was deposited following the impact of a large meteor, comet or asteroid with the earth. Furthermore, this bolide impact (the meteor, comet or asteroid colliding with the earth’s surface) could have caused the extinction of the dinosaurs. However, conclusive evidence – especially evidence of the meteor, comet or asteroid collision with earth – was required to support the theory and to eliminate rival hypotheses. Then, in the 1990s, scientists discovered a massive meteor crater (the Chicxulub Crater), 110 miles in diameter, on the edge of the Yucatán Peninsula, extending into the Gulf of Mexico, which dates to the period in question. Scientists concluded that the 6-mile-diameter bolide that formed the crater struck the earth at 40,000 miles per hour and released 2 million times more energy than the most powerful nuclear bomb ever detonated. The resulting darkness could have plunged the earth’s temperatures into the freezing zone, killing some three-quarters of the plant and animal species on Earth, including dinosaurs, within weeks.
Scientists reached the above conclusion through inductive reasoning –i.e. they used a small piece of evidence (iridium) about a specific observation to reach a more general conclusion. Inductive and deductive analysis – analytical approaches discussed in the previous chapter – play different roles in theory-building and theory testing. The inductive approach (inductive-statistical) is often associated with theory development. t’s a grounded theory-building approach whereby a researcher makes a detailed observation of a case or few cases, to derive broad generalizations and ideas that apply to a broader set of similar cases. Characteristic of qualitative small-N analysis, this approach aims to generate meanings from the data set collected in order to identify patterns and relationships to build a theory. Patterns, resemblances, and regularities are observed in order to reach conclusions (or to generate theory). The deductive (hypothetico-deductive) approach is most often useful in theory testing. Characteristic of quantitative large-N analysis, in deductive analysis a researcher begins with a theory, then conducts research in order to test whether that theory or hypothesis is supported by specific evidence. Extending or modifying an existing theory to fit new reality is a deductive exercise in theory testing.
Whether one applies inductive or deductive analysis, theory-building involves a series of steps from the identification and definition of concepts to the expression of their relationship in a theoretical statement, the construction of a rationale, and the specification of measurements (Shoemaker, Jr, and Lasorsa 2003) [170-171] detail ten steps in theory building, in “How to Build Social Science Theories,” the most important of which are:
Observation: Start with a problem, some unexpected results, an anomaly, an observation of something unusual, something you would like to know the effects of, or something you would like to know the causes of.
Conceptualization: Identify (or formulate) the key concepts involved in the phenomenon of interest. Try to come up with concepts that are observable and measurable.
Hypothesizing: On the basis of careful observation and literature review, try to think of as many causes (or as many effects) of the key concepts as you can. Postulate causal linkages (between your concepts).
Measurement: operationalize key concepts and specify how you will measure them in terms of independent and dependent variables.
Theoretical linkage: Specify the theoretical rationale for the hypotheses. Why should they be expected to be true? Use logic and/or other theories to show your argument is reasonable, to convince that the concepts are causally linked in the way you have specified.
Hypothesis testing: Try to think in terms of multiple hypotheses that are alternative explanations for the same phenomenon. Empirically demonstrate why one (your) hypothesis is true and the other is false.
Checkbox 2: Case Studies in Theory-building and Theory Testing
Theory Building: Some Social Requisites of democracy, S. M. Lipset (1959)
Lipset developed one of the most influential theories of democracy which suggested that some social changes associated with economic development are requisite for the emergence and functioning of democracy. Does economic development lead to the emergence of democracy? And, if so, why? The key concept in his analysis is “modernization” or the transition from traditional, rural, agrarian society to a secular, urban, industrial society. Lipset observed that the average wealth, degree of industrialization and urbanization, and level of education is much higher for the more democratic countries. He then hypothesized that economic development, which he estimated through measures of income, urbanization, industrialization, and education, and the associated basic changes in the class structure, values, and attitudes of society, are the causes for the development of democracy in industrialized countries. In his words “the more well-to-do a nation, the greater the chances that it will sustain democracy” (p. 75). Lipset reasoned that increase in wealth provides economic security to the working class (a guard against revolution); enlarges the size of the middle class, which moderates conflict by rewarding moderate parties and punishing extremist ones; and alleviates lower class threats to the upper class, which opposes democracy when wealth inequalities are extreme. Moreover, increased income levels also improve society’s receptivity to norms of democratic tolerance, and increase voluntary associations that constitute key institutional intermediaries in democracy. Modern education is particularly relevant for cultivating a political culture – i.e. greater voting choice, political participation, tolerance, and media consumption – associated with democracy and political stability. In short, Lipset concluded, without such changes in social structure and values that come with modernization it is impossible for a country to experience transition to democracy and its consolidation. Theory Testing I: Modernization: Theories and Facts, A. Przeworski and F. Limongi (1997), Przeworski and Limongi test Lipset’s theory by reexamining the relationship between economic development and democracy put forth by him. They formulate and test two hypotheses derived from Lipset’s explanation: (a) democracy may be more likely to emerge as countries develop economically – i.e. the endogenous explanation or modernization theory or (b) democracy may be established independently of economic development but may be more likely to survive in developed countries – i.e. the exogenous explanation. Przeworski and Limongi test these hypotheses through a quantitative analysis of 135 countries (224 political regimes in total) for the period 1950-1990, using data on levels of development measured by income per capita. They refute the endogenous explanation by, first, observing that transitions to democracy are “increasingly likely as per capita income of dictatorships rises but only until it reaches a level of about $ 6,000, above which”dictatorships become more stable as countries become more affluent” (p. 159). Their findings confirm the second hypothesis by showing that economic development has a strong impact on the survival of democracies; in fact, “the probability that democracy survives increases monotonically with per capita income.” Except in Argentina, no democracy ever fell in a country with a per capita income higher than $6,055, while thirty-nine out of sixty-nine democracies did fall in countries that were poorer (p. 165). Przeworski and Limongi further observe that the emergence of democracy is linked to economic development in “old” industrialized Western countries, because development didn’t have much of an impact on the collapse of dictatorships in “new” countries postwar and the stability of democracy increases much more with economic development in the old than in the new countries. In sum, modernization theory is correct only with regard to the old countries. ,Theory Testing II: Indigenous Democratization, C. Boix and S. Stokes (2003) ,In yet another test of Lipset’s theory, Boix and Stokes reexamine the causal relationship between economic development and democracy more rigorously. Directly challenging Przeworski and Limongi on theoretical and empirical grounds, they hypothesize that development is both an endogenous and exogenous cause of democracy. Empirically, they replicate Przeworski and Limonigi’s results to show that the latter’s findings fail on three tests of robustness. First, Boix and Stokes reason out, their observation that few transitions to democracy at high levels of income is in fact consistent with endogenous democratization, because “at a per capita income of $7,000, the effects of development on political regime have already taken place: countries that were going to develop and democratize had already done so before reaching the range of the very rich” (p. 524). Second, Przeworski and Limongi’s sample is subject to “selection problems” because the year 1950 (where their data begins) is late to draw a complete story of democratization in rich countries. Using additional data for the period 1800-1949, Boix and Stokes demonstrate that per capita income has a strong positive and statistically significant effect on transitions to democracy from the mid-nineteenth century until World War II. Finally, Przeworski and Limongi’s analysis suffers from omitted variable bias. Boix and Stokes control for additional factors (i.e. international forces and oil) to find out that economic development still makes democratization more likely. Furthermore, rather than higher income per se income equality is the causal mechanism that links economic development to democracy; as countries develop, incomes are more equally distributed, which makes the wealthy to countenance democracy as the median voter favors an equitable system.
3.6 Conclusion
A social scientific theory is a generalized explanation of causally related patterns of events, behaviors, or processes. A theory is not data, facts, typologies, or mere empirical findings because theories must go well beyond objective facts or conceptual constructs to include propositions, explanations, and observable and testable falsifiable statements. Theories differ from various other forms of non-scientific claims or knowledge because they are established using objective scientific methods (theory-building), and they are amenable to further testing, confirmation, and refutation using the same scientific methods (theory testing). Theory-building and testing are two interrelated scientific endeavors that apply the scientific method, but they vary in their epistemological approach, main goals and tasks, and their end results.
Social reality is much more complex than we can possibly comprehend or fully explain. As such our theories tend to be limited, if not outright wrong, for reasons related to limited data, unobserved relations, or systematic bias, among other shortfalls. Despite these limitations, however, social scientific theories are still our only hope to better understand our social and political world. Theories are invaluable to describe events and processes, explain relationships between two or more events and process, and to make more accurate predictions whether some events or processes are bound to occur in relation to other events or processes. As a result theories should be informative, objective, accurate, and broadly useful. Different traditions in the social sciences may hold different standards of what constitutes a good theory, but it is generally understood that parsimony, generalizability, observable implication, and falsifiability are some basic elements of what constitutes a well-crafted theory.
3.7 Application Questions
Suppose a political science student is interested in voters who are fed up with “human” politicians and demanding to vote for divine, all-powerful alien leaders. What are the valid steps in developing a theory of benign alien dictatorship?
Suppose another student wants to estimate the effect of oil wealth on democratic backslide in Venezuela in the past two decades. We already know that oil wealth is highly detrimental to democracy and boosts authoritarian regime durability in low income countries. Is the student engaged in theory-building or theory testing exercise? How is she supposed to proceed in offering an explanation of recent political experience of Venezuela in conjunction with its oil-dominated economy?
3.8 Key Terms
Concept: the basic unit of thinking in theory building or an abstract idea that offers a point of view for understanding our experiences or observations, an idea of a phenomenon formed by mentally combining its attributes, or a mental image that, when operationalized, helps to organize the analysis of data.
Falsifiability: the possibility of a claim, hypothesis or theory to be proven wrong.
Hypotheses: tentative answers to a research question. In causal analysis, a hypothesis is an "educated guess" or a conjecture about the relationship between one or more empirical phenomena (i.e. independent variable) and another phenomenon (i.e. dependent variable). Since hypotheses are proposed relationships, they may turn out to be incorrect and not supported by the empirical evidence.
Literature review: a systematic examination and interpretation of the existing scholarship for the purpose of informing further research on a topic.
Theory: the conceptual and explanatory understanding that is an essential point of departure in conducting research, and that in turn is revised in light of research. Different (i.e. qualitative and quantitative) analytic traditions have divergent norms about the appropriate structure and content of these understandings.
3.9 Answers to Application Questions
The student is trying to develop a theory that explains why voters are frustrated with politicians and favor an alien dictatorship. The valid steps are to: a. formulate a question. b. define the key concepts “human” politician, corruption, and alien dictatorship. c. formulate a hypothesis, that is, to assume the venality of moral human politicians leads voters to support incorruptible aliens or alien leaders are charismatic compared to ordinary politicians, using use careful observation or literature review. d. measure corruption among politicians and the incorruptibility of aliens. e. test both hypotheses against empirical evidence. f. empirically demonstrate why one (your) hypothesis is true and the alternative hypothesis is false.
The students is involved in theory testing because existing theories of petroleum and political regimes show oil is corrosive to new democracies. She gathers data on annual oil revenues for Venezuela in the past twenty years and figure if increase or decrease in oil revenues are correlated with the decline of democracy in the country. She has to explain why oil have had a damaging effect on Venezuela’ democracy by empirically showing that it corrupted democratic institutions, destabilized the national economy, and/or strengthened the coercive capacity of the regime.