RESEARCH DESIGN[1]
Charles McCall, California State University, Bakersfield. 

·        Subtopics

 

·        The Research Design

·        The Research Report

·        Exercises

---

 

The Research Design

Good research is not accidental. It requires careful planning as well as careful execution. Before launching a research project, social scientists prepare a research design, in which they set forth their plans for the research they intend to undertake. Often, such designs are formal in nature, as when a scholar prepares a research grant request to a funding agency, or when a student submits a proposal for a dissertation, thesis, or other research project.

Even when a research design is not a formal requirement, it is a step in the research process that should not be bypassed. With a good design the student will be able to foresee and avoid numerous obstacles and pitfalls, save time in the long run, and produce a superior finished product.

The length and complexity of research designs obviously vary considerably, but any sound design, be it for a massive, government or foundation supported study, or for an undergraduate term paper, will do the following things:

1.      Identify the problem clearly and justify its selection.

2.      Review previously published literature dealing with the problem area.

3.      Clearly and explicitly specify hypotheses central to the problem selected.

4.      Clearly describe the data which will be necessary for an adequate test of the hypotheses and explain how such data will be obtained.

5.      Describe the methods of analysis which will be applied to the data in determining whether or not the hypotheses are false.

1.         As you can see, step 1 requires you to state the problem and justify its choice; that is, you should explain your reasons for selecting it. It is correct to say that the choice of a topic depends principally on an investigator's interest and values. Nonetheless, there are some general criteria which we can use to evaluate the relative worth of topics. For example, how important is the topic for development of knowledge? How would the answer to questions which you pose for yourself affect other parts of the knowledge structure within political science? Would it extend "theories" to areas not yet explored? Would it replicate earlier findings in a different setting? Would it resolve present inconsistencies in our interpretation of evidence or create inconsistencies by calling into doubt currently accepted interpretations? A research project which adds to our knowledge in such ways is bound to be considered important.

While the promise of acquisition of knowledge is an important criterion in evaluating research proposals, it is not the only standard we apply. If additional research on the problem will improve the social, political, or economic well-being of people, that fact helps justify the choice of a research topic. Would the answers to the questions that you pose for yourself affect the achievement of accepted social values? Might such answers cause changes in accepted social values? Positive answers to such questions suggest that the research proposal is one which is important.

A third criterion which we use in evaluating a choice of research problems is the criterion of economy. How easily can the topic be studied? We are not likely to establish a moon colony simply to discover whether or not the relationship between sense of political efficacy and political participation which we've discovered repeatedly in Western nations persists among those living on our permanent satellite. No one is likely to support a proposal to expend great amounts of time, skill, and money on a project likely to add little to our knowledge or to human welfare.

2.         Before beginning any research project, you should familiarize yourself with the work that has already been done in your area, so that your project can build on and extend (or, if necessary, correct) the findings of those who have preceded you. A summary of this review of the literature should be included in your research design. Such a review is helpful not only in systematically gathering and organizing existing substantive knowledge about the topic but also in sensitizing scholars to the methods and procedures which have been used by others investigating the same general area.

An increasing number of scholarly journals are now available online through the University of Toronto Library at www.library.utoronto.ca

3.         The third major step in a research design is to clearly state the hypotheses to be tested. As used here, an hypothesis is a declarative statement positing a relationship between variables. In its simplest form an hypothesis contains these elements: an independent variable; a dependent variable; the nature of the relationship between the two.

The development of hypotheses belongs to that area of inquiry which has been called the logic of discovery. The logic of discovery deals with the identification of good reasons for suggesting an hypothesis--reasons for putting it forward or testing it, not reasons for accepting it. One good reason for subjecting an hypothesis to test is that it is commonly believed. Gerald Pomper in his book, Elections in America, noted that the literature of political science was replete with claims that party platforms were unrelated to legislative enactments. Pomper also noticed that there was very little systematic evidence in the literature supporting the hypothesis; thus he decided to test it. His research falsified the hypothesis, which had been a standard one in political science, and therefore has been of great importance to the discipline.

A second way in which hypotheses are developed is through the use of analogy. Hypotheses are often based on metaphors drawn from our environment. Darwin explained that it was his perception of the analogy between Malthus' theory of human population and processes in the animal kingdom which led him to develop the theory of evolution. The analogy between chess and international politics is an old one in political science, and in the last thirty years we have discovered that a careful study of games can be quite suggestive for the development of hypotheses regarding the ways in which nations treat each other.

Another successful use of analogy in the social sciences is evident in the application of formulae developed to describe the spread of communicable diseases as formulae to describe the spread of rumors in society. Not all analogies are good ones. The test of a good analogy is that it suggests numerous and significant hypotheses and that those hypotheses are not refuted by systematic testing.

A third way of developing hypotheses is retroduction. That method which Arthur Goldburg has defined "as an imaginative leap, a flight of fancy, taken to account for observed phenomena" is insufficiently understood. Many of you will know the tale regarding the discovery of Archimedes' Law that a solid body will, when immersed in water, suffer a loss in weight equal to the weight of the displaced water. As the story goes, Archimedes had long been perplexed by a problem set for him by the tyrant of Syracuse. He had commanded that a votive crown of pure gold be prepared for placement in one of the temples, but gossip concerning the goldsmith suggested that some silver had been mixed with the gold in the manufacture of the crown. Archimedes was charged to discover without injuring the crown whether or not that was the case.

The story continues that Archimedes, while taking a bath noticed that his limbs were unusually light when in the water and that in proportion as his body was immersed in the tub, the water spilled over its edges. That perception led to instantaneous realization that he could solve the problem. But if the myth has any basis, the fact that Archimedes after long study was hit so quickly by the solution to the problem that he was moved to leap out of the tub and run home naked shouting "eureka!" does not mean that the solution was not a result of his long study.

Regardless of the way in which the hypotheses have been discovered or developed, it is crucial that they be stated clearly. If they are clearly stated, the distinction between independent and dependent variables should be obvious. Naturally, careful statement of the hypotheses also entails clear, conceptual definition of the major variables involved.

4.         After you have stated your hypotheses clearly, you are ready to proceed to the fourth major stage in the research design, the description of the data required to check the hypotheses and the explanation of how the data is to be obtained. Your hypotheses determine in large part the kind of data required and suggest methods for collecting it.

Let us assume that you plan to test the assertion that conservatism is related to social class such that the higher peoples' social class, the more likely they are to be conservative. Your first task is to operationalize the variables. How will you identify "conservatives" and "liberals" in terms of operations which others could repeat? How will you identify in similar terms a person's social class?

Suppose you decide to operationalize conservatism on the basis of a scale based on a series of questions about political issues. You then need to specify the questions to be included in the scale and describe the method by which responses to those questions will be scored and summarized. Now what about social class? Suppose you decide to operationalize social class on the basis of income. Suppose further that you decide to group your data into the following categories: less than $15,000, $15,000 to $24,999, $25,000 to $49,999, and over $50,000.

Your next step is to explain how the necessary data is to be obtained. In general, this might involve either primary analysis of data which you will collect especially for this project, or secondary analysis of data that has already been collected for some other purpose. In any case, the data that is to be analyzed must be described fully and accurately. The goal is to give readers sufficient information to allow them to successfully evaluate your proposed project and the analysis and conclusions which you will present later.

The main point to remember is that in principle the data you plan to analyze must be capable of falsifying the hypothesis if the hypothesis is in fact untrue. What will count against it? That is probably the most important question in building theory. Indeed, the principle which should govern the selection of evidence to test the hypotheses is the principle that the chances of discovering decisive negative evidence should be maximized. The best design is one which would most clearly and quickly expose the error in a working hypothesis. That means that we go out of our way to look for negative evidence.

5.         You are now ready to turn to the fifth major element in a good research design, the identification of the methods of analysis appropriate for treating your data. Be as specific as possible in describing the approach you will use, but do not propose to employ a technique that you do not really understand in the hope that this will somehow give your design an aura of erudition.

The Research Report

A research report contains all the elements of a research design, and in addition presents, interprets, and evaluates the findings of the research. If you have developed a sufficiently careful research design (and have been rather lucky), you may be able to incorporate your design in its entirety in your research report. In practice, however, few research projects go precisely as planned, and the original design will likely have undergone some modification before the final report is presented.

In presenting your findings, the necessary evidence should be set forth and the conclusions based upon it should be clearly stated. Many beginning scholars fall victim to the temptation of presenting too much evidence and analysis in their research reports. Remember the problem you set out to attack. Interesting findings which are tangential to the problem should be omitted from oral presentations and set out in footnotes or in an appendix of written reports. If you include them in the main body of the presentation, you may only succeed in directing the audience's attention away from your principal focus.

It is in this central portion of the presentation that you will show your essential tables and graphs. Throughout the report, your main objective must be to communicate clearly, and this applies just as much to your tables as it does to the other parts of the report. Tables and graphs should be structured and titled so that the audience can quickly understand the figures presented in them. Proofread your tables and graphs with extra care for it is very easy to make an error in copying figures.

The concluding section of the presentation, like the opening one, performs several functions. You should begin by summarizing the findings clearly and concisely, showing how they confirm or disconfirm the hypotheses with which you started. This section also includes your own critique of your research indicating the shortcomings of your work and suggesting how they might be overcome with sufficient resources. Finally, the concluding section of the report should identify further questions raised by the findings of your study which demand future research.

Exercises

Using the American National Election 2000 Subset, pick a political attitude (for example, capital punishment) or behavior (for example, whether or not the respondent voted in the last presidential election).  Select some other variables (there's no magic number, but maybe a half dozen or so) that you think might explain variation in this attitude or behavior.

Write a research report in which you (1) explain the problem you have chosen to investigate, (2) set forth your hypotheses and the reasoning behind them (include, where it's not obvious, an explanation of the concepts your variables are intended to measure), (3) review and summarize the relevant scholarly literature (if your library subscribes to J-STOR, that would be an excellent place to start), and (4) describe, explain, and interpret your analysis and results.  Include contingency tables, measures of association and significance, and control variables where appropriate.