METHODS: Limitations of the Study
Some Issues to Consider While Examining the Findings
The 1,600 people who participated in the Death Penalty Study have provided valuable insights into how U.S. residents (91% were currently registered voters) feel and act toward the death penalty. However, the study has four limitations.
Non-probability sample. With probability samples, researchers can estimate, for example, the percentage of adults who agree with the opinion that murderers should be executed. With non-probability samples, such statistical claims cannot be made. The goal in the Death Penalty Study was to conduct an experiment and collect data from a sample that matched key population characteristics. While 'statistical significance' is reported as a matter of convention, it is not as useful as Eta-Squared as a means of interpreting findings.
Exploratory study. Exploratory studies explore new territory where prior research is thin or missing altogether. In this study, researchers used an innovative technique (mock jury) to examine intended actions regarding executions.
New measures. In doing this exploratory study, researchers developed their own measures of such characteristics as vicarious and actual sadism. While these indices of key characteristics are reliable and face valid, additional research is needed to confirm their utility by comparing results of the new measures against established scales (established general measures of sadism, for example).
Cross-sectional survey. Data were collected at one point in time. Except for the experiment testing different opinions about the death penalty, these findings are suggestive of causal relationships, but causality cannot be demonstrated. For example, researchers suggest that vicarious and actual sadism influences (partially causes) people to play the role of executioners, rather than abstract supporters or abolitionists. This is a reasonable explanation, but other explanations may also 'fit' the same data.