Question of Validity
Are We Measuring What
We Think We're Measuring?
Reliability and validity are two things that social scientists pay attention to when they measure ideas like belief systems. Belief systems are invisible. They are ideas that people store in their minds. Although belief systems are invisible, people can agree or disagree with opinions that are related to those belief systems.
In the illustration, a social scientist interviews an ordinary person on the phone. The social scientists can't crawl into the mind of the ordinary person answering the phone. But she can ask whether that person agrees or disagrees with a series of opinions about the death penalty.
Reliability has to do with consistency. Does agreement with one opinion match up with agreements with other opinions in the same cluster? Reliability is something we can measure, with statistical tools like Cronbach's alpha.
Validity is how well the questions we are asking (visible) match the ideas or beliefs (invisible) tucked away in the mind of the person on the phone. In an exploratory study such as this, we depend heavily on what's called face validity. That is, on the face of it, do the opinions in the questionnaire match the ideas or beliefs that we seek to measure?