Reliability and Item Analysis Introductory Overview - Basic Ideas

Suppose you want to construct a questionnaire to measure people's prejudices against foreign-made cars. You could start out by generating a number of items such as: "Foreign cars lack personality," "Foreign cars all look the same," etc. You could then submit those questionnaire items to a group of subjects (for example, people who have never owned a foreign-made car). You could ask subjects to indicate their agreement with these statements on a 9-point scale, anchored at 1=disagree and 9=agree.

True scores and error
Let us now consider more closely what is meant by precise measurement in this case. We hypothesize that there is such a thing (theoretical construct) as "prejudice against foreign cars," and that each item "taps" into this concept to some extent. Therefore, we can say that a subject's response to a particular item reflects two aspects: first, the response reflects the prejudice against foreign cars, and second, it will reflect some esoteric aspect of the respective question. For example, consider the item "Foreign cars all look the same." A subject's agreement or disagreement with that statement will partially depend on his or her general prejudices, and partially on some other aspects of the question or person. For example, the subject may have a friend who just bought a very different looking foreign car.
Testing hypotheses about relationships between items and tests
To test specific hypotheses about the relationship between sets of items or different tests (e.g., whether two sets of items measure the same construct, analyze multi-trait, multi-method matrices, etc.) use the Structural Equation Modeling (SEPATH) module.