The "Criterion Problem"

Let's review from PSYCH 281 (Lesson 4, page 3):

Criteria are evaluative standards that can be used as yardsticks for measuring [work]. Criteria are important for many decisions in organizations. We rely on these standards and measures to appraise employees' performance, validate selection devices and training programs, and make decisions about pay and promotion.  Poor criteria beget poor organizational decisions.

Unfortunately, developing and measuring criteria well is a long-standing challenge for I/O psychologists. Researchers have been writing about this difficulty, called the criterion problem, for decades (e.g., Flanagan, 1956; Austin & Villanova, 1992).

Why is it difficult to determine standards of performance? First, performance behavior is very complex and dynamic; it changes over time. Second, performance behavior is multidimensional. Third, no single criterion measures performance perfectly. More than one criterion can be used, but decisions about how to combine and weigh the individual criteria within the composite criterion create additional complexity. Lastly, criteria may differ across organizations. For example, a service industry may focus on performance related to communicating with customers, while a manufacturing job may measure production numbers.

Note: A composite criterion is a weighted combination of multiple criteria that results in a singular index of performance (Nagle, 1953). For example, weekly activities are weighted most heavily in this class, accounting for almost half of your total grade, while the quizzes account for less than 30%.