Confusing Delivery with Accomplishment

Many HR managers and consultants can remember the days when the true value of training was measured by the number of courses offered or the number of persons trained, or more likely both. In essence, the more training the better.

Thankfully the development of evidence-based management practices has cured us of that ailment. More likely we now look for measures that actually connect improvements in individual and organizational performance to a training intervention. One of the earliest proponents of this approach was Donald L. Kirkpatrick. His volume titled, Evaluating Training Programs (1998), identified four levels of measurement:

1. Reaction
2. Learning
3. Behavior
4. Results

The first level is the simplest and for the most part the least effective in measuring improvements. It is the “feel good” tool, the evaluation handed out at the end of a class asking students to rate the instructor, environment, materials and other similar issues based on the students’ personal reactions. As a consequence, an instructor might receive a great score if only because she was witty, or he was charming, or at some point in the afternoon he gave out bags of candy. What this level of evaluation doesn’t measure is anything related to learning, behavior change, or the effect of the training effort on how the organization performs (results).

A common tool in determining whether students actually learned the course material (level 2 measurement) is the post-test, an exam that quizzes students on the material taught during the class. Of course even that type of measure does not mean that someone scoring 100 on the test actually used the concepts and techniques in the workplace. For example, someone might quite certainly understand how to conduct a performance review based on standards identified in the class, and therefore correctly answer questions related to performance review on the test. Does that same person actually behave that way in the workplace? That would be a level 3 measurement that would require the use of measurement tools at some point after the class has been completed.

The most difficult of all measurements is level 4: results. Let’s suppose that in fact all managers and supervisors score high on a level 3 assessment of their ability to conduct a performance review. Do the behaviors for which the training was conducted result in greater staff productivity? The problem in making this connection is that there are many other variables that might also simultaneously influence staff performance. For example, during the same period of time, the company might have introduced new technology affecting performance outcomes. Or, the employees under scrutiny might have developed better work processes quite independent of the performance review process. Isolating the effect of the training can be both complicated and expensive.