Effectiveness Studies

Training – What’s the Point If It’s Not Effective?

Many companies do not know if their training is effective. Relying on quizzes at the end of the course may measure short-term memory, but it does not measure understanding, long-term retention, or successful implementation.  It also does not measure non-training conditions that influence the effective of training.   The absence of more in-depth effectiveness studies is a reason many companies may be blindly throw away money on training.

Training is Only Beneficial If (using slip, trip and fall training as an example):

  • The goals must be realistic.
    • If the learning objectives of the training are met, will the changed behavior result in the goals of the training being achieved?
  • The knowledge or skill can change behavior that needs to be changed.
    • Training addresses slip-resistant footwear, adequate lighting, unimpeded vision, cordoning off spills before they can be mopped up, etc.
  • The behavioral change is designed to solve a specific problem.
    • Training addresses the major causes of slips, trips and falls and how to avoid them.
  • The training is understood by the trainee.
    • Training is in a language and at an educational level that can be understood by the trainee.
  • The training is more than checking off a compliance box.
    • Training builds motivation to change behavior, not just to go through the motions.
  • The training is communicated successfully through good instructional design.
    • Training uses approaches, such as interactive activities, that are best for adult learning.
  • The knowledge or skill is retained. 
    • Anyone can retain knowledge for 60-90 days, but is it retained after 9 months to a year?
  • The knowledge or skill is implemented.
    • Does the trainee use the knowledge or skills as taught consistently and over the long-term?
  • The employee’s use of the knowledge or skill is supported by the employee’s supervisor.
    • Does the supervisor support and even mentor the trainee in putting the training into practice, or indicate that it probably won’t work or takes too much time?
  • The employee’s use of the knowledge or skill is supported by the upper management.
    • Does upper management consistently emphasize the importance of employee health and safety and back it up with action?
  • The implementation by the trainee has the desired effect.
    • Does consistent implementation of training, the trained behavior, reduce the frequency and severity of slips, trips and falls?
  • The benefits of the training outweighs the cost.
    • Training dollars are not well-spent if behavior does not change, or the intended reductions in injuries and illness are not realized.

If any one of these elements is missing, there is a good chance that the training will fail to solve the problem.

What is a training effectiveness study?

Whether you are training one person or the whole workforce, you want to know:

  1. Did the trainee learn the material?  Were the learning objectives met?
  2. Has the trainee retained the knowledge (information, skills)?
  3. Has the trainee used the new learning on the job?
  4.  Has the new learning proved useful in accomplishing your goals?

Number 1 should be answered immediately after the course, with an exam or other means of determining if the trainee understood the key course content/concepts.  Along with a course evaluation, this provides valuable information.  Don’t stop there! That is a mistake because questions 2-4 are even more important.

In many cases, learning is forgotten as soon as the final exam is passed.  An effectiveness study will provide data that can help you know whether:

  • you trained the right people
  • the training was “sticky” (i.e., trainees were able to recall the learning)
  • the training was beneficial

Questions 2-4 should be assessed after some time has passed. 

Often when there is an assessment, it is done 60-90 days after training.  While there is benefit to a quick turnaround, you will often miss the more important elements. 

  • Most people can retain knowledge for 60-90 days, but do they still remember it after 9 months to a year? 
  • Were all people trained at the same time, or did it happen over the course of weeks or months?
  • Did they have the opportunity to use their new knowledge or skills in the amount of time available? 
  • Has enough time elapsed to determine whether your training goals have been met?

The timing of the effectiveness study should be determined by these types of questions.   If you don’t find answers to these questions, you won’t know whether you have trained the wrong group of workers, or if the depth of learning was so shallow that it can no longer be recalled. You won’t know if the learning may be recalled but is not being used appropriately, or if the learning proves not to be as useful as intended.  Nine months to one year after training may be a reasonable time.