Evaluate Training: Measuring Effectiveness

Overview

The goal of training is to help a learner improve their competence, capacity, and performance. Training helps learners gain new knowledge and skill. The most effective training also helps learners apply this information to their workplace, a process known as transfer of learning or simply learning transfer. Training effectiveness refers to how well your training supports learning and learning transfer. There are many ways to evaluate training effectiveness.

What to evaluate

You should assess both learning and learning transfer, whenever possible.

Evaluate learning

Design your evaluation to assess how successfully the learner met the training's learning objectives. Look at the combined results for all learners to help you understand their learning and identify data trends that indicate challenging topics for your learners—which might show a need to improve course content or instruction.

Evaluate learning transfer

Design your evaluation to assess how successfully the learner can apply what they learned when they return to the workplace. Look at the combined results for all learners to help you understand if learning transfer is occurring and which topics are challenging for learners to apply—which might show a need for follow-up support for the learners.

When to evaluate training effectiveness

Your data collection methods and timing will depend on your evaluation questions. Determine what information you need while considering what is feasible. Your time, resources, and training scope should drive your evaluation approach.

Before and after training

The best way to evaluate any change in learning is through assessment before and after the training. Conduct a pretest before and a posttest after your training and then compare the results.

Considerations:

• The test can include a demonstration to assess skill in addition to knowledge, if needed.
• Using only a posttest, without a pretest, can provide an assessment of skill or knowledge proficiency. This allows you to know if learners achieved a certain level of knowledge or skill by the end of the training, but you will not know if there was a change in learning. Learners might have already had the knowledge or skill at the start of the training.

Retrospective pre/post assessment

If you are unable to assess learning through a pretest and posttest, consider using a retrospective pre/post assessment that asks learners to self-assess their knowledge before and after the course as part of a postcourse evaluation.

During training

Build knowledge or skill assessment into the training, like knowledge checks, quizzes or observations. This can provide evaluation data and reinforce learning at the same time.

Considerations:

• In an e-learning, you might have knowledge checks throughout the course to help reinforce learning. Those same knowledge checks give you data on how learners progress during the course.
• For an in-person training, you might ask your learners questions to assess their comprehension or use an activity to gauge how they apply what they are learning. This provides real-time information for the instructor to reinforce content or adapt as needed.

Immediately after training

In many situations, you might only be able to gather information from your learners immediately after the training ends. You can design your postcourse evaluation to assess learning and predict learning transfer immediately after the course ends, while learners are available to respond.

Considerations:

• Postcourse evaluations traditionally focus on learner satisfaction, but the research shows learner satisfaction does not determine a training's effectiveness.
• Postcourse evaluations cannot objectively assess learning or learning transfer, but focusing on the areas with the strongest, most consistent relationships with learning and learning transfer can provide you with meaningful data about training effectiveness.
  • Based on factors that predict learning transfer, Section 3 of the Recommended Training Effectiveness Questions for Postcourse Evaluations User Guide can help you determine your training's effectiveness.

Delayed evaluation or follow-up

Delayed evaluation, also called follow-up evaluation, is the best way to assess learning transfer. This helps training developers understand how much information learners retained, and if they have applied what they learned on the job. You can follow-up with learners to assess learning transfer after they have had time to go back to their workplace and apply what they have learned.

Considerations:

• For some trainings, it might also be appropriate to follow-up with learners' supervisors.
• The timing of your delayed evaluation should be based on your program resources, the specific topic of the training, and learners' capacity to apply what they learned in the workplace.

Recommended training effectiveness questions

These questions focus on themes strongly associated with learning and learning transfer that can be measured in-person or online with adult learners. For more information, visit the Recommended Training Effectiveness Questions for Postcourse Evaluations User Guide. The user guide includes the recommended questions, rationale for each question, additional context or caveats that you should consider when using the questions, and references to the research that supports the questions.

You can adapt the questions to fit your training evaluation plan and data collection tools. The Example Postcourse and Follow-Up Evaluations use the Training Effectiveness questions along with additional questions that inform quality improvement and decision making.

For postcourse evaluations

1. Rate your knowledge of the course topic before the course.

• Not at all knowledgeable
• Slightly knowledgeable
• Moderately knowledgeable
• Very knowledgeable
• Extremely knowledgeable

2. Rate your knowledge of (or skill in) the course topic now after the course.

• Not at all knowledgeable
• Slightly knowledgeable
• Moderately knowledgeable
• Very knowledgeable
• Extremely knowledgeable

3. How relevant is this course to your current work?

• Not at all relevant
• Slightly relevant
• Moderately relevant
• Very relevant
• Extremely relevant

4. What is your opinion of the balance of lecture and interactivity in this course?

• Too much lecture and not enough interactive learning
• Right amount of both lecture and interactive learning
• Too much interactive learning and not enough lecture

5. Will you use what you learned in this course in your work?

• Definitely not
• Probably not
• Possibly
• Probably yes
• Definitely yes
• Not applicable — I did not learn anything new from this course

6. What factors will keep you from using the content of this course in your work? (Select all that apply)

• I need additional training in the subject matter
• I will not have the resources I need
• I will not be provided opportunities to use what I learned
• I will not have the time to use what I learned
• My supervisor will not support me in using what I learned
• My colleagues will not support me in using what I learned
• The course content is not relevant to my current work
• Other (please specify):

7. What, if anything, do you plan to use from this course? (open-ended)

8. How could this course be improved to make it a more effective learning experience? (open-ended)

9. What part of this course was most helpful to your learning? (open-ended)

For delayed evaluation

1. To what extent have you used what you learned in this course in your work?

• Not applicable—I did not learn anything new from this course
• Not at all [if selected, go to question on barriers]
• Some [if selected, go to question on barriers]
• A lot [if selected go to question on facilitators]

2. What did you use from this course? (open-ended)

3. What factors helped you use the content of this course in your work? (Select all that apply)

• I had reminders of key learning concepts or skills
• I had the resources I needed
• I had opportunities to use what I learned
• I had time to apply what I learned
• My supervisor supported me in using what I learned
• My colleagues supported me in using what I learned
• Other (please specify):

4. What factors kept you from using the content of this course in your work? (Select all that apply)

• I need additional training in the subject matter
• I did not remember the course content well enough to use it
• I did not have the resources I needed
• I did not have opportunities to use what I learned
• I did not have the time to use what I learned
• My supervisor did not support me in using what I learned
• My colleagues did not support me in using what I learned
• The course content was not relevant to my work
• Other (please specify):

Resources

• Recommended Training Effectiveness Questions for Postcourse Evaluations User Guide
• Training Effectiveness Predictors
• Learning-Transfer Evaluation Model