Training evaluation in the modern organization

  • 6 minutes read

Implementing training evaluation with the purpose of proving its value is not an easy task.

Let’s say, hypothetically, that your manager is asking for some budget cuts. He suggests to look at the training costs, as that is a pretty large sum and maybe it can be tackled. Your well implemented needs analysis revealed you need that training plan as it is because it achieves the set objectives. But at the next review, you will have to prove a great return on investment. In the meanwhile, you have to make sure the training programs are relevant, engaging and effective for all involved. You know participants’ experience is as important as the organizational objectives.

It’s a bit easier to measure the ROI of technical skills training. For example, you can look and the productivity numbers before and after the training is implemented. But you need a good system to value training that has intangible results. Let’s break this down systematically.

The first thing that usually comes to mind regarding training evaluation is the Kirkpatrick model. The first version was published over 50 years ago. Even though it was constantly updated, I cannot help but wonder if implementing it is enough. The model consists of 4 levels and each one represents a more precise measure of the effectiveness of a training program.

The Kirkpatrick training evaluation 4 levels model

training evaluation

The 4 levels are: reaction, learning, behaviour and results. Let’s see about each one in detail, explore how to apply them and see what are their limitations.

1. Reaction – Did the participant like the training?

This type of evaluation is quick and pretty easy to obtain. Many companies use: feedback forms, verbal reactions, satisfaction surveys or watching participants’ body language during the training.

The next step is to analyse the feedback and consider the changes that can be made. Then, share the information with the trainer, adding your own conclusions of the evaluation.

2. Learning – Did the trainee understand the information and gained some new knowledge?

Make sure you already have the learning objectives set for the training. They should be understood by all parties involved. Then, before the session, test the participants to determine their skills, attitudes and knowledge, as well as their confidence and commitment, depending on the case. Then measure again after the training. You can also use methods as interview or observation.

3. Behaviour – Did the training make the participant better at his job?

Change is assessed with observation and interview over time. You can also integrate the use of new skills into their tasks so people have the chance to demonstrate what they know. This level requires the cooperation and skills of direct managers, to assess and coach their team members.

4. Results – Did the company increase profits, sales, customer satisfaction, etc.? What is the ROI?

The measures for ROI are usually already in place through the management systems and reporting. The challenge here is to relate them to the trainee. For example, some outcomes to consider are: fewer client or staff complaints, increased production, higher employee or client retention, increased sales, higher morale, reduced absentee rates, non-compliance, achievement of standards and accreditations, quality ratings, increase in internal management promotions, etc.

Also, go back to the needs analysis that entailed the training. What were the business performance factors that the training had to improve? Use them to measure and relate to organizational return achieved.

Beyond Kirkpatrick model

While I consider these four levels useful, I think they have some limitations. Also, numerous works of researchers show them as incomplete. They believe it encouraged many to focus too narrowly on the evaluation of training alone.

Expanding the application of levels of evaluation beyond training is important. You should use a more integrated approach to evaluation that also incorporates other learning and performance improvement interventions. For example, initiatives of strategic planning, integrated work teams, career planning and mentoring can be included. Learning does not end when participants step out of the training room, so the other learning opportunities the organization can offer should be taken into account. Evaluating them will improve the methods and maximize their results.

Also, as a downside to the model, Roger Kaufman stated that a 5th level should be introduced, which is concerned with societal impact. He believed that by adding this level, organizations will make sure that the changes they made were meaningful and will be seen as such also by employees.

A solution to the Kirkpatrick model limitations

What we usually do in instructional design is using this model backward. We are first laying down the results we want. You can do that by expressing the objectives desired for: individual or small groups (microlevel), the organization (macrolevel), and external clients and the society (megalevel). Then select the best learning solutions that will accomplish those results. It will be a combination of diverse learning methods. It can include training. Or, game based activities. Sometimes, the best solutions might not be training at all. For example it can be a reward, a change in internal culture, an integrated team work, a career plan session or a coaching and mentoring program. Today, non-formal methods of organizational learning are often more popular and effective.

Other suggestions for approaching training evaluation

  • Firstly, you need to understand and explain to all concerned that there is no “cookbook approach” in training evaluation.
  • It should be discussed with the participant exactly what is going to be measured before, throughout and after the training program, based on their current roles. That way they know what to expect and will fully grasp what is being assessed.
  • Remember training has many more purposes beyond developing skills, such as: focusing energy on issues, finding solutions, supporting other interventions, creating a community based on shared experience or promoting change. Evaluate acordingly.
  • You should not neglect the other half of the behaviour change. A successful training involves two phases: acquisition and maintenance of behaviour. The acquisition is most of the times the only one evaluated, as it usually happens during formal training. To maintain the desired behaviour of employees your organization should:
    • allow the behaviour to occur with some frequency
    • not punish it
    • reinforce it
    • not reinforce behaviours which conflict with the desired behaviour
  • Cultivate leadership in your employees and do training knowledge transfer. For the occasion, make participants into trainers for their respective departments and teams.
  • Consider people’s learning styles, they are essentially a perspective of people’s preferred working, thinking and communicating styles.
  • And last but not least, choose a training company that understands why and how to do training evaluation properly. You can be confident that they will improve their trainings to assure the best participant experience and to achieve your specified objectives.
References:
  • Evaluating Training Programs – Donald Kirkpatrick & James Kirkpatrick
  • Levels of evaluation: Beyond Kirkpatrick. Human Resource Development Quarterly – Roger Kaufman & John Keller
  • Thinking outside the Evaluation Box, article in American Society for Training & Development no 53 – Donna Abernathy
  • The Neglected Half of Behaviour Change – Karen S. Brethower
  • Photo source: www.kirkpatrickpartners.com
Insights