Right now, I bet many of you out there are thinking about the future of learning assessment. No? You should be. As I think about it, I’m drawn to just one conclusion: There isn’t much of a future unless we change how we approach it.
When most of us think about learning assessment, we think about tests—end-of-course tests, to be specific. We like to think we’re more sophisticated, but most of the time—whether dictated by tradition, (that’s how we’ve always done it), mandate (that’s what management wants), familiarity (that’s what we believe we know how to do), or tech (it’s what our LMS can support)—it’s the end-of-course test that dominates our evaluation strategy and practice.
The ever-popular end-of-course test (aka Kirkpatrick “level 2”) is important for sure, but it’s too easy for us to buy in to them as adequate predictors of on-the-job performance. Some are so weak that they may not even measure what was learned in the course, let alone what learners can do afterwards. Worse, we then take these tests and certify compliance, or competency, with a host of governmental, industry, and business standards, without knowing for sure whether the learners can actually do what they were trained to do. They showed up, therefore they learned; and since they learned, they can do. Really?
The problem, in a nutshell
How can we expect learning assessment to mean anything, especially to our clients and customers, if we can’t show that learners actually improve performance as a result of what they learned? We know better.
Well-designed (and that’s asking a lot) end-of-course tests can assist in predicting performance, but they can’t measure actual performance. For that you need to get into the field to look at and evaluate work products, measures of efficiency, error rates, customer and employee satisfaction, business results, supervisory feedback, response times, work agility, and many other data points. Good end-of-course tests, combined with measures like these, can go a long way toward certifying performance and, therefore, validating the training received.
If we want the future of learning assessment to remain weak, we can continue to put evaluation and measurement on the back burner of resources and time; or we can refocus ourselves to bring learning assessment to the forefront of what we do.
Three important steps you can take
If you want learning assessment to have a stronger future, here are three steps to enhance it in your organization:
- Take some money away from your instructional design budget and build your organization’s evaluation expertise. You may produce a few less courses, but what you do build might have a shot at actually demonstrating real performance improvement. This gives learning assessment equal footing with instructional design and technology. It’s not as sexy, but can you really have a great course if you can’t show that it works?
- Move compliance training away from measures of attendance and completion to better measures of actual performance (a hard slog, I know). As I’ve discussed before, we’ve been lulled into a false sense of security by using poor testing, or just an attendance sheet, to demonstrate competence. Despite government and industry compliance requirements, you still may be on shaky ground (legal and otherwise) if you can’t show that your learners can actually do what you’ve certified they can do.
- Put your clients and customers in charge of evaluation by letting them tell you what constitutes success and then, together, you measure it. This means having the evaluation discussion at the start of the project, not near the end. Letting your clients and customers own the evaluation outcomes not only helps to build a shared vision of what the assessments should be—at the end of the course and in the field—but it also engages them to be more involved in the entire project from the get-go.
Taking steps to elevate the importance of sound learning assessment, and then putting it into practice, won’t be easy. We’re not talking lip service here. It’s a real investment of time, money, and people. And let’s face it, for most of us, building cool courseware is much more interesting, visible, and rewarding than measuring the results. So there will be resistance. It can be a hard sell. Nevertheless, this is something you must do.
Or don’t, and see what happens.