In 1959 Donald Kirkpatrick introduced four levels of evaluation to the learning profession: reaction, learning, behavior, and results. Thirty years later, Gloria Gery courageously informed us all that we might as well just weigh our students before and after learning, rather than use the metrics we were still using to determine the effectiveness of what we do for organizations.
To Gloria’s credit, she offered revolutionary insight regarding how we could break free from evaluating learners’ reactions and learning mastery (Levels 1 and 2). She proposed a way for us to validate the transfer of learning to the workplace and measure actual business impact (Levels 3 and 4).
According to Gloria, this validation of transfer can only happen within a performance support framework, “because the connections between performance support [instances] in the actual work context is so much more direct than the distance between training events and work performance.”
Bold insight! But add another 20 years, and today most organizations are still stuck primarily in Levels 1 and 2.
We recently surveyed more than 2,000 learning professionals, and 68.5 percent reported that they are measuring the success of their learning and training initiatives through surveys, interviews, smile sheets, etc. That’s just Level 1 Kirkpatrick.
How can we move on?
Why are we still stuck in the mire of murky measurement? Because, until now, organizations couldn’t readily put in place the kind of performance support infrastructure that can actively measure the business impact of what we do. So how do we actually get there from here? I suggest starting with two basic steps.
Step one: Integrate a true performance support solution
Alter, if necessary, the way you educate and train your workforce by integrating performance support into the mix. There is much more to this than you can imagine. Creating job aids or dumping reference content into SharePoint won’t do the trick. Gery defines a performance support solution as an “orchestrated set of technology-enabled services that provide on-demand access to integrated information, guidance, advice, assistance, training, and tools to enable high-level job performance with a minimum of support from other people” (Electronic Performance Support Systems).
If you want to know what this entails then read (or review) the five articles Bob Mosher and I published between May and September last year in Learning Solutions Magazine.
Step two
With the aid of a performance support (PS) system, incorporate measurement capability and practices that will link training to business impact and ultimately to financial and strategic benefit. This can be done, it is being done, and it needs to be done! Figure 1 is the flow chart.
Figure 1: The path that leads to measurable benefits from training
For example, an organization implements Salesforce as their CRM (customer relationship management) solution. They also implement a PS system within Salesforce that provides contextual support to the sales team. The PS system also ties Salesforce to the other software applications, non-system sales tasks, and tools needed to effectively follow the company’s overarching sales process.
Sales teams then receive training on Salesforce, the other applications, and how to use other sales tools, such as completing a targeted account plan spreadsheet. During their training they use an integrated PS system that provides guidance from wherever a sales person is in the process—within any of the applications, when using any of the tools, or simply following the specific steps for prospecting a customer (learning).
On the job, the sales team works within the PS system to access the tasks and resources needed to help them work as a team throughout the sales process. Independent of where they are, they have access to what they need within “two clicks or 10 seconds” via their phone, iPad, desktop, and in some instances even print (performance support).
The performance support’s monitoring capability shows those from the sales team who are adhering to the process and working effectively within the applications, and have therefore increased their respective pipelines by X percent (business impact).
The following quarter, individuals, teams, and units meet their financial targets (financial impact), which in turn increases the market value of the organization (business impact).
And there you have the linkage from learning to financial and strategic benefit. Without performance support, there is limited capacity to do this; however, with performance support, we are able to make that shift to move beyond gathering the usual and traditional learning metrics (pass/fail rates, number of student days, percentage of course completions, smile sheets, etc.), and go on to those directly impacting the organization’s bottom line.
A fully-loaded performance support system allows us to monitor adherence to process, verify successful performance, and tie it all to improvements in actual work performance (reduced error rates, reduced process time, etc.). What’s more, it lets us correlate what the workers’ successful performance actually means to the business (faster, better, cheaper). We can turn that into hard dollar numbers with which the business will resonate and trust. Sidebar 1 summarizes typical response from organizations that have implemented this kind of performance support system.
“Performance success and/or failure can be tagged directly to hard dollars, and the best sources for that insight are the stakeholders who live with the results ... or lack of them. Impact those dollars, and you have a partner and advocate for your cause.”
Gary Wise, learning development director at Intellinex, LLC—A Xerox Company
“In a time where people Google everything, just-in-time performance support is an absolute must and is expected by top performers!”
Rose Lawyer, VP of instructional design at Huntington National Bank
Show me the ROI with performance support
With a performance support solution in place, you can track and then correlate increased productivity and reduced formal training time—to the bottom line. For example, with the implementation of a PS solution, two separate organizations were able to achieve the following:
- A six- to eight-percent increase in daily work productivity, resulting in a $2,375 cost savings per employee per year
- Instructor-led training reduced by 70 percent, as well as help desk calls reduced by 12.5 percent, resulting in a savings of more than $2 million annually and an additional $2 million increase in productivity
When you link performance to financial and strategic benefits, you can also calculate ROI. For example, in a study between a virtual classroom (VC) technology and a performance support system, the results from the comparison showed that the ROI for the PS solution was over 460 percent. This was based on only two criteria: 1) impact to productivity through reduction of workforce time spent in “search” mode; and 2) impact to productivity through reduction of formal training time. (See the study here.)
Further performance support research compared those organizations with a performance support system vs. those without, and saw the results shown in Figures 2, 3, and 4.
Figure 2: Performance support improved the percentage of employees reaching their time-on-task competency goal
Figure 3: Performance support increased the percentage of employees who reached their quality competency goal
Figure 4: Performance support increased the percentage of employees who reached their task-comprehension goal
Performance support bridges that gap between training and actual business impact by enabling effective performance in a constantly changing work environment. It also allows us to measure impact on that performance in ways training alone never could, nor will it ever be able to.
As an industry, let’s start measuring what really matters … what really matters in terms of employee performance … what really matters to our organization’s bottom line.
References
Gery, Gloria. Electronic Performance Support Systems. (1995). Gery Performance Press, Tolland MA.