With the financial advantages of big data gaining global attention, C-level managers and senior business leaders are beginning to demand better reporting and improved accountability across all organizational divisions. While the goal of measuring learning seems unreachable for most in the training division, the reality is that these technological advancements mean that disruption in the training industry is inevitable. These innovations will enable unprecedented reporting of training effectiveness, business impact, and return on training investment.

Many training professionals find it challenging to know if training was effective on the job or if part of the organizational system is not supporting the training delivered. While many training departments collect data under Kirkpatrick’s levels one or two, few are able to track data beyond that. Training departments need a methodology for finding out training needs without on the job observation or having trainees return for a test.

In the world of Big Data, organizations are increasingly using data and technology to better understand the value of their investments and efforts. As training professionals, it is imperative that we also use data to communicate the impact of our work. Many learning professionals avoid data and measurement because it seems like a daunting task, but it doesn’t have to be.

The availability of (and demand for) data around learning has grown dramatically. Tools like the Experience API (xAPI) make it increasingly easy to acquire data about learner’s activities, but this will provide little benefit to instructional designers or learners if we do not design to acquire meaningful data, know how to interpret that data, or know how to improve our learning design based on that data.

eLearning has a broken feedback loop, and it’s holding us back as a field. Because we usually can’t see our products being used, we lack the most basic information necessary to improve what we do. Traditional evaluation at best is costly and difficult to measure, and at worst either ignored all together or implemented in such a superficial way that it’s meaningless. Even good evaluation measures are not granular enough to inform future design decisions.

Everyone’s talking about Big Data, but learning systems already generate piles of data and it’s a mess. How can we move on to a more in-depth analysis of learner behavior if we can’t even straighten out the basics? Our technicians roll their eyes and complain about poor data quality in the upstream feeds. The report designers complain that the stakeholders keep changing their requirements, and our teams are burning precious hours managing unwieldy Excel spreadsheets. We don’t need a new technology. We need a new approach.

When learners are geographically dispersed, and the training is completed, it is difficult to know if the skills were transferred back on the job. In addition, it is difficult to observe the learner performing a new task, and to provide feedback for improvement and/or reinforcement of what was done well. We need to find ways to use technology to enable us to effectively evaluate learners’ ability to apply what they learn in training.

Instructional designers are seeking more than just knowledge retention from learners; we are seeking positive, quantifiable, and repeatable changes in behavior—the gold standard of eLearning results in the business world. However, these behavior changes are challenging to facilitate, and even more challenging to track.