Big data is an increasingly significant presence in eLearning circles as more managers turn to learning analytics to improve results. What can big data teach eLearning professionals and managers? How can analytics data be used to improve results? And what does it mean to improve results, anyhow?
Learning Solutions Magazine will explore the arena of learning analytics from multiple angles as it seeks answers to these and other questions. This article provides an overview of what is meant by learning analytics, how eLearning developers can use them, and how the data are often misunderstood.
Learning analytics and the LMS
A learning management system, or LMS, is the source of much data about eLearning and learners. The LMS can tell managers who is completing eLearning, how long each learner is spending, what their test scores are, and how they rated the course. In other words, an LMS is a good source of quantitative data about any aspect of learning that occurs within the LMS.
Managers can analyze this data to identify which employees are struggling and might need additional training; they can also identify learners who have successfully completed learning materials and might be ready for new challenges. Thus the data can help managers make predictions and tailor training to learners’ needs.
Studying data gleaned from learner evaluations or “smile sheets” can help managers and instructional designers improve eLearning by flagging what learners like and dislike. Similarly, data about engagement—how long learners spend with eLearning content—can drive improvements in the quality of the materials.
But learning analytics can provide much more.
Beyond the numbers
In her TED talk on big data, Susan Etlinger says, “We have to ask questions, and hard questions, to move past counting things to understanding them.” What does that mean for eLearning professionals?
Understanding numbers and data requires context. It requires information about how learners are using and applying facts and techniques studied in the eLearning module. That a learner successfully answered 90 percent of the quiz questions at the end of a module is meaningless unless the learner somehow takes that information and uses it to perform better on the job—to solve a problem, to communicate better with a direct report, or to provide a higher level of customer service, for example.
Looking at an entire learning ecosystem—learning and experiences that occur in the LMS are only one small piece of that ecosystem—can provide the data on experiences and behavior that create that context.
Some eLearning professionals suggest that the key is what? More data.
One approach is using xAPI statements to gather data from all sorts of learner activities, not only what happens inside the LMS. Learning Solutions Magazine will explore this possibility in a future article.
Another approach is to apply human ingenuity to analyzing the numbers by asking tough questions and challenging assumptions about those numbers. Etlinger, in the TED talk, shares a personal experience that underlines the importance of looking beyond key metrics and numbers. Her son was diagnosed with autism at age two, after failing to meet certain developmental metrics dealing with communication. These metrics looked at the number of words and communicative gestures he used; he was evaluated as communicating at the level of a nine-month-old baby. However, when her son was not quite four years old, Etlinger found him in front of the computer, running a series of Google searches. He’d somehow taught himself to read, type (with misspellings), use Google—and communicate.
“This is what happens when assessments and analytics overvalue one metric—in this case, verbal communication—and undervalue others, such as creative problem-solving,” Etlinger said in the talk. “Communication was hard for Isaac, and so he found a workaround to find out what he needed to know. And when you think about it, it makes a lot of sense, because forming a question is a really complex process, but he could get himself a lot of the way there by putting a word in a search box.”
Getting started
It’s easy to be overwhelmed by the amount of data available and the number of ways to analyze and use that data. Don’t worry; there’s help:
- The eLearning Guild published a white paper in December 2016, Learning Analytics: A Practical Pathway to Success, that outlines a six-step process to gathering, analyzing, and applying data on learners’ activities.
- Several sessions at Learning Solutions 2017 Conference & Expo (March 22 – 24 in Orlando, Florida) will address using analytics for performance support and improving employee experience. In fact, there are more than a dozen sessions on the subject in the Data and Measurement Track.