Whenever you use Google, Yahoo, or Bing, you have probably become used to seeing a short list of topic suggestions before you even finish typing your search string. Online retailers like Amazon.com, often suggest products to you as you shop (as in, “People who bought this also bought…”). What do think when an online retailer sends you an email suggesting it has found the products you are looking for (and they are a pretty close match)? What about when online movie services, like Netflix, recommend films to you based on prior viewing habits?
When websites appear to know you, and make recommendations to you, they are using analytics that are designed to predict what you want. Analytics make websites smarter. The predictions and recommendations they produce save us time and make the Web easier to use. Analytics personalize our user experience and enhance our relationship with the content provider.
Let’s take a closer look at analytics and how they can help improve learning and performance.
Analytics vs. reports
Analytics should not be confused with reports. Many of us are familiar with reports produced by systems such as LMSs, HRMSs, or other systems that we may use at work, or at home (home finance software, for example). Reports are very useful, but they only supply us with data: lists of items that can be filtered and sorted.
Analytics are far more sophisticated. Analytics recognize patterns in the data and illustrate them in ways that help us visualize what the data represents. Through analytics, we gain insight into how and why data patterns are occurring. Increasingly, businesses are transforming these insights into predictions and recommendations that connect the right people with the right content faster, and with greater accuracy.
Descriptive and predictive analytics
There are many categories of analytics, but two of the major types are descriptive and predictive. Descriptive analytics are used to interpret what has already happened. Predictive analytics are used to determine what might happen next.
Descriptive analytics
Many businesses and social networking services make use of descriptive analytics. For example, social networking services keep track of things like the number of posts, replies, mentions, followers, likes, and friends. These are easy to count because they have already happened. Analytics interprets this data to identify the most popular content, the most influential contributors, the hottest topics, relationships, and degrees of separations between friends or followers, and more.
Take this concept a step further. Analytics can segment users based on profile data to identify shared characteristics of people who have used certain types of content and interacted with the site in certain ways. These insights can be used to define rules that drive a customized social experience with features that list profiles you have visited recently, people who have viewed your profile, other profiles that have been also viewed by people who viewed yours, people you may know, and content that may be of interest to you.
Similarly, many retail businesses use analytics to count people who have viewed product descriptions, purchased products, opted in or out of marketing programs, bought product x after first viewing products y and z, made online and in-store purchases, and just about anything else that people do when visiting the site. Analytics can segment customers by profile data, purchase history, and other ways to help make product design, placement, and pricing decisions or refine marketing program terms, durations, and target customers.
For example, Google offers search suggestions soon after you begin typing. Its suggestions are based on data about how other people search, your location, your recent searches, and the words you are forming as you type.
Type the word lawn and Google may suggest things like lawn mower, lawn mower repair, and lawn service.
Delete your search string and retype the word law and you will see a new list that probably includes lawn mower first followed by items like lawless, law and order, and law abiding citizen. Lawn mower appears in the list because of your previous search attempt.
Add the letter y to form lawy and you will see another new list containing lawyer, and lawyers in [your vicinity].
Google uses descriptive analytics to save you time and help formulate the search string that will yield the results you seek.
Predictive analytics
Predictive analytics are used to determine what might happen next. Possible outcomes (predictions) are based on a set of inputs (predictors), ranked by probability (the likelihood they will happen.) This type of analytics is used to predict search-string suggestions, credit risk, fraud detection, health-condition risk, timing and demographics for sales and marketing offers, at-risk students, content relevance, and many other predictions.
Predictive analytics use a combination of descriptive (historic) and real-time data to create models that predict user needs as accurately as possible. There is a good deal of math and statistics involved in creating prediction models. Typically, they use probability and regression algorithms.
Probability calculates the chances of an outcome. A zero (0) probability indicates that the outcome is impossible and a probability of one (1) means that the outcome is certain. So the probability of most things falls somewhere between 0 and 1. For example, the probability that your coin toss will land on heads is 1/2 or 0.5. The probability that you will draw an ace from a deck of cards is 4/52 or about 0.077.
Regression analysis is a method used to estimate the expected changes to an outcome when one or more inputs are varied. Calculations are made to regress down towards the normal average, or mean probability of the outcome for each set of input conditions.
In predictive analytics, three key factors may be evaluated as inputs: (1) what is known about the content you seek; (2) what is known about you; and (3) what is known about others like you. Based on these three factors, a system attempts to predict what you will do next.
For example, Netflix evaluates what it knows about its movies. This may include how popular they are, awards they have won, their MPAA rating, how they are tagged by genre, and other characteristics. It also evaluates what it knows about you. This may include the genres you view most, the movies you’ve already viewed, the types of movies you watch more than once, how you rate movies, whether you tend to view movies in one sitting, and the times of day and days of the week that you tend to watch. Finally, it evaluates other people who have viewing characteristics similar to yours.
Netflix uses its predictive rating algorithms to predict the movies you are most likely to rate highly. Each row of movies you see on the menu is listed from left to right in order of your personal rating prediction. That’s why the movies displayed on each customer’s Netflix page are different.
Applying analytics to learning and performance: three examples
The ways in which one can use analytics, predictions, and recommendations to enhance learning and performance are limited only by our experience and imagination.
To get started, let’s discuss three illustrative examples: social media, knowledge management, and higher-education course selection. The first two are hypothetical; the third is a real application.
Social media for learning
Let us suppose you have recently introduced a new online environment to support a community of practice for developing the leadership skills of 2,500 front-line managers through communication and knowledge sharing. The online community enables managers to find and post articles on various topics to a blog, chat online about the articles and a variety of workplace issues, “agree” or “disagree” with specific articles, posts, and replies, and follow people whose articles and posts are of interest. A regular video interview featuring a different executive or manager each week is posted in the community site to communicate leadership values and generate discussion.
You apply descriptive analytics to interpret quantitative data like article submissions, posts, and replies and then identify the most relevant and controversial topics by region, functional area, and business unit. You use these insights to plan new video interviews and discussion topics. You may even inform upper management of controversial topics that may require guidance, clarification, or policy-making.
Your social-learning solution is designed to promote ongoing discussion and exchange. Analytics enable you to keep the pulse of what is important and relevant, and take appropriate action when needed.
Knowledge management
Let’s assume that you work for a company that manufactures and sells products. In one of the business groups in your company, a market-research team continually conducts market analysis, competitive analysis, and customer analysis. In the same business group, product-management teams design and develop various lines of products with features and price points geared toward different types of customers. In order to be effective, your company’s sales force needs to be knowledgeable about all of it: products, competitors, markets, and customers.
Formal courses can’t possibly keep up with the volatility of the information and sales people need to be in front of customers rather than in training. So you develop a knowledge exchange where market researchers and product managers can post competitive information, product information, and sales materials. Sales people can search the knowledge exchange for the latest information needed to make the sale and indicate whether or not they found specific content helpful.
Your knowledge exchange is so effective that six other business groups get on board. The system scales up to six times the amount of content. However, over time, some of the content becomes redundant, dated, no longer relevant or accurate. The combination of rapid system growth and lack of content “shelf-life” management will eventually result in overall degradation of the system’s value. For sales people, it will become more difficult to find the right information, expeditiously.
You implement predictive analytics to track content posts, search string entries, and content retrievals. You factor in what is known about the content (e.g., date published, date last updated, user rating, who published it, and how it is tagged); what is known about the user (e.g., business unit, region, rating trends, and frequency of use); and how other users with similar characteristics use the system. All this information is run through statistical algorithms to generate predictive models that enable you to make search suggestions and rank order search results based on predictions of relevance to the user. You also push email alerts to the user when items of predicted relevance are posted or updated.
You apply descriptive analytics to improve shelf-life management. Publishers of content that has not been retrieved for a given timespan or has been rated poorly are notified via email to review and update the content, or remove it from the site.
In this way, analytics dramatically increases the value of your knowledge-exchange solution to the sales force as well as to the product management and market researchers who publish content, while allowing the system to grow significantly in scale.
Higher education course selection
While many state universities are funded based on the number of students enrolled, Austin Peay State University in northern Tennessee is funded based on the number of students that graduate. To assist students in course selection, Austin Peay has developed a system called Degree Compass. It is available to students via a website and mobile app. Degree Compass evaluates what it knows about its curriculum such as degree requirements, majors, and credits. It also evaluates what it knows about the individual who is using the system, including the courses she completed, her grades, standardized test scores, and high school grades. Finally, the system evaluates what it knows about other students from previous years such as their transcript information and whether they graduated.
Degree Compass then provides a personalized course-ranking system to help students select courses that will count towards their degree. The system ranks potential courses based on (a) the course’s prerequisite requirements and relationship with the university’s general curriculum, (b) whether it fulfills unmet requirements related to the student’s major, and (c) a prediction of the grade the student is likely to receive in the course. A course’s ranking is indicated by the number of stars displayed next to it. Interestingly, data show that the system correctly predicts students who will earn a C or better in a course 90 percent of the time, grades to within 0.6 of a letter grade on average, and semester GPA within 0.02.
Resources in the field of learning analytics
Over the last 10 years, the application of analytics to learning has grown.
Columbia University Teacher’s College offers a focus in learning analytics in its master’s degree in cognitive science program.
Professional associations that sponsor conferences dedicated to learning analytics include the Society for Learning Analytics Research (SoLAR) and the International Educational Data Mining Society (IEDMS).
The Experience API (xAPI) specification allows us to record anything we want to track about anything users do, in any tool or system. You can use its tracking capabilities to supply analytics solutions with the big data needed to identify patterns, make predictions, and provide recommendations.
Commercial learning analytics software tools include LOCO-Analyst, Student Activity Monitor (SAM), and SNAPP. There are many general purpose business analytics tools that one can also use to analyze learning data.
Conclusion
When you really think about it, it is strange that businesses continue to rely on learning metrics that merely indicate whether an individual completed and passed, completed and failed, or did not complete training. Analytics provide boundless opportunities for us to collect and use much more meaningful data.
Implementing analytics solutions is not a simple task. It is likely to require specialized software and collaboration with a data analyst, perhaps a vendor, or someone in your IT department. However the opportunities analytics offer are significant.
The first step is to define what it is that you want to describe or predict. Next, identify the factors that need to be considered: what you know about your offerings, what you know about your user, and what you know about other users with similar characteristics. Then you’ll be ready to partner with a data analyst to explore various analytics methods.
Analytics can provide us with powerful, detailed insight into the effectiveness of our learning programs, improve our ability to plan for future learning needs, and enable us to predict the success of our learners. Just as many of the websites we use are getting “smarter,” saving us time, and becoming more personalized, we can apply analytics to provide the same benefits to the people who use our learning and performance solutions.