If you’ve been following along in my series of articles, you’ve covered a lot of ground. And now, it’s time to bring it all together. In this final article in the series, we’re going to look at the reporting side of the examples used in the third article: “xAPI Can Tell You What They Learned from the Video.” In that article, we looked at sending the statements to record when someone played a video and what part of the video that user played. We also looked at one way to report which answer the user selected on a quiz asking about the video. Again, the question was “What is the first animal you see in the video?” Not exactly the deepest or most probing question, I know. But it does show if the person was paying attention.
In this final outing, we’re going to look at how you can pull that data back from the LRS. Using xAPI queries, we’ll show who answered the question, what their answer was, and if they played the relevant part of the video. This way we can find out whether the video is effective. Remember, there are three practical results we can get here:
- The average user watches the video, but gets the answer incorrect. This could potentially indicate the video is not effective.
- The average user does not watch the video, but does get the answer correct. This could indicate that the question was too simple.
- The average user who did watch the video answers the question correctly, and the average user who did not watch the video answers the question incorrectly. This would tend to indicate that both the question and the video are effective.
We’re hoping to see the report trend toward the third option.
But I’m not a programmer/developer
As usual, I will show code examples so you can see how I did things. But the process is what’s really important here, so I’ll do my best to describe that outside of the code.
We have to consider what we want to know. That means knowing what data we need, how to get it, and how we can piece it together. At the end of the day, the code is simple. The strategy is the more difficult thing to put together, and that can be made much easier. (A great place to start is with a fantastic book from Sean Putman and Janet Laane Effron titled Investigating Performance: Design and Outcomes with xAPI.)
Looking at the data
So, from the previous articles in this series, we’ve generated some statements. There are two we want to look at here. The first is the information the LRS knows about when I watched the video:
{
"verb": {
"id": "http://adlnet.gov/expapi/verbs/Play",
"display": {
"en-US": "Video Played"
}
},
"version": "1.0.0",
"timestamp": "2017-08-19T13:37:22.625732+00:00",
"object": {
"definition": {
"name": {
"en-US": "Big Buck Bunny Video"
},
"description": {
"en-US": "sample description"
}
},
"id": "http://example.com/bigbuckbunnyvid.html",
"objectType": "Activity"
},
"actor": {
"mbox": "mailto:anthony@devlearn16.com",
"name": "anthony",
"objectType": "Agent"
},
"stored": "2017-08-19T13:37:22.625732+00:00",
"result": {
"extensions": {
"http://example.com/xapi/period_end": 17.4,
"http://example.com/xapi/period_start": 1.5
}
},
"id": "5125d510-00c5-4431-a3aa-9918d688d437",
"authority": {
"mbox": "mailto:techteam+xapi-tools@adlnet.gov",
"name": "xapi-tools",
"objectType": "Agent"
}
}
Let’s take a look at what we can learn from this:
- Who played the video (in this case, it was me)
- I played the video on August 19 at about 13:37 Zulu
- The video I played was my favorite example: Big Buck Bunny
- I played it from 1.5 seconds to 17.4 seconds
- I played the video (the specific verb is useful to know, as well!)
That’s a good bit of information. And we’ll be able to use all of it. Now, let’s look at what we know about my effort to answer the quiz question:
{
"verb": {
"id": "http://adlnet.gov/expapi/verbs/answered",
"display": {
"en-US": "answered"
}
},
"version": "1.0.0",
"timestamp": "2017-08-19T13:38:26.793810+00:00",
"object": {
"definition": {
"name": {
"en-US": "xAPI Video Quiz"
},
"description": {
"en-US": "Correlating quiz answers to video consumption"
}
},
"id": "http://omnesLRS.com/xapi/quiz_tracker",
"objectType": "Activity"
},
"actor": {
"mbox": "mailto:anthony@devlearn16.com",
"name": "anthony",
"objectType": "Agent"
},
"stored": "2017-08-19T13:38:26.793810+00:00",
"result": {
"extensions": {
"http://example.com/xapi/location": "15.6"
},
"response": "Bunny"
},
"id": "88291bfb-b7fb-4131-847b-e0445826461b",
"authority": {
"mbox": "mailto:techteam+xapi-tools@adlnet.gov",
"name": "xapi-tools",
"objectType": "Agent"
}
}
Similar to the video statement, we can learn a bunch of stuff from this statement:
- Someone answered the question (again, guilty)
- I answered the question (verbs matter)
- I answered the question at 13:38 Zulu on August 19
- I answered that specific question
- “Bunny” was my answer
- The answer can be found at 15.6 seconds into the video
Putting all of this together
So now we have a lot of information about this one consumer. It’s fairly easy to just eyeball the two statements and see that I watched the relevant part of the video, but I did answer the question incorrectly (the correct answer is “Bird”). But now let’s try that for a few thousand consumers. That’s not going to be so easy. We want a way to programmatically query all the relevant statements.
This is where strategy takes over: What is the exact question we want answered? I can see two possibilities:
- Of those who played the video, who answered the question and how did they answer?
- Of those who answered the question, how did they answer and did they play the video?
Essentially, this translates into one of two processes, each of which requires several steps:
- Of those who watched the video, who answered the question and how did they answer?
- Query for statements where someone played that video. (Remember, you can’t query on extensions! So we have to get all the statements.)
- Crawl through those statements for those who played the relevant bit of the video, and collect the agent IDs.
- For each agent ID we collect, query for any statements where that agent answered the quiz question.
- Of those who answered the question, how did they answer and did they watch the video?
- Query for statements where someone answered the question.
- For each agent that answered the question, query for statements where that agent played the video.
- Crawl through the video statements to see if the consumer played the relevant bits of the video. (Again, we can’t query for extensions; we must do this on the client side.)
At first glance, you might not see a difference between those processes. But there could be a huge difference here. Specifically, there’s likely to be many more statements for the video than for the question. Consumers may have watched the video several times. Or there may be a significant number who have watched the video, but haven’t progressed through the course to the quiz yet.
In the end, we may have the same amount of statements to process, but if we take the second option, we should have fewer queries to run. So, it’s more efficient to take the second route and start by querying for those who answered the test question first. And that is easily done. Using the ADL xAPI wrapper, the code looks like this :
var quizParams = ADL.XAPIWrapper.searchParams() quizParams['verb'] = "http://adlnet.gov/expapi/verbs/answered"; quizParams['activity'] = "http://omnesLRS.com/xapi/quiz_tracker"; // Now, issue the query var ret = ADL.XAPIWrapper.getStatements(quizParams);
Looking at the code, we create a variable for the query parameters in line 1. In lines 2 and 3, we set the verb and activity IDs for which we want to query. In line 6, we send the query, assigning the returned statements to the variable “ret.”
Once we have the statements, we crawl through them and start building the next query :
if (ret) { for (i = 0; i < ret.statements.length; i++) { var studentName = ret.statements[i].actor.mbox; var studentAnswer = ret.statements[i].result.response; var studentTS = ret.statements[i].timestamp + " "; var answerLocation = ret.statements[i].result.extensions["http://example.com/xapi/location"];
In line 1, we make sure that there were statements returned. If there are, then one at a time, we look at them and pull out the data we need, using the “for” loop defined in line 2. We’re collecting the student’s name (in this case it’s actually their email address), the answer selected, when they answered the question, and where the answer was provided in the video. This last one (on line 6) is by pulling from the result extension. Notice how we have to reference that, using the URL defined in that extension’s keyed pair from line 29 of the quiz statement seen above. And, the dotted notation for each of the data points is easy to follow: ret.statements[i].actor.mbox maps to the mbox defined in the actor object of the statement stored in the variable “ret.” And so on and so forth.
So now we have all the information we need for each of the statements where someone answered the question. Now, we need to see if they watched the video. This requires another query. Again, fairly easily built :
var vidParams = ADL.XAPIWrapper.searchParams()
vidParams['agent'] = '{"mbox":"' + studentName + '"}';
vidParams['verb'] = "http://adlnet.gov/expapi/verbs/Play";
//Make sure you pay attention the CASE OF THE VERB!!!!!!!!!
vidParams['activity'] = "http://example.com/bigbuckbunnyvid.html";
var vidRet = ADL.XAPIWrapper.getStatements(vidParams);
Note: First and foremost, take heed of the comment on line 4. Everything is case sensitive! You must query for the same case as you used in the original statements! So make sure you keep all of that consistent—I can’t stress that enough! (I may have lost several hours debugging that mistake. Sigh…)
So, same as the first query, we’re setting the verb and the activity IDs. But this time, we’re also using the user email address we collected from the quiz statements to search for any statements where that user played the videos. This time, we’re assigning the videos to “vidRet.”
I’ll spare you the code here, as it’s a little tedious, but I’ll include the full text below for the curious and adventurous. The process is straightforward, though: Look at each statement and see if the users played the segment of video containing the answer. We do this by looking at the result extensions for the period_start and period_end and comparing those to the answerLocation variable we defined in the quiz query-results example above. If the users played that segment of the video, then we set a variable called vidWatched to “true.” And if not, then we leave that variable as “false,” meaning they didn’t play that segment of the video. Then we build that line of the chart and move on to the next statement in the quiz query results, and do it all over again.
The finished product will look like Table 1.
Table 1: The result of the query
If we know the correct answer (Bird), then we can see who answered the question correctly and if they played that segment of the video. Now, from here, we could add code to check for correct answers, and if they played the video, and add a column to reflect that. I did not do that here, for the sake of simplicity. There’s already a lot going on with this page.
In closing…
And that’s it, folks! With only a few lines of code, we’re able to bring together two completely disparate sets of data, compare them, and then correlate that data to show if the video is effective or not. In this case, looking at the results, it is not!
Notice that of the six people who played the video, only three got the correct answer! That’s only 50 percent. That’s hardly effective. But now we can show that. We have the data to prove this. OK, technically not yet: We’d need far more data, as this sample size is far too small. But you can see how this can scale up quickly and easily. And at the end of the day, this, more than anything else, is the most exciting part of xAPI. Before xAPI happened along, it was very difficult to build reports like this pulling together this data. Now, you can do it with the push of a button (and some code behind the scenes). This is what makes xAPI special. This is what makes xAPI powerful. And this is why, in this author’s humble opinion, xAPI is the way we will track our consumers in the future. It shouldn’t be much of a stretch for you to see how, using similar queries to those I’ve used here, you could make courses adapt to a student’s results in another, prerequisite course. Or how you could build just about any other report you can imagine.
I really do hope you’ve found this series of articles helpful. I’ve tried to be as thorough as I can be without being too overwhelming. xAPI can be a little like the game Go: a moment to learn; a lifetime to master. But it doesn’t have to be. You can make it as complex or as simple as you want! It’s all up to you and your needs. All that power is there for you to use at your will. And no matter how complicated you need it to be, just remember this…
It all starts with four lines of code.
Bonus
Below is the full text for this example. I hope you can find it useful!
Note to developers: Yes, this is a painful way to do this. It requires N-many queries based on how many people took the quiz. If 1,000 people took the quiz, we’re running 1,000 queries for the video statements. The point here is to illustrate the process, not necessarily the most efficient method. In this example, I can show two query processes. So it meets the needs of this article very nicely, although isn’t really “production ready .”
<!doctype html>
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
<title>xAPI sample page</title>
<meta name="description" content="">
<meta name="author" content="">
<style>
table, th, td {
border: 1px solid black;
border-collapse: collapse;
}
</style>
<script type="text/javascript" src="./js/cryptojs_v3.1.2.js"></script>
<script type="text/javascript" src="./js/verbs.js"></script>
<script type="text/javascript" src="./js/xapistatement.js"></script>
<script type="text/javascript" src="./js/xapiwrapper.js"></script>
<script>
function config_LRS(){
// Tell the content where to send the xAPI statements
var conf = {
"endpoint" : "https://lrs.adlnet.gov/xapi/",
"auth" : "Basic " + toBase64('xapi-tools:xapi-tools')
};
function get_statements(){
config_LRS();
// Start by getting the list of statements for those who answered the quiz question
// First, we set the parameters for the query:
var quizParams = ADL.XAPIWrapper.searchParams()
quizParams['verb'] = "http://adlnet.gov/expapi/verbs/answered";
quizParams['activity'] = "http://omnesLRS.com/xapi/quiz_tracker";
// Now, issue the query
var ret = ADL.XAPIWrapper.getStatements(quizParams);
console.log("answered list: ");
console.log (ret);
var txt = " ";
if (ret) {
// walk through the list of "answer" statements to see what we have
for (i = 0; i < ret.statements.length; i++) {
var studentName = ret.statements[i].actor.mbox;
var studentAnswer = ret.statements[i].result.response;
var studentTS = ret.statements[i].timestamp + "<br>";
var answerLocation = ret.statements[i].result.extensions["http://example.com/xapi/location"];
// When we find a statement for someone who answered the question, search for statements
// for when/if that user played the video
// Set the parameters for the query for "Play"
var vidParams = ADL.XAPIWrapper.searchParams()
vidParams['agent'] = '{"mbox":"' + studentName + '"}';
vidParams['verb'] = "http://adlnet.gov/expapi/verbs/Play";
// Make sure you pay attention to the CASE OF THE VERB!!!!!!!!!
vidParams['activity'] = "http://example.com/bigbuckbunnyvid.html";
// Send the query for statements where the student played this video
var vidRet = ADL.XAPIWrapper.getStatements(vidParams);
var vidWatched = false;
// Now walk through the Play statements
for (x = 0; x < vidRet.statements.length; x++) {
if (!(vidWatched))
// Look to see if the user played the relevant segment of the video
if (answerLocation > vidRet.statements[x].result.extensions["http://example.com/xapi/period_start"])
{
if (answerLocation < vidRet.statements[x].result.extensions["http://example.com/xapi/period_end"])
{
vidWatched = true;
console.log ("vidWatched = true");
}
else { vidWatched = false;
console.log ("vidWatched = " );
}
}
else { var vidWatched = false;
console.log ("vidWatched = " );
}
}
var table = document.getElementById("result_table");
var row = table.insertRow(i + 1);
var cell0 = row.insertCell(0);
cell0.innerHTML = i;
var cell1 = row.insertCell(1);
cell1.innerHTML = studentName;
var cell2 = row.insertCell(2);
cell2.innerHTML = studentAnswer;
var cell3 = row.insertCell(3);
cell3.innerHTML = vidWatched;
}
}
}
</script>
</head>
<body>
<button type="button" onclick="get_statements()">Get Statements</button>
<br/><br/><br/>
<div id="results">
<table id="result_table" style="width:50%">
<tr>
<td> # </td>
<td><b>Student Name</b></td>
<td><b>Student Answer</b></td>
<td><b>Video Watched</b></td>
</tr>
</table>
</div>
</body>
</html>