As e-Learning professionals, we spend a lot of time designing and developing what we hope and believe are learning applications that will teach and inspire learners to greater heights of productivity and work satisfaction. Through our learning programs, we may help to save lives or build better products, to provide superior service, or even to find solutions to world problems. Usually, though, we can but hope; many times we have no contact with learners after they've finished an e-Learning course we've created.
A not-often-asked question is this: when learners have finished taking an e-Learning course, are they now experts in what they learned. The answer is rather obvious: of course not. They have simply taken a course, in the same way that someone takes a college course. Most organizations will jump at the chance to shorten the amount of time between the end of a course and demonstration of expertise, especially if it doesn't prove to be an expensive proposition. But how? Yukon Learning's product called Cameo provides one innovative approach.
How Cameo helps
Some consider that giving a pretest and a posttest as part of an e-Learning course is a good way of noting how much individuals have learned. To a degree, it can help to show the difference between before and after. But much like photos in weight loss ads, just as those who lose weight often gain it back, much of what people learn during an e-Learning course (or a classroom course, for that matter), is often lost shortly after the course is over. This is especially true if the learner doesn’t put learning to use right away.
Many factors dictate how quickly an individual reaches a level of expertise sufficient that others will consider that person someone they trust to provide proper guidance. The difficulty of the material, as well as the amount of time individuals have to further their learning, is a factor, as is on-the-job training. It may take days or it may take years.
The principle behind Cameo is simple and its execution is rather elegant. Following the end of a course, learners receive an e-mail at predetermined times, for instance once each week. The e-mail is short and sweet. It presents a question related to the material learned — a question that can be answered in just a few seconds. Each e-mail poses a different question. A database records each learner's answers. From this database, Cameo can issue several types of reports, indicating individual and aggregate results.
Cameo is a push technology. Learners don't need to remember to check in with a Website every week. In fact, learners really don’t need to do anything at all. Assuming each learner has access to e-mail (and who doesn't?) the question shows up in a learner's inbox. Because each question is quick to read and answer, learners will tend to do so. In fact, Yukon Learning reports about a 79% response rate to e-mails. That is a sky-high number.
The Cameo interface
A dashboard gives you access to everything you need to manage this process. Separate tabs give you the ability to set up business units and users, including teams. Here you also can establish courses and topics, set up sessions (your schedule), and generate reports. You can even customize the dashboard with your own logo. (See Figure 1.)
Figure 1: The Cameo dashboard is fully customizable.
Writing the questions
Questions can be simple knowledge queries — but why stop there? Questions can present a scenario with text and/or an image, and then allow the learner to choose the best answer from several. In fact, this is the recommended approach. (See Figure 2.)
Figure 2: Questions can present a scenario with text and/or an image, and then allow the learner to choose the best answer.
What is the point of sending the questions? Generally, I see a twofold advantage:
-
It reinforces the learning by keeping it fresh in the minds of learners. Rather than ending a course, never to see it again, the learning continues in snippets. If a learner answers a question incorrectly, Cameo can provide feedback to help the learner recall the material. You can even send a link to the place in the original course where the answer appears, encouraging learners to return to review that part of the course. In short, Cameo doesn’t let learners off the hook but gives them opportunities to continue keeping the material at the forefront of their consciousness.
-
It helps to indicate any shortcomings in the original courseware. If many learners miss a certain question it may help determine that the related material in the original course may not have been clear or sufficient.
Who creates the questions? You do. That is, those responsible for designing the courseware and its contents would normally also create the questions. You can load the questions and set the schedule, and then let Cameo take things from there.
Cameo can randomize questions to help mix things up a bit. Cameo randomizes questions by analyzing both the questions stored for a given topic and all the topics for a given course. It puts the two together to determine a question bank for the course. Cameo then randomly selects a scenario for each learner and keeps track of what it sent to each learner. You also have the option of sending the scenarios in sequential fashion, if, for example, you have a series of questions describing a scenario that occurs in a natural order. You can weight answers to any question and assign scores so that you can create scenarios with more than one possible correct answer, only with some of the answers being a little more correct than others.
Setting the schedule
While it is rather intuitive that you will want to send reinforcing questions after a course is over, might it make sense to send questions in advance of a course? Certainly. Sending e-mail questions on a predetermined schedule before learners take a course can also be beneficial, in at least two ways:
-
It can help determine where gaps in knowledge exist. For instance, if a very high percentage of learners choose the right answer on certain questions, it may help you decide that there is little need to cover some parts of a course in detail. If many answer a question incorrectly, you may want to focus on some material even more than you had originally anticipated.
-
It can help build enthusiasm for the course by giving learners a taste of what is to come. This is especially true when you present learners with scenarios that are interesting and make them think. Scenarios are like little stories that challenge learners. Who doesn't like a good story?
Does it ever make sense to send questions to learners during a course? It may very well be a good idea, especially if the learning is taking place over an extended period. These questions can help gauge how learners are performing while they are still taking a course.
Of course, sending questions after a course is a given, for reasons already stated.
What happens if learners don't respond to the scenario? You can choose to ignore that fact, or you can set Cameo to send a reminder to the learner automatically after a certain period. Using the reminder functionality increases the response rate to 88%, according to Yukon Learning.
Giving feedback
Learners receive feedback immediately in a second e-mail. The feedback can include the original question and the learner's answer, along with the feedback and the score for the answer given. You can provide specific feedback for each possible answer in the original scenario.
An included reference link can add additional choice-specific remediation for the learner who wants to dig deeper into the material, a great way of ensuring that there are no barriers for a learner who wants to learn more. (See Figure 3.)
Figure 3: In Cameo, feedback can include the original question and the learner's answer, the feedback, and the score for the answer given.
Generating reports
Cameo lets you generate several types of reports and organize them in several ways. You can select filters and determine what results to show. See Figure 4.
Figure 4: Filters in Cameo determine the results that appear in reports.
You can also show results graphically as pie or bar charts, as in Figure 5.
Figure 5: Cameo gives the option to report results to course administrators in chart form.
You can e-mail or print reports. You can also export them to CSV files, making it possible to open them in Excel or other applications. In a new version coming soon, you will also be able to save reports as PDF files, and you will be able to push these reports to administrators of your choosing.
One other option worth mentioning is that Cameo can send a Leaderboard report. (See Figure 6.) This provides learners with information about how their individual performance compares to the performance of all others who have taken the same course.
Figure 6: The Leaderboard report provides normative information to learners.
Cameo limitations
So where does Cameo fall short? With tools that fit other needs, such as authoring tools, there are myriad products that make it easy to compare the features of one tool with that of one of its competitors and show where one excels over the other. But since Cameo is both a new and unique product in the e-Learning space, it is difficult to make those comparisons. Nonetheless, there are some potential shortcomings that I would like to point out.
1. You are limited in the amount of text you can include in a scenario. While Cameo's designers think this is a good way to eliminate long email messages, whenever a tool vendor limits an element of their tool, it can be seen as a bit condescending. Why not let users decide what limits they should impose on themselves?
2. When typing or pasting questions and answers in Cameo, different character sets are allowed, so you can use Cameo to send scenarios in different languages. However, if you're relying on Cameo's spellchecker, know that the only dictionary available is English.
3. While Cameo's Report Wizard gives you the ability to run a number of reports, you are unable to create templates for reports that you might want to run on a regular basis. This would save managers and administrators a lot of time since they would not have to recreate the report each time they wanted to analyze the data in a specific way.
4. If you're using an LMS, you can use Cameo's API to move data into Cameo. But at the moment, this API is a "one-way street" and does not facilitate moving data back into the LMS. I am told this is an area they will address soon.
5. While I think that Cameo's pricing model works and is fair for what the tool gives you, there is currently no "pay-as-you-go" option. This may prevent some smaller organizations from using Cameo.
Conclusion
I find Cameo a very interesting product, one that is worth considering for use before, during, and especially after e-Learning. I have already decided to try it out when I teach Adobe Captivate and other classes. Before teaching a class, I will send periodic questions to my students to gauge how much they know. After the class, I will send them a question each week to remind them of what they learned and see how much they remember. The reports I generate should help me understand how to further enhance my teaching materials and style.
Depending on the results of these trials, as part of the e-Learning custom services we offer at my company, I may also begin to propose the use of Cameo to our clients, to help them gauge how well their employees or clients are learning from our materials. I believe this product fits a need that I have felt for a long time. While I can see areas for improvement, it is more than capable right now to help us enhance our services.