A client recently asked us to help them keep track of their learning outcomes throughout the development and delivery of their online course, as they needed to satisfy strict compliance requirements. The customer needed in-depth objective tracking; this meant that they needed to be able to map learning outcomes to individual parts of their course and evaluate these outcomes via rubric-based assessment. They also needed analytics that could provide detailed reporting on standard metrics and learning outcomes across all levels of their organization.
Based on these requirements, we conducted extensive research on tools that could improve the tracking and reporting of learning objectives. As activities such as tracking and reporting take place within an organization’s learning management system(s) (LMSs), we started our research by evaluating the capabilities of several popular learning management systems, and we found that some are far more capable of managing learning objectives than others.
Key factors
LMSs that allow for strong learning-objective management can be differentiated based on several key factors:
1. Rubric creation and customization
Learning management systems should allow users to easily create and customize rubrics. Rubrics are assessment tools that define the criteria being used to evaluate an assignment, and they are used to communicate expectations to students as well as for effectively grading assignments. Some LMSs allow rubrics to be stored in a repository that staff members and administrators can manage, edit, and customize.
One of the best popular LMSs we found for easy rubric generation was Canvas. Within Canvas, users have the option of creating fully customizable learning outcome rubrics quickly and easily (Figure 1); users can also search a database of existing outcomes.
Figure 1: A rubric created in Canvas LMS (source: Canvas Help Center)
Moodle had great potential for flexible rubric creation due to its open-source nature. The fact that Moodle is open-source means that you can almost fully customize it in every aspect—including rubric generation. However, while customizing Moodle was possible, we found that in practice it required an expert level of training.
2. Mapping learning outcomes
Ideally, an LMS should allow users to create and map learning outcomes across multiple areas—from interactive content to summative assessments and assignments. This provides facilitators with visibility over learning objectives to see exactly what components of individual eLearning modules are satisfying their respective learning outcomes, and by whom.
Some learning management systems offer rubric-style learning outcome mapping, though these rubrics are differentiated from the assessment-based rubrics mentioned above, as they’re used to ensure that a course adheres to training-delivery criteria rather than ensuring that a student’s work adheres to assessment criteria.
One of the most comprehensive offerings we found in regard to mapping learning outcomes was eCoach LMS. With this platform, you can map learning outcomes at a granular level, from specific lesson screens to individual quiz questions (Figure 2).
Figure
2: eCoach LMS allows you to map learning outcomes at a granular level (source:
eCoach)
Furthermore, eCoach LMS can also be bundled with eCoach Author, a cloud-based authoring tool that syncs to the LMS and allows for real-time mapping of learning objectives as you create content (Figure 3).
Figure 3: eCoach Author allows for real-time mapping of learning objectives as you create content (source: eCoach)
Uniquely, this approach means that learning objectives can be reported on, not only in assessment items, but through all course content. This was important to our client, who needed to view specific learning outcomes in order to ensure compliance.
3. Assessment and grading
A well-considered LMS should ideally allow users to assess and grade students’ work in line with a rubric or set of learning outcomes. In other words, rubrics and learning outcomes should be viewable during grading, with options provided for tallying marks using the rubric itself. The best LMSs also offer integrated feedback options; Canvas was again a standout LMS in this regard, with users able to comment on and grade student work against an existing rubric from a single window. eCoach LMS doesn’t currently have this feature, but it is apparently planned in a future release.
4. Reporting and analytics
Our customer also required granular reporting and visualization options in their course analytics to provide “at a glance” reporting that they can drill down into for each course component. eCoach LMS ticked all of these boxes, providing individual participant reports (to see which students had completed which learning objectives), while also tracking overall cohort performance on specific objectives (by department or across the entire organization).
Overall learning management system assessment
Based on the aforementioned factors, the best LMSs we found for managing learning objectives were Canvas and eCoach LMS. Moodle’s customization options meant that rubric generation was a strong point, though actual customization had to be outsourced. Moodle also lacked grading and feedback options and fared poorly in terms of analytics and reporting, which must be completed manually in external platforms such as Excel. Another LMS we evaluated was Blackboard, which lacked the customization options of Moodle, had dense and unwieldy menu systems for creating rubrics, and did not provide the same range of feedback options as systems like Canvas. As such, Blackboard proved one of the more limited LMS products in terms of managing learning objectives.
Specialized LMS platforms
A further option we examined (and one that may be suitable for organizations whose eLearning courses are almost entirely made up of compliance training) was the installation of a CLMS, or compliance learning management system. CLMSs are LMSs that focus specifically on compliance training and outcome mapping as their key priority. GRC Solutions, for example, offers a CLMS that allows for comprehensive and specialized compliance training.
However, CLMSs don’t offer the same range of learning management options as more orthodox learning management platforms: Training is delivered through a course library, meaning that users cannot be split into individual departments or courses, while user management is limited to assigning each user’s job title. CLMSs are recommended only for organizations focused solely on compliance training.
Add-on software for existing LMSs
Clearly, our client’s existing LMS did not allow for strong learning objective management, but switching platforms was not a preferred option due to the time and effort involved. For this reason, we also evaluated a range of add-on software (or plugins) that can integrate with existing learning management systems and that focus entirely on compliance training and mapping learning outcomes.
One of the most powerful add-on software products we found was Chalk & Wire. Chalk & Wire offers both LTI and ATP integration, meaning that it can integrate with almost any LMS on the market. From within their LMS, users can use Chalk & Wire to create, customize, and store rubrics, provide assessment feedback in-line, and access detailed granular analytics across multiple institutional levels. Chalk & Wire integrated with our client’s LMS with a minimum of fuss and was easy to begin using. The downfall here was that Chalk & Wire is suited to assignment-style content only, and not interactive eLearning.
Another external software suited to educators is Atlas Curriculum Mapping, which is fully customizable and has very flexible mapping options, but cannot integrate with LMSs, meaning that you have to enter data into the platform manually. Atlas Curriculum Mapping looked like a good tool for creating curriculums in collaboration with other educators, but did not suit our particular client’s needs.
Conclusion
Learning management systems
The two standouts here were Canvas and eCoach LMS, both of which offered rubric generation and learning-outcome mapping, assessment options, and detailed analytics. We would recommend Canvas for more straightforward, assessment-style learning, as it currently offers better feedback and grading options. eCoach LMS is recommended for more interactive eLearning, as you can easily map and track authored content in the eCoach platform. eCoach LMS also provided the best granular reporting and analytics, with the potential to break learning outcomes down across the entire organization, department, group, individual, or specific resource.
Add-on software
Using an LMS that contains features for managing learning objectives should be the preferred option for organizations that want to keep track of their learning outcomes, but it’s not always possible.
If your organization’s LMS does not offer adequate tracking features, add-on software such as Chalk & Wire can be helpful, though it is limited to assignment-style learning rather than interactive eLearning. Chalk & Wire successfully integrated with all the major learning management platforms we tried it on, including Moodle, Blackboard, and Canvas.