We live in a rapidly changing environment where learning leaders need to differentiate between the activities that will allow their business to grow and those that will take value away and prevent them from evolving. The challenge is identifying which is which; the solution is using ROI data.
It’s known that ROI data enables you to measure L&D’s value creation, but an additional benefit, one that may be a surprise for some, is that it can also provide insight into the best strategy to maximize your chances of future success. To assist you and fellow learning leaders in analyzing and applying your data, this article describes some strategy questions and ways to use ROI data to uncover valuable hints and identify ways of “prototyping” your department’s future.
What assets should you be creating?
If you track your ROI, one of the first insights you will get from your numbers is what types of assets provide the greatest value to the organization. Is it instructor-led training, virtual-led training, web-based training, or perhaps virtual-reality-based learning? The higher the value a modality creates, the more you want to focus your resources and talent on pursuing it.
At my organization, a few years ago, we moved from having an internal L&D team that specialized in instructor-led and basic web-based training to a new structure that allows for in-house production of advanced web-based training, videos, podcasts, micro-sites, applications, and virtual reality, among others. We’ve discovered that doing these activities internally drives more value than engaging external vendors.
What assets should you outsource?
The logical consequence of the above is defining what not to produce internally. The standard answer would be to limit the resources you allocate to forms of learning that provide low or negative value compared with what an external provider can offer you.
Using my organization again as an example, we limited the resources and talent dedicated to producing standard instructor-led and basic web-based training. The value provided by those assets was relatively low, and we found that it was more beneficial to have third parties develop them for us.
What relationships are working, and which ones need to evolve?
An unexpected outcome from doing a deep-dive analysis into our ROI data was finding that the numbers can reveal which relationships are very productive and which ones not so much.
The approach to calculating our ROI involves tracking the number of hours it took to create every asset we produced. The neat thing about it is that, in time, you can create statistical standards and understand what assets, or projects, take consistently shorter, or longer, than average to produce.
Either situation is the trigger for some very interesting conversations with your stakeholders. When the timelines are shorter than expected, it is a good opportunity to thank your partners for helping you be more productive, as well as to dig deeper into what makes projects with them so efficient. When it takes longer, it is a good chance to share feedback about what makes the engagements with them inefficient compared with other lines of business.
An incredible benefit we derived from this process was to discover that our L&D team was extremely agile. We were able to deliver within the timelines, and were very flexible when we needed to pivot under tight schedules. Agility is a two-way street, though; delays and rework were often linked to the stakeholders’ lack of readiness with content.
I'm sure that most of you don’t find this surprising, as it’s common knowledge in the industry. However, having the numbers that show this makes a world of difference: The conversation becomes a different one, based on fact—data—instead of personal impressions. It is easy, then, to focus on solutions and on how to produce a truly agile environment together.
Reality check: Is your learning strategy working?
Data provide a great reality check: Once you are measuring your output by modality, every month, quarter, and year, it is easy to see if the assets you are producing reflect your learning strategy goals.
For instance, if your objective was to double the amount of video-production as a percentage of your learning assets in the next three years, and your numbers show that in year one video output was 1% of the total, in year two was 2%, and by year three it was 3.5%, you can safely claim that you successfully surpassed your goal.
ROI numbers also provide a good antidote to confirmation bias. Sometimes we put so much effort into implementing a new modality (by buying new technologies, training designers on how to use it, and promoting its implementation) that we assume the output for that modality will follow suit. Often, that’s not the case. The fact that we are so invested is likely to distort our perception of how much we are really producing.
We experienced this firsthand in 2014, when we were convinced that our average output of web-based learning assets was 30% of our total production. Once we started measuring the actual output (in 2015), we realized it was closer to only 5%.
I know, it sounds ridiculous to think that we were off by so much. The reality is that when we did the original calculations, we were influenced by our confirmation and availability biases. We had just completed a big project where we transferred some key Instructor-led training programs to a web-based training modality. In our minds, the data made sense, but reality was different. At the time, we didn’t have a rigorous approach to track output; everything was based on estimates and multiple spreadsheets structured with assumptions from what was salient in our minds.
Are you ready to be proactive about your future?
Our ROI data started giving us valuable insights five years ago. The department we are today reflects our vision from those days. We made important decisions on what assets we had to focus on and how to evolve our relationships with the different lines of business, and we kept an eye on how our output was reflecting our vision—or not reflecting our vision. In time, it paid off.
No one knows the future. However, as with the assets you design, you can always “prototype” what your L&D function might look like in one, three, or five years from now, and use those insights to prioritize your efforts. Leveraging a fact-based approach that includes your ROI data among other sources will increase your chances of adapting, innovating, and flourishing.
Grow your leadership skills
Are you seeking the strategies and skills required to navigate the needs of today’s ever-changing workplace? Are you an experienced or aspiring leader looking for a community to connect with to explore today’s biggest learning leadership challenges?
The Learning Leaders Online Forum, March 16–17, offers a deep dive into the skills, strategies, and practices today's learning leaders need to excel. Join your peers as Juan Naranjo and other engaging presenters share their experience and expertise. Register today; or join our Learning Leaders Alliance, and enjoy free access to the Online Forum!