The Real ROI Of Learning: How Do You Measure The Impact Of L&D?

How Do You Determine The Real ROI Of L&D Initiatives?

This article is part of a series on tracking the right metrics using the right methodology to visually display and quantify the investment in learning and prove ROI. What you are about to read is a fable. The company, AshCom, is fictional, but the learning challenges faced by Kathryn, AshCom’s CLO, and her team are real and commonly shared by learning teams in large organizations. It is our hope that you will be able to connect with the characters, their challenges, and the solutions they discover. We also invite you to read the first eBook in the series.

The Morning Shadow

Status updating, problem-solving, team-building… these were the typical meetings that filled Kathryn’s daily calendar. She glanced at the appointments set on this day. Her eyes were drawn to the 10 a.m. slot. An uneasy feeling, like a shadow, followed her as she moved through her morning schedule.

Kathryn, the Chief Learning Officer of AshCom, knew this meeting was coming. She knew that her company had unexpected losses for the last quarter and that the losses were continuing into the most recent months. She had heard about the meetings that were taking place between their CEO, Laszlo, who was the Chief Operating Officer, and her friend Kurtis, the CFO. There were rumors, most of which she ignored, about potential layoffs and other significant cost-cutting measures.

She had seen this before in other companies where she had worked. They grew steadily and were profitable but at some point, they became inefficient. Companies in this position often replaced members of the C-Suite to right the ship. This meant that Kurtis was under significant strain when he emailed her asking for a 60-minute meeting, something not typical for him. He usually covered what he needed to cover in 15 minutes or less. Despite their friendship, she also knew that Kurtis was going to be far more serious than normal in the meeting. It might be uncomfortable.

Kathryn’s hunch was correct. When Kurtis entered her office, he looked tired, the kind that comes from both stress and a lack of good sleep.

“How are you doing?” asked Kathryn showing more concern for him as a person than for the conversation they were about to have.

Getting To The Root Of The Problem

“About how you would suspect given what we are in the middle of,” replied Kurtis. “We are getting to the bottom of some of the problems we need to fix but there are a lot more than we realized. Which is why we keep losing money month after month.”

“How can I help?” asked Kathryn.

“I want you to know that I am meeting with leaders throughout AshCom, so I’m not singling you or your learning team out,” replied Kurtis.

“I know,” said Kathryn, “I’ve heard some of the rumors.”

“Most of those rumors are true I suppose,” said Kurtis, looking depleted. “I want you to know that I am not here to ask you to cut your staff. That has to be one of your first concerns.”

“It is,” said Kathryn. “Look, I’ve been through this before with other companies that were struggling. Twice I’ve seen it where the first people laid off are the people on the learning team.”

“I’ve seen the same,” said Kurtis.

“On some level,” said Kathryn, “I understand. The company has learning assets built up that they can use for a while without too much damage. It’s why marketing people are also some of the first to be let go. There is a well the company can draw from for a while before any real damage is done. It isn’t like they are going to cut production or salespeople.”

“That’s pretty standard from what I’ve seen and read,” said Kurtis.

“So, no staff cuts?” asked Kathryn, more serious than Kurtis had ever seen her before.

“No. No staff cuts,” replied Kurtis. “At least not yet. I think we have identified several issues. If we fix them, we will improve our financial situation and get back to profitability. And normal life for me.”

“You have a problem that can be solved by learning?” asked Kathryn, breathing a sigh of relief.

“Yes,” replied Kurtis. “But that isn’t the end of it.”

“Let’s begin with the learning problem,” said Kathryn. “We can cover whatever bad news you have after that.”

The Financial Challenge

“Fair enough,” replied Kurtis. “Our learning challenge is related to the financial challenge. We are digging into every area of our operations and have found that our lack of solid preventative maintenance is a huge problem.”

Instinctively, Kathryn leaned back in her chair, somewhat taken aback by what she heard. “Preventative maintenance? You mean maintaining our machines? Like oiling them and cleaning them?” A light chuckle escaped.

After a moment, Kathryn continued, “So why are you here talking to me about this? You are a financial expert. I am a learning expert. Shouldn’t our COO be talking to me about this?”

“He will,” replied Kurtis. “But I wanted you to understand that this is a very serious issue. Our production per hour has dropped a lot in the last couple of years. Our scrap rate – the stuff we have to rework or recycle – has been climbing. It is millions of dollars. Our machines that should be lasting 8 years are lasting only 6 years. We are consuming more energy than we need to, and we are endangering our workers. All because our machines are not being properly maintained.”

“I know we have preventative maintenance learning,” said Kathryn. “I’m sure new employees take it when they are onboarded. We have hundreds of learning opportunities, so I’m not sure off the top of my head if this training is reinforced much after someone on the production line is onboarded.”

“I don’t know the answer to that either,” replied Kurtis. “I suspect no one was really looking at it that carefully. Now that we are, we are seeing that this is one of those problems right in front of us that we haven’t been seeing. We see it now and we need you and your team to build better and stronger learning experiences related to preventative maintenance.”

Calculating The Real ROI Of Learning

“You’ve challenged us before,” said Kathryn, “and we’ve always risen to the occasion. I’m sure we will here too. But I have to come back to my question: Why isn’t Laszlo, our COO, having this conversation with me?”

“Because I have a second reason to be here,” replied Kurtis. “You’ve heard about ‘Defend the Spend’?”

“That was one of the rumors,” said Kathryn. “That was one I actually believed. We are all going to be asked to defend our spending, correct? Maybe not the best name for it—”

“I know,” interrupted Kurtis. “But it rhymes, so we should get some credit for that.”

“Only a little,” said Kathryn. “It may put people on edge more than you realize. Especially my learning team. We know that the results of what we do are hard to put into a spreadsheet.”

“Return on investment, or ROI,” said Kurtis.

“I’m familiar,” said Kathryn. “It isn’t that people in my role don’t understand its importance. Or that we don’t care about financial performance. Learning is different than operations or sales. They have metrics and dashboards. Operations can count things like production per hour, scrap rates, and material costs. Human Resources can talk about employee retention, new hires, and wages and benefits. The sales team can show emails sent, calls made, and booked business. They can also do client surveys.”

“And you don’t have access to data like that,” replied Kurtis.

“We have some,” said Kathryn, “but not like the other divisions. They are making their impact known through numbers. It is harder to do that with what we do on the learning team. I’ve attended dozens of learning conferences, and I can tell you that the most popular sessions are always related to measuring the impact of learning or ROI or finding a scorecard to show why what we do matters.”

“But apparently you haven’t found anything you like?” asked Kurtis.

“They mostly talk about why it’s important,” said Kathryn. “What is missing is the how. How do we measure the ROI of learning?”

“That is part of the reason for me meeting with you,” said Kurtis. “As we’ve reviewed AshCom spending in the last three years, we looked for departments that had some of the largest percentage increases.”

“And we are on that list,” replied Kathryn. It was more of a statement than a question.

“Actually, near the top in terms of percentage increases,” replied Kurtis. “I did a little research and found that learning budgets in medium and large companies have increased almost 50% in the last year alone. And your spending is even higher than that.”

“I don’t want to be defensive,” said Kathryn, “but some of the projects we have taken on in the last couple of years have been expensive—like the financial literacy game you had us develop last year.”

“I know that was pricey,” said Kurtis, “but it was necessary. And I’m sure most people in management would say it was well worth the time and expense. You’ve also built an entirely virtual learning program for our remote workers. That seems to be working too.”

“But,” said Kathryn, “that doesn’t quite rise to the level of ‘Defend the Spend’ you are looking for.”

“Correct,” said Kurtis.

“So, you need my learning team to rebuild our preventative maintenance learning experiences while also creating a system for tracking and reporting return on investment related to learning?” asked Kathryn.

“Sort of,” said Kurtis. “I would like to limit the scope for the ROI question to just the preventative maintenance program. I am assuming that if you can figure it out for those learning experiences, you can build something that we can use for all learning experiences.”

Quantifying The Outcomes

“I’ll tell you again that this isn’t easy,” said Kathryn. “We can provide qualitative data points in the form of stories about people learning. We can show how many modules have been built. We can also show how many people took them and whether or not they passed the knowledge checks at an appropriate level. We can also survey learners to find out whether or not they liked the experience.”

“And all of those things are part of the story,” said Kurtis, “but I need data related to how behaviors have changed. What improved? By how much? What did those improvements do to our financial performance? And how much did it cost to build them? These are the questions we need you to answer.”

“Some learning solutions are more expensive than others,” said Kathryn. “We are considering getting into augmented and virtual reality in the coming months because we think these tools will give us better learning results than what we are getting now.”

“I only know a little about that technology,” said Kurtis, “but it is intriguing. But I can’t see us approving spending on new tools when we are not clear on the return on our current practices. I need to tell you a couple of things, but I need for them to stay between us.”

“Ok,” said Kathryn.

“We are looking at imposing a hiring freeze soon,” said Kurtis. “We are also looking at freezing wages and putting all performance bonus programs on hold for the time being. Things are serious. You’ve become a close friend over the years, so I am telling you this to make sure you understand the depth of the challenge. You will need to figure this out, and I will help wherever I can.”

“I appreciate your bluntness, and I am grateful for your friendship,” replied Kathryn. “I can promise you that rebuilding the preventative maintenance learning program and figuring out a more robust ROI system are now our top priorities. I also know you are having meetings like this every day so please let me know if I can be helpful or supportive of you.”

Kurtis thanked Kathryn and left her office. Kathryn sat quietly for a while, shaken by what she heard. Not one to be easily rattled, Kathryn wasn’t entirely sure what to do next. Her normal path would be to call in the members of her learning team and get their perspectives and insights. This did not feel like one of those moments.

Instead, she sent a quick email to Amy, a learning team consultant to AshCom, and asked for an hour of her time. The sooner the better.

Conclusion

Download the eBook The Learning Scorecard: How To Measure L&D Impact And Prove ROI to delve into the data and discover which key metrics your L&D team should consider. You can also join the webinar to discover a completely new approach to measuring ROI.

Dear Reader, if you would like to see a demo of MindSpring’s Learning Scorecard, please click here to schedule a time convenient for you and the learning experts at MindSpring will be happy to walk you through it.