Using data to make smarter learning decisions – part 1

The starting point for tackling data L&D’s challenges

Data is the hot topic for today’s learning leaders. Nine out of ten learning leaders are looking to data to help them make smarter decisions¹. Personalised, adaptive learning, artificial intelligence, and learning analytics topped Donald H Taylor’s Global sentiment survey² at the beginning of the year.

Fuelled by award-winning applications of AI³, and in-depth conversations about the user experience and demonstrating impact, the data trend has continued to build momentum throughout 2019.

However, whilst data might be the new black for L&D, not everyone is comfortable wearing it!

L&D’s data challenges

In the run-up to our joint webinar with Valamis on the 5th of December, we conducted a poll to explore the challenges learning professionals were experiencing in using data to make smarter decisions. L&D people from a wide range of industries and countries identified 109 challenges that they are facing with data as they move into 2020. Figure 1 shows that the challenges fall into five main categories:

^{Figure 1 L&D Data Challenges (n=107)}

A significant issue is the confidence of L&D to work with data, with 26% of challenges linked to either basic digital literacy or to more advanced data analytics expertise. The lack of data analytics skills was not a surprise, as it has also been widely reported by the Learning and Performance Institute⁴ and Towards Maturity⁵. But it was interesting that 17% of the confidence issues went right back to basics, asking fundamental questions about data principles, such as: What does data even mean for us? Where do we start? How do we add value with data?

Overall, the challenge of accessing data was the biggest concern—both collecting new, relevant data and accessing other data sources already in use. Technology, in theory, should promote access, but 10% of the issues raised were directly related to the tools, finding the right technology to capture and analyse data, getting those tools to work with other platforms, and, in some cases, L&D bemoaning the lack of actionable data from existing platforms.

The application of data to practical problems represented a challenge for 1 in 5 respondents, with 11% asking questions, such as how data we have can help us personalise learning, make smarter decisions, demonstrate impact and more. Nine percent expressed concern about the relevance of the available data to address the real challenges ahead.

Lack of support stems from the lack of buy-in from the wider business world which, in turn, contributes to the lack of resources (both time and money) to fully exploit the opportunity that data can provide to L&D. It was widely recognised that data from multiple sources is needed for L&D to be able to make smarter decisions that ultimately support business performance. Realising this requires a collaboration with business leaders and data experts, which is still a challenge.

The final concern raised within this sample was linked to the basic issue of trust. It was not only about the ethics and permissions involved in gathering useful data, but also the hidden assumptions—from both business leaders and learning leaders—about what good data looks like and the context in which is it collected.

Where to start?

As L&D leaders, we are surrounded by potential sources of data from our systems, our companies’ management information and external research. We can also surface data from the reflections of our learners and the questions we ask of them and our stakeholders. These data points are just individual information sources. As such, they can be meaningless when considered in isolation. They can be misleading, misused and sometimes downright dangerous when taken out of context. The sheer volume of data sources can be confusing and frustrating, but that is not our greatest challenge. Tom Davenport, author of the book Competing on Analytics: The New Science of Winning, says that “The biggest problem in the analysis process is having no idea what you are looking for in the data.”

Data only starts to make sense to us when we start to ask smart questions of ourselves and our services.

For all of these challenges, the best place to start is not with the data or developing a new set of data-analyst skills, but rather with our own curiosity. What do we want to prove, improve or even disprove? We don’t need a degree in statistical analysis to start asking questions. but we do need a curious and critical mind.

In the next blog, we explore how to put our curiosity to work by using evidence—the new “e” in learning!

Citations:

1. L&D’s Relationship with Data
2. GSS19 – Key Results Infographic
3. Travel company TUI won most innovative new learning product for their use of chatbots in Learning Technologies awards 2019
4. LPI capability map
5. Towards Maturity Transformation Journey 2019