Learning analytics

Updated

After reading this guide, you will have a more complete understanding of learning analytics, and how using it can both improve outcomes for learners and increase the efficacy of your organization’s training programs.

Discover:

What is learning analytics?

Learning analytics is the collection, analysis, and reporting of data about learners and their interactions with their learning environment, for the purpose of better understanding learning, optimizing learning environments, and improving learning outcomes.

The term “learning analytics” wasn’t widely used before 2011. It wasn’t until it appeared at the 1st International Conference on Learning and Analytics that it became a more common term.

The need to track the quality of digital learning appeared long ago. Nowadays, the teacher doesn’t need to sit in front of the learner but the need to check his or her knowledge and performance still exists.

The general level of interest in analytics is growing rapidly, especially in large companies. It’s hard to imagine a global company that doesn’t use analytical approaches in their business. It’s a popular choice to have in-house or external Learning and Development (L&D) teams to design and form a training strategy based on company goals and employee competencies.

There is a high demand across industries for learning analytics, which will be used to solve a wide range of business problems, including personnel training, measuring its effectiveness, productivity, and planning development strategies.

Why is learning analytics important?

The work environment is undergoing great changes. The modern workplace now represents social collaboration and growing technological trends.

New tools and technologies have completely changed the way we work: organizational flexibility, digitalization, cloud technologies, work messengers, online meetings, data intelligence, etc.

The Internet, gadgets and mobility has allowed us to work anywhere and ease the cooperation with people on the other side of the world.

These changes provide an opportunity to improve productivity in training programs, the quality of knowledge of a particular employee, and ultimately speed up production work. L&D professionals increasingly use learning analytics and automation to identify the weak points of learning, patterns and potential of employees.

Using analytics will give companies a competitive advantage. Organizations that are willing to invest in analytics will have access to data that will help them improve employee performance and to make better decisions. Thus, it will put them at the forefront of employee learning in the shortest possible time with the best results.

Learning analytics helps organizations develop better learning programs

Training programs are becoming more important. When we need to continually improve the skills of our employees, millions of dollars are wasted every year on ineffective education.

Without learning analytics, an organization has no way to understand the efficacy of its learning programs, if there are learners who need assistance, nor will they be able to know in which ways they are succeeding or failing in their training programs.

There is a high chance that they will continually make the same mistakes - simply because they don’t know that they are making them in the first place.

It can also provide content creators with the necessary insights to help them create better learning paths for their learners. For example, if we see students struggle with one part of the learning path that causes them to later leave the course, we may decide to exclude this part or recreate it.

We may also observe that learners complete the program in a completely different way, not as suggested in our learning path.

Research of the learning experience can be beneficial for a company and for the learner. Qualitative training programs make your employees more productive and engaged.

Learning analytics can help engage employees

Organizations with better training programs have more engaged employees. They are more productive, stay with the company longer, and lead to better business outcomes. All of which are goals for every organization, regardless of industry.

Learning analytics can be used to build advanced learning paths. E.g., using recommendation engines, learners in their environment will receive the most relevant content recommendations in accordance with their learning history. This will help them continue learning without the need for searching. If the platform has enough courses which meet the users’ requirements then a good search engine will keep them engaged.

Things like search terms and recommendation feed clicks can help content creators understand what needs to be added and what needs to be changed.

Learning analytics help erase skills gaps

Learning analytics can also help organizations develop consistency within their training. Skill gaps can cause problems within teams, and with a system that is able to identify and correct them, an organization is able to nip that problem right in the bud and solve issues before they begin.

By analyzing the employee’s skill matrix we can figure out what we are missing and create more content to support this exact skill and assign people to this content to cover all pain points in order to create a competent team.

Learning analytics can inform organizations on the most effective way to deploy resources

Perhaps most importantly, learning analytics is the most powerful tool an organization can use to understand the most effective way to deploy limited educational resources.

In a perfect world, training budgets would be unlimited, as would the amount of time we could give learners to engage with it. But more and more, the time and budget available are shrinking.

Learning analytics helps pinpoint the most effective learning strategies, saving organizations time and money while delivering great results.

The applications of learning analytics

Learning analytics is most often used in four main areas:

1. To measure key indicators of learner performance

Being able to track the performance of employees as they engage with your training program means that you can pinpoint areas where more help is needed.

Skills gaps can be swiftly closed, and additional assistance can be deployed quickly to solve problems.

2. Better support of learner development

When an organization is effectively using learning analytics, they are better able to provide the resources that a learner needs, rather than having to blindly guess as to what will be effective training materials.

This saves time and helps the learner improve in the areas that they truly require more training. It also illustrates to the learner that their development is being supported by the organization.

3. More able to understand the effectiveness of current practices and pinpoint where improvement is needed

Learning analytics can shine a spotlight on the areas where a training program is lacking. If many students are taking an online course but not passing the exam at the end, your organization can see that information and know that there is a need to change the material. 

4. Allows organizations to be more agile with training decisions and strategy

Using learning analytics means that your organization can much more effectively judge the efficacy of training programs, make adjustments where needed, put money where it will be most useful, and judge the overall direction that the training program is headed.

What are the learning analytic methods?

Within learning analytics, there are four main methods of analyzing the data collected from learners. We’ll give a brief overview here, and you can consult the linked articles for a more in-depth guide to these concepts.

Learning analytic methods

Descriptive analytics

In our thorough guide to descriptive analytics, we summarise it as follows: Descriptive analytics is a statistical method that is used to search and summarize historical data in order to identify patterns or meaning.

The data generated by learner interactions with the learning environment can be used to understand how, in the past, learners have behaved. This can help inform an organization how effective their current system is.

Descriptive analytics, however, does not help inform an organization about a user’s future behavior.

Diagnostic analytics

Where descriptive analytics tells you how something happened in the past, diagnostic analytics tells you why it happened.

Using processes like data mining, data discovery, drill down, and drill through, diagnostic analytics uses data to highlight the causes of behaviors and events.

It can be used to identify anomalies, helping organizations pinpoint areas that require further inquiry. It also uncovers causal relationships, showing how certain events might have resulted in the anomalies identified above.

Predictive analytics

Our article to predictive analytics defines it as ‘a statistical method that utilizes algorithms and machine learning to identify trends in data and predict future behaviors.’ Where descriptive and diagnostic analytics is concerned with the past, predictive is all about the future.

By using predictive analytics, an organization can identify both risks and opportunities, taking action to improve learning outcomes.

Using models such as decision trees, neural networks, and regression techniques, algorithms, and machine learning businesses can analyze data, both past, and present, in order to predict future trends.

Prescriptive analytics

According to our in-depth article, prescriptive analytics is ‘a statistical method used to generate recommendations and make decisions based on the computational findings of algorithmic models.’

Considered an extension of predictive analytics, prescriptive analytics is not used as much due to the complexity of the machine learning it needs to function. However, it can be found within some learning experience platforms.

A platform using prescriptive analytics might be able to pinpoint areas of struggle for a learner, prompting the delivery of content tailored to improve the skills that the learner is most likely to need to master in order to fully grasp the content.

Janne Hietala, Chief Visionary Officer at Valamis, explains how utilizing learning data can support learning and help drive better learning results.

Ethics and privacy in learning analytics

Of course, as with any gathering of data, organizations must be aware of potential issues regarding the ethical use and storage of data, and the privacy implications of the data gathering.

There are five general areas of ethical issues:

  1. Management, security, and the privacy of the stored data
  2. Data discovery based on the anonymized sources of information
  3. Logging of the information about data source access
  4. Accuracy and completeness of data
  5. Responsibility and obligation taking actions on knowledge

Organizations, when creating the road map for their learning analytics implementation, need to consider each of these areas.

There must be discussions about how each issue will be approached, what potential problems might be encountered, and what the most logical response to those problems would be.

The International Council for Open and Distance Education released a report in 2019 covering many of these issues, which may serve as a helpful guideline for organizations looking to make sure that they are operating ethically.

The report touches on key issues of:

  • Data ownership and control,
  • Transparency,
  • Accessibility of data,
  • Validity and reliability of data,
  • Institutional responsibility and obligation to act,
  • Communications,
  • Cultural values,
  • Inclusion,
  • Consent,
  • Student agency and responsibility.

Each organization using learning analytics should pay close attention to these areas, develop guidelines and ensure that standard operating practices are up to the standards proposed. It is wise to bring in experts in this field, both operational and legal, to advise your organization on your obligations in your country.

Challenges of Learning Analytics

As with any new technology, learning analytics brings unique problems that must be solved to get the most out of it.

1. Understanding the problem

First and foremost, you should understand the problem that you are seeking to solve by using learning analytics.

Without knowing what you want to do with it, even the most advanced learning analytic program will not be able to help you.

There are, of course, limitations to what the data can help you with - not all learning happens within a digital environment. By knowing which data will help understand the problem, you will be better able to solve it.

2. Recognising what must be built

As a relatively new field, learning analytics will require organizations to be active builders of their learning analytics programs.

Each company will have different digital environments and needs, each of which will require a different solution. This is an area where many trials and evaluations may be needed, to adjust the program as it develops.

Content that can be created by external vendors should be aligned with the same learning data guidelines.

3. Understanding for whom analytics is needed

As an organization develops its program, there are many questions to answer:

  • For whom are they developing this program?
  • Will this be used solely for training purposes with new employees? Or is it for lifetime learning within the organization?
  • Who will receive this data and take action?
  • Will there be dedicated roles within the company or will this be a task for departmental heads and managers?

All of these answers will inform how the learning analytics program should be created and managed.

4. Setting aside enough time to develop a proper program

Not only will the first version of the learning analytics program take a lot of time to develop, but an organization should also plan for multiple iterations of the program to be developed and implemented.

As the program becomes active, there will be faults and flaws uncovered that will need adjusting. It’s not a one-and-done job.

5. Handling vast amounts of data

Data comes in a massive variety of formats, types, and locations.

Many organizations have found it a challenge to create a system that can handle the demands of analyzing such a massive amount of disparate information.

Performance issues might be rampant, especially when there are significant amounts of learners being tracked.

6. Creating programs that match expectations to technical capabilities

Learning analytics as a field has opened up many exciting doors. However, as a young field, there is a lot of speculation about what it can do, rather than what it actually does.

Within an organization, there might be an idea that learning analytics can revolutionize a training program or completely change the way that customer behavior is understood.

While both of these things might be possible, it can also be that there is a limit to what learning analytics can truly do, and that can lead to issues.

7. Security

Finally, security is a massive challenge within this field.

Handling this amount of data will require there to be an equal amount of security amongst the storage of, and access to, information.

An organization should be very thoughtful about creating an environment that ensures the safety and privacy of all who access it.

The organization should build the level of security to limit and separate the rights according to roles and permissions to be compliant with EU GDPR and similar privacy laws.

About The Author

Anna Khokhlova, Business Analyst Consultant at Valamis

Anna Khokhlova

Business Analyst Consultant

Anna Khokhlova is an experienced Business Analyst Consultant working with learning data and big data analysis. Anna graduated from Petrozavodsk State University in Russia, and has a Masters degree in Business informatics. In order to better visualize complex data sets and interdependencies, Anna creates intuitive and easy-to-understand dashboards and reports. She is responsible for creating technical solutions for analytics in Valamis LXP.