• Employee Development
  • Digital Workplace
  • February 22, 2018

Diving into the Learning Experience – Machine Learning, AI and Chatbots

In my previous blog post about personalized learning, there was a case on how technology could help make learning more effective in an organizational context. As a short recap, I spoke about the digitalization of an employee's learning and experiences, which makes it possible to process and analyze those using machines and computational algorithms.

In this new post, I would like to take the next step, and show what approach we use to enable personalization within Valamis – Learning Experience Platform (LXP). But before I move to the main point, it is important to understand the context in which Valamis – LXP is operating.

LXP uses Experience API (xAPI) -method, which was previously also known as TinCan API. Using xAPI, detailed information about an employee's learning activities is gathered in the form of simple records. When those records are analyzed, they show not only the end results of learning, e.g. achieving a certificate or attending the training, but it also reveals the process in which the learner got to the end result. Which means, for example, you can see detailed information about what questions had been answered correctly and incorrectly, where employees spent more time, did they fast-forward or rewind the video content.

Optimize learning for employees

Learning data, gathered in the form of xAPI statements, is valuable on its own, and once that data is visually displayed by graphs and reports it could help in making decisions to better adjust the learning process to make it more effective. With this in place, Learning & Development (L&D) specialists can see where to put their efforts to get the most value - what type of content works better, what groups of learners need more support, what materials do not produce desired outcomes and where quiz passing rates are too low, showing that material needs to be reworked or a different learning path should be used.

So, data regarding a learner's activities and experiences in the form of xAPI statements helps people in the L&D department to optimize the learning process for an individual employee. Though there are some inefficiencies here: information needs to be analyzed by people, results of that analysis are applied by people, and, what is more important, is that it doesn't take into consideration if the optimization of an individual's learning has a positive impact on a company's business metrics.

Acknowledge training possibilities for each employee

In organizations, learning rarely happens without a reason. Most likely, it is addressing some issues and needs, small or big; relating for example to an employee's role, tasks at hand, a future promotion, changes in the organization or its business model, new technology or changes in a competitor's field.

Employee's behavior can be changed as a result of learning activities, and this can be observed from multiple sources. For example, after implementing a learning path about stress management and mindfulness techniques, days taken as sick leave for the employee might be reduced. Most likely, data about sick leave is found in a company's HR system, and when combined with data about employee experiences and learning activities, this can be an indicator of positive impact, that the learning activities are affecting employees' behavior.

Usually, in big organizations, there are plenty of untapped systems gathering data on employees. Linking that data to learning activities can reveal interesting cause-effect relationships. Not only do HR systems contain such data, but, also CRMs, where all customer-related activities and events are marked. It could be a tracking system, where support requests are marked, assigned and resolved. It could be a version control system, where individual changes for software components and configurations are submitted by developers or system administrators. It could be even a system, which collects information from sensors or other Internet-of-Things (IoT) devices.

the image illustrates a variety of untapped systems gathering data on employees

All these examples above show places where data about employee's actions could be. If that data is processed and analyzed, it could also show changes in employees' behavior over time. Most often, those changes will be related to some sort of learning happened.

Organization's performance is dependent on employee behavior

Most business metrics are presented in a way that is relative to the company's bottom line, like turnover, income, stock price etc., or non-monetary, like production volumes, customer satisfaction, employee retention etc.

An organization's strategy usually has some measurable metrics to track, and an end goal in mind. The data used for these metrics can be found in financial systems, HR systems or CRM systems. There is a possibility to see how those numbers are changing over time and whether they change in the desired direction at the required rate. But what makes those metrics change? What drives them? Arguably, it is always a result of many different people combining efforts.

As an example, let's imagine that a company identified the need to improve customer satisfaction. When customer satisfaction is low and the organizational goal is to improve that number, there are plenty of actions that can be taken to address that. And the people, employees of the organization, are the ones who do something to make this business metric improve.

Improvement in a metric can be tracked in one of the aforementioned systems, for example, in a CRM. Employee behavior has changed; actions they did were different than before, and those changes led to an improvement in the business metric. Although you can track the progress of business metrics, you don't have the ability with a CRM to track data that led to the changes.

So, is it possible to know what caused the change in behavior? Is it possible to know what is working and what isn't working? What should be avoided to prevent the waste of valuable resources?

Learning Analytics help untangle the mystery

Answers to the questions above can be found with help of analytics. Data from different systems, examples of which were mentioned above, can be processed, combined and analyzed together. As a result, it enables us to get insights on what causal relationships exist between changes in business metrics, shown in one system, and the changes in people's behavior, which is observed from a different system.

Even if it sounds simple, identifying such dependencies is quite a complex task. However, modern analytical tools make this goal more than attainable. Machines can process a huge amount of data, and uncover patterns and dependencies... Machine learning can find what employee behavior changes led to a positive effect and what didn't. At the same time, when data on learning activities and changes in behavior are analyzed together, it can show what learning activities bring value, e.g. what caused a change in employee behavior, and what didn't result in the desired outcomes.

This picture illustrates well what had been just presented:

relationships between learning activities and changes in employee's actions
Source: A.D. Detrick, 2016

So, when data from different systems has been processed and analyzed, and the relationships between learning activities and changes in employee's actions are defined, as well as those between actions and changes in business metric had been identified, this model could be then used for optimization of learning activities for improving business outcomes. This is where machine learning comes in. When analytical models are built to reflect causal relationships and then identified, the real data from existing systems is used to "train" those models, helping to recognize similar relationships in future data sets.

Machine Learning and Artificial Intelligence

Recently there has been a lot of hype surrounding ML & AI, including ideas about machines taking over the world. I think this occurs because people are scared of what they cannot understand. This is very natural, because for a big percent of the population, what machines are doing is just too complex. Machines much better at processing data, and people are scared that if they can't be better at something, then they are losing. But they are not. We cannot compete with machines when it comes to routine tasks, but machines cannot beat us in creativity.

Machines can only do what they are built to do. Machines can "learn" from data, but that means machines can identify patterns in data and then apply those patterns to the new data according to models they are programmed with. That is just a mathematical algorithm made by people. A human can't process the same amount of data that a machine can. But the same goes for the building of a skyscraper - humans can't do that without the help of machines and mechanisms, it is just not physically possible with only one brain and one person's muscle power.

So, AI in a sense is just math - there are algorithms that identify patterns in data, and then based on those, decisions can be made. A very simplified example of AI and machine learning could be that, if you hear the words "apple tree," in a song 100 times then you can begin to predict that you will hear the word "tree" after you hear "apple" in the continuation of the song. Additionally, if you hear 1000 different people saying the word "tree" with a slightly different accent, then you can identify this same word when a new person says it with a different pronunciation. And that is exactly what AI does.

How can AI help in organizational learning?

Now coming back to our main topic, how can all of this make learning in an organization more effective with the help of AI. AI in organizational learning can be applied in many different scenarios. Some examples of those are:

  • Interpretation and classification of learning materials (video to text, image classification, deep text understanding and concept classification),
  • Personalized recommendations (based on similarities with other learners, by linking learning to impact on company KPI, by building user's own profile and preferences).

Traditionally L&D professionals are in charge of the creation and curation of materials for learners, as well as recommending or assigning those materials to target users and user groups. But what if the routine work required to follow a learner's progress and distribute relevant materials can be done by a machine?

With Valamis, now you can. We decided to automate this process with the help of AI so that the "delivery" of recommended materials is made through a chatbot interface. Chatbots are an easy way for people to interact with the system because it has a simple and familiar interface people are used to using in their daily life. To begin a conversation with a chatbot, people don't need to learn a new concept, because it's similar to many texting and messaging applications. So, ValBo was born.

The Chatbot in Valamis: what is it for?

As Gartner says in their report on use cases for chatbots:

"When considering potential use cases for chatbots in the enterprise, application leaders should first assess the impact on resources. There are two options here: replacement or empowerment. Replacement is clearly easier as you don't need to consider integration with existing processes and you can build from scratch. Empowerment enhances an existing process by making it more flexible, more accommodating, more accessible and more simple for users." – Gartner, Four Use Cases for Chatbots in the Enterprise Now, Van L. Baker, Magnus Revang, 16 February 2017

What does ValBo mean for L&D?

The goal driving the creation of ValBo is to be a 24-7 assistant for a learner. At-the-ready all hours of the day in order to help the learner find what is relevant to his current context or task at hand when the learner asks. ValBo can also provide proactive recommendations for the learner, when it notices that - for example - some learning materials are unfinished. Or, for example, it can give onboarding materials for new employees or showing relevant "beginner's guide" -material when a person has been promoted, or their role has changed in the HR system. L&D's role here is to curate learning content and then adjust ValBo's recommendations using examples or training data.

In Valamis, learning content is classified and indexed with the help of IBM Watson, so ValBo can better find the relevant materials for the learner's needs.

Communications with ValBo are collected and analyzed, and are used for better understanding the learners' needs.

By having learning content classified and indexed and learners' needs understood, quality recommendations can be made and delivered through ValBo to the end user. To improve the quality of recommendations, learners' activities and previous recommendations are processed through machine learning algorithms. Think of Amazon's recommendations, like, people, who bought this item, also bought those ones.

At the same time, by building each learner's profile, you can then better understand things like: what form of learning materials he prefers, what time of day he is willing to learn the most, does he prefer bigger chunks of learning content or does he prefer microlearning. This further improves personalization of recommendations made.

Valamis also can make a connection between learning activities in Valamis and other systems so that it can identify relationships between learning activities and the positive impact it has on a user's actions which lead to a positive outcome for company's business KPIs. In this way, recommendations from ValBo will be made even more in line with company's key interests.


Information in the form of xAPI statements regarding employee's learning activities is a good source of insights for the L&D department, demonstrating what is working and what is not. When those are visually interpreted, they can show which learning materials can bring the most value.

Still, optimization at that level doesn't take into account if the "needed" learning is happening and if the learning brings about desired changes in employees' behavior.

For linking an individual's learning with organizational goals, there is a need to combine and analyze data from many sources, and this is what could be done with help of modern analytical tools.

In Valamis LXP, the role of a personal advisor is given to ValBo, our chatbot. It's also a tool for the L&D executives in the organization, for it offloads routine work related to tracking a learner's progress and optimizing recommendations based on organization's goals.

Digital Transformation of the Workforce

Learn insights about the benefits of proactive retraining and reskilling, and how you can better understand the economics of learning.