What is lifelong machine learning and its differences with other learning paradigms

Lifelong learning

1. Definition

The concept of lifelong learning was proposed around 1995 by Thrun [1]. Since then, it has been perceived in various ways. So far, lifelong learning remains an emerging field, and the understanding of it is still limited, to be honest. The detailed interpretation and design of lifelong learning vary among different domains and tasks. But still, there is a consensus in the research community on how should lifelong learning be like. With the vision, we can define lifelong learning in an informal and general way.

Definition: Lifelong learning (LL) is a continuous learning process. With the utilization of past knowledge learned in previous tasks T1, T2, …, Tn (tasks can be from different domains), the learner is able to learn the current task (or called the new task) T(n+1) efficiently and effectively. Not only should the learner optimize the performance of the current task, but also it can optimize any task by treating the rest tasks as the previous tasks. After the current task is learned, the knowledge should be updated in a consistent manner.

2. Five characteristics of lifelong learning

The definition indicates five key characteristics of LL:

1. continuous learning process;
2. knowledge accumulation and maintenance in the knowledge base;
3. the ability to use the accumulated past knowledge to help future learning;
4. the ability to discover new tasks;
5. the ability to learn while working or to learn on the job.

There is no other learning type (such as transfer learning and multi-task learning) that can meet all the five characteristics above, which differentiate them and LL. Next, I will explain how different they are.

LL’s relation with other learning paradigms

Here I explain several most related learning paradigms with LL, including transfer learning, multi-task learning, online learning, and meta-learning.

1. Transfer learning

Transfer learning is currently a popular topic. It’s also called domain adaption in the natural language process. Briefly, transfer learning involves two domains: a source domain and a target domain. The source domain usually contains a large amount of labeled training data, while the target domain only has meager or no labeled training data. The goal of transfer learning is to leverage labeled data in the source domain to help to learn in the target domain.

Transfer learning shares a similarity with LL in the aspect of leveraging previous-learned knowledge to learn the new task. However, it has lots of distinctions from LL:

• Transfer learning is not a continuous process like LL. It transfers knowledge from the source domain to the target domain only one time only. It does not retain the transferred knowledge for future use. But for LL, knowledge retention and accumulation are essential.
• Transfer learning is unidirectional. The knowledge can only be transferred from the source domain to the target domain. But in LL, the learning result from the new task can be used to improve learning in previous tasks.
• Transfer learning requires the source domain and the target domain are very similar. Otherwise, domain mismatching can be detrimental to the performance. But in LL, all tasks can be from different domains. When solving the new task, the learner should determine which previous task is appropriate for the new learning task.
• Transfer learning does not identify new tasks to be learned during the model application or learn on the job.

2. Multi-task learning

Multi-task learning learns multiple related tasks at the same time, aiming at achieving a better performance by using the relevant information shared by multiple tasks. Also, multi-task learning is helpful to prevent the problem of overfitting an individual task.

Multi-task learning is able to learn different tasks just like LL does. But it has many limitations compared to LL:

• Multi-task learning is still working in the traditional paradigm. Instead of optimizing a single task, it optimizes several tasks simultaneously. But if we view multiple tasks as one bigger task, it reduces to the traditional optimization in isolated learning where knowledge learned in previous tasks is not retained. It has no concept of continuous learning and can not handle tasks sequentially.
• Unlike LL, multi-task learning generally assumes that tasks are closely related. When some unrelated tasks are considered, the performance can deteriorate.

3. Online learning

Online learning is a learning paradigm where the training data points arrive in sequential order. When a new data point arrives, the existing model has quickly updated to produce the best model so far. Its goal is the same as classic learning, i.e., to optimize the performance of the given learning task.

Although online learning deals with future data in steaming or sequential order, its objective is very different from LL. Online learning still performs the same learning task over time, just with streaming data. Its objective is to learn more efficiently with the data arriving incrementally. LL, on the other hand, aims to learn from a sequence of different tasks, retain the knowledge learned so far, and use the knowledge to help future learning.

4. Meta-learning

Meta-learning primarily aims to learn a new task with only a small number of training examples (few-shot samples) using a model that has been trained on many other very similar tasks. It trains a meta-model based on a large set of tasks to quickly adapt to a new similar task.

One assumption of meta-learning is that the training tasks and the test/new tasks are from the same distribution (usually, they are sampled in different ratios from the same dataset). Actually, this assumption makes a major weakness and limitation. In general, LL doesn’t make the assumption. A lifelong learner is supposed to choose the pieces of previous knowledge that are applicable to the new task.

More detailed introductions about lifelong learning can be found in [2].

References

1. Life Long robot learning by Thrun, 1995.
2. Lifelong machine learning by Zhiyuan Chen and Bing Liu.