Additional Deep Learning Topics: Transfer Learning, Model Compression, and Anomaly Detection

This English version was machine-translated from the Chinese original. Technical terms may require verification.

Migration for study

The high cost of labelling data for many missions in practical application and the unavailability of adequate training data can be used for migration learning. Assuming A and B are two related tasks, A has a lot of training data that can move some of the knowledge learned from A to B.

The questions in this article can also be addressedIn-depth learning base: neuronet, optimization approach and integration、In-depth learning network architecture: CNN, RNN and Seq2SeqHow the concept of a relatively close read together is developed in different contexts.

Field Offset

So far, we have learned a lot of models for in-depth learning, so training a sort of sorter is not so hard for the moment. For example, the handwritten digital recognition problem on the classic MINIST data set. However, testing data and training data may lead to problems when they are distributed differently. It is assumed that the numbers were black and white at the time of training, but the numbers were coloured at the time of testing and the correct rate would be very low. This problem is referred to as the field shift.

There are many different types of field deviation, and the situation of changing data distribution of model input is one type. Another type is that the distribution of outputs may also change. For example, on training data, the probability of each number being present is probably the same, but on test data, the probability of each number being exported may be different, and it may be that a number is particularly likely to be exported, and that is possible. Another relatively rare type of situation is that the relationship between input and output, although it may be the same distribution, has changed. For example, the same picture in training data is marked "0", but in testing data is labelled "1.

And then we focus on the different areas of input data. Next we call the test data from the target area (target domain) and the training data from the source area (source domain). Learning on baseline data sets often ignores the problem of field deviation, assuming that training data are often distributed in the same way as test data. But when applied in practice, when training data differ slightly from test data, machines may perform poorly and need to adapt to improve machine performance.

For the area of self-adaptation, training data is one area, and testing data is another area where information learned in one area is used in another, where adaptation focuses on addressing the problem of the consistent but inconsistent distribution of characteristics in the characteristic and category.

Area adaptation

The following is an example of adaptation in the field, using handwritten digital recognition. For example, there is a body of tagged training data from the source area, which is used to train a model that can be used in different fields. In training, we must have some knowledge of the target areas where the data are tested.

As the level of understanding varies, so do the adaptation methods in the field. If there is a large body of labeled data in the target area, this does not really require field adaptation and direct training in data for the target area. If there is a small amount of labeled data in the target area, this situation can be fine-tuned using a small amount of data in the target area, which can be adapted to the area.

In this case, the problem that needs to be noted is that because the data volume of the target area is very small, care should be taken not to overcompar and not to overstate the data in the target area.

This situation, in which there is a large amount of unmarked data on the target areas, is described below. This is in fact a very realistic situation. The most basic idea is to train a feature extractor. Disparities between source and target areas are removed and distributed equally, so that the area can be adapted. This is generally done in a field-based, learning-restraint manner.

Area Panorama

The lack of knowledge of the target area is not intended to be adapted to a particular area, which is often referred to as a broad area. The broadness of the field can be divided into two different situations.

In one case, training data is very rich and covers a variety of areas, and only one field is used for testing data. At this point, we look forward to seeing how the model can bridge the differences between areas, as training data are multi-aread.

Another scenario is that training data are only in one area, while test data are in a variety of areas. Although there is only one area of data, data can be generated in multiple areas using enhanced data methods

Basic migration learning methods in in-depth learning

The most basic method of migration learning in in-depth learning is the sharing of parameters, and we train models in areas A where more data are available, downloading and transmitting some of the parameters to another model. The model addresses similar problems in the original domain and is partly structured in a similar way. For example, the parameters in the first cavities of shared contours of neural networks generally yield better results than random initialization.

This can also be called pre-training and oversight fine-tuning, similar to but different in the LLM area. This pre-training needs to be carried out entirely on the same data sets as the type, ensuring similarity of parameters. We replace the pre-training steps by downloading other people's parameters.

Lifelong learning

Catastrophe oblivion.

We'll train the machine to do a voice recognition, then teach it to do a visual recognition of mission two, and then we'll do a speech recognition. Teach it to translate the third assignment, so that it learns all three tasks together. We continue to teach machines new skills, and when they learn hundreds of skills, it becomes so powerful that humans cannot reach it, which is what we call lifelong learning (Lifelong Learning, LLL). It can also be called incremental learning.

What's the point of this learning? Incremental learning is very relevant to models that need to be supplemented by more data to go online to support the next wave of training, which is certainly costly if they start from scratch every time.

What's the hard part of lifelong learning? It appears that lifelong learning can be achieved by updating its data and parameters of its counterpart network, but in practice it is not as easy. For example, we want to train two different areas of image recognition missions, we train a simpler network to do the first hand-written digital recognition mission, and then we do mission two, with an accuracy of 90 per cent for mission one, and at this point, even without mission two, 96 per cent for mission two, which is a very good move. So we're going to use the same model, the model that we trained in mission one, and then we're going to train mission two, and we're going to find that the accuracy rate on mission two is better (97 per cent). But the worse thing is that the machine has forgotten how to do mission one, which is to drop the accuracy rate from 90% to 80% on mission one.

It may be too simple to think of a network setting and then lead to a phenomenon, but actually when we put data on mission one and task two together to get the network to study together, we find that machines can learn both tasks at the same time, which means that two separate data sets have a significant impact on the final results of the model because they are not properly disrupted. The proliferation of new and very close mandates continues to lead to the model being extremely forgotten about the tasks that have been learned.

This is what it's called.Catastropic forgettingI'm sorry. We know that even human beings may be forgotten when, here, we add the adjective of catastrophicity to the oblivion, in order to emphasize that this oblivion of models is not a general oblivion, but a particularly serious oblivion.

Lifelong learning assessment methodology

Now we can look at some of the criteria for judging the poor performance of lifelong learning techniques. Before lifelong learning, a series of tasks, usually simpler, are required to allow models to be studied.

The specific assessment is as follows, with a line of tasks first and a randomly initialized parameter, which is used for this T task to obtain the corresponding accuracy rate. Then the model learns the first task and then takes the accuracy rate separately for all the missions, which is, $R_{1,1}, ..., R_{1,T}$And so on. Until all tasks are completed, an accurate rate table is obtained to assess the results of lifelong learning.

Specific final accuracy factor formula $$\frac{1}{T}\sum_{i=1}^{T}R_{T,i}$$ Another way of assessing is to reverse migration, which is $$\frac{1}{T-1}\sum_{i=1}^{T-1}R_{T,i}-R_{T,i}$$

The main solution to lifelong learning

Addressing lifelong learning, which is primarily a matter of catastrophic oblivion, is usually the subject of several major research lines in academia. We will first begin with the first main solution, namely selective synaptic plasticity. By definition, only the connections between certain neurons in the nervous network are plasticized, and the rest must be solidified, and this approach is called a pro-regular approach.

In this way, the basic idea is that each parameter is of different importance to the tasks we have learned in the past, so when we learn about new mandates, we try not to move those that are important to the past, but to learn some other parameters that are more important to the new.

This method will give a coefficient to each parameter.$b_i$To show that he is not really important for the task of the past, we now have the correction of the punitive parameters in the new loss function and the degree of correction given to the punishment by using the degree of importance just given. This will contain the catastrophic forgotten problems of radical changes to all the original parameters.

And the next thing that matters is... $b_i$How do you set it? It's a certain parameter.$b_i$What is the importance of the mandate? There is a simple control of the variable, which is to move or change a parameter, which means that if the loss is largely not changed, it is variable in a certain range and the corresponding materiality parameter. $b_i$It's small, and the parameters are bigger.

There is, in fact, another type of method, known as the Gradient Round Memory, MMD, which is not limited by parameters but by the direction of the gradient update, and is therefore also called the Gradient-based Method. It calculates the direction of the current mission gradient, and then goes back to the direction of the historical mission, and then it takes two gradients to sum up and arrives at the actual direction of the gradient, so that continuous updating can be as close as possible to the most elegant solution that will not fall into catastrophic oblivion. The new gradient will, of course, require conditions greater than or equal to zero, otherwise it will be difficult to optimize in the best direction. Since this is just a small amount of information stored on the gradient, we can also accept it as a lifelong learning method.

Network compression

Network compression is an important direction, and a model like Bert or GPT is so big that it can be scaled down to a smaller amount of parameters, but it's actually similar to what it was, and that's what the network compression was trying to do. The main purpose of the network compression is to allow us to operate large models on the end side without relying on cloud servers.

Network Clitters

The first technology is network cutting. The network cut is about cutting out some of the parameters inside the network. And the former tree model cut the branches is one idea, completely removing the parameters and branches that have little meaning for the model.

First, train a big network. The next step is to measure the importance of each parameter or neurons in this large network, and to assess whether any parameters are not doing anything or whether any neurons are doing nothing.

The simplest way to assess the importance of parameters is to see their absolute value. If the absolute value of this parameter is greater, it may have greater impact on the entire network. Or if it's almost zero. Perhaps the less the impact on the entire network, the less impact on our mission.

The method used to assess neuron importance is to calculate the number of times this neuron output is not zero, and if it is often zero, it means that neurons are less important. We have a lot of ways to measure importance, and here we are just talking about one idea.

Cutting down the non-important neuron or non-important parameters, moving it out of the model, and getting a smaller network. But when this is done, usually the model's performance falls. Now, this smaller network can be fine-tuned by re-engineering the remaining uncut parameters. Get the training data out and retrain this smaller network. This step is a constant repetition, repeated time and again until we achieve a balance in performance and size.

In the experiment, if a large number of parameters were cut off at a time, the harm to the network could be too great and could be too big to recover with fine-tuning. So the amount of parameters removed at a time is not too much.

The units can be either in parameters or neurons, and the units as such will be significantly different in terms of achievement. When we trim the parameters, the shape of the network that we're getting later on may be irregular. This is very bad for Pytotch ' s calculations and can lead to a failure to properly accelerate calls. In practice, therefore, parametric trimmeds often turn the parameters of the corresponding position to zero, which does not allow for a direct reduction of the model. Existing papers prove that the parametric trimulations are not in most cases suitable for acceleration and are not practical.

Neurocuts, i.e., in neurons, are probably a more effective method of cutting. In fact, neurons are also understood to be re-useable in another network with existing large network parameters, and thus relatively free. When you do it, you can just change the dimension of each layer that you enter, output, and then you can use GPU to accelerate.

Why do you study big networks to cut small networks, instead of training a small network, and experimenting to prove that we can train a big network first, then reduce it, and not lose much of the right rate. But the less trained network does not have the same correct rate as the larger network cut down and smaller. There is no clear theory to explain this. The lottery hypothesis was a possible idea, and he thought that the big network was essentially a collection of small networks, and that cutting was a way of getting that good network, and that it would be better than training small networks directly.

Distillation of knowledge

The next step is to make the network smaller - knowledge distillation. Training a large network, known as the teacher network, in the distillation of knowledge. We're training on a really small network, the student network. Training a large network called the Teachers ' Network first. And the big network that makes the student network.

In the network, we trim the big network, we take out some parameters from the big network, and we turn it into a small network. The knowledge distillation is different, and this small network (the Student Network) is learning from the teacher network.

Assuming that handwritten digital recognition is done, training data are thrown into the teacher, and teachers produce output, because it is a classification issue, so the teacher output is actually a distribution. For example, the teacher's output may be that the score for this picture 1 is 0.7,7 and the score for the number is 0.2,9 and the score for the number is 0.1. And then you give the students the exact same picture, but instead of looking at the correct answer to the picture, it takes the teacher's output as the correct answer, which is 0.7, 7, 0.2, 9 and 1.

The thinking of distilling knowledge and the core thinking of cutting the front web are close, and they all think it's good to directly train a small network without a large network. Why would distilling knowledge help? A more intuitive explanation is that the teacher network provides additional information on the student network, which in fact complements its learning effects.

The teacher network does not have to be a single large network, but it can even be an integration of multiple networks, training multiple models, output multiple models, and the voting results are over. Or the result of the output of multiple models is considered the final answer.

Although in competitions, integration is often used. But in practical terms, integrated, computing is too much, so we need to study the distillation of knowledge.

There is a small technique in using knowledge distillation. This small technique is to modify the Softmax function slightly, adding a temperature (temperature) to the Softmax function. Original Softmax is $$y_i^{\prime}=\frac{\exp{(y_i)}}{\sum_j\exp{(y_j)}}$$ It's increasing the temperature.$T$The Softmax is $$y_i^{\prime}=\frac{\exp{(y_i/T)}}{\sum_j\exp{(y_j/T)}}$$ ♪ of which the temperature ♪$T$It's a super-parameter, temperature. $T$ The effect is to make the distribution, which is otherwise more concentrated, more smooth. When the TT>At $1 a million, such a Softmax would spread the original concentration and keep the same size and size, so that the network of teachers could provide extra information to the student network, thereby helping to learn.

It's too hot to change the model. Assuming the temperature is close to infinite, so all categories are given the same score, and the student network is not learning anything, so...$T$ And another super-parameter, which is the same as the learning rate.

Quantification of parameters

Next, you can describe the next technique: quantifying parameters. The quantification of parameters is to say whether only a relatively small amount of space can be used to store one parameter.

For example, it may be 64 or 32 bits for now to save a parameter. It might not be necessary to use this high precision, enough with 16 or 8 bits. So the simplest way to quantify parameters is to start with a value of 16 places, if, for example, we were to save a value when we were in a network, and now we're saving a value of eight places. Storage space, the size of the network becomes directly half the size of the original, and performance is not much lost, even sometimes with the accuracy of storage parameters getting lower, resulting in a slightly better outcome. In general, we consider using higher precision in training and quantifying parameters in reasoning, thereby reducing storage space.

There is also a method of further compression of parameters, namely, weight clustering. For example, group the parameters of the network, which are grouped according to the value of this parameter. The values are close to a group, and the number of groups to be divided is set in advance, and each group will have only one value to indicate it. So all you have to do is record which group he belongs to. Of course, this method does not help to optimize computing performance.

Regrouping can be considered directly during training, so that models can learn more closely about parameters. The value of the general weight re-grouping class is the average of all the parameters.

Dynamic calculations

Now we're talking about dynamic calculations. Dynamic calculations are like the previous ones. The goal is not the same. The first methods are simply to reduce the network, and dynamic calculations hope that the network will be free to adjust the amount of calculations it needs. So you can adapt to the platform on which you operate. A simple alternative to dynamic calculations is to train a collection of models to design parameters and calculate costs for different platforms.

How can networks be free to adjust their needs for computing resources? One possible direction is to let the network freely adjust its depth. And you can add an extra layer to this layer and to the middle. This additional layer of work is determined by the output of each hidden layer that is now classified. When the computing resources are adequate, the picture can run through all layers and get the final classification. When computing resources are inadequate, the network can decide which level it wants to do its own output.

It can also be free to determine its width, and how to let it freely determine its width. Set several different widths, and throw the same picture in. In training, the same picture is thrown in, and each network with different widths will have different outputs. We are hoping that every output will end as soon as it is close to the right answer, that the distance between all outputs and standard answers will be added to one loss, and that the loss will end when it is minimized. At this point we're using less neurons for the low performance.

The methods described above do not exist independently, and we can combine a variety of approaches to achieve better results.

Optimizing network architecture design

Optimizing network architecture is the best approach, but it is highly dependent on its research background, which requires a deep research effort to optimize the network architecture, which is also the work that the algorithm team should do.

ChatGPT and LLM

This section presents one of the most exciting applications of in-depth learning today, a large-language model, in which we focus on the rationale and the key technology behind it — pre-training.

First of all, we need to know the characteristics of this type of model, and first, ChatGPT has different outputs, so if you ask the same question, you might get very different answers. Every time the answer is regenerated. Another feature of ChatGPT is that you can continue to ask, how many rounds of interaction can be in the same conversation. He knows the past input and output. Not the information pre-prepared by the output developers, nor the same information that is searched for on the Internet.

So, what's ChatGPT really doing? As we described in Transformer, ChatGPT simply uses it as a function, input something, and output something. You can enter a sentence that will output the probability of the words that should be taken after the sentence. After taking this probability, we sample, and because of the randomity of the sampling, the answers vary from output to output.

As we described in Transformer, GPT keeps inputting new output into the codec until the end sign is exported, and finally a complete conversation is formed. As for its context capability, it is due to the fact that the text is always entered as part of the encoder.

After learning about the basic model's working methods, we need to know the key technology — pre-training. Because the parameters of the model are very large, we cannot receive so much training data to monitor, and we need to use self-supervised learning techniques, that is, pre-training. Models are pre-trained in a production format through a large number of open data sets. This is the GPT model.

After the GPT model, the next step is manual supervision of learning, and then GPT continues to study through the data provided by human teachers. That process, which we call pre-training, is being studied through a lot of web data before supervisory learning. This process of monitoring learning is called fine-tuning.

And we know that ChatGPT is not just a supervisory learning, but also a intensive learning that uses the usual PPO algorithms of intensive learning. In intensive learning, people are not just giving machine answers, but telling machines whether your answer is good or not. The benefits of enhanced learning are that human teachers who supervise learning are more difficult than supervisory learning, and in enhanced learning, human teachers are lazy and need only guide the larger direction. Enhanced learning can be used when humans are not able to give accurate answers, but can judge whether they are good or bad, for example, by writing poetry.

In conclusion, ChatGPT's learning is basically three steps - pre-training, supervision, and intensive learning.