Advanced Machine Learning: Unsupervised and Semi-Supervised Learning

This English version was machine-translated from the Chinese original. Technical terms may require verification.

Cluster

Starting with this chapter, we're introducing some of the classic unsupervised learning tools that are used extensively in data mining, especially in the EDA direction.

Cluster tasks

Clusters attempt to divide samples from data sets into subsets that are usually non-interconnected, each of which is referred to as a "cluster." Through this division, each cluster may correspond to some potential concepts.

The clustering process can only automatically form a cluster structure, and the syntax of the cluster is subject to user control.

Clusters can be used as a separate process to find the inner distribution structure of the data or as subdivisions Other learning tasks.

Cluster performance measures

References Performance measures of the cluster model

Level Cluster

We've already described it in multiple statistics, where we don't repeat the narrative, but we've changed the perspective of the idea of clustering.

System cluster is a cluster structure that attempts to divide data sets at different levels, resulting in tree formation. Datasets can be divided using a “bottom-up” aggregation strategy, or a “top-down” fragmentation strategy.

Prototype Cluster

Prototype-based clustering "Prototype" means a representative point in the sample space. Very common in real-life cluster assignments

Using different prototypes, different solvency, different algorithms are created that give different group results.

k Average value algorithm

The original k-mean algorithm is a GP problem that requires minimizing square error. $E=sum sum sum sum symbol{\{i}||2}$$ It's not easy to calculate, so we're using an iterative approach to approximate solvency. This algorithm has the character of automatic cessation, which is not the same as a system cluster.

k average algorithms also have the means to nuclei, and we map the data to higher dimensions to deal with non-linear clusters, depending on our actual data.

Quantification of learning vectors

Similar to k average algorithm,"Learning Vector Quantification, Jane LVQ) is also an attempt to find a prototype vector to paint a cluster structure

But... LVQ assumes that samples of data have category tags to support grouping, which is actually a supervisory learning algorithm The goal of training is to find a team.$n$Wedge vectors. Each prototype represents a cluster.

What's our algorithm thinking?

1. Initializing prototype vectors
2. Each sample found the nearest prototype vector.
3. Update prototype vector
4. Repeat 2-3 until cessation conditions are met. The core is how to update the prototype vector.

Intuitively, for the sample.j,$若最近的原型向量$p_i^$ 与$x_j$ 的类别标记相同，则令$p_i$ Center$x_j$ The new prototype vector is $US$P}(prime}=p{i^{}}+\eta\cdot(\boldsymbol{x}{j}-\boldsymbol{p}{i^{What's wrong? If the category labels are different, then the principle of distance change is that$|(1+\eta)\cdot||p_{i^{*}}-x_{j}||_{2}$ Ours.$\eta$Still learning.

Gaussian Mixed Cluster

What's different from before is that we use probabilistic models to express the probabilities of a cluster prototype, and we use models to study the probability of each sample falling into each category. Instead of giving a definite classification.

Density Cluster

Such algorithms assume that the cluster structure can be determined by the closeness of the sample distribution; the density cluster algorithm examines the connectivity between samples from the point of view of sample density and is based on the continuous expansion of the cluster to obtain final cluster results.

DBSCAN

DBSCAN is a famous density cluster algorithm based on the closeness of a group of neighbourhood parameters for given$n$The DDS defines the concepts below

• $\epsilon$- Neighborhood: Yeah. $x_j\in D$Other $\epsilon$- Neighborhood contains sample collections. $D$ Center with $\dot{x}j$ 的距离不大于 $\epsilon$ 的样$\text{i}{\epsilon}(\boldsymbol{x}{j})={\boldsymbol{x}{i}\in D\mid\mathrm{dist}(\boldsymbol{x}{i},\boldsymbol{x}{j})\leqslant\epsilon};$
• Directly decreasing-reachable:$x_j$ in$x_i$ Yes. $\epsilon$- In the neighborhood, and...$x_i$ is the core object, then $x_j$ by $x_i$ Direct density;
• Density-reachable: $x_i$ and $x_j$, if sample sequence exists $p_1,p_2,\ldots,p_n$ Where \\boldsymbol{p} 1=\boldsymbol{i,\dot{p} n\doteq\boldsymbol{x}j$ 且$p{i+1}$ 由$p_i$密度直达，则称 $x_j$ 由 $x i$ Density to reach;
• Density-connected: $x_i$ and $x_j$, if existing $x_k$ Make $x_i$ and $x_j$ Both by$x_k$ Density to be achieved, in other words $x_i$ and $x_j$ Density connections ... (a new sample was added as a connection)

Based on these concepts, the DNSCAN definition cluster is: the largest-density-connected collection of samples from a density-accessible relationship Which means...$x$For the core object, the collection of all the densityable samples is our cluster.

The DBSCAN algorithm only needs to select a core object, find its cluster, then find a new core object in the rest of the sample, repeat the search, and know that we think it's over.

Finally, there was no group sample called noise.

Nuclear density estimates

Nuclear density estimates are not a cluster approach, but are closely linked to the cluster. Density estimates want to find a point-intensive area to determine an unknown probability density function, which can be used for a cluster.

As a method that did not require parameters, he did not assume the existence of a probability model, but infer the probability density at the bottom.

One dollar nuclear density

The cumulative distribution function is very easy to give. $$\hat{F}(x)=\frac{1}{n}\sum_{i=1}^nI(x_i\leqslant x)$$ We can use his guide to estimate density and consider a small window. $$\hat{f}(x)=\frac{\hat{F}(x+\frac{h}{2})-\hat{F}(x-\frac{h}{2})}{h}=\frac{k/n}{h}=\frac{k}{nh}$$ $h$The choice is very important. It's very big.$h$You can smooth out the density estimates, too small.$h$It would result in too few points being included and seriously inaccurate estimates.

The nuclear density estimate relies on a non-negative, symmetrical nuclear function that builds up to 1$K$that is:$K(x)\geqslant0,K(-x)=$ $K(x)$ For all$x$and$\int K( x)$d$x= 1$I don't know. So,$K$It's actually a probability density function.

Dispersed Nuclear We can rewrite the density estimates in front of the discrete core. $$\hat{f}(x)=\frac{1}{nh}\sum_{i=1}^nK\left(\frac{x-x_i}{h}\right)$$ of which$K$Yes $$\left.K(z)=\left(begin{array}ll}1&|z|\leqslant\frac{1}{2}\0&\text{Others}\right.\right.$

In order to be more smooth, we can think about it. Gaussian nuclear Define $$K(z)=\frac{1}{\sqrt{2\pi}}\exp\left{-\frac{z^2}{2}\right}$$ It's the beginning of this era.$\hat{f}(x)$Yes. $$K\left(\frac{x-x_i}{h}\right)=\frac{1}{\sqrt{2\pi}}\exp\left{-\frac{(x-x_i)^2}{2h^2}\right}$$ At this point in the larger zone, the points are factored into local density estimates with different probabilistic weights.

Multiple nuclear density

For the estimation of one$d$Dots$x=(x_1,x_2,\cdots,x_d)^\mathrm{T}$Probability density, definition$d$The "Window" for V is$d$One of the supercubes in space, that's the one that's going to...$x$It's central and it's long.$h$the supercube. Like this.$d$The size of the dimension supercube is: $$\mathrm{vol}(H_d(h))=h^d$$ Like a dollar, we can write a nuclear density estimate. $$\hat{f}(x)=\frac{1}{nh^d}\sum_{i=1}^nK\left(\frac{x-x_i}{h}\right)$$ of which$K$It's a multiple nuclear function. It's a multiple probabilistic density function.Introduction to Machine Learning and Supervision Learning: Nuclear MethodsThe nuclear function that's in it.

We can putDispersed NuclearDefine $$\left.K(z)=\left(begin{array}ll}1&|z_j|\leqslant\frac{1}{2}\0&\text Other Organiser$$ 高斯核定义为 $$K(z)=\frac{1}{(2\pi)^{d/2}}\exp\left{-\frac{z^\mathrm{T}z}{2}\right}$$ 代入 $z=\frac{{x-x_{i}}}{h}$ 有 $$K(\boldsymbol{z})=\frac{1}{(2\pi)^{d/2}}\exp\left{-\frac{(\boldsymbol{x}-\boldsymbol{x}_i)^\mathrm{T}(\boldsymbol{x}-\boldsymbol{x}_i)}{2h^2}\right}$$

Nearest nuclear density

The way forward is to look for points within a fixed volume to estimate nuclear density; the other is the number of fixed points$k$ Allow volume changes; commonly referred to as density estimates$k$The approach to proximity is also a non-parametric approach.

Number of given neighbours$k$estimate$x$The density is as follows: $$\hat{f}(\boldsymbol{x})=\frac k{n\mathrm{vol}(S_d(h_{\boldsymbol{x}}))}$$ of which$h_x$Yes.$x$To the first of it.$k$A recent neighbor's distance.$vol(S_d(h_x))$Yes.$x$For the center,$h_x$Radius$d$Vi superspherical$S_d(h_x)$The volume. In other words, width (radio)$h_x$Now it's a dependency.$x$and$k$.

DENCLUE

On the basis of nuclear density, we can give a generic cluster method based on density; find peaks in density through gradient optimization; and then find areas with density above a certain threshold.

Definitions: $x^$是概率密度函数$f. A local large point, referred to as*Density attraction(density attractor) A density attraction from$x$Start with the gradient down. is the maximum gradient of the probability density function. By extrapolating, we can give the formula below. $US$x sum sum i=nK(\frac{\boldsymbol{t-\boldsymbol{x}\boldsymbol{x}i}{\sum{i=1}^nK(\frac{\boldsymbol{x}_t-\boldsymbol{x}_i}{h})}$$

Definition: Give a cluster$C\subseteq D$, if all points $x\in C$ They're attracted to a single density attraction. $x^$,使得$\hat{f}(x^)\geqslant\xi$,其中$\xi$ is a user-defined minimum density threshold calledCenter defined cluster (center-defined cluster)that is: That's a lot of money.)=\frac1{nh^d}\sum_{i=1}^nK\left(\frac{x^-x_i}h\right)\geqslant\xi $$

Definition: A cluster of random shapes$C\subseteq D$It's one.Density-based cluster, if there is a group of density attractions $x 1^,x_2^,\cdots, x m*, making:

• Every point.$x\in C$They're attracted to something.$x_i^*;$
• Every density attraction is more dense.$\xi$i.e.$\hat{f}(x_i^*)\geqslant\xi;$
• Any two density attractions $x i^$和$x_j^$都是密度可达的，即存在一条从$x_i^$到$x_j^$的路径，使得所有在该路径上的点$y$都有$\hat{f}(y)\geqslant\xi$。

Here's the idea of the DENCLUE algorithm.

1. Calculating the density attraction of each point $x i^$，如果大于阈值$\xi$，则将其加入吸引子集合$A$，对应点加入被点信息的集合$R(x_i^{})$
2. Find the largest subset of all attractions $C$ Make sure any of them attracts a child density.
3. The biggest subset of these attractions. $C$It forms a seed based on density, and it attracts points into the cluster to form a cluster result.

I can prove it.DBSCAN is a special case of DENCLUE based on a general nuclear density estimation grouping methodI don't know. If you order$h=\epsilon$and$\xi=$Minpts, with a discrete nucleus, DENCLUE will get the same results as DBSCAN. Each density attraction corresponds to a core point, and the aggregation of connecting core points defines a cluster of attractions based on density.

It can also prove that it's the right choice.$h$and$\xi$I don't know.K-means is also a special case of concentration based on density.and the density attraction corresponds to the cluster centre. In addition, notably,The density-based approach can change.$\xi$Thresholds, generate layersI don't know. For example, reduction$\xi$Value makes several clusters merge together. At the same time, if the peak density is greater than the decrease,$\xi$value, a new cluster may be generated.

Spectrums and graphs

This section looks at clustering on the map, he and the hierarchical clustering, the spectrolysis of the matrix, and the nuclear-based clustering, which we will then explain clearly.

Figure and matrix

Assignance$\mathbb{R}^d$Medium$n$D=x i}i=1^n$,令$A$代表这些点之间的$A pair of likeness matrices: $A = \begin{matrix}a{11}&a_{12}&\cdots&a_{1n}\a_{21}&a_{22}&\cdots&a_{2n}\\vdots&\vdots&\cdots&\vdots\a_{n1}&a_{n2}&\cdots&A \end{matrix} $ of which$A(i,j)=a_{ij}$Expression$x_i$and$x_j$Similarity descriptive statistics and similarity factors in visualization. We're asking for symmetry and non-negativeness.$a_{ij}=a_{ji}$and$a_{ij}\geqslant0$。

Matrix$A$It can be seen as a rights-based map.$G=(V,E)$It has a rights-based neighbourhood matrix, which represents the neighbourhood, thus transforming samples into graphic data for analysis.

For each vertex$x_i$We can calculate his degree.$d_i$ $$d_i=\sum_{j=1}^na_{ij}$$ So we can export the array of degrees to $$\left.\Delta=\left(\begin{array}{cc}d 1&0&\cdots&0\0&d_2&\cdots&0\\vdots&\vdots&\ddots&\vdots\0&0&\cdots&d_n\end{array}\right.\right)=\begin{pmatrix}\sum_{j=1}^na_{1j}&0&\cdots&0\0&\sum_{j=1}^na_{2j}&\cdots&0\\vdots&\vdots&\ddots&\vdots\0&0&\cdots&\sum_{j=1}^na_{nj}\end{pmatrix}$$

By dividing each line of the adjacent matrix by the degree of the corresponding node, we can get Normalization of the Neighbourhood Matrix As follows: That's right.&\frac{a_{12}}{d_1}&\cdots&\frac{a_{1n}}{d_1}\\frac{a_{21}}{d_2}&\frac{a_{22}}{d_2}&\cdots&\frac{a_{2n}}{d_2}\\vdots&\vdots&\ddots&\vdots\\frac{a_{n1}}{d_n}&\frac{a_{n2}}{d_n}&\cdots&\frac{a_{nn}}{d_n}\end{pmatrix}\end{gathered}$$

Then we can define The Lapras Matrix of the Fig. As follows: $$L=\Delta-A$$ That's... That's right.&-a_{12}&\cdots&-a_{1n}\-a_{21}&\sum_{j\neq2}a_{2j}&\cdots&-a_{2n}\\vdots&\vdots&\ddots&\vdots\-a_{n1}&-a_{n2}&\cdots&\sum j\neq}a\end{matrix} $ He's a semi-positive symmetrical matrix; one of them.$n$A non-negative real number feature and its characteristic vector is positive; the La Plas matrix of the figure can also be calibrated and then calibrated to vary and obtained A unified Tulapras matrix

Figure cutting

A picture.$k$Passepartout wants a good division.$C$This makes the same cluster more similar and the different clusters less similar, which is very intuitive. Let us begin with some basic knowledge of graphic cutting.

For a given rights chart and its similarity matrix, whatever$S,T\subset V$ We define$W(S,T)$ Other Organiser$S$The other node is$V$and $$W(S,T)=\sum_{v_i\in S}\sum_{v_j\in T}a_{ij}$$

Organisation$S\subseteq V$♪ With ♪$\bar{S}$This represents a contours of complementarities, i.e.$\bar{S}=V-S$I don't know. One of the figures (vertext cut) Defined$V$Division to$S\subset V$and$\bar{S}$I don't know. Cut weight is defined as$S$and$\bar{S}$the sum of the weights of the edges formed by the vertex, i.e.$W(S,\bar{S})$

Give one contains$k$A cluster.$C={C_1,\cdots,C_k}$One. Cluster$C_i$Size (size) Defined as the number of nodes in a cluster, i.e.$|C_i|$I don't know. A cluster. $C_i$Volume (volume) Defines the sum of all power values containing the edges of the vertex in the cluster: $$\mathrm{vol}(C_i)=\sum_{v_j\in C_i}d_j=\sum_{v_j\in C_i}\sum_{v_r\in V}a_{jr}=W(C_i,V)$$

You!$c_i$He's satisfied. $$\left.c =left{array}ll}1&v_j\in C_i\0&v_j\notin C_i\end{array}\right.\right.$$

Here's the weight of the cut in matrix mode: $$00begin{aligned}W (C I},\overline{C I})&=\sum_{v_r\in C_i}\sum_{v_s\in V-C_i}a_{rs}=W(C_i,V)-W(C_i,C_i)\&== sync, corrected by elderman == @elder man He's connected to the Lapras matrix of similarity.

So, we're almost there.

Target function for grouping (minimised)

The cluster target function can be just one.$k$With regard to the optimization of roads, we would like to look for some good optimization objectives, one of two common ones.

Scale cut

$k$The ratio cut on the road is defined as follows: $US$US$\m m m m m m m m m m m=m=m=m=m=m=m=m=m=m=m=m=m=m=m=m=m=m=m=m=m=m=m=m=m=m=m=m=m=m=m=m=m=m=m=m=m=m=m=m=m=m m m=m m m=m=m m m m m m m m m m m m m m m m m m m m m m m m m m m m ,\m m m m m m m m m m ,\m m m m m m m m m m m m m ,\m m\m m m\m\m\m\m\m\m\m\m\i)}{|C_i|}=\sumI'm sorry. Proportional cut attempts to minimize from the cluster$C_i$To the others.$\overline{C}_i$to take into account the size of each cluster. It can be observed that the target function is smaller when the right value of the cut is minimized and larger.

Unfortunately, for the binary range,$c_i$The rationing target is hard for NPs. An obvious way to relax is to allow$c_i$Takes any real value.

Separability

The combined cut is similar to the ratio, except that it divides the weight of each cluster by the size of the cluster, not its size. The target function gives: $US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$US$USK K\LE{I, \overline}$US$US$US$US$US$US$$US$US$US$US$US$US$US$US$US$US$US$US$US K\K\N{, \K{L{, \E, \overline{C}i)}{\mathrm{vol}(C_i)}=\sumI don't know. There is also the same optimisation problem as proportional circumcision, which needs to be solved with relaxation.

Spectrometry

Based on the optimised algorithms given earlier (although we do not extrapolate the specific solvency process and results of the algorithms), here we can point out that we actually need only to study the given matrix, such as the La Plas matrix).$L$La Plass Matrix.$L^s$;and then calculates its feature value and feature vector, and selects the maximum feature value or the smallest feature value$k$as an indicator vector)

We finally got a series of real-value clusters.$u_i$ Both $u_n...u_{n-k+1}$ As these indicator vectors are defunct, they are not binary and need to be further addressed. We see him as a new matrix: $$\left.U=\left&|&&|\u_n&u_{n-1}&\cdots&u_{n-k+1}\|&|&&|\end{array}\right.\right)=\begin{pmatrix}u_{n,1}&u_{n-1,1}&\cdots&u_{n-k+1,1}\u_{n,2}&u_{n-1,2}&\cdots&u_{n-k+1,2}\|&|&\cdots&|\u_{n,n}&u_{n-1,n}&\cdots&u_{n-k+1,n}\end{pmatrix}$$ 每一行进行归一化处理： $$y_i=\frac{1}{\sqrt{\sum_{j=1}^ku_{n-j+1,i}^2}}(u_{n,i},u_{n-1,i},\cdots,u_{n-k+1,i})^\mathrm{T}$$ 目前每一行都是一个单位向量，如下 $$\left.Y=\left(\begin{array}{ccc}-&y_1^\mathrm{T}&-\-&y_2^\mathrm{T}&-\&\vdots&\-&y_n^\mathrm{T}&- I'm sorry. Now we can use fast concentration algorithms like K-means to make the current$n$Line Vector Considers$n$Point grouping$k$A cluster is the final cluster result.This spectral method is only appropriate for dealing with the similarity matrix and the La Plass matrix after integration.

Target function for grouping (maximized)

Let's discuss two more cluster target functions.

Average weight Objectives are defined as follows: $$\max_{\mathcal{C}}J_{aw}(\mathcal{C})=\sum_{i=1}^k\frac{W(C_i,C_i)}{|C_i|}=\sum_{i=1}^k\frac{c_i^\mathrm{T}Ac_i}{c_i^\mathrm{T}c_i}$$ We still need to find a solution.

Average weight with K-mes

We're here to discuss an interesting connection, if it's a powered neighbour matrix.$A$represents the nuclear value of a pair of points, and has$a_{ij}=K(x_i,x_j)$, the nuclear K-means square error and target can be used for grouping. SSE targets are: $$00begin{aligned}\min &=\sum_{j=1}^nK(\boldsymbol{x}i,\boldsymbol{x}j)-\sum{i=1}^k\frac{1}{|C_i|}\sum{\boldsymbol{x}r\in C_i}\sum{\boldsymbol{x}s\in C_i}K(\boldsymbol{x}r,\boldsymbol{x}s)\&=\sum{j=1}^na{jj}-\sum{i=1}^k\frac{1}{|C_i|}\sum_{v_r\in C_i}\sum_{v_s\in C_i}a_{rs}\&=\sum_{j=1}^na_{jj}-\sum_{i=1}^k\frac{c_i^\mathrm{T}Ac_i}{c_i^\mathrm{T}c_i}\&== sync, corrected by elderman == I can see that.$\sum_{j=1}^na_{jj}$Not related to cluster, minimizing SSE is maximizing the average weight of AW, and the two issues are the same as the final equivalent; for this NP problem, the nuclear K-means use a greedy hiatus to solve the problem, and the average power study its laxity.

Modularity The module wants to discuss the extent to which the same cluster is connected. $ \begin{aligned}\max {mathcal{C}&=\sum_{i=1}^k\left(\frac{c_i^\mathrm{T}Ac_i}{\mathrm{tr}(\boldsymbol{\Delta})}-\frac{(\boldsymbol{d}^\mathrm{T}c_i)^2}{\mathrm{tr}(\boldsymbol{\Delta})^2}\right)\&=\sum_{i=1}^k\left(c_i^\mathrm{T}\left(\frac{A}{\mathrm{tr}(\boldsymbol{\Delta})}\right)c_i-c_i^\mathrm{T}\left(\frac{d\cdot d_i^\mathrm{T}}{\mathrm{tr}(\boldsymbol{\Delta})^2}\right)c_i\right)\&=\sum_{i=1}^kc_i^\mathrm{T}Qc_i\end{aligned}$$ 其中$Q$是模块度矩阵为 $$Q=\mathrm{tr}(\bardsymbol{\D\cdot d i^mathrm{tr} (\bardsymbol{\Delta} There's still a need for a loose solution.

Normalized modularity equals average weights, and thus, in some cases, equals nuclear K-means

Markov Cluster

The Marcov cluster uses the original transfer matrix as the transfer matrix for the Malkov chain, hoping to obtain the final cluster results by simulating the transfer of the Mars chain, and we will briefly describe its thinking below.

Give a chart$G$. . . . . . .$A$, corresponding to the normalized adjacent matrix is$M=\Delta^{-1}A$I don't know. Matrix$M$It can be seen as$n\times n$transfer matrix (transaction matrix), with each array item$m_{ij}=\frac{a_{ij}}{d_i}$It could be seen as a node.$i$Go to Node$j$The probability. He's meeting the conditions of the Ma's chain transfer matrix.

Assuming that this is a chain of horseback, that is, the transfer probability matrix has nothing to do with the current position, then we can calculate his position.$n$Step transfer probability matrix. Last time.$n$We'll stop the calculations.

The last available transfer probability matrix can draw a transfer probability map based on which we can naturally discover clusters, such as the lower matrix. $$\boldsymbol{M}=\begin{pmatrix}&1&2&3&4&5&6&7\1&0&0&0&1&0&0&0\2&0&0&0&1&0&0&0\3&0&0&0&1&0&0&0\4&0&0&0&1&0&0&0\5&0&0&0&0&0&0.5&0.5\6&0&0&0&0&0&0.5&0.5\7&0&0&0&0&0&0.5&0.5\end{matrix} The numbering of the first column and the first line of the expression point is not the transfer probability.

Unusual detection

The most dominant abnormality algorithm is similar to ours. Explored data analysis: processing of anomaliesThe information presented in this section shows that, in some cases, anomalies are information and therefore need to be identified. The most dominant method is probabilities-based treatment, which can be judged when the probability of a sample appears below a certain threshold. As for the acquisition of this probability, it may be based on the use of generation models or on density. In any case, the overall idea of monitoring anomalies is very simple.

A lot of unusual detection methods need to be fine-tuned by a sample of complete information, and we should discuss when to choose the anomaly algorithm or the supervisory learning algorithm. When we have only a very small sample of labels, such as in the area of financial fraud detection, we have fewer cases of fraud and a relatively small number of unmarked samples, and we know that the number of fraud cases is rather small, the unusual detection may be a more appropriate method.

In fact, monitoring learning and abnormality detection uses two completely different approaches to understanding data, which model normal data and then reveal anomalies. The latter is the identification of irregular samples by means of supervision and learning, and when such samples are too few, it is difficult for him to learn all abnormal patterns, which can be avoided by abnormal testing. In other words, it is difficult to generalize what is not seen, which is limited by the algorithm itself.

Anomalous detection is more dependent on the choice of characteristics, requiring more rigorous judgement as to the value of each characteristic and whether its distribution meets the needs of the model, more than monitoring learning requirements. There is a need to construct more sophisticated features based on the experience of individuals in this field and to remove unnecessary ones.

Semi-supervisory learning

Unmarked sample

It is very common in the world to show that a large number of samples are not marked (lack of information on variables), that if traditional monitoring learning techniques are used directly, a significant amount of unmarked sample information is wasted, and that perhaps because the data set of marked samples is too small to produce good results in training.Semi-supervised learning: machine learning methods using unmarked samples

How can unmarked samples be incorporated into models? The most natural way is to mark unmarked samples, of course, but the resources that are being consumed may be too large, so we try to find another way.

We can train a model with a marked sample, then we can use the model to find the most useful sample for the model's progress, and then we can mark it; we can use a smaller sample to achieve a better result, which is called "active learning.""(active learning), the goal is to achieve the best possible performance by using as few "Query".

If no additional markers (expert knowledge) are introduced, can unmarked samples improve modelability? It's actually working.

In fact, unmarked samples do not directly contain tagging information, but if they are taken independently from the same data source and in the same distribution, they contain information on data distribution. A model would be very useful.Using characteristic distribution information from unmarked samples to help us upgrade modelability.

The use of unmarked samples necessitates some data distribution information from unmarked samples Assumptions relating to category tags

The most common is the Cluster Assumptions, which assume that the data are clustered and that the same cluster of samples belong to the same category. Another common hypothesis is "Flow-shaped." "(manifold assumption), i.e., assuming data are distributed on a flow structure, the adjacent sample has similar output values.

The popular hypothesis has no limit on output values, and it is more widely applied, and it's the same scenario that we're using now.

Semi-supervisory learning can be further divided into pure (pure) semi-supervisory learning, which assumes that unmarked samples from training data are not to be predicted, while the latter is false Unmarked samples considered in the course of the learning process are just to be predicted and the purpose of the learning is to obtain the best generalization on these unmarked samples Yes.

Generating Method

Generating methods (generative methods) are methods based directly on the generation model. Assuming that all data (whether marked or not) are generated by the same potential model.

This assumption allows us to link unmarked data to learning objectives through parameters of a potential model, while tags of unmarked data can be considered missing parameters of the model, which can usually be based on EM algorithms to provide a very similar estimate of solvency.

The difference in such methods is mainly in the assumptions of the generation model, which will produce different methods.

given sample$x$, whose real category is marked$y\in\mathcal{Y}$, where$\mathcal{Y}={1,2,\ldots,N}$Assuming that samples are generated by a Gaussian hybrid model and that each category corresponds to a Gaussian mixture. In other words, data samples are generated on the basis of the following probability density: $$ (\bardsymbol{x} =sum i\cdot p} (\bardsymbol{mbol{\i,\bardsymbol{\symbol{\Sigma}(i): $ of which, blending factor $ \\alpha i\geqslant0,\sum{i=1}^N\alpha_i=1;p(\boldsymbol{x}\mid\boldsymbol{\mu}_i,\boldsymbol{\Sigma}_i)$是样本$x$属于第$I'm a Gossian. Probability of combinations;$\boldsymbol{\mu}_i$and$\boldsymbol{\Sigma}_i$It's a parameter for the Goss mix.

Using MLE to estimate EM solver, you can judge the type of information.

Semi-supervised SVM

Semi-Supervised Support Vector Machine (S3VM) supports the promotion of vectors in semi-supervised learning.

Without considering unmarked samples, the support vector tries to find the maximum spaced hyperplatform, while after considering unmarked samples, the S3VM tries to find an overplatform that separates two marked samples and passes through the low-density area of the data,

The basic assumption here is "low-density security," which is obviously the extension of the cluster scenario after consideration of linear super-levels.

As for how to solve it, it's not here.

Figure 1. Semi-oversighted learning

A data set is given, and we can map it as a map, and each sample is assembled to match a node in the map, and if the two samples are very similar (or relevant), there is a side between the corresponding node, with the "strength" of the side being more similar (or relevant) to the sample. It's the idea of drawing machines learning.

We can imagine the nodes of the marked samples as dyed, while the unmarked samples of the nodes have not yet dyed. So, semi-supervisory learning corresponds to the process of spreading or spreading colours on the map.

Disagreement-based approach

Unlike the use of unmarked data in single-learning devices, such as the method of generation, semi-supervised SVM, and graphics-supervised learning, the use of multi-learning devices is essential for the use of unmarked data.

Co-training is an important representative of this approach, which was originally designed for multi-view data and is therefore also seen as a representative of multi-view learning. Before we introduce teamwork, let's see what multi-view data is.

In a number of practical applications, a data object tends to have multiple "attribute set " , and each property set constitutes a "view".

For example, for a film, it has several properties: the attribute set for image images, the attribute set for sound messages, the attribute set for subtitle messages, and even the attribute set for online advocacy discussions. Each property set can be seen as a view.

So, a film is a sample.$(\langle\boldsymbol{x}^1,\boldsymbol{x}^2\rangle,y)$, where$x^i$It's the sample in the view.$i$, i.e. attribute vector based on the description of the view properties, you may want to fake Freeze.$x^{1}$is the attribute vector in the image view,$x^2$is the attribute vector in the sound view;$y$It's a label. It's assumed to be the type of movie, like "action film," "love film."$(\langle x^1,x^2\rangle,y)$This kind of data is multi-view data.

Assume that different views have "compatibility" (compatibility), i.e. that they contain about output space$\gamma$The message is identical:$\gamma^{1}$This means that the marking space is distinguished from image information.$\gamma^{2}$indicates that the tag space is distinguished from sound information, and there is$\mathcal{Y}=\mathcal{Y}^1=\mathcal{Y}^2$It is clear that, on the basis of compatibility, the “complementarity” of different views of information would make it easier to build a learning device.

The synergetic training makes good use of the multi-view "compatible complementarities", assuming that the data have two full (sufficient) and independent viewing of conditions, meaning that each view contains information sufficient to produce the best learners, and "conditional independence" means that two views are independent under the given category tag. In this case, a simple method can be used to use unmarked data:

First, a taxonomy is trained on each view based on a marked sample, then each classifier is given a false mark to select its own “most sure” unmarked sample, and a pseudomark sample is provided to another classifier as an additional marked sample for training updating ... This process of “learning from each other and making progress together” continues in succession until both categories do not change or reach the predefined number of iterative wheels.

While the collaborative training process is simple, it is surprising that theoretical evidence shows that unmarked samples can be used to raise the generalization performance of the weak taxonomy to an arbitrary high if the two views are adequate and independent. However, the independence of the viewing is often difficult to satisfy in a realistic mission, and therefore the performance increase is not so large, but research suggests that, even under weaker conditions, teamwork can effectively enhance the performance of the weak taxonomy.

Co-training algorithms are themselves designed for multi-view data, but since then a number of variable algorithms have emerged that can be used in single-view data, either using different learning algorithms or using different data sampling, or even using different parameter settings to produce different learning devices that can effectively use unmarked data to enhance performance. Subsequent theoretical studies have found that such algorithms in fact require, inter alia, multiple viewing of data, and only significant differences (or differences) between learners, which can be enhanced by providing each other with false tag samples. It's not important to have multiple viewing designs.