The Kalman Filter Family: KF, EKF, UKF, and EnKF

This English version was machine-translated from the Chinese original. Technical terms may require verification.

Kalman Filter

The questions in this article can also be addressedLinear time series analysis: smooth sequence, ARMA and ARIMA、Financial time series analysis: ARCCH/GARCH effect and volatility modellingHow the concept of a relatively close read together is developed in different contexts.

Why do you need Kalman filters?

State Estimates are fundamental issues in robotics, autopilot and aerospace. Engineering systems need to combine physical models with noise-bearing sensor readings to estimate states that cannot be observed directly and accurately. Kalman Filter provided a progressive calculation method for the linear status spatial model.

Assuming that we estimate the location and speed of the vehicles, two sources of information can be used, but they all have errors:

1. Your speed sheet: shows 100 km/h, but it may actually be between 90 and 110 km/h.
2. Your GPS: Shows position, but the signal is highly drifting, possibly wandering within 50 meters.

The speed tables and GPS measurements are not identical. Kalman filters first push the position and speed of the previous moment to the current moment using a motion model, then revise the predictions with observations such as GPS. This process can be understood as assigning weights to uncertainties, but not just as a simple average of two readings.

If the sensor noise is smaller, more reference observations are made when updated; if the model predicts less uncertainty, the level of update is reduced. The two are being given a weight of ** Kalman Gain, a group of people who are not in the country. $K$** Decision.

When assumptions such as online, Goss noise are established, the Kalman filter can be estimated at a minimum average error, while maintaining the uncertainty of the estimate.

• Status $x$: Values to be sought (e.g. location, speed).
• Arguments $P$: describe the difference between the difference in the state estimate and the amount of the agreement between the variables.$P$ The smaller the model, the more concentrated the current estimate is, but it is not directly equivalent to the "Error range" $\pm 5$ Me.

Create two equations (world description)

First, you describe the system in mathematical language. Standard Kalman filtering scenario shift and observation relationship isLinearand assigns the difference between the process noise and the measurement of the noise:

State equation (physical model):

$$ x_k = Fx_{k-1} + Bu_k + w_k $$

• $x_k$: Current status.
• $F$: Status transfer matrix (e. g. next-second position = last-second position + speed) $\times$ Time.
• $u_k$: Control volume (e.g., stepping on the gas).
• $w_k$：Process Noise(Process Noise), subject to Goss distribution. $N(0, Q)$I'm sorry. Here. $Q$ It represents the degree of spectrometry of physical models.

Observation equation (sensor model):

$$ z_k = Hx_k + v_k $$

• $z_k$Sensor reading.
• $H$: Observation matrix (magnify the state to read, for example, the state is [location, velocity], the sensor only measures [location],$H$ Yeah. $[1, 0]$）。
• $v_k$：Measuring Noise(Measurement Noise) $N(0, R)$I'm sorry. Here. $R$ (c) Representing the sensor ' s insinuity.

Core algorithm

Step A: Forecast (Time Update) - Before looking at sensors, estimate where.

1. Debit status:

   $$ \hat{x}k^- = F\hat{x}{k-1} + Bu_k $$

2. Deduce uncertainty:

   $$ P_k^- = FP_{k-1}F^T + Q $$

Step B: Update (Measurement Update) - See Sensors $z_k$ After that, the projections are revised with observations.

3. Calman Gain Calculated $K$：

   $$ K_k = \frac{P_k^- H^T}{H P_k^- H^T + R} $$

   • This is the most critical formula in the Kalman filter.
   • Intuitive understanding:$K \approx \frac{\text{误差}}{\text{误差} + R}$。
   • If the sensor noises $R \to 0$, and $K \to 1$ (fully believed sensors)
   • If you're expecting a mistake, $P \to 0$, and $K \to 0$ (Full letter forecast)

4. Status of amendment (final result):

   $$ \hat{x}_k = \hat{x}_k^- + K_k(z_k - H\hat{x}_k^-) $$

   • Final estimate = projection + $K \times$ (Measurements - predicted measurements).
   • The part in brackets is calledInnovation

5. Amending uncertainty:

   $$ P_k = (I - K_kH)P_k^- $$

What do you get when you're supposed to be?

The filters keep up to date with state and uncertainty following the "Prognation-Observation-Amended" cycle:

1. Integration models and observations: Final estimate when models, noise assumptions and parameters are set reasonably $\hat{x}_k$ It is usually more stable than readings using only physical projections or single sensors.

2. Depress the part measuring noise: If the GPS signal suddenly changes, the Kalman filter will be caused by $R$(sensor noise) Large or $P$(Prodict error) is smaller and chooses not to fully believe in the jump, thus drawing a smooth track.

3. Tracking uncertainty: Coordinated matrix under observation and stability of parameters, etc. $P_k$ There is a potential for gradual stabilization. The deviations may also be distorted or dispersed when models are not matched or noise settings are not set.

Intuitively, physical models provide continuous predictions, and sensors are responsible for correcting prediction deviations, with relative weights of both being subject to uncertainty.

Engineering practice

In the code, the matrix. $F$ and $H$ It is usually determined by the laws of physics and is fixed. But... $Q$ and $R$ It's the parameters you need to "tweak."

$R$（Measuring noise association differences: can be estimated on sensor specifications, static measurements or repeated experiments.

$Q$（Process Noise Argument): Disturbing without coverage in the description status model. The greater the physical or external interference that is missing from the model, the greater the need for it to be set up and repeatedly checked through the residual and physical tracks.

$x_0$（Initial Status: First measurement values can be used or a priori tested. The initialization deviations are modified depending on the system ' s detectability and the quality of subsequent observations.

$P_0$（Initial difference: If yes $x_0$ Lack of assurance allows for a larger initial co-conformation to allow filters to refer more to observations in early updates. Values still need to match the status profile and actual error range.

Summary

The regression form of the Kalman filter is suitable for real-time state estimates, but the standard version relies on linear models and noise assumptions. When the real system is clearly non-linear, variants such as extended Kalman filters are required.

The real world is often non-linear (e.g. robots do not go straight, but turn). And then we need to introduce it.Expand EKF。

Progress: Expand the Kalman filter (EKF)

EKF is the non-linear state estimation method commonly used in the Karman filter family, and is widely used for robotics, navigation and sensor integration.

Why EkF?

Standard KF requirement state transfer and observation models are linear. The actual system often includes non-linear relationships, such as the angles in robotic motion. $\sin/\cos$and the square root of radar range $\sqrt{}$。

After a change in the distribution of Goss through non-linear functions, the strict Goss shape is usually not maintained. Directly applies the standard KF linear dissemination formula, which introduces an indescribable approximation error.

EKF performs a linear first-orderization of non-linear functions near the current estimate point. Similar to the use of a plane near the Earth ' s surface for a sufficient time range, it requires only local approximation to describe changes in the immediate state.

Mathematical mechanisms (linearization and the Yacca matrix)

In standard KF, we assume $x_k = Fx_{k-1}$I'm sorry. However, in the EKF, state transfer and observation became non-linear functions:

$$ \begin{aligned} x_k &= f(x_{k-1}, u_k) + w_k \ z_k &= h(x_k) + v_k \end{aligned} $$

• When calculating status (average): We can immediately add the estimate of the previous step to the non-linear function $f(\cdot)$It's okay.
• Count Uncertainty (Assisting P): The ACSM cannot " substitute " directly for non-linear functions. We can't just count. $P_k = f(P_{k-1})$。

To update the differences. $P$We have to find one.Linear MatrixApproximately represents the "distortion level" of the non-linear function at the current point. This matrix is...Jacobian Matrix。

EKF will set a fixed matrix in the standard KF $F$ and $H$ Replace the acoustical matrix that changes with the changing state of the state:

• $F_k = \frac{\partial f}{\partial x} \mid_{\hat{x}_{k-1}}$ (local slope of the status shift function)
• $H_k = \frac{\partial h}{\partial x} \mid_{\hat{x}_k^-}$ (local slope of observation function)

The EKF projection process can be written as follows:

1. Status prediction (retain non-linear functions): $$ \hat{x}k^- = f(\hat{x}{k-1}, u_k) $$
2. Accompanying predictions (with the Yacca matrix, approximation): $$ P_k^- = F_k P_{k-1} F_k^T + Q $$

It's like: I'm going to go around the curve, but I'm going to go around the cut-off line to estimate my margin of error.

Engineering practices and challenges

When's the EKF?

• Non-linear, with significant local linear error: UKF or particle filters (PF) can be considered.
• When the Yacima matrix is difficult to extrapolate or maintain: UKF, automatic calibration or numerical method may be considered.
• When the state distribution is clearly multiple peaks: particle filters are usually more appropriate.

The UKF avoids the visible and altruistic matrix and reduces the costs of extrapolation and maintenance in some non-linear systems, but the amount calculated, the parameter settings and the numerical stability still need to be assessed separately.

Step 2: Unscrew Kalman Filter (UKF)

When the Arby matrix is difficult to extrapolate or local linear precision is insufficient, the unscrutinized Kalman filter (UKF) can be considered.

Intuitive Understanding (The Intuition)

Remember the problem we had in the EkF? ♪ When one ♪Goss distributionThrough one.Non-linear functionsAt times, it often emerges in shape that is no longer a standard elliptical, but that may become a curved “banana”.

EKF approach: Use a first-order barometer near the average, which is almost non-linear, and therefore accuracy depends on linear effects near the current point.

UKF approach: Julier and Uhlmann suggest that instead of linearizing non-linear functions directly, a group of determinative sampling points is chosen to approximate post-functional changes in the distribution of the state.

UKF Select a group from the current status distribution Sigma dots, and then reestimate the average and the co-conforming difference by the changed point.

Mathematical: Unhappinessed Transform

This process is called Unscented Transform, UTI'm sorry. It uses a distribution of determinative samples that is similar to non-linear variations.

But unlike Monte Carlo, the UKF uses one of the most random samples.Specimen Sample (Deterministic Sampling)。

The following concrete steps are taken:

1. Select Sigma Points (Sigma Points Semenation)

Assuming your state vector. $x$ Yes. $n$ V. The UKF will be symmetrical around the average Choose $2n+1$ Point.

• Centre: $\mathcal{X}_0 = \mu$ (current average)

• Around: $\mathcal{X}_i = \mu \pm (\sqrt{(n+\lambda)P})_i$

  • Here. $\sqrt{P}$ It is the square root of the co-ordinated matrix (usually obtained through Cholesky decomposition).
  • $\lambda$ is the scaling parameters, how far are the controls on these points from the centre?

2. Non-linear transmission

This step does not require a dramatic extrapolation of the Arbya Matrix, which directly points Sigma $\mathcal{X}_i$ Substitute non-linear physical equations $f(\cdot)$：

$$ \mathcal{Y}_i = f(\mathcal{X}_i) $$

• Advantages: No one-step linearization of the visible. However, when functions are inconsistent, jump or value instability, the approximation of UKF may still be invalid and subject to experimental examination.

Reorganization distribution

Now we got a set of converted points. $\mathcal{Y}_i$I'm sorry. How do you get back to Goss distribution? Weighted average!

• New average: $\hat{y} = \sum_{i=0}^{2n} W_i^{(m)} \mathcal{Y}_i$
• New Association difference: $P_y = \sum_{i=0}^{2n} W_i^{(c)} (\mathcal{Y}_i - \hat{y})(\mathcal{Y}_i - \hat{y})^T$

Here. $W_i$ The fixed weights are calculated on the basis of the distance.

4. Complete UKF closed loops

With the predicted averages and the difference, UKF still uses the Kalman update: $$ K = P_{xy} P_{yy}^{-1} $$ $$ \hat{x} = \hat{x}^- + K(z - \hat{z}) $$

The point is, the difference between the two is that $P_{xy}$ It's also calculated by Sigma's weighting, which completely avoids the Yacca Matrix. $H$ .

Summary

UKF does not need to calculate the autonometric matrix, which is easier to achieve than EKF in some non-linear issues and may be more similar to it. It is not a generic replacement for EKF: state dimensions, calculation of budgets, model structure and realization will affect choices.

KF, EKF and UKF all need to maintain their differences and perform matrix operations. When the state dimension is large, storage and costing increases rapidly and numerical stability is more likely to be encountered. EnKF is a sample collection of near-coherent differences that is suitable for handling certain high-dimensional systems.

Step 3: Gather the Karman filter (EnKF)

EnKF is mainly oriented towards situations with large state dimensions and direct maintenance of complete matrix costs, such as reaching dimensions $10^6$ .

Standard KF requires storage and calculation of one $n \times n$ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . $P$。

• If $n = 10^6$,A matrix $P$ Yes. $10^{12}$ An element.
• We need to save this matrix. 8TB RAM。
• Direct storage and operation of such dense matrices are not usually feasible and therefore require the use of structural, approximation or sample methods.

EnKF Unobvious maintenance complete $P$, instead of a statistically approximate distribution and coordination difference of a group of samples.

EnKF no longer maintains that huge thing. $P$ The matrix, but the one that's kept. "Convergence" (Ensemble)I'm sorry. Imagine, you would have drawn a perfect ellipse. Now you just have to put a little bit of paper on it. $N$ A little bit.Like what? $N=50$ or $100$I'm not sure. Just this. $N$ The distribution of a dot is similar to that ellipse, so we can use it. $N$ One point for the Goss distribution.

There's only two steps of the cycle.

Step A: Forecast - Parallel Run Model

This step is the simplest, and the strongest place for EnkF. There is no need for an acoustical matrix, no need for linearity. You just have to take this. $N$ A sample. Each one of them is thrown into you.Non-linear physical model $f(\cdot)$ Run a little bit.

• Key points: I'll add a separate one for each sample.Random Process Noise $\mathcal{w}_i$I'm sorry. This is to prevent all samples from ending up at the same point (loss of diversity).
• This step can be paralleled on GPU, very fast.

Step B: Analysis/update (Analysis) - Update the collection with sample statistics

The forecast is made. $N$ After a sample, the next step is to use observations. $z$ Updates the whole set.

EnKF UseSample statisticsReplace the full matrix of the agreement $P$I'm sorry. The updating process consists of the following three steps:

1. Calman Gain Calculated $K$： The standard formula is:$K = P H^T (H P H^T + R)^{-1}$I'm sorry. But in EnkF, we...It doesn't exist. $P$I'm sorry. We'll calculate it directly from a sample. $PH^T$ and $HPH^T$ These two:

   • $PH^T \approx$ Status setandForecast observation clusterThe difference between the union.
   • $HPH^T \approx$ Forecast observation clusterThe self-contribution difference.
   • So we're avoiding it. $10^6 \times 10^6$ The storage and calculation of large matrices involves only small arrays (usually the number of observations is much smaller than the state dimension).

2. Update each sample (disturbation observation): To maintain statistical correctness, we cannot use only the same observations. Value $z$ Go update all the samples. We need to generate. $N$ individualNoise-bearing observations：

   $$ z_i = z + v_i, \quad v_i \sim N(0, R) $$

   And then, for every sample, $x_i$ Do Kalman update alone:

   $$ x_i^{new} = x_i + K (z_i - H x_i) $$

3. Aggregation: Update $N$ Different. $x_i^{new}$I'm sorry. EnKF retains the whole set, instead of choosing a sample from it as an output.

   • If you need a specific...Estimated value, the average of the pools is calculated:$\hat{x} = \frac{1}{N} \sum x_i^{new}$。
   • If you need to know,Uncertainty, calculate the difference of the pool:$P = \text{Cov}(x^{new})$。
   • This new collection will be used directly as input into the next step of the projection, and will be repeated.

Engineering issues

In high-dimensional applications such as WRF weather patterns and ocean circulation models, limited aggregations can lead to significant sampling errors. The actual system usually also needs to address two issues:

Hypocrisy (Spurious Correlations)

• Reason: The key assumption for EnKF is to use $N$ Samples (e.g. 50) to measure estimates $10^6$ The coordination difference for the dimension. Statistics tell us that when the sample is too small, the relevant coefficients are calculated to be huge.Sample error。
• Performance: If we calculate the matrix of differences, we find some absurdity:The temperature in Brazil is actually 0.9 for Texas wind speed.I'm sorry. It's not technically reasonable, more like mathematical coincidence.
• Consequences: When the temperature in Brazil is observed, the filter may modify the wind speed of Texas to the wrong co-convene, so that the update can be transmitted to areas without physical connection.
• Solving: Localization
  • Rationale: Introduction of physical commons — there is no correlation between too far-off states.
  • Operation: Calman gain is being calculated. $K$ , the calculated matrix of the agreement $PH^T$ Light one.Distance weight matrixI'm sorry. The closer the weight is closer to 1 and the distance exceeds a certain radius (e.g. 500 km) and is directly set to 0. That cut off those remote and false links.

Filter Diversion

• Reason: Models are always imperfect, and ours. $N$ The individual samples are often drawn from the same observations during the iterative process, resulting in their increasing similarity (the differences are smaller and smaller).
• Performance:
  1. The difference of the collection $P$ Quick approaching 0.
  2. Based on Formula $K = PH^T(\dots)^{-1}$Kalman gain. $K$ It's close to zero.
  3. Results: The filter underestimates its own uncertainty, and the Kalman gain is close to zero, and new observations are difficult to correct the deviating state estimates.
• Solve: Covariance Inflation
  • Rationale: Since the difference is always small, it's artificial.IncreaseOne, maintain sensitivity to new data.
  • Operation: Every time we're done with the prediction, we'll take all the samples. $x_i$ Deviation from average $\bar{x}$ Part times one coefficient $ \lambda > $1 (e.g. 1.01): $$ x_i^{new} = \bar{x} + \lambda (x_i - \bar{x}) $$
  • It's like injecting a little uncertainty into the system, preventing it from being too early to be blindly confident.

Summary

The main advantage of EnKF is that it directly runs non-linear models, does not require the visible maintenance of the complete matrix of differences and that the group members can calculate in parallel. A small sample can make the estimation of a million-dimensional state possible, but the approximate quality depends on aggregate size, localization, complication variation and specific physical models.

The approach described above has one thing in common:Assuming a single-peak distribution (however, it ends with a mean and a variance).

If the distribution of the state is clearly high, reliance on a single-peaker approach may not be able to retain different models. I'll think about it. Particle Filter, the distribution of the Beyers filtering in a form similar to the usual type of a gravitational particle.

Appendix: Horizontal comparison of mainstream estimation algorithms

Methodology	State dimensional (n)	Distribution assumptions	Why did you do that?	Disadvantages
KF / EKF / UKF	Dimensions $n$ The matrix size is determined. $(n \times n)$	Gaussian	If you save the average and the difference, calculate quickly, not only inverted, but also solves.	It is impossible to deal with “multi-peak” situations (e.g., not knowing which one of the two similar rooms is in).
EnKF (collecting Kalman)	Dimensions $n$ It's big. Only a few samples. $N$	It's like Goss.	To solve it. $n$ Too big to leave the matrix unattended.	The distribution of the Nang Figos remains difficult to deal with.
Particle Filter (PF)	Dimensions $n$ Not too big.	Any distribution (compared with particles)	To address the complex situations of “Multi Peaks” and “Figos”.	The size of the calculations is large and the dimensions are high enough for particles to be used (the shortage of particles).