This is about figuring out how Kalman Filter works. Here are some online materials
https://scipy-cookbook.readthedocs.io/items/KalmanFiltering.html
https://pykalman.github.io/
https://www.quantopian.com/lectures/kalman-filters#notebook_tab
http://www.thealgoengineer.com/2014/online_linear_regression_kalman_filter/
Z - true value
z - observation values
R - covariance of z
x - estimate of Z (usually assigned to 0)
P - covariance of x (usually assigned to 1)
Q - transition covariance (usually assigned to 1e-5)
Kg - Kalman Gain (a weight between observation and estimation)
Assume true value is constant
At time t-1, we guess x(t) = x(t-1), then P(t) = P(t-1) + Q
At time t, we get z(t). Then we use x(t) and z(t) to get a better x(t) and P(t) (I use x'(t) and P'(t)). Then back to previous step.
# This equals to weighted average between Z(t) and x(t-1)
how to get the new P'(t) in previous step
rewrite
Here, we use
Finally, replace x(t) and P(t) with x'(t) and P'(t). Then we can begin next iteration.
To further illustrate Kalman-Filter. We generate some sample data and observations with noises, and try to apply Kalman-Filter to our observations. Let's see what happens.
In Simple_Excample, we assume true value is -0.37727, we generate 50 observations with normally distributed noise with mean 0 and variance 0.01. We also assume process variance is 1e-5. Our initial estimation and its covariance is 0 and 1. By iteratively apply Kalman-Filter, we can then get our estimation for the true value through time.
Here is the result
Although we get huge bias at the initial guess, but our estimation converges to true value very fast.
We can see that the error is gradually reducing through iterations. And it reduces very fast at the beginning.
Python also has a library pykalman
. This module implements two algorithms for tracking: the Kalman Filter and Kalman Smoother. In addition, model parameters which are traditionally specified by hand can also be learned by the implemented EM algorithm without any labeled training data. All three algorithms are contained in the KalmanFilter class in this module.
https://pykalman.github.io/#kalman-filter-user-s-guide
Here is the result by using pykalman
Pairs trading is a market-neutral trading strategy that matches a long position with a short position in a pair of highly correlated instruments such as two stocks, exchange-traded funds (ETFs), currencies, commodities or options. The long and short positions (also called hedge ratio) of the pairs can be calculated simply by linear regression between the two assets. However, the relation between two assets isn't always a constant in real world. In this situation, Kalman-Filter is a good method to calculate the hedge ratio.
Here is the result of applying Kalman-Filter to EWC and EWA with time horizon 2010-01-01 to 2014-08-01