Early termination with truncated samples about pygmmis HOT 1 CLOSED

OK, apologies that it took a while to get back to this issue. Life...

Thanks for your clear report, I can confirm it. However, PR #12 is only a workaround, not the actual solution. So let me address the problem here.

The problem, as you have determined yourself, is not in the EM equations themselves, otherwise the solution wouldn't have converged to something meaningful, but in the reported likelihood, which is used for convergence testing.

In a completely observed GMM, the likelihood of a sample x is

p(x | GMM) = 1 / Z sum_k alpha_k p_k(x)

with Z = int dx sum_k alpha_k p_k(x). Since the p_k or normalized and the alphas sum to 1, this normalization constant is dropped. But for incomplete samples, we cannot do that. The process that led the the observed samples is

p (x | GMM, Omega) = 1 / Z' Omega(x) sum_k alpha_k p_k(x)

with Z' = int dx Omega sum_k alpha_k p_k(x). As Omega is <= 1, this normalization term boosts the value of logL. Z' tells us the fraction of the probability mass that is observed. It can be very efficiently computed with Monte Carlo integration. My last commit 0e33ac6 does exactly that. With it the drop in logL doesn't arise and #12 is not necessary.

from pygmmis.