Hello, thank you for the KSG estimator implementation. I ran the estimator for a high-dimensional normal distribution as follows:
SIZE = 32
HALF_SIZE = int(SIZE / 2)
cov = np.eye(SIZE)
cov[:HALF_SIZE, HALF_SIZE:] = 0.5 * np.eye(HALF_SIZE)
cov[HALF_SIZE:, :HALF_SIZE] = 0.5 * np.eye(HALF_SIZE)
print("True:", 0.5 * log(1 / np.linalg.det(cov)))
points = np.random.multivariate_normal(np.zeros(SIZE), cov, size=5000)
X, Y = points[:, :HALF_SIZE], points[:, HALF_SIZE:]
print("Estimated:", kraskov_mi(X, Y, k=5))
True: 2.3014565796142468
Estimated: 0.7142441789816445
This result is consistent over multiple random samples. Can you provide any intuition on why the estimated mutual information is substantially lower than the true mutual information? Is the estimate normalized in some way I'm not seeing?
Any help is appreciated ๐.