SingularMatrix error when Copula can be fitted with alternative start values about copulae HOT 4 CLOSED

Hello,

I have some suspicion where the error is coming from. But if possible, could you attach the stack trace?

from copulae.

Sure, here you go:

Traceback (most recent call last):
File "/opt/risk/Code/hedging_risk/abstract.py", line 60, in fit
self.copula.fit(data=self.data_array, est_var=True)
File "/.../site-packages/copulae/copula/base.py", line 81, in fit
CopulaEstimator(self, data, x0=x0, method=method, est_var=est_var, verbose=verbose, optim_options=optim_options)
File "/.../site-packages/copulae/copula/estimator.py", line 66, in init
self.fit() # fit the copula
File "/.../site-packages/copulae/copula/estimator.py", line 72, in fit
self._verbose).fit(m)
File "/.../site-packages/copulae/copula/est_max_likelihood.py", line 60, in fit
res = self._optimize()
File "/.../site-packages/copulae/copula/est_max_likelihood.py", line 108, in _optimize
return minimize(self.copula_log_lik, self.initial_params, **self.optim_options)
File "/.../site-packages/scipy/optimize/_minimize.py", line 611, in minimize
constraints, callback=callback, **options)
File "/.../site-packages/scipy/optimize/slsqp.py", line 379, in _minimize_slsqp
fx = func(x)
File "/.../site-packages/scipy/optimize/optimize.py", line 293, in function_wrapper
return function(*(wrapper_args + args))
File "/.../site-packages/copulae/copula/est_max_likelihood.py", line 103, in copula_log_lik
return -self.copula.log_lik(self.data)
File "/.../site-packages/copulae/elliptical/abstract.py", line 43, in log_lik
return super().log_lik(data)
File "/.../site-packages/copulae/copula/base.py", line 194, in log_lik
return self.pdf(self.pobs(data), log=True).sum()
File "/.../site-packages/copulae/utility/utils.py", line 36, in internal
res = np.asarray(f(cls, x, *args, **kwargs))
File "/.../site-packages/copulae/elliptical/gaussian.py", line 82, in pdf
d = mvn.logpdf(q, cov=sigma) - norm.logpdf(q).sum(1)
File "/.../site-packages/scipy/stats/_multivariate.py", line 487, in logpdf
psd = _PSD(cov, allow_singular=allow_singular)
File "/.../site-packages/scipy/stats/_multivariate.py", line 159, in init
raise np.linalg.LinAlgError('singular matrix')
numpy.linalg.LinAlgError: singular matrix

from copulae.

Hello, sorry I haven't gotten back on the issue for a while. I've done some tests to try and replicate the issue but I think the solution lies mainly in the data treatment (pre-processing) prior to fitting with the copula.

This error will occur if at least one column of your data is a linear combination (or extremely close) of other columns. So something like the below will trigger it.

arr = np.random.normal(size=(100, 3))
arr[:, -1] = arr[:, 0] * 0.5

from copulae.

No problem. That sounds like it might be the cause, I'm pretty sure that there are no linear combinations (two dimensional empirical data), but there are often 0-values in one of the series, so your explanation makes sense.

from copulae.