Possible edge case issue with intercept and RMSE calculated value about eps HOT 4 CLOSED

Hey @mb52089, my guess is it's due to multicollinearity. The article outlines a few ways to remedy it. One is to use a regularized regression like ridge. The GSLR gem supports ridge regression so you could try using that directly. I may add support for it to Eps at some point.

from eps.

I just re-created all the models using lightgbm.  The RMSE values that were in question are fine using lightgbm.  

DISCLAIMER: I'm not a statistician or a real data scientist.

The 1596 models are so closely related, have such overlap and share the same predictors since they are derived from a single aggregate data set (especially the 21 models from the affected user) that I'm having a hard time understanding how multicollinearity could be a problem in just 6 of the 21 models for this user, and not affect this user's other models, or any of the other 1575 models from other users. If there is colinearity it seems I should see it in all similar models that share most of the same underlying data and all of the same predictors.

But the affected models still predict well, even with the wacky RMSE, so I guess I'll just monitor the situation for now.

from eps.

I don't think it's an issue with Eps, so going to close. fwiw, I'd recommend building a single model vs individual models for users so the model can learn from all users (unless a business constraint prohibits it).

from eps.

from eps.