andybega / demspaces Goto Github PK

The number of components to extract right now is at default of 5, at some point make this an optional argument. Probably also take a look at components vs fraction variance captured for the states data to make a good new default value.

Right now even with 5 components the accuracy results are only a little bit worse than full saturated logistic regression.

outcome,from_year,for_years,gwcode,p_up,p_same,p_down
v2x_freexp_altinf,2011,2012 - 2013,2,0.020803108535437917,0.4791968914645621,0.5

Why is it always 0.5?

Make RF tune itself, or at least provide some way to obtain info on tuning from a server run. The current version uses all out of the box default settings.

To help with debugging, and especially checking how many parameters were retained. Some models are still essentially 0 covariate models. I would like to see how far back one has to scale lambda to change this.

glmnet::cv.glmnet() includes the minimum cost lambda value as well as the smallest lambda within 1 CV SE of that. Add a less conservative 0.5 CV SE option. See glmnet:::getmin():

function (lambda, cvm, cvsd) 
{
    cvmin = min(cvm, na.rm = TRUE)
    idmin = cvm <= cvmin
    lambda.min = max(lambda[idmin], na.rm = TRUE)
    idmin = match(lambda.min, lambda)
    semin = (cvm + cvsd)[idmin]
    idmin = cvm <= semin
    lambda.1se = max(lambda[idmin], na.rm = TRUE)
    list(lambda.min = lambda.min, lambda.1se = lambda.1se)
}

The relevant code needs to go in reg_logreg().

See #2, might help with that.