will it work for logistic regression? about autofeat HOT 12 CLOSED

@meddulla The regular feature selection in the AutoFeatRegression model still works, but it's more likely that you'll get a numerical error (as described in #1 ); besides that the results should still be reasonable.

from autofeat.

for now you can use the model to generate additional features (using fit_transform), which you can then use together with a classification model of your choice. for native support for classification model you have to wait for the next version

from autofeat.

for now you can use the model to generate additional features (using fit_transform), which you can then use together with a classification model of your choice
Can you share example code to make it clear how to do this

from autofeat.

here you go:

# import the model
from autofeat import AutoFeatRegression
# create a model instance
model = AutoFeatRegression()
# call fit transform with your features and labels (y needs to be a vector)
df = model.fit_transform(X, y)
# df is a pandas dataframe with your original and the new features, 
# so you can use it with your classification model
clf = LogisticRegression()
clf.fit(df, y)  # new features instead of original X

from autofeat.

Great thank
And what is about features interaction
Is your code takes it to account or consider features as independent each from others?

from autofeat.

By the way did you compared with
https://github.com/aspremon/NaiveFeatureSelection

from autofeat.

thanks for the link. no, I did not compare against this (since I was mostly focused on regression problem so far) but I'll look into it.
and yeah, one of the biggest problems in the feature selection is that many of the newly created features are highly correlated, which is why the feature selection process has multiple steps to deal with this. if the features were all independent a simple L1-regularized model (like LASSO) would get pretty good results by itself already even if you have more features than data points, but with correlated features, this doesn't work.

from autofeat.

I glad you working on this
but in your code do you treat this problem of features correlation
for both categorical and continues features ?
and if yes
do you return not only features but also
group of correlated features ?

from autofeat.

categorical features are returned as one-hot encoded vectors if you specified the columns when initializing the model.
and no, correlated features are not grouped, however, generally if they are very correlated, only one of the features should be selected as a correlated one does not bring a lot of additional information.

from autofeat.

Can FeatureSelector be used in this instance or is it better to use another feature selector afterwards (like an SGDClassifier with RFECV)?

from autofeat.

Hadn't noticed that feature selection is already run on the generated features using Lasso.

from autofeat.

As of version 1.0.0, there are two autofeat models available, AutoFeatRegressor and AutoFeatClassifier for regression and classification problems respectively, so please update with pip install --upgrade autofeat and use these models instead. The arguments for the models are mostly the same so you shouldn't have to change your code much.

from autofeat.