Different machine learning methods? about eo-learn HOT 6 CLOSED

Hello,

Referring to this notebook, all you'd need to do is to:

• import the model you prefer
• initialise the model in cell 24 and replace current LGBM initialisation

For instance for SVM:

from sklearn.svm import SVC

# in cell 24
# Set up the model with default parameters
model = SVC()

For MLP

from sklearn.neural_network import MLPClassifier

# in cell 24
# Set up the model with default parameters
model = MLPClassifier()

Nothing else would need changes.

from eo-learn.

Hey Devis,
Thank you for the reply
I tried this but unfortunately got the following error:

_ValueError Traceback (most recent call last)
in

/anaconda3/lib/python3.7/site-packages/sklearn/svm/base.py in fit(self, X, y, sample_weight)
147 X, y = check_X_y(X, y, dtype=np.float64,
148 order='C', accept_sparse='csr',
--> 149 accept_large_sparse=False)
150 y = self._validate_targets(y)
151

/anaconda3/lib/python3.7/site-packages/sklearn/utils/validation.py in check_X_y(X, y, accept_sparse, accept_large_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, multi_output, ensure_min_samples, ensure_min_features, y_numeric, warn_on_dtype, estimator)
754 ensure_min_features=ensure_min_features,
755 warn_on_dtype=warn_on_dtype,
--> 756 estimator=estimator)
757 if multi_output:
758 y = check_array(y, 'csr', force_all_finite=True, ensure_2d=False,

/anaconda3/lib/python3.7/site-packages/sklearn/utils/validation.py in check_array(array, accept_sparse, accept_large_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, ensure_min_samples, ensure_min_features, warn_on_dtype, estimator)
571 if force_all_finite:
572 _assert_all_finite(array,
--> 573 allow_nan=force_all_finite == 'allow-nan')
574
575 shape_repr = _shape_repr(array.shape)

/anaconda3/lib/python3.7/site-packages/sklearn/utils/validation.py in _assert_all_finite(X, allow_nan)
54 not allow_nan and not np.isfinite(X).all()):
55 type_err = 'infinity' if allow_nan else 'NaN, infinity'
---> 56 raise ValueError(msg_err.format(type_err, X.dtype))
57
58

ValueError: Input contains NaN, infinity or a value too large for dtype('float64').
_

from eo-learn.

OK, I forgot about the NaN values :). The two classifiers do not deal with NaNs directly as LightGBM does. You would need to replace NaNs with some value using an imputer. Different methods are available and will likely affect your results.

This should fix the issue:

from sklearn.impute import SimpleImputer

# use mean imputation
# add on cell 24 before model fitting
imputer = SimpleImputer()
features_train = imputer.fit_transform(features_train)
features_test = imputer.fit_transform(features_test)

and replace the prediction task

class PredictPatch(EOTask):
    """
    Task to make model predictions on a patch. Provide the model and the feature, 
    and the output names of labels and scores (optional)
    """
    def __init__(self, model, imputer, features_feature, predicted_labels_name, predicted_scores_name=None):
        self.model = model
        self.imputer = imputer
        self.features_feature = features_feature
        self.predicted_labels_name = predicted_labels_name
        self.predicted_scores_name = predicted_scores_name
        
    def execute(self, eopatch):
        ftrs = eopatch[self.features_feature[0]][self.features_feature[1]]
        
        t, w, h, f = ftrs.shape
        ftrs = np.moveaxis(ftrs, 0, 2).reshape(w * h, t * f)
        ftrs = imputer.fit_transform(ftrs)        

        plabels = self.model.predict(ftrs)
        plabels = plabels.reshape(w, h)
        plabels = plabels[..., np.newaxis]
        eopatch.add_feature(FeatureType.MASK_TIMELESS, self.predicted_labels_name, plabels)
        
        if self.predicted_scores_name:
            pscores = self.model.predict_proba(ftrs)
            _, d = pscores.shape
            pscores = pscores.reshape(w, h, d)
            eopatch.add_feature(FeatureType.DATA_TIMELESS, self.predicted_scores_name, pscores)
        
        return eopatch

and

# replace on cell 38
# TASK FOR PREDICTION
predict = PredictPatch(model, imputer, (FeatureType.DATA, 'FEATURES'), 'LBL_GBM', 'SCR_GBM')

from eo-learn.

Thank you very much!
It works now, but I do however still encounter one problem I was hoping you might have an idea on how to get around. Regarding the part with the ROC curves and AOC metric I get the following error:

`class_labels = np.unique(np.hstack([labels_test, labels_train]))
​
scores_test = model.predict_proba(features_test)
labels_binarized = preprocessing.label_binarize(labels_test, classes=class_labels)
​
fpr = dict()
tpr = dict()
roc_auc = dict()
​
for idx,lbl in enumerate(class_labels):
fpr[idx], tpr[idx], _ = metrics.roc_curve(labels_binarized[:, idx], scores_test[:, idx])
roc_auc[idx] = metrics.auc(fpr[idx], tpr[idx])

AttributeError Traceback (most recent call last)
in ()
1 class_labels = np.unique(np.hstack([labels_test, labels_train]))
2
----> 3 scores_test = model.predict_proba(features_test, probability=True)
4 labels_binarized = preprocessing.label_binarize(labels_test, classes=class_labels)
5

~/anaconda3/lib/python3.6/site-packages/sklearn/svm/base.py in predict_proba(self)
588 datasets.
589 """
--> 590 self._check_proba()
591 return self._predict_proba
592

~/anaconda3/lib/python3.6/site-packages/sklearn/svm/base.py in _check_proba(self)
555 def _check_proba(self):
556 if not self.probability:
--> 557 raise AttributeError("predict_proba is not available when "
558 " probability=False")
559 if self._impl not in ('c_svc', 'nu_svc'):

AttributeError: predict_proba is not available when probability=False`

So obviously there is a problem with the SVM setting probability to false. How would you proceed from here?

from eo-learn.

Please check SVC docs. Retrain setting probability=True, as AttributeError suggests :)

model = SVC(probability=True)

from eo-learn.

Thanks a lot.
It's all working now and I can compare the accuracies of the different implemented models.

from eo-learn.