Describe the bug Using a dataset with 500k rows and 27 features,

This worked for me, I was using Google collab 8GB RAM <div class="snippet-clipboar

Massive memory usage running the LazyClassifier about lazypredict HOT 3 OPEN

shankarpandala commented on May 18, 2024 1

Massive memory usage running the LazyClassifier

from lazypredict.

Comments (3)

felixvor commented on May 18, 2024 2

A workaround is to filter out high memory model architectures from the default regressors / classifiers list and to pass that custom list of models to the LazyRegressor / LazyClassifier. For example:

import lazypredict
from lazypredict.Supervised import LazyRegressor

highmem_regressors = [
    "GammaRegressor", "GaussianProcessRegressor", "KernelRidge", "QuantileRegressor"
]
regressors = [reg for reg in lazypredict.Supervised.REGRESSORS if reg[0] not in highmem_regressors]
reg = LazyRegressor(regressors=regressors, verbose=1, ignore_warnings=True, custom_metric=None)
models, predictions = reg.fit(X_train, X_test, y_train, y_test)

from lazypredict.

apostolides commented on May 18, 2024 1

Hello,

I have the same issue using a train dataset with 125K rows. I'm training the models on google colaboratory with12G ram available. Runtime crashes on 38% prompting a huge amount of allocated memory. Did you find any workarounds for this issue?

Thanks in advance.

from lazypredict.

dvijkalsi commented on May 18, 2024

This worked for me, I was using Google collab 8GB RAM

highmem_classifiers = ["LabelSpreading","LabelPropagation","BernoulliNB","KNeighborsClassifier", "ElasticNetClassifier", "GradientBoostingClassifier", "HistGradientBoostingClassifier"]

# Remove the high memory classifiers from the list
classifiers = [c for c in lazypredict.Supervised.CLASSIFIERS if c[0] not in highmem_classifiers]

clf = LazyClassifier(classifiers=classifiers, verbose=1, ignore_warnings=True, custom_metric=None)
models, predictions = clf.fit(X_train, X_test, y_train, y_test)
model_dictionary = clf.provide_models(X_train, X_test, y_train, y_test)
models

from lazypredict.

Recommend Projects