Is there a better way to save the model to the disk? about classy-classification HOT 8 CLOSED

Literally four lines of code 😁

data = {
    "furniture": ["This text is about chairs.",
               "Couches, benches and televisions.",
               "I really need to get a new sofa."],
    "kitchen": ["There also exist things like fridges.",
                "I hope to be getting a new stove today.",
                "Do you also have some ovens."]
}
classifier = classyClassifier(data=data)

with open("./classifier.pkl", "wb") as f:
    pickle.dump(classifier, f)

and then to load the model and use it:

f = open("./classifier.pkl", "rb")
classifier = pickle.load(f)
print(classifier("I am looking for kitchen appliances."))

from classy-classification.

#4

from classy-classification.

There are ways to save and load these kind of models, but I have not gotten around to properly implementing this yet, since it will take up significant time, and I didn't have too much spare time the past few months.

from classy-classification.

As a work-around, you can use the stand alone usage and simply safe the sklearn classifier as a separate component.

from classy-classification.

Let me know if this makes sense and works for you. If so, it might be an easy hot-fix to include the feature of passing such a classifier within the spacy pipeline along with the embedding model.

from classy-classification.

I will give it a try in a couple of hours and let you know. Thanks a lot for your reply for now 😊

from classy-classification.

Hi! Thank you again for your help. I finally solved the problem following your advice of using the standalone usage. I saved the entire classifier object with pickle.dump() and retrieved afterward with pickle as well. It only takes a couple of seconds to load the model.
This library is pure gold btw 😍

from classy-classification.

from classy-classification.