[BUG] Movielens not working in python 3.7 due to Pandera library about recommenders HOT 15 CLOSED

This also affects Python 3.8.

from recommenders.

Also here

from recommenders.

But maybe the typing dependency is the problem.

from recommenders.

Thanks, so pandera needs to handle this. We need to remove the dependency then.

from recommenders.

from recommenders.datasets import movielens

I could only replicate the issue in 3.7, not in 3.8. @john0isaac what machine did you use? mine is a Standard_E4ds_v4 (4 cores, 32 GB RAM, 150 GB disk). What did you do to get this error? I tried

conda create -n foo python=3.8
conda activate foo
pip install recommenders
python
Python 3.8.18 (default, Sep 11 2023, 13:40:15) 
[GCC 11.2.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from recommenders.datasets import movielens
>>> 

from recommenders.

Thanks @john0isaac I don't have quota for this type of machine, I am afraid. I should make a commit for Python 3.8 too. What happens if you try with higher Python versions on that machine?

from recommenders.

@anargyri It's serverless doesn't consume lots of money and enables the spark jobs don't worry about quota.

As for your question as this is a preconfigured compute there are only two options python 3.8 or python 3.10 and python 3.10 is not supported by the recommenders package so, you need to fix this in order for this environment to work on Azure ML.

from recommenders.

I am fixing this in a new PR. Until we release the fix on PyPI, one way to bypass the issue is after you have done pip install recommenders you do a pip install "pandera[strategies]>=0.6.5,<0.18"
This should uninstall the latest version of pandera (which is causing the error) and install the previous one.

from recommenders.

@loomlike it seems we only use pandera when creating a fake movielens dataset, do you know why we are using that dependency and if we can drop it?

from recommenders.

FYI @SimonYansenZhao @anargyri more problems with dependencies

from recommenders.

This is strange because they say that pandera requires Python >=3.7

from recommenders.

@anargyri the error @miguelgfierro is sharing is from Python 3.7.17 so technically it is >= 3.7.

Yes, tracing the error goes back to the typing thing.
there is a stalk overflow post about two solutions to this problem. https://stackoverflow.com/questions/77247446/cannot-import-name-self-from-typing

from recommenders.

This is the original commit that introduced pandera btw fd33efe

from recommenders.

@anargyri

Serverless Spark compute on Azure ML

This is the machine config: 4 vCPU, 32 GB memory, 64 GB disk

Python 3.8, Scala 2.12.15, Java 1.8.0_282, .Net Core 3.1, .Net for Apache Spark 2.0, Delta Lake 1.2

from recommenders.

Thanks

from recommenders.