Add backend-agnostic worker-process data loading about keras HOT 8 CLOSED

Exposing the API provided in keras.utils.PyDataset to backends other than tensorflow would be a valuable addition.

PyDataset is usable with any backend. Nothing about it is TF-specific. Did you encounter an issue?

from keras.

Interesting, for me, the import does not work under the torch backend. I thought this was the intended behaviour:

conda create -y -n test-py-tensorflow python=3.11
conda activate test-py-tensorflow
pip install -U tensorflow keras
conda env config vars set KERAS_BACKEND=tensorflow
conda deactivate
conda activate test-py-tensorflow
python -c "from keras.utils import PyDataset"

works fine, whereas

conda create -y -n test-py-torch python=3.11
conda activate test-py-torch
pip install -U keras
conda install pytorch torchvision torchaudio cpuonly -c pytorch
conda env config vars set KERAS_BACKEND=torch
conda deactivate
conda activate test-py-torch
python -c "from keras.utils import PyDataset"

yields

ImportError: cannot import name 'PyDataset' from 'keras.utils'

Edit: Fixed the minimal example

from keras.

The import pattern should be from keras.utils import PyDataset or import keras; keras.utils.PyDataset.

from keras.

Sorry, that was a mistake I put in when creating a minimal example.

The issue seems to be a little bit deeper. When I install keras using conda install keras, I get the above issue:

from keras.utils import PyDataset
>>> ImportError: cannot import name 'PyDataset' from 'keras.utils'

When I install with pip install -U keras (as recommended by keras), I instead now get:

import keras
>>> ModuleNotFoundError: No module named 'packaging'

from keras.

Hi @LarsKue ,

Please install the packaging module using pip install packaging and try again.

from keras.

The library packaging is a dependency for Keras. Please check here.

from keras.

@SuryanarayanaY thanks, this fixes the issue. packaging should probably be auto-installed when running pip install -U keras though. Would also be nice to have the conda repos updated so that the latest torch-compatible keras version is newer than 3.1.0, which has the issue of the missing PyDataset.

from keras.

Are you satisfied with the resolution of your issue?
Yes
No

from keras.