[DataPipe] Ensure all DataPipes Meet Testing Requirements about data HOT 5 OPEN

When we have time, we might need to go over our DataPipes again to identify any missing test since there are a few DataPipe implemented recently.

Besides, for future reference, we might need to improve our testing framework to something similar to OpInfo in PyTorch Core to run the testing coverage automatically without we go over each test by ourselves.

from data.

When we have time, we might need to go over our DataPipes again to identify any missing test since there are a few DataPipe implemented recently.

Besides, for future reference, we might need to improve our testing framework to something similar to OpInfo in PyTorch Core to run the testing coverage automatically without we go over each test by ourselves.

Agreed that the OpInfo-like way is probably the best. I think our inputs and necessary setup for each test is a bit all over the place. Having tests split between two repos doesn't help either.

from data.

This is awesome. One nit note: serializable should be same as picklable IMO.

from data.

@NivekT
I am concerning about when and how we want to do graph testing. For a single DataPipe instance, the graph testing makes no sense. Then, we may want to construct a datapipe graph by ourselves. The problem is how we can guarantee the testing coverage for all use cases.

from data.

We can require each DataPipe to introduce a simple usage example graph for this purpose.

from data.