Error when initializing training about invoicenet HOT 9 CLOSED

It looks like it's not able to find any prepared data for training. What is the size of your training dataset? Did you prepare the data before training?

The most likely reason I can think of is that the size of your training dataset is smaller than the batch size you're using.

from invoicenet.

Thanks for the quick reply

I just wanted to test it real quick, so i only made a couple of test files for training:

Each JSON only contains 4 different values as well

And yes, i did prepare the data before testing:

from invoicenet.

Ah right, tensorflow dataloader throws that error since it is expected there would be some validation data. This is definitely an error that the user should at least be warned about, so thank you for pointing it out. For now, you need at least one sample for validation to satisfy the dataloader so having a dataset of at least (batch_size + 1) samples should work.

from invoicenet.

That did the job, thanks a lot.

One last question tho, is it normal for it to take this long with such a small batch? I've had it running for ~1 hour now and it's finished:

And i definitely don't have a bad pc.

Just curious if this is to be expected so i don't end up waiting a day for it to complete only for it to not be intended.

from invoicenet.

With a batch size of 4 I usually get about 0.9-1.0 batches/s on an Nvidia GTX 1050ti for an amount based field. For a general field, I get about 1.0-1.1 batches/s. It drops to about 0.65 and 0.45 respectively with a batch size of 8.

Do you have a GPU? Is CUDA 10.1 installed? I've written the setup.py script such that it automatically detects if CUDA is available and will install tensorflow-gpu if it is. Otherwise, it defaults to the tensorflow cpu bindings. Can you check the tensorflow logs to see if you're actually using the GPU if you have one?

from invoicenet.

That's my issue - i don't have a dedicated GPU on my work laptop. I just didn't expect it would be this rough on it given the limited amount of training data i threw at it, but if it's normal then i just need to be a bit patient

from invoicenet.

I am getting the same error, although I have more than 70 processed data in train and 20 in val. @naiveHobo

from invoicenet.

can you

tensorflow

please can you explain that how can i solve that, still facing same on macos using paython 3.7. @naiveHobo

from invoicenet.

training is not enough,
sorry, but I still didn't understand what data to insert for some validation data

from invoicenet.