o.e.s.storage.types.StorageChannelTask : Error occurred in storage channel#0org.eclipse.store.storage.exceptions.StorageExceptionIoWriting: Physical length 3047424 of current head file 146 is not equal its expected length of 3047612 about store HOT 7 OPEN

I tried with channel size 1, I have the same issue.

from store.

I got a similar error and once it is on the collected data is lost forever. Any idea what causes this? I took down a copy of the data from a live system using scp, could that have caused it (locked the file temporarily or something) 🤷‍♂️

from store.

@ghilainm:

Hello,
Unfortunately, we can’t reproduce this issue.
The StorageExceptionIoWriting exception indicates that the latests storage file is has not been written completely. The IndexBoundsException is similar, the storage tries to read data that is not present. Why the incomplete write happened is not visible in the provided trace.

from store.

@mstahv:
Copying an active storage is risky, as you must ensure that no write operations happen during the copy process. Even if no user code issues a store this can’t be ensured as the storage does cleanup operation if idle.
To copy a storage you need stop the storage or you can issue a full backup

from store.

@hg-ms This issue occurs to me all the time. I have stopped using eclipse store for this reason. The problem occurs on my laptop and on EC2 provisionned on AWS. Do you see any issue with the code I have provided? At least from a conceptual point of view?

Moreover, it gives the feeling that eclipse-store is either super fragile or that it can be easily used in a wrong way.

Maybe more documentation and snippets about how to use it properly would be useful? Reading the documentation I didn't find many example. Would be nice to have more use cases showcased.

from store.

Yeah, I guess so, at least for getting functional backup. In my case it is the live server that got this issue.

Another idea that might have affected this: I'm storing the data on a directory in the server and then that is mounted to the Docker container running the actual app. There might be two apps/containers accesing the same files during the update as there is "zero downtime update" configure for the system. If it is this that cause my issue, I need a different strategy (this has worked fine with RDBMSs).

I wonder still if the default is right here to throw the exeption, instead of logging some error and overriding? At least to some backup file. My case (where I'm storing weather station data for my hobby/learning project) not a biggie, but in some production system I might have lost a full day of entries (althogh store in server memory just fine), if EclipseStore would have overridden I assume would have only lost just one entry 🤷‍♂️

from store.

@mstahv:
More than one EclipseStore instance accessing the persisted data is not supported. This could explain your corrupted storage files. This is a major difference to RDBMS.

@ghilainm:
Regarding your code, I need to assume that the method createUser doesn’t modify the object graph in any way or does not do any call to EclipseStore so that there can’t be any concurrency problem, same for the class PerfData.
Looking at the store calls in the code snippet there is one unnecessary: store.storageManager.store(userDataCreated) only persists the userDataCreated object without any relation to the userData as the change (add) to user dataData is not persisted jet. This is done later, every 100th future. Here you persist the userData. The call storageManager.store(userData) perists the userData change and would also perist the referenced new UserDataCreated if not done before.

from store.