Using `copy.deepcopy` on storage causes data loss. about nicegui HOT 4 CLOSED

Very interesting, @natankeddem!

Here is an example with just a PersistentDict:

import copy
from pathlib import Path
from nicegui import observables, storage, ui

@ui.page('/')
def index():
    d = storage.PersistentDict(Path('tmp.json'))
    d['a'] = {'b': 0}
    copy.deepcopy(d['a'])
    ui.timer(0.1, lambda: print(Path('tmp.json').read_text()), once=True)

I think the deepcopy function copies some internals of the underlying ObservableDict, so that the change handler is triggered during the copy operation, which writes an empty dictionary to the JSON file.

Here is another example demonstrating how on_change is called with an empty "a" attribute:

from nicegui import observables

d = observables.ObservableDict({'a': {'b': 0}}, on_change=lambda sender: print('changed:', sender))
print('copying...')
copy.deepcopy(d['a'])

copying...
changed: ObservableChangeEventArguments(sender={'a': {}})
changed: ObservableChangeEventArguments(sender={'b': 0})

from nicegui.

It took me a bit to figure out what kept on deleting my storage. Then once I saw it was the deepcopy I guessed something similar to what you suggested but didn't dig too deep since I was trying to get some other things working on my project. To accomplish the deepcopy I ended up using the json.loads(json.dumps(app.storage.general["key"])) trick.

from nicegui.

It looks like we need to overwrite the __copy__ and __deepcopy__ methods of our observable collections:
https://docs.python.org/3/library/copy.html

from nicegui.

Thank you!

from nicegui.