KeyError: 'storageFilename' about google-photos-deduper HOT 8 CLOSED

storageFilename is set by a separate subtask that downloads the actual thumbnail images for performance purposes. Right now, if one of those subtasks fails it'll still proceed, but then run into this error, possibly others too, when calculating duplicates.

Two options I see to address:

1. Skip invalid media items during processing, logging a warning.
2. Fail the main task if a subtask fails, prompt the user to start again.

I'm leaning toward Option 2 for now, and will make that change soon. My reasoning is it's also what I'm doing when daily quota is exceeded, because if 1 failed it's likely that others may have failed as well, and the application is already optimized to skip over media items where the thumbnail has already been downloaded.

This will also benefit the project as I can get a better sense of why the subtasks are failing and not setting storageFilename, and better address the root issue.

from google-photos-deduper.

@olsw I plan to find some time over the holidays, so by end of year.

from google-photos-deduper.

So GPT suggest this:

def _get_storage_path(self, media_item) -> str:
    if "storageFilename" in media_item:
        return self.image_store.get_storage_path(media_item["storageFilename"])
    else:
        self.logger.error(f"'storageFilename' not found in media_item: {media_item}")
        return None  # Or handle this case differently, perhaps raising a custom exception

It worked, and now have a screen full of duplicates :)
Going to leave this open though, as it's probably better to fix for all.

from google-photos-deduper.

Getting the same error (KeyError: 'storageFilename') basically. I can see that some media_items have no storageFilename in their dictionary. The solution suggested by @bighippo999 does not work for me since returning None throws an error later in the game. I think that the root cause needs to be fixed where the media_items get populated and there must be a reason why there is no storageFilename saved for some. But I haven't figured out where that happens, nor why it does (not) happen yet... any help is appreciated...

from google-photos-deduper.

@mtalcott any indication when this will be released? Many thanks

from google-photos-deduper.

I think im having the same issues, and im hoping you can fix them soon.

Edit:
Welllll, something that worked for me was using Google Chrome instead of FireFox!
I installed the extention before starting program aswell. Hopefully this will help someone else aswell.

Thanks alot @mtalcott this app is awesome, got to remove so many dupe's
The only downside is that i need to shutdown my own Nginx container to get this running. But still awesome!!!!

from google-photos-deduper.

I've merged #32 which should help identify the root issue(s) at play here. Please pull the latest from main branch to get the update.

A new log/celery_worker.log will be created, I'd appreciate it if anyone is willing to share a relevant stacktrace from theirs here.

from google-photos-deduper.