s3_sync_meta issue about django-s3-storage HOT 12 CLOSED

Ah, I think you're using django-s3-storage 0.9.9, and have just been hit by an unintended consequence of the fix for #18.

Try the master branch. I've just pushed up what is almost certainly a fix for your problem.

from django-s3-storage.

yep, fixed.

from django-s3-storage.

Cool, I'll get a new release out later today.

from django-s3-storage.

0.9.10 is out

On Wed, 29 Jun 2016 at 22:34 David Binetti [email protected] wrote:

yep, fixed.

—
You are receiving this because you commented.

Reply to this email directly, view it on GitHub
#24 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AAJFCMtd7UfkKVgMXe63R8m7ptjkjBYBks5qQuTMgaJpZM4JAZ58
.

from django-s3-storage.

Might be a regression here. On 0.11.0

~ $ django-admin s3_sync_meta django.core.files.storage.default_storage
Syncing meta for django.core.files.storage.default_storage
Traceback (most recent call last):
  File "/app/.heroku/python/bin/django-admin", line 11, in <module>
    sys.exit(execute_from_command_line())
  File "/app/.heroku/python/lib/python2.7/site-packages/django/core/management/__init__.py", line 367, in execute_from_command_line
    utility.execute()
  File "/app/.heroku/python/lib/python2.7/site-packages/django/core/management/__init__.py", line 359, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/app/.heroku/python/lib/python2.7/site-packages/django/core/management/base.py", line 294, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/app/.heroku/python/lib/python2.7/site-packages/django/core/management/base.py", line 345, in execute
    output = self.handle(*args, **options)
  File "/app/.heroku/python/lib/python2.7/site-packages/django_s3_storage/management/commands/s3_sync_meta.py", line 30, in handle
    for path in storage.sync_meta_iter():
  File "/app/.heroku/python/lib/python2.7/site-packages/django_s3_storage/storage.py", line 333, in sync_meta_iter
    obj = self.meta(name)
  File "/app/.heroku/python/lib/python2.7/site-packages/django_s3_storage/storage.py", line 32, in _do_wrap_errors
    raise IOError("S3Storage error at {!r}: {}".format(name, force_text(ex)))
IOError: S3Storage error at '.': An error occurred (404) when calling the HeadObject operation: Not Found
~ $ 

from django-s3-storage.

There will hopefully be a fix for this in master.

I think that you have a key called "." in your bucket, which is causing the key name normalization to break. I've made django-s3-storage stop exploding when it encounters this, but you should probably get rid of it.

from django-s3-storage.

from django-s3-storage.

from django-s3-storage.

from django-s3-storage.

Any chance you could give me temporary access to the bucket so I can investigate? I'd imagine the answer is almost certainly not!

Otherwise, I can have a go at writing a script to try and find the problematic key.

from django-s3-storage.

from django-s3-storage.

Yes, I think an ad-hoc IAM role would do. If you're feeling paranoid, I can take a PGP email to [email protected] with the details (or a non-PGP email, if you're not feeling paranoid).

My intention is to write a read-only script to iterate over all the keys in the bucket, which will hopefully catch the offending key.

from django-s3-storage.