sha1 or other hash before upload? about django-s3-storage HOT 6 CLOSED

On Sat, 19 Mar 2022 at 07:53, Brian ***@***.***> wrote: Do you know of a way I could calculate the sha1 hash of a file prior to upload? My old code (before moving files to s3) looked like this: def save(self, *args, **kwargs): super().save(*args, **kwargs) if(self.file): self.file_size = self.file.file.size with open(self.file.path, "rb") as f: file_hash = hashlib.sha1() while chunk := f.read(8192): file_hash.update(chunk) self.file_hash = file_hash.hexdigest() super().save() But obviously the part where it's reading the file from disk no longer works. Is there a way for me to intercept the file and calculate it? Alternatively, I notice that s3 has an "etag" that might be a hash I could use (I don't know how to tell what type of hash it is), will django-s3-storage give me access to that somehow? Thank you! — Reply to this email directly, view it on GitHub <#139>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABEKCC64DXH2AWVBOO36NTVAWBXLANCNFSM5RDVU2IA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

On Mon, 21 Mar 2022 at 04:11, Brian ***@***.***> wrote: It turns out the ETag is not a reliable source for a file hash. It changes not only depending on the content of the file, but also on how it was uploaded and what type of encryption was used. https://docs.aws.amazon.com/AmazonS3/latest/API/API_Object.html I'm back to looking for a way to intercept the file before it's uploaded to s3. Does it ever reside on disk anywhere? In a temp directory or something? Here's the working code I have to save the ETag, but it doesn't really solve the problem. def save(self, *args, **kwargs): super().save(*args, **kwargs) if(self.file): self.file_size = self.file.file.size conn = s3storage.s3_connection details = conn.head_object( Bucket=s3bucket, Key=str(self.file.file), ) self.file_hash = details.get("ETag").replace('"', '') super().save() — Reply to this email directly, view it on GitHub <#139 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABEKCC2H46TULDTEHOOR53VA7YTXANCNFSM5RDVU2IA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you commented.Message ID: ***@***.***>

On Sat, 19 Mar 2022 at 18:34, Brian ***@***.***> wrote: Is there a way to get it if I'm not using the s3 storage as the default? I'm actually only using it for a handful of models that allow upload of user content. I have like storage=s3storage on individual file fields in the model. Thank you! — Reply to this email directly, view it on GitHub <#139 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABEKCBZK4ESBDZDF4U5CCTVAYMKFANCNFSM5RDVU2IA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you commented.Message ID: ***@***.***>