Custom Fetcher with high memory usage about carlos HOT 7 CLOSED

Hi @mufumade, thank you for reaching us out!

I think we need more information about the way you use Carlos to be able to be able to identify the exact problem. Could you please answer on the following questions:

1. How big are the images?
2. From where do you execute your image downloading requests? Is it in init or onAppear in your SwiftUI view?
3. Do you cancel redundant requests?
4. Did you try to limit the memory cache capacity? I believe, by default the limit is 50MB.

from carlos.

Hey thanks for taking you time for my problem!

1. All images are 500x500 px so not that large
2. See in the updated issue
3. I hope that canceling the publisher is enough?
4. Soo in the updated issue you see that I directly interface with my S3CacheFetcher that's because when the images are already cached on the disk that I do not have a memory issue. I declared the cache static somewhere in my app as well with and without a memory or disk cache but when the image has to be downloaded the memory fills up and does not get released.

I hope you have all the information you need. And thanks again!

from carlos.

@mufumade I don't see any "obvious" problems in your code. I would suggest to limit MemoryCacheLevel capacity to a smaller value (e.g. 20MB instead of default 50MB) and see if this helps. In the time being, I will investigate problem a bit deeper. Thanks!

from carlos.

@ivanlisovyi I have some updates for you. As you can see am decrypting my images after I downloaded them.

 }else if let data = data{
           if let imageData:Data = S3CacheFetcher.decryptImage(with: data, mediaItem: mediaItem){
              promise(.success(imageData as NSData))
           }
 }

To Decrypt I am using CryptoKit.AES.GCM.
For some weird reason the memory goes up when I am decrypting data there.

....
if let error = error{
    print(error)
    promise(.failure(error))
}else if let data = data{
    
    // Low memory usage
    //promise(.success(data as NSData))
    // ---
    
    // High memory usage
    let encrypt = try! AES.GCM.seal(data, using: S3CacheFetcher.symmetricKey).combined!
    let decrypt = try! AES.GCM.open(AES.GCM.SealedBox(combined: encrypt), using: S3CacheFetcher.symmetricKey)
    promise(.success(decrypt as NSData))
    // ---
}
....

I have no idea why.. I am using the ConditionedOneWayTransformationBox to decrypt images after the fetching cache level. This seams to work now.

from carlos.

@mufumade so, at the point where you resolve the promise you have your image data stored in memory 3 times: original image data you received from S3, encrypted data after seal call and decrypted data after open call. You can reduce peak memory usage by using autoreleasepool

autoreleasepool {
    let encrypt = try! AES.GCM.seal(data, using: S3CacheFetcher.symmetricKey).combined!
    let decrypt = try! AES.GCM.open(AES.GCM.SealedBox(combined: encrypt), using: S3CacheFetcher.symmetricKey)
    promise(.success(decrypt as NSData))
}

from carlos.

Yes I tried that but the main problem I had is that the memory does not get released. And by not released I mean for ever. The memory usage just keeps climbing and climbing. What I would expect is that the memory only holds one image that is shown on the screen and getting released after the image disappears.

from carlos.

MemoryCache will hold up to 50MB indefinitely by default. This is exactly the reason why I've suggested to limit MemoryCache capacity in one of my previous comments.

from carlos.