SDK's output io.Reader prevents SDK from retrying failed GET. about aws-sdk-go-v2 HOT 2 CLOSED

@jasdel What retry strategy would you suggest for this type of operation? Are there any other services in the SDK that use similar approach to the one you suggested?

from aws-sdk-go-v2.

Thanks for asking @fifarafa. The idea behind this alternative approach is to allow the SDK to wrap the concept of retrying an S3 GetObject download attempt. The C++ SDK is the only SDK that uses a similar approach to this alternative approach. With a few exceptions (Glacier), S3 is the only API with operations to download byte streams.

The retry strategy for this approach would require the user to provide a io.Writer that can be reset/reinitialized in the case the download failed, and the API does not support byte-range requests. There are two strategies that could occur during a retry. If the API supports byte-range requests, the SDK could resume from the last byte it wrote to the user provided io.Writer. Alternatively if the API does not support byte-range requests the SDK would have to start downloading the content from the beginning, and requiring some means of being able to reinitialize the writer for a fresh write.

Both of these approaches seem fragile, and error prone. After reviewing this alternative approach, I'm not convinced it is a good idea for the SDK to invert the way it exposes downloading a byte stream from a API operation, e.g. S3 GetObject. For the S3 GetObject specific usecase the SDK's s3manager.Downloader provides much more consistent behavior since it was tailor made for S3's GetObject. The Downloader has full support for retrying failed GetObject requests.

from aws-sdk-go-v2.