software.amazon.awssdk.core.exception.SdkClientException: Unable to execute HTTP request: Write timed out(short issue description) about aws-sdk-java-v2 HOT 3 CLOSED

@mubasherhussain9d

Tuning the timeout values is the proper solution in this case.

The SDK uses default timeout values that will work on most of the use cases. At the same time, we expose client configurations that you can customize to fit your specific use case, like running in an environment with high latency connectivity.

We suggest you generate the SDK client-side metrics to have an insight of the client performance in its own application environment. The metrics can be used to define what's the best timeout value to set in the client. For instructions on how to generate the metrics, check the Developer Guide and the blog post.

from aws-sdk-java-v2.

The timeout tuning mechanism is the viable solution for now but not the reliable solution, let me explain why;

there could be files of different sizes to be uploaded, so setting a fixed timeout is not a reliable solution. We have checked the different timeout values on files with different sizes, let's say, timeout = 1 min is fine for file sizes up-to 20MB or 25MB, but the very same timeout value is not fine for a file having size more then 25MB as the error occurs stating that Write timeout as mentioned above. So, how can we define a timeout value that works for all file sizes ? or, is there any other way around to manipulate the write timeout value at runtime while file is uploading, according the the internet speed and file size ?

from aws-sdk-java-v2.

A timeout that will work for all file sizes is a higher timeout. You can use AwsRequestOverrideConfiguration to set some timeouts at the request level, but not all timeout settings are available. If you don't want a timeout that is too high for the smaller files, you can create different clients with different settings: one s3 client with lower timeouts for the small files, and another s3 client with higher timeouts for the bigger files.

I still highly recommend to generate the SDK client-side metrics. You can then interpret the SDK metrics of your client to help define better custom settings.

We do have two blog posts that show how to interpret the client-side metrics to improve resiliency of your application (note that the blog posts show Java SDK v1 code, but the theory behind is the same, and most of the v1 metrics have v2 equivalents):

• Tuning AWS Java SDK HTTP request settings for latency-aware Amazon DynamoDB applications - link
• Tuning the AWS SDK for Java to Improve Resiliency - link

Out of curiosity, have you tried using the S3CrtAsyncClient in the S3TransferManager, instead of NettyNio? It would be interesting to see how the S3 CRT client performs in comparison to Netty in this case.

from aws-sdk-java-v2.