ohnosequences / aws-scala-tools Goto Github PK

At the moment there are no tests (there are some outdated commented ones, but...)
After #38.

I tried, but the code doesn't compile. Mostly because old DynamoDB API was deprecated in 1.8 and is not included in 1.9.
@evdokim please update this

@evdokim

• latest git tag here is v0.2.3.
• version.sbt was updated last time 6(!) months ago and contains that 0.2.3
• github releases page latest tag is 0.2.9 (yes, without v)
• version key in build.sbt is set to 0.4.0-SNAPSHOT
• and finally in nispero I found dependency on v0.4.2.

Not nice and very confusing 😕
Do I need to be a Sherlock Holmes to find what is the actual version of this library?

@eparejatobes @laughedelic
We should decide which type we will use for content of requests in saws-sign library.

InputStream

Widely used in AWS SDK, dispatch, async-http-client. Worth thing from my point of view because if you pass it to other method, it should call reset on it in the end (otherwise we will send not all body of requests)

Iterator[Byte], Traversable[Byte]

It is possible to wrap InputStream to something more safe, for example to Traversable[Byte] that will be reset internal InputStream after each foreach(). Such wrappers have been already implemented in scalax.io.

Array[Byte]

Most safe approach, ant manipulations with this array can't change original body of request. But it has limitation: we should store in memory whole body of request.
Actually amazon guys do it for S3 signing:

// com.amazonaws.auth.AWS3Signer

    protected byte[] getBinaryRequestPayloadWithoutQueryParams(Request<?> request) {
        InputStream content = getBinaryRequestPayloadStreamWithoutQueryParams(request);

        try {
            content.mark(-1);
            ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
            byte[] buffer = new byte[1024 * 5];
            while (true) {
                int bytesRead = content.read(buffer);
                if (bytesRead == -1) break;

                byteArrayOutputStream.write(buffer, 0, bytesRead);
            }

            byteArrayOutputStream.close();
            content.reset();

            return byteArrayOutputStream.toByteArray();
        } catch (Exception e) {
            throw new AmazonClientException("Unable to read request payload to sign request: " + e.getMessage(), e);
        }
    }

this code put all body of request to byteArrayOutputStream.

Thus I suggest so far use Array[Byte], but it is unnecessary to consider big requests and to get know limitations for requests. I think that for some services (SQS for e.g.) request size are limited.

• https://github.com/Rhinofly/play-aws-utils - looks, uses Play WS client
• https://github.com/n8han/dispatch-aws-s3

I had to manually publish snapshots from here etc, it was needed in ohnosequences/compota

This will allow to depend only on those services that are used. Otherwise one dependency on aws-scala-tools brings dependency on all these services:

• autoscaling
• ec2
• s3
• sns
• sqs

@evdokim, clarify, please, what is the latest released version of tools?
I see that grid uses 0.2r6, but here I see commits only up to 0.2r5. And what does that rX mean?

• better imports and shorter qualified names (use .{Smth => Smth} imports)
• Option values where needed (publicDNS for example)
• return types for all methods
• Instance subclass in EC2
• remove empty parens def foo()

The current stable release is based on aws-java-sdk 1.10.34. (see 1e17b4c)
And here is what happens in runtime every time when a S3/EC2 client is created:

Exception in thread "main" java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.ObjectMapper.enable([Lcom/fasterxml/jackson/core/JsonParser$Feature;)Lcom/fasterxml/jackson/databind/ObjectMapper;
    at com.amazonaws.internal.config.InternalConfig.<clinit>(InternalConfig.java:43)
    at com.amazonaws.internal.config.InternalConfig$Factory.<clinit>(InternalConfig.java:304)
    at com.amazonaws.util.VersionInfoUtils.userAgent(VersionInfoUtils.java:139)
    at com.amazonaws.util.VersionInfoUtils.initializeUserAgent(VersionInfoUtils.java:134)
    at com.amazonaws.util.VersionInfoUtils.getUserAgent(VersionInfoUtils.java:95)
    at com.amazonaws.ClientConfiguration.<clinit>(ClientConfiguration.java:53)
    at com.amazonaws.services.ec2.AmazonEC2Client.<init>(AmazonEC2Client.java:153)
    at ohnosequences.awstools.ec2.EC2$.create(EC2.scala:361)
...

We need to downgrade to 1.10.32 (which doesn't have such problem) and release M2.

A relevant issue: aws/aws-sdk-java#555