Application for copying Amazon Kinesis data to S3.

1. Download the app
2. Configure your application (start with the template KinesisToS3.SAMPLE.properties)
3. Configure AWS access credentials:
   1. Locally, set either
      1. the environment variables AWS_ACCESS_KEY_ID=xxx and AWS_SECRET_KEY=yyy
      2. or the JVM options -Daws.accessKeyId=xxx and -D=aws.secretKey=yyy
   2. On Amazon EC2
      1. No configuration needed if you start the server instance with a proper IAM role (highly recommended!)
4. Run the app: java -jar kinesis-to-s3-[version].jar configuration.properties

Checkout the source using Git. Initialize the amazon-kinesis-connectors submodule using git submodule update --init --recursive

The main class is com.github.chenkirk.aws.KinesisToS3

Build with Maven. Running

mvn clean package

will create a runnable jar at ./target/kinesis-to-s3-[version].jar

Licensed under the MIT license.

Uses the Amazon Kinesis Client Library, the Amazon Kinesis Connector Library and the AWS SDK for Java. These are also embedded in the runnable jar. See the AWS licensing terms.

The Kinesis Connector sample code contains code for copying Kinesis data to S3, but is not easily reusable. This application contains the minimal code for the copying process and nothing else.