Comments (3)
Hi Steven. Well, you can potentially do this.
there are 2 factory methods for kinesis fs2 steams
The default one is most easy to use, you pass all predefined required clients, and you are ready to stream. However, the ConfigBuilder is hidden from you, and you cannot provide the multi-steam config as the library user.
The second one was created mostly because we needed to test our implementation, and at the time we started this library, there were scarce resources for mocking kinesis SDK, especially to mimic such cases as shard ends, etc.
So you can potentially use the second factory method and make your customization.
However, we need to spend more time figuring out how to make it more user-friendly.
from fs2-aws.
@stevebakh hi Steve, here is new proposal on how to build kinesis streaming #1030
from fs2-aws.
Awesome, thanks Dmytro, and apologies for the radio silence! Will take a closer look at the new solution - we're currently just creating two consumers, each with their own lease table. Looking forward to seeing how we can integrate the above.
from fs2-aws.
Related Issues (20)
- Remove DynamoDB streaming support HOT 5
- Kinesis validates the AWS credentials and fail with fake ones (LocalStack) HOT 2
- Expose AWS GetObjectResponse when downloading files
- Vulnerability in netty-nio-client HOT 3
- S3 multipart uploads seem difficult to verify HOT 3
- Release with AWS SDK 2.21 HOT 1
- S3 Multipart Upload, no way to add tags HOT 3
- limitations: stream upload (for small files)
- S3 Invalid Range 416 error when trying to read empty directory HOT 5
- README is out of date for SQS example HOT 1
- Kinesis - Handle exceptions and keep consuming messages HOT 2
- Where are the artifacts? HOT 3
- Artifact not resolving (5.x & 4.x) HOT 1
- SqsConfig class is misplaced, it seems
- SQS default fetch message count is greater than max allowed HOT 2
- S3.create returns `F[S3[F]]` when it could just be pure? HOT 1
- Remove fs2-aws-ciris
- Rework KinesisSettings HOT 1
- Dynamodb StreamScan hangs forever if a client error happens HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fs2-aws.