Comments (7)
ivanyu
commented on July 21, 2024
1
@javierquevedo is this something you'd like to have: https://github.com/aiven/aiven-kafka-connect-s3/blob/383aa99faa87433548c8caac8b9ee639ce580d65/README.md#aws_s3_prefix ?
from s3-connector-for-apache-kafka.
javierquevedo
commented on July 21, 2024
Any plans to address this anytime soon?
from s3-connector-for-apache-kafka.
ivanyu
commented on July 21, 2024
@javierquevedo yep, I will be working on this.
from s3-connector-for-apache-kafka.
chasdevs
commented on July 21, 2024
Our company uses a prefix like so: {source}/{eventName}/{year}/{month}/{day}/{hour}. Would it be possible to prefix by year/month/day/hour?
from s3-connector-for-apache-kafka.
joekohlsdorf
commented on July 21, 2024
I see that this has now been implemented, could you tag a new release so that it becomes usable?
from s3-connector-for-apache-kafka.
ivanyu
commented on July 21, 2024
@chasdevs we're re-working the templating approach, it will be more like in the sibling GCS connector. It will give a greater degree of flexibility for configuring file names.
from s3-connector-for-apache-kafka.
ivanyu
commented on July 21, 2024
@joekohlsdorf 2.6.0 should have it
from s3-connector-for-apache-kafka.
Related Issues (20)
- file in s3 without return line for each event HOT 1
- file.name.prefix doesn't seem to work
- Random UnknownHostException resolving S3 hostname HOT 1
- Just hold latest 10 record of each key or just hold data of 10 minutes ago HOT 4
- The s3 file only has startoffset. How to obtain the endoffset HOT 2
- AWS ECS Task or (likely) EKS pod with pre-mounted IAM credentials unsupported
- Support dynamic fields partitioned without requiring Avro scheme
- There is an issue handling avro schema with default value defined as String HOT 2
- java.lang.NoClassDefFoundError HOT 1
- S3 Source Connector HOT 7
- Consumer offsets not committed when using KeyRecordGrouper
- Out-of-Memory errors when sinking large topics HOT 4
- Use environment variables for AWS Access Key ID/Secret Access Key Config
- master incompatible with latest cp-kafka-connect Docker image HOT 1
- AivenKafkaConnectS3SinkConnector should inherit from SinkConnector
- Published artifacts available for download? HOT 3
- does csv output require the input to be ByteArrayConverter ? HOT 1
- csv output with headers as 1st row HOT 2
- Delivery semantic HOT 1
- Issue on "Date Format" using the S3 Sink Connector from Avro to Parquet HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from s3-connector-for-apache-kafka.