Giter Club home page Giter Club logo

flink-kotlin's Introduction

Kotlin support for Apache Flink

This package provides type information and specialized type serializers for kotlin data classes (including Pair and Triple), as well as for Kotlin Map and Collection types.

Installation

The package is available on Maven Central and Github, using Gradle:

implementation("com.lapanthere:flink-kotlin:0.1.0")

Usage

TypeInformation

Using the createTypeInformation that will return a Kotlin friendly TypeInformation for data classes, collections, maps, etc:

dataStream.process(
    processFunction(),
    createTypeInformation<Row>()
)

It also supports fields name in the definition of keys, i.e. you will be able to use name of fields directly:

dataStream.join(another).where("name").equalTo("personName")

You can also annotate your data classes with the @TypeInfo annotation:

@TypeInfo(DataClassTypeInfoFactory::class)
data class Record(
    val name: String,
    val value: Long
)

Schema evolution

Schema evolution for data classes follow this set of rules:

  1. Fields can be removed. Once removed, the previous value for the removed field will be dropped in future checkpoints and savepoints.
  2. New fields can be added. The new field will be initialized to the default value for its type, as defined by Java.
  3. Declared fields types cannot change.
  4. Class name of type cannot change, including the namespace of the class.
  5. Null fields are not supported.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.