eventuate-tram / eventuate-tram-docs Goto Github PK

This is the manual for the Eventuate Tram and Eventuate Tram Saga projects

Home Page: http://eventuate.io/docs/manual/eventuate-tram/latest/

License: Other

eventuate-tram-docs's Introduction

Eventuate Tram Documentation

This is the manual for the Eventuate Tram and Eventuate Tram Saga projects.

Eventuate Tram is a platform for managing distributed data in a microservice architecture. It implements the Saga pattern, which implements transactions multiple services, and the CQRS pattern, which implements queries that retrieve data from multiple services. Eventuate Tram enables a services to send and receive messages as part of an database transaction. It does this by implementing the Transactional outbox pattern. This pattern is foundation of maintaining data consistency within a microservice architecture.

You can build the HTML documentation by running ./gradlew asciidoctor.

eventuate-tram-docs's People

Contributors

Stargazers

Watchers

Forkers

dartartem microservice-hub vhussein vibhupbazact dartvandru zscomehuyue guangdliu ossgroupp

eventuate-tram-docs's Issues

Document finer-grained transport configuration

Currently, the docs describe classes such as TramJdbcKafkaConfiguration which bundle together producer (JDBC-based) and consumer (whatever message broker). CQRS services, for example, only use the consumer and so use currently undocumented configuration classes.

Document how to configure Kafka topics

See https://eventuate-users.slack.com/archives/C3L9MNMQC/p1589999456102900

I assume that you are using Kafka. You can specify the default at the broker level: http://kafka.apache.org/documentation/#num.partitions or you can create/reconfigure a topic with the desired number using kafka-topics.sh. For example, see this answer on stack overflow: https://stackoverflow.com/questions/22152269/how-to-specify-number-of-partitions-on-kafka-2-8

Document EVENTUATE_CDC_READER_READER1_MONITORINGSCHEMA

eventuate-foundation/eventuate-cdc#74

Document message handlers and transaction management

Along with SqlTableBasedDuplicateMessageDetector etc

Document how to customize JSON mapping

For example:

  static {
    JSonMapper.objectMapper.registerModule(new JavaTimeModule());
    JSonMapper.objectMapper.disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS);
  }

Document MessageHandlerDecorator

Add CDC container configuration examples

it would be good to have examples to copy/paste

Simple - single pipeline - different databases: MySQL, Postgres WAL, MS SQL server
Complex - multiple pipelines

Document cleanup of message and received_messages table

Describe how a user needs to setup a "cronjob" to periodically DELETE rows from tables:

message
received_messages

Document CommandWithDestinationBuilder.withExtraHeaders()

See eventuate-tram/eventuate-tram-sagas@d52b5af

Document Kafka Consumer configuration properties

See https://eventuate-users.slack.com/archives/C3L9MNMQC/p1589374364098900

Answer: prefix eventuate.local.kafka.consumer.properties.

Mysql encoding

eventuate-tram/eventuate-tram-core#70 (comment)

Document Eventuate Tram Testing support

Document how to use other message brokers in getting-started guides

Troubleshooting an unhealthy CDC due to failing binlogEntryReaderHealthCheck

  "details": {
    "binlogEntryReaderHealthCheck": {
      "status": "DOWN",
      "details": {
        "error-1": "Reader with id reader1 has not received message for 333039 milliseconds",
        "detail-1": "Reader with id reader1 is connected"
      }
    },

Look for exceptions in log
Verify cdc_monitoring exists
Verify that DB is configured for replication.

Schema
Server configuration

For example, https://github.com/eventuate-foundation/eventuate-common/blob/master/postgres/Dockerfile

curl --location -o SpringBoot.jar https://dl.bintray.com/eventuateio-oss/eventuate-maven-rc/io/eventuate/tram/core/eventuate-tram-cdc-mysql-service/0.21.3.RC3/eventuate-tram-cdc-mysql-service-0.21.3.RC3.jar

export SPRING_DATASOURCE_URL=jdbc:mysql://mysql/eventuate
export SPRING_DATASOURCE_USERNAME=mysqluser
export SPRING_DATASOURCE_PASSWORD=mysqlpw
export SPRING_DATASOURCE_DRIVER_CLASS_NAME=com.mysql.jdbc.Driver
export EVENTUATELOCAL_KAFKA_BOOTSTRAP_SERVERS=kafka:9092
export EVENTUATELOCAL_ZOOKEEPER_CONNECTION_STRING=zookeeper:2181
export EVENTUATELOCAL_CDC_DB_USER_NAME=root
export EVENTUATELOCAL_CDC_DB_PASSWORD=rootpassword
export EVENTUATELOCAL_CDC_POLLING_INTERVAL_IN_MILLISECONDS=500
export EVENTUATELOCAL_CDC_MAX_EVENTS_PER_POLLING=1000
export EVENTUATELOCAL_CDC_MAX_ATTEMPTS_FOR_POLLING=100
export EVENTUATELOCAL_CDC_POLLING_RETRY_INTERVAL_IN_MILLISECONDS=500
export EVENTUATELOCAL_CDC_READER_NAME=${EVENTUATELOCAL_CDC_READER_NAME}
export EVENTUATELOCAL_CDC_OFFSET_STORE_KEY=${EVENTUATELOCAL_CDC_OFFSET_STORE_KEY}
export EVENTUATELOCAL_CDC_MYSQL_BINLOG_CLIENT_UNIQUE_ID=${EVENTUATELOCAL_CDC_MYSQL_BINLOG_CLIENT_UNIQUE_ID}
export EVENTUATELOCAL_CDC_READ_OLD_DEBEZIUM_DB_OFFSET_STORAGE_TOPIC=${EVENTUATELOCAL_CDC_READ_OLD_DEBEZIUM_DB_OFFSET_STORAGE_TOPIC}

java -jar SpringBoot.jar ...

Document new 'backpressure' related features in Kafka Consumer

See eventuate-foundation/eventuate-messaging-kafka@7d0902c