Giter Club home page Giter Club logo

confluent-cloud-hybrid-demo's Introduction

confluent-cloud-hybrid-demo


Detailed video of every step here:
---->

https://drive.google.com/file/d/1MQC5X1bzX7G6sivb77Nao2t7I-O15nA8/view?usp=sharing


First clone the git repo, and enter in the folder:
git clone https://github.com/angietd94/confluent-cloud-hybrid-demo.git
cd confluent-cloud-hybrid-demo

Modify first the config.sh file with your values! If done, skip this part.

###Guide: to find values in Confluent Cloud:

CLOUD_KEY=""
CLOUD_SECRET="
"

Cloud Api Key

ORG_ID="----****"
IMAGE_DESCRIPTION

ENVID="env-****"
IMAGE_DESCRIPTION

export CLUSTERID="lkc-***"
insidde the cluster information . Cluster > Cluster Settings.

#Either us, eu
export GEO="eu"
export REGION="eu-central-1"

CLUSTERAPIKEY=""
CLUSTERAPISECRET="
"
IMAGE_DESCRIPTION


#for example, aws or gcp
CLOUDPROVIDER="aws"

SR_API_KEY="
"
SR_API_SECRET="
"
SCHEMA_REGISTRY_URL="
***+"
The URL in the down-right side!
IMAGE_DESCRIPTION


BOOTSTRAP_SERVERS="*****:****"
![IMAGE_DESCRIPTION](img/bootstrap.png)


CLICKHOUSE DATA

export TOPIC=""
export DATABASE_NAME="default"
export DATABASE_PASSWORD="
"
export DATABASE_HOSTNAME="**"
export CLICKHOUSE_HOSTNAME="
"
IMAGE_DESCRIPTION

############

Second part: Launching the script!

then:

chmod +x executable.sh 
chmod +x config_files.sh
chmod +x config.sh
chmod +x ccloud-generate-cp-configs.sh
./executable.sh

More information on the part of Auto generating:
Example: Autogenerate Self-Managed Component Configs for Confluent Cloud
https://docs.confluent.io/cloud/current/cp-component/auto-generate-configs.html
Can you see the Control center here?
http://localhost:9021/clusters

and the plugins here?

http://localhost:8083/connector-plugins
What I see:
IMAGE_DESCRIPTION



if YES!

CREATE A TOPIC , I called it simple_example_avro for example with a simple schema:

{
  "doc": "Sample schema to help you get started.",
  "fields": [
    {
      "name": "age",
      "type": "int"
    },
    {
      "doc": "The string is a unicode character sequence.",
      "name": "name",
      "type": "string"
    }
  ],
  "name": "sampleRecord",
  "namespace": "com.mycorp.mynamespace",
  "type": "record"
}

and then after this launch config_files.sh making sure all values are good (they should) This will add a connector for clickhouse

./config_file.sh

You can verify it here, or in the control center!
http://localhost:8083/connectors/

CREATE A TABLE IN CLICKHOUSE SIMILAR TO THE STRUCTURE OF YOUR TOPIC
for example:

CREATE OR REPLACE TABLE simple_example_avro
(
    age Int64,
    name String
)
ENGINE = MergeTree
ORDER BY name;

Try send messages in CC something lik:

{
	"name": "angelica",
	"orderid": 29
}

And the outcome would look like this! IMAGE_DESCRIPTION

Also some videos here that go through all the process: https://drive.google.com/file/d/1xUpOMX-cFQDH5BVCfKHOc4IMn8uYmNxV/view https://drive.google.com/file/d/1Zr-Nkpmpd2leGuErGQRAPJWxmiheWZTB/view?usp=sharing

confluent-cloud-hybrid-demo's People

Contributors

angietd94 avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.