Kafka is an opensource distributed streaming platform that simplifies data intergration between systems. A stream is a pipeline to which your applications can push data. You can find more info on kafka's Official site
Kafka system has three main component:
-
A Producer: The service which produces the data that needs to be broadcast
-
A Broker: This is Kafka itself, which acts as a middle man between the producer and the consumer. It utilises the power of API's to get and broadcast data
-
A Consumer: The service that utilises the data which the broker will broadcast
-
If you are running mac OSX simply type
brew install kafka- once done installing run
brew services start kafka
- once done installing run
-
for linux user follow installation instruction from here
-
By default Kafka runs on port
9092
- Basic knowledge of python
- python 3
- Kafka installed
- Virtualenv.
- pip installed
clone this repo:
$ git clone [email protected]:amwaleh/Simple-stream-Kafka.git
Create a virtualenv and activate it inside your project directory:
$ virtualenv env && source env/bin/activate
Install required dependencies
pip install kafka-python opencv-python Flask
You will need two terminal to run the application
In the first terminal run consumer.py. open a terminal and type:
(env)$ python producer.py
In the second terminal run producer.py and pass in a message "hello word"
(env)$ python consumer.py
Open your browser and navigate to http://0.0.0.0:5000
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent