Docker Kafka Quickstart

A minimal configuration for a dockerized kafka project.

Usage:

Run this command to build kafka and zookeeper containers, and create the network kafka-network, where your containers can access the endpoint kafka:29092.

# creates the network `kafka-network` and build and
# runs the containers for zookeeper and kafka
docker-compose -f "docker-compose.kafka.yml" up -d --build # remove -d if you want to see logs in the same shell

You can then run as many broker/consumer containers on your kafka-network, in this case: generator is a broker, and indicator is a consumer.

# runs the generator and indicator containers
docker-compose up -d --build

If you check the logs in your generator container you should be seeing the messages sent to the queuing.sentences.test topic:

### KAFKA_BROKER_URL kafka:29092
09:35:09.135969 {'time': '09:35:08.107419', 'player': 'player2', 'sentence': 'sDZRJk nwENFG I sR VOOZNWll aGDdZV'}
09:35:10.138498 {'time': '09:35:10.138207', 'player': 'player2', 'sentence': 'vgIXxE  sa'}
09:35:11.141023 {'time': '09:35:11.140690', 'player': 'player1', 'sentence': 'E raH jzIeKHqXZF JGGB   tx  M oLusi '}
09:35:12.142630 {'time': '09:35:12.142360', 'player': 'player2', 'sentence': 'WxsPlteC  yxh  ogME dvbtFn l b TxVB  gaMFZe'}
09:35:13.144693 {'time': '09:35:13.144454', 'player': 'player1', 'sentence': ' aIRVm W TeIcKuigy  HPLOvJrQSXSl ZlLHlL FRH'}
09:35:14.146182 {'time': '09:35:14.145966', 'player': 'player1', 'sentence': 'cmMH k phZ LapRdGv C geTDzIT'}
09:35:15.147609 {'time': '09:35:15.147378', 'player': 'player2', 'sentence': 'QihJPzjFM WmkmDzMLl DkyydRVeyEI Rm'}
09:35:16.149716 {'time': '09:35:16.149502', 'player': 'player1', 'sentence': '  C '}
09:35:17.151371 {'time': '09:35:17.151155', 'player': 'player2', 'sentence': ''}
09:35:18.152736 {'time': '09:35:18.152508', 'player': 'player1', 'sentence': 'Oh tgfSl tY iBrxMHf aNX h'}
...

And the logs for indicator show the messages recieved on the queuing.sentences.test topic:

### KAFKA_BROKER_URL kafka:29092
Connecting...
Connected to queuing.sentences.test topic!
09:35:13.320414 {'time': '09:35:13.144454', 'player': 'player1', 'sentence': ' aIRVm W TeIcKuigy  HPLOvJrQSXSl ZlLHlL FRH'}
09:35:14.150851 {'time': '09:35:14.145966', 'player': 'player1', 'sentence': 'cmMH k phZ LapRdGv C geTDzIT'}
09:35:15.279800 {'time': '09:35:15.147378', 'player': 'player2', 'sentence': 'QihJPzjFM WmkmDzMLl DkyydRVeyEI Rm'}
09:35:16.153846 {'time': '09:35:16.149502', 'player': 'player1', 'sentence': '  C '}
09:35:17.156841 {'time': '09:35:17.151155', 'player': 'player2', 'sentence': ''}
09:35:18.156372 {'time': '09:35:18.152508', 'player': 'player1', 'sentence': 'Oh tgfSl tY iBrxMHf aNX h'}
09:35:19.160985 {'time': '09:35:19.154674', 'player': 'player2', 'sentence': 'qgSAzkUNm  fBQyruw S U   PvLnharO '}
09:35:20.183376 {'time': '09:35:20.157269', 'player': 'player1', 'sentence': 'OlX SW  a gcpXVGHxW'}

Refs:

For more information about using kafka with Docker: https://hub.docker.com/r/wurstmeister/kafka/

A minimal configuration for a dockerized kafka project.

Related tags

Overview

Docker Kafka Quickstart

Usage:

Refs:

Owner

Nouamane Tazi

Unzip Japanese Shift-JIS zip archives on non-Japanese systems.

Astroquery is an astropy affiliated package that contains a collection of tools to access online Astronomical data.

A minimalist starknet amm adapted from StarkWare's amm.

Run the Tianxunet software on the Xiaoyao Android simulator

Web站点选优工具 - 优化GitHub的打开速度、高效Clone

Python calculator made with tkinter package

Demodulate and error correct FIS-B and ADS-B signals on 978 MHz.

Py-Parser est un parser de code python en python encore en plien dévlopement.

A responsive package for Buttons, DropMenus and Combinations

RxPY - The Reactive Extensions for Python (RxPY)

Python wrapper around Apple App Store Api

One Ansible Module for using LINE notify API to send notification. It can be required in the collection list.

a simple functional programming language compiler written in python

Keyboard Layout Change - Extension for Ulauncher

Pre-crisis Risk Management for Personal Finance

VHDL to Discrete Logic on PCB Flow

A code base for python programs the goal is to integrate all the useful and essential functions

Generate Openbox Menus from a easy to write configuration file.

My solutions to Advent of Code 2021 (written in Python)

Tutorial on Tempo, Beat and Downbeat estimation