aebd885204
- Cleanup in schema-registry samples (final variable). - Add instructions in tools/kafka/docker-compose to run a quick test using Kafka console scripts.
87 lines
2.9 KiB
Plaintext
87 lines
2.9 KiB
Plaintext
== Docker Compose scripts for Kafka
|
|
|
|
[#run_kafka_cluster]
|
|
=== Kafka Cluster
|
|
Runs a multi-node Kafka cluster w/ 3 brokers available at `locahost:9091,localhost:9092,localhost:9093`.
|
|
|
|
To start the brokers run the following command:
|
|
[source,shell]
|
|
----
|
|
docker-compose -f ./kafka-cluster.yml up
|
|
----
|
|
To stop the brokers run the following command:
|
|
[source,shell]
|
|
----
|
|
docker-compose -f ./kafka-cluster.yml down
|
|
----
|
|
|
|
TIP: To start the containers in the foreground simply add `-d` to the start command. To stop containers that are running in the foreground simple issue `CTRL-C` command.
|
|
|
|
Running a quick test:
|
|
|
|
Now that we started our clusters, let us try to produce data to a topic on the cluster and then consume from it.
|
|
|
|
To produce data to a topic named `input-topic`:
|
|
|
|
```
|
|
docker exec -it broker1 /bin/kafka-console-producer --bootstrap-server localhost:29091 --topic input-topic
|
|
```
|
|
|
|
Note the use of bootstrap-server at `localhost:29091`.
|
|
This port is internal to the docker container.
|
|
|
|
To consume from the `input-topic`:
|
|
|
|
```
|
|
docker exec -it broker1 /bin/kafka-console-consumer --bootstrap-server localhost:29091 --topic input-topic
|
|
```
|
|
|
|
Enter some data on the producer console and hit enter, you should see the same data gets echoed on the consumer console.
|
|
|
|
=== Control Center UI
|
|
Runs a https://docs.confluent.io/platform/current/control-center/index.html[Confluent Control Center] that exposes a UI at http://locahost:9021.
|
|
|
|
NOTE: The scripts must be chained together as the UI depends on the brokers
|
|
|
|
To start the brokers and the Control Center UI run the following command:
|
|
[source,shell]
|
|
----
|
|
docker-compose -f ./kafka-cluster.yml -f ./control-center-ui.yml up
|
|
----
|
|
To stop the brokers and the Control Center UI run the following command:
|
|
[source,shell]
|
|
----
|
|
docker-compose -f ./kafka-cluster.yml -f ./control-center-ui.yml down
|
|
----
|
|
|
|
=== Schema Registry
|
|
Runs a https://docs.confluent.io/platform/current/schema-registry/index.html[Confluent Schema Registry] available at http://locahost:8081.
|
|
|
|
NOTE: The scripts must be chained together as the schema registry depends on the brokers
|
|
|
|
To start the brokers and the schema registry run the following command:
|
|
[source,shell]
|
|
----
|
|
docker-compose -f ./kafka-cluster.yml -f ./schema-registry.yml up
|
|
----
|
|
To stop the brokers and the schema registry run the following command:
|
|
[source,shell]
|
|
----
|
|
docker-compose -f ./kafka-cluster.yml -f ./schema-registry.yml down
|
|
----
|
|
|
|
[#_all_the_things]
|
|
=== All The Things
|
|
The scripts can be chained together to start the cluster, UI and schema registry with the following commmand:
|
|
[source,shell]
|
|
----
|
|
docker-compose -f kafka-cluster.yml -f ./control-center-ui.yml -f ./schema-registry.yml up
|
|
----
|
|
Likewise, to stop all the things:
|
|
[source,shell]
|
|
----
|
|
docker-compose -f kafka-cluster.yml -f ./control-center-ui.yml -f ./schema-registry.yml down
|
|
----
|
|
|
|
TIP: The only "UI" representation of the schema registry is by navigating to a `Topic` and then clicking the `Schema` tab for the topic
|