SpringCloud/spring-cloud-stream
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Kafka Streams binder metrics

Browse Source 
Export Kafka Streams metrics available through KafkaStreams#metrics into a Micrometer MeterRegistry.
Add documentation for how to access metrics.
Modify test to verify metrics.

Resolves #543
This commit is contained in:
Soby Chacko
2019-10-03 22:47:54 -04:00
parent 98431ed8a0
commit ecc8715b0c
6 changed files with 175 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

23
docs/src/main/asciidoc/kafka-streams.adoc
View File

@@ -1063,4 +1063,25 @@ Here is an example of using custom state stores with functional style described
}
----
These state stores can be then accessed by the applications directly.
These state stores can be then accessed by the applications directly.
==== Accessing Kafka Streams Metrics
Spring Cloud Stream Kafka Streams binder provides a basic mechanism for accessing Kafka Streams metrics exported through a MircoMeter `MeterRegistry`.
Kafka Streams metrics that are available through `KafkaStreams#metrics()` are exported to this meter registry by the binder.
The metrics exported are from the consumers, producers, admin-client and the stream itself.
The metrics exported by the binder are exported with the format of metrics group name followed by a dot and then the actual metric name.
All dashes in the original metric information is replaced with dots.
For e.g. the metric name `network-io-total` from the metric group `consumer-metrics` is available in the micrometer registry as `consumer.metrics.network.io.total`.
Similarly, the metric `commit-total` from `stream-metrics` is available as `stream.metrics.commit.total`.
You can either programmatically access the Micrometer `MeterRegistry` in the application and then iterate through the available gauges or use Spring Boot actuator to access the metrics through a REST endpoint.
When accessing through the Boot actuator endpoint, make sure to add `metrics` to the property `management.endpoints.web.exposure.include`.
Then you can access `/acutator/metrics` to get a list of all the available metrics which then can be individually accessed through the same URL (`/actuator/metrics/<metric-name>`).
Anything beyond the info level metrics available through `KafkaStreams#metrics()`, (for e.g. the debugging level metrics) are still only available through JMX after you set the `metrics.recording.level` to `DEBUG`.
Kafka Streams, by default, set this level to `INFO`.
https://kafka.apache.org/documentation/#kafka_streams_monitoring[Please see this section] from Kafka Streams documentation for more details.
In a future release, binder may support exporting these DEBUG level metrics as well through Micrometer.