SpringCloud/spring-cloud-sleuth
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Kafka instrumentation enhancements (#1936)

Browse Source 
* Bump javadocs since version to 3.1.0

* Extend integration tests to cover all Kafka clients

* Add autoconfiguration module for kafka instrumentation

* Refactor instrumentation for reactive Kafka Receiver

* Refactor instrumentation for reactive Kafka Receiver

* Add docs for Kafka instrumentation

* Revert "Refactor instrumentation for reactive Kafka Receiver"

This reverts commit 58c8f2fa

* Revert "Revert "Refactor instrumentation for reactive Kafka Receiver""

This reverts commit 450a9f8c

* Remove empty test

* Resolve comments from PR 1936

* Revert whitespaces

* Revert whitespaces in common tests pom.xml

* Fix autoconfig to consider generics when registering beans.
Only register reactor-kafka beans if the dependency is on the classpath.

* Split autoconfig for kafka and reactor-kafka
This commit is contained in:
Flaviu Mureșan
2021-05-07 22:20:37 +02:00
committed by GitHub
parent 507e918c39
commit 59c170d147
32 changed files with 1266 additions and 232 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
docs/src/main/asciidoc/documentation-overview.adoc
View File

@@ -77,6 +77,7 @@ Need more details about {project-full-name}'s core features?
Finally, we have topics related to instrumentation integrations:
* *Integrations:*
<<integrations.adoc#sleuth-kafka-integration, Apache Kafka>> |
<<integrations.adoc#sleuth-async-integration, Asynchronous Communication>> |
<<integrations.adoc#sleuth-http-client-integration, HTTP Client Integration>> |
<<integrations.adoc#sleuth-http-server-integration, HTTP Server Integration>> |
@@ -91,4 +92,4 @@ Finally, we have topics related to instrumentation integrations:
<<integrations.adoc#sleuth-runnablecallable-integration, Runnable and Callable>> |
<<integrations.adoc#sleuth-rxjava-integration, RxJava>> |
<<integrations.adoc#sleuth-circuitbreaker-integration, Spring Cloud CircuitBreaker>> |
<<integrations.adoc#sleuth-quartz-integration, Quartz>>
<<integrations.adoc#sleuth-quartz-integration, Quartz>>

19
docs/src/main/asciidoc/integrations.adoc
View File

@@ -5,6 +5,25 @@ include::_attributes.adoc[]
In this section, we describe how to customize various parts of Spring Cloud Sleuth.
[[sleuth-kafka-integration]]
== Apache Kafka
This feature is available for all tracer implementations.
We decorate the Kafka clients (`KafkaProducer` and `KafkaConsumer`) to create a span for each event that is produced or consumed. You can disable this feature by setting the value of `spring.sleuth.kafka.enabled` to `false`.
IMPORTANT: You have to register the `Producer` or `Consumer` as beans in order for Sleuth's auto-configuration to decorate them. When you then inject the beans, the expected type must be `Producer` or `Consumer` (and NOT e.g. `KafkaProducer`).
We also provide `TracingKafkaProducerFactory` and `TracingKafkaConsumerFactory` to be used with the https://projectreactor.io/docs/kafka/release/reference/[Reactor Kafka] clients (`KafkaSender` and `KafkaReceiver`, respectively). See an example in the snippet below:
[source,java,indent=0]
----
@Bean
KafkaReceiver<String, String> reactiveKafkaReceiver(TracingKafkaConsumerFactory tracingKafkaConsumerFactory, KafkaReceiverOptions kafkaReceiverOptions) {
return KafkaReceiver.create(tracingKafkaConsumerFactory, kafkaReceiverOptions);
}
----
[[sleuth-async-integration]]
== Asynchronous Communication

5
spring-cloud-sleuth-autoconfigure/pom.xml
View File

@@ -275,6 +275,11 @@
<artifactId>spring-jms</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>io.projectreactor.kafka</groupId>
<artifactId>reactor-kafka</artifactId>
<optional>true</optional>
</dependency>
<!-- GRPC Optional Dependencies -->
<dependency>
<groupId>io.github.lognet</groupId>

74
spring-cloud-sleuth-autoconfigure/src/main/java/org/springframework/cloud/sleuth/autoconfig/instrument/kafka/TracingKafkaAutoConfiguration.java Normal file
View File

@@ -0,0 +1,74 @@
/*
* Copyright 2013-2021 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.cloud.sleuth.autoconfig.instrument.kafka;
import org.apache.kafka.clients.KafkaClient;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.springframework.beans.factory.BeanFactory;
import org.springframework.boot.autoconfigure.AutoConfigureAfter;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBean;
import org.springframework.boot.autoconfigure.condition.ConditionalOnClass;
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.cloud.sleuth.Tracer;
import org.springframework.cloud.sleuth.autoconfig.brave.BraveAutoConfiguration;
import org.springframework.cloud.sleuth.instrument.kafka.TracingKafkaPropagatorGetter;
import org.springframework.cloud.sleuth.instrument.kafka.TracingKafkaPropagatorSetter;
import org.springframework.cloud.sleuth.propagation.Propagator;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
/**
* {@link org.springframework.boot.autoconfigure.EnableAutoConfiguration
* Auto-configuration} that registers instrumentation for Kafka.
*
* @author Anders Clausen
* @author Flaviu Muresan
* @since 3.1.0
*/
@Configuration(proxyBeanMethods = false)
@ConditionalOnClass(KafkaClient.class)
@ConditionalOnBean(Tracer.class)
@AutoConfigureAfter(BraveAutoConfiguration.class)
@ConditionalOnProperty(value = "spring.sleuth.kafka.enabled", matchIfMissing = true)
public class TracingKafkaAutoConfiguration {
@Bean
@ConditionalOnMissingBean(value = ProducerRecord.class, parameterizedContainer = Propagator.Setter.class)
Propagator.Setter<ProducerRecord<?, ?>> tracingKafkaPropagationSetter() {
return new TracingKafkaPropagatorSetter();
}
@Bean
@ConditionalOnMissingBean(value = ConsumerRecord.class, parameterizedContainer = Propagator.Getter.class)
Propagator.Getter<ConsumerRecord<?, ?>> tracingKafkaPropagationGetter() {
return new TracingKafkaPropagatorGetter();
}
@Bean
static TracingKafkaProducerBeanPostProcessor tracingKafkaProducerBeanPostProcessor(BeanFactory beanFactory) {
return new TracingKafkaProducerBeanPostProcessor(beanFactory);
}
@Bean
static TracingKafkaConsumerBeanPostProcessor tracingKafkaConsumerBeanPostProcessor(BeanFactory beanFactory) {
return new TracingKafkaConsumerBeanPostProcessor(beanFactory);
}
}

49
spring-cloud-sleuth-autoconfigure/src/main/java/org/springframework/cloud/sleuth/autoconfig/instrument/kafka/TracingKafkaConsumerBeanPostProcessor.java Normal file
View File

@@ -0,0 +1,49 @@
/*
* Copyright 2013-2021 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.cloud.sleuth.autoconfig.instrument.kafka;
import org.apache.kafka.clients.consumer.Consumer;
import org.springframework.beans.BeansException;
import org.springframework.beans.factory.BeanFactory;
import org.springframework.beans.factory.config.BeanPostProcessor;
import org.springframework.cloud.sleuth.instrument.kafka.TracingKafkaConsumer;
/**
* Bean post processor for {@link org.apache.kafka.clients.consumer.Consumer}.
*
* @author Anders Clausen
* @author Flaviu Muresan
* @since 3.1.0
*/
public class TracingKafkaConsumerBeanPostProcessor implements BeanPostProcessor {
private final BeanFactory beanFactory;
public TracingKafkaConsumerBeanPostProcessor(BeanFactory beanFactory) {
this.beanFactory = beanFactory;
}
@Override
public Object postProcessAfterInitialization(Object bean, String beanName) throws BeansException {
if (bean instanceof Consumer && !(bean instanceof TracingKafkaConsumer)) {
return new TracingKafkaConsumer<>((Consumer) bean, beanFactory);
}
return bean;
}
}

49
spring-cloud-sleuth-autoconfigure/src/main/java/org/springframework/cloud/sleuth/autoconfig/instrument/kafka/TracingKafkaProducerBeanPostProcessor.java Normal file
View File

@@ -0,0 +1,49 @@
/*
* Copyright 2013-2021 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.cloud.sleuth.autoconfig.instrument.kafka;
import org.apache.kafka.clients.producer.Producer;
import org.springframework.beans.BeansException;
import org.springframework.beans.factory.BeanFactory;
import org.springframework.beans.factory.config.BeanPostProcessor;
import org.springframework.cloud.sleuth.instrument.kafka.TracingKafkaProducer;
/**
* Bean post processor for {@link org.apache.kafka.clients.producer.Producer}.
*
* @author Anders Clausen
* @author Flaviu Muresan
* @since 3.1.0
*/
public class TracingKafkaProducerBeanPostProcessor implements BeanPostProcessor {
private final BeanFactory beanFactory;
public TracingKafkaProducerBeanPostProcessor(BeanFactory beanFactory) {
this.beanFactory = beanFactory;
}
@Override
public Object postProcessAfterInitialization(Object bean, String beanName) throws BeansException {
if (bean instanceof Producer && !(bean instanceof TracingKafkaProducer)) {
return new TracingKafkaProducer<>((Producer) bean, beanFactory);
}
return bean;
}
}

61
spring-cloud-sleuth-autoconfigure/src/main/java/org/springframework/cloud/sleuth/autoconfig/instrument/kafka/TracingReactorKafkaAutoConfiguration.java Normal file
View File

@@ -0,0 +1,61 @@
/*
* Copyright 2013-2021 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.cloud.sleuth.autoconfig.instrument.kafka;
import reactor.kafka.receiver.KafkaReceiver;
import org.springframework.beans.factory.BeanFactory;
import org.springframework.boot.autoconfigure.AutoConfigureAfter;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBean;
import org.springframework.boot.autoconfigure.condition.ConditionalOnClass;
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.cloud.sleuth.Tracer;
import org.springframework.cloud.sleuth.autoconfig.brave.BraveAutoConfiguration;
import org.springframework.cloud.sleuth.instrument.kafka.TracingKafkaConsumerFactory;
import org.springframework.cloud.sleuth.instrument.kafka.TracingKafkaProducerFactory;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
/**
* {@link org.springframework.boot.autoconfigure.EnableAutoConfiguration
* Auto-configuration} that registers instrumentation for Reactor Kafka.
*
* @author Anders Clausen
* @author Flaviu Muresan
* @since 3.1.0
*/
@Configuration(proxyBeanMethods = false)
@ConditionalOnClass(KafkaReceiver.class)
@ConditionalOnBean(Tracer.class)
@AutoConfigureAfter(BraveAutoConfiguration.class)
@ConditionalOnProperty(value = "spring.sleuth.kafka.enabled", matchIfMissing = true)
public class TracingReactorKafkaAutoConfiguration {
@Bean
@ConditionalOnMissingBean
TracingKafkaProducerFactory tracingKafkaProducerFactory(BeanFactory beanFactory) {
return new TracingKafkaProducerFactory(beanFactory);
}
@Bean
@ConditionalOnMissingBean
TracingKafkaConsumerFactory tracingKafkaConsumerFactory(BeanFactory beanFactory) {
return new TracingKafkaConsumerFactory(beanFactory);
}
}

4
spring-cloud-sleuth-autoconfigure/src/main/java/org/springframework/cloud/sleuth/autoconfig/instrument/messaging/TraceSpringMessagingAutoConfiguration.java
View File

@@ -45,13 +45,13 @@ class TraceSpringMessagingAutoConfiguration {
}
@Bean
@ConditionalOnMissingBean
@ConditionalOnMissingBean(value = MessageHeaderAccessor.class, parameterizedContainer = Propagator.Setter.class)
Propagator.Setter<MessageHeaderAccessor> traceMessagePropagationSetter() {
return new MessageHeaderPropagatorSetter();
}
@Bean
@ConditionalOnMissingBean
@ConditionalOnMissingBean(value = MessageHeaderAccessor.class, parameterizedContainer = Propagator.Getter.class)
Propagator.Getter<MessageHeaderAccessor> traceMessagePropagationGetter() {
return new MessageHeaderPropagatorGetter();
}

6
spring-cloud-sleuth-autoconfigure/src/main/resources/META-INF/additional-spring-configuration-metadata.json
View File

@@ -41,6 +41,12 @@
"description": "Enable tracing for WebSockets.",
"defaultValue": true
},
{
"name": "spring.sleuth.kafka.enabled",
"type": "java.lang.Boolean",
"description": "Enable instrumenting of Apache Kafka clients.",
"defaultValue": true
},
{
"name": "spring.sleuth.async.enabled",
"type": "java.lang.Boolean",

1
spring-cloud-sleuth-autoconfigure/src/main/resources/META-INF/spring.factories
View File

@@ -1,5 +1,6 @@
# Auto Configuration
org.springframework.boot.autoconfigure.EnableAutoConfiguration=\
org.springframework.cloud.sleuth.autoconfig.instrument.kafka.TracingKafkaAutoConfiguration,\
org.springframework.cloud.sleuth.autoconfig.instrument.async.TraceAsyncAutoConfiguration,\
org.springframework.cloud.sleuth.autoconfig.instrument.async.TraceAsyncCustomAutoConfiguration,\
org.springframework.cloud.sleuth.autoconfig.instrument.async.TraceAsyncDefaultAutoConfiguration,\

2
spring-cloud-sleuth-instrumentation/src/main/java/org/springframework/cloud/sleuth/instrument/kafka/KafkaTracingCallback.java
View File

@@ -31,7 +31,7 @@ import org.springframework.cloud.sleuth.Tracer;
*
* @author Anders Clausen
* @author Flaviu Muresan
* @since 3.0.3
* @since 3.1.0
*/
public class KafkaTracingCallback implements Callback {

39
spring-cloud-sleuth-instrumentation/src/main/java/org/springframework/cloud/sleuth/instrument/kafka/TracingKafkaConsumer.java
View File

@@ -37,8 +37,11 @@ import org.apache.kafka.common.MetricName;
import org.apache.kafka.common.PartitionInfo;
import org.apache.kafka.common.TopicPartition;
import org.springframework.beans.factory.BeanFactory;
import org.springframework.cloud.sleuth.Span;
import org.springframework.cloud.sleuth.propagation.Propagator;
import org.springframework.core.ParameterizedTypeReference;
import org.springframework.core.ResolvableType;
/**
* This decorates a Kafka {@link Consumer}. It creates and completes a
@@ -47,21 +50,39 @@ import org.springframework.cloud.sleuth.propagation.Propagator;
*
* @author Anders Clausen
* @author Flaviu Muresan
* @since 3.0.3
* @since 3.1.0
*/
public class TracingKafkaConsumer<K, V> implements Consumer<K, V> {
private final BeanFactory beanFactory;
private final Consumer<K, V> delegate;
private final Propagator propagator;
private Propagator propagator;
private final Propagator.Getter<ConsumerRecord<?, ?>> extractor;
private Propagator.Getter<ConsumerRecord<?, ?>> extractor;
public TracingKafkaConsumer(Consumer<K, V> consumer, Propagator propagator,
Propagator.Getter<ConsumerRecord<?, ?>> getter) {
public TracingKafkaConsumer(Consumer<K, V> consumer, BeanFactory beanFactory) {
this.delegate = consumer;
this.propagator = propagator;
this.extractor = getter;
this.beanFactory = beanFactory;
}
private Propagator propagator() {
if (this.propagator == null) {
this.propagator = this.beanFactory.getBean(Propagator.class);
}
return this.propagator;
}
private Propagator.Getter<ConsumerRecord<?, ?>> extractor() {
if (this.extractor == null) {
this.extractor = (Propagator.Getter<ConsumerRecord<?, ?>>) beanFactory
.getBeanProvider(ResolvableType.forClassWithGenerics(Propagator.Getter.class,
ResolvableType.forType(new ParameterizedTypeReference<ConsumerRecord<?, ?>>() {
})))
.getIfAvailable();
}
return this.extractor;
}
@Override
@@ -109,7 +130,7 @@ public class TracingKafkaConsumer<K, V> implements Consumer<K, V> {
public ConsumerRecords<K, V> poll(long l) {
ConsumerRecords<K, V> consumerRecords = this.delegate.poll(l);
for (ConsumerRecord<K, V> consumerRecord : consumerRecords) {
KafkaTracingUtils.buildAndFinishSpan(consumerRecord, this.propagator, this.extractor);
KafkaTracingUtils.buildAndFinishSpan(consumerRecord, propagator(), extractor());
}
return consumerRecords;
}
@@ -118,7 +139,7 @@ public class TracingKafkaConsumer<K, V> implements Consumer<K, V> {
public ConsumerRecords<K, V> poll(Duration duration) {
ConsumerRecords<K, V> consumerRecords = this.delegate.poll(duration);
for (ConsumerRecord<K, V> consumerRecord : consumerRecords) {
KafkaTracingUtils.buildAndFinishSpan(consumerRecord, this.propagator, this.extractor);
KafkaTracingUtils.buildAndFinishSpan(consumerRecord, propagator(), extractor());
}
return consumerRecords;
}

49
spring-cloud-sleuth-instrumentation/src/main/java/org/springframework/cloud/sleuth/instrument/kafka/TracingKafkaConsumerFactory.java Normal file
View File

@@ -0,0 +1,49 @@
/*
* Copyright 2013-2021 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.cloud.sleuth.instrument.kafka;
import org.apache.kafka.clients.consumer.Consumer;
import reactor.kafka.receiver.KafkaReceiver;
import reactor.kafka.receiver.ReceiverOptions;
import reactor.kafka.receiver.internals.ConsumerFactory;
import org.springframework.beans.factory.BeanFactory;
/**
* This decorates a Reactor Kafka {@link ConsumerFactory} to create decorated consumers of
* type {@link TracingKafkaConsumer}. This can be used by the {@link KafkaReceiver}
* factory methods to create instrumented receivers.
*
* @author Anders Clausen
* @author Flaviu Muresan
* @since 3.1.0
*/
public class TracingKafkaConsumerFactory extends ConsumerFactory {
private final BeanFactory beanFactory;
public TracingKafkaConsumerFactory(BeanFactory beanFactory) {
super();
this.beanFactory = beanFactory;
}
@Override
public <K, V> Consumer<K, V> createConsumer(ReceiverOptions<K, V> receiverOptions) {
return new TracingKafkaConsumer<>(super.createConsumer(receiverOptions), beanFactory);
}
}

53
spring-cloud-sleuth-instrumentation/src/main/java/org/springframework/cloud/sleuth/instrument/kafka/TracingKafkaProducer.java
View File

@@ -35,9 +35,12 @@ import org.apache.kafka.common.PartitionInfo;
import org.apache.kafka.common.TopicPartition;
import org.apache.kafka.common.errors.ProducerFencedException;
import org.springframework.beans.factory.BeanFactory;
import org.springframework.cloud.sleuth.Span;
import org.springframework.cloud.sleuth.Tracer;
import org.springframework.cloud.sleuth.propagation.Propagator;
import org.springframework.core.ParameterizedTypeReference;
import org.springframework.core.ResolvableType;
/**
* This decorates a Kafka {@link Producer} and creates a {@link Span.Kind#PRODUCER} span
@@ -46,26 +49,50 @@ import org.springframework.cloud.sleuth.propagation.Propagator;
*
* @author Anders Clausen
* @author Flaviu Muresan
* @since 3.0.3
* @since 3.1.0
*/
public class TracingKafkaProducer<K, V> implements Producer<K, V> {
private static final Log log = LogFactory.getLog(TracingKafkaProducer.class);
private final BeanFactory beanFactory;
private final Producer<K, V> delegate;
private final Tracer tracer;
private Tracer tracer;
private final Propagator propagator;
private Propagator propagator;
private final Propagator.Setter<ProducerRecord<?, ?>> injector;
private Propagator.Setter<ProducerRecord<?, ?>> injector;
public TracingKafkaProducer(Producer<K, V> producer, Tracer tracer, Propagator propagator,
Propagator.Setter<ProducerRecord<?, ?>> setter) {
public TracingKafkaProducer(Producer<K, V> producer, BeanFactory beanFactory) {
this.delegate = producer;
this.tracer = tracer;
this.propagator = propagator;
this.injector = setter;
this.beanFactory = beanFactory;
}
private Tracer tracer() {
if (this.tracer == null) {
this.tracer = this.beanFactory.getBean(Tracer.class);
}
return this.tracer;
}
private Propagator propagator() {
if (this.propagator == null) {
this.propagator = this.beanFactory.getBean(Propagator.class);
}
return this.propagator;
}
private Propagator.Setter<ProducerRecord<?, ?>> injector() {
if (this.injector == null) {
this.injector = (Propagator.Setter<ProducerRecord<?, ?>>) beanFactory
.getBeanProvider(ResolvableType.forClassWithGenerics(Propagator.Setter.class,
ResolvableType.forType(new ParameterizedTypeReference<ProducerRecord<?, ?>>() {
})))
.getIfAvailable();
}
return this.injector;
}
@Override
@@ -107,15 +134,15 @@ public class TracingKafkaProducer<K, V> implements Producer<K, V> {
@Override
public Future<RecordMetadata> send(ProducerRecord<K, V> producerRecord, Callback callback) {
Span.Builder spanBuilder = tracer.spanBuilder().kind(Span.Kind.PRODUCER).name("kafka.produce")
Span.Builder spanBuilder = tracer().spanBuilder().kind(Span.Kind.PRODUCER).name("kafka.produce")
.tag("kafka.topic", producerRecord.topic());
Span span = spanBuilder.start();
this.propagator.inject(span.context(), producerRecord, this.injector);
try (Tracer.SpanInScope spanInScope = tracer.withSpan(span)) {
propagator().inject(span.context(), producerRecord, injector());
try (Tracer.SpanInScope spanInScope = tracer().withSpan(span)) {
if (log.isDebugEnabled()) {
log.debug("Created producer span " + span);
}
return this.delegate.send(producerRecord, new KafkaTracingCallback(callback, tracer, span));
return this.delegate.send(producerRecord, new KafkaTracingCallback(callback, tracer(), span));
}
}

17
spring-cloud-sleuth-instrumentation/src/main/java/org/springframework/cloud/sleuth/instrument/kafka/TracingKafkaProducerFactory.java
View File

@@ -21,8 +21,7 @@ import reactor.kafka.sender.KafkaSender;
import reactor.kafka.sender.SenderOptions;
import reactor.kafka.sender.internals.ProducerFactory;
import org.springframework.cloud.sleuth.Tracer;
import org.springframework.cloud.sleuth.propagation.Propagator;
import org.springframework.beans.factory.BeanFactory;
/**
* This decorates a Reactor Kafka {@link ProducerFactory} to create decorated producers of
@@ -31,24 +30,20 @@ import org.springframework.cloud.sleuth.propagation.Propagator;
*
* @author Anders Clausen
* @author Flaviu Muresan
* @since 3.0.3
* @since 3.1.0
*/
public class TracingKafkaProducerFactory extends ProducerFactory {
private final Tracer tracer;
private final BeanFactory beanFactory;
private final Propagator propagator;
public TracingKafkaProducerFactory(Tracer tracer, Propagator propagator) {
public TracingKafkaProducerFactory(BeanFactory beanFactory) {
super();
this.tracer = tracer;
this.propagator = propagator;
this.beanFactory = beanFactory;
}
@Override
public <K, V> Producer<K, V> createProducer(SenderOptions<K, V> senderOptions) {
return new TracingKafkaProducer<>(super.createProducer(senderOptions), tracer, propagator,
new TracingKafkaPropagatorSetter());
return new TracingKafkaProducer<>(super.createProducer(senderOptions), beanFactory);
}
}

2
spring-cloud-sleuth-instrumentation/src/main/java/org/springframework/cloud/sleuth/instrument/kafka/TracingKafkaPropagatorGetter.java
View File

@@ -30,7 +30,7 @@ import org.springframework.cloud.sleuth.propagation.Propagator;
*
* @author Anders Clausen
* @author Flaviu Muresan
* @since 3.0.3
* @since 3.1.0
*/
public class TracingKafkaPropagatorGetter implements Propagator.Getter<ConsumerRecord<?, ?>> {

2
spring-cloud-sleuth-instrumentation/src/main/java/org/springframework/cloud/sleuth/instrument/kafka/TracingKafkaPropagatorSetter.java
View File

@@ -26,7 +26,7 @@ import org.springframework.cloud.sleuth.propagation.Propagator;
*
* @author Anders Clausen
* @author Flaviu Muresan
* @since 3.0.3
* @since 3.1.0
*/
public class TracingKafkaPropagatorSetter implements Propagator.Setter<ProducerRecord<?, ?>> {

112
spring-cloud-sleuth-instrumentation/src/main/java/org/springframework/cloud/sleuth/instrument/kafka/TracingKafkaReceiver.java
View File

@@ -1,112 +0,0 @@
/*
* Copyright 2013-2021 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.cloud.sleuth.instrument.kafka;
import java.util.function.Function;
import org.apache.kafka.clients.consumer.Consumer;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;
import reactor.kafka.receiver.KafkaReceiver;
import reactor.kafka.receiver.ReceiverRecord;
import reactor.kafka.sender.TransactionManager;
import org.springframework.cloud.sleuth.Span;
import org.springframework.cloud.sleuth.propagation.Propagator;
/**
* This decorates a reactive {@link KafkaReceiver} and creates and completes a
* {@link Span.Kind#CONSUMER} span for each record received. This span will be a child
* span of the one extracted from the record headers.
*
* @author Anders Clausen
* @author Flaviu Muresan
* @since 3.0.3
*/
public class TracingKafkaReceiver<K, V> implements KafkaReceiver<K, V> {
private final KafkaReceiver<K, V> delegate;
private final Propagator propagator;
private final Propagator.Getter<ConsumerRecord<?, ?>> extractor;
public TracingKafkaReceiver(KafkaReceiver<K, V> receiver, Propagator propagator,
Propagator.Getter<ConsumerRecord<?, ?>> getter) {
this.delegate = receiver;
this.propagator = propagator;
this.extractor = getter;
}
@Override
public Flux<ReceiverRecord<K, V>> receive(Integer integer) {
return buildAndFinishSpanOnNextReceiverRecord(this.delegate.receive(integer));
}
@Override
public Flux<ReceiverRecord<K, V>> receive() {
return buildAndFinishSpanOnNextReceiverRecord(this.delegate.receive());
}
@Override
public Flux<Flux<ConsumerRecord<K, V>>> receiveAutoAck(Integer integer) {
return this.delegate.receiveAutoAck(integer).map(this::buildAndFinishSpanOnNextConsumerRecord);
}
@Override
public Flux<Flux<ConsumerRecord<K, V>>> receiveAutoAck() {
return this.delegate.receiveAutoAck().map(this::buildAndFinishSpanOnNextConsumerRecord);
}
@Override
public Flux<ConsumerRecord<K, V>> receiveAtmostOnce(Integer integer) {
return this.buildAndFinishSpanOnNextConsumerRecord(this.delegate.receiveAtmostOnce(integer));
}
@Override
public Flux<ConsumerRecord<K, V>> receiveAtmostOnce() {
return this.buildAndFinishSpanOnNextConsumerRecord(this.delegate.receiveAtmostOnce());
}
@Override
public Flux<Flux<ConsumerRecord<K, V>>> receiveExactlyOnce(TransactionManager transactionManager) {
return this.delegate.receiveExactlyOnce(transactionManager).map(this::buildAndFinishSpanOnNextConsumerRecord);
}
@Override
public Flux<Flux<ConsumerRecord<K, V>>> receiveExactlyOnce(TransactionManager transactionManager, Integer integer) {
return this.delegate.receiveExactlyOnce(transactionManager, integer)
.map(this::buildAndFinishSpanOnNextConsumerRecord);
}
@Override
public <T> Mono<T> doOnConsumer(Function<Consumer<K, V>, ? extends T> function) {
return this.delegate.doOnConsumer(function);
}
private Flux<ConsumerRecord<K, V>> buildAndFinishSpanOnNextConsumerRecord(Flux<ConsumerRecord<K, V>> flux) {
return flux.doOnNext(consumerRecord -> KafkaTracingUtils.buildAndFinishSpan(consumerRecord, this.propagator,
this.extractor));
}
private Flux<ReceiverRecord<K, V>> buildAndFinishSpanOnNextReceiverRecord(Flux<ReceiverRecord<K, V>> flux) {
return flux.doOnNext(consumerRecord -> KafkaTracingUtils.buildAndFinishSpan(consumerRecord, this.propagator,
this.extractor));
}
}

14
spring-cloud-sleuth-instrumentation/src/test/java/org/springframework/cloud/sleuth/instrument/kafka/TracingKafkaConsumerTest.java
View File

@@ -35,6 +35,8 @@ import org.mockito.Mock;
import org.mockito.Mockito;
import org.mockito.junit.jupiter.MockitoExtension;
import org.springframework.beans.factory.BeanFactory;
import org.springframework.beans.factory.support.StaticListableBeanFactory;
import org.springframework.cloud.sleuth.propagation.Propagator;
import static org.mockito.ArgumentMatchers.eq;
@@ -48,6 +50,9 @@ public class TracingKafkaConsumerTest {
@Mock(answer = Answers.RETURNS_DEEP_STUBS)
Propagator propagator;
@Mock
Propagator.Getter<ConsumerRecord<?, ?>> extractor;
@Test
void should_delegate_poll_calls() {
Duration pollTimeout = Duration.of(5, ChronoUnit.SECONDS);
@@ -57,11 +62,18 @@ public class TracingKafkaConsumerTest {
ConsumerRecords<String, String> records = new ConsumerRecords<>(map);
BDDMockito.given(kafkaConsumer.poll(pollTimeout)).willReturn(records);
TracingKafkaConsumer<String, String> tracingKafkaConsumer = new TracingKafkaConsumer<>(kafkaConsumer,
propagator, new TracingKafkaPropagatorGetter());
beanFactory());
tracingKafkaConsumer.poll(pollTimeout);
Mockito.verify(kafkaConsumer).poll(eq(pollTimeout));
}
private BeanFactory beanFactory() {
StaticListableBeanFactory beanFactory = new StaticListableBeanFactory();
beanFactory.addBean("propagator", this.propagator);
beanFactory.addBean("extractor", this.extractor);
return beanFactory;
}
}

17
spring-cloud-sleuth-instrumentation/src/test/java/org/springframework/cloud/sleuth/instrument/kafka/TracingKafkaProducerTest.java
View File

@@ -28,6 +28,8 @@ import org.mockito.Mock;
import org.mockito.Mockito;
import org.mockito.junit.jupiter.MockitoExtension;
import org.springframework.beans.factory.BeanFactory;
import org.springframework.beans.factory.support.StaticListableBeanFactory;
import org.springframework.cloud.sleuth.Tracer;
import org.springframework.cloud.sleuth.propagation.Propagator;
import org.springframework.test.util.ReflectionTestUtils;
@@ -52,8 +54,8 @@ public class TracingKafkaProducerTest {
ProducerRecord<String, String> testRecord = new ProducerRecord<>("test", "test");
Callback callback = (record, ex) -> {
};
TracingKafkaProducer<String, String> tracingKafkaProducer = new TracingKafkaProducer<>(kafkaProducer, tracer,
propagator, new TracingKafkaPropagatorSetter());
TracingKafkaProducer<String, String> tracingKafkaProducer = new TracingKafkaProducer<>(kafkaProducer,
beanFactory());
tracingKafkaProducer.send(testRecord, callback);
@@ -65,8 +67,8 @@ public class TracingKafkaProducerTest {
ProducerRecord<String, String> testRecord = new ProducerRecord<>("test", "test");
Callback callback = (record, ex) -> {
};
TracingKafkaProducer<String, String> tracingKafkaProducer = new TracingKafkaProducer<>(kafkaProducer, tracer,
propagator, new TracingKafkaPropagatorSetter());
TracingKafkaProducer<String, String> tracingKafkaProducer = new TracingKafkaProducer<>(kafkaProducer,
beanFactory());
tracingKafkaProducer.send(testRecord, callback);
@@ -76,4 +78,11 @@ public class TracingKafkaProducerTest {
BDDAssertions.then(ReflectionTestUtils.getField(callbackArgument.getValue(), "callback")).isEqualTo(callback);
}
private BeanFactory beanFactory() {
StaticListableBeanFactory beanFactory = new StaticListableBeanFactory();
beanFactory.addBean("tracer", this.tracer);
beanFactory.addBean("propagator", this.propagator);
return beanFactory;
}
}

61
spring-cloud-sleuth-instrumentation/src/test/java/org/springframework/cloud/sleuth/instrument/kafka/TracingKafkaReceiverTest.java
View File

@@ -1,61 +0,0 @@
/*
* Copyright 2013-2021 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.cloud.sleuth.instrument.kafka;
import java.util.function.Predicate;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Answers;
import org.mockito.BDDMockito;
import org.mockito.Mock;
import org.mockito.Mockito;
import org.mockito.junit.jupiter.MockitoExtension;
import reactor.core.publisher.Flux;
import reactor.kafka.receiver.KafkaReceiver;
import reactor.kafka.receiver.ReceiverOffset;
import reactor.kafka.receiver.ReceiverRecord;
import reactor.test.StepVerifier;
import org.springframework.cloud.sleuth.propagation.Propagator;
@ExtendWith(MockitoExtension.class)
public class TracingKafkaReceiverTest {
@Mock
KafkaReceiver<String, String> kafkaReceiver;
@Mock(answer = Answers.RETURNS_DEEP_STUBS)
Propagator propagator;
@Test
void should_delegate_receive_calls() {
ReceiverOffset receiverOffset = BDDMockito.mock(ReceiverOffset.class);
ConsumerRecord<String, String> record = new ConsumerRecord<>("topic", 0, 1, "test-key", "test-value");
ReceiverRecord<String, String> receiverRecord = new ReceiverRecord<>(record, receiverOffset);
BDDMockito.given(kafkaReceiver.receive()).willReturn(Flux.just(receiverRecord));
TracingKafkaReceiver<String, String> tracingKafkaReceiver = new TracingKafkaReceiver<>(kafkaReceiver,
propagator, new TracingKafkaPropagatorGetter());
StepVerifier.create(tracingKafkaReceiver.receive()).expectNextMatches(Predicate.isEqual(receiverRecord))
.verifyComplete();
Mockito.verify(kafkaReceiver).receive();
}
}

5
tests/brave/spring-cloud-sleuth-instrumentation-kafka-tests/pom.xml
View File

@@ -70,7 +70,10 @@
<dependency>
<groupId>io.projectreactor.kafka</groupId>
<artifactId>reactor-kafka</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>io.projectreactor</groupId>
<artifactId>reactor-test</artifactId>
</dependency>
<dependency>
<groupId>org.testcontainers</groupId>

65
tests/brave/spring-cloud-sleuth-instrumentation-kafka-tests/src/test/java/org/springframework/cloud/sleuth/brave/instrument/kafka/KafkaConsumerTest.java Normal file
View File

@@ -0,0 +1,65 @@
/*
* Copyright 2013-2021 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.cloud.sleuth.brave.instrument.kafka;
import java.time.Duration;
import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.assertj.core.api.BDDAssertions;
import org.junit.jupiter.api.Test;
import org.springframework.cloud.sleuth.Span;
import org.springframework.cloud.sleuth.brave.BraveTestTracing;
import org.springframework.cloud.sleuth.exporter.FinishedSpan;
import org.springframework.cloud.sleuth.instrument.kafka.KafkaTestUtils;
import org.springframework.cloud.sleuth.test.TestTracingAware;
import static org.awaitility.Awaitility.await;
public class KafkaConsumerTest extends org.springframework.cloud.sleuth.instrument.kafka.KafkaConsumerTest {
BraveTestTracing testTracing;
@Override
public TestTracingAware tracerTest() {
if (this.testTracing == null) {
this.testTracing = new BraveTestTracing();
}
return this.testTracing;
}
@Test
public void should_consider_native_headers() {
KafkaProducer<String, String> kafkaProducer = KafkaTestUtils
.buildTestKafkaProducer(kafkaContainer.getBootstrapServers());
ProducerRecord<String, String> producerRecord = new ProducerRecord<>(testTopic, "test", "test");
producerRecord.headers().add("b3", "000000000000000a-000000000000000b-1-000000000000000a".getBytes());
kafkaProducer.send(producerRecord);
kafkaProducer.close();
await().atMost(Duration.ofSeconds(5)).until(() -> receivedCounter.intValue() == 1);
BDDAssertions.then(this.tracer.currentSpan()).isNull();
BDDAssertions.then(this.spans).hasSize(1);
FinishedSpan span = this.spans.get(0);
BDDAssertions.then(span.getKind()).isEqualTo(Span.Kind.CONSUMER);
BDDAssertions.then(span.getTraceId()).isEqualTo("000000000000000a");
BDDAssertions.then(span.getParentId()).isEqualTo("000000000000000b");
}
}

28
tests/brave/spring-cloud-sleuth-instrumentation-kafka-tests/src/test/java/org/springframework/cloud/sleuth/brave/instrument/kafka/KafkaProducerTest.java
View File

@@ -16,6 +16,15 @@
package org.springframework.cloud.sleuth.brave.instrument.kafka;
import java.util.Optional;
import java.util.concurrent.TimeUnit;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.apache.kafka.common.header.Header;
import org.assertj.core.api.BDDAssertions;
import org.junit.jupiter.api.Test;
import org.springframework.cloud.sleuth.brave.BraveTestTracing;
import org.springframework.cloud.sleuth.test.TestTracingAware;
@@ -31,4 +40,23 @@ public class KafkaProducerTest extends org.springframework.cloud.sleuth.instrume
return this.testTracing;
}
@Test
public void should_inject_native_headers() throws InterruptedException {
ProducerRecord<String, String> producerRecord = new ProducerRecord<>(testTopic, "test", "test");
startKafkaConsumer();
this.kafkaProducer.send(producerRecord);
ConsumerRecord<String, String> consumerRecord = consumerRecords.poll(5, TimeUnit.SECONDS);
BDDAssertions.then(consumerRecord).isNotNull();
BDDAssertions.then(getHeaderValueOrNull(consumerRecord, "X-B3-TraceId")).isNotNull();
BDDAssertions.then(getHeaderValueOrNull(consumerRecord, "X-B3-SpanId")).isNotNull();
BDDAssertions.then(getHeaderValueOrNull(consumerRecord, "X-B3-Sampled")).isNotNull();
}
private static String getHeaderValueOrNull(ConsumerRecord<?, ?> consumerRecord, String header) {
return Optional.ofNullable(consumerRecord).map(ConsumerRecord::headers)
.map(headers -> headers.lastHeader(header)).map(Header::value).map(String::new).orElse(null);
}
}

64
tests/brave/spring-cloud-sleuth-instrumentation-kafka-tests/src/test/java/org/springframework/cloud/sleuth/brave/instrument/kafka/KafkaReceiverTest.java Normal file
View File

@@ -0,0 +1,64 @@
/*
* Copyright 2013-2021 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.cloud.sleuth.brave.instrument.kafka;
import java.time.Duration;
import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.assertj.core.api.BDDAssertions;
import org.junit.jupiter.api.Test;
import org.springframework.cloud.sleuth.Span;
import org.springframework.cloud.sleuth.brave.BraveTestTracing;
import org.springframework.cloud.sleuth.exporter.FinishedSpan;
import org.springframework.cloud.sleuth.instrument.kafka.KafkaTestUtils;
import org.springframework.cloud.sleuth.test.TestTracingAware;
import static org.awaitility.Awaitility.await;
public class KafkaReceiverTest extends org.springframework.cloud.sleuth.instrument.kafka.KafkaReceiverTest {
BraveTestTracing testTracing;
@Override
public TestTracingAware tracerTest() {
if (this.testTracing == null) {
this.testTracing = new BraveTestTracing();
}
return this.testTracing;
}
@Test
public void should_consider_native_headers() {
KafkaProducer<String, String> kafkaProducer = KafkaTestUtils
.buildTestKafkaProducer(kafkaContainer.getBootstrapServers());
ProducerRecord<String, String> producerRecord = new ProducerRecord<>(testTopic, "test", "test");
producerRecord.headers().add("b3", "000000000000000a-000000000000000b-1-000000000000000a".getBytes());
kafkaProducer.send(producerRecord);
await().atMost(Duration.ofSeconds(5)).until(() -> receivedCounter.intValue() == 1);
BDDAssertions.then(this.tracer.currentSpan()).isNull();
BDDAssertions.then(this.spans).hasSize(1);
FinishedSpan span = this.spans.get(0);
BDDAssertions.then(span.getKind()).isEqualTo(Span.Kind.CONSUMER);
BDDAssertions.then(span.getTraceId()).isEqualTo("000000000000000a");
BDDAssertions.then(span.getParentId()).isEqualTo("000000000000000b");
}
}

69
tests/brave/spring-cloud-sleuth-instrumentation-kafka-tests/src/test/java/org/springframework/cloud/sleuth/brave/instrument/kafka/KafkaSenderTest.java Normal file
View File

@@ -0,0 +1,69 @@
/*
* Copyright 2013-2021 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.cloud.sleuth.brave.instrument.kafka;
import java.util.Optional;
import java.util.concurrent.TimeUnit;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.apache.kafka.common.header.Header;
import org.assertj.core.api.BDDAssertions;
import org.junit.jupiter.api.Test;
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;
import reactor.kafka.sender.SenderRecord;
import reactor.kafka.sender.SenderResult;
import reactor.test.StepVerifier;
import org.springframework.cloud.sleuth.brave.BraveTestTracing;
import org.springframework.cloud.sleuth.test.TestTracingAware;
public class KafkaSenderTest extends org.springframework.cloud.sleuth.instrument.kafka.KafkaSenderTest {
BraveTestTracing testTracing;
@Override
public TestTracingAware tracerTest() {
if (this.testTracing == null) {
this.testTracing = new BraveTestTracing();
}
return this.testTracing;
}
@Test
public void should_inject_native_headers() throws InterruptedException {
ProducerRecord<String, String> producerRecord = new ProducerRecord<>(testTopic, "test", "test");
startKafkaConsumer();
Flux<SenderResult<Object>> senderResultFlux = this.kafkaSender
.send(Mono.just(SenderRecord.create(producerRecord, null)));
StepVerifier.create(senderResultFlux).expectNextCount(1).verifyComplete();
ConsumerRecord<String, String> consumerRecord = consumerRecords.poll(5, TimeUnit.SECONDS);
BDDAssertions.then(consumerRecord).isNotNull();
BDDAssertions.then(getHeaderValueOrNull(consumerRecord, "X-B3-TraceId")).isNotNull();
BDDAssertions.then(getHeaderValueOrNull(consumerRecord, "X-B3-SpanId")).isNotNull();
BDDAssertions.then(getHeaderValueOrNull(consumerRecord, "X-B3-Sampled")).isNotNull();
}
private static String getHeaderValueOrNull(ConsumerRecord<?, ?> consumerRecord, String header) {
return Optional.ofNullable(consumerRecord).map(ConsumerRecord::headers)
.map(headers -> headers.lastHeader(header)).map(Header::value).map(String::new).orElse(null);
}
}

5
tests/common/pom.xml
View File

@@ -149,6 +149,11 @@
<artifactId>brave-tests</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>io.projectreactor</groupId>
<artifactId>reactor-test</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>io.projectreactor.kafka</groupId>
<artifactId>reactor-kafka</artifactId>

159
tests/common/src/main/java/org/springframework/cloud/sleuth/instrument/kafka/KafkaConsumerTest.java Normal file
View File

@@ -0,0 +1,159 @@
/*
* Copyright 2013-2021 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.cloud.sleuth.instrument.kafka;
import java.time.Duration;
import java.util.HashMap;
import java.util.Map;
import java.util.UUID;
import java.util.concurrent.Executors;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.regex.Pattern;
import org.apache.kafka.clients.consumer.ConsumerConfig;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.apache.kafka.clients.consumer.ConsumerRecords;
import org.apache.kafka.clients.consumer.KafkaConsumer;
import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.apache.kafka.common.serialization.StringDeserializer;
import org.assertj.core.api.BDDAssertions;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Answers;
import org.mockito.BDDMockito;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import org.testcontainers.containers.KafkaContainer;
import org.testcontainers.containers.wait.strategy.Wait;
import org.testcontainers.junit.jupiter.Container;
import org.testcontainers.junit.jupiter.Testcontainers;
import org.testcontainers.utility.DockerImageName;
import org.springframework.beans.factory.BeanFactory;
import org.springframework.cloud.sleuth.Span;
import org.springframework.cloud.sleuth.Tracer;
import org.springframework.cloud.sleuth.exporter.FinishedSpan;
import org.springframework.cloud.sleuth.propagation.Propagator;
import org.springframework.cloud.sleuth.test.TestSpanHandler;
import org.springframework.cloud.sleuth.test.TestTracingAwareSupplier;
import org.springframework.core.ParameterizedTypeReference;
import org.springframework.core.ResolvableType;
import static org.awaitility.Awaitility.await;
@Testcontainers
@ExtendWith(MockitoExtension.class)
public abstract class KafkaConsumerTest implements TestTracingAwareSupplier {
protected String testTopic;
protected Tracer tracer = tracerTest().tracing().tracer();
protected Propagator propagator = tracerTest().tracing().propagator();
protected TestSpanHandler spans = tracerTest().handler();
protected TracingKafkaConsumer<String, String> kafkaConsumer;
private final AtomicBoolean consumerRun = new AtomicBoolean();
protected final AtomicInteger receivedCounter = new AtomicInteger(0);
@Mock(answer = Answers.RETURNS_DEEP_STUBS)
BeanFactory beanFactory;
@Container
protected static final KafkaContainer kafkaContainer = new KafkaContainer(
DockerImageName.parse("confluentinc/cp-kafka:6.1.1")).withExposedPorts(9093)
.waitingFor(Wait.forListeningPort());
@BeforeAll
static void setupAll() {
kafkaContainer.start();
}
@AfterAll
static void destroyAll() {
kafkaContainer.stop();
}
@BeforeEach
void setup() {
BDDMockito.given(this.beanFactory.getBean(Propagator.class)).willReturn(this.propagator);
BDDMockito.given(this.beanFactory.getBeanProvider(ResolvableType.forClassWithGenerics(Propagator.Getter.class,
ResolvableType.forType(new ParameterizedTypeReference<ConsumerRecord<?, ?>>() {
}))).getIfAvailable()).willReturn(new TracingKafkaPropagatorGetter());
testTopic = UUID.randomUUID().toString();
Map<String, Object> consumerProperties = new HashMap<>();
consumerProperties.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, kafkaContainer.getBootstrapServers());
consumerProperties.put(ConsumerConfig.GROUP_ID_CONFIG, "test-consumer-group");
consumerProperties.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
consumerProperties.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
consumerProperties.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");
kafkaConsumer = new TracingKafkaConsumer<>(new KafkaConsumer<>(consumerProperties), beanFactory);
consumerRun.set(true);
Executors.newSingleThreadExecutor().execute(() -> doStartKafkaConsumer(receivedCounter));
}
@AfterEach
void destroy() {
consumerRun.set(false);
}
@Test
public void should_create_and_finish_consumer_span() {
KafkaProducer<String, String> kafkaProducer = KafkaTestUtils
.buildTestKafkaProducer(kafkaContainer.getBootstrapServers());
ProducerRecord<String, String> producerRecord = new ProducerRecord<>(testTopic, "test", "test");
kafkaProducer.send(producerRecord);
kafkaProducer.close();
await().atMost(Duration.ofSeconds(5)).until(() -> receivedCounter.intValue() == 1);
BDDAssertions.then(this.tracer.currentSpan()).isNull();
BDDAssertions.then(this.spans).hasSize(1);
FinishedSpan span = this.spans.get(0);
BDDAssertions.then(span.getKind()).isEqualTo(Span.Kind.CONSUMER);
BDDAssertions.then(span.getTags()).isNotEmpty();
BDDAssertions.then(span.getTags().get("kafka.topic")).isEqualTo(testTopic);
BDDAssertions.then(span.getTags().get("kafka.offset")).isEqualTo("0");
BDDAssertions.then(span.getTags().get("kafka.partition")).isEqualTo("0");
}
private void doStartKafkaConsumer(AtomicInteger receivedCounter) {
this.kafkaConsumer.subscribe(Pattern.compile(this.testTopic));
while (this.consumerRun.get()) {
ConsumerRecords<String, String> records = this.kafkaConsumer.poll(Duration.ofSeconds(1));
for (ConsumerRecord<String, String> record : records) {
receivedCounter.incrementAndGet();
}
}
this.kafkaConsumer.close();
}
@Override
public void cleanUpTracing() {
this.spans.clear();
}
}

93
tests/common/src/main/java/org/springframework/cloud/sleuth/instrument/kafka/KafkaProducerTest.java
View File

@@ -19,34 +19,56 @@ package org.springframework.cloud.sleuth.instrument.kafka;
import java.time.Duration;
import java.util.HashMap;
import java.util.Map;
import java.util.UUID;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.Executors;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.regex.Pattern;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.apache.kafka.clients.consumer.ConsumerRecords;
import org.apache.kafka.clients.consumer.KafkaConsumer;
import org.apache.kafka.clients.producer.Callback;
import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerConfig;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.apache.kafka.common.serialization.StringSerializer;
import org.assertj.core.api.BDDAssertions;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Answers;
import org.mockito.BDDMockito;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import org.testcontainers.containers.KafkaContainer;
import org.testcontainers.containers.wait.strategy.Wait;
import org.testcontainers.junit.jupiter.Container;
import org.testcontainers.junit.jupiter.Testcontainers;
import org.testcontainers.utility.DockerImageName;
import org.springframework.beans.factory.BeanFactory;
import org.springframework.cloud.sleuth.Span;
import org.springframework.cloud.sleuth.Tracer;
import org.springframework.cloud.sleuth.exporter.FinishedSpan;
import org.springframework.cloud.sleuth.propagation.Propagator;
import org.springframework.cloud.sleuth.test.TestSpanHandler;
import org.springframework.cloud.sleuth.test.TestTracingAwareSupplier;
import org.springframework.core.ParameterizedTypeReference;
import org.springframework.core.ResolvableType;
import static org.awaitility.Awaitility.await;
@Testcontainers
@ExtendWith(MockitoExtension.class)
public abstract class KafkaProducerTest implements TestTracingAwareSupplier {
protected String testTopic;
protected Tracer tracer = tracerTest().tracing().tracer();
protected Propagator propagator = tracerTest().tracing().propagator();
@@ -55,39 +77,82 @@ public abstract class KafkaProducerTest implements TestTracingAwareSupplier {
protected TracingKafkaProducer<String, String> kafkaProducer;
private final AtomicBoolean consumerRun = new AtomicBoolean();
protected final BlockingQueue<ConsumerRecord<String, String>> consumerRecords = new LinkedBlockingQueue<>();
@Mock(answer = Answers.RETURNS_DEEP_STUBS)
BeanFactory beanFactory;
@Container
protected final KafkaContainer kafkaContainer = new KafkaContainer(
DockerImageName.parse("confluentinc/cp-kafka:5.2.1")).withExposedPorts(9093)
protected static final KafkaContainer kafkaContainer = new KafkaContainer(
DockerImageName.parse("confluentinc/cp-kafka:6.1.1")).withExposedPorts(9093)
.waitingFor(Wait.forListeningPort());
@BeforeAll
static void setupAll() {
kafkaContainer.start();
}
@AfterAll
static void destroyAll() {
kafkaContainer.stop();
}
@BeforeEach
void setup() {
kafkaContainer.start();
Map<String, Object> properties = new HashMap<>();
properties.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, kafkaContainer.getBootstrapServers());
properties.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
properties.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
kafkaProducer = new TracingKafkaProducer<>(new KafkaProducer<>(properties), tracer, propagator,
new TracingKafkaPropagatorSetter());
BDDMockito.given(this.beanFactory.getBean(Tracer.class)).willReturn(this.tracer);
BDDMockito.given(this.beanFactory.getBean(Propagator.class)).willReturn(this.propagator);
BDDMockito.given(this.beanFactory.getBeanProvider(ResolvableType.forClassWithGenerics(Propagator.Setter.class,
ResolvableType.forType(new ParameterizedTypeReference<ProducerRecord<?, ?>>() {
}))).getIfAvailable()).willReturn(new TracingKafkaPropagatorSetter());
testTopic = UUID.randomUUID().toString();
Map<String, Object> producerProperties = new HashMap<>();
producerProperties.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, kafkaContainer.getBootstrapServers());
producerProperties.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
producerProperties.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
kafkaProducer = new TracingKafkaProducer<>(new KafkaProducer<>(producerProperties), beanFactory);
consumerRun.set(true);
consumerRecords.clear();
}
@AfterEach
void destroy() {
kafkaContainer.stop();
this.kafkaProducer.close();
consumerRun.set(false);
}
@Test
public void should_create_and_finish_producer_span() {
AtomicBoolean acknowledged = new AtomicBoolean(false);
Callback callback = (metadata, ex) -> acknowledged.set(true);
ProducerRecord<String, String> producerRecord = new ProducerRecord<>("spring-cloud-sleuth-otel-topic", "test",
"test");
ProducerRecord<String, String> producerRecord = new ProducerRecord<>(testTopic, "test", "test");
this.kafkaProducer.send(producerRecord, callback);
await().atMost(Duration.ofSeconds(5)).until(acknowledged::get);
BDDAssertions.then(this.tracer.currentSpan()).isNull();
BDDAssertions.then(this.spans).isNotEmpty();
BDDAssertions.then(this.spans.get(0).getKind()).isEqualTo(Span.Kind.PRODUCER);
BDDAssertions.then(this.spans).hasSize(1);
FinishedSpan span = this.spans.get(0);
BDDAssertions.then(span.getKind()).isEqualTo(Span.Kind.PRODUCER);
BDDAssertions.then(span.getTags().get("kafka.topic")).isEqualTo(testTopic);
}
protected void startKafkaConsumer() {
Executors.newSingleThreadExecutor().execute(this::doStartKafkaConsumer);
}
private void doStartKafkaConsumer() {
KafkaConsumer<String, String> kafkaConsumer = KafkaTestUtils
.buildTestKafkaConsumer(kafkaContainer.getBootstrapServers());
kafkaConsumer.subscribe(Pattern.compile(testTopic));
while (consumerRun.get()) {
ConsumerRecords<String, String> records = kafkaConsumer.poll(Duration.ofSeconds(1));
for (ConsumerRecord<String, String> record : records) {
consumerRecords.offer(record);
}
}
kafkaConsumer.close();
}
@Override

150
tests/common/src/main/java/org/springframework/cloud/sleuth/instrument/kafka/KafkaReceiverTest.java Normal file
View File

@@ -0,0 +1,150 @@
/*
* Copyright 2013-2021 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.cloud.sleuth.instrument.kafka;
import java.time.Duration;
import java.util.Collections;
import java.util.HashMap;
import java.util.Map;
import java.util.UUID;
import java.util.concurrent.atomic.AtomicInteger;
import org.apache.kafka.clients.consumer.ConsumerConfig;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.apache.kafka.common.serialization.StringDeserializer;
import org.assertj.core.api.BDDAssertions;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Answers;
import org.mockito.BDDMockito;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import org.testcontainers.containers.KafkaContainer;
import org.testcontainers.containers.wait.strategy.Wait;
import org.testcontainers.junit.jupiter.Container;
import org.testcontainers.junit.jupiter.Testcontainers;
import org.testcontainers.utility.DockerImageName;
import reactor.core.Disposable;
import reactor.core.scheduler.Schedulers;
import reactor.kafka.receiver.KafkaReceiver;
import reactor.kafka.receiver.ReceiverOptions;
import org.springframework.beans.factory.BeanFactory;
import org.springframework.cloud.sleuth.Span;
import org.springframework.cloud.sleuth.Tracer;
import org.springframework.cloud.sleuth.exporter.FinishedSpan;
import org.springframework.cloud.sleuth.propagation.Propagator;
import org.springframework.cloud.sleuth.test.TestSpanHandler;
import org.springframework.cloud.sleuth.test.TestTracingAwareSupplier;
import org.springframework.core.ParameterizedTypeReference;
import org.springframework.core.ResolvableType;
import static org.awaitility.Awaitility.await;
@Testcontainers
@ExtendWith(MockitoExtension.class)
public abstract class KafkaReceiverTest implements TestTracingAwareSupplier {
protected String testTopic;
protected Tracer tracer = tracerTest().tracing().tracer();
protected Propagator propagator = tracerTest().tracing().propagator();
protected TestSpanHandler spans = tracerTest().handler();
private Disposable consumerSubscription;
protected final AtomicInteger receivedCounter = new AtomicInteger(0);
@Mock(answer = Answers.RETURNS_DEEP_STUBS)
BeanFactory beanFactory;
@Container
protected static final KafkaContainer kafkaContainer = new KafkaContainer(
DockerImageName.parse("confluentinc/cp-kafka:6.1.1")).withExposedPorts(9093)
.waitingFor(Wait.forListeningPort());
@BeforeAll
static void setupAll() {
kafkaContainer.start();
}
@AfterAll
static void destroyAll() {
kafkaContainer.stop();
}
@BeforeEach
void setup() {
BDDMockito.given(this.beanFactory.getBean(Propagator.class)).willReturn(this.propagator);
BDDMockito.given(this.beanFactory.getBeanProvider(ResolvableType.forClassWithGenerics(Propagator.Getter.class,
ResolvableType.forType(new ParameterizedTypeReference<ConsumerRecord<?, ?>>() {
}))).getIfAvailable()).willReturn(new TracingKafkaPropagatorGetter());
testTopic = UUID.randomUUID().toString();
Map<String, Object> consumerProperties = new HashMap<>();
consumerProperties.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, kafkaContainer.getBootstrapServers());
consumerProperties.put(ConsumerConfig.GROUP_ID_CONFIG, "test-consumer-group");
consumerProperties.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
consumerProperties.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
consumerProperties.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");
ReceiverOptions<String, String> options = ReceiverOptions.create(consumerProperties);
options = options.withKeyDeserializer(new StringDeserializer()).withValueDeserializer(new StringDeserializer())
.subscription(Collections.singletonList(testTopic));
KafkaReceiver<String, String> kafkaReceiver = KafkaReceiver.create(new TracingKafkaConsumerFactory(beanFactory),
options);
this.consumerSubscription = kafkaReceiver.receive().subscribeOn(Schedulers.single())
.subscribe(record -> receivedCounter.incrementAndGet());
this.receivedCounter.set(0);
}
@AfterEach
void destroy() {
this.consumerSubscription.dispose();
}
@Test
public void should_create_and_finish_consumer_span() {
KafkaProducer<String, String> kafkaProducer = KafkaTestUtils
.buildTestKafkaProducer(kafkaContainer.getBootstrapServers());
ProducerRecord<String, String> producerRecord = new ProducerRecord<>(testTopic, "test", "test");
kafkaProducer.send(producerRecord);
await().atMost(Duration.ofSeconds(5)).until(() -> receivedCounter.intValue() == 1);
BDDAssertions.then(this.tracer.currentSpan()).isNull();
BDDAssertions.then(this.spans).hasSize(1);
FinishedSpan span = this.spans.get(0);
BDDAssertions.then(span.getKind()).isEqualTo(Span.Kind.CONSUMER);
BDDAssertions.then(span.getTags()).isNotEmpty();
BDDAssertions.then(span.getTags().get("kafka.topic")).isEqualTo(testTopic);
BDDAssertions.then(span.getTags().get("kafka.offset")).isEqualTo("0");
BDDAssertions.then(span.getTags().get("kafka.partition")).isEqualTo("0");
}
@Override
public void cleanUpTracing() {
this.spans.clear();
}
}

169
tests/common/src/main/java/org/springframework/cloud/sleuth/instrument/kafka/KafkaSenderTest.java Normal file
View File

@@ -0,0 +1,169 @@
/*
* Copyright 2013-2021 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.cloud.sleuth.instrument.kafka;
import java.time.Duration;
import java.util.HashMap;
import java.util.Map;
import java.util.UUID;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.Executors;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.regex.Pattern;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.apache.kafka.clients.consumer.ConsumerRecords;
import org.apache.kafka.clients.consumer.KafkaConsumer;
import org.apache.kafka.clients.producer.ProducerConfig;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.apache.kafka.common.serialization.StringSerializer;
import org.assertj.core.api.BDDAssertions;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Answers;
import org.mockito.BDDMockito;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import org.testcontainers.containers.KafkaContainer;
import org.testcontainers.containers.wait.strategy.Wait;
import org.testcontainers.junit.jupiter.Container;
import org.testcontainers.junit.jupiter.Testcontainers;
import org.testcontainers.utility.DockerImageName;
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;
import reactor.kafka.sender.KafkaSender;
import reactor.kafka.sender.SenderOptions;
import reactor.kafka.sender.SenderRecord;
import reactor.kafka.sender.SenderResult;
import reactor.test.StepVerifier;
import org.springframework.beans.factory.BeanFactory;
import org.springframework.cloud.sleuth.Span;
import org.springframework.cloud.sleuth.Tracer;
import org.springframework.cloud.sleuth.exporter.FinishedSpan;
import org.springframework.cloud.sleuth.propagation.Propagator;
import org.springframework.cloud.sleuth.test.TestSpanHandler;
import org.springframework.cloud.sleuth.test.TestTracingAwareSupplier;
import org.springframework.core.ParameterizedTypeReference;
import org.springframework.core.ResolvableType;
@Testcontainers
@ExtendWith(MockitoExtension.class)
public abstract class KafkaSenderTest implements TestTracingAwareSupplier {
protected String testTopic;
protected Tracer tracer = tracerTest().tracing().tracer();
protected Propagator propagator = tracerTest().tracing().propagator();
protected TestSpanHandler spans = tracerTest().handler();
protected KafkaSender<String, String> kafkaSender;
private final AtomicBoolean consumerRun = new AtomicBoolean();
protected final BlockingQueue<ConsumerRecord<String, String>> consumerRecords = new LinkedBlockingQueue<>();
@Mock(answer = Answers.RETURNS_DEEP_STUBS)
BeanFactory beanFactory;
@Container
protected static final KafkaContainer kafkaContainer = new KafkaContainer(
DockerImageName.parse("confluentinc/cp-kafka:6.1.1")).withExposedPorts(9093)
.waitingFor(Wait.forListeningPort());
@BeforeAll
static void setupAll() {
kafkaContainer.start();
}
@AfterAll
static void destroyAll() {
kafkaContainer.stop();
}
@BeforeEach
void setup() {
BDDMockito.given(this.beanFactory.getBean(Tracer.class)).willReturn(this.tracer);
BDDMockito.given(this.beanFactory.getBean(Propagator.class)).willReturn(this.propagator);
BDDMockito.given(this.beanFactory.getBeanProvider(ResolvableType.forClassWithGenerics(Propagator.Setter.class,
ResolvableType.forType(new ParameterizedTypeReference<ProducerRecord<?, ?>>() {
}))).getIfAvailable()).willReturn(new TracingKafkaPropagatorSetter());
testTopic = UUID.randomUUID().toString();
Map<String, Object> producerProperties = new HashMap<>();
producerProperties.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, kafkaContainer.getBootstrapServers());
producerProperties.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
producerProperties.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
this.kafkaSender = KafkaSender.create(new TracingKafkaProducerFactory(beanFactory),
SenderOptions.create(producerProperties));
consumerRun.set(true);
consumerRecords.clear();
}
@AfterEach
void destroy() {
consumerRun.set(false);
}
@Test
public void should_create_and_finish_producer_span() throws InterruptedException {
ProducerRecord<String, String> producerRecord = new ProducerRecord<>(testTopic, "test", "test");
startKafkaConsumer();
Flux<SenderResult<Object>> senderResultFlux = this.kafkaSender
.send(Mono.just(SenderRecord.create(producerRecord, null)));
StepVerifier.create(senderResultFlux).expectNextCount(1).verifyComplete();
consumerRecords.poll(5, TimeUnit.SECONDS);
BDDAssertions.then(this.tracer.currentSpan()).isNull();
BDDAssertions.then(this.spans).hasSize(1);
FinishedSpan span = this.spans.get(0);
BDDAssertions.then(span.getKind()).isEqualTo(Span.Kind.PRODUCER);
BDDAssertions.then(span.getTags().get("kafka.topic")).isEqualTo(testTopic);
}
protected void startKafkaConsumer() {
Executors.newSingleThreadExecutor().execute(this::doStartKafkaConsumer);
}
private void doStartKafkaConsumer() {
KafkaConsumer<String, String> kafkaConsumer = KafkaTestUtils
.buildTestKafkaConsumer(kafkaContainer.getBootstrapServers());
kafkaConsumer.subscribe(Pattern.compile(this.testTopic));
while (this.consumerRun.get()) {
ConsumerRecords<String, String> records = kafkaConsumer.poll(Duration.ofSeconds(1));
for (ConsumerRecord<String, String> record : records) {
this.consumerRecords.offer(record);
}
}
kafkaConsumer.close();
}
@Override
public void cleanUpTracing() {
this.spans.clear();
}
}

52
tests/common/src/main/java/org/springframework/cloud/sleuth/instrument/kafka/KafkaTestUtils.java Normal file
View File

@@ -0,0 +1,52 @@
/*
* Copyright 2013-2021 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.cloud.sleuth.instrument.kafka;
import java.util.HashMap;
import java.util.Map;
import org.apache.kafka.clients.consumer.ConsumerConfig;
import org.apache.kafka.clients.consumer.KafkaConsumer;
import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerConfig;
import org.apache.kafka.common.serialization.StringDeserializer;
import org.apache.kafka.common.serialization.StringSerializer;
public final class KafkaTestUtils {
private KafkaTestUtils() {
}
public static KafkaProducer<String, String> buildTestKafkaProducer(String bootstrapServers) {
Map<String, Object> producerProperties = new HashMap<>();
producerProperties.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServers);
producerProperties.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
producerProperties.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
return new KafkaProducer<>(producerProperties);
}
public static KafkaConsumer<String, String> buildTestKafkaConsumer(String bootstrapServers) {
Map<String, Object> consumerProperties = new HashMap<>();
consumerProperties.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServers);
consumerProperties.put(ConsumerConfig.GROUP_ID_CONFIG, "test-consumer-group");
consumerProperties.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
consumerProperties.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
consumerProperties.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");
return new KafkaConsumer<>(consumerProperties);
}
}