I had flakes on TraceAsyncIntegrationTests and it was hard to tell why
as there were so many problems with it. It used a normal list in async
setting, it unnecessarily created extra spans, it added a hook inside
the async method which if passed only told you it was in the middle of
an op, not the end of it. The method danced around making failure in
console almost impossible to decipher.
This took me a few hours to make just a normal test. Hopefully, the
others will keep passing as I can't afford time to redo all of them.
`SpanHandler` is the base type for the now deprecated `FinishedSpanHandler`.
Notable, it can not just handle things at the end of a recording, but also the
beginning.
For example, this permits set-once baggage without the HTTP abstraction:
```java
static final BaggageField EPOCH_SECONDS = BaggageField.create("epoch_seconds");
static final class RootOnlyBaggage extends SpanHandler {
@Override
public boolean begin(TraceContext context, MutableSpan span, @Nullable TraceContext parent) {
if (EPOCH_SECONDS.getValue(context) == null) { // only set at the first span
long epochSeconds = System.currentTimeMillis() / 1000;
EPOCH_SECONDS.updateValue(context, String.valueOf(epochSeconds));
}
return true;
}
@Override public boolean end(TraceContext context, MutableSpan span, Cause cause) {
Tags.BAGGAGE_FIELD.tag(EPOCH_SECONDS, context, span);
return true;
}
}
```
As the parent is available, it can also facilitate advanced tasks like counting
children, or summarizing entire local roots.
See https://github.com/openzipkin/brave/tree/master/brave/src/test/java/brave/features/handler
and https://github.com/openzipkin/brave/blob/master/brave/src/main/java/brave/handler/SpanHandler.java for more
`ExceptionLoggingFilter` logs "Uncaught exception thrown" to error level
when there is a synchronous exception not otherwise swallowed. This is a
cure worse than the disease. This disables it by default and the 3.x
should end the years of problems it caused.
Before, spring-cloud-sleuth-zipkin had to import `SamplerAutoConfiguration`
directly to unwind a sampler ordering problem caused by `TraceAutoConfiguration`
defining the default `Sampler` bean.
This fixes that by moving the default `Sampler` to where it belongs
(`SamplerAutoConfiguration`) and having `TraceAutoConfiguration` import
the sampling configuration directly as opposed to relying on auto-configuration
ordering. Finally it removes the mistake of setting `SamplerAutoConfiguration`
as auto-configuration in the first place.
The name `SamplerAutoConfiguration` was left alone because changing it would
interfere with 3rd party code that formerly imported it to correct this issue
in their non-zipkin setups.
Fixes#1618
To reduce confusion and overhead, the following custom spring-messaging headers added in Sleuth 1.0 are no longer sent, and a log warning is issued once if they are by outside code.
* spanId
* spanSampled
* spanParentSpanId
* spanTraceId
* spanFlags
Sending the above headers actually increases the headers by up to 10 because they are duplicated in the "native" part of messages. This overhead is extreme especially if messages never leave the process.
The solution is to only send [b3 single format](https://github.com/openzipkin/b3-propagation#single-header), which has been in sleuth since 2.0 and is compatible with JMS. The B3 single format is always parsed and takes precedence, even if multiple headers are sent, so this is a safe change.
Note: Unlike RPC, messaging spans never join with their parent. Better performance is achieved by not propagating the producer's parentId downstream.
Note: Deprecated spring-messaging headers such "spanTraceId" as are still read in Sleuth 3.x. However, they will not be at some point in the future. Please pay attention to the log messages and update your code if you are accidentally using them.
This moves code and properties under org.springframework.cloud.sleuth.baggage
Properties are now under "spring.sleuth.baggage"
Those coming from 2.x should migrate with the following:
* spring.sleuth.baggage-keys -> `BaggagePropagationCustomizer`
* spring.sleuth.local-keys -> spring.sleuth.baggage.local-fields
* spring.sleuth.propagation-keys -> spring.sleuth.baggage.remote-fields
* spring.sleuth.propagation.tag.whitelisted-keys -> spring.sleuth.baggage.tag-fields
* spring.sleuth.log.slf4j.whitelisted-mdc-keys -> spring.sleuth.baggage.correlation-fields
* spring.sleuth.log.slf4j.enabled -> spring.sleuth.baggage.correlation-enabled
Those using MDC should know that for performance reasons, we no longer
set the following fields: `parentId` `spanExportable`.
We also do not set fields to "dirty" by default. Doing this by default
raised the overhead substantially, especially in reactive applications.
Those who want sleuth to override manual `MDC.put` operations as
described in #1416, or with to add back `parentId` `spanExportable` need
to define their own bean:
Ex. this is the former setup:
```java
@Bean CorrelationScopeDecorator oldConfig(List<String> myFieldNames) {
CorrelationScopeDecorator.Builder builder = MDCScopeDecorator.newBuilder().clear()
.add(SingleCorrelationField.create(BaggageFields.TRACE_ID))
.add(SingleCorrelationField.create(BaggageFields.PARENT_ID))
.add(SingleCorrelationField.create(BaggageFields.SPAN_ID))
.add(SingleCorrelationField.newBuilder(BaggageFields.SAMPLED)
.name("spanExportable").build());
// Set all fields dirty, so that any changes made by MDC directly are reverted.
for (String name : myFieldNames) {
builder.add(SingleCorrelationField.newBuilder(BaggageField.create(name))
.dirty().build());
}
return builder.build();
}
```
In many places, the trace context of callbacks was accidentally set to
the client span, not the invocation context. I noticed a hack trying to
work around this. This code fixes all the problems around context. It
also removes some sporadic logging, which was only applied to a few
hooks.
Finally, this adds Brave tests which would have caught the problems
earlier. Notably, there is still more work to do as this will not help
with duplicate instrumentation, which is normal when reactor-netty is
the WebClient's HTTP connector.
