3. Features

[Important]Important

If using Zipkin, configure the percentage of spans exported using spring.sleuth.sampler.percentage (default 0.1, i.e. 10%). Otherwise you might think that Sleuth is not working cause it’s omitting some spans.

[Note]Note

the SLF4J MDC is always set and logback users will immediately see the trace and span ids in logs per the example above. Other logging systems have to configure their own formatter to get the same result. The default is logging.pattern.level set to %5p [${spring.zipkin.service.name:${spring.application.name:-}},%X{X-B3-TraceId:-},%X{X-B3-SpanId:-},%X{X-Span-Export:-}] (this is a Spring Boot feature for logback users). This means that if you’re not using SLF4J this pattern WILL NOT be automatically applied.

3.1 Introduction to Brave

[Important]Important

Starting with version 2.0.0 Spring Cloud Sleuth uses Brave as the tracing library. For your convenience we’re embedding part of the Brave’s docs here.

Brave is a library used to capture and report latency information about distributed operations to Zipkin. Most users won’t use Brave directly, rather libraries or frameworks than employ Brave on their behalf.

This module includes tracer creates and joins spans that model the latency of potentially distributed work. It also includes libraries to propagate the trace context over network boundaries, for example, via http headers.

3.1.1 Tracing

Most importantly, you need a brave.Tracer, configured to [report to Zipkin] (https://github.com/openzipkin/zipkin-reporter-java).

Here’s an example setup that sends trace data (spans) to Zipkin over http (as opposed to Kafka).

class MyClass {

    private final Tracer tracer;

    // Tracer will be autowired
    MyClass(Tracer tracer) {
        this.tracer = tracer;
    }

    void doSth() {
        Span span = tracer.newTrace().name("encode").start();
        // ...
    }
}
[Important]Important

If your span contains a name greater than 50 chars, then that name will be truncated to 50 chars. Your names have to be explicit and concrete. Big names lead to latency issues and sometimes even thrown exceptions.

3.1.2 Tracing

The tracer creates and joins spans that model the latency of potentially distributed work. It can employ sampling to reduce overhead in process or to reduce the amount of data sent to Zipkin.

Spans returned by a tracer report data to Zipkin when finished, or do nothing if unsampled. After starting a span, you can annotate events of interest or add tags containing details or lookup keys.

Spans have a context which includes trace identifiers that place it at the correct spot in the tree representing the distributed operation.

3.1.3 Local Tracing

When tracing local code, just run it inside a span.

Span span = tracer.newTrace().name("encode").start();
try {
  doSomethingExpensive();
} finally {
  span.finish();
}

In the above example, the span is the root of the trace. In many cases, you will be a part of an existing trace. When this is the case, call newChild instead of newTrace

Span span = tracer.newChild(root.context()).name("encode").start();
try {
  doSomethingExpensive();
} finally {
  span.finish();
}

3.1.4 Customizing spans

Once you have a span, you can add tags to it, which can be used as lookup keys or details. For example, you might add a tag with your runtime version.

span.tag("clnt/finagle.version", "6.36.0");

When exposing the ability to customize spans to third parties, prefer brave.SpanCustomizer as opposed to brave.Span. The former is simpler to understand and test, and doesn’t tempt users with span lifecycle hooks.

interface MyTraceCallback {
  void request(Request request, SpanCustomizer customizer);
}

Since brave.Span implements brave.SpanCustomizer, it is just as easy for you to pass to users.

Ex.

for (MyTraceCallback callback : userCallbacks) {
  callback.request(request, span);
}

3.1.5 Implicitly looking up the current span

Sometimes you won’t know if a trace is in progress or not, and you don’t want users to do null checks. brave.CurrentSpanCustomizer adds to any span that’s in progress or drops data accordingly.

Ex.

// user code can then inject this without a chance of it being null.
@Autowire SpanCustomizer span;

void userCode() {
  span.annotate("tx.started");
  ...
}

3.1.6 RPC tracing

Check for instrumentation written here and Zipkin’s list before rolling your own RPC instrumentation!

RPC tracing is often done automatically by interceptors. Under the scenes, they add tags and events that relate to their role in an RPC operation.

Here’s an example of a client span:

// before you send a request, add metadata that describes the operation
span = tracer.newTrace().name("get").type(CLIENT);
span.tag("clnt/finagle.version", "6.36.0");
span.tag(TraceKeys.HTTP_PATH, "/api");
span.remoteEndpoint(Endpoint.builder()
    .serviceName("backend")
    .ipv4(127 << 24 | 1)
    .port(8080).build());

// when the request is scheduled, start the span
span.start();

// if you have callbacks for when data is on the wire, note those events
span.annotate(Constants.WIRE_SEND);
span.annotate(Constants.WIRE_RECV);

// when the response is complete, finish the span
span.finish();

One-Way tracing

Sometimes you need to model an asynchronous operation, where there is a request, but no response. In normal RPC tracing, you use span.finish() which indicates the response was received. In one-way tracing, you use span.flush() instead, as you don’t expect a response.

Here’s how a client might model a one-way operation

// start a new span representing a client request
oneWaySend = tracer.newSpan(parent).kind(Span.Kind.CLIENT);

// Add the trace context to the request, so it can be propagated in-band
tracing.propagation().injector(Request::addHeader)
                     .inject(oneWaySend.context(), request);

// fire off the request asynchronously, totally dropping any response
request.execute();

// start the client side and flush instead of finish
oneWaySend.start().flush();

And here’s how a server might handle this..

// pull the context out of the incoming request
extractor = tracing.propagation().extractor(Request::getHeader);

// convert that context to a span which you can name and add tags to
oneWayReceive = nextSpan(tracer, extractor.extract(request))
    .name("process-request")
    .kind(SERVER)
    ... add tags etc.

// start the server side and flush instead of finish
oneWayReceive.start().flush();

// you should not modify this span anymore as it is complete. However,
// you can create children to represent follow-up work.
next = tracer.newSpan(oneWayReceive.context()).name("step2").start();

Note The above propagation logic is a simplified version of our [http handlers](https://github.com/openzipkin/sleuth/tree/master/instrumentation/http#http-server).

There’s a working example of a one-way span [here](src/test/java/sleuth/features/async/OneWaySpanTest.java).