Files
spring-cloud-sleuth/spring-cloud-sleuth.xml
2018-01-21 08:30:33 +00:00

1859 lines
105 KiB
XML
Raw Blame History

This file contains invisible Unicode characters
This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
<?xml version="1.0" encoding="UTF-8"?>
<?asciidoc-toc maxdepth="8"?>
<?asciidoc-numbered?>
<book xmlns="http://docbook.org/ns/docbook" xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en">
<info>
<title>Spring Cloud Sleuth</title>
<date>2018-01-21</date>
<author>
<personname>
<firstname>Adrian Cole, Spencer Gibb, Marcin Grzejszczak, Dave Syer</firstname>
</personname>
</author>
<authorinitials>A</authorinitials>
</info>
<preface>
<title></title>
<simpara><emphasis role="strong">2.0.0.BUILD-SNAPSHOT</emphasis></simpara>
</preface>
<chapter xml:id="_introduction">
<title>Introduction</title>
<simpara>Spring Cloud Sleuth implements a distributed tracing solution for <link xl:href="http://cloud.spring.io">Spring Cloud</link>.</simpara>
<section xml:id="_terminology">
<title>Terminology</title>
<simpara>Spring Cloud Sleuth borrows <link xl:href="http://research.google.com/pubs/pub36356.html">Dapper&#8217;s</link> terminology.</simpara>
<simpara><emphasis role="strong">Span:</emphasis> The basic unit of work. For example, sending an RPC is a new span, as is sending a response to an
RPC. Span&#8217;s are identified by a unique 64-bit ID for the span and another 64-bit ID for the trace the span
is a part of. Spans also have other data, such as descriptions, timestamped events, key-value
annotations (tags), the ID of the span that caused them, and process ID&#8217;s (normally IP address).</simpara>
<simpara>Spans are started and stopped, and they keep track of their timing information. Once you create a
span, you must stop it at some point in the future.</simpara>
<tip>
<simpara>The initial span that starts a trace is called a <literal>root span</literal>. The value of span id
of that span is equal to trace id.</simpara>
</tip>
<simpara><emphasis role="strong">Trace:</emphasis> A set of spans forming a tree-like structure. For example, if you are running a distributed
big-data store, a trace might be formed by a put request.</simpara>
<simpara><emphasis role="strong">Annotation:</emphasis> is used to record existence of an event in time. With
<link xl:href="https://github.com/openzipkin/brave">Brave</link> instrumentation we no longer need to set special events
for <link xl:href="https://zipkin.io/">Zipkin</link> to understand who the client and server are and where
the request started and where it has ended. For learning purposes
however we will mark these events to highlight what kind
of an action took place.</simpara>
<itemizedlist>
<listitem>
<simpara><emphasis role="strong">cs</emphasis> - Client Sent - The client has made a request. This annotation depicts the start of the span.</simpara>
</listitem>
<listitem>
<simpara><emphasis role="strong">sr</emphasis> - Server Received - The server side got the request and will start processing it.
If one subtracts the cs timestamp from this timestamp one will receive the network latency.</simpara>
</listitem>
<listitem>
<simpara><emphasis role="strong">ss</emphasis> - Server Sent - Annotated upon completion of request processing (when the response
got sent back to the client). If one subtracts the sr timestamp from this timestamp one
will receive the time needed by the server side to process the request.</simpara>
</listitem>
<listitem>
<simpara><emphasis role="strong">cr</emphasis> - Client Received - Signifies the end of the span. The client has successfully received the
response from the server side. If one subtracts the cs timestamp from this timestamp one
will receive the whole time needed by the client to receive the response from the server.</simpara>
</listitem>
</itemizedlist>
<simpara>Visualization of what <emphasis role="strong">Span</emphasis> and <emphasis role="strong">Trace</emphasis> will look in a system together with the Zipkin annotations:</simpara>
<informalfigure>
<mediaobject>
<imageobject>
<imagedata fileref="https://raw.githubusercontent.com/spring-cloud/spring-cloud-sleuth/master/docs/src/main/asciidoc/images/trace-id.png"/>
</imageobject>
<textobject><phrase>Trace Info propagation</phrase></textobject>
</mediaobject>
</informalfigure>
<simpara>Each color of a note signifies a span (7 spans - from <emphasis role="strong">A</emphasis> to <emphasis role="strong">G</emphasis>). If you have such information in the note:</simpara>
<screen>Trace Id = X
Span Id = D
Client Sent</screen>
<simpara>That means that the current span has <emphasis role="strong">Trace-Id</emphasis> set to <emphasis role="strong">X</emphasis>, <emphasis role="strong">Span-Id</emphasis> set to <emphasis role="strong">D</emphasis>. Also, the
<emphasis role="strong">Client Sent</emphasis> event took place.</simpara>
<simpara>This is how the visualization of the parent / child relationship of spans would look like:</simpara>
<informalfigure>
<mediaobject>
<imageobject>
<imagedata fileref="https://raw.githubusercontent.com/spring-cloud/spring-cloud-sleuth/master/docs/src/main/asciidoc/images/parents.png"/>
</imageobject>
<textobject><phrase>Parent child relationship</phrase></textobject>
</mediaobject>
</informalfigure>
</section>
<section xml:id="_purpose">
<title>Purpose</title>
<simpara>In the following sections the example from the image above will be taken into consideration.</simpara>
<section xml:id="_distributed_tracing_with_zipkin">
<title>Distributed tracing with Zipkin</title>
<simpara>Altogether there are <emphasis role="strong">7 spans</emphasis> . If you go to traces in Zipkin you will see this number in the second trace:</simpara>
<informalfigure>
<mediaobject>
<imageobject>
<imagedata fileref="https://raw.githubusercontent.com/spring-cloud/spring-cloud-sleuth/master/docs/src/main/asciidoc/images/zipkin-traces.png"/>
</imageobject>
<textobject><phrase>Traces</phrase></textobject>
</mediaobject>
</informalfigure>
<simpara>However if you pick a particular trace then you will see <emphasis role="strong">4 spans</emphasis>:</simpara>
<informalfigure>
<mediaobject>
<imageobject>
<imagedata fileref="https://raw.githubusercontent.com/spring-cloud/spring-cloud-sleuth/master/docs/src/main/asciidoc/images/zipkin-ui.png"/>
</imageobject>
<textobject><phrase>Traces Info propagation</phrase></textobject>
</mediaobject>
</informalfigure>
<note>
<simpara>When picking a particular trace you will see merged spans. That means that if there were 2 spans sent to
Zipkin with Server Received and Server Sent / Client Received and Client Sent
annotations then they will presented as a single span.</simpara>
</note>
<simpara>Why is there a difference between the 7 and 4 spans in this case?</simpara>
<itemizedlist>
<listitem>
<simpara>2 spans come from <literal>http:/start</literal> span. It has the Server Received (SR) and Server Sent (SS) annotations.</simpara>
</listitem>
<listitem>
<simpara>2 spans come from the RPC call from <literal>service1</literal> to <literal>service2</literal> to the <literal>http:/foo</literal> endpoint. The Client Sent (CS)
and Client Received (CR) events took place on <literal>service1</literal> side. Server Received (SR) and Server Sent (SS) events took place
on the <literal>service2</literal> side. Physically there are 2 spans but they form 1 logical span related to an RPC call.</simpara>
</listitem>
<listitem>
<simpara>2 spans come from the RPC call from <literal>service2</literal> to <literal>service3</literal> to the <literal>http:/bar</literal> endpoint. The Client Sent (CS)
and Client Received (CR) events took place on <literal>service2</literal> side. Server Received (SR) and Server Sent (SS) events took place
on the <literal>service3</literal> side. Physically there are 2 spans but they form 1 logical span related to an RPC call.</simpara>
</listitem>
<listitem>
<simpara>2 spans come from the RPC call from <literal>service2</literal> to <literal>service4</literal> to the <literal>http:/baz</literal> endpoint. The Client Sent (CS)
and Client Received (CR) events took place on <literal>service2</literal> side. Server Received (SR) and Server Sent (SS) events took place
on the <literal>service4</literal> side. Physically there are 2 spans but they form 1 logical span related to an RPC call.</simpara>
</listitem>
</itemizedlist>
<simpara>So if we count the physical spans we have <emphasis role="strong">1</emphasis> from <literal>http:/start</literal>, <emphasis role="strong">2</emphasis> from <literal>service1</literal> calling <literal>service2</literal>, <emphasis role="strong">2</emphasis> form <literal>service2</literal>
calling <literal>service3</literal> and <emphasis role="strong">2</emphasis> from <literal>service2</literal> calling <literal>service4</literal>. Altogether <emphasis role="strong">7</emphasis> spans.</simpara>
<simpara>Logically we see the information of <emphasis role="strong">Total Spans: 4</emphasis> because we have <emphasis role="strong">1</emphasis> span related to the incoming request
to <literal>service1</literal> and <emphasis role="strong">3</emphasis> spans related to RPC calls.</simpara>
</section>
<section xml:id="_visualizing_errors">
<title>Visualizing errors</title>
<simpara>Zipkin allows you to visualize errors in your trace. When an exception was thrown and wasn&#8217;t caught then we&#8217;re
setting proper tags on the span which Zipkin can properly colorize. You could see in the list of traces one
trace that was in red color. That&#8217;s because there was an exception thrown.</simpara>
<simpara>If you click that trace then you&#8217;ll see a similar picture</simpara>
<informalfigure>
<mediaobject>
<imageobject>
<imagedata fileref="https://raw.githubusercontent.com/spring-cloud/spring-cloud-sleuth/master/docs/src/main/asciidoc/images/zipkin-error-traces.png"/>
</imageobject>
<textobject><phrase>Error Traces</phrase></textobject>
</mediaobject>
</informalfigure>
<simpara>Then if you click on one of the spans you&#8217;ll see the following</simpara>
<informalfigure>
<mediaobject>
<imageobject>
<imagedata fileref="https://raw.githubusercontent.com/spring-cloud/spring-cloud-sleuth/master/docs/src/main/asciidoc/images/zipkin-error-trace-screenshot.png"/>
</imageobject>
<textobject><phrase>Error Traces Info propagation</phrase></textobject>
</mediaobject>
</informalfigure>
<simpara>As you can see you can easily see the reason for an error and the whole stacktrace related to it.</simpara>
</section>
<section xml:id="_distributed_tracing_with_brave">
<title>Distributed tracing with Brave</title>
<simpara>Starting with version <literal>2.0.0</literal>, Spring Cloud Sleuth uses
<link xl:href="https://github.com/openzipkin/brave">Brave</link> as the tracing library. That means
that Sleuth no longer takes care of storing the context but it delegates
that work to Brave.</simpara>
<simpara>Due to the fact that Sleuth had different naming / tagging
conventions than Brave, we&#8217;ve decided to follow the Brave&#8217;s
conventions from now on. However, if you want to use the legacy
Sleuth approaches, it&#8217;s enough to set the <literal>spring.sleuth.http.legacy.enabled</literal> property
to <literal>true</literal>.</simpara>
</section>
<section xml:id="_live_examples">
<title>Live examples</title>
<figure>
<title>Click Pivotal Web Services icon to see it live!</title>
<mediaobject>
<imageobject>
<imagedata fileref="https://raw.githubusercontent.com/spring-cloud/spring-cloud-sleuth/master/docs/src/main/asciidoc/images/pws.png" contentwidth="150" contentdepth="74"/>
</imageobject>
<textobject><phrase>Zipkin deployed on Pivotal Web Services</phrase></textobject>
</mediaobject>
</figure>
<simpara><link xl:href="http://docssleuth-zipkin-server.cfapps.io/">Click here to see it live!</link></simpara>
<simpara>The dependency graph in Zipkin would look like this:</simpara>
<informalfigure>
<mediaobject>
<imageobject>
<imagedata fileref="https://raw.githubusercontent.com/spring-cloud/spring-cloud-sleuth/master/docs/src/main/asciidoc/images/dependencies.png"/>
</imageobject>
<textobject><phrase>Dependencies</phrase></textobject>
</mediaobject>
</informalfigure>
<figure>
<title>Click Pivotal Web Services icon to see it live!</title>
<mediaobject>
<imageobject>
<imagedata fileref="https://raw.githubusercontent.com/spring-cloud/spring-cloud-sleuth/master/docs/src/main/asciidoc/images/pws.png" contentwidth="150" contentdepth="74"/>
</imageobject>
<textobject><phrase>Zipkin deployed on Pivotal Web Services</phrase></textobject>
</mediaobject>
</figure>
<simpara><link xl:href="http://docssleuth-zipkin-server.cfapps.io/dependency">Click here to see it live!</link></simpara>
</section>
<section xml:id="_log_correlation">
<title>Log correlation</title>
<simpara>When grepping the logs of those four applications by trace id equal to e.g. <literal>2485ec27856c56f4</literal> one would get the following:</simpara>
<screen>service1.log:2016-02-26 11:15:47.561 INFO [service1,2485ec27856c56f4,2485ec27856c56f4,true] 68058 --- [nio-8081-exec-1] i.s.c.sleuth.docs.service1.Application : Hello from service1. Calling service2
service2.log:2016-02-26 11:15:47.710 INFO [service2,2485ec27856c56f4,9aa10ee6fbde75fa,true] 68059 --- [nio-8082-exec-1] i.s.c.sleuth.docs.service2.Application : Hello from service2. Calling service3 and then service4
service3.log:2016-02-26 11:15:47.895 INFO [service3,2485ec27856c56f4,1210be13194bfe5,true] 68060 --- [nio-8083-exec-1] i.s.c.sleuth.docs.service3.Application : Hello from service3
service2.log:2016-02-26 11:15:47.924 INFO [service2,2485ec27856c56f4,9aa10ee6fbde75fa,true] 68059 --- [nio-8082-exec-1] i.s.c.sleuth.docs.service2.Application : Got response from service3 [Hello from service3]
service4.log:2016-02-26 11:15:48.134 INFO [service4,2485ec27856c56f4,1b1845262ffba49d,true] 68061 --- [nio-8084-exec-1] i.s.c.sleuth.docs.service4.Application : Hello from service4
service2.log:2016-02-26 11:15:48.156 INFO [service2,2485ec27856c56f4,9aa10ee6fbde75fa,true] 68059 --- [nio-8082-exec-1] i.s.c.sleuth.docs.service2.Application : Got response from service4 [Hello from service4]
service1.log:2016-02-26 11:15:48.182 INFO [service1,2485ec27856c56f4,2485ec27856c56f4,true] 68058 --- [nio-8081-exec-1] i.s.c.sleuth.docs.service1.Application : Got response from service2 [Hello from service2, response from service3 [Hello from service3] and from service4 [Hello from service4]]</screen>
<simpara>If you&#8217;re using a log aggregating tool like <link xl:href="https://www.elastic.co/products/kibana">Kibana</link>,
<link xl:href="http://www.splunk.com/">Splunk</link> etc. you can order the events that took place. An example of
Kibana would look like this:</simpara>
<informalfigure>
<mediaobject>
<imageobject>
<imagedata fileref="https://raw.githubusercontent.com/spring-cloud/spring-cloud-sleuth/master/docs/src/main/asciidoc/images/kibana.png"/>
</imageobject>
<textobject><phrase>Log correlation with Kibana</phrase></textobject>
</mediaobject>
</informalfigure>
<simpara>If you want to use <link xl:href="https://www.elastic.co/guide/en/logstash/current/index.html">Logstash</link> here is the Grok pattern for Logstash:</simpara>
<screen>filter {
# pattern matching logback pattern
grok {
match =&gt; { "message" =&gt; "%{TIMESTAMP_ISO8601:timestamp}\s+%{LOGLEVEL:severity}\s+\[%{DATA:service},%{DATA:trace},%{DATA:span},%{DATA:exportable}\]\s+%{DATA:pid}\s+---\s+\[%{DATA:thread}\]\s+%{DATA:class}\s+:\s+%{GREEDYDATA:rest}" }
}
}</screen>
<note>
<simpara>If you want to use Grok together with the logs from Cloud Foundry you have to use this pattern:</simpara>
</note>
<screen>filter {
# pattern matching logback pattern
grok {
match =&gt; { "message" =&gt; "(?m)OUT\s+%{TIMESTAMP_ISO8601:timestamp}\s+%{LOGLEVEL:severity}\s+\[%{DATA:service},%{DATA:trace},%{DATA:span},%{DATA:exportable}\]\s+%{DATA:pid}\s+---\s+\[%{DATA:thread}\]\s+%{DATA:class}\s+:\s+%{GREEDYDATA:rest}" }
}
}</screen>
<section xml:id="_json_logback_with_logstash">
<title>JSON Logback with Logstash</title>
<simpara>Often you do not want to store your logs in a text file but in a JSON file that Logstash can immediately pick. To do that you have to do the following (for readability
we&#8217;re passing the dependencies in the <literal>groupId:artifactId:version</literal> notation.</simpara>
<simpara><emphasis role="strong">Dependencies setup</emphasis></simpara>
<itemizedlist>
<listitem>
<simpara>Ensure that Logback is on the classpath (<literal>ch.qos.logback:logback-core</literal>)</simpara>
</listitem>
<listitem>
<simpara>Add Logstash Logback encode - example for version <literal>4.6</literal> : <literal>net.logstash.logback:logstash-logback-encoder:4.6</literal></simpara>
</listitem>
</itemizedlist>
<simpara><emphasis role="strong">Logback setup</emphasis></simpara>
<simpara>Below you can find an example of a Logback configuration (file named <link xl:href="https://github.com/spring-cloud-samples/sleuth-documentation-apps/blob/master/service1/src/main/resources/logback-spring.xml">logback-spring.xml</link>) that:</simpara>
<itemizedlist>
<listitem>
<simpara>logs information from the application in a JSON format to a <literal>build/${spring.application.name}.json</literal> file</simpara>
</listitem>
<listitem>
<simpara>has commented out two additional appenders - console and standard log file</simpara>
</listitem>
<listitem>
<simpara>has the same logging pattern as the one presented in the previous section</simpara>
</listitem>
</itemizedlist>
<programlisting language="xml" linenumbering="unnumbered">&lt;?xml version="1.0" encoding="UTF-8"?&gt;
&lt;configuration&gt;
&lt;include resource="org/springframework/boot/logging/logback/defaults.xml"/&gt;
&lt;springProperty scope="context" name="springAppName" source="spring.application.name"/&gt;
&lt;!-- Example for logging into the build folder of your project --&gt;
&lt;property name="LOG_FILE" value="${BUILD_FOLDER:-build}/${springAppName}"/&gt;
&lt;!-- You can override this to have a custom pattern --&gt;
&lt;property name="CONSOLE_LOG_PATTERN"
value="%clr(%d{yyyy-MM-dd HH:mm:ss.SSS}){faint} %clr(${LOG_LEVEL_PATTERN:-%5p}) %clr(${PID:- }){magenta} %clr(---){faint} %clr([%15.15t]){faint} %clr(%-40.40logger{39}){cyan} %clr(:){faint} %m%n${LOG_EXCEPTION_CONVERSION_WORD:-%wEx}"/&gt;
&lt;!-- Appender to log to console --&gt;
&lt;appender name="console" class="ch.qos.logback.core.ConsoleAppender"&gt;
&lt;filter class="ch.qos.logback.classic.filter.ThresholdFilter"&gt;
&lt;!-- Minimum logging level to be presented in the console logs--&gt;
&lt;level&gt;DEBUG&lt;/level&gt;
&lt;/filter&gt;
&lt;encoder&gt;
&lt;pattern&gt;${CONSOLE_LOG_PATTERN}&lt;/pattern&gt;
&lt;charset&gt;utf8&lt;/charset&gt;
&lt;/encoder&gt;
&lt;/appender&gt;
&lt;!-- Appender to log to file --&gt;
&lt;appender name="flatfile" class="ch.qos.logback.core.rolling.RollingFileAppender"&gt;
&lt;file&gt;${LOG_FILE}&lt;/file&gt;
&lt;rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"&gt;
&lt;fileNamePattern&gt;${LOG_FILE}.%d{yyyy-MM-dd}.gz&lt;/fileNamePattern&gt;
&lt;maxHistory&gt;7&lt;/maxHistory&gt;
&lt;/rollingPolicy&gt;
&lt;encoder&gt;
&lt;pattern&gt;${CONSOLE_LOG_PATTERN}&lt;/pattern&gt;
&lt;charset&gt;utf8&lt;/charset&gt;
&lt;/encoder&gt;
&lt;/appender&gt;
&lt;!-- Appender to log to file in a JSON format --&gt;
&lt;appender name="logstash" class="ch.qos.logback.core.rolling.RollingFileAppender"&gt;
&lt;file&gt;${LOG_FILE}.json&lt;/file&gt;
&lt;rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"&gt;
&lt;fileNamePattern&gt;${LOG_FILE}.json.%d{yyyy-MM-dd}.gz&lt;/fileNamePattern&gt;
&lt;maxHistory&gt;7&lt;/maxHistory&gt;
&lt;/rollingPolicy&gt;
&lt;encoder class="net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder"&gt;
&lt;providers&gt;
&lt;timestamp&gt;
&lt;timeZone&gt;UTC&lt;/timeZone&gt;
&lt;/timestamp&gt;
&lt;pattern&gt;
&lt;pattern&gt;
{
"severity": "%level",
"service": "${springAppName:-}",
"trace": "%X{X-B3-TraceId:-}",
"span": "%X{X-B3-SpanId:-}",
"parent": "%X{X-B3-ParentSpanId:-}",
"exportable": "%X{X-Span-Export:-}",
"pid": "${PID:-}",
"thread": "%thread",
"class": "%logger{40}",
"rest": "%message"
}
&lt;/pattern&gt;
&lt;/pattern&gt;
&lt;/providers&gt;
&lt;/encoder&gt;
&lt;/appender&gt;
&lt;root level="INFO"&gt;
&lt;appender-ref ref="console"/&gt;
&lt;!-- uncomment this to have also JSON logs --&gt;
&lt;!--&lt;appender-ref ref="logstash"/&gt;--&gt;
&lt;!--&lt;appender-ref ref="flatfile"/&gt;--&gt;
&lt;/root&gt;
&lt;/configuration&gt;</programlisting>
<note>
<simpara>If you&#8217;re using a custom <literal>logback-spring.xml</literal> then you have to pass the <literal>spring.application.name</literal> in
<literal>bootstrap</literal> instead of <literal>application</literal> property file. Otherwise your custom logback file won&#8217;t read the property properly.</simpara>
</note>
</section>
</section>
<section xml:id="_propagating_span_context">
<title>Propagating Span Context</title>
<simpara>The span context is the state that must get propagated to any child Spans across process boundaries.
Part of the Span Context is the Baggage. The trace and span IDs are a required part of the span context.
Baggage is an optional part.</simpara>
<simpara>Baggage is a set of key:value pairs stored in the span context. Baggage travels together with the trace
and is attached to every span. Spring Cloud Sleuth will understand that a header is baggage related if the HTTP
header is prefixed with <literal>baggage-</literal> and for messaging it starts with <literal>baggage_</literal>.</simpara>
<important>
<simpara>There&#8217;s currently no limitation of the count or size of baggage items. However, keep in mind that
too many can decrease system throughput or increase RPC latency. In extreme cases, it could crash the app due
to exceeding transport-level message or header capacity.</simpara>
</important>
<simpara>Example of setting baggage on a span:</simpara>
<programlisting language="java" linenumbering="unnumbered">Unresolved directive in intro.adoc - include::https://raw.githubusercontent.com/spring-cloud/spring-cloud-sleuth/master/spring-cloud-sleuth-core/src/test/java/org/springframework/cloud/brave/instrument/web/multiple/MultipleHopsIntegrationTests.java[tags=baggage,indent=0]
}</programlisting>
<section xml:id="_baggage_vs_span_tags">
<title>Baggage vs. Span Tags</title>
<simpara>Baggage travels with the trace (i.e. every child span contains the baggage of its parent). Zipkin has no knowledge of
baggage and will not even receive that information.</simpara>
<simpara>Tags are attached to a specific span - they are presented for that particular span only. However you
can search by tag to find the trace, where there exists a span having the searched tag value.</simpara>
<simpara>If you want to be able to lookup a span based on baggage, you should add corresponding entry as a tag in the root span.</simpara>
<important>
<simpara>Remember that the span needs to be in scope!</simpara>
</important>
<programlisting language="java" linenumbering="unnumbered">Unresolved directive in intro.adoc - include::https://raw.githubusercontent.com/spring-cloud/spring-cloud-sleuth/master/spring-cloud-sleuth-core/src/test/java/org/springframework/cloud/brave/instrument/web/multiple/MultipleHopsIntegrationTests.java[tags=baggage_tag,indent=0]</programlisting>
</section>
</section>
</section>
<section xml:id="_adding_to_the_project">
<title>Adding to the project</title>
<important>
<simpara>To ensure that your application name is properly displayed in Zipkin
set the <literal>spring.application.name</literal> property in <literal>bootstrap.yml</literal>.</simpara>
</important>
<section xml:id="_only_sleuth_log_correlation">
<title>Only Sleuth (log correlation)</title>
<simpara>If you want to profit only from Spring Cloud Sleuth without the Zipkin integration just add
the <literal>spring-cloud-starter-sleuth</literal> module to your project.</simpara>
<formalpara role="primary">
<title>Maven</title>
<para>
<programlisting language="xml" linenumbering="unnumbered">&lt;dependencyManagement&gt; <co xml:id="CO1-1"/>
&lt;dependencies&gt;
&lt;dependency&gt;
&lt;groupId&gt;org.springframework.cloud&lt;/groupId&gt;
&lt;artifactId&gt;spring-cloud-dependencies&lt;/artifactId&gt;
&lt;version&gt;${release.train.version}&lt;/version&gt;
&lt;type&gt;pom&lt;/type&gt;
&lt;scope&gt;import&lt;/scope&gt;
&lt;/dependency&gt;
&lt;/dependencies&gt;
&lt;/dependencyManagement&gt;
&lt;dependency&gt; <co xml:id="CO1-2"/>
&lt;groupId&gt;org.springframework.cloud&lt;/groupId&gt;
&lt;artifactId&gt;spring-cloud-starter-sleuth&lt;/artifactId&gt;
&lt;/dependency&gt;</programlisting>
</para>
</formalpara>
<calloutlist>
<callout arearefs="CO1-1">
<para>In order not to pick versions by yourself it&#8217;s much better if you add the dependency management via
the Spring BOM</para>
</callout>
<callout arearefs="CO1-2">
<para>Add the dependency to <literal>spring-cloud-starter-sleuth</literal></para>
</callout>
</calloutlist>
<formalpara role="secondary">
<title>Gradle</title>
<para>
<programlisting language="groovy" linenumbering="unnumbered">dependencyManagement { <co xml:id="CO2-1"/>
imports {
mavenBom "org.springframework.cloud:spring-cloud-dependencies:${releaseTrainVersion}"
}
}
dependencies { <co xml:id="CO2-2"/>
compile "org.springframework.cloud:spring-cloud-starter-sleuth"
}</programlisting>
</para>
</formalpara>
<calloutlist>
<callout arearefs="CO2-1">
<para>In order not to pick versions by yourself it&#8217;s much better if you add the dependency management via
the Spring BOM</para>
</callout>
<callout arearefs="CO2-2">
<para>Add the dependency to <literal>spring-cloud-starter-sleuth</literal></para>
</callout>
</calloutlist>
</section>
<section xml:id="_sleuth_with_zipkin_via_http">
<title>Sleuth with Zipkin via HTTP</title>
<simpara>If you want both Sleuth and Zipkin just add the <literal>spring-cloud-starter-zipkin</literal> dependency.</simpara>
<formalpara role="primary">
<title>Maven</title>
<para>
<programlisting language="xml" linenumbering="unnumbered">&lt;dependencyManagement&gt; <co xml:id="CO3-1"/>
&lt;dependencies&gt;
&lt;dependency&gt;
&lt;groupId&gt;org.springframework.cloud&lt;/groupId&gt;
&lt;artifactId&gt;spring-cloud-dependencies&lt;/artifactId&gt;
&lt;version&gt;${release.train.version}&lt;/version&gt;
&lt;type&gt;pom&lt;/type&gt;
&lt;scope&gt;import&lt;/scope&gt;
&lt;/dependency&gt;
&lt;/dependencies&gt;
&lt;/dependencyManagement&gt;
&lt;dependency&gt; <co xml:id="CO3-2"/>
&lt;groupId&gt;org.springframework.cloud&lt;/groupId&gt;
&lt;artifactId&gt;spring-cloud-starter-zipkin&lt;/artifactId&gt;
&lt;/dependency&gt;</programlisting>
</para>
</formalpara>
<calloutlist>
<callout arearefs="CO3-1">
<para>In order not to pick versions by yourself it&#8217;s much better if you add the dependency management via
the Spring BOM</para>
</callout>
<callout arearefs="CO3-2">
<para>Add the dependency to <literal>spring-cloud-starter-zipkin</literal></para>
</callout>
</calloutlist>
<formalpara role="secondary">
<title>Gradle</title>
<para>
<programlisting language="groovy" linenumbering="unnumbered">dependencyManagement { <co xml:id="CO4-1"/>
imports {
mavenBom "org.springframework.cloud:spring-cloud-dependencies:${releaseTrainVersion}"
}
}
dependencies { <co xml:id="CO4-2"/>
compile "org.springframework.cloud:spring-cloud-starter-zipkin"
}</programlisting>
</para>
</formalpara>
<calloutlist>
<callout arearefs="CO4-1">
<para>In order not to pick versions by yourself it&#8217;s much better if you add the dependency management via
the Spring BOM</para>
</callout>
<callout arearefs="CO4-2">
<para>Add the dependency to <literal>spring-cloud-starter-zipkin</literal></para>
</callout>
</calloutlist>
</section>
<section xml:id="_sleuth_with_zipkin_via_rabbitmq_or_kafka">
<title>Sleuth with Zipkin via RabbitMQ or Kafka</title>
<simpara>If you want to use RabbitMQ or Kafka instead of http, add the <literal>spring-rabbit</literal> or <literal>spring-kafka</literal>
dependencies. The default destination name is <literal>zipkin</literal>.</simpara>
<simpara><emphasis>Note: <literal>spring-cloud-sleuth-stream</literal> is deprecated and incompatible with these destinations</emphasis></simpara>
<simpara>If you want Sleuth over RabbitMQ add the <literal>spring-cloud-starter-zipkin</literal> and <literal>spring-rabbit</literal>
dependencies.</simpara>
<formalpara role="primary">
<title>Maven</title>
<para>
<programlisting language="xml" linenumbering="unnumbered">&lt;dependencyManagement&gt; <co xml:id="CO5-1"/>
&lt;dependencies&gt;
&lt;dependency&gt;
&lt;groupId&gt;org.springframework.cloud&lt;/groupId&gt;
&lt;artifactId&gt;spring-cloud-dependencies&lt;/artifactId&gt;
&lt;version&gt;${release.train.version}&lt;/version&gt;
&lt;type&gt;pom&lt;/type&gt;
&lt;scope&gt;import&lt;/scope&gt;
&lt;/dependency&gt;
&lt;/dependencies&gt;
&lt;/dependencyManagement&gt;
&lt;dependency&gt; <co xml:id="CO5-2"/>
&lt;groupId&gt;org.springframework.cloud&lt;/groupId&gt;
&lt;artifactId&gt;spring-cloud-starter-zipkin&lt;/artifactId&gt;
&lt;/dependency&gt;
&lt;dependency&gt; <co xml:id="CO5-3"/>
&lt;groupId&gt;org.springframework.amqp&lt;/groupId&gt;
&lt;artifactId&gt;spring-rabbit&lt;/artifactId&gt;
&lt;/dependency&gt;</programlisting>
</para>
</formalpara>
<calloutlist>
<callout arearefs="CO5-1">
<para>In order not to pick versions by yourself it&#8217;s much better if you add the dependency management via
the Spring BOM</para>
</callout>
<callout arearefs="CO5-2">
<para>Add the dependency to <literal>spring-cloud-starter-zipkin</literal> - that way all dependent dependencies will be downloaded</para>
</callout>
<callout arearefs="CO5-3">
<para>To automatically configure rabbit, simply add the spring-rabbit dependency</para>
</callout>
</calloutlist>
<formalpara role="secondary">
<title>Gradle</title>
<para>
<programlisting language="groovy" linenumbering="unnumbered">dependencyManagement { <co xml:id="CO6-1"/>
imports {
mavenBom "org.springframework.cloud:spring-cloud-dependencies:${releaseTrainVersion}"
}
}
dependencies {
compile "org.springframework.cloud:spring-cloud-starter-zipkin" <co xml:id="CO6-2"/>
compile "org.springframework.amqp:spring-rabbit" <co xml:id="CO6-3"/>
}</programlisting>
</para>
</formalpara>
<calloutlist>
<callout arearefs="CO6-1">
<para>In order not to pick versions by yourself it&#8217;s much better if you add the dependency management via
the Spring BOM</para>
</callout>
<callout arearefs="CO6-2">
<para>Add the dependency to <literal>spring-cloud-starter-zipkin</literal> - that way all dependent dependencies will be downloaded</para>
</callout>
<callout arearefs="CO6-3">
<para>To automatically configure rabbit, simply add the spring-rabbit dependency</para>
</callout>
</calloutlist>
</section>
</section>
</chapter>
<chapter xml:id="_additional_resources">
<title>Additional resources</title>
<simpara><emphasis role="strong">Marcin Grzejszczak talking about Spring Cloud Sleuth and Zipkin</emphasis></simpara>
<simpara><link xl:href="https://www.youtube.com/watch?v=eQV71Mw1u1c">click here to see the video</link></simpara>
</chapter>
<chapter xml:id="_features">
<title>Features</title>
<itemizedlist>
<listitem>
<simpara>Adds trace and span ids to the Slf4J MDC, so you can extract all the logs from a given trace or span in a log aggregator. Example logs:</simpara>
<screen>2016-02-02 15:30:57.902 INFO [bar,6bfd228dc00d216b,6bfd228dc00d216b,false] 23030 --- [nio-8081-exec-3] ...
2016-02-02 15:30:58.372 ERROR [bar,6bfd228dc00d216b,6bfd228dc00d216b,false] 23030 --- [nio-8081-exec-3] ...
2016-02-02 15:31:01.936 INFO [bar,46ab0d418373cbc9,46ab0d418373cbc9,false] 23030 --- [nio-8081-exec-4] ...</screen>
<simpara>notice the <literal>[appname,traceId,spanId,exportable]</literal> entries from the MDC:</simpara>
<itemizedlist>
<listitem>
<simpara><emphasis role="strong">spanId</emphasis> - the id of a specific operation that took place</simpara>
</listitem>
<listitem>
<simpara><emphasis role="strong">appname</emphasis> - the name of the application that logged the span</simpara>
</listitem>
<listitem>
<simpara><emphasis role="strong">traceId</emphasis> - the id of the latency graph that contains the span</simpara>
</listitem>
<listitem>
<simpara><emphasis role="strong">exportable</emphasis> - whether the log should be exported to Zipkin or not. When would you like the span not to be
exportable? In the case in which you want to wrap some operation in a Span and have it written to the logs
only.</simpara>
</listitem>
</itemizedlist>
</listitem>
<listitem>
<simpara>Provides an abstraction over common distributed tracing data models: traces, spans (forming a DAG), annotations,
key-value annotations. Loosely based on HTrace, but Zipkin (Dapper) compatible.</simpara>
</listitem>
<listitem>
<simpara>Sleuth records timing information to aid in latency analysis. Using sleuth, you can pinpoint causes of
latency in your applications. Sleuth is written to not log too much, and to not cause your production application to crash.</simpara>
<itemizedlist>
<listitem>
<simpara>propagates structural data about your call-graph in-band, and the rest out-of-band.</simpara>
</listitem>
<listitem>
<simpara>includes opinionated instrumentation of layers such as HTTP</simpara>
</listitem>
<listitem>
<simpara>includes sampling policy to manage volume</simpara>
</listitem>
<listitem>
<simpara>can report to a Zipkin system for query and visualization</simpara>
</listitem>
</itemizedlist>
</listitem>
<listitem>
<simpara>Instruments common ingress and egress points from Spring applications (servlet filter, async endpoints,
rest template, scheduled actions, message channels, zuul filters, feign client).</simpara>
</listitem>
<listitem>
<simpara>Sleuth includes default logic to join a trace across http or messaging boundaries. For example, http propagation
works via Zipkin-compatible request headers. This propagation logic is defined and customized via
<literal>SpanInjector</literal> and <literal>SpanExtractor</literal> implementations.</simpara>
</listitem>
<listitem>
<simpara>Sleuth gives you the possibility to propagate context (also known as baggage) between processes. That means that if you set on a Span
a baggage element then it will be sent downstream either via HTTP or messaging to other processes.</simpara>
</listitem>
<listitem>
<simpara>Provides a way to create / continue spans and add tags and logs via annotations.</simpara>
</listitem>
<listitem>
<simpara>If <literal>spring-cloud-sleuth-zipkin</literal> then the app will generate and collect Zipkin-compatible traces.
By default it sends them via HTTP to a Zipkin server on localhost (port 9411).
Configure the location of the service using <literal>spring.zipkin.baseUrl</literal>.</simpara>
<itemizedlist>
<listitem>
<simpara>If you depend on <literal>spring-rabbit</literal> or <literal>spring-kafka</literal> your app will send traces to a broker instead of http.</simpara>
</listitem>
<listitem>
<simpara>Note: <literal>spring-cloud-sleuth-stream</literal> is deprecated and should no longer be used.</simpara>
</listitem>
</itemizedlist>
</listitem>
<listitem>
<simpara>Spring Cloud Sleuth is <link xl:href="http://opentracing.io/">OpenTracing</link> compatible</simpara>
</listitem>
</itemizedlist>
<important>
<simpara>If using Zipkin, configure the percentage of spans exported using <literal>spring.sleuth.sampler.percentage</literal>
(default 0.1, i.e. 10%). <emphasis role="strong">Otherwise you might think that Sleuth is not working cause it&#8217;s omitting some spans.</emphasis></simpara>
</important>
<note>
<simpara>the SLF4J MDC is always set and logback users will immediately see the trace and span ids in logs per the example
above. Other logging systems have to configure their own formatter to get the same result. The default is
<literal>logging.pattern.level</literal> set to <literal>%5p [${spring.zipkin.service.name:${spring.application.name:-}},%X{X-B3-TraceId:-},%X{X-B3-SpanId:-},%X{X-Span-Export:-}]</literal>
(this is a Spring Boot feature for logback users).
<emphasis role="strong">This means that if you&#8217;re not using SLF4J this pattern WILL NOT be automatically applied</emphasis>.</simpara>
</note>
<section xml:id="_introduction_to_brave">
<title>Introduction to Brave</title>
<important>
<simpara>Starting with version <literal>2.0.0</literal> Spring Cloud Sleuth uses
<link xl:href="https://github.com/openzipkin/brave">Brave</link> as the tracing library.
For your convenience we&#8217;re embedding part of the Brave&#8217;s docs here.</simpara>
</important>
<simpara>Brave is a library used to capture and report latency information about
distributed operations to Zipkin. Most users won&#8217;t use Brave directly,
rather libraries or frameworks than employ Brave on their behalf.</simpara>
<simpara>This module includes tracer creates and joins spans that model the
latency of potentially distributed work. It also includes libraries to
propagate the trace context over network boundaries, for example, via
http headers.</simpara>
<section xml:id="_tracing">
<title>Tracing</title>
<simpara>Most importantly, you need a <literal>brave.Tracer</literal>, configured to [report to Zipkin]
(<link xl:href="https://github.com/openzipkin/zipkin-reporter-java">https://github.com/openzipkin/zipkin-reporter-java</link>).</simpara>
<simpara>Here&#8217;s an example setup that sends trace data (spans) to Zipkin over
http (as opposed to Kafka).</simpara>
<programlisting language="java" linenumbering="unnumbered">class MyClass {
private final Tracer tracer;
// Tracer will be autowired
MyClass(Tracer tracer) {
this.tracer = tracer;
}
void doSth() {
Span span = tracer.newTrace().name("encode").start();
// ...
}
}</programlisting>
<important>
<simpara>If your span contains a name greater than 50 chars, then that name will
be truncated to 50 chars. Your names have to be explicit and concrete. Big names lead to
latency issues and sometimes even thrown exceptions.</simpara>
</important>
</section>
<section xml:id="_tracing_2">
<title>Tracing</title>
<simpara>The tracer creates and joins spans that model the latency of potentially
distributed work. It can employ sampling to reduce overhead in process
or to reduce the amount of data sent to Zipkin.</simpara>
<simpara>Spans returned by a tracer report data to Zipkin when finished, or do
nothing if unsampled. After starting a span, you can annotate events of
interest or add tags containing details or lookup keys.</simpara>
<simpara>Spans have a context which includes trace identifiers that place it at
the correct spot in the tree representing the distributed operation.</simpara>
</section>
<section xml:id="_local_tracing">
<title>Local Tracing</title>
<simpara>When tracing local code, just run it inside a span.</simpara>
<programlisting language="java" linenumbering="unnumbered">Span span = tracer.newTrace().name("encode").start();
try {
doSomethingExpensive();
} finally {
span.finish();
}</programlisting>
<simpara>In the above example, the span is the root of the trace. In many cases,
you will be a part of an existing trace. When this is the case, call
<literal>newChild</literal> instead of <literal>newTrace</literal></simpara>
<programlisting language="java" linenumbering="unnumbered">Span span = tracer.newChild(root.context()).name("encode").start();
try {
doSomethingExpensive();
} finally {
span.finish();
}</programlisting>
</section>
<section xml:id="_customizing_spans">
<title>Customizing spans</title>
<simpara>Once you have a span, you can add tags to it, which can be used as lookup
keys or details. For example, you might add a tag with your runtime
version.</simpara>
<programlisting language="java" linenumbering="unnumbered">span.tag("clnt/finagle.version", "6.36.0");</programlisting>
<simpara>When exposing the ability to customize spans to third parties, prefer
<literal>brave.SpanCustomizer</literal> as opposed to <literal>brave.Span</literal>. The former is simpler to
understand and test, and doesn&#8217;t tempt users with span lifecycle hooks.</simpara>
<programlisting language="java" linenumbering="unnumbered">interface MyTraceCallback {
void request(Request request, SpanCustomizer customizer);
}</programlisting>
<simpara>Since <literal>brave.Span</literal> implements <literal>brave.SpanCustomizer</literal>, it is just as easy for you
to pass to users.</simpara>
<simpara>Ex.</simpara>
<programlisting language="java" linenumbering="unnumbered">for (MyTraceCallback callback : userCallbacks) {
callback.request(request, span);
}</programlisting>
</section>
<section xml:id="_implicitly_looking_up_the_current_span">
<title>Implicitly looking up the current span</title>
<simpara>Sometimes you won&#8217;t know if a trace is in progress or not, and you don&#8217;t
want users to do null checks. <literal>brave.CurrentSpanCustomizer</literal> adds to any
span that&#8217;s in progress or drops data accordingly.</simpara>
<simpara>Ex.</simpara>
<programlisting language="java" linenumbering="unnumbered">// user code can then inject this without a chance of it being null.
@Autowire SpanCustomizer span;
void userCode() {
span.annotate("tx.started");
...
}</programlisting>
</section>
<section xml:id="_rpc_tracing">
<title>RPC tracing</title>
<simpara>Check for <link xl:href="https://github.com/openzipkin/sleuth/tree/master/instrumentation">instrumentation written here</link>
and <link xl:href="http://zipkin.io/pages/existing_instrumentations.html">Zipkin&#8217;s list</link>
before rolling your own RPC instrumentation!</simpara>
<simpara>RPC tracing is often done automatically by interceptors. Under the scenes,
they add tags and events that relate to their role in an RPC operation.</simpara>
<simpara>Here&#8217;s an example of a client span:</simpara>
<programlisting language="java" linenumbering="unnumbered">// before you send a request, add metadata that describes the operation
span = tracer.newTrace().name("get").type(CLIENT);
span.tag("clnt/finagle.version", "6.36.0");
span.tag(TraceKeys.HTTP_PATH, "/api");
span.remoteEndpoint(Endpoint.builder()
.serviceName("backend")
.ipv4(127 &lt;&lt; 24 | 1)
.port(8080).build());
// when the request is scheduled, start the span
span.start();
// if you have callbacks for when data is on the wire, note those events
span.annotate(Constants.WIRE_SEND);
span.annotate(Constants.WIRE_RECV);
// when the response is complete, finish the span
span.finish();</programlisting>
<section xml:id="_one_way_tracing">
<title>One-Way tracing</title>
<simpara>Sometimes you need to model an asynchronous operation, where there is a
request, but no response. In normal RPC tracing, you use <literal>span.finish()</literal>
which indicates the response was received. In one-way tracing, you use
<literal>span.flush()</literal> instead, as you don&#8217;t expect a response.</simpara>
<simpara>Here&#8217;s how a client might model a one-way operation</simpara>
<programlisting language="java" linenumbering="unnumbered">// start a new span representing a client request
oneWaySend = tracer.newSpan(parent).kind(Span.Kind.CLIENT);
// Add the trace context to the request, so it can be propagated in-band
tracing.propagation().injector(Request::addHeader)
.inject(oneWaySend.context(), request);
// fire off the request asynchronously, totally dropping any response
request.execute();
// start the client side and flush instead of finish
oneWaySend.start().flush();</programlisting>
<simpara>And here&#8217;s how a server might handle this..</simpara>
<programlisting language="java" linenumbering="unnumbered">// pull the context out of the incoming request
extractor = tracing.propagation().extractor(Request::getHeader);
// convert that context to a span which you can name and add tags to
oneWayReceive = nextSpan(tracer, extractor.extract(request))
.name("process-request")
.kind(SERVER)
... add tags etc.
// start the server side and flush instead of finish
oneWayReceive.start().flush();
// you should not modify this span anymore as it is complete. However,
// you can create children to represent follow-up work.
next = tracer.newSpan(oneWayReceive.context()).name("step2").start();</programlisting>
<simpara><emphasis role="strong">Note</emphasis> The above propagation logic is a simplified version of our [http handlers](<link xl:href="https://github.com/openzipkin/sleuth/tree/master/instrumentation/http#http-server">https://github.com/openzipkin/sleuth/tree/master/instrumentation/http#http-server</link>).</simpara>
<simpara>There&#8217;s a working example of a one-way span [here](src/test/java/sleuth/features/async/OneWaySpanTest.java).</simpara>
</section>
</section>
</section>
</chapter>
<chapter xml:id="_sampling">
<title>Sampling</title>
<simpara>Sampling may be employed to reduce the data collected and reported out
of process. When a span isn&#8217;t sampled, it adds no overhead (noop).</simpara>
<simpara>Sampling is an up-front decision, meaning that the decision to report
data is made at the first operation in a trace, and that decision is
propagated downstream.</simpara>
<simpara>By default, there&#8217;s a global sampler that applies a single rate to all
traced operations. <literal>Tracer.Builder.sampler</literal> is how you indicate this,
and it defaults to trace every request.</simpara>
<section xml:id="_declarative_sampling">
<title>Declarative sampling</title>
<simpara>Some need to sample based on the type or annotations of a java method.</simpara>
<simpara>Most users will use a framework interceptor which automates this sort of
policy. Here&#8217;s how they might work internally.</simpara>
<programlisting language="java" linenumbering="unnumbered">// derives a sample rate from an annotation on a java method
DeclarativeSampler&lt;Traced&gt; sampler = DeclarativeSampler.create(Traced::sampleRate);
@Around("@annotation(traced)")
public Object traceThing(ProceedingJoinPoint pjp, Traced traced) throws Throwable {
Span span = tracing.tracer().newTrace(sampler.sample(traced))...
try {
return pjp.proceed();
} finally {
span.finish();
}
}</programlisting>
</section>
<section xml:id="_custom_sampling">
<title>Custom sampling</title>
<simpara>You may want to apply different policies depending on what the operation
is. For example, you might not want to trace requests to static resources
such as images, or you might want to trace all requests to a new api.</simpara>
<simpara>Most users will use a framework interceptor which automates this sort of
policy. Here&#8217;s how they might work internally.</simpara>
<programlisting language="java" linenumbering="unnumbered">Span newTrace(Request input) {
SamplingFlags flags = SamplingFlags.NONE;
if (input.url().startsWith("/experimental")) {
flags = SamplingFlags.SAMPLED;
} else if (input.url().startsWith("/static")) {
flags = SamplingFlags.NOT_SAMPLED;
}
return tracer.newTrace(flags);
}</programlisting>
<simpara>Note: the above is the basis for the built-in <link xl:href="https://github.com/openzipkin/sleuth/tree/master/instrumentation/http">http sampler</link></simpara>
</section>
<section xml:id="_sampling_in_spring_cloud_sleuth">
<title>Sampling in Spring Cloud Sleuth</title>
<simpara>Spring Cloud Sleuth by default sets all spans to non-exportable.
That means that you will see traces in logs, but not in any remote store.
For testing the default is often enough, and it probably is all you need
if you are only using the logs (e.g. with an ELK aggregator). If you are
exporting span data to Zipkin, there is also an <literal>Sampler.ALWAYS_SAMPLE</literal>
that exports everything and a <literal>ProbabilityBasedSampler</literal> that samples a
fixed fraction of spans.</simpara>
<note>
<simpara>The <literal>ProbabilityBasedSampler</literal> is the default if you are using
<literal>spring-cloud-sleuth-zipkin</literal>. You can
configure the exports using <literal>spring.sleuth.sampler.probability</literal>. The passed
value needs to be a double from <literal>0.0</literal> to <literal>1.0</literal>.</simpara>
</note>
<simpara>A sampler can be installed just by creating a bean definition, e.g:</simpara>
<programlisting language="java" linenumbering="unnumbered">@Bean
public Sampler defaultSampler() {
return Sampler.ALWAYS_SAMPLE;
}</programlisting>
<tip>
<simpara>You can set the HTTP header <literal>X-B3-Flags</literal> to <literal>1</literal> or when doing messaging you can
set <literal>spanFlags</literal> header to <literal>1</literal>. Then the current span will be forced to be exportable
regardless of the sampling decision.</simpara>
</tip>
</section>
</chapter>
<chapter xml:id="_propagation">
<title>Propagation</title>
<simpara>Propagation is needed to ensure activity originating from the same root
are collected together in the same trace. The most common propagation
approach is to copy a trace context from a client sending an RPC request
to a server receiving it.</simpara>
<simpara>For example, when an downstream Http call is made, its trace context is
sent along with it, encoded as request headers:</simpara>
<screen> Client Span Server Span
┌──────────────────┐ ┌──────────────────┐
│ │ │ │
│ TraceContext │ Http Request Headers │ TraceContext │
│ ┌──────────────┐ │ ┌───────────────────┐ │ ┌──────────────┐ │
│ │ TraceId │ │ │ X─B3─TraceId │ │ │ TraceId │ │
│ │ │ │ │ │ │ │ │ │
│ │ ParentSpanId │ │ Extract │ X─B3─ParentSpanId │ Inject │ │ ParentSpanId │ │
│ │ ├─┼─────────&gt;│ ├────────┼&gt;│ │ │
│ │ SpanId │ │ │ X─B3─SpanId │ │ │ SpanId │ │
│ │ │ │ │ │ │ │ │ │
│ │ Sampled │ │ │ X─B3─Sampled │ │ │ Sampled │ │
│ └──────────────┘ │ └───────────────────┘ │ └──────────────┘ │
│ │ │ │
└──────────────────┘ └──────────────────┘</screen>
<simpara>The names above are from <link xl:href="https://github.com/openzipkin/b3-propagation">B3 Propagation</link>,
which is built-in to Brave and has implementations in many languages and
frameworks.</simpara>
<simpara>Most users will use a framework interceptor which automates propagation.
Here&#8217;s how they might work internally.</simpara>
<simpara>Here&#8217;s what client-side propagation might look like</simpara>
<programlisting language="java" linenumbering="unnumbered">// configure a function that injects a trace context into a request
injector = tracing.propagation().injector(Request.Builder::addHeader);
// before a request is sent, add the current span's context to it
injector.inject(span.context(), request);</programlisting>
<simpara>Here&#8217;s what server-side propagation might look like</simpara>
<programlisting language="java" linenumbering="unnumbered">// configure a function that extracts the trace context from a request
extracted = tracing.propagation().extractor(Request::getHeader);
// when a server receives a request, it joins or starts a new trace
span = tracer.nextSpan(extracted, request);</programlisting>
<section xml:id="_propagating_extra_fields">
<title>Propagating extra fields</title>
<simpara>Sometimes you need to propagate extra fields, such as a request ID or an alternate trace context.
For example, if you are in a Cloud Foundry environment, you might want to pass the request ID:</simpara>
<programlisting language="java" linenumbering="unnumbered">// when you initialize the builder, define the extra field you want to propagate
tracingBuilder.propagationFactory(
ExtraFieldPropagation.newFactory(B3Propagation.FACTORY, "x-vcap-request-id")
);
// later, you can tag that request ID or use it in log correlation
requestId = ExtraFieldPropagation.get("x-vcap-request-id");</programlisting>
<simpara>You may also need to propagate a trace context you aren&#8217;t using. For example, you may be in an
Amazon Web Services environment, but not reporting data to X-Ray. To ensure X-Ray can co-exist
correctly, pass-through its tracing header like so.</simpara>
<programlisting language="java" linenumbering="unnumbered">tracingBuilder.propagationFactory(
ExtraFieldPropagation.newFactory(B3Propagation.FACTORY, "x-amzn-trace-id")
);</programlisting>
<section xml:id="_prefixed_fields">
<title>Prefixed fields</title>
<simpara>You can also prefix fields, if they follow a common pattern. For example, the following will
propagate the field "x-vcap-request-id" as-is, but send the fields "country-code" and "user-id"
on the wire as "x-baggage-country-code" and "x-baggage-user-id" respectively.</simpara>
<simpara>Setup your tracing instance with allowed fields:</simpara>
<programlisting language="java" linenumbering="unnumbered">tracingBuilder.propagationFactory(
ExtraFieldPropagation.newFactoryBuilder(B3Propagation.FACTORY)
.addField("x-vcap-request-id")
.addPrefixedFields("baggage-", Arrays.asList("country-code", "user-id"))
.build()
);</programlisting>
<simpara>Later, you can call below to affect the country code of the current trace context</simpara>
<programlisting language="java" linenumbering="unnumbered">ExtraFieldPropagation.set("country-code", "FO");
String countryCode = ExtraFieldPropagation.get("country-code");</programlisting>
<simpara>Or, if you have a reference to a trace context, use it explicitly</simpara>
<programlisting language="java" linenumbering="unnumbered">ExtraFieldPropagation.set(span.context(), "country-code", "FO");
String countryCode = ExtraFieldPropagation.get(span.context(), "country-code");</programlisting>
<important>
<simpara>In comparison to previous versions of Sleuth, with
Brave it&#8217;s required to pass the list of baggage keys.
There are two properties to achieve this. Via the <literal>spring.sleuth.baggage-keys</literal> you set keys
that will get prefixed with <literal>baggage-</literal> for http calls and <literal>baggage_</literal> for messaging. You can also pass
a list of prefixed keys that will be whitelisted without any prefix via
<literal>spring.sleuth.prefixed-keys</literal> property.</simpara>
</important>
</section>
<section xml:id="_extracting_a_propagated_context">
<title>Extracting a propagated context</title>
<simpara>The <literal>TraceContext.Extractor&lt;C&gt;</literal> reads trace identifiers and sampling status
from an incoming request or message. The carrier is usually a request object
or headers.</simpara>
<simpara>This utility is used in standard instrumentation like [HttpServerHandler](../instrumentation/http/src/main/java/sleuth/http/HttpServerHandler.java),
but can also be used for custom RPC or messaging code.</simpara>
<simpara><literal>TraceContextOrSamplingFlags</literal> is usually only used with <literal>Tracer.nextSpan(extracted)</literal>, unless you are
sharing span IDs between a client and a server.</simpara>
</section>
<section xml:id="_sharing_span_ids_between_client_and_server">
<title>Sharing span IDs between client and server</title>
<simpara>A normal instrumentation pattern is creating a span representing the server
side of an RPC. <literal>Extractor.extract</literal> might return a complete trace context when
applied to an incoming client request. <literal>Tracer.joinSpan</literal> attempts to continue
the this trace, using the same span ID if supported, or creating a child span
if not. When span ID is shared, data reported includes a flag saying so.</simpara>
<simpara>Here&#8217;s an example of B3 propagation:</simpara>
<screen> ┌───────────────────┐ ┌───────────────────┐
Incoming Headers │ TraceContext │ │ TraceContext │
┌───────────────────┐(extract)│ ┌───────────────┐ │(join)│ ┌───────────────┐ │
│ X─B3-TraceId │─────────┼─┼&gt; TraceId │ │──────┼─┼&gt; TraceId │ │
│ │ │ │ │ │ │ │ │ │
│ X─B3-ParentSpanId │─────────┼─┼&gt; ParentSpanId │ │──────┼─┼&gt; ParentSpanId │ │
│ │ │ │ │ │ │ │ │ │
│ X─B3-SpanId │─────────┼─┼&gt; SpanId │ │──────┼─┼&gt; SpanId │ │
└───────────────────┘ │ │ │ │ │ │ │ │
│ │ │ │ │ │ Shared: true │ │
│ └───────────────┘ │ │ └───────────────┘ │
└───────────────────┘ └───────────────────┘</screen>
<simpara>Some propagation systems only forward the parent span ID, detected when
<literal>Propagation.Factory.supportsJoin() == false</literal>. In this case, a new span ID is
always provisioned and the incoming context determines the parent ID.</simpara>
<simpara>Here&#8217;s an example of AWS propagation:</simpara>
<screen> ┌───────────────────┐ ┌───────────────────┐
x-amzn-trace-id │ TraceContext │ │ TraceContext │
┌───────────────────┐(extract)│ ┌───────────────┐ │(join)│ ┌───────────────┐ │
│ Root │─────────┼─┼&gt; TraceId │ │──────┼─┼&gt; TraceId │ │
│ │ │ │ │ │ │ │ │ │
│ Parent │─────────┼─┼&gt; SpanId │ │──────┼─┼&gt; ParentSpanId │ │
└───────────────────┘ │ └───────────────┘ │ │ │ │ │
└───────────────────┘ │ │ SpanId: New │ │
│ └───────────────┘ │
└───────────────────┘</screen>
<simpara>Note: Some span reporters do not support sharing span IDs. For example, if you
set <literal>Tracing.Builder.spanReporter(amazonXrayOrGoogleStackdrive)</literal>, disable join
via <literal>Tracing.Builder.supportsJoin(false)</literal>. This will force a new child span on
<literal>Tracer.joinSpan()</literal>.</simpara>
</section>
<section xml:id="_implementing_propagation">
<title>Implementing Propagation</title>
<simpara><literal>TraceContext.Extractor&lt;C&gt;</literal> is implemented by a <literal>Propagation.Factory</literal> plugin. Internally, this code
will create the union type <literal>TraceContextOrSamplingFlags</literal> with one of the following:
* <literal>TraceContext</literal> if trace and span IDs were present.
* <literal>TraceIdContext</literal> if a trace ID was present, but not span IDs.
* <literal>SamplingFlags</literal> if no identifiers were present</simpara>
<simpara>Some <literal>Propagation</literal> implementations carry extra data from point of extraction (ex reading incoming
headers) to injection (ex writing outgoing headers). For example, it might carry a request ID. When
implementations have extra data, here&#8217;s how they handle it.
* If a <literal>TraceContext</literal> was extracted, add the extra data as <literal>TraceContext.extra()</literal>
* Otherwise, add it as <literal>TraceContextOrSamplingFlags.extra()</literal>, which <literal>Tracer.nextSpan</literal> handles.</simpara>
</section>
</section>
</chapter>
<chapter xml:id="_current_tracing_component">
<title>Current Tracing Component</title>
<simpara>Brave supports a "current tracing component" concept which should only
be used when you have no other means to get a reference. This was made
for JDBC connections, as they often initialize prior to the tracing
component.</simpara>
<simpara>The most recent tracing component instantiated is available via
<literal>Tracing.current()</literal>. You there&#8217;s also a shortcut to get only the tracer
via <literal>Tracing.currentTracer()</literal>. If you use either of these methods, do
noot cache the result. Instead, look them up each time you need them.</simpara>
</chapter>
<chapter xml:id="_current_span">
<title>Current Span</title>
<simpara>Brave supports a "current span" concept which represents the in-flight
operation. <literal>Tracer.currentSpan()</literal> can be used to add custom tags to a
span and <literal>Tracer.nextSpan()</literal> can be used to create a child of whatever
is in-flight.</simpara>
<section xml:id="_setting_a_span_in_scope_manually">
<title>Setting a span in scope manually</title>
<simpara>When writing new instrumentation, it is important to place a span you
created in scope as the current span. Not only does this allow users to
access it with <literal>Tracer.currentSpan()</literal>, but it also allows customizations
like SLF4J MDC to see the current trace IDs.</simpara>
<simpara><literal>Tracer.withSpanInScope(Span)</literal> facilitates this and is most conveniently
employed via the try-with-resources idiom. Whenever external code might
be invoked (such as proceeding an interceptor or otherwise), place the
span in scope like this.</simpara>
<programlisting language="java" linenumbering="unnumbered">try (SpanInScope ws = tracer.withSpanInScope(span)) {
return inboundRequest.invoke();
} finally { // note the scope is independent of the span
span.finish();
}</programlisting>
<simpara>In edge cases, you may need to clear the current span temporarily. For
example, launching a task that should not be associated with the current
request. To do this, simply pass null to <literal>withSpanInScope</literal>.</simpara>
<programlisting language="java" linenumbering="unnumbered">try (SpanInScope cleared = tracer.withSpanInScope(null)) {
startBackgroundThread();
}</programlisting>
</section>
</chapter>
<chapter xml:id="_instrumentation">
<title>Instrumentation</title>
<simpara>Spring Cloud Sleuth instruments all your Spring application
automatically, so you shouldn&#8217;t have to do anything to activate
it. The instrumentation is added using a variety of technologies
according to the stack that is available, e.g. for a servlet web
application we use a <literal>Filter</literal>, and for Spring Integration we use
<literal>ChannelInterceptors</literal>.</simpara>
<simpara>You can customize the keys used in span tags. To limit the volume of
span data, by default an HTTP request will be tagged only with a
handful of metadata like the status code, host and URL. You can add
request headers by configuring <literal>spring.sleuth.keys.http.headers</literal> (a
list of header names).</simpara>
<note>
<simpara>Remember that tags are only collected and exported if there is a
<literal>Sampler</literal> that allows it (by default there is not, so there is no
danger of accidentally collecting too much data without configuring
something).</simpara>
</note>
</chapter>
<chapter xml:id="_span_lifecycle">
<title>Span lifecycle</title>
<simpara>You can do the following operations on the Span by means of <emphasis role="strong">brave.Tracer</emphasis>:</simpara>
<itemizedlist>
<listitem>
<simpara><link linkend="creating-and-finishing-spans">start</link> - when you start a span its name is assigned and start timestamp is recorded.</simpara>
</listitem>
<listitem>
<simpara><link linkend="creating-and-finishing-spans">close</link> - the span gets finished (the end time of the span is recorded) and if
the span is <emphasis role="strong">sampled</emphasis> then it will be eligible for collection to e.g. Zipkin.</simpara>
</listitem>
<listitem>
<simpara><link linkend="continuing-spans">continue</link> - a new instance of span will be created whereas it will be a copy of the
one that it continues.</simpara>
</listitem>
<listitem>
<simpara><link linkend="continuing-spans">detach</link> - the span doesn&#8217;t get stopped or closed. It only gets removed from the current thread.</simpara>
</listitem>
<listitem>
<simpara><link linkend="creating-spans-with-explicit-parent">create with explicit parent</link> - you can create a new span and set an explicit parent to it</simpara>
</listitem>
</itemizedlist>
<tip>
<simpara>Spring Cloud Sleuth creates the instance of <literal>Tracer</literal> for you. In order to use it,
all you need is to just autowire it.</simpara>
</tip>
<section xml:id="creating-and-finishing-spans">
<title>Creating and finishing spans</title>
<simpara>You can manually create spans by using the <emphasis role="strong">Tracer</emphasis>.</simpara>
<programlisting language="java" linenumbering="unnumbered">// Start a span. If there was a span present in this thread it will become
// the `newSpan`'s parent.
Span newSpan = this.tracer.nextSpan().name("calculateTax");
try (Tracer.SpanInScope ws = this.tracer.withSpanInScope(newSpan.start())) {
// ...
// You can tag a span
newSpan.tag("taxValue", taxValue);
// ...
// You can log an event on a span
newSpan.annotate("taxCalculated");
} finally {
// Once done remember to finish the span. This will allow collecting
// the span to send it to Zipkin
newSpan.finish();
}</programlisting>
<simpara>In this example we could see how to create a new instance of span. Assuming that there already
was a span present in this thread then it would become the parent of that span.</simpara>
<important>
<simpara>Always clean after you create a span! Don&#8217;t forget to finish a span if you want to send it to Zipkin.</simpara>
</important>
<important>
<simpara>If your span contains a name greater than 50 chars, then that name will
be truncated to 50 chars. Your names have to be explicit and concrete. Big names lead to
latency issues and sometimes even thrown exceptions.</simpara>
</important>
</section>
<section xml:id="continuing-spans">
<title>Continuing spans</title>
<simpara>Sometimes you don&#8217;t want to create a new span but you want to continue one. Example of such a
situation might be (of course it all depends on the use-case):</simpara>
<itemizedlist>
<listitem>
<simpara><emphasis role="strong">AOP</emphasis> - If there was already a span created before an aspect was reached then you might not want to create a new span.</simpara>
</listitem>
<listitem>
<simpara><emphasis role="strong">Hystrix</emphasis> - executing a Hystrix command is most likely a logical part of the current processing. It&#8217;s in fact
only a technical implementation detail that you wouldn&#8217;t necessarily want to reflect in tracing as a separate being.</simpara>
</listitem>
</itemizedlist>
<simpara>To continue a span you can use <emphasis role="strong">brave.Tracer</emphasis>.</simpara>
<programlisting language="java" linenumbering="unnumbered">// let's assume that we're in a thread Y and we've received
// the `initialSpan` from thread X
Span continuedSpan = this.tracer.joinSpan(newSpan.context());
try {
// ...
// You can tag a span
continuedSpan.tag("taxValue", taxValue);
// ...
// You can log an event on a span
continuedSpan.annotate("taxCalculated");
} finally {
// Once done remember to flush the span. That means that
// it will get reported but the span itself is not yet finished
continuedSpan.flush();
}</programlisting>
</section>
<section xml:id="creating-spans-with-explicit-parent">
<title>Creating spans with an explicit parent</title>
<simpara>There is a possibility that you want to start a new span and provide an explicit parent of that span.
Let&#8217;s assume that the parent of a span is in one thread and you want to start a new span in another thread.
In Brave, whenever you call <literal>nextSpan()</literal>, it&#8217;s creating one in reference
to the span being currently in scope. It&#8217;s enough to just put
the span in scope and then call <literal>nextSpan()</literal>, as presented in the example below:</simpara>
<programlisting language="java" linenumbering="unnumbered">// let's assume that we're in a thread Y and we've received
// the `initialSpan` from thread X. `initialSpan` will be the parent
// of the `newSpan`
Span newSpan = null;
try (Tracer.SpanInScope ws = this.tracer.withSpanInScope(initialSpan)) {
newSpan = this.tracer.nextSpan().name("calculateCommission");
// ...
// You can tag a span
newSpan.tag("commissionValue", commissionValue);
// ...
// You can log an event on a span
newSpan.annotate("commissionCalculated");
} finally {
// Once done remember to finish the span. This will allow collecting
// the span to send it to Zipkin. The tags and events set on the
// newSpan will not be present on the parent
if (newSpan != null) {
newSpan.finish();
}
}</programlisting>
<important>
<simpara>After having created such a span remember to finish it, otherwise it will not get
reported to e.g. Zipkin</simpara>
</important>
</section>
</chapter>
<chapter xml:id="_naming_spans">
<title>Naming spans</title>
<simpara>Picking a span name is not a trivial task. Span name should depict an operation name. The name should
be low cardinality (e.g. not include identifiers).</simpara>
<simpara>Since there is a lot of instrumentation going on some of the span names will be
artificial like:</simpara>
<itemizedlist>
<listitem>
<simpara><literal>controller-method-name</literal> when received by a Controller with a method name <literal>conrollerMethodName</literal></simpara>
</listitem>
<listitem>
<simpara><literal>async</literal> for asynchronous operations done via wrapped <literal>Callable</literal> and <literal>Runnable</literal>.</simpara>
</listitem>
<listitem>
<simpara><literal>@Scheduled</literal> annotated methods will return the simple name of the class.</simpara>
</listitem>
</itemizedlist>
<simpara>Fortunately, for the asynchronous processing you can provide explicit naming.</simpara>
<section xml:id="__spanname_annotation">
<title>@SpanName annotation</title>
<simpara>You can name the span explicitly via the <literal>@SpanName</literal> annotation.</simpara>
<programlisting language="java" linenumbering="unnumbered">@SpanName("calculateTax")
class TaxCountingRunnable implements Runnable {
@Override public void run() {
// perform logic
}
}</programlisting>
<simpara>In this case, when processed in the following manner:</simpara>
<programlisting language="java" linenumbering="unnumbered">Runnable runnable = new TraceRunnable(tracer, spanNamer, errorParser,
new TaxCountingRunnable());
Future&lt;?&gt; future = executorService.submit(runnable);
// ... some additional logic ...
future.get();</programlisting>
<simpara>The span will be named <literal>calculateTax</literal>.</simpara>
</section>
<section xml:id="_tostring_method">
<title>toString() method</title>
<simpara>It&#8217;s pretty rare to create separate classes for <literal>Runnable</literal> or <literal>Callable</literal>. Typically one creates an anonymous
instance of those classes. You can&#8217;t annotate such classes thus to override that, if there is no <literal>@SpanName</literal> annotation present,
we&#8217;re checking if the class has a custom implementation of the <literal>toString()</literal> method.</simpara>
<simpara>So executing such code:</simpara>
<programlisting language="java" linenumbering="unnumbered">Runnable runnable = new TraceRunnable(tracer, spanNamer, errorParser, new Runnable() {
@Override public void run() {
// perform logic
}
@Override public String toString() {
return "calculateTax";
}
});
Future&lt;?&gt; future = executorService.submit(runnable);
// ... some additional logic ...
future.get();</programlisting>
<simpara>will lead in creating a span named <literal>calculateTax</literal>.</simpara>
</section>
</chapter>
<chapter xml:id="_managing_spans_with_annotations">
<title>Managing spans with annotations</title>
<section xml:id="_rationale">
<title>Rationale</title>
<simpara>The main arguments for this features are</simpara>
<itemizedlist>
<listitem>
<simpara>api-agnostic means to collaborate with a span</simpara>
<itemizedlist>
<listitem>
<simpara>use of annotations allows users to add to a span with no library dependency on a span api.
This allows Sleuth to change its core api less impact to user code.</simpara>
</listitem>
</itemizedlist>
</listitem>
<listitem>
<simpara>reduced surface area for basic span operations.</simpara>
<itemizedlist>
<listitem>
<simpara>without this feature one has to use the span api, which has lifecycle commands that
could be used incorrectly. By only exposing scope, tag and log functionality, users can
collaborate without accidentally breaking span lifecycle.</simpara>
</listitem>
</itemizedlist>
</listitem>
<listitem>
<simpara>collaboration with runtime generated code</simpara>
<itemizedlist>
<listitem>
<simpara>with libraries such as Spring Data / Feign the implementations of interfaces are generated
at runtime thus span wrapping of objects was tedious. Now you can provide annotations
over interfaces and arguments of those interfaces</simpara>
</listitem>
</itemizedlist>
</listitem>
</itemizedlist>
</section>
<section xml:id="_creating_new_spans">
<title>Creating new spans</title>
<simpara>If you really don&#8217;t want to take care of creating local spans manually you can profit from the
<literal>@NewSpan</literal> annotation. Also we give you the <literal>@SpanTag</literal> annotation to add tags in an automated
fashion.</simpara>
<simpara>Let&#8217;s look at some examples of usage.</simpara>
<programlisting language="java" linenumbering="unnumbered">@NewSpan
void testMethod();</programlisting>
<simpara>Annotating the method without any parameter will lead to a creation of a new span whose name
will be equal to annotated method name.</simpara>
<programlisting language="java" linenumbering="unnumbered">@NewSpan("customNameOnTestMethod4")
void testMethod4();</programlisting>
<simpara>If you provide the value in the annotation (either directly or via the <literal>name</literal> parameter) then
the created span will have the name as the provided value.</simpara>
<programlisting language="java" linenumbering="unnumbered">// method declaration
@NewSpan(name = "customNameOnTestMethod5")
void testMethod5(@SpanTag("testTag") String param);
// and method execution
this.testBean.testMethod5("test");</programlisting>
<simpara>You can combine both the name and a tag. Let&#8217;s focus on the latter. In this case whatever the value of
the annotated method&#8217;s parameter runtime value will be - that will be the value of the tag. In our sample
the tag key will be <literal>testTag</literal> and the tag value will be <literal>test</literal>.</simpara>
<programlisting language="java" linenumbering="unnumbered">@NewSpan(name = "customNameOnTestMethod3")
@Override
public void testMethod3() {
}</programlisting>
<simpara>You can place the <literal>@NewSpan</literal> annotation on both the class and an interface. If you override the
interface&#8217;s method and provide a different value of the <literal>@NewSpan</literal> annotation then the most
concrete one wins (in this case <literal>customNameOnTestMethod3</literal> will be set).</simpara>
</section>
<section xml:id="_continuing_spans">
<title>Continuing spans</title>
<simpara>If you want to just add tags and annotations to an existing span it&#8217;s enough
to use the <literal>@ContinueSpan</literal> annotation as presented below. Note that in contrast
with the <literal>@NewSpan</literal> annotation you can also add logs via the <literal>log</literal> parameter:</simpara>
<programlisting language="java" linenumbering="unnumbered">// method declaration
@ContinueSpan(log = "testMethod11")
void testMethod11(@SpanTag("testTag11") String param);
// method execution
this.testBean.testMethod11("test");
this.testBean.testMethod13();</programlisting>
<simpara>That way the span will get continued and:</simpara>
<itemizedlist>
<listitem>
<simpara>logs with name <literal>testMethod11.before</literal> and <literal>testMethod11.after</literal> will be created</simpara>
</listitem>
<listitem>
<simpara>if an exception will be thrown a log <literal>testMethod11.afterFailure</literal> will also be created</simpara>
</listitem>
<listitem>
<simpara>tag with key <literal>testTag11</literal> and value <literal>test</literal> will be created</simpara>
</listitem>
</itemizedlist>
</section>
<section xml:id="_more_advanced_tag_setting">
<title>More advanced tag setting</title>
<simpara>There are 3 different ways to add tags to a span. All of them are controlled by the <literal>SpanTag</literal> annotation.
Precedence is:</simpara>
<itemizedlist>
<listitem>
<simpara>try with the bean of <literal>TagValueResolver</literal> type and provided name</simpara>
</listitem>
<listitem>
<simpara>if one hasn&#8217;t provided the bean name, try to evaluate an expression. We&#8217;re searching for a <literal>TagValueExpressionResolver</literal> bean.
The default implementation uses SPEL expression resolution.</simpara>
</listitem>
<listitem>
<simpara>if one hasn&#8217;t provided any expression to evaluate just return a <literal>toString()</literal> value of the parameter</simpara>
</listitem>
</itemizedlist>
<section xml:id="_custom_extractor">
<title>Custom extractor</title>
<simpara>The value of the tag for following method will be computed by an implementation of <literal>TagValueResolver</literal> interface.
Its class name has to be passed as the value of the <literal>resolver</literal> attribute.</simpara>
<simpara>Having such an annotated method:</simpara>
<programlisting language="java" linenumbering="unnumbered">@NewSpan
public void getAnnotationForTagValueResolver(@SpanTag(key = "test", resolver = TagValueResolver.class) String test) {
}</programlisting>
<simpara>and such a <literal>TagValueResolver</literal> bean implementation</simpara>
<programlisting language="java" linenumbering="unnumbered">@Bean(name = "myCustomTagValueResolver")
public TagValueResolver tagValueResolver() {
return parameter -&gt; "Value from myCustomTagValueResolver";
}</programlisting>
<simpara>Will lead to setting of a tag value equal to <literal>Value from myCustomTagValueResolver</literal>.</simpara>
</section>
<section xml:id="_resolving_expressions_for_value">
<title>Resolving expressions for value</title>
<simpara>Having such an annotated method:</simpara>
<programlisting language="java" linenumbering="unnumbered">@NewSpan
public void getAnnotationForTagValueExpression(@SpanTag(key = "test", expression = "length() + ' characters'") String test) {
}</programlisting>
<simpara>and no custom implementation of a <literal>TagValueExpressionResolver</literal> will lead to evaluation of the SPEL expression and a tag with value <literal>4 characters</literal> will be set on the span.
If you want to use some other expression resolution mechanism you can create your own implementation
of the bean.</simpara>
</section>
<section xml:id="_using_tostring_method">
<title>Using toString method</title>
<simpara>Having such an annotated method:</simpara>
<programlisting language="java" linenumbering="unnumbered">@NewSpan
public void getAnnotationForArgumentToString(@SpanTag("test") Long param) {
}</programlisting>
<simpara>if executed with a value of <literal>15</literal> will lead to setting of a tag with a String value of <literal>"15"</literal>.</simpara>
</section>
</section>
</chapter>
<chapter xml:id="_customizations">
<title>Customizations</title>
<section xml:id="_spring_integration">
<title>Spring Integration</title>
</section>
<section xml:id="_http">
<title>HTTP</title>
</section>
<section xml:id="_tracefilter">
<title>TraceFilter</title>
<simpara>You can also modify the behaviour of the <literal>TraceFilter</literal> - the component that is responsible
for processing the input HTTP request and adding tags basing on the HTTP response. You can customize
the tags, or modify the response headers by registering your own instance of the <literal>TraceFilter</literal> bean.</simpara>
<simpara>In the following example we will register the <literal>TraceFilter</literal> bean and we will add the
<literal>ZIPKIN-TRACE-ID</literal> response header containing the current Span&#8217;s trace id. Also we will
add to the Span a tag with key <literal>custom</literal> and a value <literal>tag</literal>.</simpara>
<programlisting language="java" linenumbering="unnumbered">@Component
@Order(TraceFilter.ORDER + 1)
class MyFilter extends GenericFilterBean {
private final Tracer tracer;
MyFilter(Tracer tracer) {
this.tracer = tracer;
}
@Override public void doFilter(ServletRequest request, ServletResponse response,
FilterChain chain) throws IOException, ServletException {
Span currentSpan = this.tracer.currentSpan();
then(currentSpan).isNotNull();
// for readability we're returning trace id in a hex form
((HttpServletResponse) response)
.addHeader("ZIPKIN-TRACE-ID",
currentSpan.context().traceIdString());
// we can also add some custom tags
currentSpan.tag("custom", "tag");
chain.doFilter(request, response);
}
}</programlisting>
</section>
<section xml:id="_custom_service_name">
<title>Custom service name</title>
<simpara>By default Sleuth assumes that when you send a span to Zipkin, you want the span&#8217;s service name
to be equal to <literal>spring.application.name</literal> value. That&#8217;s not always the case though. There
are situations in which you want to explicitly provide a different service name for all spans coming
from your application. To achieve that it&#8217;s enough to just pass the following property
to your application to override that value (example for <literal>foo</literal> service name):</simpara>
<programlisting language="yaml" linenumbering="unnumbered">spring.zipkin.service.name: foo</programlisting>
</section>
<section xml:id="_customization_of_reported_spans">
<title>Customization of reported spans</title>
<simpara>Before reporting spans to e.g. Zipkin you can be interested in modifying that span in some way.
You can achieve that by using the <literal>SpanAdjuster</literal> interface.</simpara>
<simpara>In Sleuth we&#8217;re generating spans with a fixed name. Some users want to modify the name depending on values
of tags. Implementation of the <literal>SpanAdjuster</literal> interface can be used to alter that name. Example:</simpara>
<simpara>Example. If you register two beans of <literal>SpanAdjuster</literal> type:</simpara>
<programlisting language="java" linenumbering="unnumbered">@Bean SpanAdjuster adjusterOne() {
return span -&gt; span.toBuilder().name("foo").build();
}
@Bean SpanAdjuster adjusterTwo() {
return span -&gt; span.toBuilder().name(span.name() + " bar").build();
}</programlisting>
<simpara>This will lead in changing the name of the reported span to <literal>foo bar</literal>, just before it gets reported (e.g. to Zipkin).</simpara>
</section>
<section xml:id="_host_locator">
<title>Host locator</title>
<important>
<simpara>This section is about defining <emphasis role="strong">host</emphasis> from service discovery. It&#8217;s <emphasis role="strong">NOT</emphasis>
about finding Zipkin in service discovery.</simpara>
</important>
<simpara>In order to define the host that is corresponding to a particular span we need to resolve the host name
and port. The default approach is to take it from server properties. If those for some reason are not set
then we&#8217;re trying to retrieve the host name from the network interfaces.</simpara>
<simpara>If you have the discovery client enabled and prefer to retrieve the host address from the registered
instance in a service registry then you have to set the property (it&#8217;s applicable for both HTTP and
Stream based span reporting).</simpara>
<programlisting language="yaml" linenumbering="unnumbered">spring.zipkin.locator.discovery.enabled: true</programlisting>
</section>
</chapter>
<chapter xml:id="_sending_spans_to_zipkin">
<title>Sending spans to Zipkin</title>
<simpara>By default if you add <literal>spring-cloud-starter-zipkin</literal> as a dependency to your project,
when the span is closed, it will be sent to Zipkin over HTTP. The communication
is asynchronous. You can configure the URL by setting the <literal>spring.zipkin.baseUrl</literal>
property as follows:</simpara>
<programlisting language="yaml" linenumbering="unnumbered">spring.zipkin.baseUrl: http://192.168.99.100:9411/</programlisting>
<simpara>If you want to find Zipkin via service discovery it&#8217;s enough to pass the
Zipkin&#8217;s service id inside the URL (example for <literal>zipkinserver</literal> service id)</simpara>
<programlisting language="yaml" linenumbering="unnumbered">spring.zipkin.baseUrl: http://zipkinserver/</programlisting>
</chapter>
<chapter xml:id="_zipkin_stream_span_consumer">
<title>Zipkin Stream Span Consumer</title>
<important>
<simpara>The suggested approach is to use the Zipkin&#8217;s
native support for message based span sending. Starting from
Edgware Zipkin Stream server is deprecated and in Finchley
it got removed.</simpara>
</important>
<simpara>Please refer to the <link xl:href="http://cloud.spring.io/spring-cloud-static/Dalston.SR4/multi/multi__span_data_as_messages.html#_zipkin_consumer">Dalston Documentaion</link>
on how to create a Stream Zipkin server.</simpara>
</chapter>
<chapter xml:id="_integrations">
<title>Integrations</title>
<section xml:id="_opentracing">
<title>OpenTracing</title>
<simpara>Spring Cloud Sleuth is <link xl:href="http://opentracing.io/">OpenTracing</link> compatible. If you have
OpenTracing on the classpath we will automatically register the OpenTracing
<literal>Tracer</literal> bean. If you wish to disable this just set <literal>spring.sleuth.opentracing.enabled</literal> to <literal>false</literal></simpara>
</section>
<section xml:id="_runnable_and_callable">
<title>Runnable and Callable</title>
<simpara>If you&#8217;re wrapping your logic in <literal>Runnable</literal> or <literal>Callable</literal> it&#8217;s enough to wrap those classes in their Sleuth representative.</simpara>
<simpara>Example for <literal>Runnable</literal>:</simpara>
<programlisting language="java" linenumbering="unnumbered">Runnable runnable = new Runnable() {
@Override
public void run() {
// do some work
}
@Override
public String toString() {
return "spanNameFromToStringMethod";
}
};
// Manual `TraceRunnable` creation with explicit "calculateTax" Span name
Runnable traceRunnable = new TraceRunnable(tracer, spanNamer, errorParser,
runnable, "calculateTax");
// Wrapping `Runnable` with `Tracing`. That way the current span will be available
// in the thread of `Runnable`
Runnable traceRunnableFromTracer = tracing.currentTraceContext().wrap(runnable);</programlisting>
<simpara>Example for <literal>Callable</literal>:</simpara>
<programlisting language="java" linenumbering="unnumbered">Callable&lt;String&gt; callable = new Callable&lt;String&gt;() {
@Override
public String call() throws Exception {
return someLogic();
}
@Override
public String toString() {
return "spanNameFromToStringMethod";
}
};
// Manual `TraceCallable` creation with explicit "calculateTax" Span name
Callable&lt;String&gt; traceCallable = new TraceCallable&lt;&gt;(tracer, spanNamer, errorParser,
callable, "calculateTax");
// Wrapping `Callable` with `Tracing`. That way the current span will be available
// in the thread of `Callable`
Callable&lt;String&gt; traceCallableFromTracer = tracing.currentTraceContext().wrap(callable);</programlisting>
<simpara>That way you will ensure that a new Span is created and closed for each execution.</simpara>
</section>
<section xml:id="_hystrix">
<title>Hystrix</title>
<section xml:id="_custom_concurrency_strategy">
<title>Custom Concurrency Strategy</title>
<simpara>We&#8217;re registering a custom <link xl:href="https://github.com/Netflix/Hystrix/wiki/Plugins#concurrencystrategy"><literal>HystrixConcurrencyStrategy</literal></link>
that wraps all <literal>Callable</literal> instances into their Sleuth representative -
the <literal>TraceCallable</literal>. The strategy either starts or continues a span depending on the fact whether tracing was already going
on before the Hystrix command was called. To disable the custom Hystrix Concurrency Strategy set the <literal>spring.sleuth.hystrix.strategy.enabled</literal> to <literal>false</literal>.</simpara>
</section>
<section xml:id="_manual_command_setting">
<title>Manual Command setting</title>
<simpara>Assuming that you have the following <literal>HystrixCommand</literal>:</simpara>
<programlisting language="java" linenumbering="unnumbered">HystrixCommand&lt;String&gt; hystrixCommand = new HystrixCommand&lt;String&gt;(setter) {
@Override
protected String run() throws Exception {
return someLogic();
}
};</programlisting>
<simpara>In order to pass the tracing information you have to wrap the same logic in the Sleuth version of the <literal>HystrixCommand</literal> which is the
<literal>TraceCommand</literal>:</simpara>
<programlisting language="java" linenumbering="unnumbered">TraceCommand&lt;String&gt; traceCommand = new TraceCommand&lt;String&gt;(tracer, traceKeys, setter) {
@Override
public String doRun() throws Exception {
return someLogic();
}
};</programlisting>
</section>
</section>
<section xml:id="_rxjava">
<title>RxJava</title>
<simpara>We&#8217;re registering a custom <link xl:href="https://github.com/ReactiveX/RxJava/wiki/Plugins#rxjavaschedulershook"><literal>RxJavaSchedulersHook</literal></link>
that wraps all <literal>Action0</literal> instances into their Sleuth representative -
the <literal>TraceAction</literal>. The hook either starts or continues a span depending on the fact whether tracing was already going
on before the Action was scheduled. To disable the custom RxJavaSchedulersHook set the <literal>spring.sleuth.rxjava.schedulers.hook.enabled</literal> to <literal>false</literal>.</simpara>
<simpara>You can define a list of regular expressions for thread names, for which you don&#8217;t want a Span to be created. Just provide a comma separated list
of regular expressions in the <literal>spring.sleuth.rxjava.schedulers.ignoredthreads</literal> property.</simpara>
</section>
<section xml:id="_http_integration">
<title>HTTP integration</title>
<simpara>Features from this section can be disabled by providing the <literal>spring.sleuth.web.enabled</literal> property with value equal to <literal>false</literal>.</simpara>
<section xml:id="_http_filter">
<title>HTTP Filter</title>
<simpara>Via the <literal>TraceFilter</literal> all sampled incoming requests result in creation of a Span. That Span&#8217;s name is <literal>http:</literal> + the path to which
the request was sent. E.g. if the request was sent to <literal>/foo/bar</literal> then the name will be <literal>http:/foo/bar</literal>. You can configure which URIs you would
like to skip via the <literal>spring.sleuth.web.skipPattern</literal> property. If you have <literal>ManagementServerProperties</literal> on classpath then
its value of <literal>contextPath</literal> gets appended to the provided skip pattern.</simpara>
</section>
<section xml:id="_handlerinterceptor">
<title>HandlerInterceptor</title>
<simpara>Since we want the span names to be precise we&#8217;re using a <literal>TraceHandlerInterceptor</literal> that either wraps an
existing <literal>HandlerInterceptor</literal> or is added directly to the list of existing <literal>HandlerInterceptors</literal>. The
<literal>TraceHandlerInterceptor</literal> adds a special request attribute to the given <literal>HttpServletRequest</literal>. If the
the <literal>TraceFilter</literal> doesn&#8217;t see this attribute set it will create a "fallback" span which is an additional
span created on the server side so that the trace is presented properly in the UI. Seeing that most likely
signifies that there is a missing instrumentation. In that case please file an issue in Spring Cloud Sleuth.</simpara>
</section>
<section xml:id="_async_servlet_support">
<title>Async Servlet support</title>
<simpara>If your controller returns a <literal>Callable</literal> or a <literal>WebAsyncTask</literal> Spring Cloud Sleuth will continue the existing span instead of creating a new one.</simpara>
</section>
<section xml:id="_webflux_support">
<title>WebFlux support</title>
<simpara>Via the <literal>TraceWebFilter</literal> all sampled incoming requests result in creation of a Span. That Span&#8217;s name is <literal>http:</literal> + the path to which
the request was sent. E.g. if the request was sent to <literal>/foo/bar</literal> then the name will be <literal>http:/foo/bar</literal>. You can configure which URIs you would
like to skip via the <literal>spring.sleuth.web.skipPattern</literal> property. If you have <literal>ManagementServerProperties</literal> on classpath then
its value of <literal>contextPath</literal> gets appended to the provided skip pattern.</simpara>
</section>
</section>
<section xml:id="_http_client_integration">
<title>HTTP client integration</title>
<section xml:id="_synchronous_rest_template">
<title>Synchronous Rest Template</title>
<simpara>We&#8217;re injecting a <literal>RestTemplate</literal> interceptor that ensures that all the tracing information is passed to the requests. Each time a
call is made a new Span is created. It gets closed upon receiving the response. In order to block the synchronous <literal>RestTemplate</literal> features
just set <literal>spring.sleuth.web.client.enabled</literal> to <literal>false</literal>.</simpara>
<important>
<simpara>You have to register <literal>RestTemplate</literal> as a bean so that the interceptors will get injected.
If you create a <literal>RestTemplate</literal> instance with a <literal>new</literal> keyword then the instrumentation WILL NOT work.</simpara>
</important>
</section>
<section xml:id="_asynchronous_rest_template">
<title>Asynchronous Rest Template</title>
<important>
<simpara>Starting with Sleuth <literal>2.0.0</literal> we no longer register
a bean of <literal>AsyncRestTemplate</literal> type. It&#8217;s up to you to create such
a bean. Then we will instrument it.</simpara>
</important>
<simpara>To block the <literal>AsyncRestTemplate</literal> features set <literal>spring.sleuth.web.async.client.enabled</literal> to <literal>false</literal>.
To disable creation of the default <literal>TraceAsyncClientHttpRequestFactoryWrapper</literal> set <literal>spring.sleuth.web.async.client.factory.enabled</literal>
to <literal>false</literal>. If you don&#8217;t want to create <literal>AsyncRestClient</literal> at all set <literal>spring.sleuth.web.async.client.template.enabled</literal> to <literal>false</literal>.</simpara>
<section xml:id="_multiple_asynchronous_rest_templates">
<title>Multiple Asynchronous Rest Templates</title>
<simpara>Sometimes you need to use multiple implementations of Asynchronous Rest Template. In the following snippet you
can see an example of how to set up such a custom <literal>AsyncRestTemplate</literal>.</simpara>
<programlisting language="java" linenumbering="unnumbered">@Configuration
@EnableAutoConfiguration
static class Config {
@Bean(name = "customAsyncRestTemplate")
public AsyncRestTemplate traceAsyncRestTemplate() {
return new AsyncRestTemplate(asyncClientFactory(), clientHttpRequestFactory());
}
private ClientHttpRequestFactory clientHttpRequestFactory() {
ClientHttpRequestFactory clientHttpRequestFactory = new CustomClientHttpRequestFactory();
//CUSTOMIZE HERE
return clientHttpRequestFactory;
}
private AsyncClientHttpRequestFactory asyncClientFactory() {
AsyncClientHttpRequestFactory factory = new CustomAsyncClientHttpRequestFactory();
//CUSTOMIZE HERE
return factory;
}
}</programlisting>
</section>
</section>
<section xml:id="_webclient">
<title>WebClient</title>
<simpara>We inject a <literal>ExchangeFilterFunction</literal> implementation that creates a span and via on success and on
error callbacks takes care of closing client side spans.</simpara>
<important>
<simpara>You have to register <literal>WebClient</literal> as a bean so that the tracing instrumention gets applied.
If you create a <literal>WebClient</literal> instance with a <literal>new</literal> keyword then the instrumentation WILL NOT work.</simpara>
</important>
</section>
<section xml:id="_traverson">
<title>Traverson</title>
<simpara>If you&#8217;re using the <link xl:href="http://docs.spring.io/spring-hateoas/docs/current/reference/html/#client.traverson">Traverson</link> library
it&#8217;s enough for you to inject a <literal>RestTemplate</literal> as a bean into your Traverson object. Since <literal>RestTemplate</literal>
is already intercepted, you will get full support of tracing in your client. Below you can find a pseudo code
of how to do that:</simpara>
<programlisting language="java" linenumbering="unnumbered">@Autowired RestTemplate restTemplate;
Traverson traverson = new Traverson(URI.create("http://some/address"),
MediaType.APPLICATION_JSON, MediaType.APPLICATION_JSON_UTF8).setRestOperations(restTemplate);
// use Traverson</programlisting>
</section>
</section>
<section xml:id="_feign">
<title>Feign</title>
<simpara>By default Spring Cloud Sleuth provides integration with feign via the <literal>TraceFeignClientAutoConfiguration</literal>. You can disable it entirely
by setting <literal>spring.sleuth.feign.enabled</literal> to false. If you do so then no Feign related instrumentation will take place.</simpara>
<simpara>Part of Feign instrumentation is done via a <literal>FeignBeanPostProcessor</literal>. You can disable it by providing the <literal>spring.sleuth.feign.processor.enabled</literal> equal to <literal>false</literal>.
If you set it like this then Spring Cloud Sleuth will not instrument any of your custom Feign components. All the default instrumentation
however will be still there.</simpara>
</section>
<section xml:id="_asynchronous_communication">
<title>Asynchronous communication</title>
<section xml:id="__async_annotated_methods">
<title>@Async annotated methods</title>
<simpara>In Spring Cloud Sleuth we&#8217;re instrumenting async related components so that the tracing information is passed between threads.
You can disable this behaviour by setting the value of <literal>spring.sleuth.async.enabled</literal> to <literal>false</literal>.</simpara>
<simpara>If you annotate your method with <literal>@Async</literal> then we&#8217;ll automatically create a new Span with the following characteristics:</simpara>
<itemizedlist>
<listitem>
<simpara>if the method is annotated with <literal>@SpanName</literal> then the value of the annotation will be the Span&#8217;s name</simpara>
</listitem>
<listitem>
<simpara>if the method is <emphasis role="strong">not</emphasis> annotated with <literal>@SpanName</literal> the Span name will be the annotated method name</simpara>
</listitem>
<listitem>
<simpara>the Span will be tagged with that method&#8217;s class name and the method name too</simpara>
</listitem>
</itemizedlist>
</section>
<section xml:id="__scheduled_annotated_methods">
<title>@Scheduled annotated methods</title>
<simpara>In Spring Cloud Sleuth we&#8217;re instrumenting scheduled method execution so that the tracing information is passed between threads. You can disable this behaviour
by setting the value of <literal>spring.sleuth.scheduled.enabled</literal> to <literal>false</literal>.</simpara>
<simpara>If you annotate your method with <literal>@Scheduled</literal> then we&#8217;ll automatically create a new Span with the following characteristics:</simpara>
<itemizedlist>
<listitem>
<simpara>the Span name will be the annotated method name</simpara>
</listitem>
<listitem>
<simpara>the Span will be tagged with that method&#8217;s class name and the method name too</simpara>
</listitem>
</itemizedlist>
<simpara>If you want to skip Span creation for some <literal>@Scheduled</literal> annotated classes you can set the
<literal>spring.sleuth.scheduled.skipPattern</literal> with a regular expression that will match the fully qualified name of the
<literal>@Scheduled</literal> annotated class.</simpara>
<tip>
<simpara>If you are using <literal>spring-cloud-sleuth-stream</literal> and <literal>spring-cloud-netflix-hystrix-stream</literal> together, Span will be created for each Hystrix metrics and sent to Zipkin. This may be annoying. You can prevent this by setting <literal>spring.sleuth.scheduled.skipPattern=org.springframework.cloud.netflix.hystrix.stream.HystrixStreamTask</literal></simpara>
</tip>
</section>
<section xml:id="_executor_executorservice_and_scheduledexecutorservice">
<title>Executor, ExecutorService and ScheduledExecutorService</title>
<simpara>We&#8217;re providing <literal>LazyTraceExecutor</literal>, <literal>TraceableExecutorService</literal> and <literal>TraceableScheduledExecutorService</literal>. Those implementations
are creating Spans each time a new task is submitted, invoked or scheduled.</simpara>
<simpara>Here you can see an example of how to pass tracing information with <literal>TraceableExecutorService</literal> when working with <literal>CompletableFuture</literal>:</simpara>
<programlisting language="java" linenumbering="unnumbered">CompletableFuture&lt;Long&gt; completableFuture = CompletableFuture.supplyAsync(() -&gt; {
// perform some logic
return 1_000_000L;
}, new TraceableExecutorService(beanFactory, executorService,
// 'calculateTax' explicitly names the span - this param is optional
"calculateTax"));</programlisting>
<important>
<simpara>Sleuth doesn&#8217;t work with <literal>parallelStream()</literal> out of the box. If you want
to have the tracing information propagated through the stream you have to use the
approach with <literal>supplyAsync(...)</literal> as presented above.</simpara>
</important>
<section xml:id="_customization_of_executors">
<title>Customization of Executors</title>
<simpara>Sometimes you need to set up a custom instance of the <literal>AsyncExecutor</literal>. In the following snippet you
can see an example of how to set up such a custom <literal>Executor</literal>.</simpara>
<programlisting language="java" linenumbering="unnumbered">@Configuration
@EnableAutoConfiguration
@EnableAsync
static class CustomExecutorConfig extends AsyncConfigurerSupport {
@Autowired BeanFactory beanFactory;
@Override public Executor getAsyncExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
// CUSTOMIZE HERE
executor.setCorePoolSize(7);
executor.setMaxPoolSize(42);
executor.setQueueCapacity(11);
executor.setThreadNamePrefix("MyExecutor-");
// DON'T FORGET TO INITIALIZE
executor.initialize();
return new LazyTraceExecutor(this.beanFactory, executor);
}
}</programlisting>
</section>
</section>
</section>
<section xml:id="_messaging">
<title>Messaging</title>
<simpara>Spring Cloud Sleuth integrates with <link xl:href="http://projects.spring.io/spring-integration/">Spring Integration</link>. It creates spans for publish and
subscribe events. To disable Spring Integration instrumentation, set <literal>spring.sleuth.integration.enabled</literal> to false.</simpara>
<simpara>You can provide the <literal>spring.sleuth.integration.patterns</literal> pattern to explicitly
provide the names of channels that you want to include for tracing. By default all channels
are included.</simpara>
<important>
<simpara>When using the <literal>Executor</literal> to build a Spring Integration <literal>IntegrationFlow</literal> remember to use the <emphasis role="strong">untraced</emphasis> version of the <literal>Executor</literal>.
Decorating Spring Integration Executor Channel with <literal>TraceableExecutorService</literal> will cause the spans to be improperly closed.</simpara>
</important>
</section>
<section xml:id="_zuul">
<title>Zuul</title>
<simpara>We&#8217;re registering Zuul filters to propagate the tracing information (the request header is enriched with tracing data).
To disable Zuul support set the <literal>spring.sleuth.zuul.enabled</literal> property to <literal>false</literal>.</simpara>
</section>
</chapter>
<chapter xml:id="_running_examples">
<title>Running examples</title>
<simpara>You can find the running examples deployed in the <link xl:href="https://run.pivotal.io/">Pivotal Web Services</link>. Check them out in the following links:</simpara>
<itemizedlist>
<listitem>
<simpara><link xl:href="http://docssleuth-zipkin-server.cfapps.io/">Zipkin for apps presented in the samples to the top</link></simpara>
</listitem>
<listitem>
<simpara><link xl:href="http://docsbrewing-zipkin-server.cfapps.io/">Zipkin for Brewery on PWS</link>, its <link xl:href="https://github.com/spring-cloud-samples/brewery">Github Code</link></simpara>
</listitem>
</itemizedlist>
</chapter>
</book>