1859 lines
105 KiB
XML
1859 lines
105 KiB
XML
<?xml version="1.0" encoding="UTF-8"?>
|
||
<?asciidoc-toc maxdepth="8"?>
|
||
<?asciidoc-numbered?>
|
||
<book xmlns="http://docbook.org/ns/docbook" xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en">
|
||
<info>
|
||
<title>Spring Cloud Sleuth</title>
|
||
<date>2018-01-21</date>
|
||
<author>
|
||
<personname>
|
||
<firstname>Adrian Cole, Spencer Gibb, Marcin Grzejszczak, Dave Syer</firstname>
|
||
</personname>
|
||
</author>
|
||
<authorinitials>A</authorinitials>
|
||
</info>
|
||
<preface>
|
||
<title></title>
|
||
<simpara><emphasis role="strong">2.0.0.BUILD-SNAPSHOT</emphasis></simpara>
|
||
</preface>
|
||
<chapter xml:id="_introduction">
|
||
<title>Introduction</title>
|
||
<simpara>Spring Cloud Sleuth implements a distributed tracing solution for <link xl:href="http://cloud.spring.io">Spring Cloud</link>.</simpara>
|
||
<section xml:id="_terminology">
|
||
<title>Terminology</title>
|
||
<simpara>Spring Cloud Sleuth borrows <link xl:href="http://research.google.com/pubs/pub36356.html">Dapper’s</link> terminology.</simpara>
|
||
<simpara><emphasis role="strong">Span:</emphasis> The basic unit of work. For example, sending an RPC is a new span, as is sending a response to an
|
||
RPC. Span’s are identified by a unique 64-bit ID for the span and another 64-bit ID for the trace the span
|
||
is a part of. Spans also have other data, such as descriptions, timestamped events, key-value
|
||
annotations (tags), the ID of the span that caused them, and process ID’s (normally IP address).</simpara>
|
||
<simpara>Spans are started and stopped, and they keep track of their timing information. Once you create a
|
||
span, you must stop it at some point in the future.</simpara>
|
||
<tip>
|
||
<simpara>The initial span that starts a trace is called a <literal>root span</literal>. The value of span id
|
||
of that span is equal to trace id.</simpara>
|
||
</tip>
|
||
<simpara><emphasis role="strong">Trace:</emphasis> A set of spans forming a tree-like structure. For example, if you are running a distributed
|
||
big-data store, a trace might be formed by a put request.</simpara>
|
||
<simpara><emphasis role="strong">Annotation:</emphasis> is used to record existence of an event in time. With
|
||
<link xl:href="https://github.com/openzipkin/brave">Brave</link> instrumentation we no longer need to set special events
|
||
for <link xl:href="https://zipkin.io/">Zipkin</link> to understand who the client and server are and where
|
||
the request started and where it has ended. For learning purposes
|
||
however we will mark these events to highlight what kind
|
||
of an action took place.</simpara>
|
||
<itemizedlist>
|
||
<listitem>
|
||
<simpara><emphasis role="strong">cs</emphasis> - Client Sent - The client has made a request. This annotation depicts the start of the span.</simpara>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara><emphasis role="strong">sr</emphasis> - Server Received - The server side got the request and will start processing it.
|
||
If one subtracts the cs timestamp from this timestamp one will receive the network latency.</simpara>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara><emphasis role="strong">ss</emphasis> - Server Sent - Annotated upon completion of request processing (when the response
|
||
got sent back to the client). If one subtracts the sr timestamp from this timestamp one
|
||
will receive the time needed by the server side to process the request.</simpara>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara><emphasis role="strong">cr</emphasis> - Client Received - Signifies the end of the span. The client has successfully received the
|
||
response from the server side. If one subtracts the cs timestamp from this timestamp one
|
||
will receive the whole time needed by the client to receive the response from the server.</simpara>
|
||
</listitem>
|
||
</itemizedlist>
|
||
<simpara>Visualization of what <emphasis role="strong">Span</emphasis> and <emphasis role="strong">Trace</emphasis> will look in a system together with the Zipkin annotations:</simpara>
|
||
<informalfigure>
|
||
<mediaobject>
|
||
<imageobject>
|
||
<imagedata fileref="https://raw.githubusercontent.com/spring-cloud/spring-cloud-sleuth/master/docs/src/main/asciidoc/images/trace-id.png"/>
|
||
</imageobject>
|
||
<textobject><phrase>Trace Info propagation</phrase></textobject>
|
||
</mediaobject>
|
||
</informalfigure>
|
||
<simpara>Each color of a note signifies a span (7 spans - from <emphasis role="strong">A</emphasis> to <emphasis role="strong">G</emphasis>). If you have such information in the note:</simpara>
|
||
<screen>Trace Id = X
|
||
Span Id = D
|
||
Client Sent</screen>
|
||
<simpara>That means that the current span has <emphasis role="strong">Trace-Id</emphasis> set to <emphasis role="strong">X</emphasis>, <emphasis role="strong">Span-Id</emphasis> set to <emphasis role="strong">D</emphasis>. Also, the
|
||
<emphasis role="strong">Client Sent</emphasis> event took place.</simpara>
|
||
<simpara>This is how the visualization of the parent / child relationship of spans would look like:</simpara>
|
||
<informalfigure>
|
||
<mediaobject>
|
||
<imageobject>
|
||
<imagedata fileref="https://raw.githubusercontent.com/spring-cloud/spring-cloud-sleuth/master/docs/src/main/asciidoc/images/parents.png"/>
|
||
</imageobject>
|
||
<textobject><phrase>Parent child relationship</phrase></textobject>
|
||
</mediaobject>
|
||
</informalfigure>
|
||
</section>
|
||
<section xml:id="_purpose">
|
||
<title>Purpose</title>
|
||
<simpara>In the following sections the example from the image above will be taken into consideration.</simpara>
|
||
<section xml:id="_distributed_tracing_with_zipkin">
|
||
<title>Distributed tracing with Zipkin</title>
|
||
<simpara>Altogether there are <emphasis role="strong">7 spans</emphasis> . If you go to traces in Zipkin you will see this number in the second trace:</simpara>
|
||
<informalfigure>
|
||
<mediaobject>
|
||
<imageobject>
|
||
<imagedata fileref="https://raw.githubusercontent.com/spring-cloud/spring-cloud-sleuth/master/docs/src/main/asciidoc/images/zipkin-traces.png"/>
|
||
</imageobject>
|
||
<textobject><phrase>Traces</phrase></textobject>
|
||
</mediaobject>
|
||
</informalfigure>
|
||
<simpara>However if you pick a particular trace then you will see <emphasis role="strong">4 spans</emphasis>:</simpara>
|
||
<informalfigure>
|
||
<mediaobject>
|
||
<imageobject>
|
||
<imagedata fileref="https://raw.githubusercontent.com/spring-cloud/spring-cloud-sleuth/master/docs/src/main/asciidoc/images/zipkin-ui.png"/>
|
||
</imageobject>
|
||
<textobject><phrase>Traces Info propagation</phrase></textobject>
|
||
</mediaobject>
|
||
</informalfigure>
|
||
<note>
|
||
<simpara>When picking a particular trace you will see merged spans. That means that if there were 2 spans sent to
|
||
Zipkin with Server Received and Server Sent / Client Received and Client Sent
|
||
annotations then they will presented as a single span.</simpara>
|
||
</note>
|
||
<simpara>Why is there a difference between the 7 and 4 spans in this case?</simpara>
|
||
<itemizedlist>
|
||
<listitem>
|
||
<simpara>2 spans come from <literal>http:/start</literal> span. It has the Server Received (SR) and Server Sent (SS) annotations.</simpara>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara>2 spans come from the RPC call from <literal>service1</literal> to <literal>service2</literal> to the <literal>http:/foo</literal> endpoint. The Client Sent (CS)
|
||
and Client Received (CR) events took place on <literal>service1</literal> side. Server Received (SR) and Server Sent (SS) events took place
|
||
on the <literal>service2</literal> side. Physically there are 2 spans but they form 1 logical span related to an RPC call.</simpara>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara>2 spans come from the RPC call from <literal>service2</literal> to <literal>service3</literal> to the <literal>http:/bar</literal> endpoint. The Client Sent (CS)
|
||
and Client Received (CR) events took place on <literal>service2</literal> side. Server Received (SR) and Server Sent (SS) events took place
|
||
on the <literal>service3</literal> side. Physically there are 2 spans but they form 1 logical span related to an RPC call.</simpara>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara>2 spans come from the RPC call from <literal>service2</literal> to <literal>service4</literal> to the <literal>http:/baz</literal> endpoint. The Client Sent (CS)
|
||
and Client Received (CR) events took place on <literal>service2</literal> side. Server Received (SR) and Server Sent (SS) events took place
|
||
on the <literal>service4</literal> side. Physically there are 2 spans but they form 1 logical span related to an RPC call.</simpara>
|
||
</listitem>
|
||
</itemizedlist>
|
||
<simpara>So if we count the physical spans we have <emphasis role="strong">1</emphasis> from <literal>http:/start</literal>, <emphasis role="strong">2</emphasis> from <literal>service1</literal> calling <literal>service2</literal>, <emphasis role="strong">2</emphasis> form <literal>service2</literal>
|
||
calling <literal>service3</literal> and <emphasis role="strong">2</emphasis> from <literal>service2</literal> calling <literal>service4</literal>. Altogether <emphasis role="strong">7</emphasis> spans.</simpara>
|
||
<simpara>Logically we see the information of <emphasis role="strong">Total Spans: 4</emphasis> because we have <emphasis role="strong">1</emphasis> span related to the incoming request
|
||
to <literal>service1</literal> and <emphasis role="strong">3</emphasis> spans related to RPC calls.</simpara>
|
||
</section>
|
||
<section xml:id="_visualizing_errors">
|
||
<title>Visualizing errors</title>
|
||
<simpara>Zipkin allows you to visualize errors in your trace. When an exception was thrown and wasn’t caught then we’re
|
||
setting proper tags on the span which Zipkin can properly colorize. You could see in the list of traces one
|
||
trace that was in red color. That’s because there was an exception thrown.</simpara>
|
||
<simpara>If you click that trace then you’ll see a similar picture</simpara>
|
||
<informalfigure>
|
||
<mediaobject>
|
||
<imageobject>
|
||
<imagedata fileref="https://raw.githubusercontent.com/spring-cloud/spring-cloud-sleuth/master/docs/src/main/asciidoc/images/zipkin-error-traces.png"/>
|
||
</imageobject>
|
||
<textobject><phrase>Error Traces</phrase></textobject>
|
||
</mediaobject>
|
||
</informalfigure>
|
||
<simpara>Then if you click on one of the spans you’ll see the following</simpara>
|
||
<informalfigure>
|
||
<mediaobject>
|
||
<imageobject>
|
||
<imagedata fileref="https://raw.githubusercontent.com/spring-cloud/spring-cloud-sleuth/master/docs/src/main/asciidoc/images/zipkin-error-trace-screenshot.png"/>
|
||
</imageobject>
|
||
<textobject><phrase>Error Traces Info propagation</phrase></textobject>
|
||
</mediaobject>
|
||
</informalfigure>
|
||
<simpara>As you can see you can easily see the reason for an error and the whole stacktrace related to it.</simpara>
|
||
</section>
|
||
<section xml:id="_distributed_tracing_with_brave">
|
||
<title>Distributed tracing with Brave</title>
|
||
<simpara>Starting with version <literal>2.0.0</literal>, Spring Cloud Sleuth uses
|
||
<link xl:href="https://github.com/openzipkin/brave">Brave</link> as the tracing library. That means
|
||
that Sleuth no longer takes care of storing the context but it delegates
|
||
that work to Brave.</simpara>
|
||
<simpara>Due to the fact that Sleuth had different naming / tagging
|
||
conventions than Brave, we’ve decided to follow the Brave’s
|
||
conventions from now on. However, if you want to use the legacy
|
||
Sleuth approaches, it’s enough to set the <literal>spring.sleuth.http.legacy.enabled</literal> property
|
||
to <literal>true</literal>.</simpara>
|
||
</section>
|
||
<section xml:id="_live_examples">
|
||
<title>Live examples</title>
|
||
<figure>
|
||
<title>Click Pivotal Web Services icon to see it live!</title>
|
||
<mediaobject>
|
||
<imageobject>
|
||
<imagedata fileref="https://raw.githubusercontent.com/spring-cloud/spring-cloud-sleuth/master/docs/src/main/asciidoc/images/pws.png" contentwidth="150" contentdepth="74"/>
|
||
</imageobject>
|
||
<textobject><phrase>Zipkin deployed on Pivotal Web Services</phrase></textobject>
|
||
</mediaobject>
|
||
</figure>
|
||
<simpara><link xl:href="http://docssleuth-zipkin-server.cfapps.io/">Click here to see it live!</link></simpara>
|
||
<simpara>The dependency graph in Zipkin would look like this:</simpara>
|
||
<informalfigure>
|
||
<mediaobject>
|
||
<imageobject>
|
||
<imagedata fileref="https://raw.githubusercontent.com/spring-cloud/spring-cloud-sleuth/master/docs/src/main/asciidoc/images/dependencies.png"/>
|
||
</imageobject>
|
||
<textobject><phrase>Dependencies</phrase></textobject>
|
||
</mediaobject>
|
||
</informalfigure>
|
||
<figure>
|
||
<title>Click Pivotal Web Services icon to see it live!</title>
|
||
<mediaobject>
|
||
<imageobject>
|
||
<imagedata fileref="https://raw.githubusercontent.com/spring-cloud/spring-cloud-sleuth/master/docs/src/main/asciidoc/images/pws.png" contentwidth="150" contentdepth="74"/>
|
||
</imageobject>
|
||
<textobject><phrase>Zipkin deployed on Pivotal Web Services</phrase></textobject>
|
||
</mediaobject>
|
||
</figure>
|
||
<simpara><link xl:href="http://docssleuth-zipkin-server.cfapps.io/dependency">Click here to see it live!</link></simpara>
|
||
</section>
|
||
<section xml:id="_log_correlation">
|
||
<title>Log correlation</title>
|
||
<simpara>When grepping the logs of those four applications by trace id equal to e.g. <literal>2485ec27856c56f4</literal> one would get the following:</simpara>
|
||
<screen>service1.log:2016-02-26 11:15:47.561 INFO [service1,2485ec27856c56f4,2485ec27856c56f4,true] 68058 --- [nio-8081-exec-1] i.s.c.sleuth.docs.service1.Application : Hello from service1. Calling service2
|
||
service2.log:2016-02-26 11:15:47.710 INFO [service2,2485ec27856c56f4,9aa10ee6fbde75fa,true] 68059 --- [nio-8082-exec-1] i.s.c.sleuth.docs.service2.Application : Hello from service2. Calling service3 and then service4
|
||
service3.log:2016-02-26 11:15:47.895 INFO [service3,2485ec27856c56f4,1210be13194bfe5,true] 68060 --- [nio-8083-exec-1] i.s.c.sleuth.docs.service3.Application : Hello from service3
|
||
service2.log:2016-02-26 11:15:47.924 INFO [service2,2485ec27856c56f4,9aa10ee6fbde75fa,true] 68059 --- [nio-8082-exec-1] i.s.c.sleuth.docs.service2.Application : Got response from service3 [Hello from service3]
|
||
service4.log:2016-02-26 11:15:48.134 INFO [service4,2485ec27856c56f4,1b1845262ffba49d,true] 68061 --- [nio-8084-exec-1] i.s.c.sleuth.docs.service4.Application : Hello from service4
|
||
service2.log:2016-02-26 11:15:48.156 INFO [service2,2485ec27856c56f4,9aa10ee6fbde75fa,true] 68059 --- [nio-8082-exec-1] i.s.c.sleuth.docs.service2.Application : Got response from service4 [Hello from service4]
|
||
service1.log:2016-02-26 11:15:48.182 INFO [service1,2485ec27856c56f4,2485ec27856c56f4,true] 68058 --- [nio-8081-exec-1] i.s.c.sleuth.docs.service1.Application : Got response from service2 [Hello from service2, response from service3 [Hello from service3] and from service4 [Hello from service4]]</screen>
|
||
<simpara>If you’re using a log aggregating tool like <link xl:href="https://www.elastic.co/products/kibana">Kibana</link>,
|
||
<link xl:href="http://www.splunk.com/">Splunk</link> etc. you can order the events that took place. An example of
|
||
Kibana would look like this:</simpara>
|
||
<informalfigure>
|
||
<mediaobject>
|
||
<imageobject>
|
||
<imagedata fileref="https://raw.githubusercontent.com/spring-cloud/spring-cloud-sleuth/master/docs/src/main/asciidoc/images/kibana.png"/>
|
||
</imageobject>
|
||
<textobject><phrase>Log correlation with Kibana</phrase></textobject>
|
||
</mediaobject>
|
||
</informalfigure>
|
||
<simpara>If you want to use <link xl:href="https://www.elastic.co/guide/en/logstash/current/index.html">Logstash</link> here is the Grok pattern for Logstash:</simpara>
|
||
<screen>filter {
|
||
# pattern matching logback pattern
|
||
grok {
|
||
match => { "message" => "%{TIMESTAMP_ISO8601:timestamp}\s+%{LOGLEVEL:severity}\s+\[%{DATA:service},%{DATA:trace},%{DATA:span},%{DATA:exportable}\]\s+%{DATA:pid}\s+---\s+\[%{DATA:thread}\]\s+%{DATA:class}\s+:\s+%{GREEDYDATA:rest}" }
|
||
}
|
||
}</screen>
|
||
<note>
|
||
<simpara>If you want to use Grok together with the logs from Cloud Foundry you have to use this pattern:</simpara>
|
||
</note>
|
||
<screen>filter {
|
||
# pattern matching logback pattern
|
||
grok {
|
||
match => { "message" => "(?m)OUT\s+%{TIMESTAMP_ISO8601:timestamp}\s+%{LOGLEVEL:severity}\s+\[%{DATA:service},%{DATA:trace},%{DATA:span},%{DATA:exportable}\]\s+%{DATA:pid}\s+---\s+\[%{DATA:thread}\]\s+%{DATA:class}\s+:\s+%{GREEDYDATA:rest}" }
|
||
}
|
||
}</screen>
|
||
<section xml:id="_json_logback_with_logstash">
|
||
<title>JSON Logback with Logstash</title>
|
||
<simpara>Often you do not want to store your logs in a text file but in a JSON file that Logstash can immediately pick. To do that you have to do the following (for readability
|
||
we’re passing the dependencies in the <literal>groupId:artifactId:version</literal> notation.</simpara>
|
||
<simpara><emphasis role="strong">Dependencies setup</emphasis></simpara>
|
||
<itemizedlist>
|
||
<listitem>
|
||
<simpara>Ensure that Logback is on the classpath (<literal>ch.qos.logback:logback-core</literal>)</simpara>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara>Add Logstash Logback encode - example for version <literal>4.6</literal> : <literal>net.logstash.logback:logstash-logback-encoder:4.6</literal></simpara>
|
||
</listitem>
|
||
</itemizedlist>
|
||
<simpara><emphasis role="strong">Logback setup</emphasis></simpara>
|
||
<simpara>Below you can find an example of a Logback configuration (file named <link xl:href="https://github.com/spring-cloud-samples/sleuth-documentation-apps/blob/master/service1/src/main/resources/logback-spring.xml">logback-spring.xml</link>) that:</simpara>
|
||
<itemizedlist>
|
||
<listitem>
|
||
<simpara>logs information from the application in a JSON format to a <literal>build/${spring.application.name}.json</literal> file</simpara>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara>has commented out two additional appenders - console and standard log file</simpara>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara>has the same logging pattern as the one presented in the previous section</simpara>
|
||
</listitem>
|
||
</itemizedlist>
|
||
<programlisting language="xml" linenumbering="unnumbered"><?xml version="1.0" encoding="UTF-8"?>
|
||
<configuration>
|
||
<include resource="org/springframework/boot/logging/logback/defaults.xml"/>
|
||
|
||
<springProperty scope="context" name="springAppName" source="spring.application.name"/>
|
||
<!-- Example for logging into the build folder of your project -->
|
||
<property name="LOG_FILE" value="${BUILD_FOLDER:-build}/${springAppName}"/>
|
||
|
||
<!-- You can override this to have a custom pattern -->
|
||
<property name="CONSOLE_LOG_PATTERN"
|
||
value="%clr(%d{yyyy-MM-dd HH:mm:ss.SSS}){faint} %clr(${LOG_LEVEL_PATTERN:-%5p}) %clr(${PID:- }){magenta} %clr(---){faint} %clr([%15.15t]){faint} %clr(%-40.40logger{39}){cyan} %clr(:){faint} %m%n${LOG_EXCEPTION_CONVERSION_WORD:-%wEx}"/>
|
||
|
||
<!-- Appender to log to console -->
|
||
<appender name="console" class="ch.qos.logback.core.ConsoleAppender">
|
||
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
|
||
<!-- Minimum logging level to be presented in the console logs-->
|
||
<level>DEBUG</level>
|
||
</filter>
|
||
<encoder>
|
||
<pattern>${CONSOLE_LOG_PATTERN}</pattern>
|
||
<charset>utf8</charset>
|
||
</encoder>
|
||
</appender>
|
||
|
||
<!-- Appender to log to file -->
|
||
<appender name="flatfile" class="ch.qos.logback.core.rolling.RollingFileAppender">
|
||
<file>${LOG_FILE}</file>
|
||
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
|
||
<fileNamePattern>${LOG_FILE}.%d{yyyy-MM-dd}.gz</fileNamePattern>
|
||
<maxHistory>7</maxHistory>
|
||
</rollingPolicy>
|
||
<encoder>
|
||
<pattern>${CONSOLE_LOG_PATTERN}</pattern>
|
||
<charset>utf8</charset>
|
||
</encoder>
|
||
</appender>
|
||
|
||
<!-- Appender to log to file in a JSON format -->
|
||
<appender name="logstash" class="ch.qos.logback.core.rolling.RollingFileAppender">
|
||
<file>${LOG_FILE}.json</file>
|
||
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
|
||
<fileNamePattern>${LOG_FILE}.json.%d{yyyy-MM-dd}.gz</fileNamePattern>
|
||
<maxHistory>7</maxHistory>
|
||
</rollingPolicy>
|
||
<encoder class="net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder">
|
||
<providers>
|
||
<timestamp>
|
||
<timeZone>UTC</timeZone>
|
||
</timestamp>
|
||
<pattern>
|
||
<pattern>
|
||
{
|
||
"severity": "%level",
|
||
"service": "${springAppName:-}",
|
||
"trace": "%X{X-B3-TraceId:-}",
|
||
"span": "%X{X-B3-SpanId:-}",
|
||
"parent": "%X{X-B3-ParentSpanId:-}",
|
||
"exportable": "%X{X-Span-Export:-}",
|
||
"pid": "${PID:-}",
|
||
"thread": "%thread",
|
||
"class": "%logger{40}",
|
||
"rest": "%message"
|
||
}
|
||
</pattern>
|
||
</pattern>
|
||
</providers>
|
||
</encoder>
|
||
</appender>
|
||
|
||
<root level="INFO">
|
||
<appender-ref ref="console"/>
|
||
<!-- uncomment this to have also JSON logs -->
|
||
<!--<appender-ref ref="logstash"/>-->
|
||
<!--<appender-ref ref="flatfile"/>-->
|
||
</root>
|
||
</configuration></programlisting>
|
||
<note>
|
||
<simpara>If you’re using a custom <literal>logback-spring.xml</literal> then you have to pass the <literal>spring.application.name</literal> in
|
||
<literal>bootstrap</literal> instead of <literal>application</literal> property file. Otherwise your custom logback file won’t read the property properly.</simpara>
|
||
</note>
|
||
</section>
|
||
</section>
|
||
<section xml:id="_propagating_span_context">
|
||
<title>Propagating Span Context</title>
|
||
<simpara>The span context is the state that must get propagated to any child Spans across process boundaries.
|
||
Part of the Span Context is the Baggage. The trace and span IDs are a required part of the span context.
|
||
Baggage is an optional part.</simpara>
|
||
<simpara>Baggage is a set of key:value pairs stored in the span context. Baggage travels together with the trace
|
||
and is attached to every span. Spring Cloud Sleuth will understand that a header is baggage related if the HTTP
|
||
header is prefixed with <literal>baggage-</literal> and for messaging it starts with <literal>baggage_</literal>.</simpara>
|
||
<important>
|
||
<simpara>There’s currently no limitation of the count or size of baggage items. However, keep in mind that
|
||
too many can decrease system throughput or increase RPC latency. In extreme cases, it could crash the app due
|
||
to exceeding transport-level message or header capacity.</simpara>
|
||
</important>
|
||
<simpara>Example of setting baggage on a span:</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">Unresolved directive in intro.adoc - include::https://raw.githubusercontent.com/spring-cloud/spring-cloud-sleuth/master/spring-cloud-sleuth-core/src/test/java/org/springframework/cloud/brave/instrument/web/multiple/MultipleHopsIntegrationTests.java[tags=baggage,indent=0]
|
||
}</programlisting>
|
||
<section xml:id="_baggage_vs_span_tags">
|
||
<title>Baggage vs. Span Tags</title>
|
||
<simpara>Baggage travels with the trace (i.e. every child span contains the baggage of its parent). Zipkin has no knowledge of
|
||
baggage and will not even receive that information.</simpara>
|
||
<simpara>Tags are attached to a specific span - they are presented for that particular span only. However you
|
||
can search by tag to find the trace, where there exists a span having the searched tag value.</simpara>
|
||
<simpara>If you want to be able to lookup a span based on baggage, you should add corresponding entry as a tag in the root span.</simpara>
|
||
<important>
|
||
<simpara>Remember that the span needs to be in scope!</simpara>
|
||
</important>
|
||
<programlisting language="java" linenumbering="unnumbered">Unresolved directive in intro.adoc - include::https://raw.githubusercontent.com/spring-cloud/spring-cloud-sleuth/master/spring-cloud-sleuth-core/src/test/java/org/springframework/cloud/brave/instrument/web/multiple/MultipleHopsIntegrationTests.java[tags=baggage_tag,indent=0]</programlisting>
|
||
</section>
|
||
</section>
|
||
</section>
|
||
<section xml:id="_adding_to_the_project">
|
||
<title>Adding to the project</title>
|
||
<important>
|
||
<simpara>To ensure that your application name is properly displayed in Zipkin
|
||
set the <literal>spring.application.name</literal> property in <literal>bootstrap.yml</literal>.</simpara>
|
||
</important>
|
||
<section xml:id="_only_sleuth_log_correlation">
|
||
<title>Only Sleuth (log correlation)</title>
|
||
<simpara>If you want to profit only from Spring Cloud Sleuth without the Zipkin integration just add
|
||
the <literal>spring-cloud-starter-sleuth</literal> module to your project.</simpara>
|
||
<formalpara role="primary">
|
||
<title>Maven</title>
|
||
<para>
|
||
<programlisting language="xml" linenumbering="unnumbered"><dependencyManagement> <co xml:id="CO1-1"/>
|
||
<dependencies>
|
||
<dependency>
|
||
<groupId>org.springframework.cloud</groupId>
|
||
<artifactId>spring-cloud-dependencies</artifactId>
|
||
<version>${release.train.version}</version>
|
||
<type>pom</type>
|
||
<scope>import</scope>
|
||
</dependency>
|
||
</dependencies>
|
||
</dependencyManagement>
|
||
|
||
<dependency> <co xml:id="CO1-2"/>
|
||
<groupId>org.springframework.cloud</groupId>
|
||
<artifactId>spring-cloud-starter-sleuth</artifactId>
|
||
</dependency></programlisting>
|
||
</para>
|
||
</formalpara>
|
||
<calloutlist>
|
||
<callout arearefs="CO1-1">
|
||
<para>In order not to pick versions by yourself it’s much better if you add the dependency management via
|
||
the Spring BOM</para>
|
||
</callout>
|
||
<callout arearefs="CO1-2">
|
||
<para>Add the dependency to <literal>spring-cloud-starter-sleuth</literal></para>
|
||
</callout>
|
||
</calloutlist>
|
||
<formalpara role="secondary">
|
||
<title>Gradle</title>
|
||
<para>
|
||
<programlisting language="groovy" linenumbering="unnumbered">dependencyManagement { <co xml:id="CO2-1"/>
|
||
imports {
|
||
mavenBom "org.springframework.cloud:spring-cloud-dependencies:${releaseTrainVersion}"
|
||
}
|
||
}
|
||
|
||
dependencies { <co xml:id="CO2-2"/>
|
||
compile "org.springframework.cloud:spring-cloud-starter-sleuth"
|
||
}</programlisting>
|
||
</para>
|
||
</formalpara>
|
||
<calloutlist>
|
||
<callout arearefs="CO2-1">
|
||
<para>In order not to pick versions by yourself it’s much better if you add the dependency management via
|
||
the Spring BOM</para>
|
||
</callout>
|
||
<callout arearefs="CO2-2">
|
||
<para>Add the dependency to <literal>spring-cloud-starter-sleuth</literal></para>
|
||
</callout>
|
||
</calloutlist>
|
||
</section>
|
||
<section xml:id="_sleuth_with_zipkin_via_http">
|
||
<title>Sleuth with Zipkin via HTTP</title>
|
||
<simpara>If you want both Sleuth and Zipkin just add the <literal>spring-cloud-starter-zipkin</literal> dependency.</simpara>
|
||
<formalpara role="primary">
|
||
<title>Maven</title>
|
||
<para>
|
||
<programlisting language="xml" linenumbering="unnumbered"><dependencyManagement> <co xml:id="CO3-1"/>
|
||
<dependencies>
|
||
<dependency>
|
||
<groupId>org.springframework.cloud</groupId>
|
||
<artifactId>spring-cloud-dependencies</artifactId>
|
||
<version>${release.train.version}</version>
|
||
<type>pom</type>
|
||
<scope>import</scope>
|
||
</dependency>
|
||
</dependencies>
|
||
</dependencyManagement>
|
||
|
||
<dependency> <co xml:id="CO3-2"/>
|
||
<groupId>org.springframework.cloud</groupId>
|
||
<artifactId>spring-cloud-starter-zipkin</artifactId>
|
||
</dependency></programlisting>
|
||
</para>
|
||
</formalpara>
|
||
<calloutlist>
|
||
<callout arearefs="CO3-1">
|
||
<para>In order not to pick versions by yourself it’s much better if you add the dependency management via
|
||
the Spring BOM</para>
|
||
</callout>
|
||
<callout arearefs="CO3-2">
|
||
<para>Add the dependency to <literal>spring-cloud-starter-zipkin</literal></para>
|
||
</callout>
|
||
</calloutlist>
|
||
<formalpara role="secondary">
|
||
<title>Gradle</title>
|
||
<para>
|
||
<programlisting language="groovy" linenumbering="unnumbered">dependencyManagement { <co xml:id="CO4-1"/>
|
||
imports {
|
||
mavenBom "org.springframework.cloud:spring-cloud-dependencies:${releaseTrainVersion}"
|
||
}
|
||
}
|
||
|
||
dependencies { <co xml:id="CO4-2"/>
|
||
compile "org.springframework.cloud:spring-cloud-starter-zipkin"
|
||
}</programlisting>
|
||
</para>
|
||
</formalpara>
|
||
<calloutlist>
|
||
<callout arearefs="CO4-1">
|
||
<para>In order not to pick versions by yourself it’s much better if you add the dependency management via
|
||
the Spring BOM</para>
|
||
</callout>
|
||
<callout arearefs="CO4-2">
|
||
<para>Add the dependency to <literal>spring-cloud-starter-zipkin</literal></para>
|
||
</callout>
|
||
</calloutlist>
|
||
</section>
|
||
<section xml:id="_sleuth_with_zipkin_via_rabbitmq_or_kafka">
|
||
<title>Sleuth with Zipkin via RabbitMQ or Kafka</title>
|
||
<simpara>If you want to use RabbitMQ or Kafka instead of http, add the <literal>spring-rabbit</literal> or <literal>spring-kafka</literal>
|
||
dependencies. The default destination name is <literal>zipkin</literal>.</simpara>
|
||
<simpara><emphasis>Note: <literal>spring-cloud-sleuth-stream</literal> is deprecated and incompatible with these destinations</emphasis></simpara>
|
||
<simpara>If you want Sleuth over RabbitMQ add the <literal>spring-cloud-starter-zipkin</literal> and <literal>spring-rabbit</literal>
|
||
dependencies.</simpara>
|
||
<formalpara role="primary">
|
||
<title>Maven</title>
|
||
<para>
|
||
<programlisting language="xml" linenumbering="unnumbered"><dependencyManagement> <co xml:id="CO5-1"/>
|
||
<dependencies>
|
||
<dependency>
|
||
<groupId>org.springframework.cloud</groupId>
|
||
<artifactId>spring-cloud-dependencies</artifactId>
|
||
<version>${release.train.version}</version>
|
||
<type>pom</type>
|
||
<scope>import</scope>
|
||
</dependency>
|
||
</dependencies>
|
||
</dependencyManagement>
|
||
|
||
<dependency> <co xml:id="CO5-2"/>
|
||
<groupId>org.springframework.cloud</groupId>
|
||
<artifactId>spring-cloud-starter-zipkin</artifactId>
|
||
</dependency>
|
||
<dependency> <co xml:id="CO5-3"/>
|
||
<groupId>org.springframework.amqp</groupId>
|
||
<artifactId>spring-rabbit</artifactId>
|
||
</dependency></programlisting>
|
||
</para>
|
||
</formalpara>
|
||
<calloutlist>
|
||
<callout arearefs="CO5-1">
|
||
<para>In order not to pick versions by yourself it’s much better if you add the dependency management via
|
||
the Spring BOM</para>
|
||
</callout>
|
||
<callout arearefs="CO5-2">
|
||
<para>Add the dependency to <literal>spring-cloud-starter-zipkin</literal> - that way all dependent dependencies will be downloaded</para>
|
||
</callout>
|
||
<callout arearefs="CO5-3">
|
||
<para>To automatically configure rabbit, simply add the spring-rabbit dependency</para>
|
||
</callout>
|
||
</calloutlist>
|
||
<formalpara role="secondary">
|
||
<title>Gradle</title>
|
||
<para>
|
||
<programlisting language="groovy" linenumbering="unnumbered">dependencyManagement { <co xml:id="CO6-1"/>
|
||
imports {
|
||
mavenBom "org.springframework.cloud:spring-cloud-dependencies:${releaseTrainVersion}"
|
||
}
|
||
}
|
||
|
||
dependencies {
|
||
compile "org.springframework.cloud:spring-cloud-starter-zipkin" <co xml:id="CO6-2"/>
|
||
compile "org.springframework.amqp:spring-rabbit" <co xml:id="CO6-3"/>
|
||
}</programlisting>
|
||
</para>
|
||
</formalpara>
|
||
<calloutlist>
|
||
<callout arearefs="CO6-1">
|
||
<para>In order not to pick versions by yourself it’s much better if you add the dependency management via
|
||
the Spring BOM</para>
|
||
</callout>
|
||
<callout arearefs="CO6-2">
|
||
<para>Add the dependency to <literal>spring-cloud-starter-zipkin</literal> - that way all dependent dependencies will be downloaded</para>
|
||
</callout>
|
||
<callout arearefs="CO6-3">
|
||
<para>To automatically configure rabbit, simply add the spring-rabbit dependency</para>
|
||
</callout>
|
||
</calloutlist>
|
||
</section>
|
||
</section>
|
||
</chapter>
|
||
<chapter xml:id="_additional_resources">
|
||
<title>Additional resources</title>
|
||
<simpara><emphasis role="strong">Marcin Grzejszczak talking about Spring Cloud Sleuth and Zipkin</emphasis></simpara>
|
||
|
||
<simpara><link xl:href="https://www.youtube.com/watch?v=eQV71Mw1u1c">click here to see the video</link></simpara>
|
||
</chapter>
|
||
<chapter xml:id="_features">
|
||
<title>Features</title>
|
||
<itemizedlist>
|
||
<listitem>
|
||
<simpara>Adds trace and span ids to the Slf4J MDC, so you can extract all the logs from a given trace or span in a log aggregator. Example logs:</simpara>
|
||
<screen>2016-02-02 15:30:57.902 INFO [bar,6bfd228dc00d216b,6bfd228dc00d216b,false] 23030 --- [nio-8081-exec-3] ...
|
||
2016-02-02 15:30:58.372 ERROR [bar,6bfd228dc00d216b,6bfd228dc00d216b,false] 23030 --- [nio-8081-exec-3] ...
|
||
2016-02-02 15:31:01.936 INFO [bar,46ab0d418373cbc9,46ab0d418373cbc9,false] 23030 --- [nio-8081-exec-4] ...</screen>
|
||
<simpara>notice the <literal>[appname,traceId,spanId,exportable]</literal> entries from the MDC:</simpara>
|
||
<itemizedlist>
|
||
<listitem>
|
||
<simpara><emphasis role="strong">spanId</emphasis> - the id of a specific operation that took place</simpara>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara><emphasis role="strong">appname</emphasis> - the name of the application that logged the span</simpara>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara><emphasis role="strong">traceId</emphasis> - the id of the latency graph that contains the span</simpara>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara><emphasis role="strong">exportable</emphasis> - whether the log should be exported to Zipkin or not. When would you like the span not to be
|
||
exportable? In the case in which you want to wrap some operation in a Span and have it written to the logs
|
||
only.</simpara>
|
||
</listitem>
|
||
</itemizedlist>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara>Provides an abstraction over common distributed tracing data models: traces, spans (forming a DAG), annotations,
|
||
key-value annotations. Loosely based on HTrace, but Zipkin (Dapper) compatible.</simpara>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara>Sleuth records timing information to aid in latency analysis. Using sleuth, you can pinpoint causes of
|
||
latency in your applications. Sleuth is written to not log too much, and to not cause your production application to crash.</simpara>
|
||
<itemizedlist>
|
||
<listitem>
|
||
<simpara>propagates structural data about your call-graph in-band, and the rest out-of-band.</simpara>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara>includes opinionated instrumentation of layers such as HTTP</simpara>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara>includes sampling policy to manage volume</simpara>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara>can report to a Zipkin system for query and visualization</simpara>
|
||
</listitem>
|
||
</itemizedlist>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara>Instruments common ingress and egress points from Spring applications (servlet filter, async endpoints,
|
||
rest template, scheduled actions, message channels, zuul filters, feign client).</simpara>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara>Sleuth includes default logic to join a trace across http or messaging boundaries. For example, http propagation
|
||
works via Zipkin-compatible request headers. This propagation logic is defined and customized via
|
||
<literal>SpanInjector</literal> and <literal>SpanExtractor</literal> implementations.</simpara>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara>Sleuth gives you the possibility to propagate context (also known as baggage) between processes. That means that if you set on a Span
|
||
a baggage element then it will be sent downstream either via HTTP or messaging to other processes.</simpara>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara>Provides a way to create / continue spans and add tags and logs via annotations.</simpara>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara>If <literal>spring-cloud-sleuth-zipkin</literal> then the app will generate and collect Zipkin-compatible traces.
|
||
By default it sends them via HTTP to a Zipkin server on localhost (port 9411).
|
||
Configure the location of the service using <literal>spring.zipkin.baseUrl</literal>.</simpara>
|
||
<itemizedlist>
|
||
<listitem>
|
||
<simpara>If you depend on <literal>spring-rabbit</literal> or <literal>spring-kafka</literal> your app will send traces to a broker instead of http.</simpara>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara>Note: <literal>spring-cloud-sleuth-stream</literal> is deprecated and should no longer be used.</simpara>
|
||
</listitem>
|
||
</itemizedlist>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara>Spring Cloud Sleuth is <link xl:href="http://opentracing.io/">OpenTracing</link> compatible</simpara>
|
||
</listitem>
|
||
</itemizedlist>
|
||
<important>
|
||
<simpara>If using Zipkin, configure the percentage of spans exported using <literal>spring.sleuth.sampler.percentage</literal>
|
||
(default 0.1, i.e. 10%). <emphasis role="strong">Otherwise you might think that Sleuth is not working cause it’s omitting some spans.</emphasis></simpara>
|
||
</important>
|
||
<note>
|
||
<simpara>the SLF4J MDC is always set and logback users will immediately see the trace and span ids in logs per the example
|
||
above. Other logging systems have to configure their own formatter to get the same result. The default is
|
||
<literal>logging.pattern.level</literal> set to <literal>%5p [${spring.zipkin.service.name:${spring.application.name:-}},%X{X-B3-TraceId:-},%X{X-B3-SpanId:-},%X{X-Span-Export:-}]</literal>
|
||
(this is a Spring Boot feature for logback users).
|
||
<emphasis role="strong">This means that if you’re not using SLF4J this pattern WILL NOT be automatically applied</emphasis>.</simpara>
|
||
</note>
|
||
<section xml:id="_introduction_to_brave">
|
||
<title>Introduction to Brave</title>
|
||
<important>
|
||
<simpara>Starting with version <literal>2.0.0</literal> Spring Cloud Sleuth uses
|
||
<link xl:href="https://github.com/openzipkin/brave">Brave</link> as the tracing library.
|
||
For your convenience we’re embedding part of the Brave’s docs here.</simpara>
|
||
</important>
|
||
<simpara>Brave is a library used to capture and report latency information about
|
||
distributed operations to Zipkin. Most users won’t use Brave directly,
|
||
rather libraries or frameworks than employ Brave on their behalf.</simpara>
|
||
<simpara>This module includes tracer creates and joins spans that model the
|
||
latency of potentially distributed work. It also includes libraries to
|
||
propagate the trace context over network boundaries, for example, via
|
||
http headers.</simpara>
|
||
<section xml:id="_tracing">
|
||
<title>Tracing</title>
|
||
<simpara>Most importantly, you need a <literal>brave.Tracer</literal>, configured to [report to Zipkin]
|
||
(<link xl:href="https://github.com/openzipkin/zipkin-reporter-java">https://github.com/openzipkin/zipkin-reporter-java</link>).</simpara>
|
||
<simpara>Here’s an example setup that sends trace data (spans) to Zipkin over
|
||
http (as opposed to Kafka).</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">class MyClass {
|
||
|
||
private final Tracer tracer;
|
||
|
||
// Tracer will be autowired
|
||
MyClass(Tracer tracer) {
|
||
this.tracer = tracer;
|
||
}
|
||
|
||
void doSth() {
|
||
Span span = tracer.newTrace().name("encode").start();
|
||
// ...
|
||
}
|
||
}</programlisting>
|
||
<important>
|
||
<simpara>If your span contains a name greater than 50 chars, then that name will
|
||
be truncated to 50 chars. Your names have to be explicit and concrete. Big names lead to
|
||
latency issues and sometimes even thrown exceptions.</simpara>
|
||
</important>
|
||
</section>
|
||
<section xml:id="_tracing_2">
|
||
<title>Tracing</title>
|
||
<simpara>The tracer creates and joins spans that model the latency of potentially
|
||
distributed work. It can employ sampling to reduce overhead in process
|
||
or to reduce the amount of data sent to Zipkin.</simpara>
|
||
<simpara>Spans returned by a tracer report data to Zipkin when finished, or do
|
||
nothing if unsampled. After starting a span, you can annotate events of
|
||
interest or add tags containing details or lookup keys.</simpara>
|
||
<simpara>Spans have a context which includes trace identifiers that place it at
|
||
the correct spot in the tree representing the distributed operation.</simpara>
|
||
</section>
|
||
<section xml:id="_local_tracing">
|
||
<title>Local Tracing</title>
|
||
<simpara>When tracing local code, just run it inside a span.</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">Span span = tracer.newTrace().name("encode").start();
|
||
try {
|
||
doSomethingExpensive();
|
||
} finally {
|
||
span.finish();
|
||
}</programlisting>
|
||
<simpara>In the above example, the span is the root of the trace. In many cases,
|
||
you will be a part of an existing trace. When this is the case, call
|
||
<literal>newChild</literal> instead of <literal>newTrace</literal></simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">Span span = tracer.newChild(root.context()).name("encode").start();
|
||
try {
|
||
doSomethingExpensive();
|
||
} finally {
|
||
span.finish();
|
||
}</programlisting>
|
||
</section>
|
||
<section xml:id="_customizing_spans">
|
||
<title>Customizing spans</title>
|
||
<simpara>Once you have a span, you can add tags to it, which can be used as lookup
|
||
keys or details. For example, you might add a tag with your runtime
|
||
version.</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">span.tag("clnt/finagle.version", "6.36.0");</programlisting>
|
||
<simpara>When exposing the ability to customize spans to third parties, prefer
|
||
<literal>brave.SpanCustomizer</literal> as opposed to <literal>brave.Span</literal>. The former is simpler to
|
||
understand and test, and doesn’t tempt users with span lifecycle hooks.</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">interface MyTraceCallback {
|
||
void request(Request request, SpanCustomizer customizer);
|
||
}</programlisting>
|
||
<simpara>Since <literal>brave.Span</literal> implements <literal>brave.SpanCustomizer</literal>, it is just as easy for you
|
||
to pass to users.</simpara>
|
||
<simpara>Ex.</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">for (MyTraceCallback callback : userCallbacks) {
|
||
callback.request(request, span);
|
||
}</programlisting>
|
||
</section>
|
||
<section xml:id="_implicitly_looking_up_the_current_span">
|
||
<title>Implicitly looking up the current span</title>
|
||
<simpara>Sometimes you won’t know if a trace is in progress or not, and you don’t
|
||
want users to do null checks. <literal>brave.CurrentSpanCustomizer</literal> adds to any
|
||
span that’s in progress or drops data accordingly.</simpara>
|
||
<simpara>Ex.</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">// user code can then inject this without a chance of it being null.
|
||
@Autowire SpanCustomizer span;
|
||
|
||
void userCode() {
|
||
span.annotate("tx.started");
|
||
...
|
||
}</programlisting>
|
||
</section>
|
||
<section xml:id="_rpc_tracing">
|
||
<title>RPC tracing</title>
|
||
<simpara>Check for <link xl:href="https://github.com/openzipkin/sleuth/tree/master/instrumentation">instrumentation written here</link>
|
||
and <link xl:href="http://zipkin.io/pages/existing_instrumentations.html">Zipkin’s list</link>
|
||
before rolling your own RPC instrumentation!</simpara>
|
||
<simpara>RPC tracing is often done automatically by interceptors. Under the scenes,
|
||
they add tags and events that relate to their role in an RPC operation.</simpara>
|
||
<simpara>Here’s an example of a client span:</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">// before you send a request, add metadata that describes the operation
|
||
span = tracer.newTrace().name("get").type(CLIENT);
|
||
span.tag("clnt/finagle.version", "6.36.0");
|
||
span.tag(TraceKeys.HTTP_PATH, "/api");
|
||
span.remoteEndpoint(Endpoint.builder()
|
||
.serviceName("backend")
|
||
.ipv4(127 << 24 | 1)
|
||
.port(8080).build());
|
||
|
||
// when the request is scheduled, start the span
|
||
span.start();
|
||
|
||
// if you have callbacks for when data is on the wire, note those events
|
||
span.annotate(Constants.WIRE_SEND);
|
||
span.annotate(Constants.WIRE_RECV);
|
||
|
||
// when the response is complete, finish the span
|
||
span.finish();</programlisting>
|
||
<section xml:id="_one_way_tracing">
|
||
<title>One-Way tracing</title>
|
||
<simpara>Sometimes you need to model an asynchronous operation, where there is a
|
||
request, but no response. In normal RPC tracing, you use <literal>span.finish()</literal>
|
||
which indicates the response was received. In one-way tracing, you use
|
||
<literal>span.flush()</literal> instead, as you don’t expect a response.</simpara>
|
||
<simpara>Here’s how a client might model a one-way operation</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">// start a new span representing a client request
|
||
oneWaySend = tracer.newSpan(parent).kind(Span.Kind.CLIENT);
|
||
|
||
// Add the trace context to the request, so it can be propagated in-band
|
||
tracing.propagation().injector(Request::addHeader)
|
||
.inject(oneWaySend.context(), request);
|
||
|
||
// fire off the request asynchronously, totally dropping any response
|
||
request.execute();
|
||
|
||
// start the client side and flush instead of finish
|
||
oneWaySend.start().flush();</programlisting>
|
||
<simpara>And here’s how a server might handle this..</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">// pull the context out of the incoming request
|
||
extractor = tracing.propagation().extractor(Request::getHeader);
|
||
|
||
// convert that context to a span which you can name and add tags to
|
||
oneWayReceive = nextSpan(tracer, extractor.extract(request))
|
||
.name("process-request")
|
||
.kind(SERVER)
|
||
... add tags etc.
|
||
|
||
// start the server side and flush instead of finish
|
||
oneWayReceive.start().flush();
|
||
|
||
// you should not modify this span anymore as it is complete. However,
|
||
// you can create children to represent follow-up work.
|
||
next = tracer.newSpan(oneWayReceive.context()).name("step2").start();</programlisting>
|
||
<simpara><emphasis role="strong">Note</emphasis> The above propagation logic is a simplified version of our [http handlers](<link xl:href="https://github.com/openzipkin/sleuth/tree/master/instrumentation/http#http-server">https://github.com/openzipkin/sleuth/tree/master/instrumentation/http#http-server</link>).</simpara>
|
||
<simpara>There’s a working example of a one-way span [here](src/test/java/sleuth/features/async/OneWaySpanTest.java).</simpara>
|
||
</section>
|
||
</section>
|
||
</section>
|
||
</chapter>
|
||
<chapter xml:id="_sampling">
|
||
<title>Sampling</title>
|
||
<simpara>Sampling may be employed to reduce the data collected and reported out
|
||
of process. When a span isn’t sampled, it adds no overhead (noop).</simpara>
|
||
<simpara>Sampling is an up-front decision, meaning that the decision to report
|
||
data is made at the first operation in a trace, and that decision is
|
||
propagated downstream.</simpara>
|
||
<simpara>By default, there’s a global sampler that applies a single rate to all
|
||
traced operations. <literal>Tracer.Builder.sampler</literal> is how you indicate this,
|
||
and it defaults to trace every request.</simpara>
|
||
<section xml:id="_declarative_sampling">
|
||
<title>Declarative sampling</title>
|
||
<simpara>Some need to sample based on the type or annotations of a java method.</simpara>
|
||
<simpara>Most users will use a framework interceptor which automates this sort of
|
||
policy. Here’s how they might work internally.</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">// derives a sample rate from an annotation on a java method
|
||
DeclarativeSampler<Traced> sampler = DeclarativeSampler.create(Traced::sampleRate);
|
||
|
||
@Around("@annotation(traced)")
|
||
public Object traceThing(ProceedingJoinPoint pjp, Traced traced) throws Throwable {
|
||
Span span = tracing.tracer().newTrace(sampler.sample(traced))...
|
||
try {
|
||
return pjp.proceed();
|
||
} finally {
|
||
span.finish();
|
||
}
|
||
}</programlisting>
|
||
</section>
|
||
<section xml:id="_custom_sampling">
|
||
<title>Custom sampling</title>
|
||
<simpara>You may want to apply different policies depending on what the operation
|
||
is. For example, you might not want to trace requests to static resources
|
||
such as images, or you might want to trace all requests to a new api.</simpara>
|
||
<simpara>Most users will use a framework interceptor which automates this sort of
|
||
policy. Here’s how they might work internally.</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">Span newTrace(Request input) {
|
||
SamplingFlags flags = SamplingFlags.NONE;
|
||
if (input.url().startsWith("/experimental")) {
|
||
flags = SamplingFlags.SAMPLED;
|
||
} else if (input.url().startsWith("/static")) {
|
||
flags = SamplingFlags.NOT_SAMPLED;
|
||
}
|
||
return tracer.newTrace(flags);
|
||
}</programlisting>
|
||
<simpara>Note: the above is the basis for the built-in <link xl:href="https://github.com/openzipkin/sleuth/tree/master/instrumentation/http">http sampler</link></simpara>
|
||
</section>
|
||
<section xml:id="_sampling_in_spring_cloud_sleuth">
|
||
<title>Sampling in Spring Cloud Sleuth</title>
|
||
<simpara>Spring Cloud Sleuth by default sets all spans to non-exportable.
|
||
That means that you will see traces in logs, but not in any remote store.
|
||
For testing the default is often enough, and it probably is all you need
|
||
if you are only using the logs (e.g. with an ELK aggregator). If you are
|
||
exporting span data to Zipkin, there is also an <literal>Sampler.ALWAYS_SAMPLE</literal>
|
||
that exports everything and a <literal>ProbabilityBasedSampler</literal> that samples a
|
||
fixed fraction of spans.</simpara>
|
||
<note>
|
||
<simpara>The <literal>ProbabilityBasedSampler</literal> is the default if you are using
|
||
<literal>spring-cloud-sleuth-zipkin</literal>. You can
|
||
configure the exports using <literal>spring.sleuth.sampler.probability</literal>. The passed
|
||
value needs to be a double from <literal>0.0</literal> to <literal>1.0</literal>.</simpara>
|
||
</note>
|
||
<simpara>A sampler can be installed just by creating a bean definition, e.g:</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">@Bean
|
||
public Sampler defaultSampler() {
|
||
return Sampler.ALWAYS_SAMPLE;
|
||
}</programlisting>
|
||
<tip>
|
||
<simpara>You can set the HTTP header <literal>X-B3-Flags</literal> to <literal>1</literal> or when doing messaging you can
|
||
set <literal>spanFlags</literal> header to <literal>1</literal>. Then the current span will be forced to be exportable
|
||
regardless of the sampling decision.</simpara>
|
||
</tip>
|
||
</section>
|
||
</chapter>
|
||
<chapter xml:id="_propagation">
|
||
<title>Propagation</title>
|
||
<simpara>Propagation is needed to ensure activity originating from the same root
|
||
are collected together in the same trace. The most common propagation
|
||
approach is to copy a trace context from a client sending an RPC request
|
||
to a server receiving it.</simpara>
|
||
<simpara>For example, when an downstream Http call is made, its trace context is
|
||
sent along with it, encoded as request headers:</simpara>
|
||
<screen> Client Span Server Span
|
||
┌──────────────────┐ ┌──────────────────┐
|
||
│ │ │ │
|
||
│ TraceContext │ Http Request Headers │ TraceContext │
|
||
│ ┌──────────────┐ │ ┌───────────────────┐ │ ┌──────────────┐ │
|
||
│ │ TraceId │ │ │ X─B3─TraceId │ │ │ TraceId │ │
|
||
│ │ │ │ │ │ │ │ │ │
|
||
│ │ ParentSpanId │ │ Extract │ X─B3─ParentSpanId │ Inject │ │ ParentSpanId │ │
|
||
│ │ ├─┼─────────>│ ├────────┼>│ │ │
|
||
│ │ SpanId │ │ │ X─B3─SpanId │ │ │ SpanId │ │
|
||
│ │ │ │ │ │ │ │ │ │
|
||
│ │ Sampled │ │ │ X─B3─Sampled │ │ │ Sampled │ │
|
||
│ └──────────────┘ │ └───────────────────┘ │ └──────────────┘ │
|
||
│ │ │ │
|
||
└──────────────────┘ └──────────────────┘</screen>
|
||
<simpara>The names above are from <link xl:href="https://github.com/openzipkin/b3-propagation">B3 Propagation</link>,
|
||
which is built-in to Brave and has implementations in many languages and
|
||
frameworks.</simpara>
|
||
<simpara>Most users will use a framework interceptor which automates propagation.
|
||
Here’s how they might work internally.</simpara>
|
||
<simpara>Here’s what client-side propagation might look like</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">// configure a function that injects a trace context into a request
|
||
injector = tracing.propagation().injector(Request.Builder::addHeader);
|
||
|
||
// before a request is sent, add the current span's context to it
|
||
injector.inject(span.context(), request);</programlisting>
|
||
<simpara>Here’s what server-side propagation might look like</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">// configure a function that extracts the trace context from a request
|
||
extracted = tracing.propagation().extractor(Request::getHeader);
|
||
|
||
// when a server receives a request, it joins or starts a new trace
|
||
span = tracer.nextSpan(extracted, request);</programlisting>
|
||
<section xml:id="_propagating_extra_fields">
|
||
<title>Propagating extra fields</title>
|
||
<simpara>Sometimes you need to propagate extra fields, such as a request ID or an alternate trace context.
|
||
For example, if you are in a Cloud Foundry environment, you might want to pass the request ID:</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">// when you initialize the builder, define the extra field you want to propagate
|
||
tracingBuilder.propagationFactory(
|
||
ExtraFieldPropagation.newFactory(B3Propagation.FACTORY, "x-vcap-request-id")
|
||
);
|
||
|
||
// later, you can tag that request ID or use it in log correlation
|
||
requestId = ExtraFieldPropagation.get("x-vcap-request-id");</programlisting>
|
||
<simpara>You may also need to propagate a trace context you aren’t using. For example, you may be in an
|
||
Amazon Web Services environment, but not reporting data to X-Ray. To ensure X-Ray can co-exist
|
||
correctly, pass-through its tracing header like so.</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">tracingBuilder.propagationFactory(
|
||
ExtraFieldPropagation.newFactory(B3Propagation.FACTORY, "x-amzn-trace-id")
|
||
);</programlisting>
|
||
<section xml:id="_prefixed_fields">
|
||
<title>Prefixed fields</title>
|
||
<simpara>You can also prefix fields, if they follow a common pattern. For example, the following will
|
||
propagate the field "x-vcap-request-id" as-is, but send the fields "country-code" and "user-id"
|
||
on the wire as "x-baggage-country-code" and "x-baggage-user-id" respectively.</simpara>
|
||
<simpara>Setup your tracing instance with allowed fields:</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">tracingBuilder.propagationFactory(
|
||
ExtraFieldPropagation.newFactoryBuilder(B3Propagation.FACTORY)
|
||
.addField("x-vcap-request-id")
|
||
.addPrefixedFields("baggage-", Arrays.asList("country-code", "user-id"))
|
||
.build()
|
||
);</programlisting>
|
||
<simpara>Later, you can call below to affect the country code of the current trace context</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">ExtraFieldPropagation.set("country-code", "FO");
|
||
String countryCode = ExtraFieldPropagation.get("country-code");</programlisting>
|
||
<simpara>Or, if you have a reference to a trace context, use it explicitly</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">ExtraFieldPropagation.set(span.context(), "country-code", "FO");
|
||
String countryCode = ExtraFieldPropagation.get(span.context(), "country-code");</programlisting>
|
||
<important>
|
||
<simpara>In comparison to previous versions of Sleuth, with
|
||
Brave it’s required to pass the list of baggage keys.
|
||
There are two properties to achieve this. Via the <literal>spring.sleuth.baggage-keys</literal> you set keys
|
||
that will get prefixed with <literal>baggage-</literal> for http calls and <literal>baggage_</literal> for messaging. You can also pass
|
||
a list of prefixed keys that will be whitelisted without any prefix via
|
||
<literal>spring.sleuth.prefixed-keys</literal> property.</simpara>
|
||
</important>
|
||
</section>
|
||
<section xml:id="_extracting_a_propagated_context">
|
||
<title>Extracting a propagated context</title>
|
||
<simpara>The <literal>TraceContext.Extractor<C></literal> reads trace identifiers and sampling status
|
||
from an incoming request or message. The carrier is usually a request object
|
||
or headers.</simpara>
|
||
<simpara>This utility is used in standard instrumentation like [HttpServerHandler](../instrumentation/http/src/main/java/sleuth/http/HttpServerHandler.java),
|
||
but can also be used for custom RPC or messaging code.</simpara>
|
||
<simpara><literal>TraceContextOrSamplingFlags</literal> is usually only used with <literal>Tracer.nextSpan(extracted)</literal>, unless you are
|
||
sharing span IDs between a client and a server.</simpara>
|
||
</section>
|
||
<section xml:id="_sharing_span_ids_between_client_and_server">
|
||
<title>Sharing span IDs between client and server</title>
|
||
<simpara>A normal instrumentation pattern is creating a span representing the server
|
||
side of an RPC. <literal>Extractor.extract</literal> might return a complete trace context when
|
||
applied to an incoming client request. <literal>Tracer.joinSpan</literal> attempts to continue
|
||
the this trace, using the same span ID if supported, or creating a child span
|
||
if not. When span ID is shared, data reported includes a flag saying so.</simpara>
|
||
<simpara>Here’s an example of B3 propagation:</simpara>
|
||
<screen> ┌───────────────────┐ ┌───────────────────┐
|
||
Incoming Headers │ TraceContext │ │ TraceContext │
|
||
┌───────────────────┐(extract)│ ┌───────────────┐ │(join)│ ┌───────────────┐ │
|
||
│ X─B3-TraceId │─────────┼─┼> TraceId │ │──────┼─┼> TraceId │ │
|
||
│ │ │ │ │ │ │ │ │ │
|
||
│ X─B3-ParentSpanId │─────────┼─┼> ParentSpanId │ │──────┼─┼> ParentSpanId │ │
|
||
│ │ │ │ │ │ │ │ │ │
|
||
│ X─B3-SpanId │─────────┼─┼> SpanId │ │──────┼─┼> SpanId │ │
|
||
└───────────────────┘ │ │ │ │ │ │ │ │
|
||
│ │ │ │ │ │ Shared: true │ │
|
||
│ └───────────────┘ │ │ └───────────────┘ │
|
||
└───────────────────┘ └───────────────────┘</screen>
|
||
<simpara>Some propagation systems only forward the parent span ID, detected when
|
||
<literal>Propagation.Factory.supportsJoin() == false</literal>. In this case, a new span ID is
|
||
always provisioned and the incoming context determines the parent ID.</simpara>
|
||
<simpara>Here’s an example of AWS propagation:</simpara>
|
||
<screen> ┌───────────────────┐ ┌───────────────────┐
|
||
x-amzn-trace-id │ TraceContext │ │ TraceContext │
|
||
┌───────────────────┐(extract)│ ┌───────────────┐ │(join)│ ┌───────────────┐ │
|
||
│ Root │─────────┼─┼> TraceId │ │──────┼─┼> TraceId │ │
|
||
│ │ │ │ │ │ │ │ │ │
|
||
│ Parent │─────────┼─┼> SpanId │ │──────┼─┼> ParentSpanId │ │
|
||
└───────────────────┘ │ └───────────────┘ │ │ │ │ │
|
||
└───────────────────┘ │ │ SpanId: New │ │
|
||
│ └───────────────┘ │
|
||
└───────────────────┘</screen>
|
||
<simpara>Note: Some span reporters do not support sharing span IDs. For example, if you
|
||
set <literal>Tracing.Builder.spanReporter(amazonXrayOrGoogleStackdrive)</literal>, disable join
|
||
via <literal>Tracing.Builder.supportsJoin(false)</literal>. This will force a new child span on
|
||
<literal>Tracer.joinSpan()</literal>.</simpara>
|
||
</section>
|
||
<section xml:id="_implementing_propagation">
|
||
<title>Implementing Propagation</title>
|
||
<simpara><literal>TraceContext.Extractor<C></literal> is implemented by a <literal>Propagation.Factory</literal> plugin. Internally, this code
|
||
will create the union type <literal>TraceContextOrSamplingFlags</literal> with one of the following:
|
||
* <literal>TraceContext</literal> if trace and span IDs were present.
|
||
* <literal>TraceIdContext</literal> if a trace ID was present, but not span IDs.
|
||
* <literal>SamplingFlags</literal> if no identifiers were present</simpara>
|
||
<simpara>Some <literal>Propagation</literal> implementations carry extra data from point of extraction (ex reading incoming
|
||
headers) to injection (ex writing outgoing headers). For example, it might carry a request ID. When
|
||
implementations have extra data, here’s how they handle it.
|
||
* If a <literal>TraceContext</literal> was extracted, add the extra data as <literal>TraceContext.extra()</literal>
|
||
* Otherwise, add it as <literal>TraceContextOrSamplingFlags.extra()</literal>, which <literal>Tracer.nextSpan</literal> handles.</simpara>
|
||
</section>
|
||
</section>
|
||
</chapter>
|
||
<chapter xml:id="_current_tracing_component">
|
||
<title>Current Tracing Component</title>
|
||
<simpara>Brave supports a "current tracing component" concept which should only
|
||
be used when you have no other means to get a reference. This was made
|
||
for JDBC connections, as they often initialize prior to the tracing
|
||
component.</simpara>
|
||
<simpara>The most recent tracing component instantiated is available via
|
||
<literal>Tracing.current()</literal>. You there’s also a shortcut to get only the tracer
|
||
via <literal>Tracing.currentTracer()</literal>. If you use either of these methods, do
|
||
noot cache the result. Instead, look them up each time you need them.</simpara>
|
||
</chapter>
|
||
<chapter xml:id="_current_span">
|
||
<title>Current Span</title>
|
||
<simpara>Brave supports a "current span" concept which represents the in-flight
|
||
operation. <literal>Tracer.currentSpan()</literal> can be used to add custom tags to a
|
||
span and <literal>Tracer.nextSpan()</literal> can be used to create a child of whatever
|
||
is in-flight.</simpara>
|
||
<section xml:id="_setting_a_span_in_scope_manually">
|
||
<title>Setting a span in scope manually</title>
|
||
<simpara>When writing new instrumentation, it is important to place a span you
|
||
created in scope as the current span. Not only does this allow users to
|
||
access it with <literal>Tracer.currentSpan()</literal>, but it also allows customizations
|
||
like SLF4J MDC to see the current trace IDs.</simpara>
|
||
<simpara><literal>Tracer.withSpanInScope(Span)</literal> facilitates this and is most conveniently
|
||
employed via the try-with-resources idiom. Whenever external code might
|
||
be invoked (such as proceeding an interceptor or otherwise), place the
|
||
span in scope like this.</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">try (SpanInScope ws = tracer.withSpanInScope(span)) {
|
||
return inboundRequest.invoke();
|
||
} finally { // note the scope is independent of the span
|
||
span.finish();
|
||
}</programlisting>
|
||
<simpara>In edge cases, you may need to clear the current span temporarily. For
|
||
example, launching a task that should not be associated with the current
|
||
request. To do this, simply pass null to <literal>withSpanInScope</literal>.</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">try (SpanInScope cleared = tracer.withSpanInScope(null)) {
|
||
startBackgroundThread();
|
||
}</programlisting>
|
||
</section>
|
||
</chapter>
|
||
<chapter xml:id="_instrumentation">
|
||
<title>Instrumentation</title>
|
||
<simpara>Spring Cloud Sleuth instruments all your Spring application
|
||
automatically, so you shouldn’t have to do anything to activate
|
||
it. The instrumentation is added using a variety of technologies
|
||
according to the stack that is available, e.g. for a servlet web
|
||
application we use a <literal>Filter</literal>, and for Spring Integration we use
|
||
<literal>ChannelInterceptors</literal>.</simpara>
|
||
<simpara>You can customize the keys used in span tags. To limit the volume of
|
||
span data, by default an HTTP request will be tagged only with a
|
||
handful of metadata like the status code, host and URL. You can add
|
||
request headers by configuring <literal>spring.sleuth.keys.http.headers</literal> (a
|
||
list of header names).</simpara>
|
||
<note>
|
||
<simpara>Remember that tags are only collected and exported if there is a
|
||
<literal>Sampler</literal> that allows it (by default there is not, so there is no
|
||
danger of accidentally collecting too much data without configuring
|
||
something).</simpara>
|
||
</note>
|
||
</chapter>
|
||
<chapter xml:id="_span_lifecycle">
|
||
<title>Span lifecycle</title>
|
||
<simpara>You can do the following operations on the Span by means of <emphasis role="strong">brave.Tracer</emphasis>:</simpara>
|
||
<itemizedlist>
|
||
<listitem>
|
||
<simpara><link linkend="creating-and-finishing-spans">start</link> - when you start a span its name is assigned and start timestamp is recorded.</simpara>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara><link linkend="creating-and-finishing-spans">close</link> - the span gets finished (the end time of the span is recorded) and if
|
||
the span is <emphasis role="strong">sampled</emphasis> then it will be eligible for collection to e.g. Zipkin.</simpara>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara><link linkend="continuing-spans">continue</link> - a new instance of span will be created whereas it will be a copy of the
|
||
one that it continues.</simpara>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara><link linkend="continuing-spans">detach</link> - the span doesn’t get stopped or closed. It only gets removed from the current thread.</simpara>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara><link linkend="creating-spans-with-explicit-parent">create with explicit parent</link> - you can create a new span and set an explicit parent to it</simpara>
|
||
</listitem>
|
||
</itemizedlist>
|
||
<tip>
|
||
<simpara>Spring Cloud Sleuth creates the instance of <literal>Tracer</literal> for you. In order to use it,
|
||
all you need is to just autowire it.</simpara>
|
||
</tip>
|
||
<section xml:id="creating-and-finishing-spans">
|
||
<title>Creating and finishing spans</title>
|
||
<simpara>You can manually create spans by using the <emphasis role="strong">Tracer</emphasis>.</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">// Start a span. If there was a span present in this thread it will become
|
||
// the `newSpan`'s parent.
|
||
Span newSpan = this.tracer.nextSpan().name("calculateTax");
|
||
try (Tracer.SpanInScope ws = this.tracer.withSpanInScope(newSpan.start())) {
|
||
// ...
|
||
// You can tag a span
|
||
newSpan.tag("taxValue", taxValue);
|
||
// ...
|
||
// You can log an event on a span
|
||
newSpan.annotate("taxCalculated");
|
||
} finally {
|
||
// Once done remember to finish the span. This will allow collecting
|
||
// the span to send it to Zipkin
|
||
newSpan.finish();
|
||
}</programlisting>
|
||
<simpara>In this example we could see how to create a new instance of span. Assuming that there already
|
||
was a span present in this thread then it would become the parent of that span.</simpara>
|
||
<important>
|
||
<simpara>Always clean after you create a span! Don’t forget to finish a span if you want to send it to Zipkin.</simpara>
|
||
</important>
|
||
<important>
|
||
<simpara>If your span contains a name greater than 50 chars, then that name will
|
||
be truncated to 50 chars. Your names have to be explicit and concrete. Big names lead to
|
||
latency issues and sometimes even thrown exceptions.</simpara>
|
||
</important>
|
||
</section>
|
||
<section xml:id="continuing-spans">
|
||
<title>Continuing spans</title>
|
||
<simpara>Sometimes you don’t want to create a new span but you want to continue one. Example of such a
|
||
situation might be (of course it all depends on the use-case):</simpara>
|
||
<itemizedlist>
|
||
<listitem>
|
||
<simpara><emphasis role="strong">AOP</emphasis> - If there was already a span created before an aspect was reached then you might not want to create a new span.</simpara>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara><emphasis role="strong">Hystrix</emphasis> - executing a Hystrix command is most likely a logical part of the current processing. It’s in fact
|
||
only a technical implementation detail that you wouldn’t necessarily want to reflect in tracing as a separate being.</simpara>
|
||
</listitem>
|
||
</itemizedlist>
|
||
<simpara>To continue a span you can use <emphasis role="strong">brave.Tracer</emphasis>.</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">// let's assume that we're in a thread Y and we've received
|
||
// the `initialSpan` from thread X
|
||
Span continuedSpan = this.tracer.joinSpan(newSpan.context());
|
||
try {
|
||
// ...
|
||
// You can tag a span
|
||
continuedSpan.tag("taxValue", taxValue);
|
||
// ...
|
||
// You can log an event on a span
|
||
continuedSpan.annotate("taxCalculated");
|
||
} finally {
|
||
// Once done remember to flush the span. That means that
|
||
// it will get reported but the span itself is not yet finished
|
||
continuedSpan.flush();
|
||
}</programlisting>
|
||
</section>
|
||
<section xml:id="creating-spans-with-explicit-parent">
|
||
<title>Creating spans with an explicit parent</title>
|
||
<simpara>There is a possibility that you want to start a new span and provide an explicit parent of that span.
|
||
Let’s assume that the parent of a span is in one thread and you want to start a new span in another thread.
|
||
In Brave, whenever you call <literal>nextSpan()</literal>, it’s creating one in reference
|
||
to the span being currently in scope. It’s enough to just put
|
||
the span in scope and then call <literal>nextSpan()</literal>, as presented in the example below:</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">// let's assume that we're in a thread Y and we've received
|
||
// the `initialSpan` from thread X. `initialSpan` will be the parent
|
||
// of the `newSpan`
|
||
Span newSpan = null;
|
||
try (Tracer.SpanInScope ws = this.tracer.withSpanInScope(initialSpan)) {
|
||
newSpan = this.tracer.nextSpan().name("calculateCommission");
|
||
// ...
|
||
// You can tag a span
|
||
newSpan.tag("commissionValue", commissionValue);
|
||
// ...
|
||
// You can log an event on a span
|
||
newSpan.annotate("commissionCalculated");
|
||
} finally {
|
||
// Once done remember to finish the span. This will allow collecting
|
||
// the span to send it to Zipkin. The tags and events set on the
|
||
// newSpan will not be present on the parent
|
||
if (newSpan != null) {
|
||
newSpan.finish();
|
||
}
|
||
}</programlisting>
|
||
<important>
|
||
<simpara>After having created such a span remember to finish it, otherwise it will not get
|
||
reported to e.g. Zipkin</simpara>
|
||
</important>
|
||
</section>
|
||
</chapter>
|
||
<chapter xml:id="_naming_spans">
|
||
<title>Naming spans</title>
|
||
<simpara>Picking a span name is not a trivial task. Span name should depict an operation name. The name should
|
||
be low cardinality (e.g. not include identifiers).</simpara>
|
||
<simpara>Since there is a lot of instrumentation going on some of the span names will be
|
||
artificial like:</simpara>
|
||
<itemizedlist>
|
||
<listitem>
|
||
<simpara><literal>controller-method-name</literal> when received by a Controller with a method name <literal>conrollerMethodName</literal></simpara>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara><literal>async</literal> for asynchronous operations done via wrapped <literal>Callable</literal> and <literal>Runnable</literal>.</simpara>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara><literal>@Scheduled</literal> annotated methods will return the simple name of the class.</simpara>
|
||
</listitem>
|
||
</itemizedlist>
|
||
<simpara>Fortunately, for the asynchronous processing you can provide explicit naming.</simpara>
|
||
<section xml:id="__spanname_annotation">
|
||
<title>@SpanName annotation</title>
|
||
<simpara>You can name the span explicitly via the <literal>@SpanName</literal> annotation.</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">@SpanName("calculateTax")
|
||
class TaxCountingRunnable implements Runnable {
|
||
|
||
@Override public void run() {
|
||
// perform logic
|
||
}
|
||
}</programlisting>
|
||
<simpara>In this case, when processed in the following manner:</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">Runnable runnable = new TraceRunnable(tracer, spanNamer, errorParser,
|
||
new TaxCountingRunnable());
|
||
Future<?> future = executorService.submit(runnable);
|
||
// ... some additional logic ...
|
||
future.get();</programlisting>
|
||
<simpara>The span will be named <literal>calculateTax</literal>.</simpara>
|
||
</section>
|
||
<section xml:id="_tostring_method">
|
||
<title>toString() method</title>
|
||
<simpara>It’s pretty rare to create separate classes for <literal>Runnable</literal> or <literal>Callable</literal>. Typically one creates an anonymous
|
||
instance of those classes. You can’t annotate such classes thus to override that, if there is no <literal>@SpanName</literal> annotation present,
|
||
we’re checking if the class has a custom implementation of the <literal>toString()</literal> method.</simpara>
|
||
<simpara>So executing such code:</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">Runnable runnable = new TraceRunnable(tracer, spanNamer, errorParser, new Runnable() {
|
||
@Override public void run() {
|
||
// perform logic
|
||
}
|
||
|
||
@Override public String toString() {
|
||
return "calculateTax";
|
||
}
|
||
});
|
||
Future<?> future = executorService.submit(runnable);
|
||
// ... some additional logic ...
|
||
future.get();</programlisting>
|
||
<simpara>will lead in creating a span named <literal>calculateTax</literal>.</simpara>
|
||
</section>
|
||
</chapter>
|
||
<chapter xml:id="_managing_spans_with_annotations">
|
||
<title>Managing spans with annotations</title>
|
||
<section xml:id="_rationale">
|
||
<title>Rationale</title>
|
||
<simpara>The main arguments for this features are</simpara>
|
||
<itemizedlist>
|
||
<listitem>
|
||
<simpara>api-agnostic means to collaborate with a span</simpara>
|
||
<itemizedlist>
|
||
<listitem>
|
||
<simpara>use of annotations allows users to add to a span with no library dependency on a span api.
|
||
This allows Sleuth to change its core api less impact to user code.</simpara>
|
||
</listitem>
|
||
</itemizedlist>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara>reduced surface area for basic span operations.</simpara>
|
||
<itemizedlist>
|
||
<listitem>
|
||
<simpara>without this feature one has to use the span api, which has lifecycle commands that
|
||
could be used incorrectly. By only exposing scope, tag and log functionality, users can
|
||
collaborate without accidentally breaking span lifecycle.</simpara>
|
||
</listitem>
|
||
</itemizedlist>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara>collaboration with runtime generated code</simpara>
|
||
<itemizedlist>
|
||
<listitem>
|
||
<simpara>with libraries such as Spring Data / Feign the implementations of interfaces are generated
|
||
at runtime thus span wrapping of objects was tedious. Now you can provide annotations
|
||
over interfaces and arguments of those interfaces</simpara>
|
||
</listitem>
|
||
</itemizedlist>
|
||
</listitem>
|
||
</itemizedlist>
|
||
</section>
|
||
<section xml:id="_creating_new_spans">
|
||
<title>Creating new spans</title>
|
||
<simpara>If you really don’t want to take care of creating local spans manually you can profit from the
|
||
<literal>@NewSpan</literal> annotation. Also we give you the <literal>@SpanTag</literal> annotation to add tags in an automated
|
||
fashion.</simpara>
|
||
<simpara>Let’s look at some examples of usage.</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">@NewSpan
|
||
void testMethod();</programlisting>
|
||
<simpara>Annotating the method without any parameter will lead to a creation of a new span whose name
|
||
will be equal to annotated method name.</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">@NewSpan("customNameOnTestMethod4")
|
||
void testMethod4();</programlisting>
|
||
<simpara>If you provide the value in the annotation (either directly or via the <literal>name</literal> parameter) then
|
||
the created span will have the name as the provided value.</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">// method declaration
|
||
@NewSpan(name = "customNameOnTestMethod5")
|
||
void testMethod5(@SpanTag("testTag") String param);
|
||
|
||
// and method execution
|
||
this.testBean.testMethod5("test");</programlisting>
|
||
<simpara>You can combine both the name and a tag. Let’s focus on the latter. In this case whatever the value of
|
||
the annotated method’s parameter runtime value will be - that will be the value of the tag. In our sample
|
||
the tag key will be <literal>testTag</literal> and the tag value will be <literal>test</literal>.</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">@NewSpan(name = "customNameOnTestMethod3")
|
||
@Override
|
||
public void testMethod3() {
|
||
}</programlisting>
|
||
<simpara>You can place the <literal>@NewSpan</literal> annotation on both the class and an interface. If you override the
|
||
interface’s method and provide a different value of the <literal>@NewSpan</literal> annotation then the most
|
||
concrete one wins (in this case <literal>customNameOnTestMethod3</literal> will be set).</simpara>
|
||
</section>
|
||
<section xml:id="_continuing_spans">
|
||
<title>Continuing spans</title>
|
||
<simpara>If you want to just add tags and annotations to an existing span it’s enough
|
||
to use the <literal>@ContinueSpan</literal> annotation as presented below. Note that in contrast
|
||
with the <literal>@NewSpan</literal> annotation you can also add logs via the <literal>log</literal> parameter:</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">// method declaration
|
||
@ContinueSpan(log = "testMethod11")
|
||
void testMethod11(@SpanTag("testTag11") String param);
|
||
|
||
// method execution
|
||
this.testBean.testMethod11("test");
|
||
this.testBean.testMethod13();</programlisting>
|
||
<simpara>That way the span will get continued and:</simpara>
|
||
<itemizedlist>
|
||
<listitem>
|
||
<simpara>logs with name <literal>testMethod11.before</literal> and <literal>testMethod11.after</literal> will be created</simpara>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara>if an exception will be thrown a log <literal>testMethod11.afterFailure</literal> will also be created</simpara>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara>tag with key <literal>testTag11</literal> and value <literal>test</literal> will be created</simpara>
|
||
</listitem>
|
||
</itemizedlist>
|
||
</section>
|
||
<section xml:id="_more_advanced_tag_setting">
|
||
<title>More advanced tag setting</title>
|
||
<simpara>There are 3 different ways to add tags to a span. All of them are controlled by the <literal>SpanTag</literal> annotation.
|
||
Precedence is:</simpara>
|
||
<itemizedlist>
|
||
<listitem>
|
||
<simpara>try with the bean of <literal>TagValueResolver</literal> type and provided name</simpara>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara>if one hasn’t provided the bean name, try to evaluate an expression. We’re searching for a <literal>TagValueExpressionResolver</literal> bean.
|
||
The default implementation uses SPEL expression resolution.</simpara>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara>if one hasn’t provided any expression to evaluate just return a <literal>toString()</literal> value of the parameter</simpara>
|
||
</listitem>
|
||
</itemizedlist>
|
||
<section xml:id="_custom_extractor">
|
||
<title>Custom extractor</title>
|
||
<simpara>The value of the tag for following method will be computed by an implementation of <literal>TagValueResolver</literal> interface.
|
||
Its class name has to be passed as the value of the <literal>resolver</literal> attribute.</simpara>
|
||
<simpara>Having such an annotated method:</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">@NewSpan
|
||
public void getAnnotationForTagValueResolver(@SpanTag(key = "test", resolver = TagValueResolver.class) String test) {
|
||
}</programlisting>
|
||
<simpara>and such a <literal>TagValueResolver</literal> bean implementation</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">@Bean(name = "myCustomTagValueResolver")
|
||
public TagValueResolver tagValueResolver() {
|
||
return parameter -> "Value from myCustomTagValueResolver";
|
||
}</programlisting>
|
||
<simpara>Will lead to setting of a tag value equal to <literal>Value from myCustomTagValueResolver</literal>.</simpara>
|
||
</section>
|
||
<section xml:id="_resolving_expressions_for_value">
|
||
<title>Resolving expressions for value</title>
|
||
<simpara>Having such an annotated method:</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">@NewSpan
|
||
public void getAnnotationForTagValueExpression(@SpanTag(key = "test", expression = "length() + ' characters'") String test) {
|
||
}</programlisting>
|
||
<simpara>and no custom implementation of a <literal>TagValueExpressionResolver</literal> will lead to evaluation of the SPEL expression and a tag with value <literal>4 characters</literal> will be set on the span.
|
||
If you want to use some other expression resolution mechanism you can create your own implementation
|
||
of the bean.</simpara>
|
||
</section>
|
||
<section xml:id="_using_tostring_method">
|
||
<title>Using toString method</title>
|
||
<simpara>Having such an annotated method:</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">@NewSpan
|
||
public void getAnnotationForArgumentToString(@SpanTag("test") Long param) {
|
||
}</programlisting>
|
||
<simpara>if executed with a value of <literal>15</literal> will lead to setting of a tag with a String value of <literal>"15"</literal>.</simpara>
|
||
</section>
|
||
</section>
|
||
</chapter>
|
||
<chapter xml:id="_customizations">
|
||
<title>Customizations</title>
|
||
<section xml:id="_spring_integration">
|
||
<title>Spring Integration</title>
|
||
|
||
</section>
|
||
<section xml:id="_http">
|
||
<title>HTTP</title>
|
||
|
||
</section>
|
||
<section xml:id="_tracefilter">
|
||
<title>TraceFilter</title>
|
||
<simpara>You can also modify the behaviour of the <literal>TraceFilter</literal> - the component that is responsible
|
||
for processing the input HTTP request and adding tags basing on the HTTP response. You can customize
|
||
the tags, or modify the response headers by registering your own instance of the <literal>TraceFilter</literal> bean.</simpara>
|
||
<simpara>In the following example we will register the <literal>TraceFilter</literal> bean and we will add the
|
||
<literal>ZIPKIN-TRACE-ID</literal> response header containing the current Span’s trace id. Also we will
|
||
add to the Span a tag with key <literal>custom</literal> and a value <literal>tag</literal>.</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">@Component
|
||
@Order(TraceFilter.ORDER + 1)
|
||
class MyFilter extends GenericFilterBean {
|
||
|
||
private final Tracer tracer;
|
||
|
||
MyFilter(Tracer tracer) {
|
||
this.tracer = tracer;
|
||
}
|
||
|
||
@Override public void doFilter(ServletRequest request, ServletResponse response,
|
||
FilterChain chain) throws IOException, ServletException {
|
||
Span currentSpan = this.tracer.currentSpan();
|
||
then(currentSpan).isNotNull();
|
||
// for readability we're returning trace id in a hex form
|
||
((HttpServletResponse) response)
|
||
.addHeader("ZIPKIN-TRACE-ID",
|
||
currentSpan.context().traceIdString());
|
||
// we can also add some custom tags
|
||
currentSpan.tag("custom", "tag");
|
||
chain.doFilter(request, response);
|
||
}
|
||
}</programlisting>
|
||
</section>
|
||
<section xml:id="_custom_service_name">
|
||
<title>Custom service name</title>
|
||
<simpara>By default Sleuth assumes that when you send a span to Zipkin, you want the span’s service name
|
||
to be equal to <literal>spring.application.name</literal> value. That’s not always the case though. There
|
||
are situations in which you want to explicitly provide a different service name for all spans coming
|
||
from your application. To achieve that it’s enough to just pass the following property
|
||
to your application to override that value (example for <literal>foo</literal> service name):</simpara>
|
||
<programlisting language="yaml" linenumbering="unnumbered">spring.zipkin.service.name: foo</programlisting>
|
||
</section>
|
||
<section xml:id="_customization_of_reported_spans">
|
||
<title>Customization of reported spans</title>
|
||
<simpara>Before reporting spans to e.g. Zipkin you can be interested in modifying that span in some way.
|
||
You can achieve that by using the <literal>SpanAdjuster</literal> interface.</simpara>
|
||
<simpara>In Sleuth we’re generating spans with a fixed name. Some users want to modify the name depending on values
|
||
of tags. Implementation of the <literal>SpanAdjuster</literal> interface can be used to alter that name. Example:</simpara>
|
||
<simpara>Example. If you register two beans of <literal>SpanAdjuster</literal> type:</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">@Bean SpanAdjuster adjusterOne() {
|
||
return span -> span.toBuilder().name("foo").build();
|
||
}
|
||
|
||
@Bean SpanAdjuster adjusterTwo() {
|
||
return span -> span.toBuilder().name(span.name() + " bar").build();
|
||
}</programlisting>
|
||
<simpara>This will lead in changing the name of the reported span to <literal>foo bar</literal>, just before it gets reported (e.g. to Zipkin).</simpara>
|
||
</section>
|
||
<section xml:id="_host_locator">
|
||
<title>Host locator</title>
|
||
<important>
|
||
<simpara>This section is about defining <emphasis role="strong">host</emphasis> from service discovery. It’s <emphasis role="strong">NOT</emphasis>
|
||
about finding Zipkin in service discovery.</simpara>
|
||
</important>
|
||
<simpara>In order to define the host that is corresponding to a particular span we need to resolve the host name
|
||
and port. The default approach is to take it from server properties. If those for some reason are not set
|
||
then we’re trying to retrieve the host name from the network interfaces.</simpara>
|
||
<simpara>If you have the discovery client enabled and prefer to retrieve the host address from the registered
|
||
instance in a service registry then you have to set the property (it’s applicable for both HTTP and
|
||
Stream based span reporting).</simpara>
|
||
<programlisting language="yaml" linenumbering="unnumbered">spring.zipkin.locator.discovery.enabled: true</programlisting>
|
||
</section>
|
||
</chapter>
|
||
<chapter xml:id="_sending_spans_to_zipkin">
|
||
<title>Sending spans to Zipkin</title>
|
||
<simpara>By default if you add <literal>spring-cloud-starter-zipkin</literal> as a dependency to your project,
|
||
when the span is closed, it will be sent to Zipkin over HTTP. The communication
|
||
is asynchronous. You can configure the URL by setting the <literal>spring.zipkin.baseUrl</literal>
|
||
property as follows:</simpara>
|
||
<programlisting language="yaml" linenumbering="unnumbered">spring.zipkin.baseUrl: http://192.168.99.100:9411/</programlisting>
|
||
<simpara>If you want to find Zipkin via service discovery it’s enough to pass the
|
||
Zipkin’s service id inside the URL (example for <literal>zipkinserver</literal> service id)</simpara>
|
||
<programlisting language="yaml" linenumbering="unnumbered">spring.zipkin.baseUrl: http://zipkinserver/</programlisting>
|
||
</chapter>
|
||
<chapter xml:id="_zipkin_stream_span_consumer">
|
||
<title>Zipkin Stream Span Consumer</title>
|
||
<important>
|
||
<simpara>The suggested approach is to use the Zipkin’s
|
||
native support for message based span sending. Starting from
|
||
Edgware Zipkin Stream server is deprecated and in Finchley
|
||
it got removed.</simpara>
|
||
</important>
|
||
<simpara>Please refer to the <link xl:href="http://cloud.spring.io/spring-cloud-static/Dalston.SR4/multi/multi__span_data_as_messages.html#_zipkin_consumer">Dalston Documentaion</link>
|
||
on how to create a Stream Zipkin server.</simpara>
|
||
</chapter>
|
||
<chapter xml:id="_integrations">
|
||
<title>Integrations</title>
|
||
<section xml:id="_opentracing">
|
||
<title>OpenTracing</title>
|
||
<simpara>Spring Cloud Sleuth is <link xl:href="http://opentracing.io/">OpenTracing</link> compatible. If you have
|
||
OpenTracing on the classpath we will automatically register the OpenTracing
|
||
<literal>Tracer</literal> bean. If you wish to disable this just set <literal>spring.sleuth.opentracing.enabled</literal> to <literal>false</literal></simpara>
|
||
</section>
|
||
<section xml:id="_runnable_and_callable">
|
||
<title>Runnable and Callable</title>
|
||
<simpara>If you’re wrapping your logic in <literal>Runnable</literal> or <literal>Callable</literal> it’s enough to wrap those classes in their Sleuth representative.</simpara>
|
||
<simpara>Example for <literal>Runnable</literal>:</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">Runnable runnable = new Runnable() {
|
||
@Override
|
||
public void run() {
|
||
// do some work
|
||
}
|
||
|
||
@Override
|
||
public String toString() {
|
||
return "spanNameFromToStringMethod";
|
||
}
|
||
};
|
||
// Manual `TraceRunnable` creation with explicit "calculateTax" Span name
|
||
Runnable traceRunnable = new TraceRunnable(tracer, spanNamer, errorParser,
|
||
runnable, "calculateTax");
|
||
// Wrapping `Runnable` with `Tracing`. That way the current span will be available
|
||
// in the thread of `Runnable`
|
||
Runnable traceRunnableFromTracer = tracing.currentTraceContext().wrap(runnable);</programlisting>
|
||
<simpara>Example for <literal>Callable</literal>:</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">Callable<String> callable = new Callable<String>() {
|
||
@Override
|
||
public String call() throws Exception {
|
||
return someLogic();
|
||
}
|
||
|
||
@Override
|
||
public String toString() {
|
||
return "spanNameFromToStringMethod";
|
||
}
|
||
};
|
||
// Manual `TraceCallable` creation with explicit "calculateTax" Span name
|
||
Callable<String> traceCallable = new TraceCallable<>(tracer, spanNamer, errorParser,
|
||
callable, "calculateTax");
|
||
// Wrapping `Callable` with `Tracing`. That way the current span will be available
|
||
// in the thread of `Callable`
|
||
Callable<String> traceCallableFromTracer = tracing.currentTraceContext().wrap(callable);</programlisting>
|
||
<simpara>That way you will ensure that a new Span is created and closed for each execution.</simpara>
|
||
</section>
|
||
<section xml:id="_hystrix">
|
||
<title>Hystrix</title>
|
||
<section xml:id="_custom_concurrency_strategy">
|
||
<title>Custom Concurrency Strategy</title>
|
||
<simpara>We’re registering a custom <link xl:href="https://github.com/Netflix/Hystrix/wiki/Plugins#concurrencystrategy"><literal>HystrixConcurrencyStrategy</literal></link>
|
||
that wraps all <literal>Callable</literal> instances into their Sleuth representative -
|
||
the <literal>TraceCallable</literal>. The strategy either starts or continues a span depending on the fact whether tracing was already going
|
||
on before the Hystrix command was called. To disable the custom Hystrix Concurrency Strategy set the <literal>spring.sleuth.hystrix.strategy.enabled</literal> to <literal>false</literal>.</simpara>
|
||
</section>
|
||
<section xml:id="_manual_command_setting">
|
||
<title>Manual Command setting</title>
|
||
<simpara>Assuming that you have the following <literal>HystrixCommand</literal>:</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">HystrixCommand<String> hystrixCommand = new HystrixCommand<String>(setter) {
|
||
@Override
|
||
protected String run() throws Exception {
|
||
return someLogic();
|
||
}
|
||
};</programlisting>
|
||
<simpara>In order to pass the tracing information you have to wrap the same logic in the Sleuth version of the <literal>HystrixCommand</literal> which is the
|
||
<literal>TraceCommand</literal>:</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">TraceCommand<String> traceCommand = new TraceCommand<String>(tracer, traceKeys, setter) {
|
||
@Override
|
||
public String doRun() throws Exception {
|
||
return someLogic();
|
||
}
|
||
};</programlisting>
|
||
</section>
|
||
</section>
|
||
<section xml:id="_rxjava">
|
||
<title>RxJava</title>
|
||
<simpara>We’re registering a custom <link xl:href="https://github.com/ReactiveX/RxJava/wiki/Plugins#rxjavaschedulershook"><literal>RxJavaSchedulersHook</literal></link>
|
||
that wraps all <literal>Action0</literal> instances into their Sleuth representative -
|
||
the <literal>TraceAction</literal>. The hook either starts or continues a span depending on the fact whether tracing was already going
|
||
on before the Action was scheduled. To disable the custom RxJavaSchedulersHook set the <literal>spring.sleuth.rxjava.schedulers.hook.enabled</literal> to <literal>false</literal>.</simpara>
|
||
<simpara>You can define a list of regular expressions for thread names, for which you don’t want a Span to be created. Just provide a comma separated list
|
||
of regular expressions in the <literal>spring.sleuth.rxjava.schedulers.ignoredthreads</literal> property.</simpara>
|
||
</section>
|
||
<section xml:id="_http_integration">
|
||
<title>HTTP integration</title>
|
||
<simpara>Features from this section can be disabled by providing the <literal>spring.sleuth.web.enabled</literal> property with value equal to <literal>false</literal>.</simpara>
|
||
<section xml:id="_http_filter">
|
||
<title>HTTP Filter</title>
|
||
<simpara>Via the <literal>TraceFilter</literal> all sampled incoming requests result in creation of a Span. That Span’s name is <literal>http:</literal> + the path to which
|
||
the request was sent. E.g. if the request was sent to <literal>/foo/bar</literal> then the name will be <literal>http:/foo/bar</literal>. You can configure which URIs you would
|
||
like to skip via the <literal>spring.sleuth.web.skipPattern</literal> property. If you have <literal>ManagementServerProperties</literal> on classpath then
|
||
its value of <literal>contextPath</literal> gets appended to the provided skip pattern.</simpara>
|
||
</section>
|
||
<section xml:id="_handlerinterceptor">
|
||
<title>HandlerInterceptor</title>
|
||
<simpara>Since we want the span names to be precise we’re using a <literal>TraceHandlerInterceptor</literal> that either wraps an
|
||
existing <literal>HandlerInterceptor</literal> or is added directly to the list of existing <literal>HandlerInterceptors</literal>. The
|
||
<literal>TraceHandlerInterceptor</literal> adds a special request attribute to the given <literal>HttpServletRequest</literal>. If the
|
||
the <literal>TraceFilter</literal> doesn’t see this attribute set it will create a "fallback" span which is an additional
|
||
span created on the server side so that the trace is presented properly in the UI. Seeing that most likely
|
||
signifies that there is a missing instrumentation. In that case please file an issue in Spring Cloud Sleuth.</simpara>
|
||
</section>
|
||
<section xml:id="_async_servlet_support">
|
||
<title>Async Servlet support</title>
|
||
<simpara>If your controller returns a <literal>Callable</literal> or a <literal>WebAsyncTask</literal> Spring Cloud Sleuth will continue the existing span instead of creating a new one.</simpara>
|
||
</section>
|
||
<section xml:id="_webflux_support">
|
||
<title>WebFlux support</title>
|
||
<simpara>Via the <literal>TraceWebFilter</literal> all sampled incoming requests result in creation of a Span. That Span’s name is <literal>http:</literal> + the path to which
|
||
the request was sent. E.g. if the request was sent to <literal>/foo/bar</literal> then the name will be <literal>http:/foo/bar</literal>. You can configure which URIs you would
|
||
like to skip via the <literal>spring.sleuth.web.skipPattern</literal> property. If you have <literal>ManagementServerProperties</literal> on classpath then
|
||
its value of <literal>contextPath</literal> gets appended to the provided skip pattern.</simpara>
|
||
</section>
|
||
</section>
|
||
<section xml:id="_http_client_integration">
|
||
<title>HTTP client integration</title>
|
||
<section xml:id="_synchronous_rest_template">
|
||
<title>Synchronous Rest Template</title>
|
||
<simpara>We’re injecting a <literal>RestTemplate</literal> interceptor that ensures that all the tracing information is passed to the requests. Each time a
|
||
call is made a new Span is created. It gets closed upon receiving the response. In order to block the synchronous <literal>RestTemplate</literal> features
|
||
just set <literal>spring.sleuth.web.client.enabled</literal> to <literal>false</literal>.</simpara>
|
||
<important>
|
||
<simpara>You have to register <literal>RestTemplate</literal> as a bean so that the interceptors will get injected.
|
||
If you create a <literal>RestTemplate</literal> instance with a <literal>new</literal> keyword then the instrumentation WILL NOT work.</simpara>
|
||
</important>
|
||
</section>
|
||
<section xml:id="_asynchronous_rest_template">
|
||
<title>Asynchronous Rest Template</title>
|
||
<important>
|
||
<simpara>Starting with Sleuth <literal>2.0.0</literal> we no longer register
|
||
a bean of <literal>AsyncRestTemplate</literal> type. It’s up to you to create such
|
||
a bean. Then we will instrument it.</simpara>
|
||
</important>
|
||
<simpara>To block the <literal>AsyncRestTemplate</literal> features set <literal>spring.sleuth.web.async.client.enabled</literal> to <literal>false</literal>.
|
||
To disable creation of the default <literal>TraceAsyncClientHttpRequestFactoryWrapper</literal> set <literal>spring.sleuth.web.async.client.factory.enabled</literal>
|
||
to <literal>false</literal>. If you don’t want to create <literal>AsyncRestClient</literal> at all set <literal>spring.sleuth.web.async.client.template.enabled</literal> to <literal>false</literal>.</simpara>
|
||
<section xml:id="_multiple_asynchronous_rest_templates">
|
||
<title>Multiple Asynchronous Rest Templates</title>
|
||
<simpara>Sometimes you need to use multiple implementations of Asynchronous Rest Template. In the following snippet you
|
||
can see an example of how to set up such a custom <literal>AsyncRestTemplate</literal>.</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">@Configuration
|
||
@EnableAutoConfiguration
|
||
static class Config {
|
||
|
||
@Bean(name = "customAsyncRestTemplate")
|
||
public AsyncRestTemplate traceAsyncRestTemplate() {
|
||
return new AsyncRestTemplate(asyncClientFactory(), clientHttpRequestFactory());
|
||
}
|
||
|
||
private ClientHttpRequestFactory clientHttpRequestFactory() {
|
||
ClientHttpRequestFactory clientHttpRequestFactory = new CustomClientHttpRequestFactory();
|
||
//CUSTOMIZE HERE
|
||
return clientHttpRequestFactory;
|
||
}
|
||
|
||
private AsyncClientHttpRequestFactory asyncClientFactory() {
|
||
AsyncClientHttpRequestFactory factory = new CustomAsyncClientHttpRequestFactory();
|
||
//CUSTOMIZE HERE
|
||
return factory;
|
||
}
|
||
}</programlisting>
|
||
</section>
|
||
</section>
|
||
<section xml:id="_webclient">
|
||
<title>WebClient</title>
|
||
<simpara>We inject a <literal>ExchangeFilterFunction</literal> implementation that creates a span and via on success and on
|
||
error callbacks takes care of closing client side spans.</simpara>
|
||
<important>
|
||
<simpara>You have to register <literal>WebClient</literal> as a bean so that the tracing instrumention gets applied.
|
||
If you create a <literal>WebClient</literal> instance with a <literal>new</literal> keyword then the instrumentation WILL NOT work.</simpara>
|
||
</important>
|
||
</section>
|
||
<section xml:id="_traverson">
|
||
<title>Traverson</title>
|
||
<simpara>If you’re using the <link xl:href="http://docs.spring.io/spring-hateoas/docs/current/reference/html/#client.traverson">Traverson</link> library
|
||
it’s enough for you to inject a <literal>RestTemplate</literal> as a bean into your Traverson object. Since <literal>RestTemplate</literal>
|
||
is already intercepted, you will get full support of tracing in your client. Below you can find a pseudo code
|
||
of how to do that:</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">@Autowired RestTemplate restTemplate;
|
||
|
||
Traverson traverson = new Traverson(URI.create("http://some/address"),
|
||
MediaType.APPLICATION_JSON, MediaType.APPLICATION_JSON_UTF8).setRestOperations(restTemplate);
|
||
// use Traverson</programlisting>
|
||
</section>
|
||
</section>
|
||
<section xml:id="_feign">
|
||
<title>Feign</title>
|
||
<simpara>By default Spring Cloud Sleuth provides integration with feign via the <literal>TraceFeignClientAutoConfiguration</literal>. You can disable it entirely
|
||
by setting <literal>spring.sleuth.feign.enabled</literal> to false. If you do so then no Feign related instrumentation will take place.</simpara>
|
||
<simpara>Part of Feign instrumentation is done via a <literal>FeignBeanPostProcessor</literal>. You can disable it by providing the <literal>spring.sleuth.feign.processor.enabled</literal> equal to <literal>false</literal>.
|
||
If you set it like this then Spring Cloud Sleuth will not instrument any of your custom Feign components. All the default instrumentation
|
||
however will be still there.</simpara>
|
||
</section>
|
||
<section xml:id="_asynchronous_communication">
|
||
<title>Asynchronous communication</title>
|
||
<section xml:id="__async_annotated_methods">
|
||
<title>@Async annotated methods</title>
|
||
<simpara>In Spring Cloud Sleuth we’re instrumenting async related components so that the tracing information is passed between threads.
|
||
You can disable this behaviour by setting the value of <literal>spring.sleuth.async.enabled</literal> to <literal>false</literal>.</simpara>
|
||
<simpara>If you annotate your method with <literal>@Async</literal> then we’ll automatically create a new Span with the following characteristics:</simpara>
|
||
<itemizedlist>
|
||
<listitem>
|
||
<simpara>if the method is annotated with <literal>@SpanName</literal> then the value of the annotation will be the Span’s name</simpara>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara>if the method is <emphasis role="strong">not</emphasis> annotated with <literal>@SpanName</literal> the Span name will be the annotated method name</simpara>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara>the Span will be tagged with that method’s class name and the method name too</simpara>
|
||
</listitem>
|
||
</itemizedlist>
|
||
</section>
|
||
<section xml:id="__scheduled_annotated_methods">
|
||
<title>@Scheduled annotated methods</title>
|
||
<simpara>In Spring Cloud Sleuth we’re instrumenting scheduled method execution so that the tracing information is passed between threads. You can disable this behaviour
|
||
by setting the value of <literal>spring.sleuth.scheduled.enabled</literal> to <literal>false</literal>.</simpara>
|
||
<simpara>If you annotate your method with <literal>@Scheduled</literal> then we’ll automatically create a new Span with the following characteristics:</simpara>
|
||
<itemizedlist>
|
||
<listitem>
|
||
<simpara>the Span name will be the annotated method name</simpara>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara>the Span will be tagged with that method’s class name and the method name too</simpara>
|
||
</listitem>
|
||
</itemizedlist>
|
||
<simpara>If you want to skip Span creation for some <literal>@Scheduled</literal> annotated classes you can set the
|
||
<literal>spring.sleuth.scheduled.skipPattern</literal> with a regular expression that will match the fully qualified name of the
|
||
<literal>@Scheduled</literal> annotated class.</simpara>
|
||
<tip>
|
||
<simpara>If you are using <literal>spring-cloud-sleuth-stream</literal> and <literal>spring-cloud-netflix-hystrix-stream</literal> together, Span will be created for each Hystrix metrics and sent to Zipkin. This may be annoying. You can prevent this by setting <literal>spring.sleuth.scheduled.skipPattern=org.springframework.cloud.netflix.hystrix.stream.HystrixStreamTask</literal></simpara>
|
||
</tip>
|
||
</section>
|
||
<section xml:id="_executor_executorservice_and_scheduledexecutorservice">
|
||
<title>Executor, ExecutorService and ScheduledExecutorService</title>
|
||
<simpara>We’re providing <literal>LazyTraceExecutor</literal>, <literal>TraceableExecutorService</literal> and <literal>TraceableScheduledExecutorService</literal>. Those implementations
|
||
are creating Spans each time a new task is submitted, invoked or scheduled.</simpara>
|
||
<simpara>Here you can see an example of how to pass tracing information with <literal>TraceableExecutorService</literal> when working with <literal>CompletableFuture</literal>:</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">CompletableFuture<Long> completableFuture = CompletableFuture.supplyAsync(() -> {
|
||
// perform some logic
|
||
return 1_000_000L;
|
||
}, new TraceableExecutorService(beanFactory, executorService,
|
||
// 'calculateTax' explicitly names the span - this param is optional
|
||
"calculateTax"));</programlisting>
|
||
<important>
|
||
<simpara>Sleuth doesn’t work with <literal>parallelStream()</literal> out of the box. If you want
|
||
to have the tracing information propagated through the stream you have to use the
|
||
approach with <literal>supplyAsync(...)</literal> as presented above.</simpara>
|
||
</important>
|
||
<section xml:id="_customization_of_executors">
|
||
<title>Customization of Executors</title>
|
||
<simpara>Sometimes you need to set up a custom instance of the <literal>AsyncExecutor</literal>. In the following snippet you
|
||
can see an example of how to set up such a custom <literal>Executor</literal>.</simpara>
|
||
<programlisting language="java" linenumbering="unnumbered">@Configuration
|
||
@EnableAutoConfiguration
|
||
@EnableAsync
|
||
static class CustomExecutorConfig extends AsyncConfigurerSupport {
|
||
|
||
@Autowired BeanFactory beanFactory;
|
||
|
||
@Override public Executor getAsyncExecutor() {
|
||
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
|
||
// CUSTOMIZE HERE
|
||
executor.setCorePoolSize(7);
|
||
executor.setMaxPoolSize(42);
|
||
executor.setQueueCapacity(11);
|
||
executor.setThreadNamePrefix("MyExecutor-");
|
||
// DON'T FORGET TO INITIALIZE
|
||
executor.initialize();
|
||
return new LazyTraceExecutor(this.beanFactory, executor);
|
||
}
|
||
}</programlisting>
|
||
</section>
|
||
</section>
|
||
</section>
|
||
<section xml:id="_messaging">
|
||
<title>Messaging</title>
|
||
<simpara>Spring Cloud Sleuth integrates with <link xl:href="http://projects.spring.io/spring-integration/">Spring Integration</link>. It creates spans for publish and
|
||
subscribe events. To disable Spring Integration instrumentation, set <literal>spring.sleuth.integration.enabled</literal> to false.</simpara>
|
||
<simpara>You can provide the <literal>spring.sleuth.integration.patterns</literal> pattern to explicitly
|
||
provide the names of channels that you want to include for tracing. By default all channels
|
||
are included.</simpara>
|
||
<important>
|
||
<simpara>When using the <literal>Executor</literal> to build a Spring Integration <literal>IntegrationFlow</literal> remember to use the <emphasis role="strong">untraced</emphasis> version of the <literal>Executor</literal>.
|
||
Decorating Spring Integration Executor Channel with <literal>TraceableExecutorService</literal> will cause the spans to be improperly closed.</simpara>
|
||
</important>
|
||
</section>
|
||
<section xml:id="_zuul">
|
||
<title>Zuul</title>
|
||
<simpara>We’re registering Zuul filters to propagate the tracing information (the request header is enriched with tracing data).
|
||
To disable Zuul support set the <literal>spring.sleuth.zuul.enabled</literal> property to <literal>false</literal>.</simpara>
|
||
</section>
|
||
</chapter>
|
||
<chapter xml:id="_running_examples">
|
||
<title>Running examples</title>
|
||
<simpara>You can find the running examples deployed in the <link xl:href="https://run.pivotal.io/">Pivotal Web Services</link>. Check them out in the following links:</simpara>
|
||
<itemizedlist>
|
||
<listitem>
|
||
<simpara><link xl:href="http://docssleuth-zipkin-server.cfapps.io/">Zipkin for apps presented in the samples to the top</link></simpara>
|
||
</listitem>
|
||
<listitem>
|
||
<simpara><link xl:href="http://docsbrewing-zipkin-server.cfapps.io/">Zipkin for Brewery on PWS</link>, its <link xl:href="https://github.com/spring-cloud-samples/brewery">Github Code</link></simpara>
|
||
</listitem>
|
||
</itemizedlist>
|
||
</chapter>
|
||
</book> |