154 lines
23 KiB
HTML
154 lines
23 KiB
HTML
<html><head>
|
|
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
|
|
<title>54. Customizations</title><link rel="stylesheet" type="text/css" href="css/manual-multipage.css"><meta name="generator" content="DocBook XSL Stylesheets V1.78.1"><link rel="home" href="multi_spring-cloud.html" title="Spring Cloud"><link rel="up" href="multi__spring_cloud_sleuth.html" title="Part VII. Spring Cloud Sleuth"><link rel="prev" href="multi__managing_spans_with_annotations.html" title="53. Managing spans with annotations"><link rel="next" href="multi__sending_spans_to_zipkin.html" title="55. Sending spans to Zipkin"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">54. Customizations</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="multi__managing_spans_with_annotations.html">Prev</a> </td><th width="60%" align="center">Part VII. Spring Cloud Sleuth</th><td width="20%" align="right"> <a accesskey="n" href="multi__sending_spans_to_zipkin.html">Next</a></td></tr></table><hr></div><div class="chapter"><div class="titlepage"><div><div><h2 class="title"><a name="_customizations" href="#_customizations"></a>54. Customizations</h2></div></div></div><p>Thanks to the <code class="literal">SpanInjector</code> and <code class="literal">SpanExtractor</code> you can customize the way spans
|
|
are created and propagated.</p><p>There are currently two built-in ways to pass tracing information between processes:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem">via Spring Integration</li><li class="listitem">via HTTP</li></ul></div><p>Span ids are extracted from Zipkin-compatible (B3) headers (either <code class="literal">Message</code>
|
|
or HTTP headers), to start or join an existing trace. Trace information is
|
|
injected into any outbound requests so the next hop can extract them.</p><p>The default way of coding tracing context is done via the <code class="literal">b3</code> header that contains the
|
|
<code class="literal">traceId-spanId-sampled</code> notation (e.g. <code class="literal">0000000000000005-0000000000000004-1</code>).
|
|
For backward compatibility, if the <code class="literal">b3</code> header is not present, we also check if
|
|
<code class="literal">X-B3</code> entries are present, and retrieve tracing context from there e.g.
|
|
(<code class="literal">X-B3-TraceId: 0000000000000005</code>, <code class="literal">X-B3-SpanId: 0000000000000004</code>, <code class="literal">X-B3-Sampled: 1</code>).</p><p>The key change in comparison to the previous versions of Sleuth is that Sleuth is implementing
|
|
the Open Tracing’s <code class="literal">TextMap</code> notion. In Sleuth it’s called <code class="literal">SpanTextMap</code>. Basically the idea
|
|
is that any means of communication (e.g. message, http request, etc.) can be abstracted via
|
|
a <code class="literal">SpanTextMap</code>. This abstraction defines how one can insert data into the carrier and
|
|
how to retrieve it from there. Thanks to this if you want to instrument a new HTTP library
|
|
that uses a <code class="literal">FooRequest</code> as a mean of sending HTTP requests then you have to create an
|
|
implementation of a <code class="literal">SpanTextMap</code> that delegates calls to <code class="literal">FooRequest</code> in terms of retrieval
|
|
and insertion of HTTP headers.</p><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="_spring_integration" href="#_spring_integration"></a>54.1 Spring Integration</h2></div></div></div><p>For Spring Integration there are 2 interfaces responsible for creation of a Span from a <code class="literal">Message</code>.
|
|
These are:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><code class="literal">MessagingSpanTextMapExtractor</code></li><li class="listitem"><code class="literal">MessagingSpanTextMapInjector</code></li></ul></div><p>You can override them by providing your own implementation.</p></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="_http" href="#_http"></a>54.2 HTTP</h2></div></div></div><p>For HTTP there are 2 interfaces responsible for creation of a Span from a <code class="literal">Message</code>.
|
|
These are:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><code class="literal">HttpSpanExtractor</code></li><li class="listitem"><code class="literal">HttpSpanInjector</code></li></ul></div><p>You can override them by providing your own implementation.</p></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="_example" href="#_example"></a>54.3 Example</h2></div></div></div><p>Let’s assume that instead of the standard Zipkin compatible tracing HTTP header names
|
|
you have</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem">for trace id - <code class="literal">correlationId</code></li><li class="listitem">for span id - <code class="literal">mySpanId</code></li></ul></div><p>This is a an example of a <code class="literal">SpanExtractor</code></p><pre class="programlisting"><span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">static</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> CustomHttpSpanExtractor <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">implements</span> HttpSpanExtractor {
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Override</span></em> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> Span joinTrace(SpanTextMap carrier) {
|
|
Map<String, String> map = TextMapUtil.asMap(carrier);
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">long</span> traceId = Span.hexToId(map.get(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"correlationid"</span>));
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">long</span> spanId = Span.hexToId(map.get(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"myspanid"</span>));
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-comment">// extract all necessary headers</span>
|
|
Span.SpanBuilder builder = Span.builder().traceId(traceId).spanId(spanId);
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-comment">// build rest of the Span</span>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> builder.build();
|
|
}
|
|
}
|
|
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">static</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> CustomHttpSpanInjector <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">implements</span> HttpSpanInjector {
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Override</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">void</span> inject(Span span, SpanTextMap carrier) {
|
|
carrier.put(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"correlationId"</span>, span.traceIdString());
|
|
carrier.put(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"mySpanId"</span>, Span.idToHex(span.getSpanId()));
|
|
}
|
|
}</pre><p>And you could register it like this:</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@Bean</span></em>
|
|
HttpSpanInjector customHttpSpanInjector() {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">new</span> CustomHttpSpanInjector();
|
|
}
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Bean</span></em>
|
|
HttpSpanExtractor customHttpSpanExtractor() {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">new</span> CustomHttpSpanExtractor();
|
|
}</pre><p>Spring Cloud Sleuth does not add trace/span related headers to the Http Response for security reasons. If you need the headers then a custom <code class="literal">SpanInjector</code>
|
|
that injects the headers into the Http Response and a Servlet filter which makes use of this can be added the following way:</p><pre class="programlisting"><span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">static</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> CustomHttpServletResponseSpanInjector <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">extends</span> ZipkinHttpSpanInjector {
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Override</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">void</span> inject(Span span, SpanTextMap carrier) {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">super</span>.inject(span, carrier);
|
|
carrier.put(Span.TRACE_ID_NAME, span.traceIdString());
|
|
carrier.put(Span.SPAN_ID_NAME, Span.idToHex(span.getSpanId()));
|
|
}
|
|
}
|
|
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">static</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> HttpResponseInjectingTraceFilter <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">extends</span> GenericFilterBean {
|
|
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">private</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">final</span> Tracer tracer;
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">private</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">final</span> HttpSpanInjector spanInjector;
|
|
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> HttpResponseInjectingTraceFilter(Tracer tracer, HttpSpanInjector spanInjector) {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">this</span>.tracer = tracer;
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">this</span>.spanInjector = spanInjector;
|
|
}
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Override</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">void</span> doFilter(ServletRequest request, ServletResponse servletResponse, FilterChain filterChain) <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">throws</span> IOException, ServletException {
|
|
HttpServletResponse response = (HttpServletResponse) servletResponse;
|
|
Span currentSpan = <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">this</span>.tracer.getCurrentSpan();
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">this</span>.spanInjector.inject(currentSpan, <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">new</span> HttpServletResponseTextMap(response));
|
|
filterChain.doFilter(request, response);
|
|
}
|
|
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> HttpServletResponseTextMap <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">implements</span> SpanTextMap {
|
|
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">private</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">final</span> HttpServletResponse delegate;
|
|
|
|
HttpServletResponseTextMap(HttpServletResponse delegate) {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">this</span>.delegate = delegate;
|
|
}
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Override</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> Iterator<Map.Entry<String, String>> iterator() {
|
|
Map<String, String> map = <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">new</span> HashMap<>();
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">for</span> (String header : <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">this</span>.delegate.getHeaderNames()) {
|
|
map.put(header, <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">this</span>.delegate.getHeader(header));
|
|
}
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> map.entrySet().iterator();
|
|
}
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Override</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">void</span> put(String key, String value) {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">this</span>.delegate.addHeader(key, value);
|
|
}
|
|
}
|
|
}</pre><p>And you could register them like this:</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@Bean</span></em> HttpSpanInjector customHttpServletResponseSpanInjector() {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">new</span> CustomHttpServletResponseSpanInjector();
|
|
}
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Bean</span></em>
|
|
HttpResponseInjectingTraceFilter responseInjectingTraceFilter(Tracer tracer) {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">new</span> HttpResponseInjectingTraceFilter(tracer, customHttpServletResponseSpanInjector());
|
|
}</pre></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="_tracefilter" href="#_tracefilter"></a>54.4 TraceFilter</h2></div></div></div><p>You can also modify the behaviour of the <code class="literal">TraceFilter</code> - the component that is responsible
|
|
for processing the input HTTP request and adding tags basing on the HTTP response. You can customize
|
|
the tags, or modify the response headers by registering your own instance of the <code class="literal">TraceFilter</code> bean.</p><p>In the following example we will register the <code class="literal">TraceFilter</code> bean and we will add the
|
|
<code class="literal">ZIPKIN-TRACE-ID</code> response header containing the current Span’s trace id. Also we will
|
|
add to the Span a tag with key <code class="literal">custom</code> and a value <code class="literal">tag</code>.</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@Bean</span></em>
|
|
TraceFilter myTraceFilter(BeanFactory beanFactory, <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">final</span> Tracer tracer) {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">new</span> TraceFilter(beanFactory) {
|
|
<em><span class="hl-annotation" style="color: gray">@Override</span></em> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">protected</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">void</span> addResponseTags(HttpServletResponse response,
|
|
Throwable e) {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-comment">// execute the default behaviour</span>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">super</span>.addResponseTags(response, e);
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-comment">// for readability we're returning trace id in a hex form</span>
|
|
response.addHeader(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"ZIPKIN-TRACE-ID"</span>,
|
|
Span.idToHex(tracer.getCurrentSpan().getTraceId()));
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-comment">// we can also add some custom tags</span>
|
|
tracer.addTag(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"custom"</span>, <span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"tag"</span>);
|
|
}
|
|
};
|
|
}</pre><p>To change the order of <code class="literal">TraceFilter</code> registration, please set the
|
|
<code class="literal">spring.sleuth.web.filter-order</code> property.</p></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="_custom_sa_tag_in_zipkin" href="#_custom_sa_tag_in_zipkin"></a>54.5 Custom SA tag in Zipkin</h2></div></div></div><p>Sometimes you want to create a manual Span that will wrap a call to an external service which is not instrumented.
|
|
What you can do is to create a span with the <code class="literal">peer.service</code> tag that will contain a value of the service that you want to call.
|
|
Below you can see an example of a call to Redis that is wrapped in such a span.</p><pre class="programlisting">org.springframework.cloud.sleuth.Span newSpan = tracer.createSpan(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"redis"</span>);
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">try</span> {
|
|
newSpan.tag(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"redis.op"</span>, <span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"get"</span>);
|
|
newSpan.tag(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"lc"</span>, <span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"redis"</span>);
|
|
newSpan.logEvent(org.springframework.cloud.sleuth.Span.CLIENT_SEND);
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-comment">// call redis service e.g</span>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-comment">// return (SomeObj) redisTemplate.opsForHash().get("MYHASH", someObjKey);</span>
|
|
} <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">finally</span> {
|
|
newSpan.tag(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"peer.service"</span>, <span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"redisService"</span>);
|
|
newSpan.tag(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"peer.ipv4"</span>, <span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"1.2.3.4"</span>);
|
|
newSpan.tag(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"peer.port"</span>, <span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"1234"</span>);
|
|
newSpan.logEvent(org.springframework.cloud.sleuth.Span.CLIENT_RECV);
|
|
tracer.close(newSpan);
|
|
}</pre><div class="important" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Important"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Important]" src="images/important.png"></td><th align="left">Important</th></tr><tr><td align="left" valign="top"><p>Remember not to add both <code class="literal">peer.service</code> tag and the <code class="literal">SA</code> tag! You have to add only <code class="literal">peer.service</code>.</p></td></tr></table></div></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="_custom_service_name" href="#_custom_service_name"></a>54.6 Custom service name</h2></div></div></div><p>By default Sleuth assumes that when you send a span to Zipkin, you want the span’s service name
|
|
to be equal to <code class="literal">spring.application.name</code> value. That’s not always the case though. There
|
|
are situations in which you want to explicitly provide a different service name for all spans coming
|
|
from your application. To achieve that it’s enough to just pass the following property
|
|
to your application to override that value (example for <code class="literal">foo</code> service name):</p><pre class="programlisting"><span xmlns:d="http://docbook.org/ns/docbook" class="hl-attribute">spring.zipkin.service.name</span>: foo</pre></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="_customization_of_reported_spans" href="#_customization_of_reported_spans"></a>54.7 Customization of reported spans</h2></div></div></div><p>Before reporting spans to e.g. Zipkin you can be interested in modifying that span in some way.
|
|
You can achieve that by using the <code class="literal">SpanAdjuster</code> interface.</p><p>Example of usage:</p><p>In Sleuth we’re generating spans with a fixed name. Some users want to modify the name depending on values
|
|
of tags. Implementation of the <code class="literal">SpanAdjuster</code> interface can be used to alter that name. Example:</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@Bean</span></em>
|
|
SpanAdjuster customSpanAdjuster() <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">{</span>
|
|
return span -> span.toBuilder().name(scrub(span.getName())).build();
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">}</span></pre><p>This will lead in changing the name of the reported span just before it gets sent to Zipkin.</p><div class="important" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Important"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Important]" src="images/important.png"></td><th align="left">Important</th></tr><tr><td align="left" valign="top"><p>Your <code class="literal">SpanReporter</code> should inject the <code class="literal">SpanAdjuster</code> and
|
|
allow span manipulation before the actual reporting is done.</p></td></tr></table></div></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="_host_locator" href="#_host_locator"></a>54.8 Host locator</h2></div></div></div><p>In order to define the host that is corresponding to a particular span we need to resolve the host name
|
|
and port. The default approach is to take it from server properties. If those for some reason are not set
|
|
then we’re trying to retrieve the host name from the network interfaces.</p><p>If you have the discovery client enabled and prefer to retrieve the host address from the registered
|
|
instance in a service registry then you have to set the property (it’s applicable for both HTTP and
|
|
Stream based span reporting).</p><pre class="programlisting"><span xmlns:d="http://docbook.org/ns/docbook" class="hl-attribute">spring.zipkin.locator.discovery.enabled</span>: <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">true</span></pre></div></div><div class="navfooter"><hr><table width="100%" summary="Navigation footer"><tr><td width="40%" align="left"><a accesskey="p" href="multi__managing_spans_with_annotations.html">Prev</a> </td><td width="20%" align="center"><a accesskey="u" href="multi__spring_cloud_sleuth.html">Up</a></td><td width="40%" align="right"> <a accesskey="n" href="multi__sending_spans_to_zipkin.html">Next</a></td></tr><tr><td width="40%" align="left" valign="top">53. Managing spans with annotations </td><td width="20%" align="center"><a accesskey="h" href="multi_spring-cloud.html">Home</a></td><td width="40%" align="right" valign="top"> 55. Sending spans to Zipkin</td></tr></table></div></body></html> |