SpringCloud/spring-cloud-sleuth
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Don't talk too much in the feature intro

Browse Source
This commit is contained in:
Adrian Cole
2020-04-15 13:12:45 +08:00
parent 354142e725
commit 07baf87bdf
3 changed files with 71 additions and 218 deletions
Download Patch File Download Diff File Expand all files Collapse all files

137
README.adoc
View File

@@ -152,141 +152,6 @@ frontend.log:2020-04-09 17:45:40.574 ERROR [frontend,5e8eeec48b08e26882aba313eb0
Above, you'll notice the trace ID is `5e8eeec48b08e26882aba313eb08f0a4`, for
example. This log configuration was automatically setup by Sleuth.
If you use a log aggregating tool (such as https://www.elastic.co/products/kibana[Kibana], https://www.splunk.com/[Splunk], and others), you can order the events that took place.
An example from Kibana would resemble the following image:
image::https://raw.githubusercontent.com/spring-cloud/spring-cloud-sleuth/{branch}/docs/src/main/asciidoc/images/kibana.png[Log correlation with Kibana]
If you want to use https://www.elastic.co/guide/en/logstash/current/index.html[Logstash], the following listing shows the Grok pattern for Logstash:
[source]
filter {
# pattern matching logback pattern
grok {
match => { "message" => "%{TIMESTAMP_ISO8601:timestamp}\s+%{LOGLEVEL:severity}\s+\[%{DATA:service},%{DATA:trace},%{DATA:span}\]\s+%{DATA:pid}\s+---\s+\[%{DATA:thread}\]\s+%{DATA:class}\s+:\s+%{GREEDYDATA:rest}" }
}
date {
match => ["timestamp", "ISO8601"]
}
mutate {
remove_field => ["timestamp"]
}
}
NOTE: If you want to use Grok together with the logs from Cloud Foundry, you have to use the following pattern:
[source]
filter {
# pattern matching logback pattern
grok {
match => { "message" => "(?m)OUT\s+%{TIMESTAMP_ISO8601:timestamp}\s+%{LOGLEVEL:severity}\s+\[%{DATA:service},%{DATA:trace},%{DATA:span}\]\s+%{DATA:pid}\s+---\s+\[%{DATA:thread}\]\s+%{DATA:class}\s+:\s+%{GREEDYDATA:rest}" }
}
date {
match => ["timestamp", "ISO8601"]
}
mutate {
remove_field => ["timestamp"]
}
}
==== JSON Logback with Logstash
Often, you do not want to store your logs in a text file but in a JSON file that Logstash can immediately pick.
To do so, you have to do the following (for readability, we pass the dependencies in the `groupId:artifactId:version` notation).
*Dependencies Setup*
. Ensure that Logback is on the classpath (`ch.qos.logback:logback-core`).
. Add Logstash Logback encode. For example, to use version `4.6`, add `net.logstash.logback:logstash-logback-encoder:4.6`.
*Logback Setup*
Consider the following example of a Logback configuration file (logback-spring.xml).
[source,xml]
-----
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<include resource="org/springframework/boot/logging/logback/defaults.xml"/>
<springProperty scope="context" name="springAppName" source="spring.application.name"/>
<!-- Example for logging into the build folder of your project -->
<property name="LOG_FILE" value="${BUILD_FOLDER:-build}/${springAppName}"/>
<!-- You can override this to have a custom pattern -->
<property name="CONSOLE_LOG_PATTERN"
value="%clr(%d{yyyy-MM-dd HH:mm:ss.SSS}){faint} %clr(${LOG_LEVEL_PATTERN:-%5p}) %clr(${PID:- }){magenta} %clr(---){faint} %clr([%15.15t]){faint} %clr(%-40.40logger{39}){cyan} %clr(:){faint} %m%n${LOG_EXCEPTION_CONVERSION_WORD:-%wEx}"/>
<!-- Appender to log to console -->
<appender name="console" class="ch.qos.logback.core.ConsoleAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<!-- Minimum logging level to be presented in the console logs-->
<level>DEBUG</level>
</filter>
<encoder>
<pattern>${CONSOLE_LOG_PATTERN}</pattern>
<charset>utf8</charset>
</encoder>
</appender>
<!-- Appender to log to file -->
<appender name="flatfile" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${LOG_FILE}</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${LOG_FILE}.%d{yyyy-MM-dd}.gz</fileNamePattern>
<maxHistory>7</maxHistory>
</rollingPolicy>
<encoder>
<pattern>${CONSOLE_LOG_PATTERN}</pattern>
<charset>utf8</charset>
</encoder>
</appender>
<!-- Appender to log to file in a JSON format -->
<appender name="logstash" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${LOG_FILE}.json</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${LOG_FILE}.json.%d{yyyy-MM-dd}.gz</fileNamePattern>
<maxHistory>7</maxHistory>
</rollingPolicy>
<encoder class="net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder">
<providers>
<timestamp>
<timeZone>UTC</timeZone>
</timestamp>
<pattern>
<pattern>
{
"timestamp": "@timestamp",
"severity": "%level",
"service": "${springAppName:-}",
"trace": "%X{traceId:-}",
"span": "%X{spanId:-}",
"pid": "${PID:-}",
"thread": "%thread",
"class": "%logger{40}",
"rest": "%message"
}
</pattern>
</pattern>
</providers>
</encoder>
</appender>
<root level="INFO">
<appender-ref ref="console"/>
<!-- uncomment this to have also JSON logs -->
<!--<appender-ref ref="logstash"/>-->
<!--<appender-ref ref="flatfile"/>-->
</root>
</configuration>
-----
That Logback configuration file:
* Logs information from the application in a JSON format to a `build/${spring.application.name}.json` file.
* Has commented out two additional appenders: console and standard log file.
* Has the same logging pattern as the one presented in the previous section.
NOTE: If you use a custom `logback-spring.xml`, you must pass the `spring.application.name` in the `bootstrap` rather than the `application` property file.
Otherwise, your custom logback file does not properly read the property.
=== Service Dependency Graph
When you consider distributed tracing tracks requests, it makes sense that
trace data can paint a picture of your architecture.
@@ -296,7 +161,7 @@ including the count of calls and how many errors exist.
The example application will make a simple diagram like this, but your real
environment diagram may be more complex.
image::https://raw.githubusercontent.com/spring-cloud/spring-cloud-sleuth/{branch}/docs/src/main/asciidoc/images/zipkin-depedendencies.png[Zipkin Dependencies]
image::https://raw.githubusercontent.com/spring-cloud/spring-cloud-sleuth/{branch}/docs/src/main/asciidoc/images/zipkin-dependencies.png[Zipkin Dependencies]
*Note*: Production environments will generate a lot of data. You will likely
need to run a separate service to aggregate the dependency graph. You can learn

64
docs/src/main/asciidoc/features.adoc
View File

@@ -49,70 +49,6 @@ frontend.log:2020-04-09 17:45:40.574 ERROR [frontend,5e8eeec48b08e26882aba313eb0
Above, you'll notice the trace ID is `5e8eeec48b08e26882aba313eb08f0a4`, for
example. This log configuration was automatically setup by Sleuth.
If you use a log aggregating tool (such as https://www.elastic.co/products/kibana[Kibana], https://www.splunk.com/[Splunk], and others), you can order the events that took place.
An example from Kibana would resemble the following image:
image::https://raw.githubusercontent.com/spring-cloud/spring-cloud-sleuth/{branch}/docs/src/main/asciidoc/images/kibana.png[Log correlation with Kibana]
If you want to use https://www.elastic.co/guide/en/logstash/current/index.html[Logstash], the following listing shows the Grok pattern for Logstash:
[source]
filter {
# pattern matching logback pattern
grok {
match => { "message" => "%{TIMESTAMP_ISO8601:timestamp}\s+%{LOGLEVEL:severity}\s+\[%{DATA:service},%{DATA:trace},%{DATA:span}\]\s+%{DATA:pid}\s+---\s+\[%{DATA:thread}\]\s+%{DATA:class}\s+:\s+%{GREEDYDATA:rest}" }
}
date {
match => ["timestamp", "ISO8601"]
}
mutate {
remove_field => ["timestamp"]
}
}
NOTE: If you want to use Grok together with the logs from Cloud Foundry, you have to use the following pattern:
[source]
filter {
# pattern matching logback pattern
grok {
match => { "message" => "(?m)OUT\s+%{TIMESTAMP_ISO8601:timestamp}\s+%{LOGLEVEL:severity}\s+\[%{DATA:service},%{DATA:trace},%{DATA:span}\]\s+%{DATA:pid}\s+---\s+\[%{DATA:thread}\]\s+%{DATA:class}\s+:\s+%{GREEDYDATA:rest}" }
}
date {
match => ["timestamp", "ISO8601"]
}
mutate {
remove_field => ["timestamp"]
}
}
==== JSON Logback with Logstash
Often, you do not want to store your logs in a text file but in a JSON file that Logstash can immediately pick.
To do so, you have to do the following (for readability, we pass the dependencies in the `groupId:artifactId:version` notation).
*Dependencies Setup*
. Ensure that Logback is on the classpath (`ch.qos.logback:logback-core`).
. Add Logstash Logback encode. For example, to use version `4.6`, add `net.logstash.logback:logstash-logback-encoder:4.6`.
*Logback Setup*
Consider the following example of a Logback configuration file (logback-spring.xml).
[source,xml]
-----
include::{project-root}/docs/src/main/asciidoc/logback-spring.xml[]
-----
That Logback configuration file:
* Logs information from the application in a JSON format to a `build/${spring.application.name}.json` file.
* Has commented out two additional appenders: console and standard log file.
* Has the same logging pattern as the one presented in the previous section.
NOTE: If you use a custom `logback-spring.xml`, you must pass the `spring.application.name` in the `bootstrap` rather than the `application` property file.
Otherwise, your custom logback file does not properly read the property.
=== Service Dependency Graph
When you consider distributed tracing tracks requests, it makes sense that
trace data can paint a picture of your architecture.

88
docs/src/main/asciidoc/spring-cloud-sleuth.adoc
View File

@@ -624,14 +624,6 @@ object, you will have to create a bean of `zipkin2.reporter.Sender` type.
}
----
== Zipkin Stream Span Consumer
IMPORTANT: We recommend using Zipkin's native support for message-based span sending.
Starting from the Edgware release, the Zipkin Stream server is deprecated.
In the Finchley release, it got removed.
If for some reason you need to create the deprecated Stream Zipkin server, see the https://cloud.spring.io/spring-cloud-static/Dalston.SR4/multi/multi__span_data_as_messages.html#_zipkin_consumer[Dalston Documentation].
== Integrations
=== OpenTracing
@@ -1012,17 +1004,77 @@ To turn off this feature, set the `spring.sleuth.quartz.enabled` property to `fa
For projects depending on Project Reactor such as Spring Cloud Gateway, we suggest turning the `spring.sleuth.reactor.decorate-on-each` option to `false`. That way an increased performance gain should be observed in comparison to the standard instrumentation mechanism. What this option does is it will wrap decorate `onLast` operator instead of `onEach` which will result in creation of far fewer objects. The downside of this is that when Project Reactor will change threads, the trace propagation will continue without issues, however anything relying on the `ThreadLocal` such as e.g. MDC entries can be buggy.
== Log integration
Sleuth configures the logging context with variables including the service name
(`%{spring.zipkin.service.name}`) and the trace ID (`%{traceId}`). These help
you connect logs with distributed traces and allow you choice in what tools you
use to troubleshoot your services.
If you use a log aggregating tool (such as https://www.elastic.co/products/kibana[Kibana], https://www.splunk.com/[Splunk], and others), you can order the events that took place.
An example from Kibana would resemble the following image:
image::https://raw.githubusercontent.com/spring-cloud/spring-cloud-sleuth/{branch}/docs/src/main/asciidoc/images/kibana.png[Log correlation with Kibana]
If you want to use https://www.elastic.co/guide/en/logstash/current/index.html[Logstash], the following listing shows the Grok pattern for Logstash:
[source]
filter {
# pattern matching logback pattern
grok {
match => { "message" => "%{TIMESTAMP_ISO8601:timestamp}\s+%{LOGLEVEL:severity}\s+\[%{DATA:service},%{DATA:trace},%{DATA:span}\]\s+%{DATA:pid}\s+---\s+\[%{DATA:thread}\]\s+%{DATA:class}\s+:\s+%{GREEDYDATA:rest}" }
}
date {
match => ["timestamp", "ISO8601"]
}
mutate {
remove_field => ["timestamp"]
}
}
NOTE: If you want to use Grok together with the logs from Cloud Foundry, you have to use the following pattern:
[source]
filter {
# pattern matching logback pattern
grok {
match => { "message" => "(?m)OUT\s+%{TIMESTAMP_ISO8601:timestamp}\s+%{LOGLEVEL:severity}\s+\[%{DATA:service},%{DATA:trace},%{DATA:span}\]\s+%{DATA:pid}\s+---\s+\[%{DATA:thread}\]\s+%{DATA:class}\s+:\s+%{GREEDYDATA:rest}" }
}
date {
match => ["timestamp", "ISO8601"]
}
mutate {
remove_field => ["timestamp"]
}
}
=== JSON Logback with Logstash
Often, you do not want to store your logs in a text file but in a JSON file that Logstash can immediately pick.
To do so, you have to do the following (for readability, we pass the dependencies in the `groupId:artifactId:version` notation).
*Dependencies Setup*
. Ensure that Logback is on the classpath (`ch.qos.logback:logback-core`).
. Add Logstash Logback encode. For example, to use version `4.6`, add `net.logstash.logback:logstash-logback-encoder:4.6`.
*Logback Setup*
Consider the following example of a Logback configuration file (logback-spring.xml).
[source,xml]
-----
include::{project-root}/docs/src/main/asciidoc/logback-spring.xml[]
-----
That Logback configuration file:
* Logs information from the application in a JSON format to a `build/${spring.application.name}.json` file.
* Has commented out two additional appenders: console and standard log file.
* Has the same logging pattern as the one presented in the previous section.
NOTE: If you use a custom `logback-spring.xml`, you must pass the `spring.application.name` in the `bootstrap` rather than the `application` property file.
Otherwise, your custom logback file does not properly read the property.
== Configuration properties
To see the list of all Sleuth related configuration properties please check link:appendix.html[the Appendix page].
== Running examples
You can see the running examples deployed in the https://run.pivotal.io/[Pivotal Web Services].
Check them out at the following links:
* https://docssleuth-zipkin-server.cfapps.io/[Zipkin for apps presented in the samples to the top]. First make
a request to https://docssleuth-service1.cfapps.io/start[Service 1] and then check out the trace in Zipkin.
* https://docsbrewing-zipkin-server.cfapps.io/[Zipkin for Brewery on PWS], its https://github.com/spring-cloud-samples/brewery[Github Code].
Ensure that you've picked the lookback period of 7 days. If there are no traces, go to https://docsbrewing-presenting.cfapps.io/[Presenting application]
and order some beers. Then check Zipkin for traces.