Spring/spring-integration
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Remove Legacy Metrics

Browse Source 
- Simplify MBeans - instead of wrapping to expose lifecycle methods,
  implement `ManageableLifecycle`. Register an additional MBean for
  polled endpoints to control the lifecycle.

* Polishing

- Move `QueueChannel` `@ManagedAttribute`s to `QueueChannelOperations`
- Make all `AbstractEndpoints` `IntegrationManagedResource`s and remove `ManagedEndpoint`
  to allow exposure of any `@Managed*` methods (including those on `Pausable`)
- Revert to `Lifecycle` for classes that are not related to endpoints
- Remove legacy metrics from docs
This commit is contained in:
Gary Russell
2020-08-07 12:56:57 -04:00
committed by GitHub
parent da5d002d64
commit 1beb854fb4
148 changed files with 533 additions and 7064 deletions
Download Patch File Download Diff File Expand all files Collapse all files

16
src/reference/asciidoc/graph.adoc
View File

@@ -91,20 +91,8 @@ A Spring Integration application with only the default components would expose a
----
====
NOTE: Version 5.2 has deprecated the legacy metrics in favor of Micrometer meters as discussed <<./metrics.adoc#metrics-management,Metrics Management>>.
While not shown above, the legacy metrics (under the `stats` child node) will continue to appear in the graph, but with an extra child node `"deprecated" : "stats are deprecated in favor of sendTimers and receiveCounters"`.
With some JSON serializers, you can suppress the inclusion of legacy statistics using several techniques; for example, with Jackson, you can register a `SimpleModule` configured with a `NullSerializer` with the `ObjectMapper`:
====
[source, java]
----
objectMapper.registerModule(new SimpleModule()
.addSerializer(IntegrationNode.Stats.class, NullSerializer.instance));
----
====
The resulting json contains `"stats" : null`.
NOTE: Version 5.2 deprecated the legacy metrics in favor of Micrometer meters as discussed <<./metrics.adoc#metrics-management,Metrics Management>>.
The legacy metrics were removed in Version 5.4 and will no longer appear in the graph.
In the preceding example, the graph consists of three top-level elements.

59
src/reference/asciidoc/jmx.adoc
View File

@@ -251,7 +251,7 @@ The following example shows how to declare an instance of an `IntegrationMBeanEx
The MBean exporter is orthogonal to the one provided in Spring core.
It registers message channels and message handlers but does not register itself.
You can expose the exporter itself (and certain other components in Spring Integration) by using the standard `<context:mbean-export/>` tag.
The exporter has some metrics attached to it -- for instance, a count of the number of active handlers and the number of queued messages.
The exporter has some metrics attached to it -- for instance, a count of the number of handlers and the number of queued messages.
It also has a useful operation, as discussed in <<jmx-mbean-shutdown>>.
=====
@@ -365,69 +365,14 @@ These resulted in a significant performance improvement of the JMX statistics co
However, it has some implications for user code in a few specific (uncommon) situations.
These changes are detailed below, with a caution where necessary.
Metrics Capture::
Previously, `MessageSource`, `MessageChannel`, and `MessageHandler` metrics were captured by wrapping the object in a JDK dynamic proxy to intercept appropriate method calls and capture the statistics.
The proxy was added when an integration MBean exporter was declared in the context.
+
Now, the statistics are captured by the beans themselves.
See <<./metrics.adoc#metrics-management,Metrics and Management>> for more information.
+
WARNING: This change means that you no longer automatically get an MBean or statistics for custom `MessageHandler` implementations, unless those custom handlers extend `AbstractMessageHandler`.
The simplest way to resolve this is to extend `AbstractMessageHandler`.
If you cannot do so, another work around is to implement the `MessageHandlerMetrics` interface.
For convenience, a `DefaultMessageHandlerMetrics` is provided to capture and report statistics.
You should invoke the `beforeHandle` and `afterHandle` at the appropriate times.
Your `MessageHandlerMetrics` methods can then delegate to this object to obtain each statistic.
Similarly, `MessageSource` implementations must extend `AbstractMessageSource` or implement `MessageSourceMetrics`.
Message sources capture only a count, so there is no provided convenience class.
You should maintain the count in an `AtomicLong` field.
+
The removal of the proxy has two additional benefits:
+
* Stack traces in exceptions are reduced (when JMX is enabled) because the proxy is not on the stack
* Cases where two MBeans were exported for the same bean now only export a single MBean with consolidated attributes and operations (see the MBean consolidation bullet, later).
Resolution::
`System.nanoTime()` (rather than `System.currentTimeMillis()`) is now used to capture times .
This may provide more accuracy on some JVMs, espcially when you expect durations of less than one millisecond.
Setting Initial Statistics Collection State::
Previously, when JMX was enabled, all sources, channels, and handlers captured statistics.
You can now control whether the statistics are enabled on an individual component.
Further, you can capture simple counts on `MessageChannel` instances and `MessageHandler` instances instead of capturing the complete time-based statistics.
This can have significant performance implications, because you can selectively configure where you need detailed statistics and enable and disable collection at runtime.
+
See <<./metrics.adoc#metrics-management,Metrics and Management>>.
@IntegrationManagedResource::
Similar to the `@ManagedResource` annotation, the `@IntegrationManagedResource` marks a class as being eligible to be exported as an MBean.
However, it is exported only if the application context has an `IntegrationMBeanExporter`.
+
Certain Spring Integration classes (in the `org.springframework.integration`) package) that were previously annotated with`@ManagedResource` are now annotated with both `@ManagedResource` and `@IntegrationManagedResource`.
Certain Spring Integration classes (in the `org.springframework.integration`) package) that were previously annotated with `@ManagedResource` are now annotated with both `@ManagedResource` and `@IntegrationManagedResource`.
This is for backwards compatibility (see the next item).
Such MBeans are exported by any context `MBeanServer` or by an `IntegrationMBeanExporter` (but not both -- if both exporters are present, the bean is exported by the integration exporter if the bean matches a `managed-components` pattern).
Consolidated MBeans::
Certain classes within the framework (mapping routers, for example) have additional attributes and operations over and above those provided by metrics and `Lifecycle`.
We use a `Router` as an example here.
+
Previously, beans of these types were exported as two distinct MBeans:
+
* The metrics MBean (with an object name such as `intDomain:type=MessageHandler,name=myRouter,bean=endpoint`).
This MBean had metrics attributes and metrics/Lifecycle operations.
* A second MBean (with an object name such as `ctxDomain:name=org.springframework.integration.config.` `RouterFactoryBean#0`,type=MethodInvokingRouter`) was exported with the channel mappings attribute and operations.
+
Now the attributes and operations are consolidated into a single MBean.
The object name depends on the exporter.
If exported by the integration MBean exporter, the object name is, for example: `intDomain:type=MessageHandler,name=myRouter,bean=endpoint`.
If exported by another exporter, the object name is, for example: `ctxDomain:name=org.springframework.integration.config.` `RouterFactoryBean#0,type=MethodInvokingRouter`.
There is no difference between these MBeans (aside from the object name), except that the statistics are not enabled (the attributes are `0`) by exporters other than the integration exporter.
You can enable statistics at runtime by using the JMX operations.
When exported by the integration MBean exporter, the initial state can be managed as described earlier.
+
WARNING: If you currently use the second MBean to change, for example, channel mappings and you use the integration MBean exporter, note that the object name has changed because of the MBean consolidation.
There is no change if you are not using the integration MBean exporter.
MBean Exporter Bean Name Patterns::
Previously, the `managed-components` patterns were inclusive only.
If a bean name matched one of the patterns, it would be included.

270
src/reference/asciidoc/metrics.adoc
View File

@@ -5,21 +5,14 @@ This section describes how to capture metrics for Spring Integration.
In recent versions, we have relied more on Micrometer (see https://micrometer.io), and we plan to use Micrometer even more in future releases.
[[configuring-metrics-capture]]
==== Configuring Metrics Capture
==== Legacy Metrics
NOTE: Prior to version 4.2, metrics were only available when JMX was enabled.
See <<./jmx.adoc#jmx,JMX Support>>.
Legacy metrics were removed in Version 5.4; see Micrometer Integration below.
To enable `MessageSource`, `MessageChannel`, and `MessageHandler` metrics, add an `<int:management/>` bean to the application context (in XML) or annotate one of your `@Configuration` classes with `@EnableIntegrationManagement` (in Java).
`MessageSource` instances maintain only counts, `MessageChannel` instances and `MessageHandler` instances maintain duration statistics in addition to counts.
See <<mgmt-channel-features>> and <<mgmt-handler-features>>, later in this chapter.
==== Disabling Logging in High Volume Environments
Doing so causes the automatic registration of the `IntegrationManagementConfigurer` bean in the application context.
Only one such bean can exist in the context, and, if registered manually via a `<bean/>` definition, it must have the bean name set to `integrationManagementConfigurer`.
This bean applies its configuration to beans after all beans in the context have been instantiated.
In addition to metrics, you can control debug logging in the main message flow.
In very high volume applications, even calls to `isDebugEnabled()` can be quite expensive with some logging subsystems.
You can control debug logging in the main message flow.
In very high volume applications, calls to `isDebugEnabled()` can be quite expensive with some logging subsystems.
You can disable all such logging to avoid this overhead.
Exception logging (debug or otherwise) is not affected by this setting.
@@ -28,13 +21,8 @@ The following listing shows the available options for controlling logging:
====
[source, xml]
----
<int:management
default-logging-enabled="true" <1>
default-counts-enabled="false" <2>
default-stats-enabled="false" <3>
counts-enabled-patterns="foo, !baz, ba*" <4>
stats-enabled-patterns="fiz, buz" <5>
metrics-factory="myMetricsFactory" /> <6>
<int:management default-logging-enabled="true"/> <1>
----
[source, java]
@@ -42,12 +30,8 @@ The following listing shows the available options for controlling logging:
@Configuration
@EnableIntegration
@EnableIntegrationManagement(
defaultLoggingEnabled = "true", <1>
defaultCountsEnabled = "false", <2>
defaultStatsEnabled = "false", <3>
countsEnabled = { "foo", "${count.patterns}" }, <4>
statsEnabled = { "qux", "!*" }, <5>
MetricsFactory = "myMetricsFactory") <6>
defaultLoggingEnabled = "true" <1>)
public static class ContextConfiguration {
...
}
@@ -59,64 +43,20 @@ Set to 'true' to enable debug logging (if also enabled by the logging subsystem)
Only applied if you have not explicitly configured the setting in a bean definition.
The default is `true`.
<2> Enable or disable count metrics for components that do not match one of the patterns in <4>.
Only applied if you have not explicitly configured the setting in a bean definition.
The default is `false`.
<3> Enable or disable statistical metrics for components that do not match one of the patterns in <5>.
Only applied if you have not explicitly configured the setting in a bean definition.
The default is 'false'.
<4> A comma-delimited list of patterns for beans for which counts should be enabled.
You can negate the pattern with `!`.
First match (positive or negative) wins.
In the unlikely event that you have a bean name starting with `!`, escape the `!` in the pattern.
For example, `\!something` positively matches a bean named `!something`.
<5> A comma-delimited list of patterns for beans for which statistical metrics should be enabled.
You can negate the pattern\ with `!`.
First match (positive or negative) wins.
In the unlikely event that you have a bean name starting with `!`, escape the `!` in the pattern.
`\!something` positively matches a bean named `!something`.
The collection of statistics implies the collection of counts.
<6> A reference to a `MetricsFactory`.
See <<mgmt-metrics-factory>>.
At runtime, counts and statistics can be obtained by calling `getChannelMetrics`, `getHandlerMetrics` and `getSourceMetrics` (all from the `IntegrationManagementConfigurer` class), which return `MessageChannelMetrics`, `MessageHandlerMetrics`, and `MessageSourceMetrics`, respectively.
See the https://docs.spring.io/spring-integration/api/index.html[Javadoc] for complete information about these classes.
When JMX is enabled (see <<./jmx.adoc#jmx,JMX Support>>), `IntegrationMBeanExporter` also exposes these metrics.
IMPORTANT:
`defaultLoggingEnabled`, `defaultCountsEnabled`, and `defaultStatsEnabled` are applied only if you have not explicitly configured the corresponding setting in a bean definition.
Starting with version 5.0.2, the framework automatically detects whether the application context has a single `MetricsFactory` bean and, if so, uses it instead of the default metrics factory.
IMPORTANT: These legacy metrics have been deprecated in favor of Micrometer metrics discussed below.
Legacy metrics support will be removed in a future release.
IMPORTANT: `defaultLoggingEnabled` is applied only if you have not explicitly configured the corresponding setting in a bean definition.
[[micrometer-integration]]
==== Micrometer Integration
Starting with version 5.0.3, the presence of a https://micrometer.io/[Micrometer] `MeterRegistry` in the application context triggers support for Micrometer metrics in addition to the built-in metrics (note that the legacy built-in metrics will be removed in a future release).
IMPORTANT: Micrometer was first supported in version 5.0.2, but changes were made to the Micrometer `Meters` in version 5.0.3 to make them more suitable for use in dimensional systems.
Further changes were made in 5.0.4.
If you use Micrometer, a minimum of version 5.0.4 is recommended, since some of the changes in 5.0.4 were breaking API changes.
Starting with version 5.0.3, the presence of a https://micrometer.io/[Micrometer] `MeterRegistry` in the application context triggers support for Micrometer metrics.
To use Micrometer, add one of the `MeterRegistry` beans to the application context.
If the `IntegrationManagementConfigurer` detects exactly one `MeterRegistry` bean, it configures a `MicrometerMetricsCaptor` bean with a name of `integrationMicrometerMetricsCaptor`.
For each `MessageHandler` and `MessageChannel`, timers are registered.
For each `MessageSource`, a counter is registered.
This only applies to objects that extend `AbstractMessageHandler`, `AbstractMessageChannel`, and `AbstractMessageSource` (which is the case for most framework components).
With Micrometer metrics, the `statsEnabled` flag has no effect, since statistics capture is delegated to Micrometer.
The `countsEnabled` flag controls whether the Micrometer `Meter` instances are updated when processing each message.
The `Timer` Meters for send operations on message channels have the following names or tags:
* `name`: `spring.integration.send`
@@ -178,191 +118,3 @@ and
* `tag`: `type:channel`
* `tag`: `name:<componentName>`
* `description`: `The remaining capacity of the queue channel`
[[mgmt-channel-features]]
==== `MessageChannel` Metric Features
These legacy metrics will be removed in a future release.
See <<micrometer-integration>>.
Message channels report metrics according to their concrete type.
If you are looking at a `DirectChannel`, you see statistics for the send operation.
If it is a `QueueChannel`, you also see statistics for the receive operation as well as the count of messages that are currently buffered by this `QueueChannel`.
In both cases, some metrics are simple counters (message count and error count), and some are estimates of averages of interesting quantities.
The algorithms used to calculate these estimates are described briefly in the following table.
.MessageChannel Metrics
[cols="1,2,3", options="header"]
|===
| Metric Type
| Example
| Algorithm
| Count
| Send Count
| Simple incrementer.
Increases by one when an event occurs.
| Error Count
| Send Error Count
| Simple incrementer.
Increases by one when an send results in an error.
| Duration
| Send Duration (method execution time in milliseconds)
| Exponential moving average with decay factor (ten by default).
Average of the method execution time over roughly the last ten (by default) measurements.
| Rate
| Send Rate (number of operations per second)
| Inverse of Exponential moving average of the interval between events with decay in time (lapsing over 60 seconds by default) and per measurement (last ten events by default).
| Error Rate
| Send Error Rate (number of errors per second)
| Inverse of exponential moving average of the interval between error events with decay in time (lapsing over 60 seconds by default) and per measurement (last ten events by default).
| Ratio
| Send Success Ratio (ratio of successful to total sends)
| Estimate the success ratio as the exponential moving average of the series composed of values (1 for success and 0 for failure, decaying as per the rate measurement over time and events by default).
The error ratio is: 1 - success ratio.
|===
[[mgmt-handler-features]]
==== MessageHandler Metric Features
These legacy metrics will be removed in a future release.
See <<micrometer-integration>>.
The following table shows the statistics maintained for message handlers.
Some metrics are simple counters (message count and error count), and one is an estimate of averages of send duration.
The algorithms used to calculate these estimates are described briefly in the following table:
.MessageHandlerMetrics
[cols="1,2,3", options="header"]
|===
| Metric Type
| Example
| Algorithm
| Count
| Handle Count
| Simple incrementer.
Increases by one when an event occurs.
| Error Count
| Handler Error Count
| Simple incrementer.
Increases by one when an invocation results in an error.
| Active Count
| Handler Active Count
| Indicates the number of currently active threads currently invoking the handler (or any downstream synchronous flow).
| Duration
| Handle Duration (method execution time in milliseconds)
| Exponential moving average with decay factor (ten by default).
Average of the method execution time over roughly the last ten (default) measurements.
|===
[[mgmt-statistics]]
==== Time-Based Average Estimates
A feature of the time-based average estimates is that they decay with time if no new measurements arrive.
To help interpret the behavior over time, the time (in seconds) since the last measurement is also exposed as a metric.
There are two basic exponential models: decay per measurement (appropriate for duration and anything where the number of measurements is part of the metric) and decay per time unit (more suitable for rate measurements where the time in between measurements is part of the metric).
Both models depend on the fact that `S(n) = sum(i=0,i=n) w(i) x(i)` has a special form when `w(i) = r^i`, with `r=constant`: `S(n) = x(n) + r S(n-1)` (so you only have to store `S(n-1)` (not the whole series `x(i)`) to generate a new metric estimate from the last measurement).
The algorithms used in the duration metrics use `r=exp(-1/M)` with `M=10`.
The net effect is that the estimate, `S(n)`, is more heavily weighted to recent measurements and is composed roughly of the last `M` measurements.
So `M` is the "`window`" or lapse rate of the estimate.
For the vanilla moving average, `i` is a counter over the number of measurements.
For the rate, we interpret `i` as the elapsed time or a combination of elapsed time and a counter (so the metric estimate contains contributions roughly from the last `M` measurements and the last `T` seconds).
[[mgmt-metrics-factory]]
==== Metrics Factory
A strategy interface `MetricsFactory` has been introduced to let you provide custom channel metrics for your `MessageChannel` instances and `MessageHandler` instances.
By default, a `DefaultMetricsFactory` provides a default implementation of `MessageChannelMetrics` and `MessageHandlerMetrics`, <<configuring-metrics-capture,described earlier>>.
To override the default `MetricsFactory`, configure it as <<configuring-metrics-capture,described earlier>>, by providing a reference to your `MetricsFactory` bean instance.
You can either customize the default implementations, as described in the next section, or provide completely different
implementations by extending `AbstractMessageChannelMetrics` or `AbstractMessageHandlerMetrics`.
See also <<micrometer-integration>>.
In addition to the default metrics factory <<configuring-metrics-capture,described earlier>>, the framework provides the `AggregatingMetricsFactory`.
This factory creates `AggregatingMessageChannelMetrics` and `AggregatingMessageHandlerMetrics` instances.
In very high volume scenarios, the cost of capturing statistics can be prohibitive (the time to make two calls to the system and
store the data for each message).
The aggregating metrics aggregate the response time over a sample of messages.
This can save significant CPU time.
CAUTION: The statistics are likely to be skewed if messages arrive in bursts.
These metrics are intended for use with high, constant-volume, message rates.
The following example shows how to define an aggregrating metrics factory:
====
[source, xml]
----
<bean id="aggregatingMetricsFactory"
class="org.springframework.integration.support.management.AggregatingMetricsFactory">
<constructor-arg value="1000" /> <!-- sample size -->
</bean>
----
====
The preceding configuration aggregates the duration over 1000 messages.
Counts (send and error) are maintained per-message, but the statistics are per 1000 messages.
===== Customizing the Default Channel and Handler Statistics
See <<mgmt-statistics>> and the https://docs.spring.io/spring-integration/api/index.html[Javadoc] for the `ExponentialMovingAverage*` classes for more information about these values.
By default, the `DefaultMessageChannelMetrics` and `DefaultMessageHandlerMetrics` use a "`window`" of ten measurements,
a rate period of one second (meaning rate per second) and a decay lapse period of one minute.
If you wish to override these defaults, you can provide a custom `MetricsFactory` that returns appropriately configured
metrics and provide a reference to it in the MBean exporter, as <<mgmt-metrics-factory,described earlier>>.
The following example shows how to do so:
====
[source,java]
----
public static class CustomMetrics implements MetricsFactory {
@Override
public AbstractMessageChannelMetrics createChannelMetrics(String name) {
return new DefaultMessageChannelMetrics(name,
new ExponentialMovingAverage(20, 1000000.),
new ExponentialMovingAverageRate(2000, 120000, 30, true),
new ExponentialMovingAverageRatio(130000, 40, true),
new ExponentialMovingAverageRate(3000, 140000, 50, true));
}
@Override
public AbstractMessageHandlerMetrics createHandlerMetrics(String name) {
return new DefaultMessageHandlerMetrics(name, new ExponentialMovingAverage(20, 1000000.));
}
}
----
====
===== Advanced Customization
The customizations described earlier are wholesale and apply to all appropriate beans exported by the MBean exporter.
This is the extent of customization available when you use XML configuration.
Individual beans can be provided with different implementations using by Java `@Configuration` or programmatically at
runtime (after the application context has been refreshed) by invoking the `configureMetrics` methods on
`AbstractMessageChannel` and `AbstractMessageHandler`.
===== Performance Improvement
Previously, the time-based metrics (see <<mgmt-statistics>>) were calculated in real time.
The statistics are now calculated when retrieved instead.
This resulted in a significant performance improvement, at the expense of a small amount of additional memory for each statistic.
As <<configuring-metrics-capture,discussed earlier>>, you can disable the statistics altogether while retaining the MBean that allows the invocation of `Lifecycle` methods.

2
src/reference/asciidoc/whats-new.adoc
View File

@@ -47,6 +47,8 @@ See <<./gateway.adoc#gateway-default-reply-channel,Setting the Default Reply Cha
The aggregator (and resequencer) can now expire orphaned groups (groups in a persistent store where no new messages arrive after an application restart).
See <<./aggregator.adoc#aggregator-expiring-groups, Aggregator Expiring Groups>> for more information.
The legacy metrics that were replaced by Micrometer meters have been removed.
[[x5.4-tcp]]
=== TCP Changes