* INT-4389: Default to SimpleSeqSizeReleaseStrategy
JIRA: https://jira.spring.io/browse/INT-4389
An `AbstractCorrelatingMessageHandler` fallback to the
`SimpleSequenceSizeReleaseStrategy` when `releaseStrategy` isn't
provided.
* Make `ReleaseStrategyFactoryBean` to fallback to the
`SimpleSequenceSizeReleaseStrategy` for consistency.
* * Fix `ResequencerParser` to fallback to the null `releaseStrategy`.
This way a subsequent `releasePartialSequences` will set a
`SequenceSizeReleaseStrategy` as a default one
* * Fix `BarrierMessageHandler` to populate a default `CorrelationStrategy`
* * More polishing to `BarrierMessageHandler`
JIRA: https://jira.spring.io/browse/INT-4466
When we schedule group for force complete in the
`AbstractCorrelatingMessageHandler`, we don't track a group `timestamp`
and its `lastModified` before the scheduled task.
This way, in the cluster environment, we may schedule several tasks
for different messages and the first started may release the group too
early.
Just because we extract a `MessageGroup` from the store already in the
task per se.
* Propagate the actual `timestamp` and `lastModified` from group before
scheduling task.
Compare these value with the actual group metadata in the
`processForceRelease()` before performing real `forceRelease()`.
This way we restore behavior before fixing memory leak, when we
propagated full `MessageGroup` to the scheduled task
* Implement `ResequencingMessageHandler.getComponentType()` for
consistency
* Remove unnecessary overhead with the `volatile` on many
`AbstractCorrelatingMessageHandler` properties, which hardly ever can be
changed at runtime
**Cherry-pick to 5.0.x and 4.3.x**
JIRA: https://jira.spring.io/browse/INT-4467
The package tangles is caused by the explicit classes declarations.
* Since we use only method definitions from those classes, there is just
enough to use class names and use reflection to get class objects and
then get methods from them
_The fix is fully compatible for back-porting_
**Cherry-pick to 5.0.x**
* Wrap non-`StandardIntegrationFlow` into Proxy
In previous version all the `IntegrationFlow` beans have been replaced
by the `StandardIntegrationFlow` i the `IntegrationFlowBeanPostProcessor`
if they are lambda.
That works for Java, but doesn't with Kotlin, since lambdas i last one
are not synthetic classes.
Therefore some Java DSL definitions (especially `.subFlowMapping()`)
don't work consistently in two languages.
* Introduce `IntegrationFlowLifecycleAdvice` to wrap all the
non-`StandardIntegrationFlow`s (excluding `IntegrationFlowAdapter`)
into the `Proxy` to expose `SmartLifecycle` and `getInputChannel()`
operations and delegate them to the internal `StandardIntegrationFlow`
created by the `IntegrationFlowBeanPostProcessor`.
* This way any custom `IntegrationFlow` implementations can be used
for manual flow registration via `IntegrationFlowContext`
* Polish `RouterDslTests.kt` for the `@Bean`s for sub-flows.
* Document in the `dsl.adoc` a request-reply approach for the case
when `.subFlowMapping()` refers to an `IntegrationFlow` `@Bean`.
* Polishing for the `FlowServiceTests` since all the
non-`StandardIntegrationFlow`s and not-`IntegrationFlowAdapter`s are
wrapped now to the Proxy.
* Add missing `from()` delegations into the `IntegrationFlowAdapter`
* Polishing for the `ManualFlowTests` since all the `IntegrationFlow`
now are `Lifecycle` after wrapping to the Proxy.
* Add JavaDocs to the IntegrationFlowLifecycleAdvice and polishing for the dsl.adoc
* * Add JavaDocs for the `IntegrationFlowBeanPostProcessor.processIntegrationFlowImpl()`
* Improve JavaDoc for the `RouterSpec.subFlowMapping()`
* Assert in the `FlowServiceTests` that proxied custom flow implements
all the expected interfaces
https://build.spring.io/browse/INT-SI50X-JOB1-56
We can't wait for the latch in the interruptable code flow;
we can't have a round-robing election guarantees.
* Add `Thread.sleep(LockRegistryLeaderInitiator.this.busyWaitMillis)`
to the `LockRegistryLeaderInitiator` when we restart the main task
* Remove latches waiting and thread shifting from the
`RedisLockRegistryLeaderInitiatorTests`
* Use long `busyWaitMillis` for yielding initiator to let the second
candidate to be elected
**Cherry-pick to 5.0.x**
https://build.spring.io/browse/INT-FATS5IC-501/
When we try to wait for the `Latch` in the interruptable code flow,
it is a fact that we step away from the waiting and end up with the
race condition downstream.
* Wrap `Latch` in the interruptable `publishOnRevoked()` code to the
`Executor.execute()`
* Remove `deleteTimeoutMillis` option from the `RedisLockRegistry`
since it doesn't make sense in the interruptable code.
* Add `RedisLockRegistry.setExecutor()` to allow to inject an external
`Executor`
* Add more debug logging into the `LockRegistryLeaderInitiator`
**Cherry-pick to 5.0.x**
https://build.spring.io/browse/INT-MASTER-1024
When the Lettuce Redis client catches an `InterruptedException`, it is
wrapped to the `RedisCommandInterruptedException`, therefore when we
catch an exception on our code level it is not an `InterruptedException`
anymore and we can't proceed in the loop because the tread is
`interrupted` already.
* Check the `interrupted` alongside with the `InterruptedException`
to restart a loop from a fresh thread.
**Cherry-pick to 5.0.x**
* Use Lettuce Redis client; Redis module fixes
The Lettuce client is based on the Netty and more stable, than Jedis
therefore we get a performance improvement for tests
(it saves us at about 30 seconds).
Also this client doesn't fail for me on Windows sporadically (very often)
with the `ConnectionClosedException`
* After switching to the Netty-based client, we expose the interrupted
Thread issue in the `LockRegistryLeaderInitiator`.
If we interrupted (expected behavior), we try to unlock calling
`RedisLockRegistry`, but Netty client reject our request because the
thread is interrupted, therefore we never delete the lock when we yield
our leadership.
Fix the issue with shifting a `RedisTemplate.delete()` operation to the
`ExecutorService` when the current thread is interrupted
* Allow to configure such an `ExecutorService` and timeout to wait for
the `submit()` result
* Refactor `RedisAvailableRule` and all the Redis tests do not expose
the target `RedisConnectionFactory` implementation.
* Make all the test-cases based on the `connectionFactory` created by
the `RedisAvailableTests.setupConnectionFactory()`
* Tweak some unnecessary timeouts and sleeps for better tests task
throughput
* Add a `Log4j2LevelAdjuster` into the `RedisLockRegistryLeaderInitiatorTests`
https://build.spring.io/browse/INT-MASTER-1017
Looks like non-`volatile` `size` property in the `LinkedList` is not
updated properly for the `contains()` assertion, therefore we end up
with the `ConcurrentModificationException`
* Move `assertThat()` into the `synchronized (overridePresent)` block
to enforce CPU cache flushing and, therefore, have an actual value
for the `LinkedList.size` during iteration
JIRA: https://jira.spring.io/browse/INT-4461
Convert `byte[]` to `String` using `URF-8` by default.
Add optional evaluate methods to JsonPathUtils with a Charset to use when
converting `byte[]` to `String`.
This is not currently exposed using SpEL. It can be done, but probably not worth
the effort until somebody asks for it.
**cherry-pick to 5.0.x, 4.3.x**
* Use `BAIS`
For better back-pressure handling do not perform active operations
directly on the current `Flux` (e.g. via `doOnNext()`).
It's better to postpone such a handling to back-pressure ready operators
Like in this `FluxMessageChannel` case, the `.handle()` operator is
wrapping the "hard" `send()` operation.
Also include `.errorStrategyContinue()` do not stop during message
processing
* INT-4458: Do not expose recursive generics API
JIRA: https://jira.spring.io/browse/INT-4458
According Kotlin generics system restrictions we can't expose API
based on the recursive generics, like we have with Java DSL for JMS
* Replace explicit recursive generics on factory method with the wildcard (`?`).
This way both Kotlin and Java are able to instantiate target object and perform
the proper chain API auto-completion
* Fix `JmsInboundChannelAdapterSpec` to deal with provided `S` type for the
`configureListenerContainer()` - the way we can configure a `JmsDefaultListenerContainerSpec`
* Add `kotlin-spring` Gradle plugin to avoid extra `open` modificator on `@Bean` methods
* Add `JmsDslKotlinTests.kt`
**Cherry-pick to 5.0.x excluding Kotlin support**
* Fix `Amqp` DSL factory for recursive generics
* Polishing for Kotlin tests
JIRA: https://jira.spring.io/browse/INT-4456
Since Kotlin compiles lambdas different way than Java, they are not
synthetic classes at runtime anymore.
Therefore we fallback to the method invocation logic, but since Java 8
interfaces has `default` methods as well the `MessagingMethodInvokerHelper`
can't select the target method for invocation
* Use explicit `Function.apply()` for non-synthetic implementations
* Add `RouterDslTests` Kotlin-based test-case
**Cherry-pick to 5.0.x**
* Add more explicit methods for invokers
The `transactionWithCommitAndAdvices()` uses `SimpleRepeatAdvice`
which essentially performs `doPoll()` twice.
In this case we don't have enough messages in the `goodInputWithAdvice`
queue
* Add one more message to the `goodInputWithAdvice` queue during testing
* Increase all the timeouts
* Increase capacity in queues for the `transactionWithCommitAndAdvices()`
**Cherry-pick to 5.0.x and 4.3.x**
JIRA: https://jira.spring.io/browse/INT-4455
When using java config, `setChannelMapping` NPEs due to no AC.
**cherry-pick to 4.3.x, and to master, removing AC check**
* Polishing - PR comment
* Resolve TODO in the `ErrorMessageExceptionTypeRouter`
* INT-4446 Improve EmbeddedJsonHeadersMessageMapper
JIRA: https://jira.spring.io/browse/INT-4446
* Do not recreate message if not necessarily
* Do not let to generate `id` and `timestamp` if they are not mapped
* Use `smartMatch` to allow to configure negative patterns
* Introduce `PatternMatchUtils.smartMatchIgnoreCase()` for convenience
**Cherry-pick to 5.0.x**
* * Polishing `EmbeddedJsonHeadersMessageMapper`
JIRA: https://jira.spring.io/browse/INT-4447
When we get an exception during `this.lock.unlock()`, we don't revoke
leadership.
In case of external resource (e.g. JDBC) this may cause a race condition
when the second candidate is selected as leader when connection comes back
* Catch `this.lock.unlock()` exceptions and log them under DEBUG.
This way we proceed to the `handleRevoked()` logic
**Cherry-pick to 5.0.x and 4.3.x**
https://build.spring.io/browse/INT-MASTER-999/
Since `BeanFactoryTypeConverterTests` and `CallerBlocksPolicyTests`
uses too long `Thread.sleep()` there is no guarantee that they are going
to be performed after expected 10 seconds
* Decrease `Thread.sleep()` in those tests
* Increase timeouts in the `RoundRobinDispatcherConcurrentTests` and
`ManualFlowTests`
* Fix `PollerAdviceTests` to re-use `TaskScheduler` from the ctx
instead of local, not closed instance
* Use `OnlyOnceTrigger` instead of local implementations
* Change the `primary` `Trigger` bean to the `PeriodicTrigger` as well.
The minimum interval for the `CronTrigger` is 1 seconds - it doesn't
matter for this test-case
JIRA: https://jira.spring.io/browse/INT-4438
The `SmartLifecycleRoleController` is based on the `MultiValueMap`
which used internally a `List` for the values.
With such an architecture we can add the same value several times.
On the other hand we are iterating over `Lifecycle`s in the role and
build a `Map` for their running status.
In this case when `NamesComponent`s return the same name the Java
`Collectors.toMap()` fails with a duplicate key error.
In any cases it would be better do not allow to add the same lifecylce
several time to the role or different with the same name.
* Add search logic to the `addLifecycleToRole()` to fail fast with the
`IllegalArgumentException` because a lifecycle with the same name is
already present in the role
**Cherry-pick to 5.0.x**
* Remove redundant `this.initialized = false` from the
`AbstractPollingEndpoint.doStop()`
Add `allEndpointsRunning()` verification to the `EndpointRoleParserTests`
Polishing
https://build.spring.io/browse/INT-SI50X-20
Looks like 5 seconds is not enough to wait for the test completion,
especially on CI server
* Refactor `PollingConsumerEndpointTests` to use `OnlyOnceTrigger`
from the test-support module instead of local `Trigger` implementation
* Increase timeout for latch in the `OnlyOnceTrigger` to 10 seconds
**Cherry-pick to 5.0.x**
JIRA: https://jira.spring.io/browse/INT-4440
The previous fix eliminated an extra `generateId()` call, but at the
same introduced regression do not populate `id` and `timestamp` from
the serialized state, e.g. after JSON transferring over the network
* Introduce a couple utility methods in the `MutableMessageHeaders`
to extract and parse `id` and `timestamp` from the provided headers
**Cherry-pick to 5.0.x**
JIRA: https://jira.spring.io/browse/INT-4441
Possible concurrent updates to `AbstractCorrelatingMessageHandler.groupIds` and
`expireGroupScheduledFutures`.
JIRA: https://jira.spring.io/browse/INT-4434
There were a restriction introduced since Java DSL `1.2` do not use
`IntegrationFlow` beans for sub-flow definitions, e.g. in routers.
It is considered as regression by community because it worked before
in version `1.1`
* Introduce `IntegrationFlow.getInputChannel()` to be able to bridge
from the main flow to the flow which is treated as sub-flow.
In most cases we talk about an independent bean for the `IntegrationFlow`
which can be used as a stand along one and as a sub-flow in other flow
**Cherry-pick to 5.0.x**
Add JavaDocs to the `EndpointSpec.obtainInputChannelFromFlow()`
JIRA: https://jira.spring.io/browse/INT-4433
The current `MessagePublishingInterceptor` behavior is to parse expressions
on each method invocation what is not so efficient at runtime
* Introduce `default` `Expression`-based method to the `PublisherMetadataSource`
contract and call existing String-based methods for backward compatibility.
* Deprecate String-based `PublisherMetadataSource` methods in favor of newly
introduced `Expression`-based
* Implement new `getExpressionForPayload()` and `getExpressionsForHeaders()`
in all the `PublisherMetadataSource` implementations
* Cache parsed `Expression` s during initialization in the `PublisherMetadataSource`
implementations or do that on demand in the `MethodAnnotationPublisherMetadataSource`
by provided method basis
* Introduce `MethodAnnotationPublisherMetadataSource#metadataCacheLimit` and populate
its value from the `@EnablePublisher` or `<int:annotation-configuration>`
* Implement `entrySet()` and `values()` in the `ExpressionEvalMap`
**Cherry-pick to 5.0.x**
* Add `@SuppressWarnings("varargs")` to avoid compilation warning
* Remove LRU cache logic - it's fine to cache all the info about methods
in the classpath
For better traceability for errors during lock acquiring add DEBUG
logging message in the `catch` block before returning back to the main
loop for the next acquiring attempt
**Cherry-pick to 5.0.x and 4.3.x**
JIRA: https://jira.spring.io/browse/INT-4437
The `ScatterGatherHandler` overrides a `replyChannel` header for the
scatter message to its internal queue and doesn't reinstate the original
`replyChannel` header when producer a gather result message
* Rebuild gather result message with population a proper `replyChannel`
header from the request message and removing a `gatherResultChannel`
header
**Cherry-pick to 5.0.3 and 4.3.x**
Polishing
JIRA: https://jira.spring.io/browse/INT-4390
Phase I: flow context/registration cycle
- extract interfaces
- rename implementations to `Standard...`
Phase II - dsl<->dsl.channel tangles
Move the channel specs to dsl.
Fix missing refactorings
(Not related to DSL) - rename core `event` package to `events`
- avoid collision with event module package
- fix tangle caused by `MessageGroupExpiredEvent`.
Phase III - dsl<->config.dsl
- move classes from config.dsl to dsl
(Not related to DSL) - fix many tangles caused by graph being a sub-package of support.management
- make `graph` a top-level package
(Not related to DSL) - move `IntegrationManagementConfigurer` from
`management` to `config` - tangle between core and management
* Polishing Copyrights, diamonds, some JavaDocs and What's New
JIRA: https://jira.spring.io/browse/INT-4430
When Iterator-based `FileSplitter` splits the file, and an exception
throws in downstream flow (in the same thread), the exception propagates
to the caller leaving underlying file reader opened.
This commit changes `AbstractMessageSplitter` the way, that,
when any exception happens, if Iterator implements `java.io.Closeable`,
its `close()` method will be called before propagating exception.
Also `FileSplitter`'s underlying iterator implements `Closeable` now.
* Make `CloseableIterator` to follow `Closeable` contract.
Now `CloseableIterator.close()` declares `IOException` and can be used
as base interface for `FunctionIterator`.
* Adjust tests.
Adjust tests to reflect the fact that we call `close()` on the reader
one more time in the end of iterator.
**Cherry-pick to 5.0.x and 4.3.x**
JIRA: https://jira.spring.io/browse/INT-4428
Using `increaseNestingLevel()` and `decreaseNestingLevel()` is not
thread-safe and may cause a race conditions
* Use `MethodParameter.nested()` instead which creates and cache a new
`MethodParameter` for the nested generic type
**Cherry-pick to 5.0.x**
JIRA: https://jira.spring.io/browse/INT-4392
Avoid the need for reflection on the `MicrometerMetricsCaptor`.
- move facade interfaces to a sub-package.
Remove deprecated `MicrometerMetricsFactory.
Fixesspring-projects/spring-integration#2388
During `HeaderEnricherSpec` refactoring the `adviceChain` population
has been missed, alongside with many other `AbstractReplyProducingMessageHandler`
options from the `ConsumerEndpointSpec`
* Refactor `HeaderEnricherSpec` to build `HeaderEnricher` and an
appropriate `MessageTransformingHandler` from the ctor to be able
to pick up an `adviceChain` automatically in the `ConsumerEndpointSpec.get()`
**Cherry-pick to 5.0.x**
JIRA: https://jira.spring.io/browse/INT-4392
JIRA: https://jira.spring.io/browse/INT-4391
JIRA: https://jira.spring.io/browse/INT-4393
Resolve class and leaf package tangles in:
- core (except DSL)
- file
- http
- move `StaticMessageHeaderAccessor` to root `integration` package
- move acknowledgment related classes to a new `acks` package
- move `MMIH` to `handler.support`, alongside argument resolvers, also `HandlerMethodArgumentResolversHolder`
- move `ErrorMessagePublisher` to `core`
- move `PatternMatchUtils` to `support.utils`
- move `MessageSourceManagement` to `support.management`
- remove direct reference to `MicrometerMetricsCapter` from `IntegrationManagementSupport`
- add new class `MetricsCaptorLoader`
- move `OperationsCallback` to inner interface in `RemoteFileOperations`
- move `HttpContextUtils` to `config`
* Simplify MetricsCaptor loading.
* Polishing
JIRA: https://jira.spring.io/browse/INT-4423
Add a facade on top of micrometer so the dependency can
be optional.t
Next iteration - PR comments
Checkstyle
More polishing; static classes etc.
checkstyle
Further polishing; docs; auto-register `MicrometerMetricsCaptor`.
Polishing - move captor load to a static method on the captor.
* Fix Checkstyle violation
* Some code style polishing
JIRA: https://jira.spring.io/browse/INT-4417
Provide more flexibility with bean naming when using EIP Annotations with
Java configuration.
* Polishing - PR Comments
* More polishing - remove `@Inherited`; disallow with more than one EIP annotation.
JIRA: https://jira.spring.io/browse/INT-4421
If a component invoked by an SPCA threw a `MessagingException` with no
`failedMessage`, the resulting ErrorMessage payload had no `failedMessage`.
The `UnicastingDispatcher` had a check for this so it wasn't an issue
as long as at least one `DirectChannel` was between the poller and the
component.
Promote the wrapping code to `IntgrationUtils` and invoke it from
places that blindly rethrew `MessagingException`s.
