JIRA: https://jira.spring.io/browse/INT-4486
The `SmartLifecycle.stop(Runnable callback)` must always call the
`callback` in the end independently of the internal state
* Revise all the `SmartLifecycle` implementations for the proper
`callback` handling
**Cherry-pick to 5.0.x and 4.3.x**
* GH-2467: JdbcLockRegistry should retry on DeadlockLoserDataAccessException
Fixes#2467
MySQL 5.7.15 introduced setting `innodb_deadlock_detect` (enabled by
default). As a result MySQL JDBC driver throws
`DeadlockLoserDataAccessException` when deadlock is detected.
`JdbcLockRegistry` doesn't handle it causing lock to be lost.
* Retry `doLock()` on data access deadlock instead of loosing the lock
* Use TransientDataAccessException instead of derived exceptions
# Conflicts:
# spring-integration-jdbc/src/main/java/org/springframework/integration/jdbc/lock/JdbcLockRegistry.java
Fixes https://github.com/spring-projects/spring-integration/issues/2471
Call `close()` on the client whenever the connection is lost or can't be
established, to release resources in the client.
**cherry-pick to 5.0.x, 4.3.x**
(cherry picked from commit f9cea64)
# Conflicts:
# spring-integration-mqtt/src/test/java/org/springframework/integration/mqtt/MqttAdapterTests.java
JIRA: https://jira.spring.io/browse/INT-4482
The outer try/catch sends an `ErrorMessage` for all exceptions; it should
only do so for `MessageConversionException`. Integration flow exceptions
will have been already handled by `MessageProducerSupport`.
Also, populate the raw message header consistently - previously it only
was populated for flow exceptions. Although the LEFE contains the raw
message, it should be in the `ErrorMessage` header for consistency.
**cherry-pick to 5.0.x, 4.3.x**
* Polishing - PR Comments
# Conflicts:
# spring-integration-amqp/src/main/java/org/springframework/integration/amqp/inbound/AmqpInboundChannelAdapter.java
# spring-integration-amqp/src/test/java/org/springframework/integration/amqp/dsl/AmqpTests.java
JIRA: https://jira.spring.io/browse/INT-4481
* Now each generated jar has `licence.txt` and `notice.txt` entries in
its `META-INF`
* Copy `notice.txt` from the Spring Framework with an appropriate
polishing
* INT-4471: PubSubChannel: Add errorHandler warn
JIRA: https://jira.spring.io/browse/INT-4471
* When an `Executor` is not provided, log warn that the provided
`ErrorHandler` is ignored.
**Cherry-pick to 5.0.x and 4.3.x**
* * Polish warn message
JIRA: https://jira.spring.io/browse/INT-4466
When we schedule group for force complete in the
`AbstractCorrelatingMessageHandler`, we don't track a group `timestamp`
and its `lastModified` before the scheduled task.
This way, in the cluster environment, we may schedule several tasks
for different messages and the first started may release the group too
early.
Just because we extract a `MessageGroup` from the store already in the
task per se.
* Propagate the actual `timestamp` and `lastModified` from group before
scheduling task.
Compare these value with the actual group metadata in the
`processForceRelease()` before performing real `forceRelease()`.
This way we restore behavior before fixing memory leak, when we
propagated full `MessageGroup` to the scheduled task
* Implement `ResequencingMessageHandler.getComponentType()` for
consistency
* Remove unnecessary overhead with the `volatile` on many
`AbstractCorrelatingMessageHandler` properties, which hardly ever can be
changed at runtime
**Cherry-pick to 5.0.x and 4.3.x**
JIRA: https://jira.spring.io/browse/INT-4465
There is a one second delay before a socket close is propagated if there is an active
assembler. This is generally only a problem with deserializers that use EOF to signal
message end (such as the `ByteArrayElasticRawDeserializer`).
Attempt to insert an EOF marker into the buffer queue so that the `getNextBuffer()` will
exit immediately on `close()` if it is blocked awaiting a buffer.
**cherry-pick to 5.0.x, 4.3.x**
(cherry picked from commit d9186f1)
# Conflicts:
# spring-integration-ip/src/test/java/org/springframework/integration/ip/tcp/connection/TcpNioConnectionTests.java
JIRA: https://jira.spring.io/browse/INT-4463
Certain options, such as `maxInFlight` were not exposed.
Deprecate the setters on the factory and allow the user to inject a pre-configured
`MqttConnectOptions`, thus making all (and any new) properties available to be
configured.
# Conflicts:
# spring-integration-mqtt/src/test/java/org/springframework/integration/mqtt/MqttAdapterTests.java
# Conflicts:
# spring-integration-mqtt/src/test/java/org/springframework/integration/mqtt/MqttAdapterTests.java
# src/reference/asciidoc/mqtt.adoc
JIRA: https://jira.spring.io/browse/INT-4464
A `ZookeeperMetadataStore.get()` is based on the `this.cache` variable.
This one is initialized in the `start()`.
* Assert `isRunning()` in the `get()` before using.
**Cherry-pick to 5.0.x and 4.3.x**
JIRA: https://jira.spring.io/browse/INT-4461
Convert `byte[]` to `String` using `URF-8` by default.
Add optional evaluate methods to JsonPathUtils with a Charset to use when
converting `byte[]` to `String`.
This is not currently exposed using SpEL. It can be done, but probably not worth
the effort until somebody asks for it.
**cherry-pick to 5.0.x, 4.3.x**
* Use `BAIS`
(cherry picked from commit e987387)
# Conflicts:
# spring-integration-core/src/main/java/org/springframework/integration/json/JsonPathUtils.java
# spring-integration-core/src/test/java/org/springframework/integration/json/JsonPathTests.java
JIRA: https://jira.spring.io/browse/INT-4455
When using java config, `setChannelMapping` NPEs due to no AC.
**cherry-pick to 4.3.x, and to master, removing AC check**
* Polishing - PR comment
# Conflicts:
# spring-integration-core/src/main/java/org/springframework/integration/router/ErrorMessageExceptionTypeRouter.java
The message in the `QueueChannel` appears for consuming a bit earlier
than TX is committed
* Introduce `afterCommitLatch` into the test verify the state
when TX is really committed and data is removed from DB
**Cherry-pick to 5.0.x and 4.3.x**
(cherry picked from commit e4e4ee0)
# Conflicts:
# spring-integration-jdbc/src/test/java/org/springframework/integration/jdbc/JdbcMessageStoreChannelTests-context.xml
# spring-integration-jdbc/src/test/java/org/springframework/integration/jdbc/JdbcMessageStoreChannelTests.java
JIRA: https://jira.spring.io/browse/INT-4447
When we get an exception during `this.lock.unlock()`, we don't revoke
leadership.
In case of external resource (e.g. JDBC) this may cause a race condition
when the second candidate is selected as leader when connection comes back
* Catch `this.lock.unlock()` exceptions and log them under DEBUG.
This way we proceed to the `handleRevoked()` logic
**Cherry-pick to 5.0.x and 4.3.x**
# Conflicts:
# spring-integration-core/src/main/java/org/springframework/integration/support/leader/LockRegistryLeaderInitiator.java
https://build.spring.io/browse/INTSAMPLES-NIGHTLY-2413
When we fail with the
`java.net.BindException: Address already in use (Bind failed)` in the
`TcpNetServerConnectionFactory.run()`, the `serverSocket` property
remains `null` and we get `NPE` in the `catch` block trying to `close()`
the socket.
* Call `stop()` instead which has all the required protections.
**Cherry-pick to 5.0.x and 4.3.x**
https://build.spring.io/browse/INT-FATS5IC-457
The H2 `1.4.197` has fixed some issues
(in particular https://github.com/h2database/h2database/issues/178)
and introduced some breaking changes for us, although they are
reasonably good.
* Fix `JdbcOutboundGatewayParserTests` to use correct identifiers
for returned generated keys instead of previously used `SCOPE_IDENTITY()`
**Cherry-pick to 5.0.x and 4.3.x**
(cherry picked from commit 7b86ca7)
# Conflicts:
# build.gradle
https://build.spring.io/browse/INT-AT42SIO-820/
The test configs use `fixed-rate="100"`.
This way when Inbound Channel Adapter is declared earlier in the
config than `<jdbc:embedded-database>`, there is a chance that we start
to poll DB before it is really initialized.
* Reorder config so embedded DB is initialized before an
Inbound Channel Adapter
**Cherry-pick to 5.0.x, 4.3.x and 4.2.x**
(cherry picked from commit da6f47c)
# Conflicts:
# spring-integration-jdbc/src/test/java/org/springframework/integration/jdbc/config/pollingWithSelectParameterSourceJdbcInboundChannelAdapterTest.xml
JIRA: https://jira.spring.io/browse/INT-4441
Possible concurrent updates to `AbstractCorrelatingMessageHandler.groupIds` and
`expireGroupScheduledFutures`.
(cherry picked from commit f9d5198)
# Conflicts:
# spring-integration-core/src/main/java/org/springframework/integration/aggregator/AbstractCorrelatingMessageHandler.java
For better traceability for errors during lock acquiring add DEBUG
logging message in the `catch` block before returning back to the main
loop for the next acquiring attempt
**Cherry-pick to 5.0.x and 4.3.x**
JIRA: https://jira.spring.io/browse/INT-4430
When Iterator-based `FileSplitter` splits the file, and an exception
throws in downstream flow (in the same thread), the exception propagates
to the caller leaving underlying file reader opened.
This commit changes `AbstractMessageSplitter` the way, that,
when any exception happens, if Iterator implements `java.io.Closeable`,
its `close()` method will be called before propagating exception.
Also `FileSplitter`'s underlying iterator implements `Closeable` now.
* Make `CloseableIterator` to follow `Closeable` contract.
Now `CloseableIterator.close()` declares `IOException` and can be used
as base interface for `FunctionIterator`.
* Adjust tests.
Adjust tests to reflect the fact that we call `close()` on the reader
one more time in the end of iterator.
**Cherry-pick to 5.0.x and 4.3.x**
(cherry picked from commit 7f25cba)
# Conflicts:
# spring-integration-core/src/main/java/org/springframework/integration/splitter/AbstractMessageSplitter.java
# spring-integration-core/src/main/java/org/springframework/integration/util/FunctionIterator.java
# spring-integration-file/src/main/java/org/springframework/integration/file/splitter/FileSplitter.java
# spring-integration-file/src/test/java/org/springframework/integration/file/remote/StreamingInboundTests.java
# spring-integration-file/src/test/java/org/springframework/integration/file/splitter/FileSplitterTests.java
https://build.spring.io/browse/INT-MJATS41-1279
Looks like there is a race condition when our polling rate around
data base is too fast and we have access to the DB files even during
application context close.
* Stop polling channel adapter in the tests explicitly after test methods
* Increase some tests performance decreasing timeouts to wait
* Increase timeouts and performance in the `StreamTransformerParserTests`
# Conflicts:
# build.gradle
# spring-integration-jdbc/src/test/java/org/springframework/integration/jdbc/JdbcMessageStoreChannelIntegrationTests-context.xml
# spring-integration-jdbc/src/test/java/org/springframework/integration/jdbc/JdbcMessageStoreChannelIntegrationTests.java
# spring-integration-jdbc/src/test/java/org/springframework/integration/jdbc/JdbcMessageStoreChannelOnePollerIntegrationTests-context.xml
# spring-integration-jdbc/src/test/java/org/springframework/integration/jdbc/JdbcMessageStoreChannelOnePollerIntegrationTests.java
# spring-integration-stomp/src/test/java/org/springframework/integration/stomp/client/StompServerIntegrationTests.java
Cherry-picked from cb0d43db6b
JIRA: https://jira.spring.io/browse/INT-4402
Make `GlobalChannelInterceptorProcessor` as a `BeanPostProcessor`,
so it can apply global `ChannelInterceptor`s to any initialized channel
beans, even those created at runtime, like in case of dynamic flows with
Java DSL
**Cherry-pick to 4.3.x**
* Add `postProcessDynamicBeans` global property with `false` by default
* Rely on the `postProcessDynamicBeans` property in the
`GlobalChannelInterceptorProcessor.postProcessAfterInitialization()`
* Add `postProcessDynamicBeans=true` to the
`GlobalChannelInterceptorTests-context.xml` to be sure in the test-case
that option works
* Document the option and effect from the global channel interceptors
JIRA: https://jira.spring.io/browse/INT-4396
When target distributed `Lock` implementation throws an exception, e.g.
in case of no connection to the service, the `LockRegistryLeaderInitiator`
exists the loop and can come back to the elections only after restart
* Catch all the exception on `this.lock.tryLock()` and resubmit
`LeaderSelector` for a new locking cycle if
`LockRegistryLeaderInitiator.isRunning()` and `InterruptedException`
* Remove diagnostics from the `JdbcLockRegistryLeaderInitiatorTests`
since this fix confirms that we just didn't have a reconnect logic before
when this test failed sporadically
**Cherry-pick to 4.3.x**
# Conflicts:
# spring-integration-core/src/main/java/org/springframework/integration/support/leader/LockRegistryLeaderInitiator.java
# spring-integration-jdbc/src/test/java/org/springframework/integration/jdbc/leader/JdbcLockRegistryLeaderInitiatorTests.java
JIRA: https://jira.spring.io/browse/INT-4394
Looks like Jackson has became much smarter and now it stores
the target type for the object, not returned interface.
We can't use Jackson annotations on the Framework classes and
we can't restrict the `messagingAwareMapper()` `ObjectMapper` just
to use fields for all the object passed through it.
As a compromise solution we shouldn't use
`Collections.unmodifiableList()` in the getter.
The copy of the internal collection is pretty enough to protect
our object from mutation.
**Cherry-pick to 4.3.x (excluding `build.gradle` change)**
# Conflicts:
# build.gradle
https://build.spring.io/browse/INT-FATS5IC-400
When we run several concurrent builds (e.g. CI) and use the same shared
Redis server we may end up with the case when one process reads data
populated by another because we use the same key (groupId in our case)
* Fix `RedisMessageGroupStoreTests` to use `UUID.randomUUID()`
for the `groupId`
**Cherry-pick to 4.3.x**
# Conflicts:
# spring-integration-redis/src/test/java/org/springframework/integration/redis/store/RedisMessageGroupStoreTests.java
