JIRA: https://jira.spring.io/browse/INT-4481
* Now each generated jar has `licence.txt` and `notice.txt` entries in
its `META-INF`
* Copy `notice.txt` from the Spring Framework with an appropriate
polishing
https://build.spring.io/browse/INT-MASTER-1058/
When we destroy manually registered `IntegrationFlow`, we need to
count with sub-flows and iterate all the dependant beans recursively
for full flow removal
**Cherry-pick to 5.0.x**
* INT-4471: PubSubChannel: Add errorHandler warn
JIRA: https://jira.spring.io/browse/INT-4471
* When an `Executor` is not provided, log warn that the provided
`ErrorHandler` is ignored.
**Cherry-pick to 5.0.x and 4.3.x**
* * Polish warn message
JIRA: https://jira.spring.io/browse/INT-4478
When we use a `GenericHandler` and an incoming payload is a `Map`, we
copy it into both arguments into the `Object` for the payload
and `Map` for the headers.
This way we lose headers in the target lambda
* Check a size of arguments on the target lambda and don't set a
payload into the `Map` argument if we have more than 1 arguments
**Cherry-pick to 5.0.x**
JIRA: https://jira.spring.io/browse/INT-4477
When running multiple roles, it is useful to be able to determine the
role for a particular context, e.g. in an event.
* Docs
**cherry-pick to 5.0.x**
(cherry picked from commit 73dbdb8)
* Fix LockRegLeaderIn for interrupted Thread.sleep
https://build.spring.io/browse/INT-FATS5IC-517
If current thread is interrupted, the `Thread.sleep()` interrupts
immediately.
In the catch block of the main loop in the `LockRegistryLeaderInitiator`
we have such a dangerous `sleep()` and don't restart election in this
candidate any more
* Move `Thread.sleep()` to else after checking the current thread for
interrupted state
* Remove `LongRunningIntegrationTest` rule from the
`RedisLockRegistryLeaderInitiatorTests` since it now works much faster
after proper `busy-wait` handling
**Cherry-pick to master**
* Ignore interruption on the sleep i catch and move on with loop
JIRA: https://jira.spring.io/browse/INT-4473
Previously, dynamic registration of integration flows with components configured
with the same `id` would fail with duplicate bean names.
Add `useFlowIdAsPrefix()` to the registration builder to enable the option.
Then, in the BPP, check the flag before naming the beans.
**cherry-pick to 5.0.x**
* Polishing according PR comments
* Widen `flowNamePrefix` responsibility in the
`IntegrationFlowBeanPostProcessor` since we may have many other
components in the dynamic flow with the same id, not only consumer
endpoints
# Conflicts:
# spring-integration-core/src/main/java/org/springframework/integration/config/dsl/IntegrationFlowBeanPostProcessor.java
# spring-integration-core/src/main/java/org/springframework/integration/dsl/context/IntegrationFlowContext.java
# spring-integration-core/src/main/java/org/springframework/integration/dsl/context/StandardIntegrationFlowContext.java
# src/reference/asciidoc/dsl.adoc
When we register `IntegrationFlow` s concurrently at runtime, we may
end up with the problem when we register the same object with the same
bean name, but in different places.
Or when we turn off bean overriding, we end up with the exception that
bean with the name already registered
* Wrap `IntegrationFlow` bean registration in the
`StandardIntegrationFlowContext` into the `Lock` when its bean name
is generating
* Make `StandardIntegrationFlowContext.registry` as `ConcurrentHashMap`
to avoid `ConcurrentModificationException` during `put()` and `remove()`
* Fix concurrency for beans registration with the generation names in
the `IntegrationFlowBeanPostProcessor` using an `IntegrationFlow` id
as a prefix for uniqueness.
**Cherry-pick to 5.0.x**
Fix generated bean name in the WebFluxDslTests
Use only single `Lock` in the `StandardIntegrationFlowContext`:
we don't need a fully blown `LockRegistry` there anymore since we have
only one synchronization block there and it is always around the same
type
* Add `What's New` note, and mention changes in the `dsl.adoc`
Minor doc polishing.
# Conflicts:
# spring-integration-core/src/main/java/org/springframework/integration/dsl/context/StandardIntegrationFlowContext.java
# spring-integration-core/src/test/java/org/springframework/integration/dsl/manualflow/ManualFlowTests.java
# src/reference/asciidoc/dsl.adoc
# src/reference/asciidoc/whats-new.adoc
https://build.spring.io/browse/INT-FATS5IC-509
Looks like `busyWait` period as `5 secs` is too aggressive and may be
two or more cycles pass until the election happens.
At this time the waiting for the `granted` latch in `10 secs`
may expire already, therefore we fail in the assertion
* Decrease `busyWait` to the `1 secs` to let the election cycle to pass
quickly
* Increase latch wait time to the `20 secs`
**Cherry-pick to 5.0.x**
(cherry picked from commit 59d6c27)
JIRA: https://jira.spring.io/browse/INT-4466
When we schedule group for force complete in the
`AbstractCorrelatingMessageHandler`, we don't track a group `timestamp`
and its `lastModified` before the scheduled task.
This way, in the cluster environment, we may schedule several tasks
for different messages and the first started may release the group too
early.
Just because we extract a `MessageGroup` from the store already in the
task per se.
* Propagate the actual `timestamp` and `lastModified` from group before
scheduling task.
Compare these value with the actual group metadata in the
`processForceRelease()` before performing real `forceRelease()`.
This way we restore behavior before fixing memory leak, when we
propagated full `MessageGroup` to the scheduled task
* Implement `ResequencingMessageHandler.getComponentType()` for
consistency
* Remove unnecessary overhead with the `volatile` on many
`AbstractCorrelatingMessageHandler` properties, which hardly ever can be
changed at runtime
**Cherry-pick to 5.0.x and 4.3.x**
JIRA: https://jira.spring.io/browse/INT-4467
The package tangles is caused by the explicit classes declarations.
* Since we use only method definitions from those classes, there is just
enough to use class names and use reflection to get class objects and
then get methods from them
_The fix is fully compatible for back-porting_
**Cherry-pick to 5.0.x**
JIRA: https://jira.spring.io/browse/INT-4465
There is a one second delay before a socket close is propagated if there is an active
assembler. This is generally only a problem with deserializers that use EOF to signal
message end (such as the `ByteArrayElasticRawDeserializer`).
Attempt to insert an EOF marker into the buffer queue so that the `getNextBuffer()` will
exit immediately on `close()` if it is blocked awaiting a buffer.
**cherry-pick to 5.0.x, 4.3.x**
(cherry picked from commit d9186f1)
https://build.spring.io/browse/INT-SI50X-JOB1-56
We can't wait for the latch in the interruptable code flow;
we can't have a round-robing election guarantees.
* Add `Thread.sleep(LockRegistryLeaderInitiator.this.busyWaitMillis)`
to the `LockRegistryLeaderInitiator` when we restart the main task
* Remove latches waiting and thread shifting from the
`RedisLockRegistryLeaderInitiatorTests`
* Use long `busyWaitMillis` for yielding initiator to let the second
candidate to be elected
**Cherry-pick to 5.0.x**
(cherry picked from commit 081d0d1)
https://build.spring.io/browse/INT-FATS5IC-501/
When we try to wait for the `Latch` in the interruptable code flow,
it is a fact that we step away from the waiting and end up with the
race condition downstream.
* Wrap `Latch` in the interruptable `publishOnRevoked()` code to the
`Executor.execute()`
* Remove `deleteTimeoutMillis` option from the `RedisLockRegistry`
since it doesn't make sense in the interruptable code.
* Add `RedisLockRegistry.setExecutor()` to allow to inject an external
`Executor`
* Add more debug logging into the `LockRegistryLeaderInitiator`
**Cherry-pick to 5.0.x**
(cherry picked from commit 51c4951)
https://build.spring.io/browse/INT-MASTER-1024
When the Lettuce Redis client catches an `InterruptedException`, it is
wrapped to the `RedisCommandInterruptedException`, therefore when we
catch an exception on our code level it is not an `InterruptedException`
anymore and we can't proceed in the loop because the tread is
`interrupted` already.
* Check the `interrupted` alongside with the `InterruptedException`
to restart a loop from a fresh thread.
**Cherry-pick to 5.0.x**
(cherry picked from commit fc47952)
JIRA: https://jira.spring.io/browse/INT-4462
When the HTTP request body is empty, the `HttpMessageReader` ends up
with the empty `Mono` which can't be evaluated to any reasonable value.
* Add fallback to `requestParams` when `Mono` for body is empty and
also when `payloadExpression` returns null
**Cherry-pick to 5.0.x**
JIRA: https://jira.spring.io/browse/INT-4463
Certain options, such as `maxInFlight` were not exposed.
Deprecate the setters on the factory and allow the user to inject a pre-configured
`MqttConnectOptions`, thus making all (and any new) properties available to be
configured.
# Conflicts:
# spring-integration-mqtt/src/test/java/org/springframework/integration/mqtt/MqttAdapterTests.java
JIRA: https://jira.spring.io/browse/INT-4464
A `ZookeeperMetadataStore.get()` is based on the `this.cache` variable.
This one is initialized in the `start()`.
* Assert `isRunning()` in the `get()` before using.
**Cherry-pick to 5.0.x and 4.3.x**
* Use Lettuce Redis client; Redis module fixes
The Lettuce client is based on the Netty and more stable, than Jedis
therefore we get a performance improvement for tests
(it saves us at about 30 seconds).
Also this client doesn't fail for me on Windows sporadically (very often)
with the `ConnectionClosedException`
* After switching to the Netty-based client, we expose the interrupted
Thread issue in the `LockRegistryLeaderInitiator`.
If we interrupted (expected behavior), we try to unlock calling
`RedisLockRegistry`, but Netty client reject our request because the
thread is interrupted, therefore we never delete the lock when we yield
our leadership.
Fix the issue with shifting a `RedisTemplate.delete()` operation to the
`ExecutorService` when the current thread is interrupted
* Allow to configure such an `ExecutorService` and timeout to wait for
the `submit()` result
* Refactor `RedisAvailableRule` and all the Redis tests do not expose
the target `RedisConnectionFactory` implementation.
* Make all the test-cases based on the `connectionFactory` created by
the `RedisAvailableTests.setupConnectionFactory()`
* Tweak some unnecessary timeouts and sleeps for better tests task
throughput
* Add a `Log4j2LevelAdjuster` into the `RedisLockRegistryLeaderInitiatorTests`
https://build.spring.io/browse/INT-FATS5IC-497
Fixes `JdbcMessageStoreTests` sporadic failure where there is no
guarantee that message are going to be polled from the group the same
order they've been inserted
Cherry-pick to `5.0.x`
(cherry picked from commit 664d67f)
JIRA: https://jira.spring.io/browse/INT-4461
Convert `byte[]` to `String` using `URF-8` by default.
Add optional evaluate methods to JsonPathUtils with a Charset to use when
converting `byte[]` to `String`.
This is not currently exposed using SpEL. It can be done, but probably not worth
the effort until somebody asks for it.
**cherry-pick to 5.0.x, 4.3.x**
* Use `BAIS`
(cherry picked from commit e987387)
* INT-4458: Do not expose recursive generics API
JIRA: https://jira.spring.io/browse/INT-4458
According Kotlin generics system restrictions we can't expose API
based on the recursive generics, like we have with Java DSL for JMS
* Replace explicit recursive generics on factory method with the wildcard (`?`).
This way both Kotlin and Java are able to instantiate target object and perform
the proper chain API auto-completion
* Fix `JmsInboundChannelAdapterSpec` to deal with provided `S` type for the
`configureListenerContainer()` - the way we can configure a `JmsDefaultListenerContainerSpec`
* Add `kotlin-spring` Gradle plugin to avoid extra `open` modificator on `@Bean` methods
* Add `JmsDslKotlinTests.kt`
**Cherry-pick to 5.0.x excluding Kotlin support**
* Fix `Amqp` DSL factory for recursive generics
* Polishing for Kotlin tests
JIRA: https://jira.spring.io/browse/INT-4456
Since Kotlin compiles lambdas different way than Java, they are not
synthetic classes at runtime anymore.
Therefore we fallback to the method invocation logic, but since Java 8
interfaces has `default` methods as well the `MessagingMethodInvokerHelper`
can't select the target method for invocation
* Use explicit `Function.apply()` for non-synthetic implementations
* Add `RouterDslTests` Kotlin-based test-case
**Cherry-pick to 5.0.x**
* Add more explicit methods for invokers
The `transactionWithCommitAndAdvices()` uses `SimpleRepeatAdvice`
which essentially performs `doPoll()` twice.
In this case we don't have enough messages in the `goodInputWithAdvice`
queue
* Add one more message to the `goodInputWithAdvice` queue during testing
* Increase all the timeouts
* Increase capacity in queues for the `transactionWithCommitAndAdvices()`
**Cherry-pick to 5.0.x and 4.3.x**
(cherry picked from commit 000f297)
JIRA: https://jira.spring.io/browse/INT-4351
The `WatchService` reacts to the events in the file system and keep
track ove the events until we poll them, e.g. via `listEligibleFiles()`
in the `WatchServiceDirectoryScanner`.
On the other hand the `SourcePollingChannelAdapter` calls the mentioned
`listEligibleFiles()` according its polling period.
At this moment the `FileListFilter` is applied to the polled files.
It may happen that `LastModifiedFileListFilter` can't accept too young
files yet and they are lost for the future consideration.
* To allow, for example, to retain young files by the
`LastModifiedFileListFilter` judgment for the future cycles add
`DiscardAwareFileListFilter` with the `DiscardCallback` support.
The `WatchServiceDirectoryScanner` now registers such a callback into
the filter and stores discarded files into the `filesToPoll` queue
for the future poll cycle.
**Cherry-pick to 5.0**
Fix compilation warnings
* Replace `DiscardCallback` with the plain `Consumer`
* Ensure uniqueness in the `WatchServiceDirectoryScanner` internal
queue via a `Set` implementation, since discard callback may be called
several times for the same file from the `CompositeFileListFilter`
according to its nature
* Add JavaDocs and Docs
Change `@since` to `5.0.5`
JIRA: https://jira.spring.io/browse/INT-4455
When using java config, `setChannelMapping` NPEs due to no AC.
**cherry-pick to 4.3.x, and to master, removing AC check**
* Polishing - PR comment
Looks like Derby is much slower than H2, plus it doesn't look so stable
having so much sporadic failures.
Looks like moving to H2 here saves for us `500 ms` test working time
**Cherry-pick to 5.0.x**
(cherry picked from commit 6dfe96d)
The message in the `QueueChannel` appears for consuming a bit earlier
than TX is committed
* Introduce `afterCommitLatch` into the test verify the state
when TX is really committed and data is removed from DB
**Cherry-pick to 5.0.x and 4.3.x**
(cherry picked from commit e4e4ee0)
Looks like we have started to fail much often
since upgrade to the `Derby-10.14.1.0`:
cb0d43db6b
That is also related to this https://jira.spring.io/browse/INT-4445
* Remove all the `drop` mentioning for Derby scripts -
it just doesn't support `IF EXISTS`.
More over we don't need to worry about drops since each test now starts
its own fresh embedded DB
**Cherry-pick to 5.0.x**
* INT-4446 Improve EmbeddedJsonHeadersMessageMapper
JIRA: https://jira.spring.io/browse/INT-4446
* Do not recreate message if not necessarily
* Do not let to generate `id` and `timestamp` if they are not mapped
* Use `smartMatch` to allow to configure negative patterns
* Introduce `PatternMatchUtils.smartMatchIgnoreCase()` for convenience
**Cherry-pick to 5.0.x**
* * Polishing `EmbeddedJsonHeadersMessageMapper`
JIRA: https://jira.spring.io/browse/INT-4447
When we get an exception during `this.lock.unlock()`, we don't revoke
leadership.
In case of external resource (e.g. JDBC) this may cause a race condition
when the second candidate is selected as leader when connection comes back
* Catch `this.lock.unlock()` exceptions and log them under DEBUG.
This way we proceed to the `handleRevoked()` logic
**Cherry-pick to 5.0.x and 4.3.x**
JIRA: https://jira.spring.io/browse/INT-4445
Looks like `/dataSource` temporary directory is busy in between different
tests.
The thought is like a high-frequently polling endpoint keeps DB resource
from removing.
* Explicitly `stop()` endpoint in the `JdbcMessageStoreChannelTests`
* Optimize
`JdbcMessageStoreChannelTests.testSendAndActivateTransactionalSend()`
to rely on the short `1` millisecond TX timeout.
This safes for us 10 seconds of the tests executions.
* Configure all the embedded DB beans for the `ignore-failures="ALL"`
as a fallback option if polling endpoint is not a cause of the concurrent
resource access.
**Cherry-pick to 5.0.x**
