Fixes https://github.com/spring-projects/spring-integration/issues/2776
Also see https://github.com/spring-projects/spring-integration/issues/2777
- reset the filter for the current file if the fetch fails
- implement `Lifecycle` and clear the `toBeReceived` queue and corresponding
filter entries
* Polishing - PR Comments **cherry-pick to all supported**
* Polishing
# Conflicts:
# spring-integration-file/src/test/java/org/springframework/integration/file/remote/StreamingInboundTests.java
# spring-integration-ftp/src/test/java/org/springframework/integration/ftp/inbound/FtpStreamingMessageSourceTests.java
# spring-integration-jdbc/src/test/java/org/springframework/integration/jdbc/StoredProcJavaConfigTests.java
* GH-2727: Ensure JDBC queries are logged
Fixes: https://github.com/spring-projects/spring-integration/issues/2727
The `JdbcTemplate` logs a message for the sql to execute when it is
supplied by the `QueryProvider`.
* Refactor `JdbcPollingChannelAdapter` to use new introduced internal
`PreparedStatementCreatorWithMaxRows` with the `QueryProvider`
* Refactor `JdbcMessageHandler` to instantiate a `generatedKeysStatementCreator`
based on the `PreparedStatementCreatorFactory`
* Verify in the `JdbcOutboundGatewayParserTests` that both fixes logs
sql queries properly
* * Implement all the delegate interfaces for the `PreparedStatementCreatorWithMaxRows`
JIRA: https://jira.spring.io/browse/INT-4574
Some RDBMS don't allow have nullable columns for primary keys
* Add NOT NULL for all the columns involved in primary keys
* Enable ModifierOrderCheck Checkstyle rule
* Fix violations for `static` and `abstract` modifier
* Remove redundant code in the `TcpNioConnection`
* Mark `connectionFactoryName` as `@Nullable` in the `TcpConnectionSupport`
ctor and its inheritors
* Fix some smells according IDEA suggestions in the affected classes
* This should fix some Sonar smells as well
* * Fix `HeaderMapperTests`
* * Polishing `TcpConnection` code style and fix Javdocs
* Fix wrong import order in tests
* Polishing for not closed application contexts
* Use diamonds
* Use proper assertion operators
* Remove redundant classes
* Improve performance in JPA tests
* Fix typos
* INT-4544: Allow runtime MBeans (un)registered
JIRA: https://jira.spring.io/browse/INT-4544
* Fix `SourcePollingChannelAdapterFactoryBean` to register an
`outputChannelName` into the `SourcePollingChannelAdapter` for late
binding, especially in case of dynamic `IntegrationFlow` registrations.
This must be back-ported to `5.0.x`
* Implement `DestructionAwareBeanPostProcessor` in the
`IntegrationMBeanExporter` for the runtime beans tracking, e.g. via
dynamic `IntegrationFlow` registrations
* Refactor `IntegrationMBeanExporter` for static and dynamic beans
registration and destruction
* Remove unused properties; introduce some new for tracking beans and
their relationship
* * Move `IntegrationMBeanExporter.registerProducer()` after
`postProcessAfterInitialization()`
* Refactor JMX test configurations to reuse MBeanServer as much as
possible and destroy the server whenever it is necessary
* * More JMX tests polishing
* * And more JMX tests polishing
* * `try...catch` in the `IntegrationMBeanExporter.postProcessAfterInitialization()`
to avoid breaking changes for the current GA phase
* Do not unregister those MBeans explicitly which weren't created at
runtime
* * Log ObjectNames in the `MBeanExporterIntegrationTests`
* * Fix `MessageMetricsAdviceTests` to rely on the application context
* Do not use a cst in the `StandardIntegrationFlowContext` for
`BeanFactory`, but an explicit `BeanDefinitionRegistry`
* Extract targets from proxies in the
`IntegrationMBeanExporter.postProcessAfterInitialization()`
* Remove logging in the `MBeanExporterIntegrationTests`
* * Fix `NotificationListeningMessageProducerTests` to reuse existing `MBeanServer`
* * Change JMX `domain` in the `DslMBeanTests` to `dsl` do not clash with
similar in the `foo` in the `MessageSourceTests`
* Use `MBeanServer` bean in the `Int2307Tests`
* Use JUnit 5ctor injection injection in the `MessageMetricsAdviceTests`
* * Fix JMX tests do not reuse existing `MBeanServer` and make them rely
on the server provided by the managed `MBeanServerFactoryBean` which
destroys a server on application context close
* Fix `StoredProcJmxManagedBeanTests-context.xml` to use `MBeanServer`
from the `<context:mbean-server>`
* INT-4381: MessageSources refactoring
JIRA: https://jira.spring.io/browse/INT-4381
* Make all the out-of-the-box `MessageSource` implementations based
on the `AbstractMessageSource`
* Fix `JdbcPollingChannelAdapterIntegrationTests` for sporadic failure
because of `fixed-rate` for the poller
* Fix HeaderEnricherTests race condition
The `errorChannel()` expect an error in the `testErrorChannel` after
`1000` ms, but at the same time the `poller` in configured for the
`3000` ms.
* Increase all the timeouts for replies
* Decrease a `fixed-delay` on the `poller`
* Some other code style polishing for the `HeaderEnricherTests`
JIRA https://jira.spring.io/browse/INT-4490
The `DefaultLockRepository.prefix` can be configured with the schema name.
And this one can be case-sensitive.
* Change format hint from the `%S` to `%s` to avoid upper-casing.
**Cherry-pick to 5.0.x and 4.3.x**
JIRA: https://jira.spring.io/browse/INT-2260
Having a feedback about confusing with the `max-rows-per-poll` property
name and its responsibility it would be better do not mention `per-poll`
at all
* Deprecate `max-rows-per-poll` in favor of new `max-rows`
* Some code style polishing, tests improvements
* Docs polishing on the matter
* Add `What's New` bullet
* Optimize `maxRows` logic
* Document vendor-specific native SELECT limiting options
* Raise warning in the parsers about deprecated `max-rows-per-poll`
Doc polishing
* GH-2467: JdbcLockRegistry should retry on DeadlockLoserDataAccessException
Fixes#2467
MySQL 5.7.15 introduced setting `innodb_deadlock_detect` (enabled by
default). As a result MySQL JDBC driver throws
`DeadlockLoserDataAccessException` when deadlock is detected.
`JdbcLockRegistry` doesn't handle it causing lock to be lost.
* Retry `doLock()` on data access deadlock instead of loosing the lock
* Use TransientDataAccessException instead of derived exceptions
JIRA: https://jira.spring.io/browse/INT-4477
When running multiple roles, it is useful to be able to determine the
role for a particular context, e.g. in an event.
* Docs
**cherry-pick to 5.0.x**
https://build.spring.io/browse/INT-FATS5IC-497
Fixes `JdbcMessageStoreTests` sporadic failure where there is no
guarantee that message are going to be polled from the group the same
order they've been inserted
Cherry-pick to `5.0.x`
We don't change JDBC scripts too often and even if we do that, it's not
so hard to copy/paste the script throughout all the supported platforms
instead of rely on the pretty old VPP tool and suffer from performance
degradation during regular day-to-day development.
Plus there is some bug in the VPP when it generates a script for the
Derby, so we end up with erroneous braces in the sentence:
```
MESSAGE_SEQUENCE BIGINT NOT NULL GENERATED ALWAYS AS IDENTITY (START WITH 1, INCREMENT BY 1),
```
I have to remove them manually each time I switch between Gradle versions
Looks like Derby is much slower than H2, plus it doesn't look so stable
having so much sporadic failures.
Looks like moving to H2 here saves for us `500 ms` test working time
**Cherry-pick to 5.0.x**
* Upgrade to Gradle 4.7 and others, polishing build
* Fix `MqttAdapterTests` according requirement of the latest Mockito
* Move `int-derby.properties` to the `/resources` from `/java`
* Fix Checkstyle violations
The message in the `QueueChannel` appears for consuming a bit earlier
than TX is committed
* Introduce `afterCommitLatch` into the test verify the state
when TX is really committed and data is removed from DB
**Cherry-pick to 5.0.x and 4.3.x**
Looks like we have started to fail much often
since upgrade to the `Derby-10.14.1.0`:
cb0d43db6b
That is also related to this https://jira.spring.io/browse/INT-4445
* Remove all the `drop` mentioning for Derby scripts -
it just doesn't support `IF EXISTS`.
More over we don't need to worry about drops since each test now starts
its own fresh embedded DB
**Cherry-pick to 5.0.x**
JIRA: https://jira.spring.io/browse/INT-4447
When we get an exception during `this.lock.unlock()`, we don't revoke
leadership.
In case of external resource (e.g. JDBC) this may cause a race condition
when the second candidate is selected as leader when connection comes back
* Catch `this.lock.unlock()` exceptions and log them under DEBUG.
This way we proceed to the `handleRevoked()` logic
**Cherry-pick to 5.0.x and 4.3.x**
JIRA: https://jira.spring.io/browse/INT-4445
Looks like `/dataSource` temporary directory is busy in between different
tests.
The thought is like a high-frequently polling endpoint keeps DB resource
from removing.
* Explicitly `stop()` endpoint in the `JdbcMessageStoreChannelTests`
* Optimize
`JdbcMessageStoreChannelTests.testSendAndActivateTransactionalSend()`
to rely on the short `1` millisecond TX timeout.
This safes for us 10 seconds of the tests executions.
* Configure all the embedded DB beans for the `ignore-failures="ALL"`
as a fallback option if polling endpoint is not a cause of the concurrent
resource access.
**Cherry-pick to 5.0.x**
https://build.spring.io/browse/INT-FATS5IC-457
The H2 `1.4.197` has fixed some issues
(in particular https://github.com/h2database/h2database/issues/178)
and introduced some breaking changes for us, although they are
reasonably good.
* Fix `JdbcOutboundGatewayParserTests` to use correct identifiers
for returned generated keys instead of previously used `SCOPE_IDENTITY()`
**Cherry-pick to 5.0.x and 4.3.x**
https://build.spring.io/browse/INT-AT42SIO-820/
The test configs use `fixed-rate="100"`.
This way when Inbound Channel Adapter is declared earlier in the
config than `<jdbc:embedded-database>`, there is a chance that we start
to poll DB before it is really initialized.
* Reorder config so embedded DB is initialized before an
Inbound Channel Adapter
**Cherry-pick to 5.0.x, 4.3.x and 4.2.x**
https://build.spring.io/browse/INT-FATS5IC-430
Looks like there is a race condition when our polling rate around
data base is too fast and we have access to the DB files even during
application context close.
* Stop polling channel adapter in the tests explicitly after test methods
* Increase some tests performance decreasing timeouts to wait
* Upgrade to Reactor-3.1.5, AssertJ-3.9.1, Derby-10.14.1.0 and some
Gradle plugins
* Upgrade to `reactor-netty-0.7.5`
* Remove workaround from the `StompServerIntegrationTests`
* Upgrade to Spring Data Kay SR5
* Upgrade to Spring Security 5.0.3
* Increase timeouts and performance in the `StreamTransformerParserTests`
JIRA: https://jira.spring.io/browse/INT-4396
When target distributed `Lock` implementation throws an exception, e.g.
in case of no connection to the service, the `LockRegistryLeaderInitiator`
exists the loop and can come back to the elections only after restart
* Catch all the exception on `this.lock.tryLock()` and resubmit
`LeaderSelector` for a new locking cycle if
`LockRegistryLeaderInitiator.isRunning()` and `InterruptedException`
* Remove diagnostics from the `JdbcLockRegistryLeaderInitiatorTests`
since this fix confirms that we just didn't have a reconnect logic before
when this test failed sporadically
**Cherry-pick to 4.3.x**
https://build.spring.io/browse/INT-MJATS41-1242
When different `DefaultLockRepository` instances use the same client id,
there is a possibility that they will update the same row in the table.
This way we have a chance that not only one obtains a lock and try to
add a value to the collection.
* Remove the test-case for the same client id as non-stable and even
dangerous by the the distributed lock purpose
* Use `tryLock(Long.MAX_VALUE)` to really ensure the wait behavior
during the concurrent loop
* Use `20` for thread pool to align with the tasks amount
**Cherry-picked to 4.3.x**
https://build.spring.io/browse/INT-MASTER-901
The `JdbcLockRegistryDifferentClientTests.testOnlyOneLock()` relies
on the `ArrayList.isEmpty()` state to proceed with the logic.
But since the `ArrayList.size` property is not `volatile`, there is no
guarantee for the proper state in the multi-threaded environment like
we have in this test-case.
* Replace `ArrayList` in the test with the `LinkedBlockingQueue` which
already rely on the `AtomicInteger` for the `size` property
* Fix `JdbcLockRegistry` tests to shutdown used `ExecutorService` s
to ensure set free threads after test suite execution.
**Cherry-pick to 4.3.x**
* INT-4376: Upgage to Log4J 2
JIRA: https://jira.spring.io/browse/INT-4376
* Deprecate Log4J 1.x components in favor of newly added
a `Log4j2LevelAdjuster` JUnit `@Rule`
* Update all the logging configuration to Log4J 2
* Polishing after testing
JIRA: https://jira.spring.io/browse/INT-2576
* Do not fetch message ids and then iterate over them to remove
one by one - just use an appropriate `DELETE` query for all messages
JIRA: https://jira.spring.io/browse/INT-4291
* Fix code style in the `ExpressionEvaluatingSqlParameterSourceFactory`
* Remove redundant
`ExpressionEvaluatingSqlParameterSourceFactory.registerSqlTypes` in
favor of the direct access to the `sqlParametersTypes` from the
`ExpressionEvaluatingSqlParameterSource` ctor
* Reformat code style in the `pollingWithSelectParameterSourceJdbcInboundChannelAdapterTest.xml`
* Fix code style in the `ExpressionEvaluatingSqlParameterSourceFactoryTests`
* Polishing `whats-new.adoc` for proper formatting and single link from
the section to the particular chapter
* Fix formatting and wording in the `jdbc.adoc`
JIRA: https://jira.spring.io/browse/INT-4291
* Fix checkstyle trailing space
*Use correct way to constants in spring xml and fix test case
* Polishing - Update javadoc and rename method name to relate more to the behavior
* Upgrade to SF-5.0.1, Reactor-3.1.1, some other minor upgrades
* Fix `HttpDslTests` and `WebFluxDslTests` for the latest SS compatibility
* Fix JavaDoc in the `ChannelMessageStorePreparedStatementSetter`
JIRA: https://jira.spring.io/browse/INT-4081
* Rename `MessageGroupPreparedStatementSetter` to `ChannelMessageStorePreparedStatementSetter`
* Add JavaDocs to the `ChannelMessageStorePreparedStatementSetter`
* Polishing `JdbcChannelMessageStore` JavaDocs; remove unused `BeanFactory`
* Polishing Docs for typos and proper text flow
JIRA: https://jira.spring.io/browse/INT-4081
* Add `MessageGroupPreparedStatementSetter` to let end-user to override
the message insertion logic
* Add tests and Docs
JIRA: https://jira.spring.io/browse/INT-4357
Updated based on code review.
Updated adocs
*Polishing code style, typos
* Add protection from NPE when `LockRegistryLeaderInitiator` isn't
supplied with the `leaderEventPublisher`
* Add `What's New` note
JIRA: https://jira.spring.io/browse/INT-4342
Use similar code to Spring AMQP to add white list support for Integration's
use of the `DeserializingMessageConverter`; introduce the `WhiteListDeserializingMessageConverter`.
Polishing
Missed this change in PR.
Fix XSD attribute
