The message in the `QueueChannel` appears for consuming a bit earlier
than TX is committed
* Introduce `afterCommitLatch` into the test verify the state
when TX is really committed and data is removed from DB
**Cherry-pick to 5.0.x and 4.3.x**
(cherry picked from commit e4e4ee0)
# Conflicts:
# spring-integration-jdbc/src/test/java/org/springframework/integration/jdbc/JdbcMessageStoreChannelTests-context.xml
# spring-integration-jdbc/src/test/java/org/springframework/integration/jdbc/JdbcMessageStoreChannelTests.java
JIRA: https://jira.spring.io/browse/INT-4447
When we get an exception during `this.lock.unlock()`, we don't revoke
leadership.
In case of external resource (e.g. JDBC) this may cause a race condition
when the second candidate is selected as leader when connection comes back
* Catch `this.lock.unlock()` exceptions and log them under DEBUG.
This way we proceed to the `handleRevoked()` logic
**Cherry-pick to 5.0.x and 4.3.x**
# Conflicts:
# spring-integration-core/src/main/java/org/springframework/integration/support/leader/LockRegistryLeaderInitiator.java
https://build.spring.io/browse/INT-FATS5IC-457
The H2 `1.4.197` has fixed some issues
(in particular https://github.com/h2database/h2database/issues/178)
and introduced some breaking changes for us, although they are
reasonably good.
* Fix `JdbcOutboundGatewayParserTests` to use correct identifiers
for returned generated keys instead of previously used `SCOPE_IDENTITY()`
**Cherry-pick to 5.0.x and 4.3.x**
(cherry picked from commit 7b86ca7)
# Conflicts:
# build.gradle
https://build.spring.io/browse/INT-AT42SIO-820/
The test configs use `fixed-rate="100"`.
This way when Inbound Channel Adapter is declared earlier in the
config than `<jdbc:embedded-database>`, there is a chance that we start
to poll DB before it is really initialized.
* Reorder config so embedded DB is initialized before an
Inbound Channel Adapter
**Cherry-pick to 5.0.x, 4.3.x and 4.2.x**
(cherry picked from commit da6f47c)
# Conflicts:
# spring-integration-jdbc/src/test/java/org/springframework/integration/jdbc/config/pollingWithSelectParameterSourceJdbcInboundChannelAdapterTest.xml
https://build.spring.io/browse/INT-MJATS41-1279
Looks like there is a race condition when our polling rate around
data base is too fast and we have access to the DB files even during
application context close.
* Stop polling channel adapter in the tests explicitly after test methods
* Increase some tests performance decreasing timeouts to wait
* Increase timeouts and performance in the `StreamTransformerParserTests`
# Conflicts:
# build.gradle
# spring-integration-jdbc/src/test/java/org/springframework/integration/jdbc/JdbcMessageStoreChannelIntegrationTests-context.xml
# spring-integration-jdbc/src/test/java/org/springframework/integration/jdbc/JdbcMessageStoreChannelIntegrationTests.java
# spring-integration-jdbc/src/test/java/org/springframework/integration/jdbc/JdbcMessageStoreChannelOnePollerIntegrationTests-context.xml
# spring-integration-jdbc/src/test/java/org/springframework/integration/jdbc/JdbcMessageStoreChannelOnePollerIntegrationTests.java
# spring-integration-stomp/src/test/java/org/springframework/integration/stomp/client/StompServerIntegrationTests.java
Cherry-picked from cb0d43db6b
JIRA: https://jira.spring.io/browse/INT-4396
When target distributed `Lock` implementation throws an exception, e.g.
in case of no connection to the service, the `LockRegistryLeaderInitiator`
exists the loop and can come back to the elections only after restart
* Catch all the exception on `this.lock.tryLock()` and resubmit
`LeaderSelector` for a new locking cycle if
`LockRegistryLeaderInitiator.isRunning()` and `InterruptedException`
* Remove diagnostics from the `JdbcLockRegistryLeaderInitiatorTests`
since this fix confirms that we just didn't have a reconnect logic before
when this test failed sporadically
**Cherry-pick to 4.3.x**
# Conflicts:
# spring-integration-core/src/main/java/org/springframework/integration/support/leader/LockRegistryLeaderInitiator.java
# spring-integration-jdbc/src/test/java/org/springframework/integration/jdbc/leader/JdbcLockRegistryLeaderInitiatorTests.java
https://build.spring.io/browse/INT-MJATS41-1242
When different `DefaultLockRepository` instances use the same client id,
there is a possibility that they will update the same row in the table.
This way we have a chance that not only one obtains a lock and try to
add a value to the collection.
* Remove the test-case for the same client id as non-stable and even
dangerous by the the distributed lock purpose
* Use `tryLock(Long.MAX_VALUE)` to really ensure the wait behavior
during the concurrent loop
* Use `20` for thread pool to align with the tasks amount
**Cherry-picked to 4.3.x**
# Conflicts:
# spring-integration-jdbc/src/test/java/org/springframework/integration/jdbc/lock/JdbcLockRegistryDifferentClientTests.java
https://build.spring.io/browse/INT-MASTER-901
The `JdbcLockRegistryDifferentClientTests.testOnlyOneLock()` relies
on the `ArrayList.isEmpty()` state to proceed with the logic.
But since the `ArrayList.size` property is not `volatile`, there is no
guarantee for the proper state in the multi-threaded environment like
we have in this test-case.
* Replace `ArrayList` in the test with the `LinkedBlockingQueue` which
already rely on the `AtomicInteger` for the `size` property
* Fix `JdbcLockRegistry` tests to shutdown used `ExecutorService` s
to ensure set free threads after test suite execution.
**Cherry-pick to 4.3.x**
# Conflicts:
# spring-integration-jdbc/src/test/java/org/springframework/integration/jdbc/lock/JdbcLockRegistryDifferentClientTests.java
# spring-integration-jdbc/src/test/java/org/springframework/integration/jdbc/lock/JdbcLockRegistryTests.java
JIRA: https://jira.spring.io/browse/INT-4353
Updated based on code review
* Polishing according PR comments
**Cherry-pick to 4.3.x**
# Conflicts:
# src/reference/asciidoc/jdbc.adoc
JIRA: https://jira.spring.io/browse/INT-4342
Use similar code to Spring AMQP to add white list support for Integration's
use of the `DeserializingMessageConverter`; introduce the `WhiteListDeserializingMessageConverter`.
Polishing
Missed this change in PR.
Fix XSD attribute
# Conflicts:
# spring-integration-core/src/main/java/org/springframework/integration/dsl/Transformers.java
# spring-integration-core/src/test/java/org/springframework/integration/config/xml/PayloadDeserializingTransformerParserTests.java
# spring-integration-core/src/test/java/org/springframework/integration/dsl/flows/IntegrationFlowTests.java
# spring-integration-jdbc/src/main/java/org/springframework/integration/jdbc/store/JdbcChannelMessageStore.java
# spring-integration-jdbc/src/main/java/org/springframework/integration/jdbc/store/channel/MessageRowMapper.java
# spring-integration-mongodb/src/main/java/org/springframework/integration/mongodb/store/MongoDbMessageStore.java
# spring-integration-mongodb/src/main/java/org/springframework/integration/mongodb/support/BinaryToMessageConverter.java
# spring-integration-mongodb/src/main/java/org/springframework/integration/mongodb/support/MongoDbMessageBytesConverter.java
JIRA: https://jira.spring.io/browse/INT-4247
* Fix `JdbcLockRegistryLeaderInitiatorTests` race condition to assert
the `initiator1` is elected eventually after yielding when the
`initiator2` is stopped
JIRA: https://jira.spring.io/browse/INT-4311
* Added `QueryTimeoutException` to retry handling in the `JdbcLockRegistry`
Polishing
**Cherry-pick to 4.3.x**
JIRA: https://jira.spring.io/browse/INT-4262
Since `IF EXISTS` doesn't exist for `DROP INDEX` command and even more
this command isn't necessary when we fully drop the table, we don't need it
as a separate command.
`DROP TABLE IF EXISTS` is fully sufficient for schema recreation requirements
* Add `NODROPINDEX` variable for VPP engine to let it do not add `DROP INDEX` command
* Fix `channel/schema-mysql.sql` for proper INDEX names
JIRA: https://jira.spring.io/browse/INT-2625
Since `jdbcOperations` is mandatory property of the `JdbcMessageStore`,
make it as constructor arg to encourage users always specify `DataSource`
or `JdbcOperations`.
With that we don't need `afterPropertiesSet()` any more
JIRA: https://jira.spring.io/browse/INT-4246
The `LockRegistryLeaderInitiator` currently does not revoke leadership
when leading instance is unable to acquire lock from the underlying lock.
This can result in multiple `LockRegistryLeaderInitiator` instances becoming
leaders in the exceptional situations such as lock timeouts.
* Handle leadership revoking when leading `LockRegistryLeaderInitiator`
is unable to acquire lock
Polish javadoc
**Cherry-pick to master**
JIRA: https://jira.spring.io/browse/INT-4202
The `StoredProcOutboundGateway` uses `MessageBuilder` directly for procedure result without any conditions.
If procedure result is `null`, `MessageBuilder.withPayload` throws `java.lang.IllegalArgumentException: payload must not be null`.
* Since the super `AbstractReplyProducingMessageHandler` class takes care about `null` reply properly via its `requiresReply` property and really uses `MessageBuilder`
for reply, just fix `StoredProcOutboundGateway` to return procedure result as is from the `handleRequestMessage()` implementation
* Fix some typos and code style
* Increase some JDBC tests performance changing `Thread.sleep()` solution to proper `PollableChannel.receive()` or iterations over expected result
**Cherry-pick to 4.3.x**
Update `@Copyright` to 2017
Make `StoredProcOutboundGateway` as `requiresReply = true` by default
Revert `setRequiresReply(true)` change
Specify `setRequiresReply(true)` only if `expectSingleResult == true` and `setRequiresReply()` hasn't been called explicitly
Conflicts:
spring-integration-jdbc/src/main/java/org/springframework/integration/jdbc/StoredProcOutboundGateway.java
spring-integration-jdbc/src/test/java/org/springframework/integration/jdbc/JdbcMessageStoreChannelIntegrationTests.java
spring-integration-jdbc/src/test/java/org/springframework/integration/jdbc/config/StoredProcOutboundGatewayParserTests.java
Resolved.
JIRA: https://jira.spring.io/browse/INT-4178
The `JdbcLockRegistry` does not retry `CannotSerializeTransactionException` thrown by `DefaultLockRepository.acquire` which in turn causes `LockRegistryLeaderInitiator`s thread to get stuck.
* catch `CannotSerializeTransactionException` on `DefaultLockRepository.acquire` and ignore it to ensure a retry
**Cherry-pick to 4.3.x**
JIRA: https://jira.spring.io/browse/INT-4176
The `DelayerHandlerRescheduleIntegrationTests` doesn't close `context` in the end of test.
That causes clashes for `beanFactoryResolver` in other tests.
When those tests are fail, we come into condition when new processes can't be started/stopped because of effect of non-stopped context
* Fix all the `DelayerHandlerRescheduleIntegrationTests` to stop `context` in the end of test
* Remove unused files in the `gemfire/store` test package
* Add current date value to the `CacheServerProcess` name to avoid clashes with other ran on the same host
* Add NPE check for `region` in the `GemfireMetadataStoreTests`
* Extract `@BeforeClass/@AfterClass` for the `CacheWritingMessageHandlerTests` to start only one Gemfire cache per class
**Cherry-pick to 4.3.x & 4.2.x**
JIRA: https://jira.spring.io/browse/INT-4058
If you hold the lock, you are the leader. This simple idea gets you
a long way if there is no "native" leader initiator. E.g. you
can use this with a RDBMS with JdbcLockRegistry.
Add some docs on leader election under "endpoints"
Make thread name for leader initiator unique
In case there are multiple instances in the same context we would
like to be able to spot them in the logs.
INT-4058: Polishing
* Code refactoring in the `LockRegistryLeaderInitiator`: SI use 120 line length
* Add `Assert.notNull()` for required properties
* Add `this.lock.unlock()` and `Thread.sleep(LockRegistryLeaderInitiator.this.busyWaitMillis)` into the `catch` block to let distributed elections to work.
Otherwise there is a big chance that we will acquire the lock and become a leader again just after `yield()`
* Change `LockContext.toString()` to use simple `String` concatenation which is optimized by compiler to the `StringBuilder`.
That let to have some better micro-performance compared with with extra `Formatter` object in case of `String.format()`
* Add `JdbcLockRegistryLeaderInitiatorTests`
* Fix `yield()` logic based on the `Future.cancel(true)`.
Since one `FutureTask.cancel(true)` makes it as `INTERRUPTED` any subsequent `yield()` does not make any effect, therefore our infinite selector loop isn't interrupted one more time.
* Reschedule the `LeaderSelector` in the `yield()` after cancel(true).
* Rework `catch` in the selector loop just to the `InterruptedException` and `return null;` to stop looping and let reschedule the selector.
* Add one more `onRevoked` logic into the `final` of the selector loop to notify that we have lost leadership during `stop()`
* Improve `JdbcLockRegistryLeaderInitiatorTests` to ensure that several `yield()` on the same initiator work well.
* Add `What's New` note.
JIRA: https://jira.spring.io/browse/INT-4048
The semantics of the locks are the same as for the default lock
registry (they have to be unlocked by the same thread that
locked them).
Updates from feedback
Add interface for JdbcClient so transactions can bind to proxy
Fix typo in exception message
Fix drop script
JdbcClient -> LockRepository
Make lock() contract more like native Lock
Make lockInterruptibly throw InterruptedException more
Delete all expired locks, not just the ones that we own
INT-4048: Polishing
* `DefaultLockRepository`: replace `prefix` only once in the `afterPropertiesSet()`
* Add JavaDocs to the `DefaultLockRepository`
* `JdbcLockRegistry`: add `Thread.sleep(100)` to lock loops do not request DB so often like in case of clean `while(true)`
* `JdbcLockRegistry`: implement more robust `tryLock(long time, TimeUnit unit)`. The logic mostly copied from `RedisLockRegistry`
* Add `testExclusiveAccess()` to be sure that JDBC locks have exclusive access to the data sequence
Document `JdbcLockRegistry`
Upgrade to SF-4.3 GA
JIRA: https://jira.spring.io/browse/INT-3997
* Created the `H2ChannelMessageStoreQueryProvider` class for the
`ChannelMessageStoreQueryProvider` interface
* The `LongRunningIntegrationTest` class needs the
`RUN_LONG_INTEGRATION_TESTS` variable defined in the 'system'
and set in true. It to let work the test methods for the
`H2TxTimeoutMessageStoreTests` class
* Updated reference documentation about `ChannelMessageStoreQueryProvider`
* Polishing
* Upgrade to SA and SF snapshots
* Address TODOs regarding those upgrades
* Resolve some other TODOs
* Replace `PropertyPlaceholderConfigurer` beans with the `<context:property-placeholder>`
* Upgrade to Spring Social Twitter 1.1.2 and resolve deprecations via mocks (https://github.com/spring-projects/spring-social-twitter/issues/91)
* Upgrade to Curator `3.1.0` and resolve deprecation in the `ZookeeperMetadataStore`
https://build.spring.io/browse/INT-B43-164/
* Fix CheckStyle for `System. out .println(` expression commenting out such a code line
* Fix generics in the `JdbcMessageStoreTests`
* Increase receive timeout in the `FileMessageHistoryTests`
* Change the `AsyncAmqpGatewayTests` logic to deal with the same channel for both `ack/nack`.
The `timeout` case without reply may end up with the `Consumer cancel`, therefore `nack` not `ack` like we expected
JIRA: https://jira.spring.io/browse/INT-3387,
https://jira.spring.io/browse/INT-3806
* Introduce
```
MessageGroupStore
void addMessagesToGroup(Object groupId, Message<?>... messages);
```
And implement it in all stores.
* Use new `addMessagesToGroup` where it is reasonable, e.g. `DelayHandler`
* Optimize test-case to use a new store method (where it is possible)
* Fix timing delays in the `JdbcMessageStoreTests`
* Introduce `PersistentMessageGroup`
* Add `AbstractMessageGroupStore#proxyMessageGroupForLazyLoad` to wrap the raw `MessageGroup` to the `PersistentMessageGroup` for lazy-load
* Rework `MessageGroupMetadata` do not be `immutable` and allow to store/restore in the `AbstractKeyValueMessageStore` only the `MessageGroupMetadata`
* Refactor `ResequencingMessageHandler` and `SequenceSizeReleaseStrategy` a bit for better performance when interact with the `MessageGroup`
* Add `AbstractMessageGroupStore#setLazyLoadMessageGroups` to switch off the `lazy-load` behavior and restore the previous full `MessageGroup` logic
* Add `What's New` note and `message-store.adoc` paragraph for the lazy-load functionality
`GroupType.PERSISTENT` and not lazy by default
PR Comments
Fix `JdbcMessageStoreTests` timing issues
Address PR comments
* Add performance test to the `ConfigurableMongoDbMessageGroupStoreTests`
* Add JavaDocs for the `MessageGroupFactory` methods
* Add `log4j.properties` into the `test` MongoDB module for better traceability
* Fix `JdbcMessageStore#getOneMessageFromGroup()` over the `doPollForMessage()` delegation.
The `jdbcTemplate.queryForObject()` requires exactly one and only one raw in `resultSet`
* Add performance test results into the `message-store.adoc`
JIRA: https://jira.spring.io/browse/INT-3996
One of the query provider implementations was missing. To avoid a maintenance
nightmare, simply reference the interface's javadoc which lists all known
implementations.
* Simple `JdbcChannelMessageStore` JavaDocs polishing
* Add `LogAdjustingTestSupport` to the `StompIntegrationTests` to trace failures in the future.
* Also change the test `Start/Stop` messages to the `WARN` level in the `LogAdjustingTestSupport`, because `DEBUG` doesn't work everywhere.
Maybe some logging systems mix misconfiguration, like we have in the WebSocket modules with the Embedded Tomcat
JIRA: https://jira.spring.io/browse/INT-3984
The `PropertySourcesPlaceholderConfigurer implements BeanFactoryPostProcessor, PriorityOrdered`
meaning that it is run before any other regular `BeanFactoryPostProcessor`,
like the `IntegrationConfigurationBeanFactoryPostProcessor` has been before.
With such an order any `BeanDefinition` declaration from the `IntegrationConfigurationBeanFactoryPostProcessor` process
ended up without `PP` resolutions.
* Rework `IntegrationConfigurationBeanFactoryPostProcessor` to the `BeanDefinitionRegistryPostProcessor`,
which makes it be loaded as early as possible. See `PostProcessorRegistrationDelegate`:
````java
// Now, invoke the postProcessBeanFactory callback of all processors handled so far.
invokeBeanFactoryPostProcessors(registryPostProcessors, beanFactory);
invokeBeanFactoryPostProcessors(regularPostProcessors, beanFactory);
````
* Modify `EnableIntegrationTests` and `ChannelSecurityInterceptorSecuredChannelAnnotationTests` to ensure that the change
meets the `PP` and `SpEL` resolutions during bean definition phase.
* Fix the timing issue in the `JdbcMessageStoreChannelIntegrationTests`
Fixes https://build.spring.io/browse/INT-MJATS41-593
* Fix `LoggingHandler` JavaDoc
* Fix `JdbcOutboundGatewayParserTests` timing and race condition issues
* Add `mock(BeanFactory.class)` to the `JpaOutboundChannelAdapterTests` for `JpaExecutor`
* Fix `JpaOutboundChannelAdapterTests` for the incorrect transaction usage
checkstyle EmptyBlock
checkstyle fixRightCurly Script
checkstyle EmptyStatement
checkstyle RightCurly
checkstyle TailingWhite
checkstyle NeedBraces
Fix the line separator in the `fixRightCurly.gradle`
fixes
fixModifiers after fixFinal
Revert CachingSessionFactory
Class is spied in tests.
checkstyle - Import Rules
checkstyle InterfaceIsType
checkstyle InnerTypeLast
checkstyle OneStatementPerLine
CovariantEquals
OneTopLevelClass
* Revert `'\n'` -> `System.lineSeparator()` in the Gradle scripts to meet Git `autocrlf = true` on Windows
* Fix timing issue with the `LastModifiedFileListFilterTests`, when the `age = 1` might not be enough for the file object when we have some delay before checking
* `gradlew clean check -x test --parallel --continue` - to collect reports
* `gradlew fixThis --parallel` - to fix all possible vulnerabilities. With `-Dfile.encoding=UTF-8` on Windows
Since the `RequireThisCheck` doesn't see parents for anonymous classes (e.g. `Runnable` callback), its report doesn't contains the outer class name with `this.`,
therefore we still have to fix those cases manually.
Thanks to the wrong `replacer` just with `this.` we have uncompilable code enough easy to find problems.
Not so easy to fix for good readability though...
* Upgrade to Grade 2.12
* Upgrade to SonarQube native plugin
The fix contains at about 300 files. So, will be done on merge.
Fix `fixThis.gradle` according PR comments
Apply `fixThis` and also `fixModifiers` for test classes.
Fix some `this.` inner issues manually.
Make code polishing for long lines after `fixThis`
Fix conflicts and vulnerabilities after the rebase
