https://build.spring.io/browse/INT-MASTER-901
The `JdbcLockRegistryDifferentClientTests.testOnlyOneLock()` relies
on the `ArrayList.isEmpty()` state to proceed with the logic.
But since the `ArrayList.size` property is not `volatile`, there is no
guarantee for the proper state in the multi-threaded environment like
we have in this test-case.
* Replace `ArrayList` in the test with the `LinkedBlockingQueue` which
already rely on the `AtomicInteger` for the `size` property
* Fix `JdbcLockRegistry` tests to shutdown used `ExecutorService` s
to ensure set free threads after test suite execution.
**Cherry-pick to 4.3.x**
# Conflicts:
# spring-integration-jdbc/src/test/java/org/springframework/integration/jdbc/lock/JdbcLockRegistryDifferentClientTests.java
# spring-integration-jdbc/src/test/java/org/springframework/integration/jdbc/lock/JdbcLockRegistryTests.java
JIRA: https://jira.spring.io/browse/INT-4353
Updated based on code review
* Polishing according PR comments
**Cherry-pick to 4.3.x**
# Conflicts:
# src/reference/asciidoc/jdbc.adoc
JIRA: https://jira.spring.io/browse/INT-4342
Use similar code to Spring AMQP to add white list support for Integration's
use of the `DeserializingMessageConverter`; introduce the `WhiteListDeserializingMessageConverter`.
Polishing
Missed this change in PR.
Fix XSD attribute
# Conflicts:
# spring-integration-core/src/main/java/org/springframework/integration/dsl/Transformers.java
# spring-integration-core/src/test/java/org/springframework/integration/config/xml/PayloadDeserializingTransformerParserTests.java
# spring-integration-core/src/test/java/org/springframework/integration/dsl/flows/IntegrationFlowTests.java
# spring-integration-jdbc/src/main/java/org/springframework/integration/jdbc/store/JdbcChannelMessageStore.java
# spring-integration-jdbc/src/main/java/org/springframework/integration/jdbc/store/channel/MessageRowMapper.java
# spring-integration-mongodb/src/main/java/org/springframework/integration/mongodb/store/MongoDbMessageStore.java
# spring-integration-mongodb/src/main/java/org/springframework/integration/mongodb/support/BinaryToMessageConverter.java
# spring-integration-mongodb/src/main/java/org/springframework/integration/mongodb/support/MongoDbMessageBytesConverter.java
JIRA: https://jira.spring.io/browse/INT-4247
* Fix `JdbcLockRegistryLeaderInitiatorTests` race condition to assert
the `initiator1` is elected eventually after yielding when the
`initiator2` is stopped
JIRA: https://jira.spring.io/browse/INT-4311
* Added `QueryTimeoutException` to retry handling in the `JdbcLockRegistry`
Polishing
**Cherry-pick to 4.3.x**
JIRA: https://jira.spring.io/browse/INT-4262
Since `IF EXISTS` doesn't exist for `DROP INDEX` command and even more
this command isn't necessary when we fully drop the table, we don't need it
as a separate command.
`DROP TABLE IF EXISTS` is fully sufficient for schema recreation requirements
* Add `NODROPINDEX` variable for VPP engine to let it do not add `DROP INDEX` command
* Fix `channel/schema-mysql.sql` for proper INDEX names
JIRA: https://jira.spring.io/browse/INT-2625
Since `jdbcOperations` is mandatory property of the `JdbcMessageStore`,
make it as constructor arg to encourage users always specify `DataSource`
or `JdbcOperations`.
With that we don't need `afterPropertiesSet()` any more
JIRA: https://jira.spring.io/browse/INT-4246
The `LockRegistryLeaderInitiator` currently does not revoke leadership
when leading instance is unable to acquire lock from the underlying lock.
This can result in multiple `LockRegistryLeaderInitiator` instances becoming
leaders in the exceptional situations such as lock timeouts.
* Handle leadership revoking when leading `LockRegistryLeaderInitiator`
is unable to acquire lock
Polish javadoc
**Cherry-pick to master**
JIRA: https://jira.spring.io/browse/INT-4202
The `StoredProcOutboundGateway` uses `MessageBuilder` directly for procedure result without any conditions.
If procedure result is `null`, `MessageBuilder.withPayload` throws `java.lang.IllegalArgumentException: payload must not be null`.
* Since the super `AbstractReplyProducingMessageHandler` class takes care about `null` reply properly via its `requiresReply` property and really uses `MessageBuilder`
for reply, just fix `StoredProcOutboundGateway` to return procedure result as is from the `handleRequestMessage()` implementation
* Fix some typos and code style
* Increase some JDBC tests performance changing `Thread.sleep()` solution to proper `PollableChannel.receive()` or iterations over expected result
**Cherry-pick to 4.3.x**
Update `@Copyright` to 2017
Make `StoredProcOutboundGateway` as `requiresReply = true` by default
Revert `setRequiresReply(true)` change
Specify `setRequiresReply(true)` only if `expectSingleResult == true` and `setRequiresReply()` hasn't been called explicitly
Conflicts:
spring-integration-jdbc/src/main/java/org/springframework/integration/jdbc/StoredProcOutboundGateway.java
spring-integration-jdbc/src/test/java/org/springframework/integration/jdbc/JdbcMessageStoreChannelIntegrationTests.java
spring-integration-jdbc/src/test/java/org/springframework/integration/jdbc/config/StoredProcOutboundGatewayParserTests.java
Resolved.
JIRA: https://jira.spring.io/browse/INT-4178
The `JdbcLockRegistry` does not retry `CannotSerializeTransactionException` thrown by `DefaultLockRepository.acquire` which in turn causes `LockRegistryLeaderInitiator`s thread to get stuck.
* catch `CannotSerializeTransactionException` on `DefaultLockRepository.acquire` and ignore it to ensure a retry
**Cherry-pick to 4.3.x**
JIRA: https://jira.spring.io/browse/INT-4176
The `DelayerHandlerRescheduleIntegrationTests` doesn't close `context` in the end of test.
That causes clashes for `beanFactoryResolver` in other tests.
When those tests are fail, we come into condition when new processes can't be started/stopped because of effect of non-stopped context
* Fix all the `DelayerHandlerRescheduleIntegrationTests` to stop `context` in the end of test
* Remove unused files in the `gemfire/store` test package
* Add current date value to the `CacheServerProcess` name to avoid clashes with other ran on the same host
* Add NPE check for `region` in the `GemfireMetadataStoreTests`
* Extract `@BeforeClass/@AfterClass` for the `CacheWritingMessageHandlerTests` to start only one Gemfire cache per class
**Cherry-pick to 4.3.x & 4.2.x**
JIRA: https://jira.spring.io/browse/INT-4058
If you hold the lock, you are the leader. This simple idea gets you
a long way if there is no "native" leader initiator. E.g. you
can use this with a RDBMS with JdbcLockRegistry.
Add some docs on leader election under "endpoints"
Make thread name for leader initiator unique
In case there are multiple instances in the same context we would
like to be able to spot them in the logs.
INT-4058: Polishing
* Code refactoring in the `LockRegistryLeaderInitiator`: SI use 120 line length
* Add `Assert.notNull()` for required properties
* Add `this.lock.unlock()` and `Thread.sleep(LockRegistryLeaderInitiator.this.busyWaitMillis)` into the `catch` block to let distributed elections to work.
Otherwise there is a big chance that we will acquire the lock and become a leader again just after `yield()`
* Change `LockContext.toString()` to use simple `String` concatenation which is optimized by compiler to the `StringBuilder`.
That let to have some better micro-performance compared with with extra `Formatter` object in case of `String.format()`
* Add `JdbcLockRegistryLeaderInitiatorTests`
* Fix `yield()` logic based on the `Future.cancel(true)`.
Since one `FutureTask.cancel(true)` makes it as `INTERRUPTED` any subsequent `yield()` does not make any effect, therefore our infinite selector loop isn't interrupted one more time.
* Reschedule the `LeaderSelector` in the `yield()` after cancel(true).
* Rework `catch` in the selector loop just to the `InterruptedException` and `return null;` to stop looping and let reschedule the selector.
* Add one more `onRevoked` logic into the `final` of the selector loop to notify that we have lost leadership during `stop()`
* Improve `JdbcLockRegistryLeaderInitiatorTests` to ensure that several `yield()` on the same initiator work well.
* Add `What's New` note.
JIRA: https://jira.spring.io/browse/INT-4048
The semantics of the locks are the same as for the default lock
registry (they have to be unlocked by the same thread that
locked them).
Updates from feedback
Add interface for JdbcClient so transactions can bind to proxy
Fix typo in exception message
Fix drop script
JdbcClient -> LockRepository
Make lock() contract more like native Lock
Make lockInterruptibly throw InterruptedException more
Delete all expired locks, not just the ones that we own
INT-4048: Polishing
* `DefaultLockRepository`: replace `prefix` only once in the `afterPropertiesSet()`
* Add JavaDocs to the `DefaultLockRepository`
* `JdbcLockRegistry`: add `Thread.sleep(100)` to lock loops do not request DB so often like in case of clean `while(true)`
* `JdbcLockRegistry`: implement more robust `tryLock(long time, TimeUnit unit)`. The logic mostly copied from `RedisLockRegistry`
* Add `testExclusiveAccess()` to be sure that JDBC locks have exclusive access to the data sequence
Document `JdbcLockRegistry`
Upgrade to SF-4.3 GA
JIRA: https://jira.spring.io/browse/INT-3997
* Created the `H2ChannelMessageStoreQueryProvider` class for the
`ChannelMessageStoreQueryProvider` interface
* The `LongRunningIntegrationTest` class needs the
`RUN_LONG_INTEGRATION_TESTS` variable defined in the 'system'
and set in true. It to let work the test methods for the
`H2TxTimeoutMessageStoreTests` class
* Updated reference documentation about `ChannelMessageStoreQueryProvider`
* Polishing
* Upgrade to SA and SF snapshots
* Address TODOs regarding those upgrades
* Resolve some other TODOs
* Replace `PropertyPlaceholderConfigurer` beans with the `<context:property-placeholder>`
* Upgrade to Spring Social Twitter 1.1.2 and resolve deprecations via mocks (https://github.com/spring-projects/spring-social-twitter/issues/91)
* Upgrade to Curator `3.1.0` and resolve deprecation in the `ZookeeperMetadataStore`
https://build.spring.io/browse/INT-B43-164/
* Fix CheckStyle for `System. out .println(` expression commenting out such a code line
* Fix generics in the `JdbcMessageStoreTests`
* Increase receive timeout in the `FileMessageHistoryTests`
* Change the `AsyncAmqpGatewayTests` logic to deal with the same channel for both `ack/nack`.
The `timeout` case without reply may end up with the `Consumer cancel`, therefore `nack` not `ack` like we expected
JIRA: https://jira.spring.io/browse/INT-3387,
https://jira.spring.io/browse/INT-3806
* Introduce
```
MessageGroupStore
void addMessagesToGroup(Object groupId, Message<?>... messages);
```
And implement it in all stores.
* Use new `addMessagesToGroup` where it is reasonable, e.g. `DelayHandler`
* Optimize test-case to use a new store method (where it is possible)
* Fix timing delays in the `JdbcMessageStoreTests`
* Introduce `PersistentMessageGroup`
* Add `AbstractMessageGroupStore#proxyMessageGroupForLazyLoad` to wrap the raw `MessageGroup` to the `PersistentMessageGroup` for lazy-load
* Rework `MessageGroupMetadata` do not be `immutable` and allow to store/restore in the `AbstractKeyValueMessageStore` only the `MessageGroupMetadata`
* Refactor `ResequencingMessageHandler` and `SequenceSizeReleaseStrategy` a bit for better performance when interact with the `MessageGroup`
* Add `AbstractMessageGroupStore#setLazyLoadMessageGroups` to switch off the `lazy-load` behavior and restore the previous full `MessageGroup` logic
* Add `What's New` note and `message-store.adoc` paragraph for the lazy-load functionality
`GroupType.PERSISTENT` and not lazy by default
PR Comments
Fix `JdbcMessageStoreTests` timing issues
Address PR comments
* Add performance test to the `ConfigurableMongoDbMessageGroupStoreTests`
* Add JavaDocs for the `MessageGroupFactory` methods
* Add `log4j.properties` into the `test` MongoDB module for better traceability
* Fix `JdbcMessageStore#getOneMessageFromGroup()` over the `doPollForMessage()` delegation.
The `jdbcTemplate.queryForObject()` requires exactly one and only one raw in `resultSet`
* Add performance test results into the `message-store.adoc`
JIRA: https://jira.spring.io/browse/INT-3996
One of the query provider implementations was missing. To avoid a maintenance
nightmare, simply reference the interface's javadoc which lists all known
implementations.
* Simple `JdbcChannelMessageStore` JavaDocs polishing
* Add `LogAdjustingTestSupport` to the `StompIntegrationTests` to trace failures in the future.
* Also change the test `Start/Stop` messages to the `WARN` level in the `LogAdjustingTestSupport`, because `DEBUG` doesn't work everywhere.
Maybe some logging systems mix misconfiguration, like we have in the WebSocket modules with the Embedded Tomcat
JIRA: https://jira.spring.io/browse/INT-3984
The `PropertySourcesPlaceholderConfigurer implements BeanFactoryPostProcessor, PriorityOrdered`
meaning that it is run before any other regular `BeanFactoryPostProcessor`,
like the `IntegrationConfigurationBeanFactoryPostProcessor` has been before.
With such an order any `BeanDefinition` declaration from the `IntegrationConfigurationBeanFactoryPostProcessor` process
ended up without `PP` resolutions.
* Rework `IntegrationConfigurationBeanFactoryPostProcessor` to the `BeanDefinitionRegistryPostProcessor`,
which makes it be loaded as early as possible. See `PostProcessorRegistrationDelegate`:
````java
// Now, invoke the postProcessBeanFactory callback of all processors handled so far.
invokeBeanFactoryPostProcessors(registryPostProcessors, beanFactory);
invokeBeanFactoryPostProcessors(regularPostProcessors, beanFactory);
````
* Modify `EnableIntegrationTests` and `ChannelSecurityInterceptorSecuredChannelAnnotationTests` to ensure that the change
meets the `PP` and `SpEL` resolutions during bean definition phase.
* Fix the timing issue in the `JdbcMessageStoreChannelIntegrationTests`
Fixes https://build.spring.io/browse/INT-MJATS41-593
* Fix `LoggingHandler` JavaDoc
* Fix `JdbcOutboundGatewayParserTests` timing and race condition issues
* Add `mock(BeanFactory.class)` to the `JpaOutboundChannelAdapterTests` for `JpaExecutor`
* Fix `JpaOutboundChannelAdapterTests` for the incorrect transaction usage
checkstyle EmptyBlock
checkstyle fixRightCurly Script
checkstyle EmptyStatement
checkstyle RightCurly
checkstyle TailingWhite
checkstyle NeedBraces
Fix the line separator in the `fixRightCurly.gradle`
fixes
fixModifiers after fixFinal
Revert CachingSessionFactory
Class is spied in tests.
checkstyle - Import Rules
checkstyle InterfaceIsType
checkstyle InnerTypeLast
checkstyle OneStatementPerLine
CovariantEquals
OneTopLevelClass
* Revert `'\n'` -> `System.lineSeparator()` in the Gradle scripts to meet Git `autocrlf = true` on Windows
* Fix timing issue with the `LastModifiedFileListFilterTests`, when the `age = 1` might not be enough for the file object when we have some delay before checking
* `gradlew clean check -x test --parallel --continue` - to collect reports
* `gradlew fixThis --parallel` - to fix all possible vulnerabilities. With `-Dfile.encoding=UTF-8` on Windows
Since the `RequireThisCheck` doesn't see parents for anonymous classes (e.g. `Runnable` callback), its report doesn't contains the outer class name with `this.`,
therefore we still have to fix those cases manually.
Thanks to the wrong `replacer` just with `this.` we have uncompilable code enough easy to find problems.
Not so easy to fix for good readability though...
* Upgrade to Grade 2.12
* Upgrade to SonarQube native plugin
The fix contains at about 300 files. So, will be done on merge.
Fix `fixThis.gradle` according PR comments
Apply `fixThis` and also `fixModifiers` for test classes.
Fix some `this.` inner issues manually.
Make code polishing for long lines after `fixThis`
Fix conflicts and vulnerabilities after the rebase
Add `fixModifiers.gradle` to run after `check` task.
The `RedundantModifierChecker` doesn't catch all errors at once.
We need run `check -> fixModifiers` pair several times to reach `SUCCESSFUL`.
The `fixThis` has been ported from the SA, but without applying.
* Polishing for `fixModifiers.gradle`
* Fix all redundant modifier with the `fixModifiers.gradle`
NOTE: Since all these task modify files on Windows we have to run them with the `-Dfile.encoding=UTF-8`.
Otherwise they are read and write with the Windows default `cp1251` making them incompatible with Unix system.
JIRAs:
https://jira.spring.io/browse/INT-3830https://jira.spring.io/browse/INT-3523https://jira.spring.io/browse/INT-3846
* Fix the OOM condition, when we `release()` `UpperBound` independently of the previous `remove` result (https://jira.spring.io/browse/INT-3846)
* Fix "confuse" around `groupCapacity`, when we really didn't care about individual groups (https://jira.spring.io/browse/INT-3523)
* Add `upperBoundTimeout` to have a hook to wait some time for the empty slot in the store (https://jira.spring.io/browse/INT-3830)
* Fix some JavaDocs warnings
* Fix some typos
* Fix inconsistency in the `DelayHandler` around `removeMessageFromGroup` when `MS` is `SimpleMessageStore`
* Remove `UpperBound.release()` operation from `SimpleMessageStore.removeGroup()`.
The waiting process should worry about the new `UpperBound` instance.
Some other polishing
`tryAcquire` outside of the `lock`
Move `tryAcquire` within the `addMessageToGroup` outside of `lock`.
But do that only for groups which already exist.
For the new groups we have a fresh `UpperBound`, so no need to worry about dead lock and
we can obtain a permit immediately.
SimpleMessageGroup: BlockingQueue -> LinkedHashSet
`SimpleMessageStore`: use "unsynchonized" `SimpleMessageGroup`
Make some synchronization fixes according to the migration to the `LinkedHashSet`
Avoid extra `Collection`
`ResequencingMessageHandler`: compare `size()` of collections instead of `containsAll()`
Fix `ConcurrentModificationException` in the `AbstractKeyValueMessageStore`
Add `SimpleMessageStore.clearMessageGroup()`
Accept polishing and fix `RedisChannelMessageStoreTests`
`@Deprecated` `MessageGroupStore.removeMessageFromGroup()`
Fix some typos
Introduce `SimpleMessageGroupFactory`
Extract `MessageGroupFactory` and address PR comments
Polishing after rebase
JavaDocs and Reference Manual
Fix JavaDocs
JIRA: https://jira.spring.io/browse/INT-3926
* Add `<poller>` for all `<outbound-adapter(gateway)>`
* Add/modify tests for `<queue>` as an input channel and particular `<poller>`
* Move XSD from the `xml` package to the `config` level to achieve the consistency throughout the project.
* `ClientWebSocketContainer`: add some synchronization to avoid race conditions: https://build.spring.io/browse/INT-B41-492
* `TomcatWebSocketTestServer`: convert to `0` port to rely on the OS resolution for `localPort`
* Add `LogAdjustingTestSupport` for STOMP test
* `SftpServerTests`: use `0` port to rely on the OS resolution for `localPort`
* `ImapMailReceiver`, `OutboundGatewayFunctionTests` (JMS), `CachingClientConnectionFactoryTests`,
`AsyncGatewayTests`, `AsyncMessagingTemplateTests`, `GatewayParserTests`, `PriorityChannelTests`, `AggregatorIntegrationTests`: increase timeout
* `EnableIntegrationTests`: use `LogAdjustingTestSupport`
* `FileOutboundChannelAdapterParserTests`: rework `Thread.sleep()` with `CountDownLatch`
* `ConnectionToConnectionTests`: increase timeout and count iteration. Previously with `1sec` we may lose some events. And we can't just rely on the `10sec`,
because the last iteration will be so long
* `TcpOutboundGatewayTests`: `500ms` is so big timeout to wait for the `Exception` that in the high load environment we can yield to other Thread so long.
Like in our case to `server` Thread to send the reply for us. Therefore decrease the Exception timeout to the `50ms` and increase server delay to `2sec`
* `StompInboundChannelAdapterWebSocketIntegrationTests`: remove `@Qualifier("taskScheduler")` as a potential candidate to test against latest SF changes.
We're fine with `SF-4.2.2` and it is just a test-case. So, I don't see reason to wait for their fix here.
STOMP: `session = null` in adapters for any transportError
Polishing
JIRA: https://jira.spring.io/browse/INT-2541
The `IntegrationMBeanExporter` doesn't register any `@ManagedResource` automatically.
Its `autoDetection` is disabled by default.
Hence such a components like `AbstractMessageGroupStore`
or `WireTap` aren't exposed to the JMX by the `IntegrationMBeanExporter`.
From other side `MBeanExporterHelper` aims to help to avoid duplicate exposing if
an `MBeanExporter` is presented in the CTX.
But `MBeanExporterHelper` did that unconditionally just for the whole `org.springframework.integration` package.
Therefore components which aren't EIP ones aren't exposed to the JMX at all.
* Fix `MBeanExporterHelper` to deal for exclusion based on the `@IntegrationManagedResource` annotation.
* Remove `@IntegrationManagedResource` from those components which aren't EIP.
* Make some polishing in the JMX tests.
