JIRA: https://jira.spring.io/browse/INT-4262
Since MySQL doesn't have `IF EXISTS` for `DROP INDEX`.
And on the other hand the entire `DROP TABLE` takes care about indexes as well,
we don't need the `DROP INDEX` command for mysql.properties
* Add `NODROPINDEX` variable to the `mysql.properties`
* Tweak `destroy.sql.vpp` to leverage `NODROPINDEX`
This commit harmonizes `LockRegistry#obtain` logic in Zookeeper and JDBC
implementations with Redis implementation
by using Java 8 `ConcurrentMap#computeIfAbsent`
* Remove `synchronized (this.locks)` from the `expireUnusedOlderThan`
implementations because `iterator()` is thread-safe from `ConcurrentHashMap`
* Fix deprecation in the `IntegrationGraphControllerRegistrar`
* Revert Spring Security version to `4.2.2`, since `5.0 B-S` is broken
JIRA: https://jira.spring.io/browse/INT-2625,
https://jira.spring.io/browse/INT-4050,
https://jira.spring.io/browse/INT-4052
Make some refactoring for consistency:
- Move `JdbcMessageStore` to the `store` package
- Make `JdbcMessageStore` only ctor-based configuration for the `JdbcOperations`
- Remove `/channel/` SQL scripts in favor of combined in the `/jdbc/` directory
- Make scripts for the `INT_CHANNEL_MESSAGE` table and indexes generated via VPP
Fix drop script generation
Fix `schema-derby.sql` for suspicious `[]`
Looks like `foundrylogic.vpp` tool generates somehow wrong SQL
Further fix for the `drop` scripts generation
Address PR comments:
* Remove deprecated code in the `JdbcMessageStore`
* Remove separate MySQL script in favor of only one for latest version
* Remove the note in the `jdbc.adoc` about recommendation to use new MySQL version
JIRA: https://jira.spring.io/browse/INT-4251
MockIntegration PoC
Demonstrate how `MockIntegration.mockMessageSource()` can be used.
Even from the XML Config!
The same semantics can be applied for any other our mocking features we will add to the `MockIntegration`
Add `testMockMessageSourceDynamicFlow()`
Improve `MockIntegration.Context.instead()` for target bean type assertion
Some refactoring, improvements and JavaDocs
* Move `MockIntegration.Context` to its own `MockIntegrationContext` class
* Rename `@MockIntegrationTest` to `@SpringIntegrationTest` since we talk there not only about mocks, but any other possible testing feature for integration
* Add `@SpringIntegrationTest#stopEndpoints()` attribute for endpoints bean names patterns
* Add `IntegrationEndpointsInitializer` to customize `AbstractEndpoint`s according options from the `@SpringIntegrationTest`
Right now it is only about making them `autoStartup = false`
* Rename `SI-test` to `SI-test-support`
* Rename `SI-mock` to `SI-test`
* Rename `@SpringIntegrationTest#stopEndpoints()` to `noAutoStartup()`
* Fix JavaDocs according PR comments
* Increase timeouts in some sporadically failed tests
* Draft for the `testing.adoc`
Fixesspring-projects/spring-integration-java-dsl#23
Fix `testing.adoc` a bit
* Document Testing Support
* Fix a couple typos in the `xml.adoc`
* Add `package-info.java` files to `/test/` sub-packages
Improve docs according PR comments
Fix typos in `TestUtils` JavaDocs
Add `What's New` note about Java DSL and Testing support
Doc Polishing
JIRA: https://jira.spring.io/browse/INT-4244
The `JpaOutboundGateway` returns a whole `Message` for reply copying request headers.
Building the reply a `AbstractMessageProducingHandler` uses `MessageBuilder` to create a new `Message` with all request headers because of `shouldCopyRequestHeaders() == true`
* To avoid some overhead with new `Message` and `MessageBuilder` for nothing, just return a JPA result as is and let super class to build the Message
* Optimize all reply producers to return `MessageBuilder` to the super `handleMessage()` process
* Return the whole `gatherResult` `Message<?>` from the `ScatterGatherHandler` to have ability to transfer reply headers as well
* Mark `DelayHandler` as `shouldCopyRequestHeaders() return false` since it doesn't modify Message and returns it as is after delay
JIRA: https://jira.spring.io/browse/INT-4246
The `LockRegistryLeaderInitiator` currently does not revoke leadership
when leading instance is unable to acquire lock from the underlying lock.
This can result in multiple `LockRegistryLeaderInitiator` instances becoming
leaders in the exceptional situations such as lock timeouts.
* Handle leadership revoking when leading `LockRegistryLeaderInitiator`
is unable to acquire lock
Polish javadoc
**Cherry-pick to master**
Conflicts:
spring-integration-core/src/main/java/org/springframework/integration/support/leader/LockRegistryLeaderInitiator.java
spring-integration-core/src/test/java/org/springframework/integration/support/leader/LockRegistryLeaderInitiatorTests.java
* Stop initiators in the end of test
JIRA: https://jira.spring.io/browse/INT-4220
Currently when an error is thrown from the event publishing the role granting is broken and we just go to the role revoking.
* Since it's just an event publishing it shouldn't effect the original leader election.
* `try...catch` event publishing in the `LeaderInitiator` and `logger.warn` an `Exception`
* Make `leader/Context` as `@FunctionalInterface` for simple Lambda use-case like `NULL_CONTEXT` - `() -> false`
* Remove all the internal `NullContext` implementations in favor of above mention Lambda
* In the `LockRegistryLeaderInitiator` use `CustomizableThreadFactory` instead of custom `ThreadFactory` for prefixing
* Add `zookeeper/leader/LeaderInitiator#getContext()` for external usage and consistency with other similar components
* Fix `CuratorContext.toString()` typo
JIRA: https://jira.spring.io/browse/INT-4202
The `StoredProcOutboundGateway` uses `MessageBuilder` directly for procedure result without any conditions.
If procedure result is `null`, `MessageBuilder.withPayload` throws `java.lang.IllegalArgumentException: payload must not be null`.
* Since the super `AbstractReplyProducingMessageHandler` class takes care about `null` reply properly via its `requiresReply` property and really uses `MessageBuilder`
for reply, just fix `StoredProcOutboundGateway` to return procedure result as is from the `handleRequestMessage()` implementation
* Fix some typos and code style
* Increase some JDBC tests performance changing `Thread.sleep()` solution to proper `PollableChannel.receive()` or iterations over expected result
**Cherry-pick to 4.3.x**
Update `@Copyright` to 2017
Make `StoredProcOutboundGateway` as `requiresReply = true` by default
Revert `setRequiresReply(true)` change
Specify `setRequiresReply(true)` only if `expectSingleResult == true` and `setRequiresReply()` hasn't been called explicitly
JIRA: https://jira.spring.io/browse/INT-4178
The `JdbcLockRegistry` does not retry `CannotSerializeTransactionException` thrown by `DefaultLockRepository.acquire` which in turn causes `LockRegistryLeaderInitiator`s thread to get stuck.
* catch `CannotSerializeTransactionException` on `DefaultLockRepository.acquire` and ignore it to ensure a retry
**Cherry-pick to 4.3.x**
(cherry picked from commit 8198ff1)
JIRA: https://jira.spring.io/browse/INT-4176
The `DelayerHandlerRescheduleIntegrationTests` doesn't close `context` in the end of test.
That causes clashes for `beanFactoryResolver` in other tests.
When those tests are fail, we come into condition when new processes can't be started/stopped because of effect of non-stopped context
* Fix all the `DelayerHandlerRescheduleIntegrationTests` to stop `context` in the end of test
* Remove unused files in the `gemfire/store` test package
* Add current date value to the `CacheServerProcess` name to avoid clashes with other ran on the same host
* Add NPE check for `region` in the `GemfireMetadataStoreTests`
* Extract `@BeforeClass/@AfterClass` for the `CacheWritingMessageHandlerTests` to start only one Gemfire cache per class
**Cherry-pick to 4.3.x & 4.2.x**
https://sonar.spring.io/component_issues?id=org.springframework.integration%3Aspring-integration%3Amaster#resolved=false|types=BUG
In `IntegrationFlowRegistration` double check locking is ok for `inputChannel`
because we're assigning an existing object, but `MessagingTemplate` constructs
a new object for which double check locking doesn't work.
In any case, for both these items, the chance of concurrent access is extremely low
and is idempotent anyway, so remove double check locking.
Several inner classes can be static.
Other minor fixes.
JIRA: https://jira.spring.io/browse/INT-4162
INT-4162: IP DSL - Phase I Connection Factories
Phase 2 - Adapters/Gateways
Phase 3 - UDP
Polishing - PR Comments
Remove unnecessary generics.
More Polishing - PR Comments
Checkstyle/Javadoc Fixes
ComponentsRegistration Improvements
Add ...Spec CTORs to avoid the issue described here:
https://github.com/spring-projects/spring-integration-java-dsl/issues/137
Also other PR comments.
Fix UDP Spec Inheritance
Jdbc Lock Test Log Adjuster
Fix Mongo Test Race Conditions
* Fox typos in the `AbstractUdpOutboundChannelAdapterSpec`
* Improve `IpIntegrationTests.testUdpInheritance()`
* Change `fixed-rate` to `fixed-delay` in the `MongoDbInboundChannelAdapterIntegrationTests-context` to pursue single-threaded environment from poller for test
JIRA: https://jira.spring.io/browse/INT-2460,
https://jira.spring.io/browse/INT-4122
Since the main purpose of the `MessageStore` to persist message for durability and only,
it doesn't make sense to modify `Message` for additional headers like `SAVED` and `CREATED_DATE`.
Such a logic should be a part of metadata stored together with the message.
And it is provided by the out-of-the-box `MessageStore` implementation.
In addition we free ourselves from the reflection operations to retain `ID` and `TIMESTAMP` headers when we add `SAVED` and `CREATED_DATE`
* Control "already saved" logic in the `JdbcMessageStore`s via `DuplicateKeyException` on the `INSERT`.
This is much effective then additional `SELECT` in case of `SAVED` before
* Control "already saved" logic in the `AbstractConfigurableMongoDbMessageStore` via `DuplicateKeyException` on the `INSERT`.
Since `MongoDbMessageStore` doesn't provide extra `messageId` field, perform extra `SELECT` before store document.
Anyway the `MongoDbMessageStore` isn't recommended for use.
We may consider to deprecate it
* Control "already saved" logic in the `AbstractKeyValueMessageStore` via `putIfAbsent` operation
With this fix we persist message in the store as is without any modifications when we perform standard serialization procedure.
Any custom serializers should consider to use `MutableMessageBuilder` if there is a requirement to retain `ID` and `TIMESTAMP`
Rework `MongoDbMetadataStore.putIfAbsent()` to normal `findAndModify()` with particular `$setOnInsert`.
Technically the MongoDB query looks like:
```
db.collection.findAndModify({
query: { _id: $key },
update: {
$setOnInsert: { value: $value } // perform modification only on upsert
},
new: false, // don't return new doc if one is upserted
upsert: true // insert the document if it does not exist
})
```
Move single import to the appropriate JavaDoc
Polishing after rebase
DEBUG messages in `doStoreIfAbsent()` implementations
* To keep track of the extra message information in the `MessageStore`, without `Message` modification, introduce `MessageMetadata` and `MessageHolder`
* Add `MessageStore#getMessageMetadata()`
* Modify MongoDb `MessageStore` to add extra `timestamp` for individual message
* Fix `ConcurrentAggregatorTests` race condition.
Since currently the default release strategy is `SimpleSequenceSizeReleaseStrategy` which is just based on the `MessageGroup` size, there is no guaranty which messages will complete the group in concurrent environment.
The test is really based on the `SequenceAwareMessageGroup` logic to discard the message with the same `correlationId`
Fix `@Copyright` format
Move cast to `MessageHolder` after `Assert.isInstanceOf(MessageHolder.class, messageHolder)`
Retain backward compatibility in the `AbstractKeyValueMessageStore`
Polishing
* Upgrade to Checkstyle `7.1`
* Relax `RequireThis` rule a bit. Right now it does the effort only in case of overlapping. See https://github.com/checkstyle/checkstyle/issues/2362 for more info
* Enable some annotation rules and provide fixes for violations
* Enable `tabs indents` rule. This was the biggest fix in this PR
* Resolve `TODO` in the `MessagingMethodInvokerHelper` and fix tests to meet `IllegalStateException` now
Remain Checkstyle version `6.16.1` and reinstate `RequireThis` rule
The latest Checkstyle has a bug with local scope variables if they have the same names as property.
Revert some literals splitting
Fix some line length exceeding and code style
* Make most functional interfaces as `@FunctionalInterface`
* Convert some abstract classes to `@FunctionalInterface` with `default` methods
* Apply Lambda style implementation in some places
* Remove `Function` in favor of similar in Java 8
* Remove redundant code from `DefaultAmqpHeaderMapper` since we are already on Spring AMQP-2.0
* Add several ctors to the `ExpressionEvaluatingMessageListProcessor`
* Populate explicit `Boolean.class` `expectedType` from the `ExpressionEvaluatingReleaseStrategy`
JIRA: https://jira.spring.io/browse/INT-3913
* Remove deprecated classes and methods/constructors, deprecated XML attributes
* Remove `TcpConnectionEventListeningMessageProducer` and rework tests logic to the `ApplicationEventListeningMessageProducer`
* Fix several typos
* Remove/rework deprecated entities mentioning
JIRA: https://jira.spring.io/browse/INT-4058
If you hold the lock, you are the leader. This simple idea gets you
a long way if there is no "native" leader initiator. E.g. you
can use this with a RDBMS with JdbcLockRegistry.
Add some docs on leader election under "endpoints"
Make thread name for leader initiator unique
In case there are multiple instances in the same context we would
like to be able to spot them in the logs.
INT-4058: Polishing
* Code refactoring in the `LockRegistryLeaderInitiator`: SI use 120 line length
* Add `Assert.notNull()` for required properties
* Add `this.lock.unlock()` and `Thread.sleep(LockRegistryLeaderInitiator.this.busyWaitMillis)` into the `catch` block to let distributed elections to work.
Otherwise there is a big chance that we will acquire the lock and become a leader again just after `yield()`
* Change `LockContext.toString()` to use simple `String` concatenation which is optimized by compiler to the `StringBuilder`.
That let to have some better micro-performance compared with with extra `Formatter` object in case of `String.format()`
* Add `JdbcLockRegistryLeaderInitiatorTests`
* Fix `yield()` logic based on the `Future.cancel(true)`.
Since one `FutureTask.cancel(true)` makes it as `INTERRUPTED` any subsequent `yield()` does not make any effect, therefore our infinite selector loop isn't interrupted one more time.
* Reschedule the `LeaderSelector` in the `yield()` after cancel(true).
* Rework `catch` in the selector loop just to the `InterruptedException` and `return null;` to stop looping and let reschedule the selector.
* Add one more `onRevoked` logic into the `final` of the selector loop to notify that we have lost leadership during `stop()`
* Improve `JdbcLockRegistryLeaderInitiatorTests` to ensure that several `yield()` on the same initiator work well.
* Add `What's New` note.
JIRA: https://jira.spring.io/browse/INT-4048
The semantics of the locks are the same as for the default lock
registry (they have to be unlocked by the same thread that
locked them).
Updates from feedback
Add interface for JdbcClient so transactions can bind to proxy
Fix typo in exception message
Fix drop script
JdbcClient -> LockRepository
Make lock() contract more like native Lock
Make lockInterruptibly throw InterruptedException more
Delete all expired locks, not just the ones that we own
INT-4048: Polishing
* `DefaultLockRepository`: replace `prefix` only once in the `afterPropertiesSet()`
* Add JavaDocs to the `DefaultLockRepository`
* `JdbcLockRegistry`: add `Thread.sleep(100)` to lock loops do not request DB so often like in case of clean `while(true)`
* `JdbcLockRegistry`: implement more robust `tryLock(long time, TimeUnit unit)`. The logic mostly copied from `RedisLockRegistry`
* Add `testExclusiveAccess()` to be sure that JDBC locks have exclusive access to the data sequence
Document `JdbcLockRegistry`
Upgrade to SF-4.3 GA
JIRA: https://jira.spring.io/browse/INT-3997
* Created the `H2ChannelMessageStoreQueryProvider` class for the
`ChannelMessageStoreQueryProvider` interface
* The `LongRunningIntegrationTest` class needs the
`RUN_LONG_INTEGRATION_TESTS` variable defined in the 'system'
and set in true. It to let work the test methods for the
`H2TxTimeoutMessageStoreTests` class
* Updated reference documentation about `ChannelMessageStoreQueryProvider`
* Polishing
* Upgrade to SA and SF snapshots
* Address TODOs regarding those upgrades
* Resolve some other TODOs
* Replace `PropertyPlaceholderConfigurer` beans with the `<context:property-placeholder>`
* Upgrade to Spring Social Twitter 1.1.2 and resolve deprecations via mocks (https://github.com/spring-projects/spring-social-twitter/issues/91)
* Upgrade to Curator `3.1.0` and resolve deprecation in the `ZookeeperMetadataStore`
https://build.spring.io/browse/INT-B43-164/
* Fix CheckStyle for `System. out .println(` expression commenting out such a code line
* Fix generics in the `JdbcMessageStoreTests`
* Increase receive timeout in the `FileMessageHistoryTests`
* Change the `AsyncAmqpGatewayTests` logic to deal with the same channel for both `ack/nack`.
The `timeout` case without reply may end up with the `Consumer cancel`, therefore `nack` not `ack` like we expected
JIRA: https://jira.spring.io/browse/INT-3387,
https://jira.spring.io/browse/INT-3806
* Introduce
```
MessageGroupStore
void addMessagesToGroup(Object groupId, Message<?>... messages);
```
And implement it in all stores.
* Use new `addMessagesToGroup` where it is reasonable, e.g. `DelayHandler`
* Optimize test-case to use a new store method (where it is possible)
* Fix timing delays in the `JdbcMessageStoreTests`
* Introduce `PersistentMessageGroup`
* Add `AbstractMessageGroupStore#proxyMessageGroupForLazyLoad` to wrap the raw `MessageGroup` to the `PersistentMessageGroup` for lazy-load
* Rework `MessageGroupMetadata` do not be `immutable` and allow to store/restore in the `AbstractKeyValueMessageStore` only the `MessageGroupMetadata`
* Refactor `ResequencingMessageHandler` and `SequenceSizeReleaseStrategy` a bit for better performance when interact with the `MessageGroup`
* Add `AbstractMessageGroupStore#setLazyLoadMessageGroups` to switch off the `lazy-load` behavior and restore the previous full `MessageGroup` logic
* Add `What's New` note and `message-store.adoc` paragraph for the lazy-load functionality
`GroupType.PERSISTENT` and not lazy by default
PR Comments
Fix `JdbcMessageStoreTests` timing issues
Address PR comments
* Add performance test to the `ConfigurableMongoDbMessageGroupStoreTests`
* Add JavaDocs for the `MessageGroupFactory` methods
* Add `log4j.properties` into the `test` MongoDB module for better traceability
* Fix `JdbcMessageStore#getOneMessageFromGroup()` over the `doPollForMessage()` delegation.
The `jdbcTemplate.queryForObject()` requires exactly one and only one raw in `resultSet`
* Add performance test results into the `message-store.adoc`
JIRA: https://jira.spring.io/browse/INT-3996
One of the query provider implementations was missing. To avoid a maintenance
nightmare, simply reference the interface's javadoc which lists all known
implementations.
* Simple `JdbcChannelMessageStore` JavaDocs polishing
* Add `LogAdjustingTestSupport` to the `StompIntegrationTests` to trace failures in the future.
* Also change the test `Start/Stop` messages to the `WARN` level in the `LogAdjustingTestSupport`, because `DEBUG` doesn't work everywhere.
Maybe some logging systems mix misconfiguration, like we have in the WebSocket modules with the Embedded Tomcat
JIRA: https://jira.spring.io/browse/INT-3984
The `PropertySourcesPlaceholderConfigurer implements BeanFactoryPostProcessor, PriorityOrdered`
meaning that it is run before any other regular `BeanFactoryPostProcessor`,
like the `IntegrationConfigurationBeanFactoryPostProcessor` has been before.
With such an order any `BeanDefinition` declaration from the `IntegrationConfigurationBeanFactoryPostProcessor` process
ended up without `PP` resolutions.
* Rework `IntegrationConfigurationBeanFactoryPostProcessor` to the `BeanDefinitionRegistryPostProcessor`,
which makes it be loaded as early as possible. See `PostProcessorRegistrationDelegate`:
````java
// Now, invoke the postProcessBeanFactory callback of all processors handled so far.
invokeBeanFactoryPostProcessors(registryPostProcessors, beanFactory);
invokeBeanFactoryPostProcessors(regularPostProcessors, beanFactory);
````
* Modify `EnableIntegrationTests` and `ChannelSecurityInterceptorSecuredChannelAnnotationTests` to ensure that the change
meets the `PP` and `SpEL` resolutions during bean definition phase.
* Fix the timing issue in the `JdbcMessageStoreChannelIntegrationTests`
Fixes https://build.spring.io/browse/INT-MJATS41-593
* Fix `LoggingHandler` JavaDoc
* Fix `JdbcOutboundGatewayParserTests` timing and race condition issues
* Add `mock(BeanFactory.class)` to the `JpaOutboundChannelAdapterTests` for `JpaExecutor`
* Fix `JpaOutboundChannelAdapterTests` for the incorrect transaction usage
checkstyle EmptyBlock
checkstyle fixRightCurly Script
checkstyle EmptyStatement
checkstyle RightCurly
checkstyle TailingWhite
checkstyle NeedBraces
Fix the line separator in the `fixRightCurly.gradle`
fixes
fixModifiers after fixFinal
Revert CachingSessionFactory
Class is spied in tests.
checkstyle - Import Rules
checkstyle InterfaceIsType
checkstyle InnerTypeLast
checkstyle OneStatementPerLine
CovariantEquals
OneTopLevelClass
* Revert `'\n'` -> `System.lineSeparator()` in the Gradle scripts to meet Git `autocrlf = true` on Windows
* Fix timing issue with the `LastModifiedFileListFilterTests`, when the `age = 1` might not be enough for the file object when we have some delay before checking
* `gradlew clean check -x test --parallel --continue` - to collect reports
* `gradlew fixThis --parallel` - to fix all possible vulnerabilities. With `-Dfile.encoding=UTF-8` on Windows
Since the `RequireThisCheck` doesn't see parents for anonymous classes (e.g. `Runnable` callback), its report doesn't contains the outer class name with `this.`,
therefore we still have to fix those cases manually.
Thanks to the wrong `replacer` just with `this.` we have uncompilable code enough easy to find problems.
Not so easy to fix for good readability though...
* Upgrade to Grade 2.12
* Upgrade to SonarQube native plugin
The fix contains at about 300 files. So, will be done on merge.
Fix `fixThis.gradle` according PR comments
Apply `fixThis` and also `fixModifiers` for test classes.
Fix some `this.` inner issues manually.
Make code polishing for long lines after `fixThis`
Fix conflicts and vulnerabilities after the rebase
Add `fixModifiers.gradle` to run after `check` task.
The `RedundantModifierChecker` doesn't catch all errors at once.
We need run `check -> fixModifiers` pair several times to reach `SUCCESSFUL`.
The `fixThis` has been ported from the SA, but without applying.
* Polishing for `fixModifiers.gradle`
* Fix all redundant modifier with the `fixModifiers.gradle`
NOTE: Since all these task modify files on Windows we have to run them with the `-Dfile.encoding=UTF-8`.
Otherwise they are read and write with the Windows default `cp1251` making them incompatible with Unix system.
