* INT-4376: Upgage to Log4J 2
JIRA: https://jira.spring.io/browse/INT-4376
* Deprecate Log4J 1.x components in favor of newly added
a `Log4j2LevelAdjuster` JUnit `@Rule`
* Update all the logging configuration to Log4J 2
* Polishing after testing
JIRA: https://jira.spring.io/browse/INT-4342
Use similar code to Spring AMQP to add white list support for Integration's
use of the `DeserializingMessageConverter`; introduce the `WhiteListDeserializingMessageConverter`.
Polishing
Missed this change in PR.
Fix XSD attribute
The `PollableChannelPublisherAdapter` is based on the poll model of the
`FluxSink` and iterate and poll downstream `PollableChannel` until
there is an item or `n > 0`.
Having `take(6)` we end up with the cancel from the downstream
`Subscriber` after the batch is filled, but at the same time we continue
to poll the upstream source because `n` is like `Long.MAX_VALUE`.
* The proper way to interact is check for the `!sink.isCancelled()`
as well.
This way cancelled `sink` won't "steal" data from other subscribers
Fix compatibility with the latest dependencies
* Upgrade to Gradle 4.1
* Upgrade as much dependencies as possible
* Fix MongoDB module to resolve deprecation in the latest driver
* Increase receive timeout in the `ResequencerTests`
* Restore generic argument for the method reference in the `ReactiveStreamsTests`
* Fix `WebFluxInboundEndpoint` for the compatibility with the latest Reactor API
* Upgrade to Gradle 3.5, SS-5.0, Hibernate-5.2.10, Mockito-2.7.22, Pah-1.1.1
And some other minor upgrades
* Fix deprecations for Mockito compatibility
* Fix `ServletWebSocketHandlerRegistry` deprecations
* Fix SD-Mongo deprecations
* Tweak JMX tests to avoid dangling threads after tests exist
* Increase timeouts in some polling tests
JIRA: https://jira.spring.io/browse/INT-4221
* Upgrade to those versions of Spring project dependencies which potentially will provide similar fix
**Cherry-pick to 4.3.x**
JIRA: https://jira.spring.io/browse/INT-4196
Add `CollectionCallback` option to the `MongoDbOutboundGateway`
Add XML support to CollectionCallback
Fix PR comments
Fix issue with javadoc parsing
* Polishing code style, JavaDocs and some Docs
Since we can have a result message in the `replyChannel` before the TX commit, a subsequent adapter `stop()` causes the current task interruption,
therefore TX rallback.
That's why we see `semaphore.acquire()` interruption and don't see message in the `afterCommitChannel`
**Cherry-pick to 4.3.x**
Change all the `BUILD-SNAPSHOT`s to their latest Milestones
Fix compatibility with those Milestones
Upgrade to Spring AMQP 2.0.0.M1
Upgrade to Spring Data Kay
Gemfire now is based on the Apache Geode, so changed all the imports to proper new packages
MongoDB now is based on the Mongo 3 Driver, therefore many breaking changes. Mostly to the mapping part
Fix Checkstyle violation
Fix race condition in the `MongoDbInboundChannelAdapterIntegrationTests` around `QueueChannel` and tx commit
Add `-s` to Travis Gradle command to see stack trace about `MongoDbMetadataStoreTests` problem
Test only MongoDB module on Travis with -d
Add addon to Travis to pull MongoDB-3.0
Fix `MongoDbAvailableRule` for MongoDB 3.0 Driver style
Increase `serverSelectionTimeout` to `100` in the `MongoDbAvailableRule`.
Looks like `0` isn't good value to get immediate answer
JIRA: https://jira.spring.io/browse/INT-4176
The `DelayerHandlerRescheduleIntegrationTests` doesn't close `context` in the end of test.
That causes clashes for `beanFactoryResolver` in other tests.
When those tests are fail, we come into condition when new processes can't be started/stopped because of effect of non-stopped context
* Fix all the `DelayerHandlerRescheduleIntegrationTests` to stop `context` in the end of test
* Remove unused files in the `gemfire/store` test package
* Add current date value to the `CacheServerProcess` name to avoid clashes with other ran on the same host
* Add NPE check for `region` in the `GemfireMetadataStoreTests`
* Extract `@BeforeClass/@AfterClass` for the `CacheWritingMessageHandlerTests` to start only one Gemfire cache per class
**Cherry-pick to 4.3.x & 4.2.x**
https://sonar.spring.io/component_issues?id=org.springframework.integration%3Aspring-integration%3Amaster#resolved=false|types=BUG
In `IntegrationFlowRegistration` double check locking is ok for `inputChannel`
because we're assigning an existing object, but `MessagingTemplate` constructs
a new object for which double check locking doesn't work.
In any case, for both these items, the chance of concurrent access is extremely low
and is idempotent anyway, so remove double check locking.
Several inner classes can be static.
Other minor fixes.
JIRA: https://jira.spring.io/browse/INT-4012
Previously to configure a `priority` for the `MessageChannel` we should configure `QueueChannel` for particular `MessageStore`.
Although the target priority logic is really in the `MessageStore` implementation, it isn't so obvious why we can't use `PriorityChannel` instance for `MessageStore` case as well.
* Add `PriorityCapableChannelMessageStore` and `MessageGroupQueue` based ctors to the `PriorityChannel` for consistency.
* Delegate the logic to the super `QueueChannel` as before
* Rework `PointToPointChannelParser` and Java DSL components to reflect a new state of the `PriorityChannel`
* Improve Docs on the matter
JIRA: https://jira.spring.io/browse/INT-4162
INT-4162: IP DSL - Phase I Connection Factories
Phase 2 - Adapters/Gateways
Phase 3 - UDP
Polishing - PR Comments
Remove unnecessary generics.
More Polishing - PR Comments
Checkstyle/Javadoc Fixes
ComponentsRegistration Improvements
Add ...Spec CTORs to avoid the issue described here:
https://github.com/spring-projects/spring-integration-java-dsl/issues/137
Also other PR comments.
Fix UDP Spec Inheritance
Jdbc Lock Test Log Adjuster
Fix Mongo Test Race Conditions
* Fox typos in the `AbstractUdpOutboundChannelAdapterSpec`
* Improve `IpIntegrationTests.testUdpInheritance()`
* Change `fixed-rate` to `fixed-delay` in the `MongoDbInboundChannelAdapterIntegrationTests-context` to pursue single-threaded environment from poller for test
JIRA: https://jira.spring.io/browse/INT-4163
When `MessageSource` is proxy, the `TransactionSynchronizationManager.getResource(this)` logic in the `MessageSource` doesn't work,
because TX resource is bound to the `Proxy` in the `SourcePollingChannelAdapter`
* Introduce `SourcePollingChannelAdapter.originalSource` property and store there a target `MessageSource` object extracted from the AOP Proxy
* Use `originalSource` as a resource to bind to the TX
* Modify `MongoDbInboundChannelAdapterIntegrationTests` to ensure that `AbstractMessageSourceAdvice` proxying the `MessageSource` doesn't effect `TransactionSynchronizationManager.getResource(this)` logic
* Refactor for some MongoDb test to rely on the `@RunWith(SpringJUnit4ClassRunner.class)` for context loading for better test class performance
**Cherry-pick to 4.3.x, 4.2.x**
Fallback to provided source if `target` from Proxy is `null`
Fix [UnusedImport] issue
Polishing - PR Comments and Closeable Warnings
Eclipse emits bogus warnings with exceptions in lambdas.
Even though the lambda might run on another thread, elipse thinks it could
cause the context to not be closed.
SPR-14854: MessageChannel is now a @FunctionalInterface
* Additional Lambda polishing and some code style fixes
JIRA: https://jira.spring.io/browse/INT-2460,
https://jira.spring.io/browse/INT-4122
Since the main purpose of the `MessageStore` to persist message for durability and only,
it doesn't make sense to modify `Message` for additional headers like `SAVED` and `CREATED_DATE`.
Such a logic should be a part of metadata stored together with the message.
And it is provided by the out-of-the-box `MessageStore` implementation.
In addition we free ourselves from the reflection operations to retain `ID` and `TIMESTAMP` headers when we add `SAVED` and `CREATED_DATE`
* Control "already saved" logic in the `JdbcMessageStore`s via `DuplicateKeyException` on the `INSERT`.
This is much effective then additional `SELECT` in case of `SAVED` before
* Control "already saved" logic in the `AbstractConfigurableMongoDbMessageStore` via `DuplicateKeyException` on the `INSERT`.
Since `MongoDbMessageStore` doesn't provide extra `messageId` field, perform extra `SELECT` before store document.
Anyway the `MongoDbMessageStore` isn't recommended for use.
We may consider to deprecate it
* Control "already saved" logic in the `AbstractKeyValueMessageStore` via `putIfAbsent` operation
With this fix we persist message in the store as is without any modifications when we perform standard serialization procedure.
Any custom serializers should consider to use `MutableMessageBuilder` if there is a requirement to retain `ID` and `TIMESTAMP`
Rework `MongoDbMetadataStore.putIfAbsent()` to normal `findAndModify()` with particular `$setOnInsert`.
Technically the MongoDB query looks like:
```
db.collection.findAndModify({
query: { _id: $key },
update: {
$setOnInsert: { value: $value } // perform modification only on upsert
},
new: false, // don't return new doc if one is upserted
upsert: true // insert the document if it does not exist
})
```
Move single import to the appropriate JavaDoc
Polishing after rebase
DEBUG messages in `doStoreIfAbsent()` implementations
* To keep track of the extra message information in the `MessageStore`, without `Message` modification, introduce `MessageMetadata` and `MessageHolder`
* Add `MessageStore#getMessageMetadata()`
* Modify MongoDb `MessageStore` to add extra `timestamp` for individual message
* Fix `ConcurrentAggregatorTests` race condition.
Since currently the default release strategy is `SimpleSequenceSizeReleaseStrategy` which is just based on the `MessageGroup` size, there is no guaranty which messages will complete the group in concurrent environment.
The test is really based on the `SequenceAwareMessageGroup` logic to discard the message with the same `correlationId`
Fix `@Copyright` format
Move cast to `MessageHolder` after `Assert.isInstanceOf(MessageHolder.class, messageHolder)`
Retain backward compatibility in the `AbstractKeyValueMessageStore`
Polishing
JIRA: https://jira.spring.io/browse/INT-4119
* Mark `MessageDocument` with the `@AccessType(AccessType.Type.PROPERTY)` for better performance via bean accessors
* Add more getters for `MessageDocument` properties
* Leave `MessageWrapper` as is since it has a complex internal logic which may be broker via accessors
* Upgrade to Checkstyle `7.1`
* Relax `RequireThis` rule a bit. Right now it does the effort only in case of overlapping. See https://github.com/checkstyle/checkstyle/issues/2362 for more info
* Enable some annotation rules and provide fixes for violations
* Enable `tabs indents` rule. This was the biggest fix in this PR
* Resolve `TODO` in the `MessagingMethodInvokerHelper` and fix tests to meet `IllegalStateException` now
Remain Checkstyle version `6.16.1` and reinstate `RequireThis` rule
The latest Checkstyle has a bug with local scope variables if they have the same names as property.
Revert some literals splitting
Fix some line length exceeding and code style
JIRA: https://jira.spring.io/browse/INT-3913
* Remove deprecated classes and methods/constructors, deprecated XML attributes
* Remove `TcpConnectionEventListeningMessageProducer` and rework tests logic to the `ApplicationEventListeningMessageProducer`
* Fix several typos
* Remove/rework deprecated entities mentioning
JIRA: https://jira.spring.io/browse/INT-4082
Starting with Spring Data 1.9 the `MappingContextIsNewStrategyFactory` relies on a newly introduced `PersistentEntities` and doesn't register entities lazily any more.
Such a change finishes with the `Unsupported entity` exception when an auditing is switched on (`<mongo:auditing/>`)
for the `MongoDbMessageStore` and `AbstractConfigurableMongoDbMessageStore` internally created `MongoTemplate` and `MessageWrapper` and `MessageDocument` SI internal entities.
* Don't register `ApplicationContext` into internally created `MongoTemplate`s since to avoid entity events emitting for `MessageWrapper` and `MessageDocument`
* Pull `MongoDbMessageBytesConverter` to the top-level class to let customize `MappingMongoConverter` properly if there is need to audit `MessageDocument` anyway,
what can be possible via external injections into `AbstractConfigurableMongoDbMessageStore` implementation
* Fix `mongodb.adoc` for some typos
**Cherry-pick to 4.2.x**
JIRA: https://jira.spring.io/browse/INT-4023
There is no guaranty that following `System.currentTimeMillis()` return the same value.
That is how the `assertEquals()` test for `created` and `modified` value may fail sporadically.
* Fix `MongoDbMessageStore` and `ConfigurableMongoDbMessageStore` do not use a new `System.currentTimeMillis()`
in case of `group create`
**Cherry-pick to 4.2.x**
https://build.spring.io/browse/INT-B43-164/
* Fix CheckStyle for `System. out .println(` expression commenting out such a code line
* Fix generics in the `JdbcMessageStoreTests`
* Increase receive timeout in the `FileMessageHistoryTests`
* Change the `AsyncAmqpGatewayTests` logic to deal with the same channel for both `ack/nack`.
The `timeout` case without reply may end up with the `Consumer cancel`, therefore `nack` not `ack` like we expected
JIRA: https://jira.spring.io/browse/INT-3387,
https://jira.spring.io/browse/INT-3806
* Introduce
```
MessageGroupStore
void addMessagesToGroup(Object groupId, Message<?>... messages);
```
And implement it in all stores.
* Use new `addMessagesToGroup` where it is reasonable, e.g. `DelayHandler`
* Optimize test-case to use a new store method (where it is possible)
* Fix timing delays in the `JdbcMessageStoreTests`
* Introduce `PersistentMessageGroup`
* Add `AbstractMessageGroupStore#proxyMessageGroupForLazyLoad` to wrap the raw `MessageGroup` to the `PersistentMessageGroup` for lazy-load
* Rework `MessageGroupMetadata` do not be `immutable` and allow to store/restore in the `AbstractKeyValueMessageStore` only the `MessageGroupMetadata`
* Refactor `ResequencingMessageHandler` and `SequenceSizeReleaseStrategy` a bit for better performance when interact with the `MessageGroup`
* Add `AbstractMessageGroupStore#setLazyLoadMessageGroups` to switch off the `lazy-load` behavior and restore the previous full `MessageGroup` logic
* Add `What's New` note and `message-store.adoc` paragraph for the lazy-load functionality
`GroupType.PERSISTENT` and not lazy by default
PR Comments
Fix `JdbcMessageStoreTests` timing issues
Address PR comments
* Add performance test to the `ConfigurableMongoDbMessageGroupStoreTests`
* Add JavaDocs for the `MessageGroupFactory` methods
* Add `log4j.properties` into the `test` MongoDB module for better traceability
* Fix `JdbcMessageStore#getOneMessageFromGroup()` over the `doPollForMessage()` delegation.
The `jdbcTemplate.queryForObject()` requires exactly one and only one raw in `resultSet`
* Add performance test results into the `message-store.adoc`
JIRA: https://jira.spring.io/browse/INT-3978
Previously the `MessageHistory.write()` always used `MessageBuilder` to populated the `history` header.
Since the `MessageBuilder` doesn't care about the `source` message type, we might lose some important information,
like `AdviceMessage.inputMessage`
* Add conditional logic into the `MessageHistory` for all known `Message<?>` implementations
* Provide appropriate tests in the `MessageHistoryTests` for known `Message<?>` implementations
* Make `AdviceMessage` generic and fix all affected code on the matter
* Since `MutableMessage` is public already, fix `MongoDbMessageStore.DBObjectToMutableMessageConverter` to use
`MutableMessage` type directly.
Add WARN for non-standard `Message` type
checkstyle EmptyBlock
checkstyle fixRightCurly Script
checkstyle EmptyStatement
checkstyle RightCurly
checkstyle TailingWhite
checkstyle NeedBraces
Fix the line separator in the `fixRightCurly.gradle`
fixes
fixModifiers after fixFinal
Revert CachingSessionFactory
Class is spied in tests.
checkstyle - Import Rules
checkstyle InterfaceIsType
checkstyle InnerTypeLast
checkstyle OneStatementPerLine
CovariantEquals
OneTopLevelClass
* Revert `'\n'` -> `System.lineSeparator()` in the Gradle scripts to meet Git `autocrlf = true` on Windows
* Fix timing issue with the `LastModifiedFileListFilterTests`, when the `age = 1` might not be enough for the file object when we have some delay before checking
* `gradlew clean check -x test --parallel --continue` - to collect reports
* `gradlew fixThis --parallel` - to fix all possible vulnerabilities. With `-Dfile.encoding=UTF-8` on Windows
Since the `RequireThisCheck` doesn't see parents for anonymous classes (e.g. `Runnable` callback), its report doesn't contains the outer class name with `this.`,
therefore we still have to fix those cases manually.
Thanks to the wrong `replacer` just with `this.` we have uncompilable code enough easy to find problems.
Not so easy to fix for good readability though...
* Upgrade to Grade 2.12
* Upgrade to SonarQube native plugin
The fix contains at about 300 files. So, will be done on merge.
Fix `fixThis.gradle` according PR comments
Apply `fixThis` and also `fixModifiers` for test classes.
Fix some `this.` inner issues manually.
Make code polishing for long lines after `fixThis`
Fix conflicts and vulnerabilities after the rebase
Add `fixModifiers.gradle` to run after `check` task.
The `RedundantModifierChecker` doesn't catch all errors at once.
We need run `check -> fixModifiers` pair several times to reach `SUCCESSFUL`.
The `fixThis` has been ported from the SA, but without applying.
* Polishing for `fixModifiers.gradle`
* Fix all redundant modifier with the `fixModifiers.gradle`
NOTE: Since all these task modify files on Windows we have to run them with the `-Dfile.encoding=UTF-8`.
Otherwise they are read and write with the Windows default `cp1251` making them incompatible with Unix system.
JIRA: https://jira.spring.io/browse/INT-3950
Previously there was a mention of the `MessageGroupStore.expireMessageGroup(groupId)` which just doesn't existing
in the Framework and never has been there.
* Fix the documentation for the existing `MessageGroupStore.expireMessageGroups(timeout)`.
Although the mention there of `Control Bus` requires to have `@ManagedOperation` on the method.
* Add `@ManagedOperation` for the `MessageGroupStore.expireMessageGroups(timeout)` and confirm with the test-case: `AggregatorWithMessageStoreParserTests`
* Fix the same docs in the XSD for `<aggregator>`
* Fix other typos in the `aggregator.adoc` and `resequencer.adoc`
JIRAs:
https://jira.spring.io/browse/INT-3830https://jira.spring.io/browse/INT-3523https://jira.spring.io/browse/INT-3846
* Fix the OOM condition, when we `release()` `UpperBound` independently of the previous `remove` result (https://jira.spring.io/browse/INT-3846)
* Fix "confuse" around `groupCapacity`, when we really didn't care about individual groups (https://jira.spring.io/browse/INT-3523)
* Add `upperBoundTimeout` to have a hook to wait some time for the empty slot in the store (https://jira.spring.io/browse/INT-3830)
* Fix some JavaDocs warnings
* Fix some typos
* Fix inconsistency in the `DelayHandler` around `removeMessageFromGroup` when `MS` is `SimpleMessageStore`
* Remove `UpperBound.release()` operation from `SimpleMessageStore.removeGroup()`.
The waiting process should worry about the new `UpperBound` instance.
Some other polishing
`tryAcquire` outside of the `lock`
Move `tryAcquire` within the `addMessageToGroup` outside of `lock`.
But do that only for groups which already exist.
For the new groups we have a fresh `UpperBound`, so no need to worry about dead lock and
we can obtain a permit immediately.
SimpleMessageGroup: BlockingQueue -> LinkedHashSet
`SimpleMessageStore`: use "unsynchonized" `SimpleMessageGroup`
Make some synchronization fixes according to the migration to the `LinkedHashSet`
Avoid extra `Collection`
`ResequencingMessageHandler`: compare `size()` of collections instead of `containsAll()`
Fix `ConcurrentModificationException` in the `AbstractKeyValueMessageStore`
Add `SimpleMessageStore.clearMessageGroup()`
Accept polishing and fix `RedisChannelMessageStoreTests`
`@Deprecated` `MessageGroupStore.removeMessageFromGroup()`
Fix some typos
Introduce `SimpleMessageGroupFactory`
Extract `MessageGroupFactory` and address PR comments
Polishing after rebase
JavaDocs and Reference Manual
Fix JavaDocs
JIRA: https://jira.spring.io/browse/INT-3886
An SO user reported a short message delivery when an NIO connection was timed out.
See JIRA for link.
We could not produce the problem with his deserializer but it did identify a problem
with the standard `ByteArrayRawSerializer`.
With NIO, socket timeouts were reported to the deserializer as a normal EOF (-1). This
caused the raw serializer to emit a short message - it's signal for end of message is the
socket closure.
We can't treat a timeout as a normal EOF.
If the socket is forcibly timed out due to no recent data, throw a `SocketTimeoutException` to
the deserializer.
Just in case the old behavior is being relied upon, a boolean has been added to restore that
behavior. This is not recommended, as using timeout to delimit messages is not reliable.
* Fix typos
* Increase `MongoDbInboundChannelAdapterIntegrationTests` timeouts
* Some code polishing
JIRA: https://jira.spring.io/browse/INT-3642
Currently, `removeMessageFromGroup` rebuilds the group on every removal.
In every case where this method is used in the framework, the result is not used.
Add `removeMessagesFromGroup` that removes a collection of messages and returns no result.
INT-3642: Polishing - PR Comments
INT-3642: Polishing and Fix Group Metadata Size
JIRA: https://jira.spring.io/browse/INT-3661
Previously there were a lot of noise from the `PostProcessorRegistrationDelegate$BeanPostProcessorChecker` for early access for beans.
That may produce some side-effects when some of `BeanFactoryPostProcessor`s won't adjust those beans.
The issue is based on two facts:
1. Loading beans from `BPP`, e.g. `IntegrationEvaluationContextAwareBeanPostProcessor` (or `ChannelSecurityInterceptorBeanPostProcessor` - https://jira.spring.io/browse/INT-3663)
2. Loading beans from `setBeanFactory()/setApplicationContext()` container methods
* Move all code from `setBeanFactory()` with access to the `BeanFactory` (e.g. `this.messageBuilderFactory = IntegrationUtils.getMessageBuilderFactory(this.beanFactory);`)
to some other lazy-load methods like `getMessageBuilderFactory()`
* Fix parser tests to remove `messageBuilderFactory` tests since there is no activity for target components to lazy-load them
* Polish some test according the new lazy-load logic
* Rework `IntegrationEvaluationContextAwareBeanPostProcessor` to the `SmartInitializingSingleton` and make it `Ordered`
* Populate `beanFactory` for the internal instance of `connectionFactory` in the `TcpSyslogReceivingChannelAdapter`
* Populate `beanFactory` for the internal `UnicastReceivingChannelAdapter` in the `UdpSyslogReceivingChannelAdapter`
* Add `log.info` that `UdpSyslogReceivingChannelAdapter` overrides `outputChannel` for the provided `UnicastReceivingChannelAdapter`
* Change the internal `MessageChannel` in the `UdpSyslogReceivingChannelAdapter` to the `FixedSubscriberChannel` for better performance
* Fix `AbstractExpressionEvaluator`
* Add JavaDocs for the `IntegrationEvaluationContextAware`
Fix `MongoDbMessageStoreClaimCheckIntegrationTests`
Addressing PR comments
