Commit Graph

712 Commits

Author SHA1 Message Date
Artem Bilan
092d876fae Remove DSL Deprecated Methods and Others
Since the DSL code is new in this version there is no reason to keep deprecated method

* Remove deprecated `resequence()` and `aggregate()` in the `IntegrationFlowDefinition`
* Remove `InternalAggregatingMessageHandler` with the `MessageGroupProcessorWrapper` delegation logic in favor of newly introduced `AbstractCorrelatingMessageHandler.setOutputProcessor()`
* Rework `Promise` Gateway Docs to the `Mono`
* Rename `AsyncGatewayTests` "promise" words to "mono"
* Resolve `com.rabbitmq.client.FlowListener` deprecation

RecipientListRouter DSL refactoring

Previously there was a DSL specific `DslRecipientListRouter` to overcome the lack of setters in the target `RecipientListRouter` and its `Recipient`

* Add `channelName` variant for the `Recipient`
* Add several new `addRecipient()` methods to the `RecipientListRouter`

All those improvements allow to avoid extra bridge (or adapter) component between Java DSL and target `RecipientListRouter`

* Make `LambdaMessageProcessor` and `GatewayMessageHandler` as `public` classes and move them to the appropriate packages

Address PR comments

Add `isLambda()` condition to `RecipientListRouterSpec` to avoid ambiguity for the provided `GenericSelector` impl

checkstyle polishing
2016-11-28 17:32:18 -05:00
Gary Russell
57806b92c9 INT-4171: Document Solution for FTPS Shared SSL
JIRA: https://jira.spring.io/browse/INT-4171

Since the solution requires reflection on `sun` classes, document only.

Also fix some PDF text overflows in (S)FTP.
2016-11-26 16:24:33 -05:00
Gary Russell
3035bc716d INT-4138: MQTT: Outbound Adapter Improvements
JIRA:https://jira.spring.io/browse/INT-4138

Expressions for topic, qos, retained.

Also change inbound mapping to `RECEIVED_...` headers.

Fix some minor asciidoc problems in (s)ftp.

Rework Qos/Retained Expressions/Defaults

Encapsulate the logic entirely in the converter.

Polishing - PR Comments
2016-11-16 11:22:10 -05:00
Artem Bilan
507764a3d6 INT-4153: Feed Java DSL and other improvements
JIRA: https://jira.spring.io/browse/INT-4153

* Remove deprecated `FeedFetcher` usage
* Introduce `Resource` based ctor for the `FeedEntryMessageSource`
* Add `SyndFeedInput` option and short-hand `preserveWireFeed` for internal `SyndFeedInput` instance
* Reflect the changes in the XSD for Feed
* Change ROME dependency from deprecated `rome-fetcher` to just `rome` as it is recommended by ROME team
* Port Java DSL for Feed module and reflect aforementioned changes in the `Feed` factory and `FeedEntryMessageSourceSpec` as well
* Document changes and mention Feed Java DSL, too
2016-11-10 14:44:23 -05:00
Artem Bilan
47cd4e4540 INT-4012: PriorityChannel refinement
JIRA: https://jira.spring.io/browse/INT-4012

Previously to configure a `priority` for the `MessageChannel` we should configure `QueueChannel` for particular `MessageStore`.
Although the target priority logic is really in the `MessageStore` implementation, it isn't so obvious why we can't use `PriorityChannel` instance for `MessageStore` case as well.

* Add `PriorityCapableChannelMessageStore` and `MessageGroupQueue` based ctors to the `PriorityChannel` for consistency.
* Delegate the logic to the super `QueueChannel` as before
* Rework `PointToPointChannelParser` and Java DSL components to reflect a new state of the `PriorityChannel`
* Improve Docs on the matter
2016-11-10 14:19:26 -05:00
Artem Bilan
5cca8e8e01 INT-3770: Add TX Support from Mid-flow
JIRA: https://jira.spring.io/browse/INT-3770,
https://jira.spring.io/browse/INT-4107

Having `TransactionHandleMessageAdvice` we can start TX from any `MessageHandler.handleMessage()`

* Add `<transactional>` alongside with the `<request-handler-advice-chain>` for those components which produce reply
* Merge `<transactional>` and `<request-handler-advice-chain>` configuration to a single `ManagedList`
* Rework JPA `<transactional>` in favor of common solution
* Some polishing and refactoring

AbstractPollingEndpoint: avoid `new ArrayList` if we don't  have `receiveOnlyAdvice`s
2016-11-07 09:06:17 -05:00
Artem Bilan
2f0b377cfb INT-4091: Aggregator: Removal for Empty Groups
JIRA: https://jira.spring.io/browse/INT-4091

When `empty-group-min-timeout` is configured and `expireGroupsUponCompletion == false` and normal or partial sequences group release happens,
schedule the group for removal after `empty-group-min-timeout`, since it is empty already.
That lets to avoid `MessageGroupStoreReaper` configuration just for cleaning empty groups.

* Retrieve the group `lastModified` to check if group is still valid for removal
* Remove `ScheduledFeature` in the remove task
* Polishing for debug messages to reflect the current logic
* Reschedule empty group removal task in case of `InterruptedException` on the `lock.lockInterruptibly()`
* Fix typos in docs

Fix race condition between `groupStore.expireMessageGroups()`and `TaskScheduler`  in the `AbstractCorrelatingMessageHandlerTests.testReaperReapsAnEmptyGroupAfterConfiguredDelay()`

Also check groupSize for removal decision

Address PR comments
2016-11-01 13:50:09 -04:00
Gary Russell
a43299213e INT-4149: Improve (S)FTP Recursive MGET
JIRA: https://jira.spring.io/browse/INT-4149

Add an option to always pass directories while recursing.

* Fix typos in docs
* Revert `SyslogReceivingChannelAdapterTests` `Thread.sleep()` fixes
2016-10-28 16:22:46 -04:00
Gary Russell
e6c70b5cc6 File Streaming Doc Polishing
http://stackoverflow.com/questions/40135591/get-string-from-spring-ftp-streaming-inbound-channel-adapter/40136021#40136021

Show an example for removing the remote file after processing.
2016-10-23 10:16:40 -04:00
Gary Russell
5a44216201 INT-4140: File Streaming Adapter: Add maxFetchSize
JIRA: https://jira.spring.io/browse/INT-4140
2016-10-18 10:32:23 -04:00
Gary Russell
3c284871c3 INT-4139: Add CORS Config to Graph Annotation
JIRA: https://jira.spring.io/browse/INT-4139

Doc Polishing

Polishing - Fix CORS Test and Revert XML

- Use standard CORS configuration with XML.
2016-10-17 14:49:30 -04:00
Artem Bilan
a1f554c04d INT-2460: Remove Message Modification Logic in MS
JIRA: https://jira.spring.io/browse/INT-2460,
https://jira.spring.io/browse/INT-4122

Since the main purpose of the `MessageStore` to persist message for durability and only,
it doesn't make sense to modify `Message` for additional headers like `SAVED` and `CREATED_DATE`.
Such a logic should be a part of metadata stored together with the message.
And it is provided by the out-of-the-box `MessageStore` implementation.
In addition we free ourselves from the reflection operations to retain `ID` and `TIMESTAMP` headers when we add `SAVED` and `CREATED_DATE`

* Control "already saved" logic in the `JdbcMessageStore`s via `DuplicateKeyException` on the `INSERT`.
This is much effective then additional `SELECT` in case of `SAVED` before
* Control "already saved" logic in the  `AbstractConfigurableMongoDbMessageStore` via `DuplicateKeyException` on the `INSERT`.
Since `MongoDbMessageStore` doesn't provide extra `messageId` field, perform extra `SELECT` before store document.
Anyway the `MongoDbMessageStore` isn't recommended for use.
We may consider to deprecate it
* Control "already saved" logic  in the `AbstractKeyValueMessageStore` via `putIfAbsent` operation

With this fix we persist message in the store as is without any modifications when we perform standard serialization procedure.
Any custom serializers should consider to use `MutableMessageBuilder` if there is a requirement to retain `ID` and `TIMESTAMP`

Rework `MongoDbMetadataStore.putIfAbsent()` to normal `findAndModify()` with particular `$setOnInsert`.
Technically the MongoDB query looks like:
```
db.collection.findAndModify({
  query: { _id: $key },
  update: {
    $setOnInsert: { value: $value } // perform modification only on upsert
  },
  new: false,   // don't return new doc if one is upserted
  upsert: true // insert the document if it does not exist
})
```

Move single import to the appropriate JavaDoc

Polishing after rebase
DEBUG messages in `doStoreIfAbsent()` implementations

* To keep track of the extra message information in the `MessageStore`, without `Message` modification, introduce `MessageMetadata` and `MessageHolder`
* Add `MessageStore#getMessageMetadata()`
* Modify MongoDb `MessageStore` to add extra `timestamp` for individual message
* Fix `ConcurrentAggregatorTests` race condition.
Since currently the default release strategy is `SimpleSequenceSizeReleaseStrategy` which is just based on the `MessageGroup` size, there is no guaranty which messages will complete the group in concurrent environment.
The test is really based on the `SequenceAwareMessageGroup` logic to discard the message with the same `correlationId`

Fix `@Copyright` format

Move cast to `MessageHolder` after `Assert.isInstanceOf(MessageHolder.class, messageHolder)`

Retain backward compatibility in the `AbstractKeyValueMessageStore`

Polishing
2016-10-13 13:19:37 -04:00
Gary Russell
7c12288d2e GH-1928: Roles: Doc Polishing
Fixes #1928 https://github.com/spring-projects/spring-integration/issues/1928
2016-10-12 11:58:33 -04:00
Gary Russell
890ad63584 INT-4137: Improve Aggregator Performance
JIRA: https://jira.spring.io/browse/INT-4137

Aggregators with `SequenceSizeReleaseStrategy` do not perform well with large groups
because of an O(n) linear search to reject (discard) duplicate sequences.

Change the aggregator to use a `SimpleSequenceSizeReleaseStrategy` by default, unless
`setReleasePartialSequences(true)`.

This avoids the linear search, which is not needed for most splitter -> ... -> aggregator
flows.

Log a warning if SSRS is used.

Polishing - PR Comments

* Remove `this.logger.isWarnEnabled()` since it is redundant when our log message is just string constant
2016-10-11 17:14:55 -04:00
Artem Bilan
8f0fa3468f INT-832: Add File Relative Path to Message Headers
JIRA: https://jira.spring.io/browse/INT-832

When we scan directory recursively for files (e.g. `WatchServiceDirectoryScanner`), it can be useful to get access to the relative path from the `Message`, e.g. on the `FileWritingMessageHandler` side to restore the original structure

* Add a `FileHeaders.FILENAME` into the outbound `Message` from the `FileReadingMessageSource`
* The result of that header is like a removal of leading `this.directory.getAbsolutePath()` in the target `File.getAbsolutePath()`.
In case of not recursion we get just only regular file name.

* Introduce `FileHeaders.RELATIVE_PATH`
* Populated `FileHeaders.RELATIVE_PATH`, `FileHeaders.FILENAME`, `FileHeaders.ORIGINAL_FILE` in the `FileReadingMessageSource`
* File `FileTailingMessageProducerSupport` to populate `FileHeaders` properly
* Introduce ctor for the `LastModifiedFileListFilter` for better Java configuration experience
* Add docs for changes

Doc Polishing
2016-10-11 13:04:21 -04:00
Gary Russell
2edf766e49 INT-4136: Support DirectMessageListenerContainer
JIRA: https://jira.spring.io/browse/INT-4136

Fix a couple typos
2016-10-11 10:46:30 -04:00
Artem Bilan
53d80b5694 INT-3903: user prefix as "" for HTTP headers
JIRA: https://jira.spring.io/browse/INT-3903

According to RFC-6648, the "X-" prefix has been deprecated and now any use-specific headers can be mapped without any prefix

* Fix `DefaultHttpHeaderMapper` for the `""` as a `userDefinedHeaderPrefix` by default

Document `X-` changes
2016-10-10 16:58:12 -04:00
Piotr Zygielo
b6aed111f4 Update bean id to reflect direction
Obvious Fix
2016-10-08 16:51:29 -04:00
Gary Russell
9673a02c7d INT-4131: Add delayExpression to AMQP Outbounds
JIRA: https://jira.spring.io/browse/INT-4131

Specify an expression on the outbound endpoints to set the `x-delay` header when
using the RabbitMQ Delayed Message Exchange plugin.

Polishing

- PR Comments
- Add `setDelay`
- Port `FunctionExpression` from DSL
- Add `SupplierExpression`

Javadoc Fixes

Use ValueExpression for delay

* Simple polishing for `SupplierExpression`
* Mention plain `delay` property in the `amqp.adoc`
2016-10-07 11:42:01 -04:00
Gary Russell
9e3156ae7d INT-4129: Add Discard Channel to Barrier Handler
JIRA: https://jira.spring.io/browse/INT-4129

Discard late arriving triggers.

* Minor code style polishing
2016-10-06 11:47:22 -04:00
Gary Russell
f9ddefec2c INT-4118: Detect EOF on stdin
JIRA: https://jira.spring.io/browse/INT-4118

Add an option to the `CharacterStreamReadingMessageSource` to detect EOF
on the stream and close the context.

Publish event instead of closing context.

Polishing - PR Comments

Polishing

Docs about stopping the poller.

* Fix typos in the docs
2016-10-05 19:31:29 -04:00
Artem Bilan
129984ec0a INT-4113: Add @Poller.errorChannel() Attribute
JIRA: https://jira.spring.io/browse/INT-4113

* To simplify `MessagePublishingErrorHandler` and avoid extra `PollerMetadata` beans, added the `errorChannel()` attribute to the `@Poller` annotation
* The `MessagePublishingErrorHandler` now supports the late-binding via new `defaultErrorChannelName`
* Docs about new `errorChannel()` attribute
* Some other docs polishing

Polishing

**Cherry-pick to 4.3.x**

Polishing docs according PR comments
2016-09-27 17:57:37 -04:00
Artem Bilan
12fb590b95 Mainly Checkstyle Violation Fixes
* Upgrade to Checkstyle `7.1`
* Relax `RequireThis` rule a bit. Right now it does the effort only in case of overlapping. See https://github.com/checkstyle/checkstyle/issues/2362 for more info
* Enable some annotation rules and provide fixes for violations
* Enable `tabs indents` rule. This was the biggest fix in this PR
* Resolve `TODO` in the `MessagingMethodInvokerHelper` and fix tests to meet `IllegalStateException` now

Remain Checkstyle version `6.16.1` and reinstate `RequireThis` rule

The latest Checkstyle has a bug with local scope variables if they have the same names as property.

Revert some literals splitting

Fix some line length exceeding and code style
2016-09-22 13:23:51 -04:00
Artem Bilan
c0f1c9ece8 INT-4106: Add JPA Java-based configuration docs
JIRA: https://jira.spring.io/browse/INT-4106

Also fix some typos in the `jpa.adoc` and remove mentioning of OpenJPA, since it doesn't support JPA 2.1
2016-09-16 12:25:36 -04:00
Artem Bilan
dbcebebf38 INT-4096: Allow Configuration of ReadOnly Headers
JIRA: https://jira.spring.io/browse/INT-4096

* Introduce `spring.integration.readOnly.headers` Integration property
* Introduce `IntegrationMessageHeaderAccessor.setReadOnlyHeaders()` and use it from the overridden `IntegrationMessageHeaderAccessor.isReadOnly()`
* Add `DefaultMessageBuilderFactory.setReadOnlyHeaders()` and delegate the value to the new `MessageBuilder.readOnlyHeaders()`
* Modify `spring.integration.properties` for the `contentType` as a `readOnly` header for testing
* Ensure that provided logic works via an appropriate modification to the `ObjectToJsonTransformerParserTests`

Increase receive timeouts in the `OutboundGatewayFunctionTests`

Fix `FileInboundTransactionTests` for slow `WatchService` issue on Linux/OS X

Also redo the `tmp` dir logic to the `TemporaryFolder` `@ClassRule`

Fix unused `import` in the `FileInboundTransactionTests`

Polishing

Conflicts:
	spring-integration-core/src/main/java/org/springframework/integration/context/IntegrationProperties.java
	spring-integration-core/src/main/resources/META-INF/spring.integration.default.properties

Some polishing and documentation

Localize `readOnly.headers` customization only in the `ObjectToJsonTransformerParserTests-context.xml`

Otherwise ti might affect many other tests and the search for the reason of failure would be enough complicated

Doc Polish
2016-09-15 15:32:20 -04:00
Artem Bilan
6790dbd03e Fix simple typo in whats-new.adoc
Otherwise `Gateway Changes` section looks like sub-paragraph of the `JMS Changes`
2016-09-08 10:42:29 -04:00
Artem Bilan
b66acf2363 Fix (S)FTP gateway samples in docs for expression
SO: http://stackoverflow.com/questions/39293574/how-to-configure-sftp-outbound-gateway-using-java-config
2016-09-02 14:56:18 -04:00
Artem Bilan
868004e9e4 Some Fixes and Improvements
* Remove `durable-subscription-name` from the `JmsMessageDrivenEndpointParser`, since we don't have such an attribute any more
* Fix `configuration.adoc` for various typos
* Remove `spring.integration.messagingAnnotations.require.componentAnnotation` and `spring.integration.messagingGateway.convertReceiveMessage` properties and their usage,
since they are not actual any more starting with SI-5.0
* Fix tests appropriately

Address PR comments and more polishing

Polishing
2016-08-31 16:22:23 -04:00
Artem Bilan
eaed954458 INT-3913 Remove/resolve deprecation from the past
JIRA: https://jira.spring.io/browse/INT-3913

* Remove deprecated classes and methods/constructors, deprecated XML attributes
* Remove `TcpConnectionEventListeningMessageProducer` and rework tests logic to the `ApplicationEventListeningMessageProducer`
* Fix several typos
* Remove/rework deprecated entities mentioning
2016-08-26 13:42:47 -04:00
Gary Russell
ea4763faa9 INT-4095: Support Limiting (S)FTP Files Fetched
JIRA: https://jira.spring.io/browse/INT-4095

Limit the number of remote files fetched on each poll (when it is necessary to fetch files).

Polishing - PR Comments

Polishing - Decouple MaxFetchSize from Poller

Polishing - PR Comments

Schemas and Docs

More Polishing

* Polishing according PR comments
2016-08-26 13:36:14 -04:00
Gary Russell
dfb6ab1e08 INT-4098: IMAP Content Rendering Consistency
JIRA: https://jira.spring.io/browse/INT-4098

Polishing - What's New

Schema Polishing

Also fix a test to align it with changes to the test mail server.
2016-08-23 12:20:25 -04:00
Kevin Bowersox
0ebb4aa0f5 Fix SPEL syntax for headers in the ftp.adoc 2016-08-23 11:09:52 -04:00
Gary Russell
9f40d896a2 INT-4092: Add errorChannel Header with Void Return
JIRA: https://jira.spring.io/browse/INT-4092

Polishing

Polishing

Polish of Polishing
2016-08-22 18:10:30 -04:00
Artem Bilan
52904c7106 INT-4099: Fix closableResource typo
JIRA: https://jira.spring.io/browse/INT-4099

The `IntegrationMessageHeaderAccessor.CLOSEABLE_RESOURCE` value has a typo

* Fix the typo in the `IntegrationMessageHeaderAccessor.CLOSEABLE_RESOURCE` constant value
* Fix the same type in the (s)ftp.adoc
2016-08-22 15:38:07 -04:00
Gary Russell
77d0becd97 INT-4066: Expose RMI Factory Bean
JIRA: https://jira.spring.io/browse/INT-4066

Allow customization.

Rename To RmiProxyFactoryBeanConfigurer

Polishing

More Polishing

Polishing
2016-08-22 15:37:06 -04:00
Artem Bilan
2a0e56ba6c jms.adoc: Fix typos in extract-payload paragraph
SO: http://stackoverflow.com/questions/39064020/spring-integration-jms
2016-08-22 10:04:42 -04:00
Gary Russell
5f2f7d8b83 INT-3941: Align JMS Config with Spring Boot
JIRA: https://jira.spring.io/browse/INT-3941

Previously, like the SF `<jms:/>` namespace, Spring Integration used a default bean
name `connectionFactory` for the JMS Connection Factory.

Spring Boot auto-configures a bean name `jmsConnectionFactory`.

Since it is still common to use Spring Integration XML configuration with Spring Boot,
it is useful to align the two so that an SI app can use the auto configured bean.

Docs polishing

Doc Polishing
2016-08-19 17:15:57 -04:00
Artem Bilan
c54f547d14 Move Reference Manual to 5.0 2016-08-18 15:57:09 -04:00
Artem Bilan
e7b30eba5c INT-4082: Fix MongoDB MessageStore for auditing
JIRA: https://jira.spring.io/browse/INT-4082

Starting with Spring Data 1.9 the `MappingContextIsNewStrategyFactory` relies on a newly introduced `PersistentEntities` and doesn't register entities lazily any more.
Such a change finishes with the `Unsupported entity` exception when an auditing is switched on (`<mongo:auditing/>`)
for the `MongoDbMessageStore` and `AbstractConfigurableMongoDbMessageStore` internally created `MongoTemplate` and `MessageWrapper` and `MessageDocument` SI internal entities.

* Don't register `ApplicationContext` into internally created `MongoTemplate`s since to avoid entity events emitting for `MessageWrapper` and `MessageDocument`
* Pull `MongoDbMessageBytesConverter` to the top-level class to let customize `MappingMongoConverter` properly if there is need to audit `MessageDocument` anyway,
what can be possible via external injections into `AbstractConfigurableMongoDbMessageStore` implementation
* Fix `mongodb.adoc` for some typos

**Cherry-pick to 4.2.x**
2016-08-17 15:29:51 -04:00
Gary Russell
c0f2d6d738 Overview Doc Polishing 2016-08-16 11:37:10 -04:00
Artem Bilan
3b7d2aaf9b Fix typos in the scatter-gather.adoc 2016-08-10 09:27:27 -04:00
Gary Russell
d0008c368e INT-4085: Access ARPMH from RequestHandlerAdvice
JIRA: https://jira.spring.io/browse/INT-4085

Polishing - PR Comments
2016-08-09 09:54:41 -04:00
Artem Bilan
d00e0cdcc1 Document Java(DSL) for (S)FTP Outbound
SO: http://stackoverflow.com/questions/38654837/retreive-files-from-ftp-using-spring-integration-ftpoutboundgateway
2016-07-29 12:45:25 -04:00
Gary Russell
6eba49c625 INT-4080: Add TcpConnectionFailedEvent
JIRA: https://jira.spring.io/browse/INT-4080
2016-07-26 12:02:14 -04:00
Vedran Pavic
91aaae1f06 Polish JDBC documentation: typos and links 2016-07-25 10:21:33 -04:00
Artem Bilan
d9f9f9e3ca INT-4064: Simplify IdempotentReInt Java Config
JIRA: https://jira.spring.io/browse/INT-4064

We configure `IdempotentReceiver` via `<int:idempotent-receiver>` component or `@IdempotentReceiver` annotation.
In case of regular Java config, e.g. direct `ConsumerEndpointFactoryBean` usage or Java DSL,
it isn't possible to configure `idempotentReceiverInterceptor` enough easy

* Introduce `if...else` logic into `ConsumerEndpointFactoryBean` to proxy `MessageHandler`,
if `adviceChain` contains an newly-introduced `HandleMessageAdvice`.
And do that independently if `MessageHandler` is `AbstractReplyProducingMessageHandler`
* Make `idempotentReceiverInterceptor extends HandleMessageAdvice`
* Skip `HandleMessageAdvice` in the `AbstractReplyProducingMessageHandler`
* Add advice applying logic into the `AbstractMethodAnnotationPostProcessor` as well

Introduce `HandleMessageAdvice` marker interceptor to cover the case when an `Advice` can be advices as well.
Remove unused `setAdviceChainIfPresent()` method in the `AbstractMethodAnnotationPostProcessor`

Document `HandleMessageAdvice`

Increase wait latch timeouts in the `LockRegistryLeaderInitiatorTests`

Doc Polishing
2016-07-22 13:56:41 -04:00
Artem Bilan
f74ddc2aa6 INT-4063: Extend Types for DefXmlPayloadConverter
JIRA: https://jira.spring.io/browse/INT-4063

Since `DefaultXmlPayloadConverter` is used from many out-of-the-box components by default,
make it more flexible with the supported input types which can be converted into `Document`

Improve `xml.adoc` about this types and also fix a lot of typos there as well

Address PR comments
2016-07-22 13:38:10 -04:00
Artem Bilan
35a5a4a4ab Fix typos in the delayer.adoc 2016-07-19 12:29:21 -04:00
Artem Bilan
9462d36de2 Add (S)FTP Inbound Java Config samples into Docs 2016-07-14 14:30:38 -04:00
Gary Russell
d3e480aa55 Minor Doc Fix 2016-07-14 11:50:42 +01:00