Commit Graph

1038 Commits

Author SHA1 Message Date
Artem Bilan
8805dd6533 GH-2243: Fix producer endpoint image
Fixes: spring-projects/spring-integration#2243

The Message is indeed a result of `MessageChannel` consumption, but
definitely not a part of interaction with the tarter system.
For example `FileWritingMessageHandler` obviously writes to file,
but don't produce any messages
2017-11-28 14:14:34 -05:00
Artem Bilan
90c46f5a79 INT-4365: Improve notPropagatedHeaders function
JIRA: https://jira.spring.io/browse/INT-4365

It is much useful to configure the `notPropagatedHeaders` as a set of
patterns to match.
In this case we can filter a group of headers with the common prefix or
suffix

* Allow to configure `AbstractMessageProducingHandler.setNotPropagatedHeaders`
as simple patterns; the `*` means filter all - not copy request headers
at all - similar to `transformer` behavior
* Add `ConsumerEndpointSpec.notPropagatedHeaders()` for Java DSL
* Add `not-propagated-headers` to the `<service-activator>`

Address PR comments; some other improvements

* Fix `ConsumerEndpointSpec#notPropagatedHeaders()` log message
* Improve `AbstractMessageProducingHandler.notPropagatedHeaders() logic
so any `*` in the set of patterns eliminates all others since it has
a highest priority
* Expose `requires-reply` for the `<transformer>` as a `true` by default
* Refactor a bit `AbstractStandardMessageHandlerFactoryBean` hierarchy to
avoid duplicated code

* Fix `message.adoc`
* Add `noHeadersPropagation` flag to the `AbstractMessageProducingHandler`
* Rework logic in the `updateNotPropagatedHeaders()` to store the array
of patterns instead of `Set` to avoid extra operation on each message
* Combine `noHeadersPropagation` with the `shouldCopyRequestHeaders()`
in the `createOutputMessage()` for logic to determine if we should start
the copy-headers procedure at all

* Revert `AbstractMessageProducingHandler.selectiveHeaderPropagation`
* Optimize `AmqpOutboundGatewayParserTests` performance from 3 secs to 0.5

Fix more NPEs in the `AbstractMessageProducingHandler`
2017-11-27 16:56:41 -05:00
Artem Bilan
7d7fad31c9 INT-4169: Document dynamic HTTP URI variables
JIRA: https://jira.spring.io/browse/INT-4169
2017-11-27 16:11:03 -05:00
Meherzad Lahewala
96aa7280a0 INT-3876: Track groups in correlation endpoints
JIRA: https://jira.spring.io/browse/INT-3876

INT-3876 track groups ids in AbstractCorrelatingMessageHandler and docs

INT-3876 Fix merge conflict and add missing link in doc

INT-3876 Move groupId check in message group processor and rework on docs

INT-3876 Remove extra space from the doc

INT-3876 Fix checkstyle refer instance variable groupid correctly

INT-3876 Update aggregator doc with more details

* Polishing `aggregator.adoc`
* Rework `AbstractCorrelatingMessageHandlerTests.testDontReapMessageOfOtherHandler`
do not use redundant options
2017-11-20 14:51:50 -05:00
Artem Bilan
1244c6c22a The log() in the end IMPORTANT note in dsl.adoc 2017-11-15 15:21:14 -05:00
Artem Bilan
7ca20e53f7 GH-2268: Add RedisHeaders.MESSAGE_SOURCE header
Resolves: /spring-projects/spring-integration#2268

* To indicate the source the Redis message in the `RedisInboundChannelAdapter`
populate the `RedisHeaders.MESSAGE_SOURCE` header to the messages to produce
* Fix the `SimpleMessageConverter` to populate the provided `MessageHeaders`
to the message to produce

* Add `toMessage(T object, @Nullable Map<String, Object> headers)`
to the `InboundMessageMapper` to propagate additional header
to the message to create
* Rework all the out-of-the-box `InboundMessageMapper` implementations
to properly propagate additional headers via `toMessage()`
from the `SimpleMessageConverter`
* Provide optimizations in the `InboundMessageMapper` implementations
do not re-create messages
* Refactor `MutableMessage.toString()` to align with the `GenericMessage`

* Add more `@Nullable` to method arguments
* Increase latch wait timeout in the `EndpointParserTests`
* Address `redis.adoc` PR comment
2017-11-15 11:06:18 -05:00
Artem Bilan
f7e75223c7 INT-4361: Add a role() hook to Java DSL
JIRA: https://jira.spring.io/browse/INT-4361

* To get a gain of the method chain code flow and avoid extra annotation
parsing, add `.role(String)` hook to the `EndpointSpec`
* Delegate the provided `role` property to the `AbstractEndpoint`
* Register `AbstractEndpoint` as itself `SmartLifecycle` in the
`SmartLifecycleRoleController`
* Add `destroy()` to the `AbstractEndpoint` and remove it from the
`SmartLifecycleRoleController`
* Provide some Java 8 code style refactoring

* Rework XML parsers and Annotation processors to populate `role`
property on the `AbstractEndpoint`
* Wrap `roleController` bean extraction to the `NoSuchBeanDefinitionException` catch
* Fix several `AbstractEndpoint` implementation to properly call
`super.onInit()` which has been missed before
2017-11-10 13:58:28 -05:00
Artem Bilan
a61327766e GH-2207: Clarify RedisMetadataStore in cluster
Resolves: spring-projects/spring-integration#2207

* Since the `WATCH` command isn't supported on clustered connections,
the `RedisMetadataStore.replace()` operation can't be used there.
Document this limitation
2017-11-10 11:37:47 -05:00
Artem Bilan
6891d5a2e2 INT-4277: Add Java DSL chapter to the Docs
JIRA: https://jira.spring.io/browse/INT-4277

*Mostly migration from the https://github.com/spring-projects/spring-integration-java-dsl/wiki/Spring-Integration-Java-DSL-Reference
* Add cross links from some appropriate chapters
* Fix log message in the `WebSocketInboundChannelAdapter`

* Document `IntegrationFlowContext`
* Document gateways via `IntegrationFlow`
2017-11-10 11:31:48 -05:00
Artem Bilan
098e8bc58f INT-4291: Post-merge polishing
JIRA: https://jira.spring.io/browse/INT-4291

* Fix code style in the `ExpressionEvaluatingSqlParameterSourceFactory`
* Remove redundant
`ExpressionEvaluatingSqlParameterSourceFactory.registerSqlTypes` in
favor of the direct access to the `sqlParametersTypes` from the
`ExpressionEvaluatingSqlParameterSource` ctor
* Reformat code style in the `pollingWithSelectParameterSourceJdbcInboundChannelAdapterTest.xml`
* Fix code style in the `ExpressionEvaluatingSqlParameterSourceFactoryTests`
* Polishing `whats-new.adoc` for proper formatting and single link from
the section to the particular chapter
* Fix formatting and wording in the `jdbc.adoc`
2017-11-06 11:33:05 -05:00
Meherzad Lahewala
854dba1366 INT-4291 Add ExprEvalSqlPSF.setSqlParameterTypes
JIRA: https://jira.spring.io/browse/INT-4291

* Fix checkstyle trailing space
*Use correct way to constants in spring xml and fix test case

* Polishing - Update javadoc and rename method name to relate more to the behavior
2017-11-06 11:19:01 -05:00
Artem Bilan
8766262399 INT-4360: Add ClientStompEncoder support
The `StompSubProtocolHandler` explicitly sets `stompCommand` header
to the `MESSAGE` value ignoring any client inputs.
In this case the message is treated as from the server and ignored on
the STOMP Broker side from the client session.

* Introduce `ClientStompEncoder` for the client side to be injected
into the `StompSubProtocolHandler` for the proper client side messages
encoding/decoding.
Override `stompCommand` header to the `SEND` value if it is `MESSAGE`
before encoding to the `byte[]` to send to the session

JIRA: https://jira.spring.io/browse/INT-4360

**Cherry-pick to 4.3.x**

Fix WebSocket test to rely on the proper client config class
and don't pick up the server config unconditionally in the test context
2017-10-31 09:25:52 -04:00
Gary Russell
06aa45e91f INT-4279: Document UDP Java Configuration
JIRA: https://jira.spring.io/browse/INT-4279

Add documentation for Java configuration of UDP channel adapters.
Enhance DSL to support host/port for outbound adapters.
2017-10-30 13:49:44 -04:00
Artem Bilan
d7d56a0444 INT-4081: Post-merge polishing and improvements
JIRA: https://jira.spring.io/browse/INT-4081

* Rename `MessageGroupPreparedStatementSetter` to `ChannelMessageStorePreparedStatementSetter`
* Add JavaDocs to the `ChannelMessageStorePreparedStatementSetter`
* Polishing `JdbcChannelMessageStore` JavaDocs; remove unused `BeanFactory`
* Polishing Docs for typos and proper text flow
2017-10-23 11:48:35 -04:00
Meherzad Lahewala
34d0859515 INT-4081: Custom PreparedStSetter for JdbcMChStore
JIRA: https://jira.spring.io/browse/INT-4081

* Add `MessageGroupPreparedStatementSetter` to let end-user to override
the message insertion logic
* Add tests and Docs
2017-10-23 11:48:34 -04:00
Artem Bilan
077b5ffffc INT-4344: AbstRemoteFileOutGw: Add assert for NPE
JIRA: https://jira.spring.io/browse/INT-4344

The `FtpOutboundGateway` provides ctors without expression for remote path.
In this case it is treated as a `working directory` but only for the
`LS`, `NLST`, `PUT` and `MPUT` commands.

* Add assertion in the `AbstractRemoteFileOutboundGateway` to discard
configuration for all other commands when `expression` is `null`

**Cherry-pick to 4.3.x**
2017-10-16 17:04:22 -04:00
Artem Bilan
bf63ef5491 GH-2253: Revert (S)FTP Source to DefaultDirScann
Fixes spring-projects/spring-integration#2253

For backward compatibility and use-cases when processed files are stored
in sub-directories in the `localDirectory` revert the
`AbstractInboundFileSynchronizingMessageSource` logic back to the
`DefaultDirectoryScanner`.
The `RecursiveDirectoryScanner` can be injected via `scanner` option
2017-10-16 16:05:44 -04:00
Artem Bilan
1fa73a9728 INT-4356: FileTransferMH: Add string SpEL setters
JIRA: https://jira.spring.io/browse/INT-4356

**Cherry-pick to 4.3.x**

Address PR comments:
* Remove unused variable in the test case
* Add string-based SpEL setters to the `AbstractInboundFileSynchronizer`
* Fix `@since` in the `FileTransferringMessageHandler` to the proper version
2017-10-16 15:51:12 -04:00
Glenn Renfro
1979f91cf5 INT-4357: LeaderInit: Add failed to lock event
JIRA: https://jira.spring.io/browse/INT-4357

Updated based on code review.

Updated adocs

*Polishing code style, typos
* Add protection from NPE when `LockRegistryLeaderInitiator` isn't
supplied with the `leaderEventPublisher`
* Add `What's New` note
2017-10-13 13:15:12 -04:00
Jorge
a7eebed252 Typo at FTPs example
Use DefaultFtpsSessionFactory instead of DefaultFtpsClientFactory
2017-10-12 15:14:42 -04:00
Glenn Renfro
e515a0d643 INT-4353: Let to set id for DefaultLockRepository
JIRA: https://jira.spring.io/browse/INT-4353

Updated based on code review

* Polishing according PR comments

**Cherry-pick to 4.3.x**
2017-09-29 09:57:59 -04:00
Venil Noronha
797e5045bf INT-4334: Adds a DirectoryScanner to RemoteFileMS
JIRA: https://jira.springsource.org/browse/INT-4334

* Add a `DirectoryScanner` setter named `setScanner` in
`AbstractInboundFileSynchronizingMessageSource` to set the
`FileReadingMessageSource`'s scanner.
* Add tests to `AbstractRemoteFileSynchronizerTests`.

Polish.

Updates xsd, tests and documentation.

Fix xsd formatting.

Addresses review comments.

* Some code style polishing
2017-09-25 21:08:51 -04:00
Venil Noronha
a65de25572 INT-4343: Add executor option to the RedisInChA
JIRA: https://jira.springsource.org/browse/INT-4343

* Add a `Executor` setter named `setTaskExecutor` in
`RedisInboundChannelAdapter` to set the container's task executor.

* Add a `task-executor` attribute to
`<int-redis:inbound-channel-adapter>`.

* Update RedisInboundChannelAdapterParser configuration validation
tests.

* Update redis.adoc and whats-new.adoc.

* Polishing according PR comments
* Improve `RedisQueueGatewayIntegrationTests` performance
2017-09-19 18:18:30 -04:00
Venil Noronha
b1865dab3a INT-4313: GemfireMetadataSt: Add Listener support
JIRA: https://jira.springsource.org/browse/INT-4313

* Add `GemfireCacheListener` in `GemfireMetadataStore` to delegate cache
events to `MetadataStoreListener`s.
* Add GemfireMetadataStore cache listener tests.
* Update ascii doc.
* Addresses review comments.
* Some polishing to Java 8 style
* Some code style polishing
* Minor Doc improvement
2017-09-18 20:58:58 -04:00
Meherzad Lahewala
40783ff547 INT-3936: GlobalChIntercep: add negative pattern
JIRA: https://jira.spring.io/browse/INT-3936

* Extract `smartMatch()` logic from the `IntegrationManagementConfigurer`
and `IntegrationMBeanExporter` into the `PatternMatchUtils` class
* Add negative (`!`) pattern matching configuration support to the
`GlobalChannelInterceptor` annotation and `<int:channel-interceptor>` component
* Code style, Docs and JavaDocs polishing
2017-09-18 13:55:00 -04:00
Artem Bilan
5263ea6dff INT-4123: Add Prefix to the Key-Value MSs
Fixes spring-projects/spring-integration#2213
JIRA: https://jira.spring.io/browse/INT-4123

Fully different `MessageStore`s can be configured for the same shared
Key-Value data-base.
Since the retrieval logic is based on the keys, that may cause the
unexpected messages expiration via `MessageGroupStoreReaper`.

* To distinguish store instances on the shared store add `prefix`
option to the `AbstractKeyValueMessageStore`

* Deprecate the `GemfireMessageStore` `Cache`-based configuration - `setIgnoreJta()` and `afterPropertiesSet()`.
The `GemfireMessageStore` relies only on an externally configured `Region`.

**Cherry-pick to 4.3.x**

Doc Polishing
2017-09-11 14:24:33 -04:00
Philipp Etschel
1f220e0e66 INT-4340: Disable XMPP Roster properly
JIRA: https://jira.spring.io/browse/INT-4340

* Roster is now disabled on login as well
* Fixed tagging and formatting issues
* Checkstyle and documented the changed behaviour as well

Polishing: rely on the `AbstractFactoryBean.getObject()`

**Cherry-pick to 4.3.x**
2017-09-11 11:02:13 -04:00
Gary Russell
0d495294ed INT-4342: White List for Payload Deserializer
JIRA: https://jira.spring.io/browse/INT-4342

Use similar code to Spring AMQP to add white list support for Integration's
use of the `DeserializingMessageConverter`; introduce the `WhiteListDeserializingMessageConverter`.

Polishing

Missed this change in PR.

Fix XSD attribute
2017-09-08 15:31:47 -04:00
Gary Russell
cfab1fb1a4 INT-4333: Don't Override Explicit Management Props
JIRA: https://jira.spring.io/browse/INT-4333

If the user has explicitly set management properties in a bean definition,
the configurer should not override those settings with defaults.
2017-09-01 15:48:11 -04:00
Artem Bilan
6c18cbcf57 INT-4326: Add endpoints.noAutoStartup property
JIRA: https://jira.spring.io/browse/INT-4326

To let to disable autoStartup for particular endpoint bean globally,
in one place, add `spring.integration.endpoints.noAutoStartup`
to the `spring.integration.properties`

So, now all the `AbstractEndpoint` checks that property for matching its
bean name to configure `autoStartup` property to `false`

**Cherry-pick to 4.3.x**
2017-09-01 15:14:47 -04:00
Oleg Zhurakousky
946cc229eb INT-4330: Add support for Supplier MessageSource
JIRA: https://jira.spring.io/browse/INT-4330

- updated documentation
- updated test
2017-08-17 12:01:03 -04:00
Artem Bilan
ab9e843cfa Fix Checkstyle violations
* Add `validateOnlyOverlapping = false` for `RequireThis` Checkstyle rule
* Add `this.` to caught places
* Use `CustomizableThreadFactory` instead of raw interface in the
`UnicastSendingMessageHandler`
2017-08-15 13:36:19 -04:00
Gary Russell
a1abae03a4 INT-4329: Fix JDBC [IMPORTANT] Block
JIRA: https://jira.spring.io/browse/INT-4329

Block wasn't closed.
2017-08-15 13:07:10 -04:00
Gary Russell
0a1306cd1b INT-4328: AMQP: Returns/Nacks: Create ErrorMessage
JIRA: https://jira.spring.io/browse/INT-4328

Add support for sending `ErrorMessage`s to the return and nack channels.

**cherry-pick to 4.3.x, but change default EMS to null (will require minor adjustment to test - set the EMS in `adapterWithReturnsAndErrorMessageStrategy`)**
2017-08-15 12:31:52 -04:00
Artem Bilan
84d60f4ab4 INT-4315: Add WebFlux module
JIRA: https://jira.spring.io/browse/INT-4315

Move Reactive components outside of HTTP module to the new WebFlux one,
including XSD, tests and documentation
Make an appropriate polishing for the `http.adoc` with cross-link
to the `webflux.adoc`

Exclude transitive `spring-webmvc` for the `spring-integration-webflux`
2017-08-11 13:38:37 -04:00
Artem Bilan
4fd32d2bd4 Document max-rows-per-poll for the JDBC Gateway
StackOverflow https://stackoverflow.com/questions/45611037
2017-08-10 16:52:44 -04:00
Artem Bilan
74b59ef8d3 INT-4316 Add ObjectToMapTransformer.mapper option
JIRA: https://jira.spring.io/browse/INT-4316

To allow to customize a JSON representation of the Object, provide a
`JsonObjectMapper`-based ctor for the `ObjectToMapTransformer`
2017-07-25 11:58:41 -04:00
Artem Bilan
784d891fa5 Fix Checkstyle violations 2017-07-24 14:05:53 -04:00
Artem Bilan
d4a99919ed INT-4300: Add WebFlux Server Support
JIRA: https://jira.spring.io/browse/INT-4300

* Add `ReactiveHttpInboundEndpoint` based on the WebFlux foundation
* Extract `BaseHttpInboundEndpoint` for common
HTTP Inbound Channel Adapters options
* Make `spring-webmvc` and `spring-webflux` as `optional` dependencies
to let end-user to choose
* Refactor `HttpContextUtils` to include constants
for newly added WebFlux support
* Introduce `BaseHttpInboundEndpoint.setRequestPayloadTypeClass()`
for raw `Class<?>` and modify existing `setRequestPayloadType()`
for the `ResolvableType`
* Refactor existing MVC tests and XML components parsers to use
new `setRequestPayloadTypeClass()`
* Add `MessagingGatewaySupport.sendAndReceiveMessageReactive()` to get
a reply from downstream flow reactive back-pressure manner
* Add `IntegrationHandlerResultHandler` implementation to let WebFlux
infrastructure to handle the `Mono<Void>` from the `ReactiveHttpInboundEndpoint`
properly
* Fix `JdbcLockRegistryLeaderInitiatorTests` race condition to assert
the `initiator1` is elected eventually after yielding when the `initiator2`
is stopped

* Fix JavaDocs issue in the `HttpRequestHandlingMessagingGateway`
* Move all the "hard" logic in the `MessagingGatewaySupport#doSendAndReceiveMessageReactive`
to the `Mono` chain ensuring back-pressure when `sendAndReceiveMessageReactive()` is
called not from the Reactive Stream

Add test-case to demonstrate SSE

JIRA: https://jira.spring.io/browse/INT-3625

Some polishing and optimization for the
`MessagingGatewaySupport.doSendAndReceiveMessageReactive()`

More optimization for `MessagingGatewaySupport`

* Upgrade to Reactor 3.1 M3
* Document WebFlux-based components

Minor Doc Polishing
2017-07-20 12:19:17 -04:00
Artem Bilan
e475d9c695 INT-4308: Document JdbcMetadataStore
JIRA: https://jira.spring.io/browse/INT-4308

* Document `JdbcMetadataStore`
* Remove unnecessary context DERBY config for the `PersistentAcceptOnceFileListFilterExternalStoreTests`
* Rework the `testFileSystemWithJdbcMetadataStore()` to use `EmbeddedDatabase` directly, without context
2017-07-20 10:55:03 -04:00
Artem Bilan
d79e70c6df GH-2197: Rely on the Document creation for Source
Resolves https://github.com/spring-projects/spring-integration/issues/2197

Sometime we would like to avoid some XML definition features (e.g. loading of the DTD).
The `XmlValidatingMessageSelector` doesn't allow to do that easily because it doesn't
rely on the `Document` creation via `DocumentBuilderFactory`, but directly
manipulates with `Source`

* Change `DefaultXmlPayloadConverter.convertToSource()` to fallback to the `convertToDocument()`
for non-Document payloads.
This way we can configure `DefaultXmlPayloadConverter` with custom `DocumentBuilderFactory` and
bypass DTD loading, for example
* Expose `xml-converter` option for the `<int-xml:validating-filter>`  component
2017-07-06 12:04:16 -04:00
Gary Russell
36ac9f2203 (S)FTP MaxFetchSize Doc Polishing and DSL
Also polish programming tips and tricks docs.

* Fix typos in docs
2017-06-27 15:45:40 -04:00
Gary Russell
71d9f173e8 Doc Polishing 2017-06-27 13:18:38 -04:00
Artem Bilan
d8382e0bd0 INT-4297: Add FileSplitter.setFirstLineAsHeader()
JIRA: https://jira.spring.io/browse/INT-4297

* Add `FileSplitter.setFirstLineAsHeader()` and extract the first line from
the context before `iterator`.
* Populate the extracted header to each subsequent line

* Don't use `Streams` to read first line
* Add XSD and Java DSL option for the `firstLineAsHeader`
* Document the feature
* Add Java DSL sample for the `Files.slitter()`

Add a note about more complex header enrichment

Doc Polishing and fix field name.
2017-06-27 13:14:10 -04:00
Gary Russell
5216dc2b02 INT-4304 (S)FTP Outbound Gateway Delete After GET
JIRA: https://jira.spring.io/browse/INT-4304

Add an option to delete remote file(s) after GET/MGET.

Polishing - log error on delete failure.
2017-06-26 14:30:08 -04:00
Gary Russell
0903df9676 INT-4303: (S)FTP MGET with FileExistsMode.IGNORE
JIRA: https://jira.spring.io/browse/INT-4303

Don't emit ignored files in the output message payload.

Also, always preserve the timestamp when using `FileExistsMode.REPLACE_IF_MODIFIED`.

Some code style polishing
2017-06-23 14:02:03 -04:00
Gary Russell
6b6782e16b (S)FTP Doc Polishing 2017-06-22 20:11:49 -04:00
Gary Russell
42dd05d655 INT-4298: Add Interface HeaderPropagationAware
JIRA: https://jira.spring.io/browse/INT-4298

Used to indicate components that can propagate headers; allows suppression.

Move Interface to AbstractMessageProducingHandler

Add Docs

* Fix typo in doc
2017-06-20 10:44:08 -04:00
Gary Russell
4ca91d912c INT-4295: XML Schema (Legacy) Documentation
JIRA: https://jira.spring.io/browse/INT-4295

Also fixed link to github in the online core schema.
2017-06-19 11:30:51 -04:00
Artem Bilan
fab0f5478f INT-4284: Remove info about readOnly headers
JIRA: https://jira.spring.io/browse/INT-4284

**Cherry-pick to 4.3.x**
2017-06-15 11:05:45 -04:00