Commit Graph

1959 Commits

Author SHA1 Message Date
Artem Bilan
54b0800d08 INT-4280: Add taskScheduler() to Java DSL
JIRA: https://jira.spring.io/browse/INT-4280

To let end-user easily to inject custom `TaskScheduler` to the endpoint
add `ConsumerEndpointSpec.taskScheduler()` option

* Address PR comments
2017-06-28 10:12:38 -04:00
Artem Bilan
d766d233f5 Fix ReactiveStreamsConsumerTests and Checkstyle
Also increase timeouts in the `RedisAvailableRule`
2017-06-21 12:12:49 -04:00
Artem Bilan
a834ce9ffb FluxMessageChannel.send: no subscribers assertion 2017-06-21 11:10:41 -04:00
Artem Bilan
ecb7d03895 Upgrade to the latest SF and fix HTTP Reactive
https://build.spring.io/browse/INT-FATS5IC-206/

Also use `AopUtils.getTargetClass()` directly in the
`ReactiveHttpRequestExecutingMessageHandler` since it never returns `null`
2017-06-21 10:43:36 -04:00
Gary Russell
42dd05d655 INT-4298: Add Interface HeaderPropagationAware
JIRA: https://jira.spring.io/browse/INT-4298

Used to indicate components that can propagate headers; allows suppression.

Move Interface to AbstractMessageProducingHandler

Add Docs

* Fix typo in doc
2017-06-20 10:44:08 -04:00
Artem Bilan
fab0f5478f INT-4284: Remove info about readOnly headers
JIRA: https://jira.spring.io/browse/INT-4284

**Cherry-pick to 4.3.x**
2017-06-15 11:05:45 -04:00
Artem Bilan
4a47a7ce45 INT-4284: Exception to overwrite id or timestamp
JIRA: https://jira.spring.io/browse/INT-4284

To inform end-user that he/she can't override `id` and `timestamp` headers
throw a `BeanInitializationException` from the `gateway`, `header-enricher`,
`enricher` and `header-filter`  configuration when `id` and `timestamp` are
explicitly provided
2017-06-14 13:02:36 -04:00
Artem Bilan
99224f54c0 INT-4290: JacksonJsonUtils: Add Trusted Packages
JIRA: https://jira.spring.io/browse/INT-4290

See CVE-2017-4995

To disallow deserialization of unknown classes,
the `JacksonJsonUtils#messagingAwareMapper()` can now be supplied
with the `trustedPackages`.
The default list is:
```
java.util
java.lang
org.springframework.messaging.support
org.springframework.integration.support
org.springframework.integration.message
org.springframework.integration.store
```
Can be configured with `*` (asterisk) with meaning trust all

**Cherry-pick to 4.3.x**

Polishing according PR comments
2017-06-14 12:35:32 -04:00
Gary Russell
71149f260a Fix s-i-core and SF @Nullable Changes 2017-06-07 15:09:43 -04:00
Marius Bogoevici
ecc3006179 INT-4288: Add blacklist headers at runtime
JIRA: https://jira.spring.io/browse/INT-4288

- Add `AbstractMessageProducingHandler.addNotPropagatedHeaders()`
so that new headers can be added without overriding existing ones.
- Add `AbstractMessageProducingHandler.getNotPropagatedHeaders()`
so that existing excluded headers can be consulted.

* Polishing JavaDocs and imports order

**Cherry-pick to 4.3.x**
2017-06-06 16:43:18 -04:00
Gary Russell
f58106ec3c INT-4033: Support SpEL in Gateway Timeouts
JIRA: https://jira.spring.io/browse/INT-4033

Provide a mechanism to support dynamic timeouts when invoking gateway methods.

Polishing - PR Comments

* Polishing `what's new`
* Add `STOMP` to the `endpoint-summary.adoc`
* Polishing code style a bit for the changes in this fix
2017-06-06 10:56:59 -04:00
Gary Russell
cb444d8f5d INT-4284: Integration Properties @Bean => static
JIRA: https://jira.spring.io/browse/INT-4284

Prevent early instantiation of config.
2017-06-05 14:49:37 -04:00
Artem Bilan
33e9ce912f INT-4284: INFO about overriding readOnly headers
JIRA: https://jira.spring.io/browse/INT-4284

Add `INFO` into the `MessageBuilder#copyHeadersIfAbsent()` when end-user
tries to populate headers which are `readOnly`

We can't throw exception on the matter since can modify `readOnlyHeaders`
and that would force end-user to add `header-filter` logic to the application.

* Document `readOnly` headers in the `message.adoc`

**Cherry-pick 4.3.x**
2017-06-05 12:53:21 -04:00
Gary Russell
2c7bf9271c INT-4283: MarkerFilePresentFileListFilters
JIRA: https://jira.spring.io/browse/INT-4283

Filter that only passes files where a corresponding "marker" file is also present.

Implementations for file and(S)FTP.

* Polishing - PR Comments

* Missed one
2017-06-02 19:31:05 -04:00
Artem Bilan
a7ad22adcf ReactiveStreamsTests: Add Reactor diagnostics
https://build.spring.io/browse/INT-FATS5IC-186
2017-06-01 14:35:59 -04:00
Gary Russell
0d28e8c5e3 Revert "Ignore Test (temporary)"
This reverts commit a800d9683a.
2017-06-01 11:16:32 -04:00
Artem Bilan
f112ecbb7e INT-4269: Add Stream & Flux Support for Splitter
JIRA: https://jira.spring.io/browse/INT-4269

* Check if `outputChannel` is `ReactiveStreamsSubscribableChannel`
to let back-pressure splitting
* Build `Flux` or `Iterator` depending in the `reactive` state
* Allow to get a size of the `iterator` if it is possible,
for example `XPathMessageSplitter`

Add tests

Fix raw type and unused import

* Add JavaDocs to the `AbstractMessageSplitter#obtainSizeIfPossible()`
* Add asserts for the `sequenceSize` populataiton in the `XPathMessageSplitter`
* Document `Stream` & `Flux` support in the splitter

Minor doc polishing
2017-05-31 14:28:28 -04:00
Gary Russell
a800d9683a Ignore Test (temporary) 2017-05-31 12:21:46 -04:00
Gary Russell
492246eb13 Fix NPE (Mock Message with no Headers) 2017-05-31 12:10:27 -04:00
Gary Russell
d90d4e4d8f INT-4283: Restore Registration of Missing Comps
JIRA: https://jira.spring.io/browse/INT-4282

Unrecognized components were not added to `targetIntegrationComponents` causing
problems such as messge-driven components not starting.
2017-05-30 11:10:26 -04:00
Artem Bilan
533fa17f76 Fix Checkstyle violation in the DelayHandlerTests 2017-05-25 14:46:44 -04:00
Gary Russell
be336a0463 Make DelayHandlerTests More Robust
https://build.spring.io/browse/INT-MASTER-683/

Wait for release task to arrive in the scheduler queue.
2017-05-25 14:04:25 -04:00
Artem Bilan
ea89682368 INT-4252 IntegrationFlow: Allow Custom Bean Names
JIRA: https://jira.spring.io/browse/INT-4252

To allow to provide arbitrary bean names for the intermediate components
in the `IntegrationFlow` change `ComponentsRegistration` to return
a `Map<Object, String>` instead of raw `Collection<Object>`

* Use the value from that map in the `IntegrationFlowBeanPostProcessor` as a
fallback option before walking into bean name generation
* Provide `containerId` option for the AMQP DSL components

* Revert `@AfterClass` in the `AmqpTests`
* Expose `id()` for the `JmsDestinationAccessorSpec`
* Register `ListenerContainer` and `JmsTemplate`
as bean via `ComponentRegistration` in the particular JMS Java DSL components
* Fix `Jms` factory to populate the `JmsDefaultListenerContainerSpec`
for the `messageDrivenChannelAdapter()` without class specified

* Refactor AMQP DSL Inbound components to deal with the new ContainerSpec API
* Remove `containerId()` in favor of `id()` in the ContainerSpec
2017-05-23 14:46:06 -04:00
Gary Russell
799dcaae9f INT-4276: Selective Header Propagation
JIRA: https://jira.spring.io/browse/INT-4276

The `readOnlyHeaders` integration property allows suppression of certain headers
globally.

Add support for suppressing propagation on individual message handlers.
2017-05-22 12:56:55 -04:00
Artem Bilan
ea6cf0f4ef INT-4245: Improve ReplyProducingMHWrapper
JIRA: https://jira.spring.io/browse/INT-4245

* Move `ReplyProducingMessageHandlerWrapper` outside of
`ServiceActivatorAnnotationPostProcessor`.
* Use it from the `ServiceActivatorFactoryBean` as well to fix
the missed `Lifecycle` control
* Increase wait timeout in the `ConnectionFactoryTests`
* Mark endpoint as `auto-startup="false"` in the SFTP parser tests
* Remove redundant `org.gradle.daemon=true` from the `gradle.properties`
since it is like that by default for a while already
2017-05-18 08:37:56 -04:00
Artem Bilan
be98a6c2d0 INT-4274: Improve IntegrationFlow Lifecycle
JIRA: https://jira.spring.io/browse/INT-4274

Since any component from the `StandardIntegrationFlow` can be started
independently, the `StandardIntegrationFlow.stop()` should propagate
the stop control to the target components independently of its previous
`running` state

Javadoc Polishing
2017-05-16 15:33:40 -04:00
Artem Bilan
4dba3065ff INT-4275: Add IntegrationFlowContext.getRegistry
JIRA: https://jira.spring.io/browse/INT-4275

To analyze the state of the `IntegrationFlowContext` provide the access to
its internal `registry`
2017-05-16 08:36:08 -04:00
Artem Bilan
6c2200a03f Revert to B-S versions for base line dependencies
Increase receive timeouts in the `ResequencerTests`
2017-05-15 10:31:08 -04:00
Gary Russell
f4a17886be INT-4272: Read Only Header Improvements
JIRA: https://jira.spring.io/browse/INT-4272

- Honor read only header configuration when building from an unchanged message
- Support adding read only headers to the message builder factory

* Move `IntegrationMessageHeaderAccessor.containsReadOnly()` to the
`MessageBuilder` to avoid confusing

**Cherry-pick to 4.3.x**
2017-05-12 10:44:57 -04:00
Artem Bilan
709cb7fe4c Remove Jackson Imports from JacksonJsonUtils
Since the classes load their imports explicitly we can't use them for
classes which provides utilities to check classpath

* Move imports to fully qualified names in the target utility method

**Cherry-pick to 4.3.x**

Move `JacksonJsonUtils.isPresent` methods to separate `JacksonPresent` class

Polishing
2017-05-11 15:39:47 -04:00
Artem Bilan
9543877c0c Fix FluxMessageChannel for Multi Sources
* Rename `ReactiveConsumer` to `ReactiveStreamsConsumer`
* Rename `FluxSubscribableChannel` to `ReactiveStreamsSubscribableChannel`
* Remove the `processor` functionality from the `FluxMessageChannel`
in favor of internal `FluxSink` as it is recommended by the Project Reactor:
> Most of the time, you should try to avoid using a Processor.
They are harder to use correctly and prone to some corner cases.

* Make connectable, upstream publishers for the `FluxMessageChannel` as
bridges to the internal `sink` via `this::send`.
This way we are able to receive data from multi sources.
When the source is completed (e.g. `Mono` in case of WebFlux response),
the downstream flow isn't completed.
* Rework `MessageChannelReactiveUtils#PollableChannelPublisherAdapter`
to be based on the `Flux.create()` and `onRequest()` to poll channel for messages
* Add one more request to the `ReactiveHttpRequestExecutingMessageHandler`
to be sure that we consume different `Mono`s by the `FluxSubscribableChannel`
properly without completion
* Upgrade to the `spring-io-plugin:0.0.7.RELEASE`

Add `IntegrationConsumer` implementation to the `ReactiveStreamsConsumer`
2017-05-10 14:02:35 -04:00
Artem Bilan
7e919a4017 INT-4267: Add JSON (De)Serializers for Messaging
JIRA: https://jira.spring.io/browse/INT-4267
Fixes: spring-projects/spring-integration#2110

The documentation clearly point that we can simply use JSON (de)serialization
with the `RedisMessageStore`, but actually it isn't so easy

* Fix `MessageGroupMetadata`, `MessageHolder`, `MessageMetadata` for Jackson
deserialization compatibility
* Add `MessageHeaders`-based ctor to the `AdviceMessage`
* Add `MessageHeadersJacksonSerializer` to serialize `MessageHeaders`
to the `HashMap` for easier deserialization afterwards
* Add deserializer implementations for all `Message` types
* Add convenient `JsonObjectMapperProvider#jacksonMessageAwareMapper()`
factory method to build `ObjectMapper` supplied with mentioned above
(de)serializers

**Cherry-pick to 4.3.10 without `MessageHolder` and `MessageMetadata`**

Address PR comments and document the feature

Doc Polishing
2017-05-10 10:31:46 -04:00
Artem Bilan
aca97e7025 Fix Race Condition in TxSyncQueueChannelTests
https://build.spring.io/browse/INT-MASTER-653

The test-case uses only 1 second to wait for the message
in the `QueueChannel`.
That isn't enough on slow environment like CI causing failure for
the current test and unexpected value in the queue for the subsequent
tests

* Fix timeouts
* Get rid of `CountDownLatch` - the same is done by the wait on queues
* Purge queue in between tests.
This way we may not have failed subsequent test, thus failure analyze
will be much easier
* Rework `testRollback()` to wait for good result after one retry over
rollback
* Remove duplicate bean definition from XML config

**Cherry-pick to 4.3.x and 4.2.x**
2017-05-08 16:13:07 -04:00
Gary Russell
2b4c343eaa INT-4270: CGLib Compatibility
JIRA: https://jira.spring.io/browse/INT-4270

Initial Commit

Remove `final` modifier from certain critical framework methods that prevent CGLib proxies working.

Since Spring Boot 2.0 now uses proxyTargetClass=true by default (and has done that for
transactional proxies since 1.4), we must relax these restrictions so that CGLib can
proxy channels (which is often done to make subflows run in a transaction).

When CGLib can't override a `final` method, the fields used within those methods are uninitialized.
In the case of `AbstractMessageChannel` this causes an NPE on the unitiallized `dataTypes` field.
With `MessageHandler`, fields like `shouldTrack` are always false.

Further, methods on `IntegrationObjectSupport` - such as `getComponentName()` return null.

I have left setters and `afterPropertiesSet` (on IOS) `final` since these will typically be called
before the object is proxied.

I suspect there will be other methods we need to open up, but perhaps we should only do them on-demand.

__cherry-pick to 4.3.x__
2017-05-05 09:37:18 -04:00
Gary Russell
572467b1c4 INT-4260: MessagePublishingErrorHandler Orig. Msg
JIRA: https://jira.spring.io/browse/INT-4260

MPEH: Populate the `ErrorMessage.originalMessage`,
if available and not the same as the `failedMessage`.

Polishing - rename exception to `MessagingExceptionWrapper`;
 make `MPEH` a subclass of `ErrorMessagePublisher`

PR Comments

Polishing - PR Comments
2017-05-03 16:32:49 -04:00
Artem Bilan
18f9fcbb59 INT-4259: ACorrelationMH: WARN group reschedule
JIRA: https://jira.spring.io/browse/INT-4259

Since DEBUG can switched off for the application, it becomes difficult
to determine the wrong configuration for the discard flow from
the scheduler thread.

* Change logging level to the `WARN` and log the full stack-trace as well
to give more information about the reason of rescheduling group for
`forceComplete`

**Cherry-pick to 4.3.x**
2017-05-03 10:04:52 -04:00
Artem Bilan
f03356ccbc INT-4265: Add Java DSL .routeByError() EIP-Method
JIRA: https://jira.spring.io/browse/INT-4265

* Add `.routeByError()` EIP-method to the `IntegrationFlowDefinition`
based on the `ErrorMessageExceptionTypeRouter`
* Add missed `dynamicChannelLimit()` option to the `RouterSpec`
* Rework `RouterSpec#RouterMappingProvider` into the `ContextRefreshedEvent`
phase initialization to be sure that dependent `MappingMessageRouterManagement`
is fully initialized before applying mapping conversions

* Rename `routeByError()` to `routeByException()` to more reflect reality
of the `ErrorMessageExceptionTypeRouter` and don't mislead about
Java's `Error`
* Polish `ErrorMessageExceptionTypeRouter` JavaDocs a bit and some Java 8
code style
* Add `ApplicationContext` assertion into the
`RouterMappingProvider.onApplicationEvent()` do not trigger delegate
initialization if `ContextRefreshedEvent` is from the different app context
2017-05-02 14:26:04 -04:00
Gary Russell
ca231763a3 Rename ReactiveChannel
- as discussed last week

TODO:
- should we take the channel outside of the `AbstractMessageChannel` hierarchy?
  - avoid blocking interceptors
  - we would lose channel metrics though
- rename `ReactiveConsumer` ?

Polishing some missed renaming
2017-05-01 13:45:35 -04:00
Artem Bilan
b869666021 INT-4261: Do not Register Resource to TX Twice
JIRA: https://jira.spring.io/browse/INT-4261

When we have some `Advice` withing TX Advice which may perform `doPoll()`
several times, we unconditionally call
`transactionSynchronizationFactory.create(resource)`.
With the out-of-the-box implementations
`DefaultTransactionSynchronizationFactory` and
`PassThroughTransactionSynchronizationFactory`
we preform `TransactionSynchronizationManager.bindResource()`.
If resource is already there, an `IllegalStateException` is thrown

* Check that resource isn't bound already to the TX and don't create a new
`TransactionalResourceSynchronization` - just return `null`
* Check in the target users for the `null` before registering synchronization

Move resource registration to TX outside of out-of-the-box factories

* Fix condition in the `AbstractPollingEndpoint` for the resource
* Increase responsiveness of TX test to decreasing `fixed-delay`
and using `receive-timeout="-1"`
2017-04-28 12:56:03 -04:00
Artem Bilan
026484c382 Compatibility with the latest dependencies
* Upgrade to Gradle 3.5, SS-5.0, Hibernate-5.2.10, Mockito-2.7.22, Pah-1.1.1
And some other minor upgrades
* Fix deprecations for Mockito compatibility
* Fix `ServletWebSocketHandlerRegistry` deprecations
* Fix SD-Mongo deprecations
* Tweak JMX tests to avoid dangling threads after tests exist
* Increase timeouts in some polling tests
2017-04-26 11:29:18 -04:00
Artem Bilan
3c047c5076 INT-4264: .publishSubscribeChannel(): fix order
JIRA: https://jira.spring.io/browse/INT-4264

When we use Java DSL `.publishSubscribeChannel()`, we expect that subscribers
will be called in the order they are defined in the flow.
But actually everything is depend on the populated `MessageHandler` implementation.
The `AbstractMessageHandler` implements `Ordered` and according the
`OrderedAwareCopyOnWriteArraySet` logic they are placed in the beginning of the
handlers set, meanwhile all others (un-ordered) are moved to the end.

* Add implicit `.bridge()` to all the subFlows added to the `PublishSubscribeSpec`.
This way all the subFlow subscribers will be ordered the same way (because of
`BridgeHandler`) and, therefore, will be preformed in the expected (as declared) order
2017-04-25 14:28:11 -04:00
Gary Russell
c66540593c INT-4257 Polishing - Subclasses: buildErrorMessage
JIRA: https://jira.spring.io/browse/INT-4257

So that subclasses can also generate error message for conditions such
as message conversion errors.

Polishing - PR Comments

Fix JMS buildErrorMessage Override

Update Copyrights
2017-04-24 13:35:13 -04:00
Artem Bilan
71b93987d7 INT-4257-2: Document ErrorMessagePublisher
JIRA: https://jira.spring.io/browse/INT-4257

* Remove deprecated `EnhancedErrorMessage`
* Document `ErrorMessagePublisher` and `ErrorMessageStrategy`
* Fix Reactor version to `3.1` in Doc
* Fix sample about Reactive Streams in the `router.adoc`
2017-04-20 11:52:40 -04:00
Artem Bilan
28058884f4 INT-4257: Introduce ErrorMessagePublisher
JIRA: https://jira.spring.io/browse/INT-4257

To make a target `ErrorMessage` customizable introduce
`ErrorMessagePublishingRecoveryCallback` and
`ErrorMessageStrategy` to inject

**Cherry-pick to 4.3.x**

Fix `ErrorMessagePublishingRecoveryCallback` generic type for
`RequestHandlerRetryAdvice` compatibility

Polishing

- Support publishing when no retry context is available
- Use a constant for the message context key

Add EnhancedErrorMessage

- contains input message at time of error message generation
- make the default RecovererErrorMessageStrategy public and extensible

Fix `RetryAdviceParserTests` for new `RecoveryCallback` architecture

Rename to `ErrorMessagePublisher`
Decouple ErrorMessagePublisher from Retry

Refactoring for better purpose reflection and JavaDocs

Use `ErrorMessageStrategy` in message producers.

Add JavaDocs to the `DefaultErrorMessageStrategy`

Deprecate EnhancedErrorMessage

Minor Polish for Subclass Use

- make setters final
- allow subclasses to supply the AttributeAccessor

Fix Tests
2017-04-19 17:54:32 -04:00
Artem Bilan
34db64c832 INT-4254: LockRegLInit: Fix unconditional unlock
JIRA: https://jira.spring.io/browse/INT-4254

The `LockRegistryLeaderInitiator#LeaderSelector` unconditional calls
`unlock()` in the `finally` block when the `Lock` might not be locked.

Another problem that `running = true` is set after submitting `LeaderSelector` task.
That might bring the problem that `LeaderSelector` won't be selected because of
`this.running` race condition

* Move `this.running = true` before submitting `LeaderSelector` task
* Move `unlock()` into the `if (this.locked)` condition

**Cherry-pick to 4.3.x**
2017-04-18 15:22:40 -04:00
Gary Russell
c5e66ec65c Fix Promise->Mono Test Name and Beans 2017-04-16 13:33:31 -04:00
Gary Russell
effca79f3f Upgrade to Reactor 3.1
Polishing

- remove catch from test
- update netty for stomp

Add reactiveErrorHandler to AbstractMessageHandler
2017-04-14 22:14:21 -04:00
Gary Russell
f751398cfe ARFOGateway: Directory LiteralExpression->Value...
Using a `LiteralExpression` in the `AbstractRemoteFileOutboundGateway` (when the
user provides a `File` for the directory) causes us to create a new `File` object
each time, instead of using the user-supplied `File`.

Use a `ValueExpression` instead.

Move the `FileWritingMessageHandler` logic to `ExpressionUtils` and use it from both places.

Also add a String setter variant for convenient Java Configuration.
2017-04-06 11:42:22 -04:00
Artem Bilan
d5c633d703 Fix LockRegLeaderInitiatorTests race condition
https://build.spring.io/browse/INT-FATS5IC-129/

Since leader selection is done on background thread in the
`LockRegistryLeaderInitiatorTests`, there is no guarantee that
after `Thread.sleep(100)` the lock will be obtained and leader state is
changed.

* Waiting for the `granted` `CountDownLatch` is much robust condition
2017-04-06 10:47:22 -04:00
Artem Bilan
19c5402079 INT-4208: Add WebFlux-Based HttpMessageHandler
JIRA: https://jira.spring.io/browse/INT-4208

Since `AsyncRestTemplate` is deprecated in Spring 5.0, it doesn't make
sense to promote that feature via our new `AsyncHttpRequestExecutingMessageHandler`
component

* Rework (and rename) `AsyncHttpRequestExecutingMessageHandler` to
`ReactiveHttpRequestExecutingMessageHandler` and make it based on the
WebFlux `WebClient`
* Fix Java DSL (`ReactiveHttpMessageHandlerSpec`) and all tests according a new
logic in the `ReactiveHttpRequestExecutingMessageHandler`
* Fix XML namespace support to use new `ReactiveHttpRequestExecutingMessageHandler`
and don't expose unused options like `converters` and `request-factory`
* Fix `What's New` and `http.adoc`
* To remain with `async` mode for the `ReactiveHttpRequestExecutingMessageHandler`
behavior support fix `AbstractMessageProducingHandler` to adapt reply `Mono`
to the `SettableListenableFuture`
* Introduce new `reactive` behavior for the `AbstractMessageProducingHandler`
when `outputChannel` is `ReactiveSubscribableChannel` and perform
`.subscribeTo(Publisher)` for `Publisher` reply
* Upgrade to Spring AMQP `2.0 M3`
* Downgrade to Spring Security `4.2.2`

Address PR comments

Add async error handling to the one-way case

Minor polishing and checkstyle fix
2017-04-05 13:00:50 -04:00