Commit Graph

2048 Commits

Author SHA1 Message Date
Artem Bilan
a3160f80f6 INT-4318: Register Jackson modules if available
JIRA: https://jira.spring.io/browse/INT-4318

Do not `ObjectMApper.findAndRegisterModules()` unconditionally.
If there is no the target module library in classpath, we end up
with the `NoClassDefFoundError`

* Copy-paste the well-known modules registration logic from the
`org.springframework.http.converter.json.Jackson2ObjectMapperBuilder`
since we can't add `spring-web`  dependency to the `SI-core`
2017-08-08 13:49:50 -04:00
Artem Bilan
5a259939a6 Reset AbstractMessageChannel.fullChannelName
Fixes spring-cloud/spring-cloud-stream#983

When the `AbstractMessageChannel.getFullChannelName()` is called
before the full bean initialization, we don't have the proper name any
more lately - that is only `unknown.channel.name`

* Reset `fullChannelName` property to `null` in the end of `onInit()`,
so the next `getFullChannelName()` will build the proper component name

**Cherry-pick to 4.3.x**
2017-08-04 13:19:46 -04:00
Artem Bilan
74b59ef8d3 INT-4316 Add ObjectToMapTransformer.mapper option
JIRA: https://jira.spring.io/browse/INT-4316

To allow to customize a JSON representation of the Object, provide a
`JsonObjectMapper`-based ctor for the `ObjectToMapTransformer`
2017-07-25 11:58:41 -04:00
Artem Bilan
784d891fa5 Fix Checkstyle violations 2017-07-24 14:05:53 -04:00
Artem Bilan
c26cac4c17 Upgrade to the released Milestones
* SF-5.0 RC3
* AMQP2.0 M5
* Security 5.0 M3
* Spring Twitter 2.0 M4
* Gradle 4.0.1
* Checkstyle 8.0
* JsonPath 2.3
* Fix deprecations and Checkstyle violations
2017-07-24 13:42:12 -04:00
Artem Bilan
5a66854c67 IntegrationFlow: optional @MessagingGateway
When we use `IntegrationFlows.from(Class<?>)`, we don't need to require
present for `@MessagingGateway`, we can just synthesize it with the
default properties

This way we can simply make gateway proxies for the `java.util.function`
interfaces which should be useful for Spring Cloud Function
2017-07-20 17:54:12 -04:00
Artem Bilan
d4a99919ed INT-4300: Add WebFlux Server Support
JIRA: https://jira.spring.io/browse/INT-4300

* Add `ReactiveHttpInboundEndpoint` based on the WebFlux foundation
* Extract `BaseHttpInboundEndpoint` for common
HTTP Inbound Channel Adapters options
* Make `spring-webmvc` and `spring-webflux` as `optional` dependencies
to let end-user to choose
* Refactor `HttpContextUtils` to include constants
for newly added WebFlux support
* Introduce `BaseHttpInboundEndpoint.setRequestPayloadTypeClass()`
for raw `Class<?>` and modify existing `setRequestPayloadType()`
for the `ResolvableType`
* Refactor existing MVC tests and XML components parsers to use
new `setRequestPayloadTypeClass()`
* Add `MessagingGatewaySupport.sendAndReceiveMessageReactive()` to get
a reply from downstream flow reactive back-pressure manner
* Add `IntegrationHandlerResultHandler` implementation to let WebFlux
infrastructure to handle the `Mono<Void>` from the `ReactiveHttpInboundEndpoint`
properly
* Fix `JdbcLockRegistryLeaderInitiatorTests` race condition to assert
the `initiator1` is elected eventually after yielding when the `initiator2`
is stopped

* Fix JavaDocs issue in the `HttpRequestHandlingMessagingGateway`
* Move all the "hard" logic in the `MessagingGatewaySupport#doSendAndReceiveMessageReactive`
to the `Mono` chain ensuring back-pressure when `sendAndReceiveMessageReactive()` is
called not from the Reactive Stream

Add test-case to demonstrate SSE

JIRA: https://jira.spring.io/browse/INT-3625

Some polishing and optimization for the
`MessagingGatewaySupport.doSendAndReceiveMessageReactive()`

More optimization for `MessagingGatewaySupport`

* Upgrade to Reactor 3.1 M3
* Document WebFlux-based components

Minor Doc Polishing
2017-07-20 12:19:17 -04:00
Artem Bilan
7a67c1f1a3 Various polishing and optimizations
* Check for `null` package in the `JacksonJsonUtils`.
Also don't call `this.delegate.typeFromId()` the second time
* Increase wait timeout in the `QueueChannelTests`
* Check `ZookeeperMetadataStore.addListener()` for `null`

**Cherry-pick to 4.3.x**
2017-07-14 16:31:46 -04:00
Artem Bilan
012bebd7bf Impl CoreSubscriber instead of Subscriber
According Reactor recommendation it would be better to `CoreSubscriber`
from Reactor instead of Reactive Streams `Subscriber` for chasing
some optimisation in case of Reactor flows

Increase `receive()` timeout in the `AttributePollingChannelAdapterParserTests`
2017-07-13 17:20:16 -04:00
Artem Bilan
6829da995f INT-4301: Make CircuitBreakerOpenException as ME
JIRA: https://jira.spring.io/browse/INT-4301

To keep the context of the `failedMessage` and avoid extra wrapping
to the `MessagingException` make
`CircuitBreakerOpenException extends MessagingException` directly
2017-07-12 18:46:55 -04:00
Artem Bilan
54b0800d08 INT-4280: Add taskScheduler() to Java DSL
JIRA: https://jira.spring.io/browse/INT-4280

To let end-user easily to inject custom `TaskScheduler` to the endpoint
add `ConsumerEndpointSpec.taskScheduler()` option

* Address PR comments
2017-06-28 10:12:38 -04:00
Artem Bilan
d766d233f5 Fix ReactiveStreamsConsumerTests and Checkstyle
Also increase timeouts in the `RedisAvailableRule`
2017-06-21 12:12:49 -04:00
Artem Bilan
a834ce9ffb FluxMessageChannel.send: no subscribers assertion 2017-06-21 11:10:41 -04:00
Artem Bilan
ecb7d03895 Upgrade to the latest SF and fix HTTP Reactive
https://build.spring.io/browse/INT-FATS5IC-206/

Also use `AopUtils.getTargetClass()` directly in the
`ReactiveHttpRequestExecutingMessageHandler` since it never returns `null`
2017-06-21 10:43:36 -04:00
Gary Russell
42dd05d655 INT-4298: Add Interface HeaderPropagationAware
JIRA: https://jira.spring.io/browse/INT-4298

Used to indicate components that can propagate headers; allows suppression.

Move Interface to AbstractMessageProducingHandler

Add Docs

* Fix typo in doc
2017-06-20 10:44:08 -04:00
Artem Bilan
fab0f5478f INT-4284: Remove info about readOnly headers
JIRA: https://jira.spring.io/browse/INT-4284

**Cherry-pick to 4.3.x**
2017-06-15 11:05:45 -04:00
Artem Bilan
4a47a7ce45 INT-4284: Exception to overwrite id or timestamp
JIRA: https://jira.spring.io/browse/INT-4284

To inform end-user that he/she can't override `id` and `timestamp` headers
throw a `BeanInitializationException` from the `gateway`, `header-enricher`,
`enricher` and `header-filter`  configuration when `id` and `timestamp` are
explicitly provided
2017-06-14 13:02:36 -04:00
Artem Bilan
99224f54c0 INT-4290: JacksonJsonUtils: Add Trusted Packages
JIRA: https://jira.spring.io/browse/INT-4290

See CVE-2017-4995

To disallow deserialization of unknown classes,
the `JacksonJsonUtils#messagingAwareMapper()` can now be supplied
with the `trustedPackages`.
The default list is:
```
java.util
java.lang
org.springframework.messaging.support
org.springframework.integration.support
org.springframework.integration.message
org.springframework.integration.store
```
Can be configured with `*` (asterisk) with meaning trust all

**Cherry-pick to 4.3.x**

Polishing according PR comments
2017-06-14 12:35:32 -04:00
Gary Russell
71149f260a Fix s-i-core and SF @Nullable Changes 2017-06-07 15:09:43 -04:00
Marius Bogoevici
ecc3006179 INT-4288: Add blacklist headers at runtime
JIRA: https://jira.spring.io/browse/INT-4288

- Add `AbstractMessageProducingHandler.addNotPropagatedHeaders()`
so that new headers can be added without overriding existing ones.
- Add `AbstractMessageProducingHandler.getNotPropagatedHeaders()`
so that existing excluded headers can be consulted.

* Polishing JavaDocs and imports order

**Cherry-pick to 4.3.x**
2017-06-06 16:43:18 -04:00
Gary Russell
f58106ec3c INT-4033: Support SpEL in Gateway Timeouts
JIRA: https://jira.spring.io/browse/INT-4033

Provide a mechanism to support dynamic timeouts when invoking gateway methods.

Polishing - PR Comments

* Polishing `what's new`
* Add `STOMP` to the `endpoint-summary.adoc`
* Polishing code style a bit for the changes in this fix
2017-06-06 10:56:59 -04:00
Gary Russell
cb444d8f5d INT-4284: Integration Properties @Bean => static
JIRA: https://jira.spring.io/browse/INT-4284

Prevent early instantiation of config.
2017-06-05 14:49:37 -04:00
Artem Bilan
33e9ce912f INT-4284: INFO about overriding readOnly headers
JIRA: https://jira.spring.io/browse/INT-4284

Add `INFO` into the `MessageBuilder#copyHeadersIfAbsent()` when end-user
tries to populate headers which are `readOnly`

We can't throw exception on the matter since can modify `readOnlyHeaders`
and that would force end-user to add `header-filter` logic to the application.

* Document `readOnly` headers in the `message.adoc`

**Cherry-pick 4.3.x**
2017-06-05 12:53:21 -04:00
Gary Russell
2c7bf9271c INT-4283: MarkerFilePresentFileListFilters
JIRA: https://jira.spring.io/browse/INT-4283

Filter that only passes files where a corresponding "marker" file is also present.

Implementations for file and(S)FTP.

* Polishing - PR Comments

* Missed one
2017-06-02 19:31:05 -04:00
Artem Bilan
a7ad22adcf ReactiveStreamsTests: Add Reactor diagnostics
https://build.spring.io/browse/INT-FATS5IC-186
2017-06-01 14:35:59 -04:00
Gary Russell
0d28e8c5e3 Revert "Ignore Test (temporary)"
This reverts commit a800d9683a.
2017-06-01 11:16:32 -04:00
Artem Bilan
f112ecbb7e INT-4269: Add Stream & Flux Support for Splitter
JIRA: https://jira.spring.io/browse/INT-4269

* Check if `outputChannel` is `ReactiveStreamsSubscribableChannel`
to let back-pressure splitting
* Build `Flux` or `Iterator` depending in the `reactive` state
* Allow to get a size of the `iterator` if it is possible,
for example `XPathMessageSplitter`

Add tests

Fix raw type and unused import

* Add JavaDocs to the `AbstractMessageSplitter#obtainSizeIfPossible()`
* Add asserts for the `sequenceSize` populataiton in the `XPathMessageSplitter`
* Document `Stream` & `Flux` support in the splitter

Minor doc polishing
2017-05-31 14:28:28 -04:00
Gary Russell
a800d9683a Ignore Test (temporary) 2017-05-31 12:21:46 -04:00
Gary Russell
492246eb13 Fix NPE (Mock Message with no Headers) 2017-05-31 12:10:27 -04:00
Gary Russell
d90d4e4d8f INT-4283: Restore Registration of Missing Comps
JIRA: https://jira.spring.io/browse/INT-4282

Unrecognized components were not added to `targetIntegrationComponents` causing
problems such as messge-driven components not starting.
2017-05-30 11:10:26 -04:00
Artem Bilan
533fa17f76 Fix Checkstyle violation in the DelayHandlerTests 2017-05-25 14:46:44 -04:00
Gary Russell
be336a0463 Make DelayHandlerTests More Robust
https://build.spring.io/browse/INT-MASTER-683/

Wait for release task to arrive in the scheduler queue.
2017-05-25 14:04:25 -04:00
Artem Bilan
ea89682368 INT-4252 IntegrationFlow: Allow Custom Bean Names
JIRA: https://jira.spring.io/browse/INT-4252

To allow to provide arbitrary bean names for the intermediate components
in the `IntegrationFlow` change `ComponentsRegistration` to return
a `Map<Object, String>` instead of raw `Collection<Object>`

* Use the value from that map in the `IntegrationFlowBeanPostProcessor` as a
fallback option before walking into bean name generation
* Provide `containerId` option for the AMQP DSL components

* Revert `@AfterClass` in the `AmqpTests`
* Expose `id()` for the `JmsDestinationAccessorSpec`
* Register `ListenerContainer` and `JmsTemplate`
as bean via `ComponentRegistration` in the particular JMS Java DSL components
* Fix `Jms` factory to populate the `JmsDefaultListenerContainerSpec`
for the `messageDrivenChannelAdapter()` without class specified

* Refactor AMQP DSL Inbound components to deal with the new ContainerSpec API
* Remove `containerId()` in favor of `id()` in the ContainerSpec
2017-05-23 14:46:06 -04:00
Gary Russell
799dcaae9f INT-4276: Selective Header Propagation
JIRA: https://jira.spring.io/browse/INT-4276

The `readOnlyHeaders` integration property allows suppression of certain headers
globally.

Add support for suppressing propagation on individual message handlers.
2017-05-22 12:56:55 -04:00
Artem Bilan
ea6cf0f4ef INT-4245: Improve ReplyProducingMHWrapper
JIRA: https://jira.spring.io/browse/INT-4245

* Move `ReplyProducingMessageHandlerWrapper` outside of
`ServiceActivatorAnnotationPostProcessor`.
* Use it from the `ServiceActivatorFactoryBean` as well to fix
the missed `Lifecycle` control
* Increase wait timeout in the `ConnectionFactoryTests`
* Mark endpoint as `auto-startup="false"` in the SFTP parser tests
* Remove redundant `org.gradle.daemon=true` from the `gradle.properties`
since it is like that by default for a while already
2017-05-18 08:37:56 -04:00
Artem Bilan
be98a6c2d0 INT-4274: Improve IntegrationFlow Lifecycle
JIRA: https://jira.spring.io/browse/INT-4274

Since any component from the `StandardIntegrationFlow` can be started
independently, the `StandardIntegrationFlow.stop()` should propagate
the stop control to the target components independently of its previous
`running` state

Javadoc Polishing
2017-05-16 15:33:40 -04:00
Artem Bilan
4dba3065ff INT-4275: Add IntegrationFlowContext.getRegistry
JIRA: https://jira.spring.io/browse/INT-4275

To analyze the state of the `IntegrationFlowContext` provide the access to
its internal `registry`
2017-05-16 08:36:08 -04:00
Artem Bilan
6c2200a03f Revert to B-S versions for base line dependencies
Increase receive timeouts in the `ResequencerTests`
2017-05-15 10:31:08 -04:00
Gary Russell
f4a17886be INT-4272: Read Only Header Improvements
JIRA: https://jira.spring.io/browse/INT-4272

- Honor read only header configuration when building from an unchanged message
- Support adding read only headers to the message builder factory

* Move `IntegrationMessageHeaderAccessor.containsReadOnly()` to the
`MessageBuilder` to avoid confusing

**Cherry-pick to 4.3.x**
2017-05-12 10:44:57 -04:00
Artem Bilan
709cb7fe4c Remove Jackson Imports from JacksonJsonUtils
Since the classes load their imports explicitly we can't use them for
classes which provides utilities to check classpath

* Move imports to fully qualified names in the target utility method

**Cherry-pick to 4.3.x**

Move `JacksonJsonUtils.isPresent` methods to separate `JacksonPresent` class

Polishing
2017-05-11 15:39:47 -04:00
Artem Bilan
9543877c0c Fix FluxMessageChannel for Multi Sources
* Rename `ReactiveConsumer` to `ReactiveStreamsConsumer`
* Rename `FluxSubscribableChannel` to `ReactiveStreamsSubscribableChannel`
* Remove the `processor` functionality from the `FluxMessageChannel`
in favor of internal `FluxSink` as it is recommended by the Project Reactor:
> Most of the time, you should try to avoid using a Processor.
They are harder to use correctly and prone to some corner cases.

* Make connectable, upstream publishers for the `FluxMessageChannel` as
bridges to the internal `sink` via `this::send`.
This way we are able to receive data from multi sources.
When the source is completed (e.g. `Mono` in case of WebFlux response),
the downstream flow isn't completed.
* Rework `MessageChannelReactiveUtils#PollableChannelPublisherAdapter`
to be based on the `Flux.create()` and `onRequest()` to poll channel for messages
* Add one more request to the `ReactiveHttpRequestExecutingMessageHandler`
to be sure that we consume different `Mono`s by the `FluxSubscribableChannel`
properly without completion
* Upgrade to the `spring-io-plugin:0.0.7.RELEASE`

Add `IntegrationConsumer` implementation to the `ReactiveStreamsConsumer`
2017-05-10 14:02:35 -04:00
Artem Bilan
7e919a4017 INT-4267: Add JSON (De)Serializers for Messaging
JIRA: https://jira.spring.io/browse/INT-4267
Fixes: spring-projects/spring-integration#2110

The documentation clearly point that we can simply use JSON (de)serialization
with the `RedisMessageStore`, but actually it isn't so easy

* Fix `MessageGroupMetadata`, `MessageHolder`, `MessageMetadata` for Jackson
deserialization compatibility
* Add `MessageHeaders`-based ctor to the `AdviceMessage`
* Add `MessageHeadersJacksonSerializer` to serialize `MessageHeaders`
to the `HashMap` for easier deserialization afterwards
* Add deserializer implementations for all `Message` types
* Add convenient `JsonObjectMapperProvider#jacksonMessageAwareMapper()`
factory method to build `ObjectMapper` supplied with mentioned above
(de)serializers

**Cherry-pick to 4.3.10 without `MessageHolder` and `MessageMetadata`**

Address PR comments and document the feature

Doc Polishing
2017-05-10 10:31:46 -04:00
Artem Bilan
aca97e7025 Fix Race Condition in TxSyncQueueChannelTests
https://build.spring.io/browse/INT-MASTER-653

The test-case uses only 1 second to wait for the message
in the `QueueChannel`.
That isn't enough on slow environment like CI causing failure for
the current test and unexpected value in the queue for the subsequent
tests

* Fix timeouts
* Get rid of `CountDownLatch` - the same is done by the wait on queues
* Purge queue in between tests.
This way we may not have failed subsequent test, thus failure analyze
will be much easier
* Rework `testRollback()` to wait for good result after one retry over
rollback
* Remove duplicate bean definition from XML config

**Cherry-pick to 4.3.x and 4.2.x**
2017-05-08 16:13:07 -04:00
Gary Russell
2b4c343eaa INT-4270: CGLib Compatibility
JIRA: https://jira.spring.io/browse/INT-4270

Initial Commit

Remove `final` modifier from certain critical framework methods that prevent CGLib proxies working.

Since Spring Boot 2.0 now uses proxyTargetClass=true by default (and has done that for
transactional proxies since 1.4), we must relax these restrictions so that CGLib can
proxy channels (which is often done to make subflows run in a transaction).

When CGLib can't override a `final` method, the fields used within those methods are uninitialized.
In the case of `AbstractMessageChannel` this causes an NPE on the unitiallized `dataTypes` field.
With `MessageHandler`, fields like `shouldTrack` are always false.

Further, methods on `IntegrationObjectSupport` - such as `getComponentName()` return null.

I have left setters and `afterPropertiesSet` (on IOS) `final` since these will typically be called
before the object is proxied.

I suspect there will be other methods we need to open up, but perhaps we should only do them on-demand.

__cherry-pick to 4.3.x__
2017-05-05 09:37:18 -04:00
Gary Russell
572467b1c4 INT-4260: MessagePublishingErrorHandler Orig. Msg
JIRA: https://jira.spring.io/browse/INT-4260

MPEH: Populate the `ErrorMessage.originalMessage`,
if available and not the same as the `failedMessage`.

Polishing - rename exception to `MessagingExceptionWrapper`;
 make `MPEH` a subclass of `ErrorMessagePublisher`

PR Comments

Polishing - PR Comments
2017-05-03 16:32:49 -04:00
Artem Bilan
18f9fcbb59 INT-4259: ACorrelationMH: WARN group reschedule
JIRA: https://jira.spring.io/browse/INT-4259

Since DEBUG can switched off for the application, it becomes difficult
to determine the wrong configuration for the discard flow from
the scheduler thread.

* Change logging level to the `WARN` and log the full stack-trace as well
to give more information about the reason of rescheduling group for
`forceComplete`

**Cherry-pick to 4.3.x**
2017-05-03 10:04:52 -04:00
Artem Bilan
f03356ccbc INT-4265: Add Java DSL .routeByError() EIP-Method
JIRA: https://jira.spring.io/browse/INT-4265

* Add `.routeByError()` EIP-method to the `IntegrationFlowDefinition`
based on the `ErrorMessageExceptionTypeRouter`
* Add missed `dynamicChannelLimit()` option to the `RouterSpec`
* Rework `RouterSpec#RouterMappingProvider` into the `ContextRefreshedEvent`
phase initialization to be sure that dependent `MappingMessageRouterManagement`
is fully initialized before applying mapping conversions

* Rename `routeByError()` to `routeByException()` to more reflect reality
of the `ErrorMessageExceptionTypeRouter` and don't mislead about
Java's `Error`
* Polish `ErrorMessageExceptionTypeRouter` JavaDocs a bit and some Java 8
code style
* Add `ApplicationContext` assertion into the
`RouterMappingProvider.onApplicationEvent()` do not trigger delegate
initialization if `ContextRefreshedEvent` is from the different app context
2017-05-02 14:26:04 -04:00
Gary Russell
ca231763a3 Rename ReactiveChannel
- as discussed last week

TODO:
- should we take the channel outside of the `AbstractMessageChannel` hierarchy?
  - avoid blocking interceptors
  - we would lose channel metrics though
- rename `ReactiveConsumer` ?

Polishing some missed renaming
2017-05-01 13:45:35 -04:00
Artem Bilan
b869666021 INT-4261: Do not Register Resource to TX Twice
JIRA: https://jira.spring.io/browse/INT-4261

When we have some `Advice` withing TX Advice which may perform `doPoll()`
several times, we unconditionally call
`transactionSynchronizationFactory.create(resource)`.
With the out-of-the-box implementations
`DefaultTransactionSynchronizationFactory` and
`PassThroughTransactionSynchronizationFactory`
we preform `TransactionSynchronizationManager.bindResource()`.
If resource is already there, an `IllegalStateException` is thrown

* Check that resource isn't bound already to the TX and don't create a new
`TransactionalResourceSynchronization` - just return `null`
* Check in the target users for the `null` before registering synchronization

Move resource registration to TX outside of out-of-the-box factories

* Fix condition in the `AbstractPollingEndpoint` for the resource
* Increase responsiveness of TX test to decreasing `fixed-delay`
and using `receive-timeout="-1"`
2017-04-28 12:56:03 -04:00
Artem Bilan
026484c382 Compatibility with the latest dependencies
* Upgrade to Gradle 3.5, SS-5.0, Hibernate-5.2.10, Mockito-2.7.22, Pah-1.1.1
And some other minor upgrades
* Fix deprecations for Mockito compatibility
* Fix `ServletWebSocketHandlerRegistry` deprecations
* Fix SD-Mongo deprecations
* Tweak JMX tests to avoid dangling threads after tests exist
* Increase timeouts in some polling tests
2017-04-26 11:29:18 -04:00