Commit Graph

8838 Commits

Author SHA1 Message Date
Gary Russell
b00a01bfca Update To Spring Framework 5.0.0.RC2, Sec 5.0.0.M2 2017-06-15 18:03:29 -04:00
Artem Bilan
fab0f5478f INT-4284: Remove info about readOnly headers
JIRA: https://jira.spring.io/browse/INT-4284

**Cherry-pick to 4.3.x**
2017-06-15 11:05:45 -04:00
Artem Bilan
4a47a7ce45 INT-4284: Exception to overwrite id or timestamp
JIRA: https://jira.spring.io/browse/INT-4284

To inform end-user that he/she can't override `id` and `timestamp` headers
throw a `BeanInitializationException` from the `gateway`, `header-enricher`,
`enricher` and `header-filter`  configuration when `id` and `timestamp` are
explicitly provided
2017-06-14 13:02:36 -04:00
Artem Bilan
99224f54c0 INT-4290: JacksonJsonUtils: Add Trusted Packages
JIRA: https://jira.spring.io/browse/INT-4290

See CVE-2017-4995

To disallow deserialization of unknown classes,
the `JacksonJsonUtils#messagingAwareMapper()` can now be supplied
with the `trustedPackages`.
The default list is:
```
java.util
java.lang
org.springframework.messaging.support
org.springframework.integration.support
org.springframework.integration.message
org.springframework.integration.store
```
Can be configured with `*` (asterisk) with meaning trust all

**Cherry-pick to 4.3.x**

Polishing according PR comments
2017-06-14 12:35:32 -04:00
Artem Bilan
cc4a166be5 Upgrade to Spring Data Kay M2 2017-06-14 12:09:28 -04:00
Artem Bilan
db692b1d8c Revert to Spring AMQP 2.0.0.M4
Since we don't have one more Spring AMQP 2.0 Milestone for upcoming
Spring Boot Milestone Spring Integration Milestone must be compatible
with current IO state.

* Replace `AmqpHeaders.RAW_MESSAGE` to the
`AmqpMessageHeaderErrorMessageStrategy.AMQP_RAW_MESSAGE` constant
2017-06-13 09:31:37 -04:00
Artem Bilan
6e38ed1dae Upgrade to Reactor 3.1.0.M2
https://build.spring.io/browse/INT-FATS5IC-197/
2017-06-12 11:56:27 -04:00
Artem Bilan
9c7c6dccfc Adapt to the latest Spring Data Redis 2017-06-08 17:06:28 -04:00
Artem Bilan
c053e46269 Revert to io.projectreactor.addons:reactor-test 2017-06-08 11:54:48 -04:00
Artem Bilan
83bd7f4bd3 Rallback to Reactor 3.1 M1
https://build.spring.io/browse/INT-MASTER-708/

Since SF hasn't been adapted to Reactor 3.1 M2, we can't upgrade either
2017-06-08 11:46:34 -04:00
Artem Bilan
1a019b85c2 Upgrade to Reactor-3.1.0.M2 & Reactor-Netty-0.6.4 2017-06-08 11:25:26 -04:00
Gary Russell
c7aa508434 Fix JMS Outbound Gateway
45ae138091 caused the reply destination to overridden to "".
2017-06-08 10:48:17 -04:00
Artem Bilan
fabc5fb725 Fix STOMP and WebSocket modules to the latest SF
* The `@MessageMapping` now requires `@SendTo` for replies
even if we are going to send to the default topic
* Fix assertion for exception message in the `StompInboundChannelAdapterWebSocketIntegrationTests`
2017-06-07 15:26:55 -04:00
Gary Russell
71149f260a Fix s-i-core and SF @Nullable Changes 2017-06-07 15:09:43 -04:00
Artem Bilan
45ae138091 Fix JMS and JMX module to the latest SF
* Fix `ControlBusParserTests` test to pass `Object.class` to the
`MessageConverter` since we can't use `null` there any more
* Fix `JmsOutboundGateway` for empty string for the
`replyDestinationName` since we can't pass `null` there any more
2017-06-07 14:34:01 -04:00
Gary Russell
2670a2872e Remove s-i-http Test Dependency from s-i-amqp
No longer needed.
2017-06-07 14:18:51 -04:00
Artem Bilan
bd53d8a1e0 SCSt-913: Fix AMQP Inbound Retry logic
Relates to spring-cloud/spring-cloud-stream#913
and 914094e51d

Also see https://github.com/spring-projects/spring-integration-kafka/pull/163

The `RetryTemplate` and `errorChannel` are mutually exclusive options:
```
Assert.state(getErrorChannel() == null, "Cannot have an 'errorChannel' property when a 'RetryTemplate' is "
	+ "provided; use an 'ErrorMessageSendingRecoverer' in the 'recoveryCallback' property to "
	+ "send an error message when retries are exhausted");
```
Therefore we don't have any error handling functionality if there is a retry but no `RecoveryCallback`.
So, don't add `RetryContext` to `ThreadLocal<AttributeAccessor>` if we don't have `RecoveryCallback` provided,
because that value is out of use then.

To make code more logical perform `attributesHolder.remove()` only for the non-retryable branch.
With that refactoring the missed `attributesHolder.remove()` has been observed in the `AmqpInboundGateway`
Add `attributesHolder.remove()` to the `RetryListener.close()` for retryable branch

**Cherry-pick to 4.3.x**
2017-06-07 12:09:52 -04:00
Marius Bogoevici
ecc3006179 INT-4288: Add blacklist headers at runtime
JIRA: https://jira.spring.io/browse/INT-4288

- Add `AbstractMessageProducingHandler.addNotPropagatedHeaders()`
so that new headers can be added without overriding existing ones.
- Add `AbstractMessageProducingHandler.getNotPropagatedHeaders()`
so that existing excluded headers can be consulted.

* Polishing JavaDocs and imports order

**Cherry-pick to 4.3.x**
2017-06-06 16:43:18 -04:00
Gary Russell
f58106ec3c INT-4033: Support SpEL in Gateway Timeouts
JIRA: https://jira.spring.io/browse/INT-4033

Provide a mechanism to support dynamic timeouts when invoking gateway methods.

Polishing - PR Comments

* Polishing `what's new`
* Add `STOMP` to the `endpoint-summary.adoc`
* Polishing code style a bit for the changes in this fix
2017-06-06 10:56:59 -04:00
Gary Russell
914094e51d INT-4256: AMQP: Conversion Errors to ErrorChannel
JIRA: https://jira.spring.io/browse/INT-4256

Also fix the JMS endpoint to use the `MessagingTemplate` instead of sending to the
error channel directly (ignored the send result).

Missing commit

Polishing - PR Comments

* Some additional polishing: remove extra `ifs`; make internal
classes `protected` for possible inheritors

Conflicts:
	spring-integration-amqp/src/main/java/org/springframework/integration/amqp/inbound/AmqpInboundChannelAdapter.java
	spring-integration-amqp/src/main/java/org/springframework/integration/amqp/inbound/AmqpInboundGateway.java

AMQP: Add Support for SCSt Error Handling

See: https://github.com/spring-cloud/spring-cloud-stream/issues/913

Add retry within `onMessage` so we have access to the converted message
as well as the original spring-amqp message, which is added to the
`ErrorMessage` as a header.

Similar to the work in spring-integration-kafka.

* Polishing JavaDocs
* Rename `AmqpMessageHeaderErrorMessageStrategy.AMQP_RAW_DATA`
to `AmqpMessageHeaderErrorMessageStrategy.AMQP_MESSAGE`

Conflicts:
	spring-integration-amqp/src/main/java/org/springframework/integration/amqp/inbound/AmqpInboundChannelAdapter.java
	spring-integration-amqp/src/main/java/org/springframework/integration/amqp/inbound/AmqpInboundGateway.java
	spring-integration-amqp/src/test/java/org/springframework/integration/amqp/inbound/InboundEndpointTests.java

* Use `AmqpHeaders.RAW_MESSAGE` as a retry context key
* Simple polishing for Java 8 code style
2017-06-06 09:48:14 -04:00
Gary Russell
b1f74577db AMQP: Use ErrorMessageStrategy
Fix a couple of places where `ErrorMessage` was created directly instead
of delegating to the configured `ErrorMessageStrategy`.
2017-06-05 14:57:35 -04:00
Gary Russell
cb444d8f5d INT-4284: Integration Properties @Bean => static
JIRA: https://jira.spring.io/browse/INT-4284

Prevent early instantiation of config.
2017-06-05 14:49:37 -04:00
Artem Bilan
33e9ce912f INT-4284: INFO about overriding readOnly headers
JIRA: https://jira.spring.io/browse/INT-4284

Add `INFO` into the `MessageBuilder#copyHeadersIfAbsent()` when end-user
tries to populate headers which are `readOnly`

We can't throw exception on the matter since can modify `readOnlyHeaders`
and that would force end-user to add `header-filter` logic to the application.

* Document `readOnly` headers in the `message.adoc`

**Cherry-pick 4.3.x**
2017-06-05 12:53:21 -04:00
Gary Russell
2c7bf9271c INT-4283: MarkerFilePresentFileListFilters
JIRA: https://jira.spring.io/browse/INT-4283

Filter that only passes files where a corresponding "marker" file is also present.

Implementations for file and(S)FTP.

* Polishing - PR Comments

* Missed one
2017-06-02 19:31:05 -04:00
Gary Russell
c3031a2c7c INT-4285: TCP: Add Elastic Raw Deserializer
JIRA: https://jira.spring.io/browse/INT-4285

Buffer grows as needed.
2017-06-02 17:37:45 -04:00
Artem Bilan
a7ad22adcf ReactiveStreamsTests: Add Reactor diagnostics
https://build.spring.io/browse/INT-FATS5IC-186
2017-06-01 14:35:59 -04:00
Gary Russell
0d28e8c5e3 Revert "Ignore Test (temporary)"
This reverts commit a800d9683a.
2017-06-01 11:16:32 -04:00
Artem Bilan
f112ecbb7e INT-4269: Add Stream & Flux Support for Splitter
JIRA: https://jira.spring.io/browse/INT-4269

* Check if `outputChannel` is `ReactiveStreamsSubscribableChannel`
to let back-pressure splitting
* Build `Flux` or `Iterator` depending in the `reactive` state
* Allow to get a size of the `iterator` if it is possible,
for example `XPathMessageSplitter`

Add tests

Fix raw type and unused import

* Add JavaDocs to the `AbstractMessageSplitter#obtainSizeIfPossible()`
* Add asserts for the `sequenceSize` populataiton in the `XPathMessageSplitter`
* Document `Stream` & `Flux` support in the splitter

Minor doc polishing
2017-05-31 14:28:28 -04:00
Gary Russell
a800d9683a Ignore Test (temporary) 2017-05-31 12:21:46 -04:00
Gary Russell
492246eb13 Fix NPE (Mock Message with no Headers) 2017-05-31 12:10:27 -04:00
Gary Russell
d90d4e4d8f INT-4283: Restore Registration of Missing Comps
JIRA: https://jira.spring.io/browse/INT-4282

Unrecognized components were not added to `targetIntegrationComponents` causing
problems such as messge-driven components not starting.
2017-05-30 11:10:26 -04:00
Artem Bilan
bb2ee967f2 INT-4002: JDBC MS Doc: DB init in the beginning
JIRA https://jira.spring.io/browse/INT-4002

To avoid some confuses during developing target applications,
warn end-users that it is important to initialize data base before
running applications with JDBC Message Stores
2017-05-25 15:58:57 -04:00
Artem Bilan
533fa17f76 Fix Checkstyle violation in the DelayHandlerTests 2017-05-25 14:46:44 -04:00
Artem Bilan
0379565264 Fix JavaDocs for Remote File components
https://build.spring.io/browse/INT-MASTER-684/
2017-05-25 14:34:29 -04:00
Gary Russell
be336a0463 Make DelayHandlerTests More Robust
https://build.spring.io/browse/INT-MASTER-683/

Wait for release task to arrive in the scheduler queue.
2017-05-25 14:04:25 -04:00
Gary Russell
851a2cfdee Fix CachingClientConnectionFactoryTests
gatewayIntegrationTest() was stealing integrationTest()'s message.

Shutdown its executor and wait for the shutdown.

Also add a meaningful toString() to TcpConnectionSupport.
2017-05-25 13:09:41 -04:00
Artem Bilan
57feb5262f Improvements for MockIntegration 2017-05-25 12:12:54 -04:00
Artem Bilan
3daeaab1b4 INT-4060: FTP Gateway: Add NLST and workDir
JIRA: https://jira.spring.io/browse/INT-4060

* Add `NLST` command to the `AbstractRemoteFileOutboundGateway` to perform
`listNames` on the target session.
Useful in case of server doesn't allow to perform `LS` or the names set is
sufficient for application requirements
* Add `workingDirExpression` to the `FtpOutboundGateway` to allow to perform
`FtpClient.changeWorkingDirectory()` based on the current request message
* Change `slf4j-log4j12` to the `testCompile` -
the FTP tests fail in the IDE with `ClassNotFoundException`

Address PR comments

* Add `nlst` to XSD config
* Reinstate `ls -1` test-case for the `FtpServerOutboundTests`
* wrap more commands to the `doInWorkingDirectory()`

Fix `MV` command in the `AbstractRemoteFileOutboundGateway`

* Implement `RemoteFileOperations#invoke(OperationsCallback<F, T>)`
for thread-bound `session`s
* Use a new `invoke()` for `put()` and `mPut()` commands in the `AbstractRemoteFileOutboundGateway`
* Add delegation for the `put()` and `mPut()` commands in the `FtpOutboundGateway`
* Add DSL support for the `workingDirExpression` and add `whats-new.adoc` note

Document changes

Address some PR comments:

* Add `invokeScope` variable to `execute` to track `ThreadLocal` session or not
* Check for `null` in the `getSession()` and fallback to regular
`sessionFactory.getSession()`.
Most likely the `invoke()` is called from other thread

Doc Polishing
2017-05-25 11:23:30 -04:00
Artem Bilan
b7801f6311 Fix CookieTests for the latest SF
https://build.spring.io/browse/INT-MASTER-679/
2017-05-24 15:31:06 -04:00
Gary Russell
4991d963e8 INT-4164: TCP Support Pushback and Subclassing
JIRA: https://jira.spring.io/browse/INT-4164

Previously, it was not possible to create a subclass of `TcpNetConnection`.
This was possible for `TcpNioConnection` via `TcpNioConnectionSupport`.

Add `TcpNetConnectionSupport` and a default implementation.

Create `AbstractTcpConnectionSupport` and add support for all implementation (Net, NIO, NIO with SSL)
to support wrapping the connection's `InputStream` in a `PushBackInputStream`.

checkstyle

Polishing - PR Comments

* Polishing TCP tests a bit:
 - mark some of them with the `LongRunningIntegrationTest` `@Rule`
 - Remove redundant `AFTER_EACH_TEST_METHOD`
 - use Lambda for the `ApplicationEventPublisher` in tests
2017-05-24 15:07:58 -04:00
Artem Bilan
ea89682368 INT-4252 IntegrationFlow: Allow Custom Bean Names
JIRA: https://jira.spring.io/browse/INT-4252

To allow to provide arbitrary bean names for the intermediate components
in the `IntegrationFlow` change `ComponentsRegistration` to return
a `Map<Object, String>` instead of raw `Collection<Object>`

* Use the value from that map in the `IntegrationFlowBeanPostProcessor` as a
fallback option before walking into bean name generation
* Provide `containerId` option for the AMQP DSL components

* Revert `@AfterClass` in the `AmqpTests`
* Expose `id()` for the `JmsDestinationAccessorSpec`
* Register `ListenerContainer` and `JmsTemplate`
as bean via `ComponentRegistration` in the particular JMS Java DSL components
* Fix `Jms` factory to populate the `JmsDefaultListenerContainerSpec`
for the `messageDrivenChannelAdapter()` without class specified

* Refactor AMQP DSL Inbound components to deal with the new ContainerSpec API
* Remove `containerId()` in favor of `id()` in the ContainerSpec
2017-05-23 14:46:06 -04:00
Gary Russell
2b99c191f6 INT-4117: File Outbound Set File Permissions
JIRA: https://jira.spring.io/browse/INT-4117

Add an option to set file permissions on the `FileWritingMessageHandler` when the file system supports it.

Polishing - PR Comments; more robust file system detection.

Add DSL support and fix NPE with missing timestamp header.

Fix javadoc

Fix javadoc

Some polishing:

* Add `chmod >= 0` assertion
* Make some assertion with string concatenation as `Supplier`-based to
deffer that string concatenation
2017-05-23 13:55:39 -04:00
Artem Bilan
c8aafde20b Downgrade to Reactor-3.1.0.M1
https://build.spring.io/browse/INT-MASTER-675/

Since SF is still based on the `Bismuth-M1`,
we can't afford ourselves to go to `BUILD-SNAPSHOT` yet

Also fix `FileTests` do not poll the same file again for `splitter` case
2017-05-23 11:08:42 -04:00
Artem Bilan
94e5271a57 Increase timeouts for RedisQueueMDrEndpointTests
https://build.spring.io/browse/INT-MJATS41-999/

**Cherry-pick to 4.3.x**
2017-05-23 09:45:52 -04:00
Gary Russell
b84a334379 INT-4166: ThreadAffinityClientConnectionFactory
JIRA: https://jira.spring.io/browse/INT-4166

Binds connections to threads.

Polishing - PR Comments

* Fix `ip.adoc` typo for `[[tcp-affinity-cf]]`
2017-05-22 19:21:01 -04:00
Gary Russell
799dcaae9f INT-4276: Selective Header Propagation
JIRA: https://jira.spring.io/browse/INT-4276

The `readOnlyHeaders` integration property allows suppression of certain headers
globally.

Add support for suppressing propagation on individual message handlers.
2017-05-22 12:56:55 -04:00
Gary Russell
b5bbb93fe8 INT-4256: AMQP: Conversion Errors to ErrorChannel
JIRA: https://jira.spring.io/browse/INT-4256

Also fix the JMS endpoint to use the `MessagingTemplate` instead of sending to the
error channel directly (ignored the send result).

Missing commit

Polishing - PR Comments

* Some additional polishing: remove extra `ifs`; make internal
classes `protected` for possible inheritors
2017-05-22 12:55:54 -04:00
Gary Russell
3cf85afe1f INT-4278: DSL: AMQP DirectMessageListenerContainer
JIRA: https://jira.spring.io/browse/INT-4278

Support configuration of inbound endpoints using `DirectMessageListenerContainer`.

**Needs migration guide update**

Polishing according PR discussion
2017-05-19 14:07:40 -04:00
Artem Bilan
079ccb84e2 Add MockMessageHandler to the Testing Framework
Fix PayloadMatcherTests for generics

Address PR comments and other improvements

* Revert `rawtypes` mode for the `PayloadMatcher`
* Make `HeaderMatcher` as `rawtypes` as well
* Make `MockMessageHandler` expect `rawtypes` for `Matcher`s.
This way we can just support `Matcher`s like `notNullValue(Message.class)`
* Rename `expect()` to `assertNext()`
* Rename `andReply()` to `thenReply()`
* Track replies are supplied in the `MockMessageHandler`
* Distinguish simple `MH` from the `MP` types in the
`MockIntegrationContext#instead()` do not let to replace simple `MH`
with fully configured `MockMessageHandler` or any other `MP` implementation.
Fail replace if types mismatch; wrap `MockMessageHandler` to simple `MH`
if it doesn't have replies when we are going to replace simple `MH`
* Wrap `MockMessageHandler` to the `Mockito.spy()` in the
`MockIntegration#mockMessageHandler()` to allow to `verify()` interaction
in the test-case

Remove wrapping `MockMH` to raw `MH` when no reply supported.
If `MockMH` isn't supplied with replies ti's safe to use it as is - no harm to target endpoint
which supposed to be last one in the flow

Some polishing and JavaDocs

More JavaDocs

Add docs for the `MockMessageHandler` and fix some JavaDocs

Make the `MockMessageHandler` with an API like:
```
MockIntegration.mockMessageHandler()
             .handleNext(Consumer<Message<?>>)
             .handleNext(Consumer<Message<?>>)
             .handleNextAndReply(Function<Message<?>, Object>)
             .handleNext(Consumer<Message<?>>)
             .handleNextAndReply(Function<Message<?>, Object>)
             .handleNextAndReply(Function<Message<?>, Object>);
```

Doc Polishing
2017-05-18 11:25:43 -04:00
Artem Bilan
ea6cf0f4ef INT-4245: Improve ReplyProducingMHWrapper
JIRA: https://jira.spring.io/browse/INT-4245

* Move `ReplyProducingMessageHandlerWrapper` outside of
`ServiceActivatorAnnotationPostProcessor`.
* Use it from the `ServiceActivatorFactoryBean` as well to fix
the missed `Lifecycle` control
* Increase wait timeout in the `ConnectionFactoryTests`
* Mark endpoint as `auto-startup="false"` in the SFTP parser tests
* Remove redundant `org.gradle.daemon=true` from the `gradle.properties`
since it is like that by default for a while already
2017-05-18 08:37:56 -04:00