Commit Graph

8881 Commits

Author SHA1 Message Date
Artem Bilan
f00eafb4f8 GH-152: Scanner & filters are mutually exclusive
Fix spring-projects/spring-integration-java-dsl#152

* Add Logic to the `FileInboundChannelAdapterSpec` do not populate
`FileListFilterFactoryBean` to the `FileReadingMessageSource`
when the `scanner` is provided.
Otherwise call the `FileReadingMessageSource` and rely on its logic to
refuse mutually exclusive `scanner` and `filter/locker` options
2017-07-31 10:52:29 -04:00
Artem Bilan
0f78d4b2ec Some JDBC tests polishing and optimization
**Cherry-pick to master**

Conflicts:
	build.gradle
	spring-integration-jdbc/src/test/java/org/springframework/integration/jdbc/store/channel/DataSource-derby-context.xml
2017-07-26 16:16:07 -04:00
Spring Buildmaster
3b03b6ac91 [artifactory-release] Next development version 2017-07-25 17:26:55 +00:00
Spring Buildmaster
5934001a62 [artifactory-release] Release version 5.0.0.M6 2017-07-25 17:26:49 +00:00
Artem Bilan
4b49c45137 Upgrade to SD Kay RC2 2017-07-25 12:50:22 -04:00
Artem Bilan
74b59ef8d3 INT-4316 Add ObjectToMapTransformer.mapper option
JIRA: https://jira.spring.io/browse/INT-4316

To allow to customize a JSON representation of the Object, provide a
`JsonObjectMapper`-based ctor for the `ObjectToMapTransformer`
2017-07-25 11:58:41 -04:00
Artem Bilan
784d891fa5 Fix Checkstyle violations 2017-07-24 14:05:53 -04:00
Artem Bilan
c26cac4c17 Upgrade to the released Milestones
* SF-5.0 RC3
* AMQP2.0 M5
* Security 5.0 M3
* Spring Twitter 2.0 M4
* Gradle 4.0.1
* Checkstyle 8.0
* JsonPath 2.3
* Fix deprecations and Checkstyle violations
2017-07-24 13:42:12 -04:00
Artem Bilan
5a66854c67 IntegrationFlow: optional @MessagingGateway
When we use `IntegrationFlows.from(Class<?>)`, we don't need to require
present for `@MessagingGateway`, we can just synthesize it with the
default properties

This way we can simply make gateway proxies for the `java.util.function`
interfaces which should be useful for Spring Cloud Function
2017-07-20 17:54:12 -04:00
Artem Bilan
d4a99919ed INT-4300: Add WebFlux Server Support
JIRA: https://jira.spring.io/browse/INT-4300

* Add `ReactiveHttpInboundEndpoint` based on the WebFlux foundation
* Extract `BaseHttpInboundEndpoint` for common
HTTP Inbound Channel Adapters options
* Make `spring-webmvc` and `spring-webflux` as `optional` dependencies
to let end-user to choose
* Refactor `HttpContextUtils` to include constants
for newly added WebFlux support
* Introduce `BaseHttpInboundEndpoint.setRequestPayloadTypeClass()`
for raw `Class<?>` and modify existing `setRequestPayloadType()`
for the `ResolvableType`
* Refactor existing MVC tests and XML components parsers to use
new `setRequestPayloadTypeClass()`
* Add `MessagingGatewaySupport.sendAndReceiveMessageReactive()` to get
a reply from downstream flow reactive back-pressure manner
* Add `IntegrationHandlerResultHandler` implementation to let WebFlux
infrastructure to handle the `Mono<Void>` from the `ReactiveHttpInboundEndpoint`
properly
* Fix `JdbcLockRegistryLeaderInitiatorTests` race condition to assert
the `initiator1` is elected eventually after yielding when the `initiator2`
is stopped

* Fix JavaDocs issue in the `HttpRequestHandlingMessagingGateway`
* Move all the "hard" logic in the `MessagingGatewaySupport#doSendAndReceiveMessageReactive`
to the `Mono` chain ensuring back-pressure when `sendAndReceiveMessageReactive()` is
called not from the Reactive Stream

Add test-case to demonstrate SSE

JIRA: https://jira.spring.io/browse/INT-3625

Some polishing and optimization for the
`MessagingGatewaySupport.doSendAndReceiveMessageReactive()`

More optimization for `MessagingGatewaySupport`

* Upgrade to Reactor 3.1 M3
* Document WebFlux-based components

Minor Doc Polishing
2017-07-20 12:19:17 -04:00
Artem Bilan
e475d9c695 INT-4308: Document JdbcMetadataStore
JIRA: https://jira.spring.io/browse/INT-4308

* Document `JdbcMetadataStore`
* Remove unnecessary context DERBY config for the `PersistentAcceptOnceFileListFilterExternalStoreTests`
* Rework the `testFileSystemWithJdbcMetadataStore()` to use `EmbeddedDatabase` directly, without context
2017-07-20 10:55:03 -04:00
Artem Bilan
f52fdd85ab Fix Gemfire tests for the latest SDG
https://build.spring.io/browse/INT-MASTER-JOB1-750
2017-07-19 17:40:51 -04:00
Artem Bilan
bccaa52bd2 Make testAll task depends on check; exclude test 2017-07-19 17:15:47 -04:00
Artem Bilan
7a67c1f1a3 Various polishing and optimizations
* Check for `null` package in the `JacksonJsonUtils`.
Also don't call `this.delegate.typeFromId()` the second time
* Increase wait timeout in the `QueueChannelTests`
* Check `ZookeeperMetadataStore.addListener()` for `null`

**Cherry-pick to 4.3.x**
2017-07-14 16:31:46 -04:00
Artem Bilan
748feea0f8 Upgrade to SD Kay B-S and fix MongoDB accordingly
https://build.spring.io/browse/INT-FATS5IC-229/
2017-07-14 11:11:50 -04:00
Artem Bilan
012bebd7bf Impl CoreSubscriber instead of Subscriber
According Reactor recommendation it would be better to `CoreSubscriber`
from Reactor instead of Reactive Streams `Subscriber` for chasing
some optimisation in case of Reactor flows

Increase `receive()` timeout in the `AttributePollingChannelAdapterParserTests`
2017-07-13 17:20:16 -04:00
Artem Bilan
6829da995f INT-4301: Make CircuitBreakerOpenException as ME
JIRA: https://jira.spring.io/browse/INT-4301

To keep the context of the `failedMessage` and avoid extra wrapping
to the `MessagingException` make
`CircuitBreakerOpenException extends MessagingException` directly
2017-07-12 18:46:55 -04:00
Artem Bilan
4663862ea6 Resolve deprecations since the latest SF 2017-07-11 11:52:11 -04:00
Kai Zimmermann
c0d4375b1e INT-4311: JdbcLocks: QueryTimeoutException retry
JIRA: https://jira.spring.io/browse/INT-4311

* Added `QueryTimeoutException` to retry handling in the `JdbcLockRegistry`

Polishing

**Cherry-pick to 4.3.x**
2017-07-11 11:33:47 -04:00
Artem Bilan
d79e70c6df GH-2197: Rely on the Document creation for Source
Resolves https://github.com/spring-projects/spring-integration/issues/2197

Sometime we would like to avoid some XML definition features (e.g. loading of the DTD).
The `XmlValidatingMessageSelector` doesn't allow to do that easily because it doesn't
rely on the `Document` creation via `DocumentBuilderFactory`, but directly
manipulates with `Source`

* Change `DefaultXmlPayloadConverter.convertToSource()` to fallback to the `convertToDocument()`
for non-Document payloads.
This way we can configure `DefaultXmlPayloadConverter` with custom `DocumentBuilderFactory` and
bypass DTD loading, for example
* Expose `xml-converter` option for the `<int-xml:validating-filter>`  component
2017-07-06 12:04:16 -04:00
Gary Russell
6c62af53f4 INT-4309 Http Header Mapper VarArgs Instead of []
JIRA: https://jira.spring.io/browse/INT-4309

More convenient for Java Config.

*Upgrade to Reactor 3.1 B-S, Reactor-Netty 0.7 B-S and Security 5.0 B-S.
Otherwise we are not compatible with the latest SF
2017-07-05 13:38:20 -04:00
Bojan Vukasovic
47003077cc INT-4306: Add support for JdbcMetadataStore
JIRA: https://jira.spring.io/browse/INT-4306

Fixed checkstyle errors

Fixed checkstyle errors

Fixed checkstyle errors and sql schema

Manually fixed derby sql schema

Fixing some issues

added region support

small refactorings

test checkstyle fix

integration tests and fixes

integration tests and fixes
2017-06-29 13:44:28 -04:00
Artem Bilan
54b0800d08 INT-4280: Add taskScheduler() to Java DSL
JIRA: https://jira.spring.io/browse/INT-4280

To let end-user easily to inject custom `TaskScheduler` to the endpoint
add `ConsumerEndpointSpec.taskScheduler()` option

* Address PR comments
2017-06-28 10:12:38 -04:00
Artem Bilan
72bd4e5bcf INT-4305: (S)FTP: Remove Local File Before Rename
JIRA: https://jira.spring.io/browse/INT-4305

The `File.renameTo()` operation may fail, therefore the content of the
local file isn't changed, but since we change `setLastModified()` anyway,
this file might be eligible for local polling.
So, we end up with the same content from local file in a new message,
meanwhile we expect a new content from the remote file

* Check the `File.renameTo()` result and attempt to `delete()`
for existing local file
* When file isn't renames remove the remote file from the `filter` to let
it be transferred one more time on the next poll.
The local file might be opened for processing, so this way we postpone a fresh
remote file for the future poll rounds
* Modify `copyFileToLocalDirectory()` to return `boolean` to reflect the fact of copy.
This way we check the real number of transferred files

**Cherry-pick to 4.3.x**

Do not transfer remote file if we can't remove local one

Polishing Log Messages
2017-06-27 16:45:15 -04:00
Gary Russell
36ac9f2203 (S)FTP MaxFetchSize Doc Polishing and DSL
Also polish programming tips and tricks docs.

* Fix typos in docs
2017-06-27 15:45:40 -04:00
Gary Russell
71d9f173e8 Doc Polishing 2017-06-27 13:18:38 -04:00
Artem Bilan
d8382e0bd0 INT-4297: Add FileSplitter.setFirstLineAsHeader()
JIRA: https://jira.spring.io/browse/INT-4297

* Add `FileSplitter.setFirstLineAsHeader()` and extract the first line from
the context before `iterator`.
* Populate the extracted header to each subsequent line

* Don't use `Streams` to read first line
* Add XSD and Java DSL option for the `firstLineAsHeader`
* Document the feature
* Add Java DSL sample for the `Files.slitter()`

Add a note about more complex header enrichment

Doc Polishing and fix field name.
2017-06-27 13:14:10 -04:00
Gary Russell
5216dc2b02 INT-4304 (S)FTP Outbound Gateway Delete After GET
JIRA: https://jira.spring.io/browse/INT-4304

Add an option to delete remote file(s) after GET/MGET.

Polishing - log error on delete failure.
2017-06-26 14:30:08 -04:00
Artem Bilan
a1c42f65d0 Fix timing issue in test for STOMP Server
https://build.spring.io/browse/INT-MASTER-733

The Netty tries to load `netty-transport-native-epoll` too long, more
then 10 seconds we expect in the `AbstractStompSessionManager`, therefore
 we end up having `StompConnectionFailedEvent` as a first event in the
queue channel for STOMP events.

* Increase connection wait timeout to `30 secs`, like it is in many other
places in Spring
* Provide some Java 8 style refactoring for the `AbstractStompSessionManager`
* Fix `StompServerIntegrationTests` to poll events from queue until
`StompSessionConnectedEvent` any way
* Upgrade to Gradle 4.0 and `spring-io-plugin-0.0.8`
* Bump `org.sonarqube` version to `2.5` - `2.1` isn't compatible with Gradle 4.0
2017-06-23 15:41:37 -04:00
Gary Russell
0903df9676 INT-4303: (S)FTP MGET with FileExistsMode.IGNORE
JIRA: https://jira.spring.io/browse/INT-4303

Don't emit ignored files in the output message payload.

Also, always preserve the timestamp when using `FileExistsMode.REPLACE_IF_MODIFIED`.

Some code style polishing
2017-06-23 14:02:03 -04:00
Artem Bilan
4679c9bdbe INT-4302: Add fileExistsMode() to Java DSL
JIRA: https://jira.spring.io/browse/INT-4302

Add `RemoteFileOutboundGatewaySpec#fileExistsMode()`
2017-06-23 13:32:55 -04:00
Artem Bilan
171d169748 Fix JDBC tests to close embedded DB
https://build.spring.io/browse/INT-MJATS41-1030

**Cherry-pick to 4.3.x**
2017-06-23 09:10:32 -04:00
Gary Russell
e16e81f780 Revert ftp test log4j 2017-06-22 20:14:50 -04:00
Gary Russell
6b6782e16b (S)FTP Doc Polishing 2017-06-22 20:11:49 -04:00
Artem Bilan
d766d233f5 Fix ReactiveStreamsConsumerTests and Checkstyle
Also increase timeouts in the `RedisAvailableRule`
2017-06-21 12:12:49 -04:00
Artem Bilan
a834ce9ffb FluxMessageChannel.send: no subscribers assertion 2017-06-21 11:10:41 -04:00
Artem Bilan
ecb7d03895 Upgrade to the latest SF and fix HTTP Reactive
https://build.spring.io/browse/INT-FATS5IC-206/

Also use `AopUtils.getTargetClass()` directly in the
`ReactiveHttpRequestExecutingMessageHandler` since it never returns `null`
2017-06-21 10:43:36 -04:00
Gary Russell
42dd05d655 INT-4298: Add Interface HeaderPropagationAware
JIRA: https://jira.spring.io/browse/INT-4298

Used to indicate components that can propagate headers; allows suppression.

Move Interface to AbstractMessageProducingHandler

Add Docs

* Fix typo in doc
2017-06-20 10:44:08 -04:00
Gary Russell
4ca91d912c INT-4295: XML Schema (Legacy) Documentation
JIRA: https://jira.spring.io/browse/INT-4295

Also fixed link to github in the online core schema.
2017-06-19 11:30:51 -04:00
Gary Russell
a38bf60332 INT-4271: FWMH: Fix Race Condition with stop()
JIRA: https://jira.spring.io/browse/INT-4271

The `Flusher.run()` is `synchronized` on the `FWMH`; a `stop()` while it's running will interrupt the task
which could cause a state to be removed without a flush actually happening (interrupt
on lock acquisition).

- synchronize `stop()` so it won't interrupt a running flusher
- detect an interrupt in the flusher and stop flushing


**Cherry-pick to 4.3.x**
2017-06-19 10:38:24 -04:00
Gary Russell
404c2c5369 INT-4296: Fix APPEND_NO_FLUSH Timing Problem
JIRA: https://jira.spring.io/browse/INT-4296

The `FileState.close()` method correctly acquires the lock to prevent closing while
writing. However, with a short flush interval, the close could occur between state
creation and the write operation acquiring the lock.

Move the state creation to within the scope of the lock acquired by the writer.
2017-06-19 10:02:32 -04:00
Spring Buildmaster
efd944cefe [artifactory-release] Next development version 2017-06-15 22:53:46 +00:00
Spring Buildmaster
207483a3a6 [artifactory-release] Release version 5.0.0.M5 2017-06-15 22:53:39 +00:00
Gary Russell
b00a01bfca Update To Spring Framework 5.0.0.RC2, Sec 5.0.0.M2 2017-06-15 18:03:29 -04:00
Artem Bilan
fab0f5478f INT-4284: Remove info about readOnly headers
JIRA: https://jira.spring.io/browse/INT-4284

**Cherry-pick to 4.3.x**
2017-06-15 11:05:45 -04:00
Artem Bilan
4a47a7ce45 INT-4284: Exception to overwrite id or timestamp
JIRA: https://jira.spring.io/browse/INT-4284

To inform end-user that he/she can't override `id` and `timestamp` headers
throw a `BeanInitializationException` from the `gateway`, `header-enricher`,
`enricher` and `header-filter`  configuration when `id` and `timestamp` are
explicitly provided
2017-06-14 13:02:36 -04:00
Artem Bilan
99224f54c0 INT-4290: JacksonJsonUtils: Add Trusted Packages
JIRA: https://jira.spring.io/browse/INT-4290

See CVE-2017-4995

To disallow deserialization of unknown classes,
the `JacksonJsonUtils#messagingAwareMapper()` can now be supplied
with the `trustedPackages`.
The default list is:
```
java.util
java.lang
org.springframework.messaging.support
org.springframework.integration.support
org.springframework.integration.message
org.springframework.integration.store
```
Can be configured with `*` (asterisk) with meaning trust all

**Cherry-pick to 4.3.x**

Polishing according PR comments
2017-06-14 12:35:32 -04:00
Artem Bilan
cc4a166be5 Upgrade to Spring Data Kay M2 2017-06-14 12:09:28 -04:00
Artem Bilan
db692b1d8c Revert to Spring AMQP 2.0.0.M4
Since we don't have one more Spring AMQP 2.0 Milestone for upcoming
Spring Boot Milestone Spring Integration Milestone must be compatible
with current IO state.

* Replace `AmqpHeaders.RAW_MESSAGE` to the
`AmqpMessageHeaderErrorMessageStrategy.AMQP_RAW_MESSAGE` constant
2017-06-13 09:31:37 -04:00
Artem Bilan
6e38ed1dae Upgrade to Reactor 3.1.0.M2
https://build.spring.io/browse/INT-FATS5IC-197/
2017-06-12 11:56:27 -04:00