Commit Graph

8906 Commits

Author SHA1 Message Date
Gary Russell
0d495294ed INT-4342: White List for Payload Deserializer
JIRA: https://jira.spring.io/browse/INT-4342

Use similar code to Spring AMQP to add white list support for Integration's
use of the `DeserializingMessageConverter`; introduce the `WhiteListDeserializingMessageConverter`.

Polishing

Missed this change in PR.

Fix XSD attribute
2017-09-08 15:31:47 -04:00
Artem Bilan
5749c5b237 INT-4320: Fix AMQP channels to declare on demand
JIRA: https://jira.spring.io/browse/INT-4320

To avoid "fail fast" situation when Broker isn't available during
application init phase, move all the declarations in
the AMQP Message Channels to the `ConnectionListener.onCreate()`.
In addition call the declaration from the `Lifecycle.start()`
2017-09-08 14:46:03 -04:00
Matthias Jeschke
b9c8a8edf8 INT-4341: RedisQueueIn: support receiveTimeout=0
JIRA: https://jira.spring.io/browse/INT-4341
Fixes spring-projects/spring-integration#2228

Add to the `RedisQueueInboundGateway` and `RedisQueueMessageDrivenEndpoint`support
for the `receiveTimeout` of 0 and block indefinitely

Added some simple tests to check if the receive timeout can be set to 0

**Cherry-pick to 4.3.x**
2017-09-06 17:55:06 -04:00
Artem Bilan
46de69dbf2 Honor the FluxSink State in the PollChPublisherAd
The `PollableChannelPublisherAdapter` is based on the poll model of the
`FluxSink` and iterate and poll downstream `PollableChannel` until
there is an item or `n > 0`.
Having `take(6)` we end up with the cancel from the downstream
`Subscriber` after the batch is filled, but at the same time we continue
to poll the upstream source because `n` is like `Long.MAX_VALUE`.

* The proper way to interact is check for the `!sink.isCancelled()`
as well.
This way cancelled `sink` won't "steal" data from other subscribers

Fix compatibility with the latest dependencies

* Upgrade to Gradle 4.1
* Upgrade as much dependencies as possible
* Fix MongoDB module to resolve deprecation in the latest driver
* Increase receive timeout in the `ResequencerTests`
* Restore generic argument for the method reference in the `ReactiveStreamsTests`
* Fix `WebFluxInboundEndpoint` for the compatibility with the latest Reactor API
2017-09-06 16:46:59 -04:00
Gary Russell
cfab1fb1a4 INT-4333: Don't Override Explicit Management Props
JIRA: https://jira.spring.io/browse/INT-4333

If the user has explicitly set management properties in a bean definition,
the configurer should not override those settings with defaults.
2017-09-01 15:48:11 -04:00
Gary Russell
3a2be8e9e0 INT-4336: Fix NPE in IntegrationGraphServer
JIRA: https://jira.spring.io/browse/INT-4336

Don't add "hidden" unreachable methods to the map in the `GatewayProxyFactoryBean`.
The methods can never be executed and may not have all expected properties; they
should not appear in the graph at all since they are unreachable.

Also add defensive coding to the `IntegrationGraphServer` to avoid NPEs when unexpected
conditions arise.
2017-09-01 15:44:55 -04:00
Artem Bilan
6c18cbcf57 INT-4326: Add endpoints.noAutoStartup property
JIRA: https://jira.spring.io/browse/INT-4326

To let to disable autoStartup for particular endpoint bean globally,
in one place, add `spring.integration.endpoints.noAutoStartup`
to the `spring.integration.properties`

So, now all the `AbstractEndpoint` checks that property for matching its
bean name to configure `autoStartup` property to `false`

**Cherry-pick to 4.3.x**
2017-09-01 15:14:47 -04:00
Gary Russell
34652f3342 INT-4332: Fix Test Race Condition 2017-08-23 09:52:52 -04:00
Oleg Zhurakousky
318741d1fd INT-4332 updated DSL with Supplier support
- add support for Supplier as sources of messages to the DSL
- add tests and javadocs
2017-08-22 17:08:30 -04:00
Oleg Zhurakousky
946cc229eb INT-4330: Add support for Supplier MessageSource
JIRA: https://jira.spring.io/browse/INT-4330

- updated documentation
- updated test
2017-08-17 12:01:03 -04:00
Artem Bilan
ab9e843cfa Fix Checkstyle violations
* Add `validateOnlyOverlapping = false` for `RequireThis` Checkstyle rule
* Add `this.` to caught places
* Use `CustomizableThreadFactory` instead of raw interface in the
`UnicastSendingMessageHandler`
2017-08-15 13:36:19 -04:00
Gary Russell
a1abae03a4 INT-4329: Fix JDBC [IMPORTANT] Block
JIRA: https://jira.spring.io/browse/INT-4329

Block wasn't closed.
2017-08-15 13:07:10 -04:00
Gary Russell
0a1306cd1b INT-4328: AMQP: Returns/Nacks: Create ErrorMessage
JIRA: https://jira.spring.io/browse/INT-4328

Add support for sending `ErrorMessage`s to the return and nack channels.

**cherry-pick to 4.3.x, but change default EMS to null (will require minor adjustment to test - set the EMS in `adapterWithReturnsAndErrorMessageStrategy`)**
2017-08-15 12:31:52 -04:00
Oleg Zhurakousky
d41d707357 Polishing ErrorMessageSendingRecoverer
- delegated creation of 'DefaultErrorMessageStrategy' to the
another constructor in cases where user provided null
- see https://github.com/spring-cloud/spring-cloud-stream/pull/1038#discussion_r132707905
for more details and justification

* Simple code style polishing

**Cherry-pick to 4.3.x**
2017-08-14 11:19:59 -04:00
Rick Hogge
9f4f07bcfd INT-4327: Optimize Bean Post Processors
JIRA: https://jira.spring.io/browse/INT-4327

To avoid reflection and annotation processing overhead on the bean classes
when we deal with non-singleton beans and there is really no any
messaging on the bean class, store bean classes to the local cache to skip
them in the future when `BPP` is applied for request/prototype beans

* Simple code style polishing

**Cherry-pick to 4.3.x**
2017-08-14 10:37:40 -04:00
Artem Bilan
6dad473219 Demonstrate error handling in the HttpDslTests
StackOverflow https://stackoverflow.com/questions/45627072/spring-integration-preserve-http-error-code-and-body-in-response
2017-08-11 13:45:51 -04:00
Artem Bilan
84d60f4ab4 INT-4315: Add WebFlux module
JIRA: https://jira.spring.io/browse/INT-4315

Move Reactive components outside of HTTP module to the new WebFlux one,
including XSD, tests and documentation
Make an appropriate polishing for the `http.adoc` with cross-link
to the `webflux.adoc`

Exclude transitive `spring-webmvc` for the `spring-integration-webflux`
2017-08-11 13:38:37 -04:00
Artem Bilan
4fd32d2bd4 Document max-rows-per-poll for the JDBC Gateway
StackOverflow https://stackoverflow.com/questions/45611037
2017-08-10 16:52:44 -04:00
Artem Bilan
a3160f80f6 INT-4318: Register Jackson modules if available
JIRA: https://jira.spring.io/browse/INT-4318

Do not `ObjectMApper.findAndRegisterModules()` unconditionally.
If there is no the target module library in classpath, we end up
with the `NoClassDefFoundError`

* Copy-paste the well-known modules registration logic from the
`org.springframework.http.converter.json.Jackson2ObjectMapperBuilder`
since we can't add `spring-web`  dependency to the `SI-core`
2017-08-08 13:49:50 -04:00
Artem Bilan
32fdd2119f INT-4324: File Outbound: parse preserve-timestamp
JIRA: https://jira.spring.io/browse/INT-4324

The `preserve-timestamp` attribute for the
`<int-file:outbound-channel-adapter>` has been missed for parsing in
the `FileWritingMessageHandlerBeanDefinitionBuilder` and propagation
in the `FileWritingMessageHandlerFactoryBean`

**Cherry-pick to 4.3.x**
2017-08-08 11:47:32 -04:00
Artem Bilan
2fe7289429 Redis DelayerHRescheduleIntTests: unique groupId
https://build.spring.io/browse/INT-AT42SIO-595/

To avoid clashes with other parallel builds on CI for different versions
on the same shared Redis make a testing `groupId` as unique using
`UUID.randomUUID()`

**Cherry-pick to 4.3.x & 4.2.x**
2017-08-07 12:21:20 -04:00
Artem Bilan
cb1784d978 INT-4325: Fix JpaExecutor bean registration
JIRA: https://jira.spring.io/browse/INT-4325

When channel adapter is defined without `id` attribute, the
`JpaInboundChannelAdapterParser` generates the name for class which
bean is never registered, therefore all the `JpaExecutor`s are registered
with the same bean name and afterward all the `JpaPollingChannelAdapter`
without `id` share the same `JpaExecutor`

* Generate bean name based on the `SourcePollingChannelAdapterFactoryBean`
which is used eventually for real bean registration
* Add test config for several channel adapters without `id`
* Optimize the `JpaInboundChannelAdapterParserTests` to start
the context only once during test class initialization

**Cherry-pick to 4.3.x**
2017-08-04 20:30:13 -04:00
Artem Bilan
5a259939a6 Reset AbstractMessageChannel.fullChannelName
Fixes spring-cloud/spring-cloud-stream#983

When the `AbstractMessageChannel.getFullChannelName()` is called
before the full bean initialization, we don't have the proper name any
more lately - that is only `unknown.channel.name`

* Reset `fullChannelName` property to `null` in the end of `onInit()`,
so the next `getFullChannelName()` will build the proper component name

**Cherry-pick to 4.3.x**
2017-08-04 13:19:46 -04:00
Artem Bilan
6aca63806f Upgrade to the latest SF, SS and Reactor
https://build.spring.io/browse/INT-FATS5IC-249
2017-08-03 19:23:35 -04:00
shawn-yang-1979
3fb902855e Update MqttMessageDeliveredEvent.java 2017-08-03 10:55:46 -04:00
Artem Bilan
f00eafb4f8 GH-152: Scanner & filters are mutually exclusive
Fix spring-projects/spring-integration-java-dsl#152

* Add Logic to the `FileInboundChannelAdapterSpec` do not populate
`FileListFilterFactoryBean` to the `FileReadingMessageSource`
when the `scanner` is provided.
Otherwise call the `FileReadingMessageSource` and rely on its logic to
refuse mutually exclusive `scanner` and `filter/locker` options
2017-07-31 10:52:29 -04:00
Artem Bilan
0f78d4b2ec Some JDBC tests polishing and optimization
**Cherry-pick to master**

Conflicts:
	build.gradle
	spring-integration-jdbc/src/test/java/org/springframework/integration/jdbc/store/channel/DataSource-derby-context.xml
2017-07-26 16:16:07 -04:00
Spring Buildmaster
3b03b6ac91 [artifactory-release] Next development version 2017-07-25 17:26:55 +00:00
Spring Buildmaster
5934001a62 [artifactory-release] Release version 5.0.0.M6 2017-07-25 17:26:49 +00:00
Artem Bilan
4b49c45137 Upgrade to SD Kay RC2 2017-07-25 12:50:22 -04:00
Artem Bilan
74b59ef8d3 INT-4316 Add ObjectToMapTransformer.mapper option
JIRA: https://jira.spring.io/browse/INT-4316

To allow to customize a JSON representation of the Object, provide a
`JsonObjectMapper`-based ctor for the `ObjectToMapTransformer`
2017-07-25 11:58:41 -04:00
Artem Bilan
784d891fa5 Fix Checkstyle violations 2017-07-24 14:05:53 -04:00
Artem Bilan
c26cac4c17 Upgrade to the released Milestones
* SF-5.0 RC3
* AMQP2.0 M5
* Security 5.0 M3
* Spring Twitter 2.0 M4
* Gradle 4.0.1
* Checkstyle 8.0
* JsonPath 2.3
* Fix deprecations and Checkstyle violations
2017-07-24 13:42:12 -04:00
Artem Bilan
5a66854c67 IntegrationFlow: optional @MessagingGateway
When we use `IntegrationFlows.from(Class<?>)`, we don't need to require
present for `@MessagingGateway`, we can just synthesize it with the
default properties

This way we can simply make gateway proxies for the `java.util.function`
interfaces which should be useful for Spring Cloud Function
2017-07-20 17:54:12 -04:00
Artem Bilan
d4a99919ed INT-4300: Add WebFlux Server Support
JIRA: https://jira.spring.io/browse/INT-4300

* Add `ReactiveHttpInboundEndpoint` based on the WebFlux foundation
* Extract `BaseHttpInboundEndpoint` for common
HTTP Inbound Channel Adapters options
* Make `spring-webmvc` and `spring-webflux` as `optional` dependencies
to let end-user to choose
* Refactor `HttpContextUtils` to include constants
for newly added WebFlux support
* Introduce `BaseHttpInboundEndpoint.setRequestPayloadTypeClass()`
for raw `Class<?>` and modify existing `setRequestPayloadType()`
for the `ResolvableType`
* Refactor existing MVC tests and XML components parsers to use
new `setRequestPayloadTypeClass()`
* Add `MessagingGatewaySupport.sendAndReceiveMessageReactive()` to get
a reply from downstream flow reactive back-pressure manner
* Add `IntegrationHandlerResultHandler` implementation to let WebFlux
infrastructure to handle the `Mono<Void>` from the `ReactiveHttpInboundEndpoint`
properly
* Fix `JdbcLockRegistryLeaderInitiatorTests` race condition to assert
the `initiator1` is elected eventually after yielding when the `initiator2`
is stopped

* Fix JavaDocs issue in the `HttpRequestHandlingMessagingGateway`
* Move all the "hard" logic in the `MessagingGatewaySupport#doSendAndReceiveMessageReactive`
to the `Mono` chain ensuring back-pressure when `sendAndReceiveMessageReactive()` is
called not from the Reactive Stream

Add test-case to demonstrate SSE

JIRA: https://jira.spring.io/browse/INT-3625

Some polishing and optimization for the
`MessagingGatewaySupport.doSendAndReceiveMessageReactive()`

More optimization for `MessagingGatewaySupport`

* Upgrade to Reactor 3.1 M3
* Document WebFlux-based components

Minor Doc Polishing
2017-07-20 12:19:17 -04:00
Artem Bilan
e475d9c695 INT-4308: Document JdbcMetadataStore
JIRA: https://jira.spring.io/browse/INT-4308

* Document `JdbcMetadataStore`
* Remove unnecessary context DERBY config for the `PersistentAcceptOnceFileListFilterExternalStoreTests`
* Rework the `testFileSystemWithJdbcMetadataStore()` to use `EmbeddedDatabase` directly, without context
2017-07-20 10:55:03 -04:00
Artem Bilan
f52fdd85ab Fix Gemfire tests for the latest SDG
https://build.spring.io/browse/INT-MASTER-JOB1-750
2017-07-19 17:40:51 -04:00
Artem Bilan
bccaa52bd2 Make testAll task depends on check; exclude test 2017-07-19 17:15:47 -04:00
Artem Bilan
7a67c1f1a3 Various polishing and optimizations
* Check for `null` package in the `JacksonJsonUtils`.
Also don't call `this.delegate.typeFromId()` the second time
* Increase wait timeout in the `QueueChannelTests`
* Check `ZookeeperMetadataStore.addListener()` for `null`

**Cherry-pick to 4.3.x**
2017-07-14 16:31:46 -04:00
Artem Bilan
748feea0f8 Upgrade to SD Kay B-S and fix MongoDB accordingly
https://build.spring.io/browse/INT-FATS5IC-229/
2017-07-14 11:11:50 -04:00
Artem Bilan
012bebd7bf Impl CoreSubscriber instead of Subscriber
According Reactor recommendation it would be better to `CoreSubscriber`
from Reactor instead of Reactive Streams `Subscriber` for chasing
some optimisation in case of Reactor flows

Increase `receive()` timeout in the `AttributePollingChannelAdapterParserTests`
2017-07-13 17:20:16 -04:00
Artem Bilan
6829da995f INT-4301: Make CircuitBreakerOpenException as ME
JIRA: https://jira.spring.io/browse/INT-4301

To keep the context of the `failedMessage` and avoid extra wrapping
to the `MessagingException` make
`CircuitBreakerOpenException extends MessagingException` directly
2017-07-12 18:46:55 -04:00
Artem Bilan
4663862ea6 Resolve deprecations since the latest SF 2017-07-11 11:52:11 -04:00
Kai Zimmermann
c0d4375b1e INT-4311: JdbcLocks: QueryTimeoutException retry
JIRA: https://jira.spring.io/browse/INT-4311

* Added `QueryTimeoutException` to retry handling in the `JdbcLockRegistry`

Polishing

**Cherry-pick to 4.3.x**
2017-07-11 11:33:47 -04:00
Artem Bilan
d79e70c6df GH-2197: Rely on the Document creation for Source
Resolves https://github.com/spring-projects/spring-integration/issues/2197

Sometime we would like to avoid some XML definition features (e.g. loading of the DTD).
The `XmlValidatingMessageSelector` doesn't allow to do that easily because it doesn't
rely on the `Document` creation via `DocumentBuilderFactory`, but directly
manipulates with `Source`

* Change `DefaultXmlPayloadConverter.convertToSource()` to fallback to the `convertToDocument()`
for non-Document payloads.
This way we can configure `DefaultXmlPayloadConverter` with custom `DocumentBuilderFactory` and
bypass DTD loading, for example
* Expose `xml-converter` option for the `<int-xml:validating-filter>`  component
2017-07-06 12:04:16 -04:00
Gary Russell
6c62af53f4 INT-4309 Http Header Mapper VarArgs Instead of []
JIRA: https://jira.spring.io/browse/INT-4309

More convenient for Java Config.

*Upgrade to Reactor 3.1 B-S, Reactor-Netty 0.7 B-S and Security 5.0 B-S.
Otherwise we are not compatible with the latest SF
2017-07-05 13:38:20 -04:00
Bojan Vukasovic
47003077cc INT-4306: Add support for JdbcMetadataStore
JIRA: https://jira.spring.io/browse/INT-4306

Fixed checkstyle errors

Fixed checkstyle errors

Fixed checkstyle errors and sql schema

Manually fixed derby sql schema

Fixing some issues

added region support

small refactorings

test checkstyle fix

integration tests and fixes

integration tests and fixes
2017-06-29 13:44:28 -04:00
Artem Bilan
54b0800d08 INT-4280: Add taskScheduler() to Java DSL
JIRA: https://jira.spring.io/browse/INT-4280

To let end-user easily to inject custom `TaskScheduler` to the endpoint
add `ConsumerEndpointSpec.taskScheduler()` option

* Address PR comments
2017-06-28 10:12:38 -04:00
Artem Bilan
72bd4e5bcf INT-4305: (S)FTP: Remove Local File Before Rename
JIRA: https://jira.spring.io/browse/INT-4305

The `File.renameTo()` operation may fail, therefore the content of the
local file isn't changed, but since we change `setLastModified()` anyway,
this file might be eligible for local polling.
So, we end up with the same content from local file in a new message,
meanwhile we expect a new content from the remote file

* Check the `File.renameTo()` result and attempt to `delete()`
for existing local file
* When file isn't renames remove the remote file from the `filter` to let
it be transferred one more time on the next poll.
The local file might be opened for processing, so this way we postpone a fresh
remote file for the future poll rounds
* Modify `copyFileToLocalDirectory()` to return `boolean` to reflect the fact of copy.
This way we check the real number of transferred files

**Cherry-pick to 4.3.x**

Do not transfer remote file if we can't remove local one

Polishing Log Messages
2017-06-27 16:45:15 -04:00
Gary Russell
36ac9f2203 (S)FTP MaxFetchSize Doc Polishing and DSL
Also polish programming tips and tricks docs.

* Fix typos in docs
2017-06-27 15:45:40 -04:00