Commit Graph

7325 Commits

Author SHA1 Message Date
Gary Russell
34683bd560 INT-3216 MGS Reaper Improvements
JIRA: https://jira.springsource.org/browse/INT-3216

Previously, After a group has been selected for completion
by the reaper, if the group is otherwise completed before
the reap AND expireGroupsOnCompletion is true AND a new
group with the same ID is created, ALL in the same millisecond,
AND the group creation timestamp is used for the reap criteria,
the new group will be incorrectly reaped.

Also, for a simple MGS, it was not necessary to re-fetch the group
if the group has already been marked complete.

- Defer reaping if the group is already complete (unless it's empty).
- Verify the timestamp hasn't changed (indicating a new group)

Add tests

- Verify no refetch of a complete SMG
- Verify no reap of a complete group (after refetch)
- Verify no reap when the timestamp changed
2013-11-24 17:53:20 -05:00
Gary Russell
ee6ad2c256 Fix Redis Test - Wait for Container to Listen
https://build.springsource.org/browse/INT-B30X-312/
2013-11-23 15:42:59 -05:00
Gary Russell
b61de881c9 INT-2932 Tcp Event More Doc Polishing
JIRA: https://jira.springsource.org/browse/INT-2932

Add source (the TcpConnetion) to the property list.
2013-11-23 15:02:04 -05:00
Artem Bilan
87798b055f INT-2491: Scripting: Variables Improvements
JIRA: https://jira.springsource.org/browse/INT-2491

* Add support variables for 'inline' scripts
* Add convenient attribute `variables` for script tags
* Introduce `BeanFactoryFallbackBinding` for groovy scripts.

INT-2491: Polishing

* Add check for duplication of provided variables
* Tests and Docs

Polishing
2013-11-22 11:57:26 -05:00
Artem Bilan
a7cabc53b0 INT-3055: HTTP Outbound uri-variables-expression
JIRA: https://jira.springsource.org/browse/INT-3055

* Add to HTTP Outbound Endpoint `uri-variables-expression`
* Add mutually exclusive check for `uri-variables-expression` with `<uri-variable>`
* Tests and Docs

INT-3055: Fixes

* `ExpressionEvalMap` now can apply `Map<String, Object>`,
but the value must have a `String` or `Expression` type.

Polishing

INT-3055 Doc Polishing
2013-11-21 15:32:27 -05:00
Gary Russell
8ab65c6dd3 INT-3214 Event Adapter Stop All Events
JIRA: https://jira.springsource.org/browse/INT-3214

Previously, ApplicationContextEvents were emitted even when the adapter
was stopped.

Do not emit ApplicationContextEvents unless they are stop or close events
and the adapter was stopped in the last 5 seconds.

An existing test case verifies the code (applicationContextEvents).

INT-3214 Polishing

Set default phase to Integer.MIN_VALUE + 1000 so the adapter
is started early, stopped late, while leaving headroom to
configure any downstream consumer to start before, stop after.
2013-11-21 21:37:16 +02:00
Artem Bilan
6a2583affd INT-2501: Set CI#autoCreate from INT properties
JIRA: https://jira.springsource.org/browse/INT-2501
2013-11-21 19:55:49 +02:00
Artem Bilan
d0501cda4d INT-3215: Redis: Squash ZSet tests to one
JIRA: https://jira.springsource.org/browse/INT-3215

Added fix for `RedisChannelParserTests#testPubSubChannelUsage()`
2013-11-21 13:49:28 +02:00
Gary Russell
2f0f676f48 INT-3213 Fix Package Tangle
JIRA: https://jira.springsource.org/browse/INT-3213

Backwards reference from ....support.channel to
....channel.registry (BeanFactoryChannelResolver).

- Move the registry interface to ....support.channel
- Move the implementation to ....channel
2013-11-20 22:04:15 -05:00
Gary Russell
c04cfc668a INT-3143 Doc Polishing
JIRA: JIRA: https://jira.springsource.org/browse/INT-3143

Also increase the timeout on an event test.
2013-11-20 17:47:35 -05:00
Artem Bilan
d2bb90f7e0 INT-3143: Add ConfigurableMongoDbMessageStore
JIRA: https://jira.springsource.org/browse/INT-3143

INT-3143: Polishing, fixed and documentation

JIRA: https://jira.springsource.org/browse/INT-3077
2013-11-20 17:23:47 -05:00
Artem Bilan
225c0b4c9f INT-3085: Use RedisProperties for RMDStore
JIRA: https://jira.springsource.org/browse/INT-3085
2013-11-20 16:30:13 -05:00
Artem Bilan
afb369c0eb INT-2500: Add Support for Global Properties
`META-INF/spring.integration.properties`

JIRA: https://jira.springsource.org/browse/INT-2500
2013-11-20 15:58:13 -05:00
Artem Bilan
9c42a65e3c INT-3159: JPA: Add max-results Alias Attribute
JIRA: https://jira.springsource.org/browse/INT-3159

* Deprecate `max-number-of-results`
* Tests and Docs

Doc Polishing
2013-11-20 15:47:47 -05:00
Artem Bilan
405fce672b INT-3212: Fix Failed Tests From CI Builds
JIRA: https://jira.springsource.org/browse/INT-3212

INT-3212 Fix Failing TCP Test

TcpNioConnections can deadlock when a fixed thread pool is being used.
The deadlock is detected after 60 seconds, but the test case failed
after 20 seconds - before the deadlock was detected.

Change the test to use a cached thread pool instead.

Also, increase concurrency by using threading in the test server
so each socket is handled on a separate thread.

Enchance connectionId to include both local and remote ports to
aid debugging.

Some minor formatting corrections.
2013-11-20 15:10:22 -05:00
Artem Bilan
6b50c49efe INT-3210: JDBC & JPA: Optimize SpEL Parsing
JIRA: https://jira.springsource.org/browse/INT-3210

* Parse `Expression` parameters only once at creation phase
* There is still tiny case, when configured parameters don't have
provided parameter, in this case an `input` determines as JavaBean and the
`name` will be populated as bean property via SpEL.

INT-3210: Add params to maps and collections

* If configured parameters don't contain provide param,
parse his expressions and add to parameters of current
`ParameterSource` and enclosing `ParameterSourceFactory`
* Move JPA `ExpressionEvaluatingParameterSource` to inner class
of `ExpressionEvaluatingParameterSourceFactory`, similar to `ExpressionEvaluatingSqlParameterSourceFactory`
2013-11-20 09:46:21 -05:00
Gary Russell
5bb6e550aa INT-3211 TCP Interceptor Events; Test Polishing
Intercepted TCP connections did not publish events.

TcpReceivingChannelAdapterTests did not close server factories after
tests.

Suppress 'no publisher' log messages.
2013-11-20 11:42:02 +02:00
Artem Bilan
945cb14407 INT-3199: Fix Inconsistency for Candidate Methods
JIRA: https://jira.springsource.org/browse/INT-3199

* Set `TypeDescriptor` to `Void.class` as default `targetParameterType` for falling-back
* Change `fall-thru` only to one method:
* Extract generic type from `Massage` parameter, so now
``` m1(Message<String> message);
    m2(String payload);
```
are equal by `payload type` and similar POJO causes ambiguity configuration exception

INT-3199: Polishing and further fixing

* Address PR's comments
* Provide more friendly exception message for `Void.class` method
* Remove the logic around `@ServiceActivator`:
 - one part was fixed within INT-3114
 - another for `RequestReplyExchanger#exchange` doesn't make sense as it is good candidate
* Add fallback to `Iterator.class` method for `Iterable` payloads
* Add tests

Polishing

INT-3199: Add `handlerMessageMethods`

* Introduce separate properties:
 - `handlerMessageMethods` for methods with `Message` parameter
 - `handlerMethod`, if there is only one candidate after configuration
* Polishing several tests according new state of `MessagingMethodInvokerHelper`'s properties

Polishing

Revert `RequestReplyExchanger` fallback

Add `fallbackMessageMethods` processing

INT-3199 Polishing

Only revert to RequestReplyExchanger if there are ambiguous
fallback methods - if the target object is an RRE, it will always
have a candidate fallback (exchange).

Add more RRE tests.

Make ParametersWrapper a public inner class - it's only used
within the context of a process() call. Making it public allows
SpEL to access its properties without reflection.
2013-11-19 17:27:56 -05:00
Gary Russell
703b24541f INT-3209 DefaultFileNameGenerator - SpEL Parsing
JIRA: https://jira.springsource.org/browse/INT-3209

Previously, the DefaultFileNameGenerator re-parsed the
SpEL expression for every message.

Compile the SpEL once (and if it is changed).

Change unit tests to match changes.
2013-11-19 15:24:26 +02:00
Gary Russell
a7d4d57090 INT-2997 Document Endpoint Inner Bean Names
JIRA: https://jira.springsource.org/browse/INT-2997

.handler and .source
2013-11-19 13:23:54 +02:00
Gary Russell
adc20970d9 INT-2870/INT-2714 Document IP MessageHeaders
JIRA: https://jira.springsource.org/browse/INT-2870
JIRA: https://jira.springsource.org/browse/INT-2714

INT-2870: Polishing IP Headers Table for PDF
2013-11-19 13:21:14 +02:00
Gary Russell
59d6f7dfc1 INT-3088 (S)FTP Outbound Gateway - PUT and MPUT
- Core support in file module
- FTP Parser and Test

https://jira.springsource.org/browse/INT-3088

INT-3088 Add SFTP Support for PUT, MPUT

INT-3088 Polishing - PR Comments

INT-3088 Docbook For PUT/MPUT
2013-11-19 12:56:48 +02:00
Artem Bilan
88de8117aa INT-3140: Add #xpath() SpEL Function Support
JIRA: https://jira.springsource.org/browse/INT-3140

* Introduce `XPathUtils`
* Add `#xpath()` tests
* Add documentation

INT-3140: Polishing according PR comments

INT-3140 Polishing

- Doc and javadoc polishing
- Fix some compiler warnings (not all xpath)
2013-11-15 15:54:14 -05:00
Gary Russell
fa59b50bdc Fix Failing Test
Producer starts before consumer (same phase) with a fast poller.

Add negative phase to the consumer so he starts first.
2013-11-15 09:27:50 -05:00
Gary Russell
836c8e2556 INT-2269 Add ReplyChannelRegistry
Allows reply channel resolution after a message has been serialized
somewhere in a flow. Previously, the reply channel was lost.

With this change, the reply channel can be registered and the header
becomes a string (channel name) that can be serialized.

JIRA: https://jira.springsource.org/browse/INT-2269

INT-2269 Rename Registry to HeaderChannelRegistry

- Extract interface
- DefaultHeaderChannelRgistry

INT-2269 Polishing; PR Comments

- Add errorChannel registration as well.

INT-2269 HeaderChannelRegistry - Fix

The internal BridgeHandler in MessagingGatewaySupport did not have
a channel resolver. When a reply was explicitly routed to the gateway's
reply channel, the String representation of the reply channel could
not be resolved to a channel.

Set the BeanFactory on the bridge handler.

Add tests.

INT-2269 HeaderChannelRegistry - Fix JMS/Enricher

The JMS inbound gateway and ContentEnricher instantiate a
MessagingGatewaySupport internally it does not get a
reference to the BeanFactory. This means that the internal
BridgeHandler cannot resolve the String representation of
the reply channel to a channel.

Make ChannelPublishingJmsMessageListener BeanFactory aware, and
propagate the bean factory to the MGS.

Set the MGS bean factory in the ContentEnricher (which is already
BFA).

INT-2269: Polishing
2013-11-15 15:23:05 +02:00
Gary Russell
1d0c28852f INT-3204 RemoteFileTemplate Phase II Part 1
- AbstractInboundFileSynchronizer

INT-3204 RemoteFileTemplate Phase II, Part 2

- AbstractRemoteFileOutboundGateway
- Encapsulate Session.readRaw and .finalizeRaw in template.get()

INT-3204 Polishing - PR Comments

JIRA: https://jira.springsource.org/browse/INT-3204

INT-3204: Polishing
2013-11-15 15:13:57 +02:00
Gary Russell
d662e7ee42 INT-3203 Introduce RemoteFileTemplate
Begin by refactoring `FileTransferringMessageHandler` to use
a `send` operation on RemoteFileTemplate.

Effectively transfer all properties from the adapter to the
template.

This will allow reuse of the send logic from the adapter in
the outbound gateway for `put` and `mput`.

JIRA: https://jira.springsource.org/browse/INT-3203
2013-11-11 17:53:50 -05:00
Gary Russell
72dd39bccf INT-2898 Add Persistent FileListFilters
- Abstract implementation
- Implementation for the file system
- Implementation for remote files ((S)FTP)

JIRA: https://jira.springsource.org/browse/INT-2898

INT-2889 Polishing

- Add (S)FTP test cases
- Docs
2013-11-07 17:31:45 -05:00
Gary Russell
7bc4fe2d0b INT-3047 SFTP Multiplex Over Single Connection
Enable the use of multiple JSch channels over a
single connection.

JIRA: https://jira.springsource.org/browse/INT-3047

INT-3047: Polishing locks logic

INT-3047 SFTP Reset Cache; Quiesce Shared Sessions

Support resetCache() on the CachingSessionFactory
- reset a shared JSch session so it is reestablished on next use
- immediately close idle sessions
- close in-use sessions as they are returned
- close the channel when an SftpSession is closed and a shared JSch is being used
- physically close a shared JSch session only when the last channel is closed

Polishing

Forgot to save the ftp.xml.

Change CachedSession.epoch to created.

INT-3047 Polishing - PR Comments

- Also add missing docs for `preserve-timestamp`

INT-3047 Polish - PR Comments

Also change epoch to nanoseconds and use a simple != comparison with the
epoch in which a session was created.
2013-11-07 21:10:12 +02:00
Gary Russell
e128c80c19 INT-2421 Support 'container-class' on JMS MDCA
Convenience for configuring a custom listener container - avoids
having to transfer many attributes from the adapter to the
bean when switching to a custom implementation.

JIRA: https://jira.springsource.org/browse/INT-2421

INT-2421 Polishing - PR Comments

Also auto-create adapter channel if necessary.
2013-11-07 20:39:54 +02:00
Gary Russell
f11f00b5ab INT-3100 Support Remote File Streaming (Inbound)
- Add readRaw and finalizeRaw methods to Session
- Add tests for SFTP and FTP

Allows retrieval of a remote file as a stream.

JIRA: https://jira.springsource.org/browse/INT-3100
2013-11-07 17:14:46 +02:00
Artem Bilan
b1b78409c8 INT-2946: Add redis:i-c-a topic-patterns Attr.
JIRA: https://jira.springsource.org/browse/INT-2946

Polishing
2013-11-06 14:56:47 -05:00
Artem Bilan
93bcfdbc55 INT-3198: Add 'recovery-interval' for redis-q-c-a
JIRA: https://jira.springsource.org/browse/INT-3198

* Expose `RedisQueueMessageDrivenEndpoint.recoveryInterval` to Namespace support
* Add parser's tests
* Rename parser test class
* Polishing for concurrency of`RedisQueueMessageDrivenEndpointTests#testInt3196Recovery` test
* Add documentation for 'recovery-interval' and Redis Events implementation

Doc Polishing

Add more delay because of unsolvable race condition in the Redis Subscription - see DATAREDIS-242
2013-11-06 14:31:27 -05:00
Gary Russell
f169555224 INT-3017 Redis TaskExecutor Changes
Don't wrap the TE in an EHTE if there is no BeanFactory.
2013-11-05 18:34:16 -05:00
Artem Bilan
4cf0e2e673 INT-3196: RedisQueueMDE: Add RecoveryInterval
Catch `RedisSystemException` in the `ListenerTask` and sleep current Thread
 with `recoveryInterval` if the Endpoint is `active` before the next `restart()`

JIRA: https://jira.springsource.org/browse/INT-3196

RedisConnFailure & RedisSys Exceptions recovery

Introduce `RedisIntegrationEvent`s

* Catch all `Exception`s on `this.boundListOperations.rightPop`
* Log them and send within `RedisExceptionEvent`
* Mark some fields with `volatile`
2013-11-05 14:06:35 -05:00
Artem Bilan
3470e33069 INT-3177: Generic RedisTemplate for PublishingMH
JIRA: https://jira.springsource.org/browse/INT-3177

* Change `RedisPublishingMessageHandler.StringRedisTemplate` to `RedisTemplate<?, ?>`
* Don't provide `serializer` to the `template`
* Use `serializer` directly for values which are not `byte[]`
* Add `RedisAvailableTests#awaitContainerSubscribed` for tests to avoid race conditions
* Refactor some tests
* Add test for `byte[]` payload

INT-3177: Generic RedisInboundChannelAdapter

Port from Spring XD: allow for `RedisInboundChannelAdapter`
to receive from Redis any object, not only String

* Make `serializer` property 'resettable' to `null`
* Change `RedisInboundChannelAdapterParser` to allow to apply
empty value from `serializer` attribute
* Add parser test and test for `byte[]` messages

INT-3177: Polishing and documentation

JIRA: https://jira.springsource.org/browse/INT-3033

* Add `topic-expression` to `<int-redis:outbound-channel-adapter>`
* Add tests and docs

INT-3177: Deprecate `RedisPublishMH.defaultTopic`

* Make `topic` and `topic-expression` attributes as mutually exclusive
* Polishing Redis Topic Parsers
* Polishing tests and docs
* Default `serializer` for Redis Topic Adapters is `StringRedisSerializer` for backward compatibility

Polishing according PR's discussion

INT-3177 Doc Polishing
2013-11-05 12:40:19 -05:00
Artem Bilan
156eeeb738 INT-3167: Polishing Twitter Documentation
JIRA: https://jira.springsource.org/browse/INT-3167

Doc Polishing
2013-11-05 10:23:07 -05:00
Gary Russell
970852741d INT-3154/INT-3137 JavaDoc Formatting
INT-3154 Use SPR Java7 stylesheet to put a box around `<pre class="code"/>`
code.

Update `<pre/>` tags to include class="code".

INT-3137 Remove NOTICE from `JdbcChannelMessageStore`.

JIRA: https://jira.springsource.org/browse/INT-3137
      https://jira.springsource.org/browse/INT-3154
2013-11-05 14:59:27 +02:00
Gary Russell
87988b5032 INT-3064 Message ID Generation
Use the SPR 4.0.0 Message ID Generation algorithm.

JIRA: https://jira.springsource.org/browse/INT-3064
2013-11-04 18:45:21 -05:00
Gary Russell
68f3990611 INT-3017 Fix Typo 2013-11-04 17:08:53 -05:00
Artem Bilan
ce1f467793 INT-3017: Redis Queue Adapters: Namespace Support
JIRA: https://jira.springsource.org/browse/INT-3017

* Add parsers tests and integration tests
* Polishing Redis XSD
* Fix `JmsMessageDrivenEndpointParser` `LifeCycle` attributes
* Catch `RedisSystemException` after `this.boundListOperations.rightPop`.
It maybe an exception about 'connection closed'.
If the `RedisQueueMessageDrivenEndpoint` is 'active' this exception is rethrown,
otherwise just logged under error category.
* Add Redis queue components docs

JIRA: https://jira.springsource.org/browse/INT-3019

INT-3017 Doc Polishing
2013-11-04 17:05:13 -05:00
Artem Bilan
5be8ef3fd8 INT-3147: (3167,3173,1941) Improve MetadataStore
Previously, `MetadataStore` couldn't be configured for Twitter Adapters
- only a global one could be used.
The `metadataKey` was generated automatically with a 'difficult' value.

* Register all `MessageSource` for `SourcePollingChannelAdapter`
as beans with id based on adapter id and prefix '.source' (INT-3147)
* Polishing parser to get rid of explicit `MessageSource` beans. (INT-3147)
* Make Feed and Twitter adapters `id` attribute as required -
now it presents a `metadataKey` for `MetadataStore` (INT-3147)
* Add to Twitter adapters a reference attribute for `MetadataStore` (INT-3173)
* Add Twitter adapters `poll-skip-period` attribute (INT-3167)
* Add and implement `MetadataStore#remove` (INT-1941)
* Make `MetadataStore` as `@ManagedResource` (INT-1941)
* Polishing tests

JIRAs:
https://jira.springsource.org/browse/INT-3147
https://jira.springsource.org/browse/INT-3167
https://jira.springsource.org/browse/INT-3173
https://jira.springsource.org/browse/INT-1941

INT-3147: Polishing and fixes

* add domain suffix to `metadataKey`
* change contract of `MetadataStore.remove`
* remove timeout window from `AbstractTwitterMessageSource`
* polishing and fix `SearchReceivingMessageSourceWithRedisTests`

INT-3147: Rebasing and polishing

INT-3147: fix 'metadata' package tangle

INT-3147 Doc Polishing
2013-11-04 12:44:12 -05:00
Gary Russell
1fb838dd1a INT-3190 Fix Package Tangle (JsonHeaders)
https://jira.springsource.org/browse/INT-3190
2013-11-04 17:34:45 +02:00
Mauro Franceschini
cd3f13ac91 INT-3193: Add ObjectToMapTransformer flatten attr
JIRA: https://jira.springsource.org/browse/INT-3193
2013-11-04 10:52:05 +02:00
Artem Bilan
4dd95c41ea INT-2809: Add/Extract JSON JavaType Headers
* Add `AmqpHeaders` headers
* Add `JavaType` headers mapping to `DefaultAmqpHeaderMapper`
* Add ability to Json Transformers to add/extract `JavaType` headers
* Make `JsonToObjectTransformer#targetClass` as non-required and do fallback
to extract type from `JavaType` headers
* Add `JavaType` extraction in the `JsonObjectMapper` implementors

JIRA: https://jira.springsource.org/browse/INT-2809

INT-2809: Polishing and refactoring

* Introduce `JsonHeaders`, `AbstractJacksonJsonObjectMapper`
* Move `TestPerson` and `TestAddress` to package level
* Now `JsonToObjectTransformer` supports not only `String` payload
* Remove json headers after transformation in the `JsonToObjectTransformer`

INT-2809 Minor Polishing

INT-2809 Docs and What's New
2013-11-01 16:22:03 -04:00
Artem Bilan
c389604c2e INT-2930: Mention correct SI version in Docs
JIRA: https://jira.springsource.org/browse/INT-2930
2013-11-01 13:45:13 -04:00
Artem Bilan
213cafb2da INT-3014: Introduce Redis Queue Adapters
https://jira.springsource.org/browse/INT-3014
https://jira.springsource.org/browse/INT-3015

* Introduce `RedisQueueMessageDrivenEndpoint` and `RedisQueueOutboundChannelAdapter`
* add Tests for them
* Remove `ChannelRegistry` artifacts
2013-11-01 11:00:11 -04:00
Gary Russell
9849e2087d INT-2628 Add ObjectNamingStrategy to IMBE
Allow customization of the generated ObjectName for the
Integration components.

JIRA: https://jira.springsource.org/browse/INT-2628

INT-2628 Polishing

- Address PR comments
- Rename `naming-strategy` to `object-naming-strategy` for consistency with `object-name-static-properties`
2013-11-01 16:47:03 +02:00
Gary Russell
01ab48d664 INT-2953 Use Buffered In/Out Streams for Files
Wrap `FileInputStream` and `FileOutputStreams` with
`BufferedInputStream`, `BufferedOutputStream` respectively.

Also use `BufferedReader`, `BufferedWriter` to wrap `FileReader`, `FileWriter`.

JIRA: https://jira.springsource.org/browse/INT-2953
2013-11-01 15:11:26 +02:00
Gary Russell
91f3674dc8 INT-3175 Add Caution About Apache HTTP Client
JIRA: https://jira.springsource.org/browse/INT-3175
2013-11-01 12:29:15 +02:00