Commit Graph

336 Commits

Author SHA1 Message Date
Yi EungJun
a9f50d32e4 Add InputStream to FileWritingMH JavaDoc
InputStream has been supported since 247232bdde.
2015-09-22 11:55:19 -04:00
Artem Bilan
d8fc94602c INT-3820: Fix MethodInvokerHelper for the Proxy
JIRA: https://jira.spring.io/browse/INT-3820

Some Proxy may be based on the specif non-user classes and advised with
the provided end-user interfaces.
The best sample is `@Repository` from Spring Data projects.
In this case the `MessagingMethodInvokerHelper` just missed the user interfaces
to consider for the candidate methods or thrown an `Exception` like `NoSuchMethodError`.

* Add `((Advised) targetObject).getProxiedInterfaces()` for the scanning algorithm
* Move the `UniqueMethodFilter` to the internal `Set<Method>` to allow iteration
and filtering for methods from the `targetClass` as well as from all those user interfaces
* Add Spring JPA repository test-case
* Polishing form some time-weak tests

Revert MessagingMethodInvokerHelper

INT-3820: Simple Proxy Interface Method Matching

No-risk solution for INT-3820.

Instead of considering all interfaces on the proxy, only look at proxy interfaces
for a single matching method name if we find no candidate or fallback methods.

Consider the complete solution for 4.3.

Add `Message<?>` method distinguishing
2015-09-09 14:33:38 -04:00
Artem Bilan
f607f83bf2 INT-3804: WatchServiceDirScan: Add OVERFLOW Event
JIRA: https://jira.spring.io/browse/INT-3804

INT-3804: Polishing

Fix race conditions.
2015-09-04 17:07:46 -04:00
Artem Bilan
057c004e6c INT-3619: Prevent scanner Options Overrides
JIRA: https://jira.spring.io/browse/INT-3619

Previously the `FileReadingMessageSource` allowed overriding `filter` and `locker` properties
on the `scanner` through its own setters.
It caused confusing and unexpected behavior.

Provide refactoring to check that provided `filter` and `locker` are applied for an internal `scanner`
instance, not for the injected one.

PR Comments
2015-08-12 16:59:35 -04:00
Artem Bilan
e408331e67 INT-3622: (S)FTP: Add MessageSessionCallback
JIRA: https://jira.spring.io/browse/INT-3622

INT-3622: Polishing
2015-08-11 09:33:34 -04:00
Artem Bilan
0f689558ab INT-3792: Fix Some SONAR Vulnerabilities
JIRA: https://jira.spring.io/browse/INT-3792

* Remove `Serializable` requirement for `ThreadStatePropagationChannelInterceptor` generic type
* Increase `PriorityChannelTests` timeout
2015-08-05 14:20:08 -04:00
Artem Bilan
ec12b289c2 INT-3786: Fix Several Sporadic Test Failures
JIRA: https://jira.spring.io/browse/INT-3786

* Increase `receiveTimeout` for several `QueueChannel` and `CountDownLatch` based tests
* Fix `FileSplitterTests` for Windows compatibility - `UTF-8` for bytes conversion
* Make some STOMP tests as `LongRunning`

The next fixing phase
2015-07-30 15:28:02 -04:00
Gary Russell
13b4facc7a INT-3491: (S)FTP Delegation Session Factory
JIRA: https://jira.spring.io/browse/INT-3491

Select the session factory at runtime.

Polishing; PR Comments

Also add/remove factories.
2015-07-30 12:39:55 -04:00
Gary Russell
9e0b2fb319 INT-3593: (S)FTP OG Partial Updates (mget/mput)
JIRA: https://jira.spring.io/browse/INT-3593

Throw a `PartialSuccessException` if an exception occurs after partial success (some files transferred).
2015-07-20 12:10:33 -04:00
Gary Russell
00540cb5df INT-3229: Fix WatchService Test
JIRA: https://jira.spring.io/browse/INT-3229
2015-07-16 15:09:59 -04:00
Gary Russell
1dba450e8c INT-3773: (S)FTP RemoteFileTemplate Fail/Ignore
JIRA: https://jira.spring.io/browse/INT-3773

When the `FileExistsMode` was `FAIL` or `IGNORE` and the remote file
did NOT exist, the file was not written.
2015-07-15 13:44:00 -04:00
Gary Russell
c986206059 INT-3229: Add WatchServiceDirectoryScanner
JIRA: https://jira.spring.io/browse/INT-3229

Polishing
2015-07-15 10:54:59 -04:00
Gary Russell
583d432f22 INT-3764: LastModifiedFileListFilter
JIRA: https://jira.spring.io/browse/INT-3764

* Add JavaDocs to the `LastModifiedFileListFilter`
* Add `Namespace Support` to the `file.adoc`
2015-07-15 09:54:17 -04:00
Gary Russell
c68c7b3412 INT-3763: (S)FTP - Inbound Remote Dir. Expression
JIRA: https://jira.spring.io/browse/INT-3763

Fix `AbstractRemoteFileSynchronizerTests` according to this change.
2015-07-09 18:26:46 -04:00
Gary Russell
cef99f3584 INT-2507: File: Fix HeadDirectoryScanner
JIRA: https://jira.spring.io/browse/INT-2507

When using a `HeadDirectoryScanner`, the use of any filter removed the
size limitation passed into the contstructor (or `queue-size` attribute
of a file inbound channel adapter.

The `HeadFilter` was overwritten.

Combine the head filter into a `CompositeFileListFilter` (creating one
if necessary).
2015-06-30 12:13:55 -04:00
Artem Bilan
9c96a14cd1 INT-3749: Revert setters for EC
JIRA: https://jira.spring.io/browse/INT-3749

Since some target application could inject there is `EvaluationContext` before, removal of
 `setIntegrationEvaluationContext()` is breaking change.

 This fix reverts all those setters.
2015-06-27 12:36:33 -04:00
Artem Bilan
3392d4e1ab INT-3749: Get rid of IECA logic
JIRA: https://jira.spring.io/browse/INT-3749

Since `IntegrationEvaluationContextAware` isn't so "context-free" resource like `BeanFactory`
 and `ApplicationContext`, but just a specific bean in the context, we can't follow with `BeanPostProcessor` logic - bad architecture by level of responsibility.
 Therefore we should follow with standard Dependency Injection mechanism to retrieve `evaluationContext` for the particular component.

 Since we can't rely on the `@Autowired` because SI can be used from the raw XML configuration,
 we use utility method instead. The pattern to get the proper `integrationEvaluationContext` is:
 ```
 @Override
 protected void onInit() throws Exception {
 		super.onInit();
 		this.evaluationContext = ExpressionUtils.createStandardEvaluationContext(getBeanFactory());
 }
 ```
2015-06-24 14:57:00 -04:00
Gary Russell
f4104c199a INT-3745: More TCP Events (Failed Correlation)
JIRA: https://jira.spring.io/browse/INT-3745

Emit events whenever a message is received that can't be correlated to
a connection (or request in the case of an outbound gateway).

Also, when sending messages, wrap the exception in a `MessagingException` so that
the `TcpConnectionExceptionEvent` provides access to the failed message.

INT-3745: Polishing - PR Comments

Fix JavaDocs errors
2015-06-19 13:49:10 -04:00
Gary Russell
65d2024937 INT-3737: (S)FTP Outbound Gateway Streaming GET
JIRA: https://jira.spring.io/browse/INT-3737

Support returning an `InputStream` from a GET operation.

This can be used in conjuction with a `<file:splitter/>` to stream a text file.

The user is responsible for releasing the session when the download is complete.

The session object is stored in the message header and `RemoteFileUtils.closeSession()` should be called
to clean up and close the session.

INT-3737: Polishing and Docs

Fix `CachedSession.close()` bug

Preventing `Caused by: java.io.IOException: Previous raw read was not finalized`
when `Session` is returned to the pool, but `readingRaw` hasn't been finalized
because the real `close()` isn't invoked in case of normal return to pool.
2015-06-19 13:39:36 -04:00
Gary Russell
3d5f7db4b2 INT-4740: FileSplitter - Add Headers
JIRA: https://jira.spring.io/browse/INT-3740

For `File` and `String` payloads add `FileHeaders.ORIGINAL_FILE` and `FileHeaders.FILENAME` headers.

Reworked to create headers once only.

Fix typos and Java > 6  API usage
2015-06-18 13:55:52 -04:00
Gary Russell
6436aa6d32 INT-3600: FileSplitter Namespace and Docs
JIRA: https://jira.spring.io/browse/INT-3600

Add `<int-file:splitter/>` namespace component.
Add `FileSplitter` documentation.

Polishing
2015-06-18 13:45:16 -04:00
Gunnar Hillert
247232bdde INT-3659: InputStream support in FileWritingMH
https://jira.spring.io/browse/INT-3659

Add `InputStream` `payload` handling to the `FileWritingMessageHandler`
2015-06-17 20:57:37 -04:00
Gary Russell
da54a1262d INT-3739: FileSplitter - Add LineCount to EOF Mark
JIRA: https://jira.spring.io/browse/INT-3739

Since iterating splitters don't have a sequence size, it would be useful to
include a line count in the EOF marker, to facilitate some downstream process
knowing that all lines have been processed, in a multi-threaded situation.
2015-06-17 17:59:00 -04:00
Gary Russell
c278548216 INT-3738: Fix FileSplitter
JIRA: https://jira.spring.io/browse/INT-3738

Don't rely on `BufferedReader.ready()` to detect EOF.

Polishing FileSplitter Fix
2015-06-17 17:21:48 -04:00
Gary Russell
51b52ea3a6 INT-3727: FileSplitter - add SOF/EOF Capability
JIRA: https://jira.spring.io/browse/INT-3727

Add a configuration option such that the `FileSplitter` can emit
Start/End messages when splitting.

Polishing according PR comments

Add toString to FileMarker
2015-06-08 13:52:48 -04:00
Gunnar Hillert
a0160e50d4 INT-3718: Add ignore-hidden to int-file:i-c-a
JIRA: https://jira.spring.io/browse/INT-3718

* If not set to `false` explicitly, add `IgnoreHiddenFileListFilter` as a default in `FileListFilterFactoryBean`
* Update XML Schema XSD and add `ignore-hidden` attribute
* Update + Add tests
* Add documentation
* Add default `IgnoreHiddenFileListFilter` also to `DefaultDirectoryScanner`
2015-05-25 22:08:12 -04:00
Gary Russell
5b0b3fc159 INT-3721: Add Flush Support to Persistent Filters
JIRA: https://jira.spring.io/browse/INT-3721

Support flushing metadata after each update.

Polishing JavaDocs
2015-05-25 21:13:41 -04:00
Gunnar Hillert
510bea55cd INT-3716: Add IgnoreHiddenFileListFilter
JIRA: https://jira.spring.io/browse/INT-3716

**Cherry-pick to 4.1.x**
2015-05-19 20:26:25 +03:00
Gary Russell
123b38a008 INT-3706: Support FileExistModes in (S)FTP Gateway
JIRA: https://jira.spring.io/browse/INT-3706

Previously, the `mode` attribute on the (S)FTP outbound gateways was ignored.

Take account of the mode on (M)GET and (M)PUT commands.
2015-05-12 13:21:26 +03:00
Gary Russell
15150ac499 INT-3699: RemoteFileTemplate Javadocs
JIRA: https://jira.spring.io/browse/INT-3699

Also javadocs for related classes.

INT-3699: Polishing - PR Comments
2015-04-15 12:49:41 +03:00
Gavin Gray
0d4c839331 INT-3684: Rebuild Tail Command on Restart
JIRA: https://jira.spring.io/browse/INT-3684
2015-03-31 11:21:28 +01:00
Artem Bilan
2bde14b742 INT-3661: Fix the eager BF access from BPPs (P I)
JIRA: https://jira.spring.io/browse/INT-3661

Previously there were a lot of noise from the `PostProcessorRegistrationDelegate$BeanPostProcessorChecker` for early access for beans.
That may produce some side-effects when some of `BeanFactoryPostProcessor`s won't adjust those beans.

The issue is based on two facts:
1. Loading beans from `BPP`, e.g. `IntegrationEvaluationContextAwareBeanPostProcessor` (or `ChannelSecurityInterceptorBeanPostProcessor` - https://jira.spring.io/browse/INT-3663)
2. Loading beans from `setBeanFactory()/setApplicationContext()` container methods

* Move all code from `setBeanFactory()` with access to the `BeanFactory` (e.g. `this.messageBuilderFactory = IntegrationUtils.getMessageBuilderFactory(this.beanFactory);`)
to some other lazy-load methods like `getMessageBuilderFactory()`
* Fix parser tests to remove `messageBuilderFactory` tests since there is no activity for target components to lazy-load them
* Polish some test according the new lazy-load logic
* Rework `IntegrationEvaluationContextAwareBeanPostProcessor` to the `SmartInitializingSingleton` and make it `Ordered`
* Populate `beanFactory` for the internal instance of `connectionFactory` in the `TcpSyslogReceivingChannelAdapter`
* Populate `beanFactory` for the internal `UnicastReceivingChannelAdapter` in the `UdpSyslogReceivingChannelAdapter`
* Add `log.info` that `UdpSyslogReceivingChannelAdapter` overrides `outputChannel` for the provided `UnicastReceivingChannelAdapter`
* Change the internal `MessageChannel` in the `UdpSyslogReceivingChannelAdapter` to the `FixedSubscriberChannel` for better performance

* Fix `AbstractExpressionEvaluator`
* Add JavaDocs for the `IntegrationEvaluationContextAware`

Fix `MongoDbMessageStoreClaimCheckIntegrationTests`

Addressing PR comments
2015-03-02 18:23:44 -05:00
Gary Russell
80a45715ac INT-3628: Use SchedulingAwareRunner for Long Tasks
JIRA: https://jira.spring.io/browse/INT-3628

Certain tasks in Spring Integration are long running. Indicate
so for the benefit of platforms such as WebLogic that can log warnings
for long running tasks otherwise.

Also fix conflicting versions of commons-io.
2015-02-28 15:16:07 +02:00
Artem Bilan
f248394682 INT-3589: Upgrade Dependencies
JIRA: https://jira.spring.io/browse/INT-3589, https://jira.spring.io/browse/INT-3624

* SF - 4.2. Fix Breaking changes for the `ApplicationEventPublisher`
* AMQP 1.5. Without issues
* Reactor - 2.0. Fix for the new `Stream` foundation
* jsonPath - 1.2.0. Fix for new `Predicate` abstraction. Add 'fail-fast error' to the `IntegrationRegistrar`
* Sshd - 0.13.0. Fix for new `VirtualFileSystemFactory` usage
* Spring Data - Fowler
* And others without issues
* Get rid of `reactor.util.StringUtils` usage

INT-3589: Address PR comments

Polishing
2015-02-17 15:09:09 -05:00
Gary Russell
195ee70568 INT-3632: Sonar Improvements
JIRA: https://jira.spring.io/browse/INT-3632

- Use Map.entrySet()
- Double check locking only works when the field(s) are volatile
- Remove unnecessary instanceof tests
2015-02-10 18:00:07 -05:00
Tony Falabella
a7d325adc1 INT-3621: Add Namespace support for appendNewLine
JIRA: https://jira.springsource.org/browse/INT-3621

Polishing for code formatting and simple polishing to Docs
2015-02-05 15:38:35 +02:00
Tony Falabella
24beee9627 INT-3620: add appendNewLine to the FileWritingMH
JIRA: https://jira.spring.io/browse/INT-3620

Add `setShouldAppendNewLine(boolean shouldAppendNewLine)` method to the
`FileWritingMessageHandler` class.  Added similar test methods to the
`FileWritingMessageHandlerTests` class.

INT-3620: `appendNewLine` just only for `FileExistsMode.APPEND`

Also add `appendNewLine` for `File` payload

INT-3620: `appendNewLine` for any `fileExistsMode`
2015-02-02 17:24:07 +02:00
Artem Bilan
34c25a22fa INT-3606: Implement Closeable/Flushable
JIRA: https://jira.spring.io/browse/INT-3606

Change `LifecycleMessageSource` usage to use just `Lifecycle`
2015-01-13 15:32:07 -05:00
Gary Russell
15eb01169d INT-3581: Support selector-expression on WireTap
JIRA: https://jira.spring.io/browse/INT-3581

Move schemas to 4.2.

Add `selector-expression` to `<wire-tap/>`.

INT-3781: Fix What's New

Bump Namespace Version to 4.2
2015-01-13 15:32:06 -05:00
Gary Russell
2581388cc2 INT-3606: Implement Closeable/Flushable
JIRA: https://jira.spring.io/browse/INT-3606

Close the store when an (S)FTP synchronizing MessageSource is stopped.

Will need another commit on 4.2.WIP to change to `Lifecycle`.

Add dirty Flag to Properties MetadataStore

Avoid unnecessary persists.

Remove unnecessary implementation of `AbstractInboundFileSynchronizingMessageSource#getComponentType()`
2015-01-13 15:35:58 +02:00
Artem Bilan
703e69ff5c INT-3584: Introduce FileSplitter
JIRA: https://jira.spring.io/browse/INT-3584

INT-3584: Move `iterator` option to ctor

Polishing

- Javadocs
- Fix String payload (missing 'else')
- Support charsets other than the default
- throw MessagingException instead of RuntimeException
- Add more tests to cover all cases

INT-3584: Throw `MessageHandlingException`
2015-01-09 09:41:09 -05:00
Gary Russell
26dae7bae7 INT-3572: Improve Accept Once Filter Performance
JIRA: https://jira.spring.io/browse/INT-3572

Previously, a blocking queue was used to hold seen files; this was used to enable
FIFO when a max capacity is set.

When used with no capacity, a queue is not needed; also the `contains` operation
on a queue requires a linear search which does not scale well for a large number
of files.

Use a `HashSet` instead to significantly improve the `contains` performance.

Only maintain a queue if a max capacity is set.

Fix `AcceptOnceFileListFilterTests` to be compatible with Java < 8
2014-12-09 18:59:00 +02:00
Artem Bilan
cb50b1565c INT-3554: Add Expression Setters
JIRA: https://jira.spring.io/browse/INT-3554

**Cherry-pick to 4.0.x**

Polishing
2014-11-07 15:58:22 -05:00
Gary Russell
e4f1ee14aa INT-3537 RemoteFileTemplate Close Stream
JIRA: https://jira.spring.io/browse/INT-3537

Stream not closed if session cannot be created.

Add `try {} finally {}`.

__cherry-pick to 4.0.x, 3.0.x__
2014-10-22 12:26:08 +03:00
Artem Bilan
85866a2ea8 INT-3427: Add JsonNodeToStringConverter
JIRA: https://jira.spring.io/browse/INT-3427

INT-3427: Workaround for String -> File converter

INT-3427: PR comments
2014-08-20 10:04:22 -04:00
Stéphane Nicoll
463c185b38 Refactor AbstractHeaderMapper
This commit updates AbstractHeaderMapper in a number of ways:

* The 'userDefinedHeaderPrefix' property has been removed as some
  advanced tests revealed that the feature is actually broken right
  now and there is no easy way to fix it. A new richer hook point
  that indicates if the property comes from the target object or the
  standard MessageHeaders has been added.
* A HeaderMatcher interface has been introduced to replace the
  lengthy checks in shouldMapHeader. Several implementations
  of that interface are provided
* An additional pattern has been added that maps any header that is
  *not* a standard header. It uses the standardHeaderPrefix property
  for that purpose
* A number of protected method that were only used to create the
  instance have been removed in favour of a non default constructor.
  Subclasses should provide those *static* values in their own default
  constructor.
* The list of transient headers can now be customized. Transient
  headers are headers that should never be mapped. The standard
  ErrorChannel and ReplyChannel headers have also been removed from
  the default transient headers list as they no longer need to be
  transient

header-mapper: Polishing
2014-08-19 12:02:01 +03:00
Artem Bilan
d69c8cb718 INT-3495 Add FileTailInChAFBean#setErrorChannel
JIRA: https://jira.spring.io/browse/INT-3495

**Cherry-pick to 4.0.x & 3.0.x**

INT-3495 Add `error-channel` to `<tail-i-c-a>`

JIRA: https://jira.spring.io/browse/INT-3495

* Fix `AbstractEndpoint` JavaDoc
2014-08-15 11:28:23 -04:00
Gary Russell
403c91801d INT-3412 (S)FTP Append, rmdir, Client Access
JIRA: https://jira.spring.io/browse/INT-3412

Initial commit - review only.

TODO:
- SFTP Tests
- Namespace/Adapter support for file append
- Docs

INT-3412 Polishing

- Addressed PR comments
- Completed SFTP implementation
- Added namespace/parser support for `FileExistsMode` (append, etc)
- Added SFTP Tests
- Created Embedded SFTP server for tests (similar to FTP)
- Converted tests that needed a real server to use the embedded server

INT-3412 Docs and Polish (PR Comments)
2014-08-04 15:08:34 +03:00
Gary Russell
a20fa9a029 INT-3464 Exceptions and AcceptOnceFileListFilter
JIRA: https://jira.spring.io/browse/INT-3464

Previously, if an IO exeption occurred while synchronizing
files, and an AcceptOnceFileListFilter is being used, files that were
not transferred would not be fetched next time.

Add strategy `ReversibleFileListFilter` that can rollback previously
accepted files.

Implement this interface on `AcceptOnceFileListFilter` and
`AbstractPersistentAcceptOnceFileListFilter`.

Add test cases.

Polishing
2014-07-24 16:56:38 +03:00
Liujiong
302edf9221 INT-3465:Content Enricher Improvements
JIRA: https://jira.spring.io/browse/INT-3465

Add support for adding/removing individual recipients to the RecipientListRouter

Modify documentation in what's new and spring-integration-4.1.xsd

Polishing

`AbstractRemoteFileOutboundGateway`: close `outputStream` before `file.delete()` to release exclusive file-lock
2014-07-23 21:42:45 +03:00