Commit Graph

397 Commits

Author SHA1 Message Date
Gary Russell
f4104c199a INT-3745: More TCP Events (Failed Correlation)
JIRA: https://jira.spring.io/browse/INT-3745

Emit events whenever a message is received that can't be correlated to
a connection (or request in the case of an outbound gateway).

Also, when sending messages, wrap the exception in a `MessagingException` so that
the `TcpConnectionExceptionEvent` provides access to the failed message.

INT-3745: Polishing - PR Comments

Fix JavaDocs errors
2015-06-19 13:49:10 -04:00
Gary Russell
65d2024937 INT-3737: (S)FTP Outbound Gateway Streaming GET
JIRA: https://jira.spring.io/browse/INT-3737

Support returning an `InputStream` from a GET operation.

This can be used in conjuction with a `<file:splitter/>` to stream a text file.

The user is responsible for releasing the session when the download is complete.

The session object is stored in the message header and `RemoteFileUtils.closeSession()` should be called
to clean up and close the session.

INT-3737: Polishing and Docs

Fix `CachedSession.close()` bug

Preventing `Caused by: java.io.IOException: Previous raw read was not finalized`
when `Session` is returned to the pool, but `readingRaw` hasn't been finalized
because the real `close()` isn't invoked in case of normal return to pool.
2015-06-19 13:39:36 -04:00
Gary Russell
3d5f7db4b2 INT-4740: FileSplitter - Add Headers
JIRA: https://jira.spring.io/browse/INT-3740

For `File` and `String` payloads add `FileHeaders.ORIGINAL_FILE` and `FileHeaders.FILENAME` headers.

Reworked to create headers once only.

Fix typos and Java > 6  API usage
2015-06-18 13:55:52 -04:00
Gary Russell
6436aa6d32 INT-3600: FileSplitter Namespace and Docs
JIRA: https://jira.spring.io/browse/INT-3600

Add `<int-file:splitter/>` namespace component.
Add `FileSplitter` documentation.

Polishing
2015-06-18 13:45:16 -04:00
Gunnar Hillert
247232bdde INT-3659: InputStream support in FileWritingMH
https://jira.spring.io/browse/INT-3659

Add `InputStream` `payload` handling to the `FileWritingMessageHandler`
2015-06-17 20:57:37 -04:00
Gary Russell
da54a1262d INT-3739: FileSplitter - Add LineCount to EOF Mark
JIRA: https://jira.spring.io/browse/INT-3739

Since iterating splitters don't have a sequence size, it would be useful to
include a line count in the EOF marker, to facilitate some downstream process
knowing that all lines have been processed, in a multi-threaded situation.
2015-06-17 17:59:00 -04:00
Gary Russell
c278548216 INT-3738: Fix FileSplitter
JIRA: https://jira.spring.io/browse/INT-3738

Don't rely on `BufferedReader.ready()` to detect EOF.

Polishing FileSplitter Fix
2015-06-17 17:21:48 -04:00
Gary Russell
51b52ea3a6 INT-3727: FileSplitter - add SOF/EOF Capability
JIRA: https://jira.spring.io/browse/INT-3727

Add a configuration option such that the `FileSplitter` can emit
Start/End messages when splitting.

Polishing according PR comments

Add toString to FileMarker
2015-06-08 13:52:48 -04:00
Gunnar Hillert
a0160e50d4 INT-3718: Add ignore-hidden to int-file:i-c-a
JIRA: https://jira.spring.io/browse/INT-3718

* If not set to `false` explicitly, add `IgnoreHiddenFileListFilter` as a default in `FileListFilterFactoryBean`
* Update XML Schema XSD and add `ignore-hidden` attribute
* Update + Add tests
* Add documentation
* Add default `IgnoreHiddenFileListFilter` also to `DefaultDirectoryScanner`
2015-05-25 22:08:12 -04:00
Gary Russell
5b0b3fc159 INT-3721: Add Flush Support to Persistent Filters
JIRA: https://jira.spring.io/browse/INT-3721

Support flushing metadata after each update.

Polishing JavaDocs
2015-05-25 21:13:41 -04:00
Gunnar Hillert
510bea55cd INT-3716: Add IgnoreHiddenFileListFilter
JIRA: https://jira.spring.io/browse/INT-3716

**Cherry-pick to 4.1.x**
2015-05-19 20:26:25 +03:00
Gary Russell
123b38a008 INT-3706: Support FileExistModes in (S)FTP Gateway
JIRA: https://jira.spring.io/browse/INT-3706

Previously, the `mode` attribute on the (S)FTP outbound gateways was ignored.

Take account of the mode on (M)GET and (M)PUT commands.
2015-05-12 13:21:26 +03:00
Gary Russell
15150ac499 INT-3699: RemoteFileTemplate Javadocs
JIRA: https://jira.spring.io/browse/INT-3699

Also javadocs for related classes.

INT-3699: Polishing - PR Comments
2015-04-15 12:49:41 +03:00
Gavin Gray
0d4c839331 INT-3684: Rebuild Tail Command on Restart
JIRA: https://jira.spring.io/browse/INT-3684
2015-03-31 11:21:28 +01:00
Artem Bilan
2bde14b742 INT-3661: Fix the eager BF access from BPPs (P I)
JIRA: https://jira.spring.io/browse/INT-3661

Previously there were a lot of noise from the `PostProcessorRegistrationDelegate$BeanPostProcessorChecker` for early access for beans.
That may produce some side-effects when some of `BeanFactoryPostProcessor`s won't adjust those beans.

The issue is based on two facts:
1. Loading beans from `BPP`, e.g. `IntegrationEvaluationContextAwareBeanPostProcessor` (or `ChannelSecurityInterceptorBeanPostProcessor` - https://jira.spring.io/browse/INT-3663)
2. Loading beans from `setBeanFactory()/setApplicationContext()` container methods

* Move all code from `setBeanFactory()` with access to the `BeanFactory` (e.g. `this.messageBuilderFactory = IntegrationUtils.getMessageBuilderFactory(this.beanFactory);`)
to some other lazy-load methods like `getMessageBuilderFactory()`
* Fix parser tests to remove `messageBuilderFactory` tests since there is no activity for target components to lazy-load them
* Polish some test according the new lazy-load logic
* Rework `IntegrationEvaluationContextAwareBeanPostProcessor` to the `SmartInitializingSingleton` and make it `Ordered`
* Populate `beanFactory` for the internal instance of `connectionFactory` in the `TcpSyslogReceivingChannelAdapter`
* Populate `beanFactory` for the internal `UnicastReceivingChannelAdapter` in the `UdpSyslogReceivingChannelAdapter`
* Add `log.info` that `UdpSyslogReceivingChannelAdapter` overrides `outputChannel` for the provided `UnicastReceivingChannelAdapter`
* Change the internal `MessageChannel` in the `UdpSyslogReceivingChannelAdapter` to the `FixedSubscriberChannel` for better performance

* Fix `AbstractExpressionEvaluator`
* Add JavaDocs for the `IntegrationEvaluationContextAware`

Fix `MongoDbMessageStoreClaimCheckIntegrationTests`

Addressing PR comments
2015-03-02 18:23:44 -05:00
Gary Russell
80a45715ac INT-3628: Use SchedulingAwareRunner for Long Tasks
JIRA: https://jira.spring.io/browse/INT-3628

Certain tasks in Spring Integration are long running. Indicate
so for the benefit of platforms such as WebLogic that can log warnings
for long running tasks otherwise.

Also fix conflicting versions of commons-io.
2015-02-28 15:16:07 +02:00
Artem Bilan
f248394682 INT-3589: Upgrade Dependencies
JIRA: https://jira.spring.io/browse/INT-3589, https://jira.spring.io/browse/INT-3624

* SF - 4.2. Fix Breaking changes for the `ApplicationEventPublisher`
* AMQP 1.5. Without issues
* Reactor - 2.0. Fix for the new `Stream` foundation
* jsonPath - 1.2.0. Fix for new `Predicate` abstraction. Add 'fail-fast error' to the `IntegrationRegistrar`
* Sshd - 0.13.0. Fix for new `VirtualFileSystemFactory` usage
* Spring Data - Fowler
* And others without issues
* Get rid of `reactor.util.StringUtils` usage

INT-3589: Address PR comments

Polishing
2015-02-17 15:09:09 -05:00
Gary Russell
195ee70568 INT-3632: Sonar Improvements
JIRA: https://jira.spring.io/browse/INT-3632

- Use Map.entrySet()
- Double check locking only works when the field(s) are volatile
- Remove unnecessary instanceof tests
2015-02-10 18:00:07 -05:00
Tony Falabella
a7d325adc1 INT-3621: Add Namespace support for appendNewLine
JIRA: https://jira.springsource.org/browse/INT-3621

Polishing for code formatting and simple polishing to Docs
2015-02-05 15:38:35 +02:00
Tony Falabella
24beee9627 INT-3620: add appendNewLine to the FileWritingMH
JIRA: https://jira.spring.io/browse/INT-3620

Add `setShouldAppendNewLine(boolean shouldAppendNewLine)` method to the
`FileWritingMessageHandler` class.  Added similar test methods to the
`FileWritingMessageHandlerTests` class.

INT-3620: `appendNewLine` just only for `FileExistsMode.APPEND`

Also add `appendNewLine` for `File` payload

INT-3620: `appendNewLine` for any `fileExistsMode`
2015-02-02 17:24:07 +02:00
Artem Bilan
34c25a22fa INT-3606: Implement Closeable/Flushable
JIRA: https://jira.spring.io/browse/INT-3606

Change `LifecycleMessageSource` usage to use just `Lifecycle`
2015-01-13 15:32:07 -05:00
Gary Russell
15eb01169d INT-3581: Support selector-expression on WireTap
JIRA: https://jira.spring.io/browse/INT-3581

Move schemas to 4.2.

Add `selector-expression` to `<wire-tap/>`.

INT-3781: Fix What's New

Bump Namespace Version to 4.2
2015-01-13 15:32:06 -05:00
Gary Russell
2581388cc2 INT-3606: Implement Closeable/Flushable
JIRA: https://jira.spring.io/browse/INT-3606

Close the store when an (S)FTP synchronizing MessageSource is stopped.

Will need another commit on 4.2.WIP to change to `Lifecycle`.

Add dirty Flag to Properties MetadataStore

Avoid unnecessary persists.

Remove unnecessary implementation of `AbstractInboundFileSynchronizingMessageSource#getComponentType()`
2015-01-13 15:35:58 +02:00
Artem Bilan
703e69ff5c INT-3584: Introduce FileSplitter
JIRA: https://jira.spring.io/browse/INT-3584

INT-3584: Move `iterator` option to ctor

Polishing

- Javadocs
- Fix String payload (missing 'else')
- Support charsets other than the default
- throw MessagingException instead of RuntimeException
- Add more tests to cover all cases

INT-3584: Throw `MessageHandlingException`
2015-01-09 09:41:09 -05:00
Gary Russell
26dae7bae7 INT-3572: Improve Accept Once Filter Performance
JIRA: https://jira.spring.io/browse/INT-3572

Previously, a blocking queue was used to hold seen files; this was used to enable
FIFO when a max capacity is set.

When used with no capacity, a queue is not needed; also the `contains` operation
on a queue requires a linear search which does not scale well for a large number
of files.

Use a `HashSet` instead to significantly improve the `contains` performance.

Only maintain a queue if a max capacity is set.

Fix `AcceptOnceFileListFilterTests` to be compatible with Java < 8
2014-12-09 18:59:00 +02:00
Artem Bilan
cb50b1565c INT-3554: Add Expression Setters
JIRA: https://jira.spring.io/browse/INT-3554

**Cherry-pick to 4.0.x**

Polishing
2014-11-07 15:58:22 -05:00
Gary Russell
e4f1ee14aa INT-3537 RemoteFileTemplate Close Stream
JIRA: https://jira.spring.io/browse/INT-3537

Stream not closed if session cannot be created.

Add `try {} finally {}`.

__cherry-pick to 4.0.x, 3.0.x__
2014-10-22 12:26:08 +03:00
Artem Bilan
85866a2ea8 INT-3427: Add JsonNodeToStringConverter
JIRA: https://jira.spring.io/browse/INT-3427

INT-3427: Workaround for String -> File converter

INT-3427: PR comments
2014-08-20 10:04:22 -04:00
Stéphane Nicoll
463c185b38 Refactor AbstractHeaderMapper
This commit updates AbstractHeaderMapper in a number of ways:

* The 'userDefinedHeaderPrefix' property has been removed as some
  advanced tests revealed that the feature is actually broken right
  now and there is no easy way to fix it. A new richer hook point
  that indicates if the property comes from the target object or the
  standard MessageHeaders has been added.
* A HeaderMatcher interface has been introduced to replace the
  lengthy checks in shouldMapHeader. Several implementations
  of that interface are provided
* An additional pattern has been added that maps any header that is
  *not* a standard header. It uses the standardHeaderPrefix property
  for that purpose
* A number of protected method that were only used to create the
  instance have been removed in favour of a non default constructor.
  Subclasses should provide those *static* values in their own default
  constructor.
* The list of transient headers can now be customized. Transient
  headers are headers that should never be mapped. The standard
  ErrorChannel and ReplyChannel headers have also been removed from
  the default transient headers list as they no longer need to be
  transient

header-mapper: Polishing
2014-08-19 12:02:01 +03:00
Artem Bilan
d69c8cb718 INT-3495 Add FileTailInChAFBean#setErrorChannel
JIRA: https://jira.spring.io/browse/INT-3495

**Cherry-pick to 4.0.x & 3.0.x**

INT-3495 Add `error-channel` to `<tail-i-c-a>`

JIRA: https://jira.spring.io/browse/INT-3495

* Fix `AbstractEndpoint` JavaDoc
2014-08-15 11:28:23 -04:00
Gary Russell
403c91801d INT-3412 (S)FTP Append, rmdir, Client Access
JIRA: https://jira.spring.io/browse/INT-3412

Initial commit - review only.

TODO:
- SFTP Tests
- Namespace/Adapter support for file append
- Docs

INT-3412 Polishing

- Addressed PR comments
- Completed SFTP implementation
- Added namespace/parser support for `FileExistsMode` (append, etc)
- Added SFTP Tests
- Created Embedded SFTP server for tests (similar to FTP)
- Converted tests that needed a real server to use the embedded server

INT-3412 Docs and Polish (PR Comments)
2014-08-04 15:08:34 +03:00
Gary Russell
a20fa9a029 INT-3464 Exceptions and AcceptOnceFileListFilter
JIRA: https://jira.spring.io/browse/INT-3464

Previously, if an IO exeption occurred while synchronizing
files, and an AcceptOnceFileListFilter is being used, files that were
not transferred would not be fetched next time.

Add strategy `ReversibleFileListFilter` that can rollback previously
accepted files.

Implement this interface on `AcceptOnceFileListFilter` and
`AbstractPersistentAcceptOnceFileListFilter`.

Add test cases.

Polishing
2014-07-24 16:56:38 +03:00
Liujiong
302edf9221 INT-3465:Content Enricher Improvements
JIRA: https://jira.spring.io/browse/INT-3465

Add support for adding/removing individual recipients to the RecipientListRouter

Modify documentation in what's new and spring-integration-4.1.xsd

Polishing

`AbstractRemoteFileOutboundGateway`: close `outputStream` before `file.delete()` to release exclusive file-lock
2014-07-23 21:42:45 +03:00
liujiong1982
5bb2141fe2 INT-3238:Delete the FTP Temporary File (get)
JIRA: https://jira.spring.io/browse/INT-2438

If a get operation fails in the outbound gateway,
clean up (remove) the temporary file if present.

Test Polishing
2014-07-22 10:46:14 -04:00
Artem Bilan
db8b164425 INT-3404: Remove MProducer from AIFileSMS
JIRA: https://jira.spring.io/browse/INT-3404

Previously, `AbstractInboundFileSynchronizingMessageSource` implemented both `MessageProducer` and `MessageSource`.
The `MessageProducer` is really redundant and confusing.
Also, it might produce some side-effects when using JavaConfig.

Provide some other simple but important fix for all `AbstractMessageSource` implementation to invoke `super.afterPropertiesSet()`

Polishing

Add mock bean factory to failing tests.

Revert state assertions in AbstractInboundFileSynchronizingMessageSource;
the fields are state, not arguments, at the time they are being
asserted.
2014-07-02 15:29:08 -04:00
Artem Bilan
f5145f531b INT-3418: Final Phase: Cleanup
INT-3418: use versioned XSDs for XSD import
2014-06-09 10:08:30 -04:00
Artem Bilan
77b31f58fd INT-3418: xsd-4.0 -> 4.1 2014-06-05 13:34:52 +03:00
Artem Bilan
c0ab912c49 INT-3418: xsd-3.0 -> 4.0 2014-06-05 13:34:41 +03:00
Artem Bilan
9e5e7514e0 INT-3418: xsd-2.2 -> 3.0 2014-06-05 13:34:30 +03:00
Artem Bilan
9bd441c8d0 INT-3418: xsd-2.1 -> 2.2 2014-06-05 13:34:19 +03:00
Artem Bilan
af1cd16027 INT-3418: xsd-2.0 -> 2.1 2014-06-05 13:33:57 +03:00
Artem Bilan
fa33f8f21d INT-3418: xsd-1.0 -> 2.0 2014-06-05 13:33:40 +03:00
Artem Bilan
cedbf6edc2 INT-3418: Prepare 4.1 VERSION
JIRA: https://jira.spring.io/browse/INT-3418

* Create 4.1 XSD
* Clean up Docs for 4.1
* Prepare WebSocket module
* Add Reactor dependency
* Remove Eclipse and SNV artifacts

Phase #1: prepare

Conflicts:
	gradle.properties
2014-05-30 21:14:21 +03:00
Gary Russell
47d47fb735 Fix JavaDoc on CompositeFileListFilter
Implied that the filter would pass if *any* of its filters
did, when it actually only passes if *all* of it filters do.
2014-04-25 18:41:43 -04:00
Gary Russell
8d6aa0fb1e INT-3374 Fix Package Tangle; Part II
JIRA: https://jira.spring.io/browse/INT-3374

Move `getMessageBuilderFactory` from `IntegrationContexUtils`
to `IntegrationUtils`.

Change DOS Newlines to Unix

`AbstractMessageGroupStore`.
2014-04-15 14:08:14 -04:00
Artem Bilan
de123c1ee4 INT-3367: Add GemfireMetadataStore
JIRA: https://jira.spring.io/browse/INT-3367

INT-3367 Polishing

Rename test class; move to metadata package; doc polishing.
2014-04-14 17:59:24 -04:00
Gary Russell
89a7743640 INT-3360 Fix Package Tangle
JIRA: https://jira.spring.io/browse/INT-3360

Move LockRegistry from util to support.
2014-04-08 12:01:51 -04:00
Gary Russell
4dee2a224b INT-3353 Concurrent RedisMetadataStore
JIRA: https://jira.spring.io/browse/INT-3353

Shared metadata for PersistentAcceptOnceFileListFilters.

INT-3353 Polishing; PR Comments
2014-04-07 17:38:33 +03:00
Gary Russell
9dee131e3c INT-3349 BeanFactory Propagation
JIRA: https://jira.spring.io/browse/INT-3349

Several FactoryBeans did not propagate the BeanFactory
to their created object(s). Beans that create messages
must have access to a bean factory to get the
message builder factory.

Fix the FactoryBeans and add a mock FB to all tests that
need one.

Add a runtime environment variable to make any infractions
fatal. This should be set to `true` on CI builds and on
framework developer environments.
2014-04-02 14:15:48 -04:00
Artem Bilan
97c270c0e2 INT-3330 EnableIntegrationMBeanExport Annotation
JIRA: https://jira.spring.io/browse/INT-3330

INT-3330: Enable SpEL evaluation

INT-3330: Polishing for `MBeanExporterHelper`

INT-3330: Fix `errorChannel` early access

INT-3330: Polishing according PR comments

Polishing - copyrights, author, docs
2014-03-27 12:53:20 -04:00