JIRA: https://jira.spring.io/browse/INT-4142
INT-4015: Streaming Remote File Inbound Adapter
Initial commit.
Reworked to emit an input stream and use the file splitter.
Add StreamTransformer.
Add CLOSABLE_RESOURCE header so we can close the session automatically.
Implement INT-3854, FTP, SFTP
(S)FTP Namespace Changes
Docs - also fixes a PDF overflow
Polishing - PR Comments
checkstyle fixes
Polishing - Add Namespace for StreamParser
Polishing - PR Comments
Fix Streaming (S)FTP Tests
https://build.spring.io/browse/INT-B43-190/
Add `AcceptOneFileListFilter`s.
Polishing
polishing
Polishing
* Polish `.travis.yml` to avoid unnecessary work on Travis CI
JIRA: https://jira.spring.io/browse/INT-4049
Since the `FileSplitter` can now return `Message<?>` as item for markers, we should ensure the original headers copying, e.g. `file_remoteSession`
Also change `FtpServerOutboundTests-context.xml` to check that new `file_marker` header works well
JIRA: https://jira.spring.io/browse/INT-4046
Since not all FTP servers provide proper `STAT` command implementation,
plus the `NLIST` doesn't work properly for directories cases, introduce the `FtpRemoteFileTemplate.ExistsMode`
to let:
* to perform `STAT` by default (previous) behavior;
* to switch to `NLIST` for `FtpRemoteFileTemplate` internal use;
* perform the full `NLIST` and `FTPClient.changeWorkingDirectory()` algorithm if needed.
* Improve (S)Ftp components to use proper `RemoteFileTemplate` for internal instantiation
* Introduce `FtpMessageHandler` to wrap `FtpRemoteFileTemplate` with the proper `NLIST` `ExistsMode`
* Cover `NLIST` switching from the `FtpOutboundChannelAdapterParser` and `FtpOutboundGatewayParser`
* Document the `FtpRemoteFileTemplate.ExistsMode`
* Fix typo in the recently introduced `RemoteFileOperations.getSession()` method name
* Add JavaDoc to `Session.exists()`
* Add `NLIST` support for the `FtpSession.exists()` to meet the API requirements
**Cherry-pick to 4.2.x and 4.1.x**
Addressing PR comments
Doc Polishing
Conflicts:
spring-integration-file/src/main/java/org/springframework/integration/file/remote/AbstractRemoteFileStreamingMessageSource.java
spring-integration-file/src/main/java/org/springframework/integration/file/remote/RemoteFileOperations.java
spring-integration-file/src/main/java/org/springframework/integration/file/remote/RemoteFileTemplate.java
spring-integration-ftp/src/main/java/org/springframework/integration/ftp/session/FtpSession.java
spring-integration-ftp/src/test/java/org/springframework/integration/ftp/config/FtpOutboundChannelAdapterParserTests.java
spring-integration-sftp/src/main/java/org/springframework/integration/sftp/gateway/SftpOutboundGateway.java
spring-integration-sftp/src/main/java/org/springframework/integration/sftp/outbound/SftpMessageHandler.java
spring-integration-sftp/src/main/java/org/springframework/integration/sftp/outbound/package-info.java
JIRA: https://jira.spring.io/browse/INT-3749
Since `IntegrationEvaluationContextAware` isn't so "context-free" resource like `BeanFactory`
and `ApplicationContext`, but just a specific bean in the context, we can't follow with `BeanPostProcessor` logic - bad architecture by level of responsibility.
Therefore we should follow with standard Dependency Injection mechanism to retrieve `evaluationContext` for the particular component.
Since we can't rely on the `@Autowired` because SI can be used from the raw XML configuration,
we use utility method instead. The pattern to get the proper `integrationEvaluationContext` is:
```
@Override
protected void onInit() throws Exception {
super.onInit();
this.evaluationContext = ExpressionUtils.createStandardEvaluationContext(getBeanFactory());
}
```
JIRA: https://jira.spring.io/browse/INT-3737
Support returning an `InputStream` from a GET operation.
This can be used in conjuction with a `<file:splitter/>` to stream a text file.
The user is responsible for releasing the session when the download is complete.
The session object is stored in the message header and `RemoteFileUtils.closeSession()` should be called
to clean up and close the session.
INT-3737: Polishing and Docs
Fix `CachedSession.close()` bug
Preventing `Caused by: java.io.IOException: Previous raw read was not finalized`
when `Session` is returned to the pool, but `readingRaw` hasn't been finalized
because the real `close()` isn't invoked in case of normal return to pool.
JIRA: https://jira.spring.io/browse/INT-3706
Previously, the `mode` attribute on the (S)FTP outbound gateways was ignored.
Take account of the mode on (M)GET and (M)PUT commands.
JIRA: https://jira.spring.io/browse/INT-3632
- Use Map.entrySet()
- Double check locking only works when the field(s) are volatile
- Remove unnecessary instanceof tests
JIRA: https://jira.spring.io/browse/INT-3581
Move schemas to 4.2.
Add `selector-expression` to `<wire-tap/>`.
INT-3781: Fix What's New
Bump Namespace Version to 4.2
JIRA: https://jira.spring.io/browse/INT-3606
Close the store when an (S)FTP synchronizing MessageSource is stopped.
Will need another commit on 4.2.WIP to change to `Lifecycle`.
Add dirty Flag to Properties MetadataStore
Avoid unnecessary persists.
Remove unnecessary implementation of `AbstractInboundFileSynchronizingMessageSource#getComponentType()`
JIRA: https://jira.spring.io/browse/INT-3412
Initial commit - review only.
TODO:
- SFTP Tests
- Namespace/Adapter support for file append
- Docs
INT-3412 Polishing
- Addressed PR comments
- Completed SFTP implementation
- Added namespace/parser support for `FileExistsMode` (append, etc)
- Added SFTP Tests
- Created Embedded SFTP server for tests (similar to FTP)
- Converted tests that needed a real server to use the embedded server
INT-3412 Docs and Polish (PR Comments)
JIRA: https://jira.spring.io/browse/INT-3464
Previously, if an IO exeption occurred while synchronizing
files, and an AcceptOnceFileListFilter is being used, files that were
not transferred would not be fetched next time.
Add strategy `ReversibleFileListFilter` that can rollback previously
accepted files.
Implement this interface on `AcceptOnceFileListFilter` and
`AbstractPersistentAcceptOnceFileListFilter`.
Add test cases.
Polishing
JIRA: https://jira.spring.io/browse/INT-2586
Add timeout properties to the `DefaultFtpSessionFactory`
avoiding the need to subclass it just to set these
common properties.
JIRA: https://jira.spring.io/browse/INT-2738
Add a `getComponentType()` method for all handlers,
message producers, and message sources that implement
`NamedComponent` (see `SPCA.getComponentType()`).
Add rome as an optional dependency - STS complained
because AtomFeedHttpMessageConverter has a dependency
on it.
JIRA: https://jira.spring.io/browse/INT-3349
Several FactoryBeans did not propagate the BeanFactory
to their created object(s). Beans that create messages
must have access to a bean factory to get the
message builder factory.
Fix the FactoryBeans and add a mock FB to all tests that
need one.
Add a runtime environment variable to make any infractions
fatal. This should be set to `true` on CI builds and on
framework developer environments.
JIRA: https://jira.spring.io/browse/INT-3041
Add namespace support to simplify configuration of
a `RequestHandlerRetryAdvice.
INT-3041 Polishing; PR Comments
* Allow a retry-advice element within the request-handler-advice-chain
* Clean up schema (should not have allowed `synchronization-factory` on
the request-handler-advice-chain.
INT-3041 Polishing; PR Comment
If an exception occurs on a session it should be physically closed
and not reused because its state is indeterminate and the next
operation might fail.
Note: The booleans within CachedSession do not need to be volatile
because it is a short-lived object only used by the current thread.
Also fixes the assertion message in RFT.get().
JIRA: https://jira.springsource.org/browse/INT-3266
INT-3266 Polishing - PR Comments
Remove need for SuppressWarnings.
Fix 2 test cases where the exception has an additional cause.
- Core support in file module
- FTP Parser and Test
https://jira.springsource.org/browse/INT-3088
INT-3088 Add SFTP Support for PUT, MPUT
INT-3088 Polishing - PR Comments
INT-3088 Docbook For PUT/MPUT
Begin by refactoring `FileTransferringMessageHandler` to use
a `send` operation on RemoteFileTemplate.
Effectively transfer all properties from the adapter to the
template.
This will allow reuse of the send logic from the adapter in
the outbound gateway for `put` and `mput`.
JIRA: https://jira.springsource.org/browse/INT-3203
Previously, `MetadataStore` couldn't be configured for Twitter Adapters
- only a global one could be used.
The `metadataKey` was generated automatically with a 'difficult' value.
* Register all `MessageSource` for `SourcePollingChannelAdapter`
as beans with id based on adapter id and prefix '.source' (INT-3147)
* Polishing parser to get rid of explicit `MessageSource` beans. (INT-3147)
* Make Feed and Twitter adapters `id` attribute as required -
now it presents a `metadataKey` for `MetadataStore` (INT-3147)
* Add to Twitter adapters a reference attribute for `MetadataStore` (INT-3173)
* Add Twitter adapters `poll-skip-period` attribute (INT-3167)
* Add and implement `MetadataStore#remove` (INT-1941)
* Make `MetadataStore` as `@ManagedResource` (INT-1941)
* Polishing tests
JIRAs:
https://jira.springsource.org/browse/INT-3147https://jira.springsource.org/browse/INT-3167https://jira.springsource.org/browse/INT-3173https://jira.springsource.org/browse/INT-1941
INT-3147: Polishing and fixes
* add domain suffix to `metadataKey`
* change contract of `MetadataStore.remove`
* remove timeout window from `AbstractTwitterMessageSource`
* polishing and fix `SearchReceivingMessageSourceWithRedisTests`
INT-3147: Rebasing and polishing
INT-3147: fix 'metadata' package tangle
INT-3147 Doc Polishing
