JIRA: https://jira.springsource.org/browse/INT-3305
Previously, if the channel can't be connected (e.g. if the server
does not support SFTP), the connection to the server remained open.
Close the session in the event of a failure.
Add test with the Apache SSHD server.
JIRA: https://jira.springsource.org/browse/INT-3209
Previously, the DefaultFileNameGenerator re-parsed the
SpEL expression for every message.
Compile the SpEL once (and if it is changed).
Change unit tests to match changes.
- Core support in file module
- FTP Parser and Test
https://jira.springsource.org/browse/INT-3088
INT-3088 Add SFTP Support for PUT, MPUT
INT-3088 Polishing - PR Comments
INT-3088 Docbook For PUT/MPUT
Begin by refactoring `FileTransferringMessageHandler` to use
a `send` operation on RemoteFileTemplate.
Effectively transfer all properties from the adapter to the
template.
This will allow reuse of the send logic from the adapter in
the outbound gateway for `put` and `mput`.
JIRA: https://jira.springsource.org/browse/INT-3203
Enable the use of multiple JSch channels over a
single connection.
JIRA: https://jira.springsource.org/browse/INT-3047
INT-3047: Polishing locks logic
INT-3047 SFTP Reset Cache; Quiesce Shared Sessions
Support resetCache() on the CachingSessionFactory
- reset a shared JSch session so it is reestablished on next use
- immediately close idle sessions
- close in-use sessions as they are returned
- close the channel when an SftpSession is closed and a shared JSch is being used
- physically close a shared JSch session only when the last channel is closed
Polishing
Forgot to save the ftp.xml.
Change CachedSession.epoch to created.
INT-3047 Polishing - PR Comments
- Also add missing docs for `preserve-timestamp`
INT-3047 Polish - PR Comments
Also change epoch to nanoseconds and use a simple != comparison with the
epoch in which a session was created.
When recursing, the returned filenames are relative to
the top level directory.
INT-3172 (S)FTP - Add Recursion to MGET Command
https://jira.springsource.org/browse/INT-3172
INT-3172 Polishing
- PR Comments
- Add Tests for SFTP (tested with Mock and real SSH)
- Enhance test to ensure subdir is in 3rd file retrieved
- Exclude "special" directories ('.' and '..') from recursion
INT-3172 Make FtpServerRule Safe For Concurrency
Previously the port was static.
- Make the port an instance variable
- Channel the `@ ClassRule` to a `@ Rule`
- Register an appropriate DefaultFtpSessionFactory bean declaration
INT-3172 Polishing; Docs
- Rename FtpServerRule to TestFtpServer
- Remove restriction on filters for MGET
- Add test for filtered MGET
- Reference docs, what's new
INT-3172 More Polish
- Fix type in reference
- Fix error message now that filter is allowed with MGET
- Remove test for filter with MGET (now valid)
INT-3172: Fix tests and typos
INT-3172: this.remoteFileSeparator for recursion
* Add `local-directory-expression` to (S)FTP Outbound Gateways
* Add `FtpServerRule` to Apache Mina embedded FtpServer
* Add tests for (M)GET and `local-directory-expression`:
FTP tests uses `FtpServerRule`, SFTP tests need testing on real SFTP server
JIRA: https://jira.springsource.org/browse/INT-2866
INT-2866: Documentation
INT-2866 Polishing
- Remove leading /
- Change \ to / in invalid test
- Clean up after sftp
Tested with real SSH.
INT-2866 Polishing - Add Mock SFTP Test
Run with -Dspring-profiles-active=realSSH to run with a real SSH server.
Assumes ftptest/ftptest account on localhost with the following directory tree in the user's root...
$ tree sftpSource/
sftpSource/
├── sftpSource1.txt
├── sftpSource2.txt
└── subSftpSource
└── subSftpSource1.txt
INT-2866: Polishing
INT-2866: change `remotePath` to `remoteDirectory`
Doc Polishing.
byte[] and String payloads were written to a temporary file.
If the same message is transferred to two different destinations
concurrently, one of the adapters could remove the temporary
file while the other was using it.
Don't use a temporary file for these payloads, simply use the
payload byte[] [or a String.getBytes()] as the InputStream
passed to the Session.write() method.
Polishing
Polishing - Add WARN For Missing File
+ Test
JIRA: https://jira.springsource.org/browse/INT-3091
Use the Spring Framework stylesheet for javadocs.
Fix 3 missing package-info files.
(Aside: remove cache-sessions from 2 @Ignored test case configuration files).
* Add XSD attributeGroup `smartLifeCycleAttributeGroup`
* Use it for all adapters tags
* Refactoring `AbstractChannelAdapterParser` hierarchy
* Refactoring some `MessageHandler`'s that should not have `autoStartup` & `phase`.
It is the responsibility of `AbstrctEndpoint`
* Polishing some tests according new logic
JIRA: https://jira.springsource.org/browse/INT-2998
INT-2998: Polishing
Polishing
- Reformat schemas where indentation changed
- Add SLC to the UDP inbound adapter
* Add `local-filename-generator-expression` attribute to FTP and SFTP Outbound Gateway
* Refactoring for `(S)FtpOutboundGatewayParser`
* Add `local-filename-generator-expression` tests
* Add 'What's New' and attribute description
JIRA: https://jira.springsource.org/browse/INT-3129
Polishing
Doc Polishing
Assert attribute is only set for get/mget
Move tests to the get gateways
There were several "helper" classes where the context was initialized
in the constructor, before the BeanFactory was passed in.
Generally fixed by adding the BeanFactory to the constructor args.
Other cases where the container-managed bean instantiated a helper
and never passed in the BeanFactory.
Finally, a fix to ExpressionUtils where the caller had a BeanFactory
but the BF did not contain an EvaluationContext factory bean, the
BeanResolver was not set up. This is unlikely in a Spring Integration
application, but added for completeness.
INT-3115 Add a BeanFactory to Test Cases
Change the WARN log in ExpressionUtils to a fatal exception to detect
cases where an EvaluationContext was created without a BeanFactory.
While this was generally in test cases, it also exposed some cases
in code where the context was initialized without a BF.
polishing on merge
Allow modification of context property accessors and
functions.
Polishing
- Fix tests to ensure the EvaluationContextFactoryBean is
available when necessary.
Polishing - Use Utility for ALL EvaluationContexts
Add Test Showing Custom Function
See SpelTransformerIntegrationTests-context.xml
Polishing
Polishing -fix Remote Sync
Polishing - Emit WARN if No BeanFactory
Polishing - Do Not Override ConversionService
Kludge to Prevent Warning When No BeanFactory
Expressions for method invoking in the MessagingMethodInvokerHelper
don't need a BeanFactory - suppress the warning.
Polishing - PR Comments - Invert Boolean
* Add `requires-reply` attribute for all adapters outbound gateways as `true` by default
* WS-outbound-gateway is still without it, because it has its own specific attribute `ignore-empty-responses`
* Make `requires-reply` as `false` by default for `jdbc:stored-proc-outbound-gateway` inasmuch as `jdbc:stored-proc-outbound-adapter`
doesn't have ability to configure `returning-resultset`
* add parser tests for `requires-reply`
JIRA: https://jira.springsource.org/browse/INT-2822
INT-2822 'requires-reply' for ws:outbound-gateway
Default false.
INT-2822: Polishing after rebase
INT-2822: deprecate 'ignore-empty-responses'
* Add 'requires-reply' section into What's New
INT-2822: remove 'ignore-empty-responses' from RM
INT-2822: Polishing after rebase
INT-2822: Rebased and polished
INT-2822: Rebased and polished
Add WARN within deprecated `AbstractWebServiceOutboundGateway#setIgnoreEmptyResponses`
Revert 'ignore-empty-responses'; Doc Polishing
The 'cache-sessions' attribute was deprecated in 2.1 in favor
of using a configured CachingConnectionFactory to provide more
flexibility, such as setting the cache size.
Remove the attribute from the schemas.
Update tests.
Add docs.
Update Migration Guide on the project Wiki.
* (S)FTP Change Commands, Options to Enums
* INT-2981 Add Support For 'mv' To File/Remote/GW
* INT-2981 (S)FTP Namespace Support and Docs
* INT-2981 Create Remote Dirs if Needed
* INT-2981 Add 'mv' Command to What's New
* INT-2981 Polishing - PR Comments
* Add javadocs to enums.
* Polish schemas to use an enumerated type for available gateway commands
When autocreateDirectory is true, and the top level directory
did not exist, the mkdirs failed because the adapter tried
to create a directory name with a zero length string.
This caused an 'String index out of range: 0' exception in
SFTP.
Stop iteration when 'pathSegment' is zero length.
Copy schemas, cleanup whitespace, update spring.schemas files,
update version in AbstractIntegrationNamespaceHandler.
For each schema compared the 2.2 version with the 3.0 version
(diff -w), ensuring the only difference is the import of
3.0 schemas (where appropriate) instead of 2.2.
* ban to use 'channel' and 'request-channel' attributes for 'outbound-channel-adapters' and 'outbound-gateways' when they are declared inside the `<chain>`
* fix usage of `<request-handler-advice-chain>` for components when they are declared inside the `<chain>`
* ban to use `<request-handler-advice-chain>` for 'outbound-channel-adapters' when they are declared inside the `<chain>` don't implement `AbstractReplyProducingMessageHandler`
* refactor `IntegrationNamespaceUtils` to use `BeanDefinition` instead of `BeanDefinitionBuilder` for 'advice-chain' parsing
* tests for new logic for 'channel' and 'request-channel' attributes within `<chain>`
* tests for new logic for `<request-handler-advice-chain>` within `<chain>`
* fix failed test on slow machine
* fix `SysLogTransformer` tests
For reference:
https://jira.springsource.org/browse/INT-2718https://jira.springsource.org/browse/INT-2721https://jira.springsource.org/browse/INT-2719
INT-2718 Polishing
Remove the need for pseudo-transactional element
INT-2727 PseudoTX
Add PseudoTransactionalTransactionManager
INT-2727
addressed PR comments
cherry picked previous code for mail module to eliminate breaking change
INT-2727
initial refactoring pseudo-tx support to use common configuration
INT-2727
finalizing pseudo-tx synchronization support
INT-2727 polishing
INT-2727 polishing based on PR comments
INT-2727 addressed PR comments
INT-2727 polishing
INT-2727 Remove PseudoTransactionalMessageSource
Instead of getResource, bind the resource holder before
receive() and then add attributes to the holder.
INT-2727 polishing
INT-2727 Polishing
Remove bind of #resource; add beforeCommit() test;
add TransactionTemplate tests.
Add general capability to advise just the handleRequestMessage
part of an AbstractReplyProducingMessageHandler.
This is to advise just the immediate operation, and not the
entire downstream flow.
Uses include:
* outbound gateway post processing
* adding retry behavior using spring-retry
* adding circuit breaker functionality
Initial commit for review.
Also need to advise simple message handlers (such as file
etc) to allow them to post-process file operations
with payload.delete(), payload.renameTo(...) etc.
INT-2250 Add Circuit Breaker Advice
INT-343 Add Retry Advice
Stateless and Stateful retry using spring-retry. Stateless
means the RetryTemplate performs the retries internally.
Stateful means the exception is thrown (e.g. to JMS container)
and the retry state is maintained by spring-retry.
INT-2215, INT-343, INT-2250 Refactoring
Factor out common abstract Advice class.
INT-2214 Catch Evaluation Expression Exceptions
If an onSuccess expression evaluation fails, add an
option so the user can decide whether such an exception is
caught, or propagated to the caller.
INT-2214 etc PR Review Polishing
INT-2214 etc Namespace Core, File, FTP
Add <request-handler-advice-chain/> to outbound endpoints.
INT-2214 etc. More Namespace Support
amqp, event, gemfire, groovy, http, ip, jdbc, jms, jmx, jpa, mail, rmi, sftp, twitter, ws, xmpp
INT-2214 etc Polishing
PR Review
INT-2214 etc Polishing
Don't catch Throwable.
Move Advice classes to handler.advice package.
Remove M3 disposition-* attributes on File/(S)FTP inbound adapters.
Add <pseudo-transactional/> and <transaction-synchronization/> elements
to <poller/>.
These elements provide the following attributes:
* on-success-expression
* on-success-result-channel
* on-failure-expression
* on-failure-result-channel
* send-timeout
<transaction-synchronization/> synchronizes these expression evaluations
with the transaction, such that they are executed immediately after
the commit/rollback.
<psuedo-transactional/> is used for a non-transactional poller.
When an <advice-chain/> is provided to the poller, <psuedo-transactional/>
and <transaction-synchronization/> are synonyms; and the behavior is
dictated by whether or not the <advice-chain/> contains a transaction
advice. It is recommended that <pseudo-transactional/> is used when the
<advice-chain/> does not have a txAdvice, and <transaction-synchronization/>
when it does, but the framework does not enforce this.
The expressions have the original (polled) message as the #root variable.
In addition, a BeanResolver is provided, allowing expressions such as
'@someBean.handleSuccess(payload)'.
MessageSources may also implement PseudoTransactionalMessageSource. This
has a number of methods allowing more flexibility in transactional and
non-transactional environents. For example, for backwards compatibility.
the mail-inbound-channel-adapter deletes its polled message after the
receive() rather than after the polled message is sent (when running in
a non-transactional poller). However, when running in a transactional
poller, the delete is done after the transaction commits (but not when
it rolls back).
In addition, MessageSources that implement this interface can optionally
provide an arbitrary object to the success/failure expressions in a
variable named '#resource'.
INT-2685 Polishing
PR Review Comments
Add tests for non-tx PseudoTransactionalMessageSource
FileReadingMessageSource supports disposition of the payload
after the message is sent (or via transaction synchronization).
The (S)FTP adapters delegate to an FRMS; add support for the
file disposition expression, result channel, and send timeout
to the (S)FTP adapters.
Also tested with ftp sample - INTSAMPLES-83 - patch will be
committed once this is in a milestone.
FTP servers return full path from listNames() whereas SFTP
returns just the filename. mget logic assumed SFTP behavior.
Add a test to see if the filename already starts with the remote
directory and remove it before calling get() (which assumes just
the filename).
Includes an @Ignored test for FTP and SFTP that runs an mget
against a server.
* fix Missing documentation of "enableDaemonThread" property of the SFTP session factory
* Reference documentation clean-up
* Document all properties of the DefaultSftpSessionFactory
INT-1141 polished code
INT-1141 added tests
INT-1141 improved how iterator is obtaind from OrderAwareLikedHashSet
INT-1141 polishing
INT-1141 stashed commit with List-based collection and performance tests
INT-1141 improved OrderedAwareLinkedHashSet
INT-1141 polishing
INT-1141, INT-2627 improved LoadBalancingStrategy and
RoundRobinLoadBalancingStrategy to obtain handler's iterator faster
INT-1141 improved getHandlerIterator method to ensure that it only executes reordering logic if there are more then one handler
INT-1141 simplified OrderedAwareLinkedHashSet to not to extend from LinkedHashSet
INT-1141
changed OrderedAwareLinkedHashSet to OrderedAwareCopyOnWriteArraySetTests, imporoved array creation in RoundRobinLoadBalancingStrategy
INT-1141 polished failing tests and RoundRobinLoadBalancingStrategy. Removed 'transient' from OrderedAwareCopyOnWriteArraySet
XSD refactoring: remove use="required" from 'request-channel' attribute of all 'outbound-gateways'
Tests for all 'outbound-gateways' inside the <chain>
Add re-init logic for nested chains
Add logic about nested element for AbstractChannelAdapterParser
Refactor of DefaultOutboundChannelAdapterParser
Test for non-last nested chain with some outbound-channel-adapter
Improve XSD for chain-type
Manual outbound-channel-adapter ability for chain
Integration tests for all outbound-channel-adapter within <chain>
Remove redundant 'return-value-required' attribute from <stored-proc-outbound-channel-adapter>
Add support 'expectReply' for FileWritingMessageHandler
INT-2275 polishing & refactor FileOutbound*Parser
HttpRequestExecutingMessageHandlerTests polishing
INT-2275: polishing JavaDoc
Add 2.2 versions of module schemas.
JMX is already covered by an existing pull request.
INT-2536 Schema Check
Bump version for schema check in
AbstractIntegrationNamespaceHandler.
INT-2536 Remove Schema Versions
Some test config files had 2.1 versioned schemas.
ExtendedSession, with the listNames() method was introduced
in 2.1.1 (INT-2492) to support mget commands on the gateway.
This was to avoid breaking any user implementations of Session
in a point release. However, we documented this was temporary
and the interfaces would be consolidated in 2.2., and
ExtendedSession would be removed.
This is that consolidation.
INT-2498:Added use-temporary-file-name to (s)ftp outbound adapters
INT-2489 polished parser to eliminate any value validation
INT-2489 removed trailing whitespaces from schema files
INT-2489 added boolean flag to detect if 'temporaryFileSuffix'
was explicitly set so the warning message is only be displayed
if renaming is turned off, but a suffix provided.
INT-2489 polishing
No options supported; send a payload of /* or * to
mget all files in the root directory.
Payload is a list of File objects representing files
copied to the 'local-directory'.
INT-2492 Polishing
Fix path issues; suppress LIST when GETing files from NLST.
INT-2492 Polishing
Add ExtendedSession so CachingConnectionFactory works.
INT-2492 SFTP, Docs
INT-2492 Polishing
INT-2492 Polishing - olegz PR Review
INT-2492 Polishing - EOL Note for ExtendedSession
The AbstractChannelAdapterParser creates an implicit DirectChannel
if the adapter has no 'channel' attribute.
The Event, TCP, and UDP channel adapter parsers did not bind
this channel to the adapter and AC initialization failed with
'outputChannel is required'.
Further, the event schema marked the channel as being 'required',
precluding this feature.
INT-2407 Remove Channel use="required"
Parsers automatically generate the channel when none is provided.
- JMX
- JDBC
- SFTP
- Redis
- Feed
- XMPP
- Mail
- FTP
- HTTP
