When recursing, the returned filenames are relative to
the top level directory.
INT-3172 (S)FTP - Add Recursion to MGET Command
https://jira.springsource.org/browse/INT-3172
INT-3172 Polishing
- PR Comments
- Add Tests for SFTP (tested with Mock and real SSH)
- Enhance test to ensure subdir is in 3rd file retrieved
- Exclude "special" directories ('.' and '..') from recursion
INT-3172 Make FtpServerRule Safe For Concurrency
Previously the port was static.
- Make the port an instance variable
- Channel the `@ ClassRule` to a `@ Rule`
- Register an appropriate DefaultFtpSessionFactory bean declaration
INT-3172 Polishing; Docs
- Rename FtpServerRule to TestFtpServer
- Remove restriction on filters for MGET
- Add test for filtered MGET
- Reference docs, what's new
INT-3172 More Polish
- Fix type in reference
- Fix error message now that filter is allowed with MGET
- Remove test for filter with MGET (now valid)
INT-3172: Fix tests and typos
INT-3172: this.remoteFileSeparator for recursion
* Add `local-directory-expression` to (S)FTP Outbound Gateways
* Add `FtpServerRule` to Apache Mina embedded FtpServer
* Add tests for (M)GET and `local-directory-expression`:
FTP tests uses `FtpServerRule`, SFTP tests need testing on real SFTP server
JIRA: https://jira.springsource.org/browse/INT-2866
INT-2866: Documentation
INT-2866 Polishing
- Remove leading /
- Change \ to / in invalid test
- Clean up after sftp
Tested with real SSH.
INT-2866 Polishing - Add Mock SFTP Test
Run with -Dspring-profiles-active=realSSH to run with a real SSH server.
Assumes ftptest/ftptest account on localhost with the following directory tree in the user's root...
$ tree sftpSource/
sftpSource/
├── sftpSource1.txt
├── sftpSource2.txt
└── subSftpSource
└── subSftpSource1.txt
INT-2866: Polishing
INT-2866: change `remotePath` to `remoteDirectory`
Doc Polishing.
byte[] and String payloads were written to a temporary file.
If the same message is transferred to two different destinations
concurrently, one of the adapters could remove the temporary
file while the other was using it.
Don't use a temporary file for these payloads, simply use the
payload byte[] [or a String.getBytes()] as the InputStream
passed to the Session.write() method.
Polishing
Polishing - Add WARN For Missing File
+ Test
JIRA: https://jira.springsource.org/browse/INT-3091
Use the Spring Framework stylesheet for javadocs.
Fix 3 missing package-info files.
(Aside: remove cache-sessions from 2 @Ignored test case configuration files).
* Add `local-filename-generator-expression` attribute to FTP and SFTP Outbound Gateway
* Refactoring for `(S)FtpOutboundGatewayParser`
* Add `local-filename-generator-expression` tests
* Add 'What's New' and attribute description
JIRA: https://jira.springsource.org/browse/INT-3129
Polishing
Doc Polishing
Assert attribute is only set for get/mget
Move tests to the get gateways
There were several "helper" classes where the context was initialized
in the constructor, before the BeanFactory was passed in.
Generally fixed by adding the BeanFactory to the constructor args.
Other cases where the container-managed bean instantiated a helper
and never passed in the BeanFactory.
Finally, a fix to ExpressionUtils where the caller had a BeanFactory
but the BF did not contain an EvaluationContext factory bean, the
BeanResolver was not set up. This is unlikely in a Spring Integration
application, but added for completeness.
INT-3115 Add a BeanFactory to Test Cases
Change the WARN log in ExpressionUtils to a fatal exception to detect
cases where an EvaluationContext was created without a BeanFactory.
While this was generally in test cases, it also exposed some cases
in code where the context was initialized without a BF.
polishing on merge
Allow modification of context property accessors and
functions.
Polishing
- Fix tests to ensure the EvaluationContextFactoryBean is
available when necessary.
Polishing - Use Utility for ALL EvaluationContexts
Add Test Showing Custom Function
See SpelTransformerIntegrationTests-context.xml
Polishing
Polishing -fix Remote Sync
Polishing - Emit WARN if No BeanFactory
Polishing - Do Not Override ConversionService
Kludge to Prevent Warning When No BeanFactory
Expressions for method invoking in the MessagingMethodInvokerHelper
don't need a BeanFactory - suppress the warning.
Polishing - PR Comments - Invert Boolean
* Add `requires-reply` attribute for all adapters outbound gateways as `true` by default
* WS-outbound-gateway is still without it, because it has its own specific attribute `ignore-empty-responses`
* Make `requires-reply` as `false` by default for `jdbc:stored-proc-outbound-gateway` inasmuch as `jdbc:stored-proc-outbound-adapter`
doesn't have ability to configure `returning-resultset`
* add parser tests for `requires-reply`
JIRA: https://jira.springsource.org/browse/INT-2822
INT-2822 'requires-reply' for ws:outbound-gateway
Default false.
INT-2822: Polishing after rebase
INT-2822: deprecate 'ignore-empty-responses'
* Add 'requires-reply' section into What's New
INT-2822: remove 'ignore-empty-responses' from RM
INT-2822: Polishing after rebase
INT-2822: Rebased and polished
INT-2822: Rebased and polished
Add WARN within deprecated `AbstractWebServiceOutboundGateway#setIgnoreEmptyResponses`
Revert 'ignore-empty-responses'; Doc Polishing
The 'cache-sessions' attribute was deprecated in 2.1 in favor
of using a configured CachingConnectionFactory to provide more
flexibility, such as setting the cache size.
Remove the attribute from the schemas.
Update tests.
Add docs.
Update Migration Guide on the project Wiki.
* (S)FTP Change Commands, Options to Enums
* INT-2981 Add Support For 'mv' To File/Remote/GW
* INT-2981 (S)FTP Namespace Support and Docs
* INT-2981 Create Remote Dirs if Needed
* INT-2981 Add 'mv' Command to What's New
* INT-2981 Polishing - PR Comments
* Add javadocs to enums.
* Polish schemas to use an enumerated type for available gateway commands
When autocreateDirectory is true, and the top level directory
did not exist, the mkdirs failed because the adapter tried
to create a directory name with a zero length string.
This caused an 'String index out of range: 0' exception in
SFTP.
Stop iteration when 'pathSegment' is zero length.
* ban to use 'channel' and 'request-channel' attributes for 'outbound-channel-adapters' and 'outbound-gateways' when they are declared inside the `<chain>`
* fix usage of `<request-handler-advice-chain>` for components when they are declared inside the `<chain>`
* ban to use `<request-handler-advice-chain>` for 'outbound-channel-adapters' when they are declared inside the `<chain>` don't implement `AbstractReplyProducingMessageHandler`
* refactor `IntegrationNamespaceUtils` to use `BeanDefinition` instead of `BeanDefinitionBuilder` for 'advice-chain' parsing
* tests for new logic for 'channel' and 'request-channel' attributes within `<chain>`
* tests for new logic for `<request-handler-advice-chain>` within `<chain>`
* fix failed test on slow machine
* fix `SysLogTransformer` tests
For reference:
https://jira.springsource.org/browse/INT-2718https://jira.springsource.org/browse/INT-2721https://jira.springsource.org/browse/INT-2719
INT-2718 Polishing
Remove the need for pseudo-transactional element
INT-2727 PseudoTX
Add PseudoTransactionalTransactionManager
INT-2727
addressed PR comments
cherry picked previous code for mail module to eliminate breaking change
INT-2727
initial refactoring pseudo-tx support to use common configuration
INT-2727
finalizing pseudo-tx synchronization support
INT-2727 polishing
INT-2727 polishing based on PR comments
INT-2727 addressed PR comments
INT-2727 polishing
INT-2727 Remove PseudoTransactionalMessageSource
Instead of getResource, bind the resource holder before
receive() and then add attributes to the holder.
INT-2727 polishing
INT-2727 Polishing
Remove bind of #resource; add beforeCommit() test;
add TransactionTemplate tests.
Add general capability to advise just the handleRequestMessage
part of an AbstractReplyProducingMessageHandler.
This is to advise just the immediate operation, and not the
entire downstream flow.
Uses include:
* outbound gateway post processing
* adding retry behavior using spring-retry
* adding circuit breaker functionality
Initial commit for review.
Also need to advise simple message handlers (such as file
etc) to allow them to post-process file operations
with payload.delete(), payload.renameTo(...) etc.
INT-2250 Add Circuit Breaker Advice
INT-343 Add Retry Advice
Stateless and Stateful retry using spring-retry. Stateless
means the RetryTemplate performs the retries internally.
Stateful means the exception is thrown (e.g. to JMS container)
and the retry state is maintained by spring-retry.
INT-2215, INT-343, INT-2250 Refactoring
Factor out common abstract Advice class.
INT-2214 Catch Evaluation Expression Exceptions
If an onSuccess expression evaluation fails, add an
option so the user can decide whether such an exception is
caught, or propagated to the caller.
INT-2214 etc PR Review Polishing
INT-2214 etc Namespace Core, File, FTP
Add <request-handler-advice-chain/> to outbound endpoints.
INT-2214 etc. More Namespace Support
amqp, event, gemfire, groovy, http, ip, jdbc, jms, jmx, jpa, mail, rmi, sftp, twitter, ws, xmpp
INT-2214 etc Polishing
PR Review
INT-2214 etc Polishing
Don't catch Throwable.
Move Advice classes to handler.advice package.
Remove M3 disposition-* attributes on File/(S)FTP inbound adapters.
Add <pseudo-transactional/> and <transaction-synchronization/> elements
to <poller/>.
These elements provide the following attributes:
* on-success-expression
* on-success-result-channel
* on-failure-expression
* on-failure-result-channel
* send-timeout
<transaction-synchronization/> synchronizes these expression evaluations
with the transaction, such that they are executed immediately after
the commit/rollback.
<psuedo-transactional/> is used for a non-transactional poller.
When an <advice-chain/> is provided to the poller, <psuedo-transactional/>
and <transaction-synchronization/> are synonyms; and the behavior is
dictated by whether or not the <advice-chain/> contains a transaction
advice. It is recommended that <pseudo-transactional/> is used when the
<advice-chain/> does not have a txAdvice, and <transaction-synchronization/>
when it does, but the framework does not enforce this.
The expressions have the original (polled) message as the #root variable.
In addition, a BeanResolver is provided, allowing expressions such as
'@someBean.handleSuccess(payload)'.
MessageSources may also implement PseudoTransactionalMessageSource. This
has a number of methods allowing more flexibility in transactional and
non-transactional environents. For example, for backwards compatibility.
the mail-inbound-channel-adapter deletes its polled message after the
receive() rather than after the polled message is sent (when running in
a non-transactional poller). However, when running in a transactional
poller, the delete is done after the transaction commits (but not when
it rolls back).
In addition, MessageSources that implement this interface can optionally
provide an arbitrary object to the success/failure expressions in a
variable named '#resource'.
INT-2685 Polishing
PR Review Comments
Add tests for non-tx PseudoTransactionalMessageSource
FileReadingMessageSource supports disposition of the payload
after the message is sent (or via transaction synchronization).
The (S)FTP adapters delegate to an FRMS; add support for the
file disposition expression, result channel, and send timeout
to the (S)FTP adapters.
Also tested with ftp sample - INTSAMPLES-83 - patch will be
committed once this is in a milestone.
FTP servers return full path from listNames() whereas SFTP
returns just the filename. mget logic assumed SFTP behavior.
Add a test to see if the filename already starts with the remote
directory and remove it before calling get() (which assumes just
the filename).
Includes an @Ignored test for FTP and SFTP that runs an mget
against a server.
INT-1141 polished code
INT-1141 added tests
INT-1141 improved how iterator is obtaind from OrderAwareLikedHashSet
INT-1141 polishing
INT-1141 stashed commit with List-based collection and performance tests
INT-1141 improved OrderedAwareLinkedHashSet
INT-1141 polishing
INT-1141, INT-2627 improved LoadBalancingStrategy and
RoundRobinLoadBalancingStrategy to obtain handler's iterator faster
INT-1141 improved getHandlerIterator method to ensure that it only executes reordering logic if there are more then one handler
INT-1141 simplified OrderedAwareLinkedHashSet to not to extend from LinkedHashSet
INT-1141
changed OrderedAwareLinkedHashSet to OrderedAwareCopyOnWriteArraySetTests, imporoved array creation in RoundRobinLoadBalancingStrategy
INT-1141 polished failing tests and RoundRobinLoadBalancingStrategy. Removed 'transient' from OrderedAwareCopyOnWriteArraySet
XSD refactoring: remove use="required" from 'request-channel' attribute of all 'outbound-gateways'
Tests for all 'outbound-gateways' inside the <chain>
Add re-init logic for nested chains
Add logic about nested element for AbstractChannelAdapterParser
Refactor of DefaultOutboundChannelAdapterParser
Test for non-last nested chain with some outbound-channel-adapter
Improve XSD for chain-type
Manual outbound-channel-adapter ability for chain
Integration tests for all outbound-channel-adapter within <chain>
Remove redundant 'return-value-required' attribute from <stored-proc-outbound-channel-adapter>
Add support 'expectReply' for FileWritingMessageHandler
INT-2275 polishing & refactor FileOutbound*Parser
HttpRequestExecutingMessageHandlerTests polishing
INT-2275: polishing JavaDoc
INT-2498:Added use-temporary-file-name to (s)ftp outbound adapters
INT-2489 polished parser to eliminate any value validation
INT-2489 removed trailing whitespaces from schema files
INT-2489 added boolean flag to detect if 'temporaryFileSuffix'
was explicitly set so the warning message is only be displayed
if renaming is turned off, but a suffix provided.
INT-2489 polishing
The AbstractChannelAdapterParser creates an implicit DirectChannel
if the adapter has no 'channel' attribute.
The Event, TCP, and UDP channel adapter parsers did not bind
this channel to the adapter and AC initialization failed with
'outputChannel is required'.
Further, the event schema marked the channel as being 'required',
precluding this feature.
INT-2407 Remove Channel use="required"
Parsers automatically generate the channel when none is provided.
- JMX
- JDBC
- SFTP
- Redis
- Feed
- XMPP
- Mail
- FTP
- HTTP
This is a significant update to the build system, including the changes
listed below. README.md has been updated with instructions on the most
important day-to-day commands.
- Eliminate buildSrc submodule
In favor of using the new bundlor and docbook-reference plugins. The
net effect is a large reduction in number of lines of build code.
Common docbook resources, stylesheets, etc are stored directly in the
docbook plugin.
This means that --recursive is no longer required when cloning and
there will never be a need to use `git submodule` commands. README
files have been updated to reflect.
Use of the new bundlor plugin also means the removal of template.mf
files from the source tree in favor of an inline approach. See
build.gradle for details. Bundlor 'import templates' are built up
programmatically and kept physically close to gradle dependency
declarations, leading to more convenience when changing these values
and hopefully fewer errors / version inconsistencies over time.
Certain tests depended on the presence of template.mf files, all of
which have recently been removed from the source tree in favor of the
new bundlor plugin which allows for inlining bundlor configuration
within the Gradle build script. These tests now create temp files
using the java.io.File API instead.
- Upgrade to Gradle 1.0-milestone-6
The m6 release is significantly faster when resolving dependencies
and has a number of valuable new features over the earlier m3
version. Review the release notes for Gradle 1.0-milestone-6 online
for full details.
- Switch to repo.springsource.org repository
Previously the project build declared as many repositories as
necessary to resolve all project dependencies.
Now depending on a single 'virtual repository' defined within the
SpringSource Artifactory instance at http://repo.springsource.org.
Currently, the virtual repository in use is 'libs-milestone', which
allows for the resolution of all "milestone-or-better" versions of
all S2 and third-party dependencies.
Should snapshot dependencies become required, this value may be
changed from 'libs-milestone' to 'libs-snapshot'. To build only
against GA releases, change the value to 'libs-release'.
- New build plan(s)
Spring Integration build plans have been updated to use the
Artifactory Bamboo plugin and publish to repo.springsource.org.
Build plans have names like 2.1.x to reflect the version under
development, not necessarily the name of the branch, as this may
change over time and across major releases.
- Improve release process
As mentioned above, Spring Integration will now use the Artifactory
Bamboo plugin to publish releases and also use Artifactory's support
for pushing builds directly into Maven Central via oss.sonatype.org.
Generate poms that contain all necessary fields for onboarding at
Maven central (scm, developers, organization, licenses, etc).
Generate -source and -javadoc poms to comply with Maven Central
onboarding rules (and for general good practice anyway).
Generation of PGP signatures, sha1 and md5 checksums are all handled
automatically by Artifactory. These are also requirements for
automated entry into Maven Central.
- Remove source-level pom generation
Automatic generation of Maven poms suitable for use in building
Spring Integration is no longer supported. Generation and
publication of poms for the purpose of dependency management remains
supported.
Sonar support has to date depended on these poms, but will be
switched over to use the Gradle Sonar plugin shortly.
- Eliminate docs subproject
Move docs/src to the root of the project and eliminate docs as a
formal subproject. This simplifies the build in a number of ways,
including removing the need for distinguishing between 'subprojects'
and 'javaprojects' as well as allowing users to build both 'api' and
'reference' docs without qualifying with a ':docs' prefix.
Also rename the src/info directory to src/dist to better reflect that
these files are packaged with the distribution. For example, the
readme.txt there is really the distribution readme, distinct from the
README.md at the root of the project which is for building from source,
etc.
added support for specifying 'temporary-remote-directory' as well as 'temporary-remote-directory-expression'
INT-2213 polishing
INT-2213 polishing docs
