Previously, `MetadataStore` couldn't be configured for Twitter Adapters
- only a global one could be used.
The `metadataKey` was generated automatically with a 'difficult' value.
* Register all `MessageSource` for `SourcePollingChannelAdapter`
as beans with id based on adapter id and prefix '.source' (INT-3147)
* Polishing parser to get rid of explicit `MessageSource` beans. (INT-3147)
* Make Feed and Twitter adapters `id` attribute as required -
now it presents a `metadataKey` for `MetadataStore` (INT-3147)
* Add to Twitter adapters a reference attribute for `MetadataStore` (INT-3173)
* Add Twitter adapters `poll-skip-period` attribute (INT-3167)
* Add and implement `MetadataStore#remove` (INT-1941)
* Make `MetadataStore` as `@ManagedResource` (INT-1941)
* Polishing tests
JIRAs:
https://jira.springsource.org/browse/INT-3147https://jira.springsource.org/browse/INT-3167https://jira.springsource.org/browse/INT-3173https://jira.springsource.org/browse/INT-1941
INT-3147: Polishing and fixes
* add domain suffix to `metadataKey`
* change contract of `MetadataStore.remove`
* remove timeout window from `AbstractTwitterMessageSource`
* polishing and fix `SearchReceivingMessageSourceWithRedisTests`
INT-3147: Rebasing and polishing
INT-3147: fix 'metadata' package tangle
INT-3147 Doc Polishing
Wrap `FileInputStream` and `FileOutputStreams` with
`BufferedInputStream`, `BufferedOutputStream` respectively.
Also use `BufferedReader`, `BufferedWriter` to wrap `FileReader`, `FileWriter`.
JIRA: https://jira.springsource.org/browse/INT-2953
https://jira.springsource.org/browse/INT-2961
onInit() relied on subclasses calling super.onInit() to apply the
advice chain and add the BeanFactory to the MessagingTemplate.
At least one extension (smpp) failed to do this, disabling these features.
Make onInit() final and call doInit() for subclass initialization (no-op in ARPMH).
INT-2961 Polishing
- do not increase visibility of doInit in subclasses.
- remove final modifiers from doInit so user subclasses can participate in initialization if needed.
When recursing, the returned filenames are relative to
the top level directory.
INT-3172 (S)FTP - Add Recursion to MGET Command
https://jira.springsource.org/browse/INT-3172
INT-3172 Polishing
- PR Comments
- Add Tests for SFTP (tested with Mock and real SSH)
- Enhance test to ensure subdir is in 3rd file retrieved
- Exclude "special" directories ('.' and '..') from recursion
INT-3172 Make FtpServerRule Safe For Concurrency
Previously the port was static.
- Make the port an instance variable
- Channel the `@ ClassRule` to a `@ Rule`
- Register an appropriate DefaultFtpSessionFactory bean declaration
INT-3172 Polishing; Docs
- Rename FtpServerRule to TestFtpServer
- Remove restriction on filters for MGET
- Add test for filtered MGET
- Reference docs, what's new
INT-3172 More Polish
- Fix type in reference
- Fix error message now that filter is allowed with MGET
- Remove test for filter with MGET (now valid)
INT-3172: Fix tests and typos
INT-3172: this.remoteFileSeparator for recursion
* Add `local-directory-expression` to (S)FTP Outbound Gateways
* Add `FtpServerRule` to Apache Mina embedded FtpServer
* Add tests for (M)GET and `local-directory-expression`:
FTP tests uses `FtpServerRule`, SFTP tests need testing on real SFTP server
JIRA: https://jira.springsource.org/browse/INT-2866
INT-2866: Documentation
INT-2866 Polishing
- Remove leading /
- Change \ to / in invalid test
- Clean up after sftp
Tested with real SSH.
INT-2866 Polishing - Add Mock SFTP Test
Run with -Dspring-profiles-active=realSSH to run with a real SSH server.
Assumes ftptest/ftptest account on localhost with the following directory tree in the user's root...
$ tree sftpSource/
sftpSource/
├── sftpSource1.txt
├── sftpSource2.txt
└── subSftpSource
└── subSftpSource1.txt
INT-2866: Polishing
INT-2866: change `remotePath` to `remoteDirectory`
Doc Polishing.
byte[] and String payloads were written to a temporary file.
If the same message is transferred to two different destinations
concurrently, one of the adapters could remove the temporary
file while the other was using it.
Don't use a temporary file for these payloads, simply use the
payload byte[] [or a String.getBytes()] as the InputStream
passed to the Session.write() method.
Polishing
Polishing - Add WARN For Missing File
+ Test
JIRA: https://jira.springsource.org/browse/INT-3091
Use the Spring Framework stylesheet for javadocs.
Fix 3 missing package-info files.
(Aside: remove cache-sessions from 2 @Ignored test case configuration files).
* Add XSD attributeGroup `smartLifeCycleAttributeGroup`
* Use it for all adapters tags
* Refactoring `AbstractChannelAdapterParser` hierarchy
* Refactoring some `MessageHandler`'s that should not have `autoStartup` & `phase`.
It is the responsibility of `AbstrctEndpoint`
* Polishing some tests according new logic
JIRA: https://jira.springsource.org/browse/INT-2998
INT-2998: Polishing
Polishing
- Reformat schemas where indentation changed
- Add SLC to the UDP inbound adapter
* Add `local-filename-generator-expression` attribute to FTP and SFTP Outbound Gateway
* Refactoring for `(S)FtpOutboundGatewayParser`
* Add `local-filename-generator-expression` tests
* Add 'What's New' and attribute description
JIRA: https://jira.springsource.org/browse/INT-3129
Polishing
Doc Polishing
Assert attribute is only set for get/mget
Move tests to the get gateways
There were several "helper" classes where the context was initialized
in the constructor, before the BeanFactory was passed in.
Generally fixed by adding the BeanFactory to the constructor args.
Other cases where the container-managed bean instantiated a helper
and never passed in the BeanFactory.
Finally, a fix to ExpressionUtils where the caller had a BeanFactory
but the BF did not contain an EvaluationContext factory bean, the
BeanResolver was not set up. This is unlikely in a Spring Integration
application, but added for completeness.
INT-3115 Add a BeanFactory to Test Cases
Change the WARN log in ExpressionUtils to a fatal exception to detect
cases where an EvaluationContext was created without a BeanFactory.
While this was generally in test cases, it also exposed some cases
in code where the context was initialized without a BF.
polishing on merge
Allow modification of context property accessors and
functions.
Polishing
- Fix tests to ensure the EvaluationContextFactoryBean is
available when necessary.
Polishing - Use Utility for ALL EvaluationContexts
Add Test Showing Custom Function
See SpelTransformerIntegrationTests-context.xml
Polishing
Polishing -fix Remote Sync
Polishing - Emit WARN if No BeanFactory
Polishing - Do Not Override ConversionService
Kludge to Prevent Warning When No BeanFactory
Expressions for method invoking in the MessagingMethodInvokerHelper
don't need a BeanFactory - suppress the warning.
Polishing - PR Comments - Invert Boolean
The SimplePool maintains an 'allocated' set, for the sole reason
of preventing a "foreign" (non-managed) object being returned.
When a pool item is detected as stale, it is removed from the pool
but remains in the 'allocated' set.
Add a test to verify the allocated size is reduced when a stale
item is popped from the pool.
Add a FileTransferringMessageHandler test (where the problem was
discovered).
Fix a test in TCP to expect a close().
Polishing
Do not allow returning null items to just release a permit - it
cannot remove the item from allocated. Clients must return the
stale item to the pool so it can be refreshed on the next get.
Only used by the TCP caching CF when returning a connection when
the factory is not running; but should not be allowed.
Also, protect against double release - not currently an issue with
existing users of SimplePool, but should be protected against. Could
cause the permit count to exceed the pool size.
Add inUse Set to the pool so we can detect attempts to release an
item that has already been released.
* Add `requires-reply` attribute for all adapters outbound gateways as `true` by default
* WS-outbound-gateway is still without it, because it has its own specific attribute `ignore-empty-responses`
* Make `requires-reply` as `false` by default for `jdbc:stored-proc-outbound-gateway` inasmuch as `jdbc:stored-proc-outbound-adapter`
doesn't have ability to configure `returning-resultset`
* add parser tests for `requires-reply`
JIRA: https://jira.springsource.org/browse/INT-2822
INT-2822 'requires-reply' for ws:outbound-gateway
Default false.
INT-2822: Polishing after rebase
INT-2822: deprecate 'ignore-empty-responses'
* Add 'requires-reply' section into What's New
INT-2822: remove 'ignore-empty-responses' from RM
INT-2822: Polishing after rebase
INT-2822: Rebased and polished
INT-2822: Rebased and polished
Add WARN within deprecated `AbstractWebServiceOutboundGateway#setIgnoreEmptyResponses`
Revert 'ignore-empty-responses'; Doc Polishing
The 'cache-sessions' attribute was deprecated in 2.1 in favor
of using a configured CachingConnectionFactory to provide more
flexibility, such as setting the cache size.
Remove the attribute from the schemas.
Update tests.
Add docs.
Update Migration Guide on the project Wiki.
Allow an application to be informed of problems on the IMAP idle
thread by emitting an event containing the exception.
Introduce IntegrationApplicationEvent hierarchy for all
events emitted by SI components.
INT-3099 Polishing (PR Comments)
Separate TCP events into discrete subclasses.
When writing files with this mode setting, the FWMH detected
that the final file already exists, but did not detect that
the temporary (default ....writing) file exists. This could
happen if two adapters were simulataneously processing the same
file.
Also, while a comment in the setter said an empty temp file
suffix is allowed, it did not work because the file rename
is performed unconditionally, fails, and throws an exception.
Add a check for the temporary file (if the suffix is not "").
Don't rename if the file was written in-place (no temp file suffix).
Add test cases for the FileExistsMode.IGNORE for both the final
and temporary files. Add a test where the temporary file suffix
is "", a ...writing file exists, the final file does not exist
and so is created by the handler. The ...writing file is untouched.
- Needs to implement SmartLifecycle and delegate to the adapter
- Needs to implement ApplicationEventPublisherAware and configure the adapter
- Don't set TaskExecutor if null
* Introduce XSD 'id' attribute for all `<chain>` sub-components
* Register `<chain>` sub-components as Beans with names based on `<chain>` 'id'
* Generate aliases for sub-components based on element 'id'
* Add `<chain>` parser test to check sub-components as Beans within `BeanFactory`
* Ignore `NestedChainParserTests` as abnormal and confused for SI-configs
* Now `<chain>` sub-components are trackable by `MessageHistory`
* Now `<chain>` sub-components are eligible for JMX export
* Polishing some failed tests
https://jira.springsource.org/browse/INT-2755https://jira.springsource.org/browse/INT-2321
INT-2755: Register as Beans only if 'id' provided
INT-2755: Set 'componentName' for handlers with 'id'
When components within `<chain>` are configured with an 'id' attribute
the 'componentName' of the handler is constructed as:
chainHandler.componentName + '.' + handler.componentName.
Otherwise it is as was before.
INT-2755: Polishing
INT-2755: changes according PR's comments
* Add INFO about preference of `id` attribute for `<chain>` in the `ChainParser`
* Change `<chain>`'s sub-components `id` generation algorithm
* Configure `<chain>`'s sub-component `componentName` in the `ChainParser` instead of in the `MessageHandlerChain`
* Add `componentName` property to the `JpaOutboundGatewayFactoryBean`
* Add a note to the Reference Manual
INT-2755: Localize duplicate 'id' within chain
INT-2755: remove child beans aliasing
* Polishing tests
* Polishing docs
INT-2755 Polishing: Handle Nested Chain Bean Names
Previously named beans within a nested chain did not get unique
bean names.
INT-2755: Polishing nested chains
Doc Polishing
Two implementations:
- native 'tail' command (e.g. tail -F -n 0 /foo/bar)
- Apache commons-io Tailer
Further changes:
- Add documentation to section "what's new"
- Add a JUnit @Rule so the OSDFTMP test doesn't fail on Windows
* (S)FTP Change Commands, Options to Enums
* INT-2981 Add Support For 'mv' To File/Remote/GW
* INT-2981 (S)FTP Namespace Support and Docs
* INT-2981 Create Remote Dirs if Needed
* INT-2981 Add 'mv' Command to What's New
* INT-2981 Polishing - PR Comments
* Add javadocs to enums.
* Polish schemas to use an enumerated type for available gateway commands
INT-2964 - JavaDoc: <emphasis> should be <em>
INT-2964 - JavaDoc: <p/> should be <p>
INT-2964 - JavaDoc polishing
* <li></li> should be wrapped in <ul></ul>
* wrap code snippets in {@code myCode()}
* change <code>false</false> to <code>false</code>
INT-2964 - Polish - Fix more JavaDoc errors
* Mockito-all dependency causes JavaDoc error. Change dependency to Mockito-Core.
When autocreateDirectory is true, and the top level directory
did not exist, the mkdirs failed because the adapter tried
to create a directory name with a zero length string.
This caused an 'String index out of range: 0' exception in
SFTP.
Stop iteration when 'pathSegment' is zero length.
Copy schemas, cleanup whitespace, update spring.schemas files,
update version in AbstractIntegrationNamespaceHandler.
For each schema compared the 2.2 version with the 3.0 version
(diff -w), ensuring the only difference is the import of
3.0 schemas (where appropriate) instead of 2.2.
The final remote directory expression processor was being
used instead of the temporary directory expression processor.
Add a test to confirm the correct processors are used.
Remove the need for pseudo-transactional element
INT-2727 PseudoTX
Add PseudoTransactionalTransactionManager
INT-2727
addressed PR comments
cherry picked previous code for mail module to eliminate breaking change
INT-2727
initial refactoring pseudo-tx support to use common configuration
INT-2727
finalizing pseudo-tx synchronization support
INT-2727 polishing
INT-2727 polishing based on PR comments
INT-2727 addressed PR comments
INT-2727 polishing
INT-2727 Remove PseudoTransactionalMessageSource
Instead of getResource, bind the resource holder before
receive() and then add attributes to the holder.
INT-2727 polishing
INT-2727 Polishing
Remove bind of #resource; add beforeCommit() test;
add TransactionTemplate tests.
* Remove hard-coded XML Schema versions in *FileInboundTransactionTests-context.xml*
* Convert spaces-to-tabs in *FileInboundTransactionTests-context.xml*
For reference see: https://jira.springsource.org/browse/INT-2708
Add general capability to advise just the handleRequestMessage
part of an AbstractReplyProducingMessageHandler.
This is to advise just the immediate operation, and not the
entire downstream flow.
Uses include:
* outbound gateway post processing
* adding retry behavior using spring-retry
* adding circuit breaker functionality
Initial commit for review.
Also need to advise simple message handlers (such as file
etc) to allow them to post-process file operations
with payload.delete(), payload.renameTo(...) etc.
INT-2250 Add Circuit Breaker Advice
INT-343 Add Retry Advice
Stateless and Stateful retry using spring-retry. Stateless
means the RetryTemplate performs the retries internally.
Stateful means the exception is thrown (e.g. to JMS container)
and the retry state is maintained by spring-retry.
INT-2215, INT-343, INT-2250 Refactoring
Factor out common abstract Advice class.
INT-2214 Catch Evaluation Expression Exceptions
If an onSuccess expression evaluation fails, add an
option so the user can decide whether such an exception is
caught, or propagated to the caller.
INT-2214 etc PR Review Polishing
INT-2214 etc Namespace Core, File, FTP
Add <request-handler-advice-chain/> to outbound endpoints.
INT-2214 etc. More Namespace Support
amqp, event, gemfire, groovy, http, ip, jdbc, jms, jmx, jpa, mail, rmi, sftp, twitter, ws, xmpp
INT-2214 etc Polishing
PR Review
INT-2214 etc Polishing
Don't catch Throwable.
Move Advice classes to handler.advice package.
Remove M3 disposition-* attributes on File/(S)FTP inbound adapters.
Add <pseudo-transactional/> and <transaction-synchronization/> elements
to <poller/>.
These elements provide the following attributes:
* on-success-expression
* on-success-result-channel
* on-failure-expression
* on-failure-result-channel
* send-timeout
<transaction-synchronization/> synchronizes these expression evaluations
with the transaction, such that they are executed immediately after
the commit/rollback.
<psuedo-transactional/> is used for a non-transactional poller.
When an <advice-chain/> is provided to the poller, <psuedo-transactional/>
and <transaction-synchronization/> are synonyms; and the behavior is
dictated by whether or not the <advice-chain/> contains a transaction
advice. It is recommended that <pseudo-transactional/> is used when the
<advice-chain/> does not have a txAdvice, and <transaction-synchronization/>
when it does, but the framework does not enforce this.
The expressions have the original (polled) message as the #root variable.
In addition, a BeanResolver is provided, allowing expressions such as
'@someBean.handleSuccess(payload)'.
MessageSources may also implement PseudoTransactionalMessageSource. This
has a number of methods allowing more flexibility in transactional and
non-transactional environents. For example, for backwards compatibility.
the mail-inbound-channel-adapter deletes its polled message after the
receive() rather than after the polled message is sent (when running in
a non-transactional poller). However, when running in a transactional
poller, the delete is done after the transaction commits (but not when
it rolls back).
In addition, MessageSources that implement this interface can optionally
provide an arbitrary object to the success/failure expressions in a
variable named '#resource'.
INT-2685 Polishing
PR Review Comments
Add tests for non-tx PseudoTransactionalMessageSource
FileReadingMessageSource supports disposition of the payload
after the message is sent (or via transaction synchronization).
The (S)FTP adapters delegate to an FRMS; add support for the
file disposition expression, result channel, and send timeout
to the (S)FTP adapters.
Also tested with ftp sample - INTSAMPLES-83 - patch will be
committed once this is in a milestone.
FTP servers return full path from listNames() whereas SFTP
returns just the filename. mget logic assumed SFTP behavior.
Add a test to see if the filename already starts with the remote
directory and remove it before calling get() (which assumes just
the filename).
Includes an @Ignored test for FTP and SFTP that runs an mget
against a server.
* Add *disposition-expression* to the inbound file adapter. This allows for operations such as *payload.delete()*, *payload.renameTo()* after the file is processed.
* The result of executing the expression (if any) is sent to the disposition-result-channel.
* Add documentation directory-expression
* Add documentation for auto-create-directory attribute
* Refactor File Adapter chapter slightly so that it flows better
For reference see: https://jira.springsource.org/browse/INT-2618
INT-2618 - Doc updated due to changes in INT-805
* rebased
* clarified the file section in the reference doc
Add capability to append to a file, if it already exists, to File Outbound Channel Adapter.
The new attribute is 'append' and is mutually exclusive with 'temporary-file-suffix' since
when appending to a file we can no longer write to a temporary file and then rename it.
INT-2470 addressed PR comments
INT-2470 PR Comments
INT-2470 polishing
changed logic behind File copy when one file needs to be appended into another
INT-2470 added WhileLockedProcessor
INT-2470 polishing
XSD refactoring: remove use="required" from 'request-channel' attribute of all 'outbound-gateways'
Tests for all 'outbound-gateways' inside the <chain>
Add re-init logic for nested chains
Add logic about nested element for AbstractChannelAdapterParser
Refactor of DefaultOutboundChannelAdapterParser
Test for non-last nested chain with some outbound-channel-adapter
Improve XSD for chain-type
Manual outbound-channel-adapter ability for chain
Integration tests for all outbound-channel-adapter within <chain>
Remove redundant 'return-value-required' attribute from <stored-proc-outbound-channel-adapter>
Add support 'expectReply' for FileWritingMessageHandler
INT-2275 polishing & refactor FileOutbound*Parser
HttpRequestExecutingMessageHandlerTests polishing
INT-2275: polishing JavaDoc
Pool based on algorithm used for spring-integration-file
CachingSessionFactory introduced by INT-2146. Refactored
that code to use the common SimplePool.
One difference to the previous implementation is the
ability to change the pool size dynamically.
If the size is reduced and more than the new size are
in use, items are closed as they are returned until
the pool size is as requested.
Initial commit.
Allow Pool Size Changes
Factor out Pool
Polishing
Pool Tests
Default forever
Javadocs, File
Polishing
INT-1871 PR Polishing
* Consistent/cleaner method names
* Track checkouts; reject release of 'foreign' objects.
* Add 'getAllocatedCount()'
