* GH-3155: Add support for Java DSL extensions
Fixes https://github.com/spring-projects/spring-integration/issues/3155
Provide an `IntegrationFlowExtension` for possible custom EI-operators
in the target project use-cases.
* * Move `IntegrationFlowExtension` tests ot its own test class
* Make all the `IntegrationComponentSpec` ctors as `protected` for possible custom extensions
* Make some `BaseIntegrationFlowDefinition` methods and properties as `protected` to get them
access from the `IntegrationFlowExtension` implementations
* Document the feature
* * Fix language and typos in docs
* * Add `protected` to one more `GatewayEndpointSpec` ctor
* Add JavaDocs to `GatewayEndpointSpec` methods
* * Add `protected` to one more `JmsPollableMessageChannelSpec` ctor
Fixes https://github.com/spring-projects/spring-integration/issues/3132
It turns out that Checkstyle EmptyBlock doesn't complain about
empty default ctor.
Plus a new check for `super();` call treats it as a violation
* Remove `super();` from all the no-arg ctors
* Code style clean up in the affected classes according
IDEA suggestions
* Fix new Sonar smells
* Move `What's New` into `changes-5.1-5.2.adoc`
* Remove version from the `XSD` files:
1. Align with SF
2. The version for XSD doesn't matter for the current jar version,
since only the version is available in classpath is from the current
jar version
3. Modify `spring.schemas` to map all the possible versions for XSD
into the current one in a jar.
This way target applications can upgrade without changing the version
for XSD location and we don't need to require `versionless` variant
any more
4. The jar in classpath can handle only its own XSD, so independently
of version or no-version variants we still may fail because the current
jar may not support end-user code any more - need some fix over there
anyway.
* Remove `checkTestConfigs` Gradle task since we don't worry about XSD
version any more
* Modify `schemaZip` Gradle task to populate XSDs into a distribution
still with a version to avoid overriding on-line XSDs for version `1.0.x`.
We may consider not do that since SF doesn't and just bite a bullet
for always overriding those on-line XSDs to the actual one from the latest
release.
Fixes https://github.com/spring-projects/spring-integration/issues/3026
**Cherry-pick to `5.1.x`**
* Populate proper `FileTransferringMessageHandler` impl from DSL spec
implementations.
This way we are able to use a provided `chmod` from Java DSL
* Added `FileTransferringMessageHandlerSpec` ctor TODO
* Update SftpTests
* Code cleanup; `@Ignore` `SftpTests.testSftpOutboundFlowWithChmod()`
since it doesn't work properly on Windows
Fixes https://github.com/spring-projects/spring-integration/issues/3038
* Add more delegating setters into the `AbstractRemoteFileOutboundGateway`
for its `RemoteFileTemplate` property
* Restrict externally provided `RemoteFileTemplate` from modifications
in the `AbstractRemoteFileOutboundGateway`: those options must be
configured on that external `RemoteFileTemplate`
* Expose new options in the Java DSL specs
* GH-3043: Add FileHeaders.REMOTE_HOST header
Fixes https://github.com/spring-projects/spring-integration/issues/3043
* Populate a `FileHeaders.REMOTE_HOST` from the
`AbstractRemoteFileStreamingMessageSource` and "get"-based commands
in the `AbstractRemoteFileOutboundGateway`
* Extract the value from the a `Session.getHost()` contract
* The `AbstractInboundFileSynchronizingMessageSource` cannot be
addressed with this because the real message is already based on the
locally stored file
* Adjust some affected tests according our code style requirements
* * Add remote file info support into `AbstractInboundFileSynchronizingMessageSource`
* Introduce a `MetadataStore` functionality into the `AbstractInboundFileSynchronizer`
to gather a remote file info an save it in the URI style against local file
* Retrieve such an info in the `AbstractInboundFileSynchronizingMessageSource`
during local file polling
* Introduce `protocol()` contract for the `AbstractInboundFileSynchronizer`
to build a proper URI in the metadata for external readers to distinguish
remote files properly
* Document the feature
* * Fix some typos in Docs
* * Rename property and header constant to the `HOST_PORT` pair
* Fix typos in Docs
* Add `remote-file-metadata-store` and `metadata-store-prefix` into XSD
of (S)FTP Inbound Channel Adapters
* Add `remoteFileMetadataStore` and `metadataStorePrefix` options
into `RemoteFileInboundChannelAdapterSpec` for Java DSL
- republish certain events as `ApplicationEvent`s.
* * Add ApacheMinaFtplet to provide the same functionality with FTP
* Fix typo
* * Polishing javadocs and event toString() methods
Fixes https://github.com/spring-projects/spring-integration/issues/2987
To resolve XSD files properly from the classpath, their HTTPS reference
must be present in the `spring.schemas` to avoid the Internet interaction
for resolving an XSD file
**Cherry-pick to 5.1.x, 5.0.x & 4.3.x**
* Add `io.spring.nohttp` Gradle plugin into the build script and allow
it only on CI server, since it is going to parse every file
* Fix tests do not use `http` any more
* Fix `test.html` in the `mail` modules tests to exclude a Namespace for
OExchange XRD protocol
* Fix some files for new line in the end of file according Checkstyle
violations
While investigating an SI build failure, I noticed that there is
a timeout available for when connecting channels.
This won't resolve the build issue (where the session closed) but
worth adding as a property.
* Optimize DefaultSftpSessionFactory
Related to https://build.spring.io/browse/INT-MASTERSPRING40-677/
Doesn't look like `DefaultSftpSessionFactory.getSession()` needs
locking around `sharedJschSession`
* change the logic in the `getSession()` to store a `sharedJschSession`
into the local variable and if it is `null` or not connected, create a
new `JSchSessionWrapper`, connect it and store to the `sharedJschSession`
back into the `sharedJschSession` property if `this.isSharedSession`.
This way we always deal with `sharedJschSession` anyway if it is valid
or create a new fresh one if that is invalid.
Without locking we always get an actual state of the `sharedJschSession`
and don't fall into the race condition when `sharedJschSession` is invalid,
but we can't connect to the SFTP channel from the `sftpSession.connect()`
* * Wrap `sharedJschSession` initialization to the lock
* * Store `sharedJschSession` back when it is really fresh and while the lock
https://build.spring.io/browse/INT-MASTER-1481
The `SftpSession.connect()` may lead to race condition when we try to
open the `channel`, but `JschSession` is closed already.
It may happen in cases when we have `DefaultSftpSessionFactory`
configured for the `isSharedSession` and that shared session may be
closed by another thread before we reach the `channel.connect()`,
because we already have left the `this.sharedSessionLock` blocking path
**Cherry-pick to 5.1.x**
* Do not fix urls for the `http://apache.org/xml/features` when we
configure XML components
* Do not fix urls for localhost and fake links in tests
* Fix all the dead links in docs
* Enable ModifierOrderCheck Checkstyle rule
* Fix violations for `static` and `abstract` modifier
* Remove redundant code in the `TcpNioConnection`
* Mark `connectionFactoryName` as `@Nullable` in the `TcpConnectionSupport`
ctor and its inheritors
* Fix some smells according IDEA suggestions in the affected classes
* This should fix some Sonar smells as well
* * Fix `HeaderMapperTests`
* * Polishing `TcpConnection` code style and fix Javdocs
JIRA: https://jira.spring.io/browse/INT-4495
Previously, `maxFetch` was applied before directories were removed from
the fetch list.
Remove the directories before filtering and applying `maxFetch`.
* Polishing - PR Comments
* More polishing
* More polishing.
* Check for empty array.
* Remove test main method.
JIRA: https://jira.springsource.org/browse/INT-4334
* Add a `DirectoryScanner` setter named `setScanner` in
`AbstractInboundFileSynchronizingMessageSource` to set the
`FileReadingMessageSource`'s scanner.
* Add tests to `AbstractRemoteFileSynchronizerTests`.
Polish.
Updates xsd, tests and documentation.
Fix xsd formatting.
Addresses review comments.
* Some code style polishing
JIRA: https://jira.spring.io/browse/INT-4283
Filter that only passes files where a corresponding "marker" file is also present.
Implementations for file and(S)FTP.
* Polishing - PR Comments
* Missed one
JIRA: https://jira.spring.io/browse/INT-4060
* Add `NLST` command to the `AbstractRemoteFileOutboundGateway` to perform
`listNames` on the target session.
Useful in case of server doesn't allow to perform `LS` or the names set is
sufficient for application requirements
* Add `workingDirExpression` to the `FtpOutboundGateway` to allow to perform
`FtpClient.changeWorkingDirectory()` based on the current request message
* Change `slf4j-log4j12` to the `testCompile` -
the FTP tests fail in the IDE with `ClassNotFoundException`
Address PR comments
* Add `nlst` to XSD config
* Reinstate `ls -1` test-case for the `FtpServerOutboundTests`
* wrap more commands to the `doInWorkingDirectory()`
Fix `MV` command in the `AbstractRemoteFileOutboundGateway`
* Implement `RemoteFileOperations#invoke(OperationsCallback<F, T>)`
for thread-bound `session`s
* Use a new `invoke()` for `put()` and `mPut()` commands in the `AbstractRemoteFileOutboundGateway`
* Add delegation for the `put()` and `mPut()` commands in the `FtpOutboundGateway`
* Add DSL support for the `workingDirExpression` and add `whats-new.adoc` note
Document changes
Address some PR comments:
* Add `invokeScope` variable to `execute` to track `ThreadLocal` session or not
* Check for `null` in the `getSession()` and fallback to regular
`sessionFactory.getSession()`.
Most likely the `invoke()` is called from other thread
Doc Polishing
JIRA: https://jira.spring.io/browse/INT-4117
Add an option to set file permissions on the `FileWritingMessageHandler` when the file system supports it.
Polishing - PR Comments; more robust file system detection.
Add DSL support and fix NPE with missing timestamp header.
Fix javadoc
Fix javadoc
Some polishing:
* Add `chmod >= 0` assertion
* Make some assertion with string concatenation as `Supplier`-based to
deffer that string concatenation
JIRA: https://jira.spring.io/browse/INT-4115
Apply `(S)FtpPersistentAcceptOnceFileListFilter` for the `(S)FtpInboundFileSynchronizer` by default to avoid cases to sync the same remote files to the local directory again.
Especially when `localFileName` strategy is applied and we end up with new local files, but with the same remote content
Accept `(S)FtpPersistentAcceptOnceFileListFilter` for streaming adapters
Make `doSetFilter()` as `protected final`
Fix tests after rebase
Fix "What's New" after rebase
Compose `PersistentAcceptOnceFileListFilter` together with the regex or pattern filters
Document such a behavior
Address PR comments for formatting and typo
JIRA: https://jira.spring.io/browse/INT-4233
INT-4233: Add (S)FTP FileInfo Header (Streaming)
JIRA: https://jira.spring.io/browse/INT-4233
Add the complete file info as JSON (when Jackson or Boon available) to the message headers
when streaming inbound.
Provide a mechanism to configure Boon to provide similar output to Jackson.
Also allow subclasses to provide their own object mapper.
Also clean up after the AMQP DSL tests, and don't use a queue `foo`. I often have such a queue
with content; this caused tests to fail.
Use SimpleJsonSerializer
Doc Polishing
Polishing - PR Comments
More Polishing
* Polishing according latest PR comments
JIRA: https://jira.spring.io/browse/INT-4221
* Upgrade to those versions of Spring project dependencies which potentially will provide similar fix
**Cherry-pick to 4.3.x**
JIRA: https://jira.spring.io/browse/INT-4175
Fixes GH-1980 (https://github.com/spring-projects/spring-integration/issues/1980)
When the `isSharedSession` is used for the `DefaultSftpSessionFactory`,
there is some race condition window when we can call the target `this.jschSession.connect()` several times and end up with the session is already connected.
Wrap `sftpSession.connect()` to the `this.sharedSessionLock.readLock().lock()` when `isSharedSession` to protect from that race condition.
Note: there is no test for this change because it is pretty tricky to build barriers between threads for this kind of race conditions.
Especially after introduction `this.sharedSessionLock.readLock().lock()` around guilty code
**Cherry-pick to 4.3.x & 4.2.x**
Move `sharedJschSession` connect logic to the existing locking block
Fix mock test to reflect the current ConnectionFactory state
JIRA: https://jira.spring.io/browse/INT-4155
- Add streaming support
- Rework tests to use dynamic flow registrations due to the test FTP server is now a test support class.
Javadoc fixes
Checkstyle fix
Polishing - PR Comments
Polishing and Port SFTP
JIRA: https://jira.spring.io/browse/INT-3770,
https://jira.spring.io/browse/INT-4107
Having `TransactionHandleMessageAdvice` we can start TX from any `MessageHandler.handleMessage()`
* Add `<transactional>` alongside with the `<request-handler-advice-chain>` for those components which produce reply
* Merge `<transactional>` and `<request-handler-advice-chain>` configuration to a single `ManagedList`
* Rework JPA `<transactional>` in favor of common solution
* Some polishing and refactoring
AbstractPollingEndpoint: avoid `new ArrayList` if we don't have `receiveOnlyAdvice`s
Polishing - PR Comments and Closeable Warnings
Eclipse emits bogus warnings with exceptions in lambdas.
Even though the lambda might run on another thread, elipse thinks it could
cause the context to not be closed.
SPR-14854: MessageChannel is now a @FunctionalInterface
* Additional Lambda polishing and some code style fixes
