* GH-3090: Add `logout() to `FtpSession.close()`
Fixes https://github.com/spring-projects/spring-integration/issues/3090
Without `logout()` the FTP session is not closed at all,
but just the connection is closed.
Some FTP servers close those sessions eventually anyway, but some just
leak with resources.
**Cherry-pick to 5.1.x & 4.3.x**
* * Migrate `SessionFactoryTests` to JUnit 5
Fixes https://github.com/spring-projects/spring-integration/issues/3026
**Cherry-pick to `5.1.x`**
* Populate proper `FileTransferringMessageHandler` impl from DSL spec
implementations.
This way we are able to use a provided `chmod` from Java DSL
* Added `FileTransferringMessageHandlerSpec` ctor TODO
* Update SftpTests
* Code cleanup; `@Ignore` `SftpTests.testSftpOutboundFlowWithChmod()`
since it doesn't work properly on Windows
Related to: https://build.spring.io/browse/INT-MASTERSPRING40-826/
If we don't close application context on the embedded (S)FTP server,
we my still have some session opened in other tests, which would cause
some unexpected behavior like extra `SessionClosedEvent` in the
test for FTP events
Fixes https://github.com/spring-projects/spring-integration/issues/3038
* Add more delegating setters into the `AbstractRemoteFileOutboundGateway`
for its `RemoteFileTemplate` property
* Restrict externally provided `RemoteFileTemplate` from modifications
in the `AbstractRemoteFileOutboundGateway`: those options must be
configured on that external `RemoteFileTemplate`
* Expose new options in the Java DSL specs
* GH-3043: Add FileHeaders.REMOTE_HOST header
Fixes https://github.com/spring-projects/spring-integration/issues/3043
* Populate a `FileHeaders.REMOTE_HOST` from the
`AbstractRemoteFileStreamingMessageSource` and "get"-based commands
in the `AbstractRemoteFileOutboundGateway`
* Extract the value from the a `Session.getHost()` contract
* The `AbstractInboundFileSynchronizingMessageSource` cannot be
addressed with this because the real message is already based on the
locally stored file
* Adjust some affected tests according our code style requirements
* * Add remote file info support into `AbstractInboundFileSynchronizingMessageSource`
* Introduce a `MetadataStore` functionality into the `AbstractInboundFileSynchronizer`
to gather a remote file info an save it in the URI style against local file
* Retrieve such an info in the `AbstractInboundFileSynchronizingMessageSource`
during local file polling
* Introduce `protocol()` contract for the `AbstractInboundFileSynchronizer`
to build a proper URI in the metadata for external readers to distinguish
remote files properly
* Document the feature
* * Fix some typos in Docs
* * Rename property and header constant to the `HOST_PORT` pair
* Fix typos in Docs
* Add `remote-file-metadata-store` and `metadata-store-prefix` into XSD
of (S)FTP Inbound Channel Adapters
* Add `remoteFileMetadataStore` and `metadataStorePrefix` options
into `RemoteFileInboundChannelAdapterSpec` for Java DSL
Fixes https://github.com/spring-projects/spring-integration/issues/3027
This is a polishing for PR https://github.com/spring-projects/spring-integration/pull/3029
* Move `KeyDirectory` in to the `RotationPolicy` since it looks like fully
coupled concept with that abstraction and the class name is so bad for
the public API
* Remove `AbstractStandardRotationPolicy` in favor of a `StandardRotationPolicy`
extendability
* Change `ftp.adoc` and `sftp.adoc` to reflect a new API reality
* Mantion these changes in the `whats-new.adoc`
- republish certain events as `ApplicationEvent`s.
* * Add ApacheMinaFtplet to provide the same functionality with FTP
* Fix typo
* * Polishing javadocs and event toString() methods
Fixes https://github.com/spring-projects/spring-integration/issues/2748
* Refactor more `MessageHandlingException`s to include `this` into an
exception message
* Revert using `MessagingException` in some places which really are not
about messaging.
This helps to wrap them into `MessageHandlingException` later in the
`MessageHandler` for the `BeanDefinition` reference
* Remove `volatile` from configuration properties in the affected
classes
* Remove already deprecated `JmsOutboundGateway.setPriority()`
* Add `resource` and `source` for `BeanDefinition` in the
`AbstractChannelAdapterParser` & `AbstractInboundGatewayParser`
* Document the feature
Fixes https://github.com/spring-projects/spring-integration/issues/2987
To resolve XSD files properly from the classpath, their HTTPS reference
must be present in the `spring.schemas` to avoid the Internet interaction
for resolving an XSD file
**Cherry-pick to 5.1.x, 5.0.x & 4.3.x**
* Do not fix urls for the `http://apache.org/xml/features` when we
configure XML components
* Do not fix urls for localhost and fake links in tests
* Fix all the dead links in docs
Fixes: https://github.com/spring-projects/spring-integration/issues/2820
The current `FtpRemoteFileTemplate.exists()` implementation obtains a
`Session` for the `NLST_AND_DIRS` mode, but doesn't close it.
* Fix the `FtpRemoteFileTemplate.exists()` method to delegate to `super`
for the proper interaction with a `Session`
**Cherry-pick to 5.1.x, 5.0.x & 4.3.x**
Resolves https://github.com/spring-projects/spring-integration/issues/2777
If the filter supports it, defer filtering until the last possible moment.
Then, the worst case scenario after a catastrophic failure (e.g. power loss),
would be that at most one file will be incorrectly filtered on restart.
Polishing and add more tests.
Polishing Javadocs
More Polishing
Final polishing
More polishing.
Polishing and docPolishing and docs.
* Fix typos in Docs
* Defer Messaging annotations process
The `AbstractMethodAnnotationPostProcessor` and its implementations
have a `beanFactory.getBean()` call for the `@Bean` methods with
Messaging annotations.
This is done, actually, from the
`MessagingAnnotationPostProcessor.postProcessAfterInitialization()`
which might be still too early in some scenarios, like Spring Cloud Feign
with its child application contexts being initialized from the
`FeignClientFactoryBean`, causing a `BeanCurrentlyInCreationException`
See https://stackoverflow.com/questions/54887963/beancurrentlyincreationexception-when-using-spring-integration-with-spring-cloud
* Implement a `SmartInitializingSingleton` for the `MessagingAnnotationPostProcessor`
and gather `Runnable` wrappers for newly introduced `postProcessMethodAndRegisterEndpointIfAny()`
to be called later in the `afterSingletonsInstantiated()` when context is
still in the initialization phase.
All runtime-registered beans are going to be processed normally from the
regular `postProcessAfterInitialization()`
**Cherry-pick to 5.1.x**
* * Fix unused imports in the `MessagingAnnotationsWithBeanAnnotationTests`
* * Fix `IntegrationEndpointsInitializer` in the testing framework to handle
all the possible `AbstractEndpoint` beans registration.
See its JavaDocs for more info
* Fix `AbstractCorrelatingMessageHandlerParser` and
`AbstractConsumerEndpointParser` to use bean names for `outputChannel`
and `discardChannel` instead of bean references.
Since `MessagingAnnotationPostProcessor` now registers endpoints and
beans for channels much later, than parsers, we can't rely on bean
references any more there.
* Fixes for failing tests which expected `outputChannel/discardChannel`
bean references, when it is already just their names for late binding.
* Apply some code style polishing for the affected classes.
* Add `@Nullable` for `MessageSelector` parameter in the `QueueChannel.purge()`
* Fix new Sonar smells
* Fix some old Sonar smells as well
* Fix Micrometer leaks in the `PollableChannel` when we register
meters, but don't remove them.
* * Fix NPE around `MetricsCaptor` in channels
* * Fix new smells according test report
* * Further Sonar smell fixes
* * More smell fixes for `MessagingMethodInvokerHelper`
* Remove `throws Exception` from `AbstractMessageHandler.destroy()`
* * Fix complexity in the `MessagingMethodInvokerHelper.processInvokeExceptionAndFallbackToExpressionIfAny()`
Mostly thanks to IDEA's plugin: https://plugins.jetbrains.com/plugin/10345-assertions2assertj
There is still a lot of work to do when complex and composite matchers are used.
* Add `awaitility` dependency and deprecate `EventuallyMatcher` in favor
of `awaitility`
* Remove Hamcrest from dependencies and disable JUnit & Hamcrest
static imports to encourage to use only AssertJ
* Migrate JUnit assumptions in rules to AssertJ's assumptions
* Deprecate some custom matchers in favor of existing in Hamcrest
after upgrading the last to version `2.1`
* Replace `ExpectedException` rules with `assertThatThrownBy()`
* Mention `MessagePredicate` in the `testing.adoc`
* Some file module Sonar fixes
* * Fixes after testing
* * Some further Sonar fixes for file module
* * Further Sonar fixes for file module
* * More Sonar fixes for file module
* * Fix `RemoteFileTemplate` issues
* * Fix new sonar smells after previous one
* * Review feedback
https://build.spring.io/browse/INT-FATS5IC-685/
Even if we stop `SourcePollingChannelAdapter`, the task on the fly is
still going to be executed.
This way we may have extra calls to our logic.
* Use `CopyOnWriteArrayList` for the `sessionSources` to avoid
`ConcurrentModificationException` during this collection read in the
tests
**Cherry-pick to 5.0.x**
* Add Reactive mode for AbstractPollingEndpoint
* When `SourcePollingChannelAdapter.outputChannel` is a
`ReactiveStreamsSubscribableChannel`, use `Flux.generate()` for polling
* Refactor `AbstractPollingEndpoint` to remove redundant `Poller` class
in favor of lambda
* Extract `pollForMessage()` method to handle TX states instead of
`Poller` class previously
* * Rebase and fix conflicts
* Polishing for GatewayProxyFactoryBeanTests
https://build.spring.io/browse/INT-MASTERSPRING40-458/
We need to stop Inbound Channel Adapter first and only then wait for
the latch.
Otherwise there is a chance that new item is added to the collection
in between `latch.await()` and `stop()`
**Cherry-pick to 5.0.x**
JIRA: https://jira.spring.io/browse/INT-4513
The `IntegrationManagementConfigurer` keeps metrics beans in its local
stores.
When we add beans at runtime, they are not removed from those caches
after their removal
* Implement `DestructionAwareBeanPostProcessor` in the `IntegrationManagementConfigurer`
and clean up caches according provided bean type and its name
* Improve `ManualFlowTests` and `FtpTests` to be sure that
`IntegrationManagementConfigurer` caches are cleared after destroying
`IntegrationFlowRegistration`
**Cherry-pick to 5.0.x**
* INT-4381: MessageSources refactoring
JIRA: https://jira.spring.io/browse/INT-4381
* Make all the out-of-the-box `MessageSource` implementations based
on the `AbstractMessageSource`
* Fix `JdbcPollingChannelAdapterIntegrationTests` for sporadic failure
because of `fixed-rate` for the poller
* Fix HeaderEnricherTests race condition
The `errorChannel()` expect an error in the `testErrorChannel` after
`1000` ms, but at the same time the `poller` in configured for the
`3000` ms.
* Increase all the timeouts for replies
* Decrease a `fixed-delay` on the `poller`
* Some other code style polishing for the `HeaderEnricherTests`
Windows is too scrupulous for non closed file resources.
Our `RemoteFileTestSupport` recreates a test directory with files
for each test.
When we don't close some file handler, we are not able to delete the
directory and we fail with subsequent tests.
* Close `InputStream` for each polled file in new tests in the
`SftpStreamingMessageSourceTests`
* Close `CLOSEABLE_RESOURCE` in new tests in the
`FtpStreamingMessageSourceTests`
* Rework `FtpStreamingMessageSourceTests.testAllContents()` do not
poll all the files on each polling cycle - this causes a race condition
when we don't close `CLOSEABLE_RESOURCE` yet in the `StreamTransformer`,
but try to proceed with recreation a test directory structure
**Cherry-pick to 5.0.x**
JIRA: https://jira.spring.io/browse/INT-4495
Previously, `maxFetch` was applied before directories were removed from
the fetch list.
Remove the directories before filtering and applying `maxFetch`.
* Polishing - PR Comments
* More polishing
* More polishing.
* Check for empty array.
* Remove test main method.
On Windows `File.delete()` causes to file to be modified first of all.
with the high polling interval (`fixedDelay(1)`) there is a chance
to pick the same file from the local directory again, since the
`AbstractPersistentAcceptOnceFileListFilter` is based on the `lastModified`
* Do not delete files manually at all - just rely on the `clean()`
with the `@Before`
**Cherry-pick to 5.0.x**
