* GH-3376: Remove gauges on application ctx close
Fixes https://github.com/spring-projects/spring-integration/issues/3376
The `MeterRegistry` may request meters on application shutdown.
The gauges for channels, handlers and message sources don't make sense
at the moment since all those beans are going to be destroyed.
* Remove gauges for channel, handler and message source numbers from the
`IntegrationManagementConfigurer.destroy()`
**Cherry-pick to 5.3.x & 5.2.x**
* * Add `MicrometerImportSelector` to conditionally load
a `MicrometerMetricsCaptorConfiguration` when `MeterRegistry`
is on class path.
* Make `MicrometerMetricsCaptorConfiguration.integrationMicrometerMetricsCaptor()`
bean dependant on the `ObjectProvider<MeterRegistry>`
* Make `IntegrationManagementConfiguration.managementConfigurer()`
dependant on the `ObjectProvider<MetricsCaptor>`.
This way the `IntegrationManagementConfigurer` is destroyed before
`MeterRegistry` when application context is closed
* Deprecate `MicrometerMetricsCaptor.loadCaptor()` in favor of
`@Import(MicrometerImportSelector.class)`
* * Add `MicrometerMetricsCaptorRegistrar` to register a `MICROMETER_CAPTOR_NAME`
bean when `MeterRegistry` is on class path and no `MICROMETER_CAPTOR_NAME` bean yet.
* Make `IntegrationManagementConfiguration.managementConfigurer()`
dependant on the `ObjectProvider<MetricsCaptor>`.
This way the `IntegrationManagementConfigurer` is destroyed before
`MeterRegistry` when application context is closed
* Deprecate `MicrometerMetricsCaptor.loadCaptor()` in favor of
`@Import(MicrometerMetricsCaptorRegistrar.class)`
* Fix test to make a `MeterRegistry` bean as `static` since
`@EnableIntegrationManagement` depends on this bean definition now
# Conflicts:
# spring-integration-core/src/main/java/org/springframework/integration/config/EnableIntegrationManagement.java
# spring-integration-core/src/main/java/org/springframework/integration/config/IntegrationManagementConfiguration.java
# spring-integration-core/src/main/java/org/springframework/integration/config/IntegrationManagementConfigurer.java
* Fix some deprecation warnings
# Conflicts:
# spring-integration-core/src/main/java/org/springframework/integration/config/IntegrationManagementConfigurer.java
Fixes https://github.com/spring-projects/spring-integration/issues/3373
The `AbstractInboundFileSynchronizer` doesn't consider that
`hostPort` from `Session` could be in an IPv6 syntax
* Parse the `hostPort` from `Session` in a manner that only the last
`:` is treated as a port delimiter
**Cherry-pick to 5.3.x & 5.2.x**
* GH-3370: Remove synchronized from RemoteFileUtils
Fixes https://github.com/spring-projects/spring-integration/issues/3370
The `synchronized` on the `RemoteFileUtils.makeDirectories()` makes an application too
slow, especially when we deal with different paths in different sessions
* Remove the `synchronized` from that method and rework `SftpSession.mkdir()`
to return `false` when "A file cannot be created if it already exists" exception
is thrown from the server.
Essentially make an `exists()` call to be sure that an exception is really related
to "file-already-exists" answer from the server
**Cherry-pick to 5.3.x, 5.2.x & 4.3.x**
* * Re-throw an exception in the `SftpSession.mkdir()`
when error code is not `4` or remote dir does not exist
* * Check `session.mkdir()` result in the
`RemoteFileUtils` to throw an `IOException` when `false`
* * Fix mock test to return `true` for `mkdir` instead of `null`
Close the pool so that any sessions returned after the factory is
`destroy()`ed are closed.
* Call `removeAllIdleItems()` in `close()`.
* Close sessions in `SftpStreamingMessageSourceTests`.
**cherry-pick to all supported branches**
Fixes: https://github.com/spring-projects/spring-integration/issues/3249
When the `CachingSessionFactory` is configured with small enough pool
and it is very likely that dead lock may happen when `RemoteFileTemplate.send()`
is used.
The problem happens when we reach the `RemoteFileTemplate.exists()` call
which is done from the internal method called from already pulled from cache
`Session`
* Fix `RemoteFileTemplate` to use a `session.exists()` instead on the provided
into the method `Session`
* Demonstrate the problem in the `SftpRemoteFileTemplateTests.testNoDeadLockOnSend()`
**Cherry-pick to 5.2.x, 5.1.x & 4.3.x**
Fix RemoteFileOutboundGWTests for the proper mock
* GH-3247: Fix `SftpSession.exists` for error code
Fixes https://github.com/spring-projects/spring-integration/issues/3247
When there is no path on the SFTP server, a `ChannelSftp.SSH_FX_NO_SUCH_FILE`
error is returned in the thrown `SftpException`.
* Fix `SftpSession.exists()` to check for the `SSH_FX_NO_SUCH_FILE` to
return `false` and re-throw an exception otherwise
* Add mock test for `SftpSession.exists()`
* Add `org.mockito.AdditionalMatchers` to `checkstyle.xml` exclusions
**Cherry-pick to 5.2.x & 5.1.x**
* * Add exists tests against Mina embedded server
Fixes https://github.com/spring-projects/spring-integration/issues/3026
**Cherry-pick to `5.1.x`**
* Populate proper `FileTransferringMessageHandler` impl from DSL spec
implementations.
This way we are able to use a provided `chmod` from Java DSL
* Added `FileTransferringMessageHandlerSpec` ctor TODO
* Update SftpTests
* Code cleanup; `@Ignore` `SftpTests.testSftpOutboundFlowWithChmod()`
since it doesn't work properly on Windows
Related to: https://build.spring.io/browse/INT-MASTERSPRING40-826/
If we don't close application context on the embedded (S)FTP server,
we my still have some session opened in other tests, which would cause
some unexpected behavior like extra `SessionClosedEvent` in the
test for FTP events
Fixes https://github.com/spring-projects/spring-integration/issues/3038
* Add more delegating setters into the `AbstractRemoteFileOutboundGateway`
for its `RemoteFileTemplate` property
* Restrict externally provided `RemoteFileTemplate` from modifications
in the `AbstractRemoteFileOutboundGateway`: those options must be
configured on that external `RemoteFileTemplate`
* Expose new options in the Java DSL specs
* GH-3043: Add FileHeaders.REMOTE_HOST header
Fixes https://github.com/spring-projects/spring-integration/issues/3043
* Populate a `FileHeaders.REMOTE_HOST` from the
`AbstractRemoteFileStreamingMessageSource` and "get"-based commands
in the `AbstractRemoteFileOutboundGateway`
* Extract the value from the a `Session.getHost()` contract
* The `AbstractInboundFileSynchronizingMessageSource` cannot be
addressed with this because the real message is already based on the
locally stored file
* Adjust some affected tests according our code style requirements
* * Add remote file info support into `AbstractInboundFileSynchronizingMessageSource`
* Introduce a `MetadataStore` functionality into the `AbstractInboundFileSynchronizer`
to gather a remote file info an save it in the URI style against local file
* Retrieve such an info in the `AbstractInboundFileSynchronizingMessageSource`
during local file polling
* Introduce `protocol()` contract for the `AbstractInboundFileSynchronizer`
to build a proper URI in the metadata for external readers to distinguish
remote files properly
* Document the feature
* * Fix some typos in Docs
* * Rename property and header constant to the `HOST_PORT` pair
* Fix typos in Docs
* Add `remote-file-metadata-store` and `metadata-store-prefix` into XSD
of (S)FTP Inbound Channel Adapters
* Add `remoteFileMetadataStore` and `metadataStorePrefix` options
into `RemoteFileInboundChannelAdapterSpec` for Java DSL
- republish certain events as `ApplicationEvent`s.
* * Add ApacheMinaFtplet to provide the same functionality with FTP
* Fix typo
* * Polishing javadocs and event toString() methods
Fixes https://github.com/spring-projects/spring-integration/issues/2987
To resolve XSD files properly from the classpath, their HTTPS reference
must be present in the `spring.schemas` to avoid the Internet interaction
for resolving an XSD file
**Cherry-pick to 5.1.x, 5.0.x & 4.3.x**
* Add `io.spring.nohttp` Gradle plugin into the build script and allow
it only on CI server, since it is going to parse every file
* Fix tests do not use `http` any more
* Fix `test.html` in the `mail` modules tests to exclude a Namespace for
OExchange XRD protocol
* Fix some files for new line in the end of file according Checkstyle
violations
While investigating an SI build failure, I noticed that there is
a timeout available for when connecting channels.
This won't resolve the build issue (where the session closed) but
worth adding as a property.
* Optimize DefaultSftpSessionFactory
Related to https://build.spring.io/browse/INT-MASTERSPRING40-677/
Doesn't look like `DefaultSftpSessionFactory.getSession()` needs
locking around `sharedJschSession`
* change the logic in the `getSession()` to store a `sharedJschSession`
into the local variable and if it is `null` or not connected, create a
new `JSchSessionWrapper`, connect it and store to the `sharedJschSession`
back into the `sharedJschSession` property if `this.isSharedSession`.
This way we always deal with `sharedJschSession` anyway if it is valid
or create a new fresh one if that is invalid.
Without locking we always get an actual state of the `sharedJschSession`
and don't fall into the race condition when `sharedJschSession` is invalid,
but we can't connect to the SFTP channel from the `sftpSession.connect()`
* * Wrap `sharedJschSession` initialization to the lock
* * Store `sharedJschSession` back when it is really fresh and while the lock
https://build.spring.io/browse/INT-MASTER-1481
The `SftpSession.connect()` may lead to race condition when we try to
open the `channel`, but `JschSession` is closed already.
It may happen in cases when we have `DefaultSftpSessionFactory`
configured for the `isSharedSession` and that shared session may be
closed by another thread before we reach the `channel.connect()`,
because we already have left the `this.sharedSessionLock` blocking path
**Cherry-pick to 5.1.x**
* Do not fix urls for the `http://apache.org/xml/features` when we
configure XML components
* Do not fix urls for localhost and fake links in tests
* Fix all the dead links in docs
Resolves https://github.com/spring-projects/spring-integration/issues/2777
If the filter supports it, defer filtering until the last possible moment.
Then, the worst case scenario after a catastrophic failure (e.g. power loss),
would be that at most one file will be incorrectly filtered on restart.
Polishing and add more tests.
Polishing Javadocs
More Polishing
Final polishing
More polishing.
Polishing and docPolishing and docs.
* Fix typos in Docs
* Defer Messaging annotations process
The `AbstractMethodAnnotationPostProcessor` and its implementations
have a `beanFactory.getBean()` call for the `@Bean` methods with
Messaging annotations.
This is done, actually, from the
`MessagingAnnotationPostProcessor.postProcessAfterInitialization()`
which might be still too early in some scenarios, like Spring Cloud Feign
with its child application contexts being initialized from the
`FeignClientFactoryBean`, causing a `BeanCurrentlyInCreationException`
See https://stackoverflow.com/questions/54887963/beancurrentlyincreationexception-when-using-spring-integration-with-spring-cloud
* Implement a `SmartInitializingSingleton` for the `MessagingAnnotationPostProcessor`
and gather `Runnable` wrappers for newly introduced `postProcessMethodAndRegisterEndpointIfAny()`
to be called later in the `afterSingletonsInstantiated()` when context is
still in the initialization phase.
All runtime-registered beans are going to be processed normally from the
regular `postProcessAfterInitialization()`
**Cherry-pick to 5.1.x**
* * Fix unused imports in the `MessagingAnnotationsWithBeanAnnotationTests`
* * Fix `IntegrationEndpointsInitializer` in the testing framework to handle
all the possible `AbstractEndpoint` beans registration.
See its JavaDocs for more info
* Fix `AbstractCorrelatingMessageHandlerParser` and
`AbstractConsumerEndpointParser` to use bean names for `outputChannel`
and `discardChannel` instead of bean references.
Since `MessagingAnnotationPostProcessor` now registers endpoints and
beans for channels much later, than parsers, we can't rely on bean
references any more there.
* Fixes for failing tests which expected `outputChannel/discardChannel`
bean references, when it is already just their names for late binding.
* Apply some code style polishing for the affected classes.
* Add `@Nullable` for `MessageSelector` parameter in the `QueueChannel.purge()`
Mostly thanks to IDEA's plugin: https://plugins.jetbrains.com/plugin/10345-assertions2assertj
There is still a lot of work to do when complex and composite matchers are used.
* Add `awaitility` dependency and deprecate `EventuallyMatcher` in favor
of `awaitility`
* Remove Hamcrest from dependencies and disable JUnit & Hamcrest
static imports to encourage to use only AssertJ
* Migrate JUnit assumptions in rules to AssertJ's assumptions
* Deprecate some custom matchers in favor of existing in Hamcrest
after upgrading the last to version `2.1`
* Replace `ExpectedException` rules with `assertThatThrownBy()`
* Mention `MessagePredicate` in the `testing.adoc`
* Enable ModifierOrderCheck Checkstyle rule
* Fix violations for `static` and `abstract` modifier
* Remove redundant code in the `TcpNioConnection`
* Mark `connectionFactoryName` as `@Nullable` in the `TcpConnectionSupport`
ctor and its inheritors
* Fix some smells according IDEA suggestions in the affected classes
* This should fix some Sonar smells as well
* * Fix `HeaderMapperTests`
* * Polishing `TcpConnection` code style and fix Javdocs
* Spring AMQP 2.1 B-S and fix imports for moved classes
* Reactor GA
* SF-5.1 B-S and exclude `org.springframework` from all the
Spring Security dependencies
* the latest Hibernate and JPA and fix the test query for the
appropriate actual syntax
* latest SSHD and add required dependency for `sshd-sftp`
* latest Groovy and extra dependency for the `groovy-dateutil`
* latest Smack for XMPP and appropriate code fixes for compatibility
* Fix `StompIntegrationTests` to use `AFTER_EACH_TEST_METHOD`:
when session is disconnected on the server, we can't interact with it
anymore
* INT-4381: MessageSources refactoring
JIRA: https://jira.spring.io/browse/INT-4381
* Make all the out-of-the-box `MessageSource` implementations based
on the `AbstractMessageSource`
* Fix `JdbcPollingChannelAdapterIntegrationTests` for sporadic failure
because of `fixed-rate` for the poller
* Fix HeaderEnricherTests race condition
The `errorChannel()` expect an error in the `testErrorChannel` after
`1000` ms, but at the same time the `poller` in configured for the
`3000` ms.
* Increase all the timeouts for replies
* Decrease a `fixed-delay` on the `poller`
* Some other code style polishing for the `HeaderEnricherTests`
JIRA: https://jira.spring.io/browse/INT-4498
* Use `FileCopyUtils` instead of `StreamUtils` in test-cases when we deal
with `InputStream` s and files resources.
* Fix reflection-based tests with AOP to use an `ArrayList` access
instead of `LinkedList` since an optimization in SF:
6d6aa72e8f
**Cherry-pick to 5.0.x and 4.3.x**
Windows is too scrupulous for non closed file resources.
Our `RemoteFileTestSupport` recreates a test directory with files
for each test.
When we don't close some file handler, we are not able to delete the
directory and we fail with subsequent tests.
* Close `InputStream` for each polled file in new tests in the
`SftpStreamingMessageSourceTests`
* Close `CLOSEABLE_RESOURCE` in new tests in the
`FtpStreamingMessageSourceTests`
* Rework `FtpStreamingMessageSourceTests.testAllContents()` do not
poll all the files on each polling cycle - this causes a race condition
when we don't close `CLOSEABLE_RESOURCE` yet in the `StreamTransformer`,
but try to proceed with recreation a test directory structure
**Cherry-pick to 5.0.x**
