Commit Graph

737 Commits

Author SHA1 Message Date
Artem Bilan
ea3e118c8c Remove request FILENAME header for MPUT (#8761)
Related to: https://stackoverflow.com/questions/77268009/how-to-use-sftp-spring-integration-mput-with-sftpoutboundgateway-when-file-objec

In some scenarios when the flow starts with a file inbound channel adapter
and then an MPUT operation is performed for remote file outbound gateway,
the populated in the beginning `FileHeaders.FILENAME` is used from
the `DefaultFileNameGenerator` for all the files from local directory to
upload.
Such a behaviour leads only to the last file in the target remote
directory and only with the name from that header.

* Fix the `AbstractRemoteFileOutboundGateway` to remove a `FileHeaders.FILENAME` header
when message is build for specific item from MPUT request.
This way an original local file is used when we upload directory.
2023-10-12 09:46:06 -04:00
Artem Bilan
ce4ce74db6 Some code clean up for RemoteFileOutboundGateway
* Also apply suggested by IDE refactoring to `FtpServerOutboundTests`
2023-10-11 14:09:02 -04:00
Artem Bilan
44433ed8a1 GH-8745: Add RFT.shouldMarkSessionAsDirty() (#8759)
* GH-8745: Add RFT.shouldMarkSessionAsDirty()

Fixes https://github.com/spring-projects/spring-integration/issues/8745

Not all errors caught in the `RemoteFileTemplate.execute()`
are fatal to mark session as dirty and physically close the target session
in the cache

* Introduce a `RemoteFileTemplate.shouldMarkSessionAsDirty()`
to consult with an exception if it is really a fatal error to close
the session in the end.
* Override `shouldMarkSessionAsDirty()` in the `RemoteFileTemplate`
implementations to check statuses of respective protocol errors

**Cherry-pick to `6.1.x` & `6.0.x`**

* * Fix tests for pool interaction

* * Fix language in Javadocs
* Add more `not dirty` statuses to `SftpRemoteFileTemplate` & `SmbRemoteFileTemplate`
2023-10-11 10:52:48 -04:00
Artem Bilan
7b4d4561f5 Update dependencies; prepare for release
* Fix Checkstyle violations for `private` inner classes where they have to be also `final`
* Fix deprecations after upgrades
* Remove manual coroutines handling in the `GatewayProxyFactoryBean`
in favor of the same functionality in the `JdkDynamicProxy`
2023-09-19 20:49:14 -04:00
Adama Sorho
73ed3eeebd GH-8691: Add (S)FTP, SMB aged file filters
Fixes https://github.com/spring-projects/spring-integration/issues/8691

* Remove setAge with TimeUnit
Turned out we don't need anymore since we're using Duration for age in FtpLastModifiedFileListFilter and SftpLastModifiedFileListFilter.
* Add changes to the docs
* Introduce AbstractLastModifiedFileListFilter
* Some code readability improvements
* Make language in the docs more official
2023-09-14 12:16:06 -04:00
Artem Bilan
6b8d37ba30 GH-2971: Add LockRegistry.executeLocked() API (#8729)
* GH-2971: Add `LockRegistry.executeLocked()` API

Fixes https://github.com/spring-projects/spring-integration/issues/2971

* Following best practice and well-known patterns with `Jdbc`, `Rest` or `Jms` templates,
introduce `default` methods into `LockRegistry` interface to make it easier to perform
tasks when within a lock.
* Since all the required logic is now covered by those `LockRegistry.executeLocked()` methods,
there is no need in the dedicated abstract `WhileLockedProcessor` class.
Deprecated it for removal in the next version
* Use a new `LockRegistry.executeLocked()` API in the `FileWritingMessageHandler`
instead of just deprecated `WhileLockedProcessor`
* To satisfy Java limitations for checked lambdas, introduce `CheckedCallable` and `CheckedRunnable` utilities
similar to interfaces in the `io.micrometer.observation.Observation`
* Change existing `CheckedFunction` to expose extra generic argument for `Throwable`
* Add dedicated chapter for distributed lock into docs
* Fix some links and typos in the docs

* * Fix Javadoc for `CheckedFunction`

* Fix language in docs

Co-authored-by: Gary Russell <grussell@vmware.com>

---------

Co-authored-by: Gary Russell <grussell@vmware.com>
2023-09-11 13:22:26 -04:00
Artem Bilan
34f901f7db Improve split() DSL (#8666)
* Introduce a `SplitterSpec` which can accept possible
splitter variants: `expression`, `function`, `ref` etc.
This way we are going to have a complex endpoint configuration
only with a single method argument.
* Use `SplitterSpec` in a newly introduced `splitWith()`
* Deprecate those `split()` methods which use `SplitterEndpointSpec`.
This method are complex enough because of their several arguments
* Refactor some common `MessageHandler` options initialization
into the `ConsumerEndpointSpec.doGet()`
* Disable failing now `MethodInvokingMessageProcessorTests.testCollectionArgument()`:
or Jackson problem, or fresh SF
* Groovy DSL will be fixed in the separate PR:
https://stackoverflow.com/questions/76595843/groovy-selects-a-defaultgroovymethods-split-instead-of-mine-one
2023-07-03 10:35:21 -04:00
pziobron
0798c8df95 GH-8659: Fix WatchService to react for renames
Fixes https://github.com/spring-projects/spring-integration/issues/8659

* GH-8659: Updating the documentation as requested

* GH-8659: Fixes requested after coder review

**Cherry-pick to `6.1.x`, `6.0.x` & `5.5.x`**
2023-06-29 14:23:33 -04:00
Artem Bilan
070c1c6e60 GH-8626: Provide cleaner transform() DSL (#8653)
* GH-8626: Provide cleaner `transform()` DSL

Fixes https://github.com/spring-projects/spring-integration/issues/8626

* Add missed `transform(String beanName, @Nullable String methodName)` API
* Introduce a `TransformerSpec` to expose a strict API to configure
transformer variants.
* Introduce `transformWith(Consumer<TransformerSpec>)` as a single point
of all possible transformer and its endpoint options
* Deprecate those `IntegrationFlowDefinition.transform()` variants
which are harder to configure as several lambda arguments

This change will make Kotlin & Groovy DSL more readable and straightforward

* * Use new `transformWith()` in tests where a deprecated API still used
* Add JavaDocs to `TransformerSpec`
* Fix generic types for `BaseIntegrationFlowDefinition.transformWith()` -
they make sense exactly on the `TransformerSpec.transformer()` only
* Apply `transformWith()` for Groovy DSL
* Introduce a new `ClassUtils.isLambda(Object candidate)`
and add a check for Groovy `Closure`
* Fix `GroovyIntegrationFlowDefinition.createConfigurerIfAny()` to propagate a `Consumer` argument
down to the `Closure`

* * Rename `TransformerSpec -> TransformerEndpointSpec` for better context
meaning of the class
* Introduce `KotlinTransformerEndpointSpec` as an extension of the `TransformerEndpointSpec`
to have an `inline fun <reified P> transformer(crossinline function: (P) -> Any)`
for Kotlin style
* Add `KotlinIntegrationFlowDefinition.transformWith(KotlinTransformerEndpointSpec)`
* Deprecate Kotlin methods which are covered by the mentioned `transformWith()`
* Fix tests to use new API
* Mentioned the change in the doc
2023-06-26 12:01:13 -04:00
abilan
34904b3cd6 Upgrade to commons-io-2.13.0; downgrade sshd-sftp
* The latest `sshd-sftp-2.10.0` has a bug not removing trailing `.`
in the "unrooted" path when we ask to create a remote directory.
It works on Windows well, but fails on UNIX file systems.
* The `commons-io-2.13.0` has `Tailer` ctors deprecated
and exposes a builder API.
It starts a tailer process though unconditionally in
its own thread.
Use `setStartThread(false)` to have the tailer process
managed by our own `TaskExecutor`
* Add `ApacheCommonsFileTailingMessageProducer.setPollingDelayDuration(Duration)`
* Deprecate a `TailerListener` impl on the `ApacheCommonsFileTailingMessageProducer`
in favor of an internal instance
2023-06-22 15:41:38 -04:00
Christian Tzolov
c38ed96ee9 GH-8643: Replace synchronized with Lock
Fixes https://github.com/spring-projects/spring-integration/issues/8643

* First pass - trivial synchronized blocks
  - Convert the "trivial" `synchronized` block into `ReentrantLock`.

* fix checkstyle

* use blocking lock

* Secon pass - handle multi-lock cases

* javadoc + year

* addres first batch of review suggestions

* fix checkstyle issues

* fix the mqtt parent/child lock monitor sharing

* fix the mqtt parent/child lock monitor sharing, v2

* patch the stomp test
2023-06-21 13:25:45 -04:00
Artem Bilan
8dd1332d0a Fix NPEs in DSL Specs (#8597)
* Fix NPEs in DSL Specs

The `BaseWsInboundGatewaySpec` and `TailAdapterSpec` don't override super methods
and when we call them we fail with NPE since `target` was not populated.

* Fix `BaseWsInboundGatewaySpec` and its inheritors to populate the `target`
from their ctors.
* Remove redundant methods from `BaseWsInboundGatewaySpec` hierarchy
* Propagate `AbstractWebServiceInboundGateway` properties directly to the target
from the `BaseWsInboundGatewaySpec` inheritors
* Override `MessageProducerSpec` methods in the `TailAdapterSpec`
to populate respective option into the `FileTailInboundChannelAdapterFactoryBean`
* Expose more missed options for producer endpoint in the `FileTailInboundChannelAdapterFactoryBean`

**Cherry-pick to `6.0.x` & `5.5.x`**

* * `acceptIfNotNull()` for `errorMessageStrategy` in the `FileTailInboundChannelAdapterFactoryBean`

* * Fix `Unmarshaller` population logic in the `MarshallingWsInboundGatewaySpec`
2023-04-18 11:12:23 -04:00
abilan
d5181bf0d7 Add Nullability support into Java DSL 2023-04-14 14:16:36 -04:00
Artem Bilan
053cc00484 GH-3557: Add maxDepth, dirPredicate to FileReadMS (#8596)
* GH-3557: Add maxDepth, dirPredicate to FileReadMS

Fixes https://github.com/spring-projects/spring-integration/issues/3557

* Expose a `watchMaxDepth` on the `FileReadingMessageSource` for its `Files.walkFileTree()` API usage
* Add `watchDirPredicate` option ot the `FileReadingMessageSource` to skip sub-tree for `Files.walkFileTree()`
scanning according to some condition against directory `Path`

* Fix language in docs

Co-authored-by: Gary Russell <grussell@vmware.com>

---------

Co-authored-by: Gary Russell <grussell@vmware.com>
2023-04-13 17:03:39 -04:00
Adel Haidar
bfc6931329 GH-3869: Add ContextHolderRequestHandlerAdvice
Fixes https://github.com/spring-projects/spring-integration/issues/3869

* Move `ContextHolderRequestHandlerAdvice` to the `core` module for more general purposes
* Add `ContextHolderRequestHandlerAdviceTests`
* Rework `DelegatingSessionFactoryTests` to rely on the `ContextHolderRequestHandlerAdvice`.
This allows us to remove unnecessary XML configuration for this test class
* Document the feature
2023-03-31 17:44:19 -04:00
Artem Bilan
1bec420fd1 Do not block by default (#8580)
Currently, many timeouts in the project are like `-1` or other negative value
with a meaning to wait indefinitely.

According to distributed systems design and bad demo developing experience
it is not OK to block forever.

* Rework most of the timeouts in the framework to be `30` seconds.
Only one remained as `1` seconds is a `PollingConsumer` where it is
better to not block even for those 30 seconds when no messages in the queue,
but let the polling task be rescheduled.
* Remove the `MessagingGatewaySupport.replyTimeout` propagation down to the
`PollingConsumer` correlator where it was a `-1` before and blocked
the polling thread on the `Queue.poll()`.
This fixed the problem with a single thread in a pool for auto-configured `TaskScheduler`.
Now with 1 seconds wait time we are able to switch to other scheduled tasks
even with only 1 thread in the pool
2023-03-21 17:43:00 -04:00
Artem Bilan
ce9f7f4321 GH-8562: Fix streaming source for remote calls (#8564)
* GH-8562: Fix streaming source for remote calls

Fixes https://github.com/spring-projects/spring-integration/issues/8562

The `AbstractRemoteFileStreamingMessageSource.doReceive()` takes files first from a `toBeReceived` queue.
When `AbstractRemoteFileStreamingMessageSource.remoteFileToMessage()` fails to fetch the file content
because of interim connection issue, we reset this file from a filter and rethrow an exception.
The next `receive()` call will just go ahead to the next entry in the `toBeReceived` queue, but the
file we have just failed for will be retried only on the next list call to the remove directory.
This essentially breaks a possible in-order target application logic.

* Introduce `AbstractRemoteFileStreamingMessageSource.strictOrder` option to clear the `toBeReceived` queue
 when we fail in the `remoteFileToMessage()`, so the next `receive()`
 call would re-fetch files from remote dir, because the filter has been reset for those files.
* Fix `AbstractFileInfo.toString()` to not perform remote calls when we just log this file.
For example, we reset the file for connection failure and log the message about it,
but it fails again because we request `size` of the file which may require a remote connection.

**Cherry-pick to `6.0.x` & `5.5.x`**

* * Revert `AbstractFileInfo` changes
* Override `toString()` in `SmbFileInfo` instead -
exactly the place where connection is used to obtain
file attributes like `size` or `lastModified`
2023-02-28 16:34:19 -05:00
abilan
9dd9f08181 Use switch expression; some other clean up 2023-01-13 16:00:19 -05:00
abilan
033cb7695b Migrate Mockito-effected test classes to JUnit 5 2022-12-21 09:42:18 -05:00
abilan
52d43ea8ed Upgrade dependencies; prepare for release
Fix XML configs for new `Mockito.mock()` factory method
2022-12-20 16:25:06 -05:00
abilan
e3ec94b230 Optimize DefaultFileNameGenerator for expression
The default expression in the `DefaultFileNameGenerator`
is based on a message header which simply can be retrieved
with much faster `FunctionExpression`

* Some other code style refactoring in the `DefaultFileNameGenerator`
2022-11-14 12:26:35 -05:00
abilan
7864658d01 GH-3686: Apply SF editor config
Fixes https://github.com/spring-projects/spring-integration/issues/3686

* Add `src/idea` with respective editor config for IntelliJ IDEA.
Must be imported into an IDE
* Add `src/eclipse` with respective editor config for Eclipse/STS
* Reformat imports in source code according a new editor config
2022-11-14 10:55:21 -05:00
Artem Bilan
6b31d970de Remove spring-integration-gemfire module
Starting with Spring Data 2022.0.0 there is not going to be GemFire (Geode) support
directly from Spring team

A source code of the `spring-integration-gemfire` is moving to Spring Integration Extensions
from where community may consider to pull it and support in their own manner
2022-10-05 10:35:19 -04:00
Artem Bilan
4aa2f91bd9 GH-3572: Migrate SFTP from jsch to sshd-sftp (#3892)
* GH-3572: Migrate SFTP from `jsch` to `sshd-sftp`

Fixes https://github.com/spring-projects/spring-integration/issues/3572

* Rework SFTP module from the JSch API to more modern `sshd-sftp`
* Migrate generics of most API from `ChannelSftp.LsEntry` to the `SftpClient.DirEntry`
* Rework `DefaultSftpSessionFactory` to deal with an `SshClient` and create `SftpClient`
wrapped to the `SftpSession`
* Implement a `ResourceKnownHostsServerKeyVerifier` to load `known-hosts` from any possible resource
* Implement an expected `SftpSession.list()` with just file name to take or pattern matching
* Remove some unused tests and their config
* Remove tests for custom `UserInfo` since we don't provide any custom out-of-the-box
* Test a new `ResourceKnownHostsServerKeyVerifier` against default `known-hosts` file

* * Some tests improvements

* * Improve generics handling for `FileUtils`
2022-09-27 13:10:11 -04:00
Artem Bilan
5080fc2f45 GH-3828: Initial Spring AOT support (#3832)
* GH-3828: Initial Spring AOT support

Fixes https://github.com/spring-projects/spring-integration/issues/3828

* Provide an infrastructure based on a new Spring AOT engine in the latest Spring Framework
* Introduce `RuntimeHintsRegistrar` impls into modules which require some reflection,
resources or proxies and serialization available in the native image
* Mark some framework method with the `@Reflective` to make their reflection
info available in the native image, for example for SpEL or JMX invocations
* Add a `GatewayProxyBeanRegistrationAotProcessor` to register proxy interfaces
info for messaging gateways (either instance of the `GatewayProxyFactoryBean`)
* Rework `ConverterRegistrar` to not use a `beanFactory.getBeansWithAnnotation()`
since it is not available after AOT phase.
Instead, register an intermediate `IntegrationConverterRegistration` bean definition
from the `IntegrationConverterInitializer`
* Refactor `GlobalChannelInterceptorInitializer` a bit according to a new logic in the
`IntegrationConverterInitializer`
* Remove `JsonNodeWrapperToJsonNodeConverter` bean registration from the
`DefaultConfiguringBeanFactoryPostProcessor` - it is added by the `ConverterRegistrar`
into the target `ConversionService`
* Fix `ParentContextTests` respectively a `JsonNodeWrapperToJsonNodeConverter` bean removal
* Refactor `XsltPayloadTransformer` to not load a `ServletContextResource`, but just use its
name for the `xslResource` condition

* * Rework AOT support according latest changes and requirements
* Remove `@Bean` reflection since it is not needed any more
* Add `AotDetector.useGeneratedArtifacts()` condition to not register beans
one more time at runtime after AOT build phase
* Fix deprecation in the WebFlux test from the latest SF
2022-09-01 13:59:07 -04:00
Artem Bilan
8f44870837 GH-3827: Fix RemoteFile GET STREAM session leak
Fixes https://github.com/spring-projects/spring-integration/issues/3827

The `AbstractRemoteFileOutboundGateway.doGet()` for a `STREAM` option
does not close the `session` in case of error.
This may lead to some leaks or exhausted caches

* Close `session` in the `catch()` of the `AbstractRemoteFileOutboundGateway.doGet()`
* Adjust the `SftpServerOutboundTests` to configure a `CachingSessionFactory`
for the `testStream()` to verify there is no leaks attempting to
`GET STREAM` non-existing remote file twice

**Cherry-pick to `5.5.x`**
2022-07-18 14:07:34 -04:00
Artem Vozhdayenko
1c461a3e2c GH-3840: Migrate Redis tests to Testcontainers
Fixes https://github.com/spring-projects/spring-integration/issues/3840

* Create a new abstraction `RedisTest` for Testcontainers-based tests
* Move existing common utility methods to the new interface
* Migrate all existing Redis tests to use this new `RedisTest` interface
* Migrate all existing Redis tests to JUnit5
* Make all existing Redis tests confirming to JUnit5 standards, such as
default methods and classes visibility instead of public
* Add a dependency for parametrized tests in JUnit5
* Improve assertions readability across tests, for instance:
assertThat(thing.size()).isEqualTo(2) -> assertThat(thing).hasSize(2)
* Add real assertions to
`RedisLockRegistryTests.twoRedisLockRegistryTest` (earlier it did not
have assertions at all)
* Reformat, rearrange and cleanup the code
* Fix a couple of small changes after the code review:
* Change base interface name to be consistent with other similar places
* Typo in javadocs
* Small tests readability improvement regarding assertions
* Add `opens java.util` to `spring-integration-redis`
 to satisfy a `ReactiveRedisStreamMessageHandlerTests.testMessageWithListPayload()`
 requirements
2022-07-15 14:47:49 -04:00
Artem Bilan
5ded860e78 Remove unused import in OSDelegatingFileTailingMP 2022-07-11 11:53:18 -04:00
Artem Bilan
f85423a436 Fix more deprecations around TaskScheduler 2022-07-11 11:41:38 -04:00
Artem Vozhdayenko
53dd050c5b GH-3623: Deprecarte an IntegrationFlows
Fixes https://github.com/spring-projects/spring-integration/issues/3623

* `IntegrationFlow` refactoring
* Apply several code style improvements and good practices
* Code style: no empty lines for methods javadocs
* make deprecated implementation reuse actual one instead of the copy-paste approach
* add whats-new comments
* Fix whats-new page according to standards
2022-07-05 15:47:30 -04:00
Artem Bilan
2022c40d55 Fix compatibility with the latest SF
* Upgrade Spring dependencies to the latest SNAPSHOTs
* Fix tests to verify against stack traces: the message
of the `NestedRuntimeException`  does not include the nested exception information.
Related to https://github.com/spring-projects/spring-framework/issues/25162
* Fix `JdbcMessageStore` and `DefaultLockRepository` to rely on the `DataIntegrityViolationException`
instead of only its `DuplicateKeyException` extension.
This is the current behavior of the SQL errors translation
* Disable `WebFluxDslTests.testValidation()` - doesn't subscribe to the reply somehow...
* Refine `SimplePool.PoolSemaphore.reducePermits()`
2022-06-27 20:30:14 -04:00
Artem Bilan
799802c0d1 Clean up some JavaDocs; remove deprecated API 2022-04-08 16:05:43 -04:00
Artem Bilan
b3ae24eac6 Fixes according latest upgrades
* Add `--add-opens` to `asciidoctor` Gradle task to avoid compilation warning
* Upgrade to Kotlin `1.6.10` and modify its Gradle task respective
* Add empty impl for new `AnnotationMetadataAdapter.getDeclaredMethods()`
* Parse HTTP methods to new `String methodNames` property of the `RequestMapping`
since `HttpMethod` is not an `enum` anymore.
* Ignore JDBC tests which rely on a map property resolution: the `[]` placeholder
is not handled in the latest SF anymore.
* Fix `WebFluxInboundEndpoint` for deprecations
2022-01-18 14:38:53 -05:00
Artem Bilan
a80b22638d Start 6.0 version
* Upgrade to Java 17, SF-6.0, Gradle 7.2
* Upgrade to Jakarta dependencies and respective namespaces
* Fix some tests for Java 17 compatibility
* Fix wrong Javadocs
* Add some missed Javadocs
* Fix more `jakarta` namespace
* Fix WS & XML modules to use Jakarta EE
* `--add-opens` in some modules for their reflection-based tests
* Disable Kafka tests which does not work on Windows; see Apache Kafka `3.0.1`
* Upgrade to JUnit `5.8.1`
* Migrate JMS tests to Artemis
* Remove RMI module as it was deprecated before
* Fix `pr-build-workflow.yml` for Java 17
* Fix JavaDocs warnings using `Xdoclint:syntax` per module, not in the top-level `api` task
* Move docs for version `6.0`
2022-01-18 14:38:50 -05:00
Mauro Molinari
15520e120c GH-3652: File gateway: don't try delete no file
Fixes spring-projects/spring-integration#3652

Do not try to delete non-existing file in `FileExistMode.REPLACE` in the `AbstractRemoteFileOutboundGateway`

**Cherry-pick to `5.4.x` & `5.3.x`**
2021-11-03 10:06:31 -04:00
Artem Bilan
fc7d338302 GH-3656: Fix package tangles
Fixes https://github.com/spring-projects/spring-integration/issues/3656

* Move `JavaUtils` from `util` package to the root one to break any possible
tangling to/from other packages.
* Deprecate an existing `JavaUtils` for backward compatibility
* Remove `@IntegrationConverter` from the `JsonNodeWrapperToJsonNodeConverter`
and register it manually in the `ConverterRegistrar`.
The bean registration for the `JsonNodeWrapperToJsonNodeConverter` must be removed
in the next `6.0`
* Remove usage of `IntegrationContextUtils` from the `support` package
2021-11-01 15:06:01 -04:00
Artem Bilan
78366e7b00 GH-3647: Use remoteDirExpression in MV command (#3651)
* GH-3647: Use remoteDirExpression in MV command

Fixes https://github.com/spring-projects/spring-integration/issues/3647

To simplify a source and renameTo remote file expressions, the `remoteDirectoryExpression`
is consulted now, when they are not full paths.
This is useful when we want just to rename a remote file in some dir

* * Add JavaDoc for `getDirectoryExpressionProcessor()`
* Fix language in docs
2021-10-21 14:12:50 -04:00
Artem Bilan
fe57fd281c Checkstyle changes
* Upgrade to Checkstyle 9.0
* apply some JavaDocs rules
* Fix JavaDocs rules violations
* Some other minor clean up in the affected classes
2021-09-28 11:55:25 -04:00
Artem Bilan
0fb64f1762 Upgrade to Kotlin 1.5
* To align with the rest of Spring portfolio, based on Spring Framework 5.3.x generation,
it is better to upgrade to the Kotlin version supported by SF
* Fix  `ClassUtils.isLambda()` to treat `$inlined$` classes as Kotlin lambdas and let
Java DSL to delegate to the `LambdaMessageProcessor` for better expected type handling.
Looks like Kotlin does not carry type info in its lambdas anymore (at least for functions).
* Add `-parameters` to the compiler args for possible method argument names discovery.
Essentially realign with the rest of portfolio
* Fix Kotlin tests for deprecated API
2021-09-13 16:46:05 -04:00
trungPa
9e512186ed GH-3549: Fix minor SonarQube smells
Fixes https://github.com/spring-projects/spring-integration/issues/3549
2021-07-23 11:07:39 -04:00
Artem Bilan
5ec8913d27 Fix pattern how Lock.unlock() is used
Related to https://build.spring.io/browse/INT-MAIN-84/

The `lock.unlock()` must be called in the `finally` block of the
nested `try..catch`, not in the outer which may just fail on `lock.lockInterruptibly()`
in which case there is just not going to be anything we can `unlock()` in the end

**Cherry-pick to `5.4.x` & `5.3.x`**
2021-07-19 17:07:54 -04:00
trungPa
9c718c37a6 GH-3424: Refactor to use logging methods from LogAccessor
Fixes https://github.com/spring-projects/spring-integration/issues/3424

* Use `LogMessage.format()` for lazily formatting

* Fix some logging statements in `JdbcChannelMessageStore`
2021-07-19 11:30:01 -04:00
Artem Bilan
0d7bbeaa4e GH-3554: Eval remote dir on each synchToLocal (#3556)
* GH-3554: Eval remote dir on each synchToLocal

Fixes https://github.com/spring-projects/spring-integration/issues/3554

The `remoteDirectoryExpression` was introduced into an
`AbstractInboundFileSynchronizer` to let end-user to evaluate a remote directory
on each poll (essentially on each `synchronizeToLocalDirectory()` call).
The fix for the https://jira.spring.io/browse/INT-4491 introduced a regression
when such an expression was evaluated only once when we call a `setRemoteDirectory()`.
So, an original purpose of this option was lost and we don't get an actual
remote dir on each poll

* Remove `evaluateRemoteDirectory()` method and its usage since it doesn't
reflect expectation of the remote dir expression property
* Reinstate the `remoteDirectoryExpression` evaluation in the
`synchronizeToLocalDirectory()`
* Propagate the result of that expression into further methods for copying
files
* Remove setting of the `remoteDirectory` variable into a global `EvaluationContext`
of the `AbstractInboundFileSynchronizer` instance since it is not thread-safe
* Instead create an `EvaluationContext` locally for each `synchronizeToLocalDirectory()`
call and set the `remoteDirectory` variable into this scoped instances
* Generate a local file name from the `localFilenameGeneratorExpression`
against locally created `EvaluationContext` with the mentioned `remoteDirectory` variable
* Cover the expected functionality with a unit-test

**Cherry-pick to `5.4.x` & `5.3.x`**

* * Fix `testRemoteDirectoryRefreshedOnEachSynchronization` according PR review
2021-04-27 16:36:54 -04:00
Artem Bilan
6d9bd59659 Fix breaking change for PersistentFileListFilter
The `AbstractPersistentAcceptOnceFileListFilter`
extends now an `AbstractDirectoryAwareFileListFilter`
and makes all the implementors to implement its `isDirectory()` method.
This is a breaking change in the point release which makes implementations
outside of the core project not compilable

The regression after: https://github.com/spring-projects/spring-integration/issues/3488
Related fix in the Spring Cloud GCP: https://github.com/spring-cloud/spring-cloud-gcp/pull/2654

* Implement an `isDirectory()` in the `AbstractPersistentAcceptOnceFileListFilter`
to let other implementations do not worry during upgrade

**Cherry-pick to `5.4.x` & `5.3.x`**
2021-04-21 11:21:50 -04:00
Artem Bilan
df62147f5d Fix new Sonar smells 2021-04-07 11:37:05 -04:00
Artem Bilan
f2009271dc Miscellaneous fixes
* Fix `MiscellaneousTests` to extend `ActiveMQMultiContextTests`
to let it to close ActiveMQ connection factory in the end of test
* Make `RequestReplyScenariosWithTempReplyQueuesTests` more robust
closing `DefaultMessageListenerContainer` and `ExecutorService`
in the end of tests
* Fix new Sonar smells
* Rework some SQL calls in the `JdbcMessageStore` to note expose a
`PreparedStatement` API
* Fix JavaDoc in the `CorrelationHandlerSpec`
2021-03-26 12:39:00 -04:00
Artem Bilan
7e9552974c Introduce a MessageGroup.condition (#3517)
* Introduce a `groupConditionSupplier` for MGS

* Add a `MessageGroup.condition` option
* Add a `MessageGroupStore.conditionSupplier` option
* Use it from the `SimpleMessageStore.addMessagesToGroup()` API
to populate `condition` (if any) into a `MessageGroup`
* Introduce a `GroupConditionProvider` contract to be implemented
on those `ReleaseStrategy` contracts which could be aware of group condition
* Populated a `GroupConditionProvider.getGroupConditionSupplier()`
into a `MessageGroupStore` from the `AbstractCorrelatingMessageHandler`
for end-user convenience
* Rework a `FileMarkerReleaseStrategy` to implement a `GroupConditionProvider`
to provide a function which produces a condition from `file_lineCount` header
of the `END` marker message
* Make the `FileMarkerReleaseStrategy` logic already based on the condition from a group
* Delegate `GroupConditionProvider` from the `FileAggregator`
* Add test for empty file aggregation

* * Implement `condition` in the `AbstractKeyValueMessageStore` and `MongoDbMessageStore`
* Test `condition` for `mongo-aggregator-config.xml` and `FileAggregatorTests` against GemFire

* * Implement `condition` in the `ConfigurableMongoDbMessageStore`

* * Implement `condition` in the `JdbcMessageStore`
* `FileAggregatorTests` against `JdbcMessageStore`
* Refactor `JdbcMessageStore` for better handling of message group metadata
* Remove unused `MARKED` column in the DDL in favor of newly introduced `CONDITION`

* * Add docs for message group condition

* * Move `conditionSupplier` option from MGS to AbstractCorrelatingMH
* Make it as a `BiFunction` to propagate existing condition alongside with the
message to consult
* Expose `groupConditionSupplier` in Java & XML DSLs

* * Fix language in docs
2021-03-23 14:23:46 -04:00
Artem Bilan
4342586361 INT-4116: Introduce FileAggregator (#3511)
* INT-4116: Introduce FileAggregator

JIRA: https://jira.spring.io/browse/INT-4116

* Implement a `FileSplitter.FileMarker`-based aggregation strategies
and utilize them in a general `FileAggregator` component
* Make `HeaderAttributeCorrelationStrategy.attributeName` as `final`; add `Assert.notEmpty()`
* Fix `AggregatorFactoryBean` and `AggregatorSpec` to parse the provided processor
for possible `CorrelationStrategy` and/or `ReleaseStrategy`
* Introduce short-cut methods into Java & Kotlin DSL for an `aggregate()` configuration
* Introduce a `FileHeaders.LINE_COUNT` for header to be populated in the `FileSplitter`.
We need this info in the `FileAggregator` to avoid possible overhead with JSON deserialization
of the `FileSplitter.FileMarker` messages
* Test and document the feature
* Improve `FileSplitter` doc for code block switch (tabs)

* * Rework FileAggregator do not use Java Streams
2021-03-16 15:06:36 -04:00
Trung Pham
f30ede47e2 GH-3507: Fix Tail producer for proper command
Fixes https://github.com/spring-projects/spring-integration/issues/3507

The `OSDelegatingFileTailingMessageProducer` passing command string to `Runtime.getRuntime().exec()`
may cause problems if spaces (and other special characters) are used in the filename.

* Use an array for command and its options to let the target `Runtime` to parse and
execute it properly

**Cherry-pick to 5.4.x, 5.3.x & 5.2.x**
2021-03-11 13:58:30 -05:00
Artem Bilan
c023816c7e Various tests fixes
* Use a tmp file in the `FileTests.testFileReadingFlow()`
for writing content and then rename it to the target file.
It looks like we may already have a file just with a short content
and it is picked up by the watch service for processing in the flow
* Rework `AggregatorWithCustomReleaseStrategyTests` to JUnit 5 and remove
extra looping logic since it just does not add any extra coverage just
performance overhead via crating and destroying the same application ctx
* Rework `ChannelAdapterParserTests` to `@SpringJUnitConfig` managed test
2021-02-25 11:33:16 -05:00