Commit Graph

10821 Commits

Author SHA1 Message Date
Artem Bilan
d61df01acf Fix DSL sample in the doc
* Improve JavaDocs for `BaseIntegrationFlowDefinition` with renaming `payloadType`
param to `expectedType` since it can also be as a `Message.class`, not only type
for the payload.
* Add JavaDoc for `LambdaMessageProcessor` ctor
2021-10-19 14:19:58 -04:00
Anthony Schweigard
7e5fef9e45 GH-3641: Handle duplicate cookies properly
Fixes https://github.com/spring-projects/spring-integration/issues/3641

When a duplicate cookie name appears in a request, an `IllegalStateException` is thrown.
The default `Collectors.toMap()` does not allow a duplicated keys.

* Handle `servletRequest.getCookies()` as a `MultiValueMap`
* Call `toSingleValueMap()` for the evaluation context variable to restore previous behavior.
The next major version must expose the `MultiValueMap` as is to give access to all cookies from end-user expressions
* Rework some HTTP tests to JUnit 5

**Cherry-pick to `5.4.x` & `5.3.x`**
2021-10-19 13:26:22 -04:00
Artem Bilan
6c47593aef Fix ExpressionUtilsTests according SF changes
Related to: https://github.com/spring-projects/spring-framework/issues/27446
2021-10-19 09:46:35 -04:00
Artem Bilan
1341a457f0 Upgrade dependencies 2021-10-18 17:30:12 -04:00
Artem Bilan
72b9e7001e GH-3643: Defer MeterRegistry bean dependency
Fixes https://github.com/spring-projects/spring-integration/issues/3643

The `ImportBeanDefinitionRegistrar` register its bean definition in the early phase.
It causes a problem with Spring Boot when an explicit `@EnableIntegrationManagement`
is declared: the `MeterRegistry` is provided later on via auto-configuration
and `MicrometerMetricsCaptor` has no knowledge about `MeterRegistry` at the
`ImportBeanDefinitionRegistrar` phase.

* Reworks `MicrometerMetricsCaptorRegistrar` to the `MicrometerMetricsCaptorConfiguration`
and use special conditional `MicrometerMetricsCaptorImportSelector` for that
`@EnableIntegrationManagement`.
This way a `MicrometerMetricsCaptor` `@Bean` consults `ObjectProvider<MeterRegistry>`
on its creating time and returns `null` if no `MeterRegistry` beans at all.
2021-10-18 17:15:45 -04:00
Artem Bilan
a03af081af Upgrade to Gradle 6.9.1; fork JavaCompile
Attempt to rely on the forked process for compilation
according these docs: https://docs.gradle.org/current/userguide/performance.html#compiler_daemon
2021-10-18 11:24:31 -04:00
Artem Bilan
34d2b0904f Fix new Sonar smells 2021-10-07 10:59:43 -04:00
Gary Russell
e2557a42c2 GH-3617: KPMH - Option To Use Template's Converter
Resolves https://github.com/spring-projects/spring-integration/issues/3617

Optionally use the `KafkaTemplate` message converter instead of the default
(or supplied) `ProducerRecordCreator`.
2021-10-06 16:10:57 -04:00
Artem Bilan
0410343f7f Rework all the MQTT tests to JUnit 5
* Use `MosquittoContainerTest` for all the MQTT integration tests
* Remove MQTT `BrokerRunning` JUnit 4 rule
* Mark `BackToBackAdapterTests` with `@LongRunningTest` as it is really too slow
2021-10-06 13:30:12 -04:00
Artem Bilan
e7c0d8dafa GH-3432: Add MQTT v5 channel adapters (#3639)
* GH-3432: Add MQTT v5 channel adapters

Fixes https://github.com/spring-projects/spring-integration/issues/3432

* Add `optional` dependency for `org.eclipse.paho:org.eclipse.paho.mqttv5.client`
* Add `MqttProtocolErrorEvent` and emit it from the `mqttErrorOccurred()` callback of the MQTT v5 client
* Add `MqttHeaderMapper` since MQTT v5 has introduced user properties pair to transfer over the protocol
* Add `Mqttv5PahoMessageHandler` as one more extension of the `AbstractMqttMessageHandler`
* Add more convenient `MqttHeaders` constants for easier headers mapping configuration
* Ensure via `Mqttv5BackToBackTests` that MQTT v5 is supported by the provided components
* Change `pr-build-workflow.yml` to use `eclipse-mosquitto` container for testing all the MQTT interactions
* Change `cyrilix/rabbitmq-mqtt` service to the `rabbitmq:management` since RabbitMQ does not support MQTT v5

* * Handle manual acks
* Add `Mqttv5PahoMessageDrivenChannelAdapter.persistence` property

* * Add documentation
* Add `MosquittoContainerTest` for TestContainers support with Mosquitto image

* Fix language in the docs after review

Co-authored-by: Gary Russell <grussell@vmware.com>

Co-authored-by: Gary Russell <grussell@vmware.com>
2021-10-06 11:38:52 -04:00
Gary Russell
b8e414d4f6 GH-3637: Fix KafkaMessageSource First Poll
Resolves https://github.com/spring-projects/spring-integration/issues/3637

If there are no records to receive, the poll blocked for `20*pollTimeout`.

Wake the consumer during partition assignment; however, this changes the behavior
when there are records present - the first poll always returns no records.

Detect that the wakeup was due to assignment and perform another poll.

Also add some debug logging, and don't create a new consumer if the source was
stopped.

**cherry-pick to 5.4.x**
2021-09-30 16:34:33 -04:00
Artem Bilan
fe57fd281c Checkstyle changes
* Upgrade to Checkstyle 9.0
* apply some JavaDocs rules
* Fix JavaDocs rules violations
* Some other minor clean up in the affected classes
2021-09-28 11:55:25 -04:00
Spring Builds
d2e23c5f03 [artifactory-release] Next development version 2021-09-21 17:44:21 +00:00
Spring Builds
6268917239 [artifactory-release] Release version 5.5.4 2021-09-21 17:44:17 +00:00
Artem Bilan
3bfabb92ee Upgrade dependencies; prepare for release 2021-09-21 12:41:00 -04:00
Artem Bilan
726b216f07 Disable TcpNioConnectionTests.testCleanup()
The timeout is too short for CI server and there is
a high chance fo race condition with such a short `nioHarvestInterval`

* Use `@DisabledIfEnvironmentVariable(named = "bamboo_buildKey")`
to exclude the test from CI server, but still run it locally or in GH Actions
2021-09-20 11:36:23 -04:00
Artem Bilan
3f6d205615 Fix link in the channel-adapter.adoc 2021-09-20 11:10:50 -04:00
Artem Bilan
b92d52fe74 Improve docs for global default poller
* Fix Kotlin deprecation warning in the test
* Replace wrong `IntervalTrigger` mentioning in the docs to the proper `PeriodicTrigger`
* Fix code snippet in the `channel-adapter.adoc`
2021-09-20 10:35:16 -04:00
Artem Bilan
0fb64f1762 Upgrade to Kotlin 1.5
* To align with the rest of Spring portfolio, based on Spring Framework 5.3.x generation,
it is better to upgrade to the Kotlin version supported by SF
* Fix  `ClassUtils.isLambda()` to treat `$inlined$` classes as Kotlin lambdas and let
Java DSL to delegate to the `LambdaMessageProcessor` for better expected type handling.
Looks like Kotlin does not carry type info in its lambdas anymore (at least for functions).
* Add `-parameters` to the compiler args for possible method argument names discovery.
Essentially realign with the rest of portfolio
* Fix Kotlin tests for deprecated API
2021-09-13 16:46:05 -04:00
Artem Bilan
0f2285d5a3 Fix reply producing to not block reactive thread
When `DirectChannel` is used for reply producing, the data is
handled on the same thread which has produced it (normally), so
if we have a request-reply afterwards (e.g. `gateway()`), this thread
is blocked waiting for reply.
When the thread is assumed to be non-blocked (e.g. Netty event loop),
the request-reply withing such a thread for the same non-blocking client
causes a deadlock: the thread waits for reply, but at the same time it
supposes to fulfil a synchronization barrier with that reply

* Fix `AbstractMessageProducingHandler.asyncNonReactiveReply()` to use
a `publishOn(Schedulers.boundedElastic())` for reply `Mono` to free
producing thread from potential downstream blocking
* Demonstrate deadlock with a new test in the `RSocketDslTests`;
the original report was against WebFlux, but conditions are really
the same: `reactor-netty` is used as a low-level client

**Cherry-pick to `5.4.x`**
2021-09-13 10:42:04 -04:00
Artem Bilan
fe300ebc38 GH-3614: JPA outbound components delete in batch (#3629)
* GH-3614: JPA outbound components delete in batch

Fixes https://github.com/spring-projects/spring-integration/issues/3614

Add an `Iterable` payload support for the
`JpaExecutor.executeOutboundJpaOperationOnPersistentMode(Message<?>)`
and `DELETE` persist mode

* Fix language in the docs after review

Co-authored-by: Gary Russell <grussell@vmware.com>

Co-authored-by: Gary Russell <grussell@vmware.com>
2021-09-10 16:34:10 -04:00
Artem Bilan
25ac230be8 GH-3627: Fix race condition NPE in MqttPahoMDCA
Fixes https://github.com/spring-projects/spring-integration/issues/3627

The `destroy()`, and therefore `stop()` could be called from the `MqttConnectionFailedEvent` handling
in the same thread resetting the `client` property to `null`.

* Check for `this.client != null` in the next block of the `connectAndSubscribe()` to avoid NPE
* Check for `isActive()` in the `scheduleReconnect()` to be sure do not reconnect if channel adapter
has been stopped already

**Cherry-pick to `5.4.x`**
2021-09-09 14:37:41 -04:00
Artem Bilan
4456caffa1 Introduce high-level API for flows composition (#3624)
* Introduce high-level API for flows composition

For better end-user experience and more smooth integration logic
decomposition and distribution introduce an `IntegrationFlows.from(IntegrationFlow)`
to let to start the current flow from existing one.
On the other hand introduce an `BaseIntegrationFlowDefinition.to(IntegrationFlow)`
to let to continue the flow logic in the other existing one.
This way we can extract some templating logic into separate `IntegrationFlow` definitions
allowing at the same time to decompose a complex flow definition into logical reusable parts

* * Add more tests

* * Fix Checkstyle violation
* Add `@SuppressWarnings("overloads")` to new `from(IntegrationFlow)` and existing `from(Publisher)`.
Technically it does not make sense since `PublisherIntegrationFlow` is not a `public` class

* * Add docs
* Fix language in JavaDocs according review

* Fix language in the docs after review

Co-authored-by: Gary Russell <grussell@vmware.com>

Co-authored-by: Gary Russell <grussell@vmware.com>
2021-09-08 17:14:29 -04:00
wangYX657211334
b8785e5f9c GH-3619: Fix Mail Receiver fetch redundant Messages
Fixes https://github.com/spring-projects/spring-integration/issues/3619

The `AbstractMailReceiver` fetche redundant messages, causing performance problems

* Use correct, already filtered, array of message to fetch

**Cherry-pick to `main` & `5.3.x`**
2021-09-02 10:49:43 -04:00
Gary Russell
55876e0e42 GH-3620: Fix Shared SFTP Session
Resolves https://github.com/spring-projects/spring-integration/issues/3620

Shared session field was captured outside of the lock, causing multiple sessions
in a multi-threaded environment.

**cherry-pick to 5.4.x, 5.3.x**
2021-08-30 10:50:37 -04:00
Spring Builds
590fb01743 [artifactory-release] Next development version 2021-08-17 19:13:21 +00:00
Spring Builds
1c554a9583 [artifactory-release] Release version 5.5.3 2021-08-17 19:13:17 +00:00
Artem Bilan
39599f82db Update dependencies; prepare for release 2021-08-17 12:11:29 -04:00
Artem Bilan
024ffd1423 Fix generics for customizeMonoReply()
Related to: https://stackoverflow.com/questions/68637283/how-to-customize-response-in-spring-integration-using-webflux-when-a-specific-er

The function provided for the `ConsumerEndpointSpec.customizeMonoReply()` may convert
incoming value to something else.
With wildcards it cannot be compiled without casting.

* Add `<T, V>` generic arg for the `customizeMonoReply()` to conform in and out types carrying.
* Modify `WebFluxDslTests` to demonstrate the problem and confirm the fix

**Cherry-pick to `5.4.x` & `5.3.x`**
2021-08-09 15:12:10 -04:00
Dave G
b949b90d66 GH-3610: Fix WebFluxMH for error handling
Fixes https://github.com/spring-projects/spring-integration/issues/3610

This fix changes the WebFluxRequestExecutingMessageHandler
The change specifically changes the contruction of the
WebClientResponseException to use the `create` factory method.

* Added changes and unit test to cover updated code

Updated code to use simplified exception construction that is more tolerant
  of larger payloads.
Updated unit tests to check for specific exception types.

* Corrected checkstyle errors

Corrected checkstyle error in imports.
Corrected whitespace between casts.
Corrected trailing whitespace.

* Added @author


**Cherry-pick to `5.4.x` & `5.3.x`**
2021-08-09 13:27:12 -04:00
Dominik Simmen
c189a125a6 Safely read expunged IMAP messages
If application is connected to a Domino mail server via IMAP, it can happen from time to time that a message is expunged.
This leads to a `MessageRemovedException` when calling `IMAPMessage#getSubject`.
And although debug is set to false this again leads to a `MessageException` and the whole integration flow stops.

* Check for `isExpunged()` before logging a filtered message

* Add `@author`
* Fix debug message for expunged to reflect reality
* Clean up new unit tests a bit

**Cherry-pick to `5.4.x`**
2021-08-09 11:23:26 -04:00
trungPa
b12540d2ac GH-3598: Fix delay in waitStopListening()
Fixes spring-projects/spring-integration#3598

`TestingUtilities.waitStopListening(serverConnectionFactory, delayArg)` actually waits 
for `delayArg * 2` milliseconds which is inconsistent with the JavaDocs.

* Fix `TestingUtilities.waitStopListening()` to sleep for `100` between attempts and 
compare `n` attempts against `delay / 100`
2021-08-09 10:32:30 -04:00
Artem Bilan
0a7ea59198 GH-3601: Bring AMQP byte code compatibility back
Fixes https://github.com/spring-projects/spring-integration/issues/3601

The issue https://github.com/spring-projects/spring-integration/issues/3584
has introduced a regression when old constructor with an
`AbstractMessageListenerContainer` was removed in favor of just
`MessageListenerContainer`.
But with that change all the dependant projects must be recompiled,
which is not a case when Spring Cloud was not released against the
latest Spring Boot.

**Cherry-pick to `5.4.x`**
2021-07-30 13:42:46 -04:00
Artem Bilan
91d09c8ad3 Add deprecation suppression for spring-kafka-2.8
Spring for Apache Kafka 2.8 has introduced an new `CommonErrorHandler`
and deprecated its retying components including `RetryingMessageListenerAdapter`.

* For proper compatibility with `spring-kafka-2.8.0`, which is going to be
a foundation for upcoming Spring Boot 2.6, it is better to suppress deprecations
and don't raise such a concern to end-users.

In the future we will revise retrying logic to expected behavior from `spring-kafka-2.8.0`.
Or will do home-made one as it is now with `AmqpInboundChannelAdapter`, for example.

See https://github.com/spring-projects/spring-integration/issues/3605
2021-07-29 16:39:57 -04:00
Gary Russell
1b7d622777 GH-3600: Fix Kafka Tx Synchronization
Resolves https://github.com/spring-projects/spring-integration/issues/3600

Previously, the `KafkaProducerMessageHandler` did not synchronize a transaction
with some other transaction (e.g. JDBC); it published in a local transaction instead.

Also configure the Gradle Kotlin Daemon to work with JDK 16.

**cherry-pick to 5.4.x**
2021-07-28 11:54:31 -04:00
Artem Bilan
1951d5230c Fix java-dsl-gateway double usage in Docs 2021-07-27 15:19:36 -04:00
trungPa
61153578c5 GH-3549: Clean up more SonarQube smells
Fixes https://github.com/spring-projects/spring-integration/issues/3549
2021-07-27 11:24:14 -04:00
Artem Bilan
2be6d86ff9 Upgrade setup-jfrog-cli to fix deprecation 2021-07-26 16:17:53 -04:00
trungPa
9e512186ed GH-3549: Fix minor SonarQube smells
Fixes https://github.com/spring-projects/spring-integration/issues/3549
2021-07-23 11:07:39 -04:00
Spring Builds
bc8246d267 [artifactory-release] Next development version 2021-07-20 16:20:43 +00:00
Spring Builds
63073e4ee4 [artifactory-release] Release version 5.5.2 2021-07-20 16:20:37 +00:00
Artem Bilan
ed143ba790 Upgrade dependencies; prepare for release 2021-07-20 11:45:53 -04:00
Artem Bilan
f64b602b23 Fix code smell in the JdbcChannelMessageStore
* Refine JavaDocs in the `JdbcChannelMessageStore`
2021-07-20 09:39:02 -04:00
Artem Bilan
5ec8913d27 Fix pattern how Lock.unlock() is used
Related to https://build.spring.io/browse/INT-MAIN-84/

The `lock.unlock()` must be called in the `finally` block of the
nested `try..catch`, not in the outer which may just fail on `lock.lockInterruptibly()`
in which case there is just not going to be anything we can `unlock()` in the end

**Cherry-pick to `5.4.x` & `5.3.x`**
2021-07-19 17:07:54 -04:00
Artem Bilan
dbc8af937b GH-3591: Add gateway() section into dsl.adoc (#3593)
* GH-3591: Add `gateway()` section into `dsl.adoc`

Fixes https://github.com/spring-projects/spring-integration/issues/3591

* Apply review requested changes
2021-07-19 15:55:53 -04:00
Artem Bilan
af9e69c251 Make MessageHistory JSON-serializable
* Add `org.springframework.integration.history` to trusted default packaged of the `JacksonJsonUtils`
* Add `@JsonCreator` to `MessageHistory` `private` ctor to let it to be created automatically by Jackson
* Add `equals()` and `hashCode()` into `MessageHistory` for the proper `Message` comparison
* Add  `MessageHistory` into headers for testing with Redis JSON (de)serialization

**Cherry-pick to `5.4.x` & `5.3.x`**
2021-07-19 12:37:05 -04:00
trungPa
9c718c37a6 GH-3424: Refactor to use logging methods from LogAccessor
Fixes https://github.com/spring-projects/spring-integration/issues/3424

* Use `LogMessage.format()` for lazily formatting

* Fix some logging statements in `JdbcChannelMessageStore`
2021-07-19 11:30:01 -04:00
Artem Bilan
39c3acb042 Fix ZMQMessageHandlerTests race condition
Use `receiveTimeOut = 0` on SUB socket and rely
on the retries from the `await().atMost()`.
Also resubscribe in the retry callback before publishing the next attempt
2021-07-09 11:38:03 -04:00
Artem Bilan
0297f6bc9f Fix dsl.adoc for the actual API around Supplier 2021-07-08 16:25:38 -04:00
Artem Bilan
ca9c9192c7 Fix new sonar smell
The `GatewayMethodMetadata.headerExpressions` is never `null`
2021-07-08 14:11:22 -04:00