Commit Graph

259 Commits

Author SHA1 Message Date
abilan
52d43ea8ed Upgrade dependencies; prepare for release
Fix XML configs for new `Mockito.mock()` factory method
2022-12-20 16:25:06 -05:00
abilan
ab1254dcf8 GH-3966: Kafka XML config: Expose more attributes
Fixes https://github.com/spring-projects/spring-integration/issues/3966

Exposed setters in the `KafkaInboundGateway` and `KafkaMessageDrivenChannelAdapter`
as an XML attributes for `kafka` namespace

**Cherry-pick to `5.5.x`**
2022-12-12 16:00:40 -05:00
abilan
66ca0743ed Fix deprecation for RetryListenerSupport
Related to https://github.com/spring-projects/spring-retry/pull/326
2022-12-01 12:24:11 -05:00
Artem Bilan
3c40b01915 Expose more reflection hints (#3951)
* Expose more reflection hints

* Bring back `@Reflective` on `Pausable` - for possible end-user usage
* Add `ReactiveMessageHandler` hint since its method is used reflectively
in the `IntegrationRSocketMessageHandler`
* Add `@Reflective` on the `ServerRSocketMessageHandler.handleConnectionSetup()`
since it is used reflectively for a `registerHandlerMethod()`
* Add `KafkaRuntimeHints` to expose `Pausable` contract on Kafka inbound endpoints
for SpEL invocation via Control Bus
* Document native images support

* Fix language in docs

Co-authored-by: Gary Russell <grussell@vmware.com>

Co-authored-by: Gary Russell <grussell@vmware.com>
2022-11-18 11:26:46 -05:00
Gary Russell
77a752934d Improve Kafka Exception Message
- when we can't determine reply topic
2022-11-15 14:41:39 -05:00
Gary Russell
e19e6d4726 GH-3942: Fix Race in Kafka OB Gateway
Resolves https://github.com/spring-projects/spring-integration/issues/3942

When determining the default reply-to topic/partition, we need to wait for assignment.

Already covered by `KafkaDslTests` (a recent build failure exposed this problem).

**No back-port - 5.5.x uses 2.7.x by default, which does not support this.**

5.5.x users can call `waitForAssignment` on the `ReplyingKafkaTemplate` that is
supplied to the gateways before sending messages.
2022-11-15 08:39:54 -05:00
abilan
7864658d01 GH-3686: Apply SF editor config
Fixes https://github.com/spring-projects/spring-integration/issues/3686

* Add `src/idea` with respective editor config for IntelliJ IDEA.
Must be imported into an IDE
* Add `src/eclipse` with respective editor config for Eclipse/STS
* Reformat imports in source code according a new editor config
2022-11-14 10:55:21 -05:00
Artem Bilan
beef2e68c7 Use unique consumer groups in KafkaDslKotlinTests 2022-10-18 13:20:38 -04:00
Artem Bilan
daaa30e67f Make replyContainer as bean in Kafka tests
It looks like `replyContainer()` is not registered as a bean,
so its lifecycle is somehow out of application context control

* Mark `replyContainer()` method as a `@Bean` in the `KafkaDslTests` and `KafkaDslKotlinTests`
to see if this fixes flaky state of the test suite
* Upgrade to Kotlin `1.7.20` and fix `KafkaDslKotlinTests` according to its requirements
2022-10-06 16:01:08 -04:00
Artem Bilan
a667171c4f Use unique Kafka consumer groups in tests
so, it doesn't cause unexpected rebalances in the global shared embedded Kafka broker
2022-10-06 10:50:01 -04:00
Gary Russell
cbfa150dfa GH-3661: Fix Javadocs for RetryingMLA
Use of the `RetryingMessageListenerAdapter` was removed in
0550380704
but the javadocs were not corrected.
2022-09-21 16:04:20 -04:00
Artem Bilan
91a8b14b00 Move dependencies to Milestones for release
* Adapt code to the changes in upstream libs
* Change consumer groups in Kafka tests to unique values to avoid clashes
2022-09-20 11:43:36 -04:00
Artem Bilan
47cfae9d14 Close volatile resource in some Kafka tests 2022-09-07 17:27:39 -04:00
Artem Bilan
96d3b005fb Add @DirtiesContext to a couple Kafka tests 2022-09-07 16:05:48 -04:00
Artem Bilan
690743ee0c Use Global Embedded Kafka whenever possible
For better test suite lifecycle (higher performance)
reuse one global embedded Kafka broker introduced in Spring for Apache Kafka `3.0`

Some tests have left with their own `EmbeddedKafkaBroker` definitions since they
rely on different partitions
2022-08-01 12:27:58 -04:00
Artem Bilan
5572c2161d Fix deprecations around ListenableFuture (#3865)
* Fix deprecations around ListenableFuture

SF has deprecated a `ListenableFuture` and API around it

* Migrate to `CompletableFuture` everywhere a `ListenableFuture` has been used
* Suppress a deprecation for `ListenableFuture` keeping the functionality until the next version
* Resolve deprecations nad removals from the latest Spring for Apache Kafka
* Fix documentation for the `ListenableFuture` in favor of `CompletableFuture`

NOTE: the AMQP module is left as is until `ListenableFuture` deprecation is resolved in Spring AMQP

* * Restore some `ListenableFuture` test for messaging gateway
2022-07-28 09:32:01 -04:00
Artem Bilan
573c29793a Fix SIK module for latest SK compatibility
Spring for Apache Kafka does not produce `ListenableFuture` anymore.

* Rework `KafkaProducerMessageHandler` to deal with the `CompletableFuture` from now on
* Add support for `CompletableFuture` replies handling into `AbstractMessageProducingHandler`
* Remove redundant cast in the `DispatcherHasNoSubscribersTests`
2022-07-25 17:19:19 -04:00
Artem Bilan
d9a42c9e14 Fix KafkaOutAdapterParserTests race condition
The `Future` for `Producer.send()` can be fulfilled before
we return it (`isDone()`), so `KafkaTemplate` short-circuits
with a `KafkaException`.

* Modify `KafkaOutboundAdapterParserTests.testSyncMode()`
to assert against `KafkaException` which may be thrown earlier
then previously expected `KafkaProducerException`

**Cherry-pick to `5.5.x`**
2022-07-18 13:05:42 -04:00
Artem Vozhdayenko
53dd050c5b GH-3623: Deprecarte an IntegrationFlows
Fixes https://github.com/spring-projects/spring-integration/issues/3623

* `IntegrationFlow` refactoring
* Apply several code style improvements and good practices
* Code style: no empty lines for methods javadocs
* make deprecated implementation reuse actual one instead of the copy-paste approach
* add whats-new comments
* Fix whats-new page according to standards
2022-07-05 15:47:30 -04:00
Artem Bilan
2022c40d55 Fix compatibility with the latest SF
* Upgrade Spring dependencies to the latest SNAPSHOTs
* Fix tests to verify against stack traces: the message
of the `NestedRuntimeException`  does not include the nested exception information.
Related to https://github.com/spring-projects/spring-framework/issues/25162
* Fix `JdbcMessageStore` and `DefaultLockRepository` to rely on the `DataIntegrityViolationException`
instead of only its `DuplicateKeyException` extension.
This is the current behavior of the SQL errors translation
* Disable `WebFluxDslTests.testValidation()` - doesn't subscribe to the reply somehow...
* Refine `SimplePool.PoolSemaphore.reducePermits()`
2022-06-27 20:30:14 -04:00
Artem Bilan
cdcc986d11 Fix KafkaMDChannelAdapter error handling
Current `IntegrationRecordMessageListener.onMessage()`
send a conversion error to the `errorChannel`, but
it does not return if it was successful leading to
the NPE `enhanceHeadersAndSaveAttributes()` because the `message`
is null

* Change the `IntegrationRecordMessageListener.onMessage()` logic to check
the result of the `sendErrorMessageIfNecessary()` and return immediately
if success.
Rethrow an exception otherwise.
2022-05-31 14:08:07 -04:00
Artem Bilan
22b74c7a33 GH-3790: Use new header constants for Kafka headers
Fixes https://github.com/spring-projects/spring-integration/issues/3790

Some `KafkaHeaders` constants have been removed and replaced with new
more meaningful

* Fix removed constants everywhere in the code and docs in favor of
newly introduced, which replaces old
2022-05-05 21:51:16 -04:00
Artem Bilan
d5501d8273 GH-3734: Support MessageHistory on Kafka deser
Fixes https://github.com/spring-projects/spring-integration/issues/3734

The `MessageHistory` is not in the trusted packages of the `DefaultKafkaHeaderMapper`
therefore it fails when `MessageHistory.read()` is performed.

* Configure default `DefaultKafkaHeaderMapper` in the `SubscribableKafkaChannel`,
`KafkaInboundGateway`, `KafkaMessageDrivenChannelAdapter` and `KafkaMessageSource`
to also trust packages exposed via `JacksonJsonUtils.DEFAULT_TRUSTED_PACKAGES`
which include a `MessageHistory`, too.

* Verify in some integration Kafka tests to be sure that `MessageHistory` is
deserialized properly in the transferred headers
* Rework some tests to use `@EmbeddedKafka` instead of `@BeforeAll/@AfterAll`

**Cherry-pick to `5.5.x`**
2022-03-22 15:39:00 -04:00
Artem Bilan
50cf364cde Fix kafka tests according latest spring-kafka 2022-03-16 17:21:40 -04:00
Gary Russell
0550380704 GH-3661: Resolve Spring Apache Kafka Deprecations
Resolves https://github.com/spring-projects/spring-integration/issues/3661

* Don't allow existing listener; remove unnecessary try/catch; add what's new.
* Fix link in whats-new.adoc
2022-03-16 14:46:10 -04:00
Artem Bilan
11c3acdf2c Remove usage of isOnlyLogRecordMetadata
The `ConsumerProperties.isOnlyLogRecordMetadata` has been removed
in Spring for Apache Kafka in favor of `KafkaUtils.setConsumerRecordFormatter()`
2022-03-15 13:48:07 -04:00
Artem Bilan
5c912b1315 Fix KafkaMessageSource for the latest SK
* Add `mockito-inline` dep to be able to mock `final` classes, e.g. `record`
2022-03-08 16:24:30 -05:00
Gary Russell
12f2084ae8 Switch to spring-kafka 3.0.0-SNAPSHOT
- deprecations are temporarily restored there
 - fix deprecation warnings
 - re-enable tests for Windows
2022-03-08 10:10:27 -05:00
Artem Bilan
b3ae24eac6 Fixes according latest upgrades
* Add `--add-opens` to `asciidoctor` Gradle task to avoid compilation warning
* Upgrade to Kotlin `1.6.10` and modify its Gradle task respective
* Add empty impl for new `AnnotationMetadataAdapter.getDeclaredMethods()`
* Parse HTTP methods to new `String methodNames` property of the `RequestMapping`
since `HttpMethod` is not an `enum` anymore.
* Ignore JDBC tests which rely on a map property resolution: the `[]` placeholder
is not handled in the latest SF anymore.
* Fix `WebFluxInboundEndpoint` for deprecations
2022-01-18 14:38:53 -05:00
Artem Bilan
a80b22638d Start 6.0 version
* Upgrade to Java 17, SF-6.0, Gradle 7.2
* Upgrade to Jakarta dependencies and respective namespaces
* Fix some tests for Java 17 compatibility
* Fix wrong Javadocs
* Add some missed Javadocs
* Fix more `jakarta` namespace
* Fix WS & XML modules to use Jakarta EE
* `--add-opens` in some modules for their reflection-based tests
* Disable Kafka tests which does not work on Windows; see Apache Kafka `3.0.1`
* Upgrade to JUnit `5.8.1`
* Migrate JMS tests to Artemis
* Remove RMI module as it was deprecated before
* Fix `pr-build-workflow.yml` for Java 17
* Fix JavaDocs warnings using `Xdoclint:syntax` per module, not in the top-level `api` task
* Move docs for version `6.0`
2022-01-18 14:38:50 -05:00
Artem Bilan
6f3fdc76c1 Some docs clean up 2021-11-15 10:00:14 -05:00
Artem Bilan
34d2b0904f Fix new Sonar smells 2021-10-07 10:59:43 -04:00
Gary Russell
e2557a42c2 GH-3617: KPMH - Option To Use Template's Converter
Resolves https://github.com/spring-projects/spring-integration/issues/3617

Optionally use the `KafkaTemplate` message converter instead of the default
(or supplied) `ProducerRecordCreator`.
2021-10-06 16:10:57 -04:00
Gary Russell
b8e414d4f6 GH-3637: Fix KafkaMessageSource First Poll
Resolves https://github.com/spring-projects/spring-integration/issues/3637

If there are no records to receive, the poll blocked for `20*pollTimeout`.

Wake the consumer during partition assignment; however, this changes the behavior
when there are records present - the first poll always returns no records.

Detect that the wakeup was due to assignment and perform another poll.

Also add some debug logging, and don't create a new consumer if the source was
stopped.

**cherry-pick to 5.4.x**
2021-09-30 16:34:33 -04:00
Artem Bilan
0fb64f1762 Upgrade to Kotlin 1.5
* To align with the rest of Spring portfolio, based on Spring Framework 5.3.x generation,
it is better to upgrade to the Kotlin version supported by SF
* Fix  `ClassUtils.isLambda()` to treat `$inlined$` classes as Kotlin lambdas and let
Java DSL to delegate to the `LambdaMessageProcessor` for better expected type handling.
Looks like Kotlin does not carry type info in its lambdas anymore (at least for functions).
* Add `-parameters` to the compiler args for possible method argument names discovery.
Essentially realign with the rest of portfolio
* Fix Kotlin tests for deprecated API
2021-09-13 16:46:05 -04:00
Artem Bilan
91d09c8ad3 Add deprecation suppression for spring-kafka-2.8
Spring for Apache Kafka 2.8 has introduced an new `CommonErrorHandler`
and deprecated its retying components including `RetryingMessageListenerAdapter`.

* For proper compatibility with `spring-kafka-2.8.0`, which is going to be
a foundation for upcoming Spring Boot 2.6, it is better to suppress deprecations
and don't raise such a concern to end-users.

In the future we will revise retrying logic to expected behavior from `spring-kafka-2.8.0`.
Or will do home-made one as it is now with `AmqpInboundChannelAdapter`, for example.

See https://github.com/spring-projects/spring-integration/issues/3605
2021-07-29 16:39:57 -04:00
Gary Russell
1b7d622777 GH-3600: Fix Kafka Tx Synchronization
Resolves https://github.com/spring-projects/spring-integration/issues/3600

Previously, the `KafkaProducerMessageHandler` did not synchronize a transaction
with some other transaction (e.g. JDBC); it published in a local transaction instead.

Also configure the Gradle Kotlin Daemon to work with JDK 16.

**cherry-pick to 5.4.x**
2021-07-28 11:54:31 -04:00
Gary Russell
0b223369ed Fix Sonar Issue 2021-06-04 11:03:14 -04:00
Gary Russell
95fce11b0a GH-3573: Add ConsumerRecord(s) to ConversionExcep.
Resolves https://github.com/spring-projects/spring-integration/issues/3573
2021-06-03 12:47:19 -04:00
Artem Bilan
c57634f085 Properly mention Apache Kafka trademark in docs (#3540) 2021-04-07 16:59:57 -04:00
Artem Bilan
df62147f5d Fix new Sonar smells 2021-04-07 11:37:05 -04:00
Artem Bilan
be34fb6866 Some test improvement; fix new Sonar smell 2021-02-26 14:20:49 -05:00
Artem Bilan
eef31d4b34 Fix new Sonar smells; optimize some tests 2021-02-24 11:34:29 -05:00
Artem Bilan
9c686ddcb7 More JVM memory for Gradle
* Fix race condition in the `KafkaOutboundAdapterParserTests`
2021-02-23 11:28:04 -05:00
Artem Bilan
ad57c6187a Fix MessageSourceTest for the latest spring-kafka
Related to: https://github.com/spring-projects/spring-kafka/issues/1659
2020-12-21 17:31:45 -05:00
Artem Bilan
bdb3ba8ebd Upgrade dependencies including Gradle
* Fix compatibility with upgraded dependencies
* Move `What's New` for `5.4` to `history`
* Add `XmlUnit` dependency since the latest AssertJ has deprecated its `isXmlEqualTo()`
* Remove `javax.annotation-api` and `apiguardian-api` as redundant dependencies
2020-12-02 17:20:46 -05:00
Artem Bilan
0331933c16 Rely on MProducerSupport.active for Flux (#3423)
* Rely on `MProducerSupport.active` for `Flux`

* Fix `MessageProducerSupport` to extract an `active` flag and set it before
`isRunning` - the `Flux` subscription relies on the `takeWhile()`
where in case of `autoStartup = false` we will never start consume because
it is set to `true` already after `doStart()`
* Refactor all the `MessageProducerSupport` implementation with similar
`active` state to use already one from the super class

**Cherry-pick to 5.3.x**

* * Remove `MessageProducerSupport.setActive()`
to not let to mutate it from the implementations
* Set `active` to `false` in the `destroy()`
* Clean up and fix typos in the affected `JmsMessageDrivenEndpoint`

* * Pull `active` flag down to the `AbstractEndpoint`
* Set `active = true` in the `start()` before calling `doStart()`
* Do same for `active = false` in the `stop()`
* Clean up `AbstractEndpoint` impls to not call `doStart/doStop` for nothing
* Refactor endpoints to rely on the `active` state from the `AbstractEndpoint`
not their own
2020-11-06 13:51:03 -05:00
Gary Russell
3cf66cc539 Fix previous commit 2020-10-19 11:48:18 -04:00
Gary Russell
b6d60bfd69 KPMH - Remove unused parameter 2020-10-19 11:32:33 -04:00
Gary Russell
75d14d32e7 Further Improve Kafka Test Run Time
Configurable `timeoutBuffer`.
2020-10-16 16:28:34 -04:00