* GH-8778: Fix KafkaMessageSource deadlock
Fixes https://github.com/spring-projects/spring-integration/issues/8778
The `KafkaMessageSource.doReceive()` have a lock around its whole body.
That includes the `pollRecord()` which can be blocked on the `KafkaConsumer.poll()`.
This way the rest of lifecycle management callbacks can be blocked until `KafkaConsumer.poll()` returns.
* Rework lifecycle management flags to `AtomicBoolean` since there is not too much work
in their respective callbacks
* Decrease a locking block in the `doReceive()` just to consumer setup part.
Leave `pollRecord()` outside of the lock
* Add `this.consumer.wakeup()` into `stopConsumer()` to break a `poll()` cycle
and return immediately for the next `close()` call
**Cherry-pick to `6.1.x` & `6.0.x`**
* * Use `compareAndSet` in `start` & `stop`
Resolves https://github.com/spring-projects/spring-integration/issues/8638
Previously, immediate failures (e.g. timeout getting metadata) were
only thrown as exceptions, and not sent to the failure channel, if present.
**cherry-pick to all supported branches**
Looks like sometime Spring Framework XML parser can properly determine the target factory method to chose,
but sometimes it does that in a wrong order.
* Add `type="java.lang.Class"` to the ctor args everywhere for `class="org.mockito.Mockito" factory-method="mock"`
`<bean>` definitions
* Expose more reflection hints
* Bring back `@Reflective` on `Pausable` - for possible end-user usage
* Add `ReactiveMessageHandler` hint since its method is used reflectively
in the `IntegrationRSocketMessageHandler`
* Add `@Reflective` on the `ServerRSocketMessageHandler.handleConnectionSetup()`
since it is used reflectively for a `registerHandlerMethod()`
* Add `KafkaRuntimeHints` to expose `Pausable` contract on Kafka inbound endpoints
for SpEL invocation via Control Bus
* Document native images support
* Fix language in docs
Co-authored-by: Gary Russell <grussell@vmware.com>
Co-authored-by: Gary Russell <grussell@vmware.com>
Resolves https://github.com/spring-projects/spring-integration/issues/3942
When determining the default reply-to topic/partition, we need to wait for assignment.
Already covered by `KafkaDslTests` (a recent build failure exposed this problem).
**No back-port - 5.5.x uses 2.7.x by default, which does not support this.**
5.5.x users can call `waitForAssignment` on the `ReplyingKafkaTemplate` that is
supplied to the gateways before sending messages.
Fixes https://github.com/spring-projects/spring-integration/issues/3686
* Add `src/idea` with respective editor config for IntelliJ IDEA.
Must be imported into an IDE
* Add `src/eclipse` with respective editor config for Eclipse/STS
* Reformat imports in source code according a new editor config
It looks like `replyContainer()` is not registered as a bean,
so its lifecycle is somehow out of application context control
* Mark `replyContainer()` method as a `@Bean` in the `KafkaDslTests` and `KafkaDslKotlinTests`
to see if this fixes flaky state of the test suite
* Upgrade to Kotlin `1.7.20` and fix `KafkaDslKotlinTests` according to its requirements
For better test suite lifecycle (higher performance)
reuse one global embedded Kafka broker introduced in Spring for Apache Kafka `3.0`
Some tests have left with their own `EmbeddedKafkaBroker` definitions since they
rely on different partitions
* Fix deprecations around ListenableFuture
SF has deprecated a `ListenableFuture` and API around it
* Migrate to `CompletableFuture` everywhere a `ListenableFuture` has been used
* Suppress a deprecation for `ListenableFuture` keeping the functionality until the next version
* Resolve deprecations nad removals from the latest Spring for Apache Kafka
* Fix documentation for the `ListenableFuture` in favor of `CompletableFuture`
NOTE: the AMQP module is left as is until `ListenableFuture` deprecation is resolved in Spring AMQP
* * Restore some `ListenableFuture` test for messaging gateway
Spring for Apache Kafka does not produce `ListenableFuture` anymore.
* Rework `KafkaProducerMessageHandler` to deal with the `CompletableFuture` from now on
* Add support for `CompletableFuture` replies handling into `AbstractMessageProducingHandler`
* Remove redundant cast in the `DispatcherHasNoSubscribersTests`
The `Future` for `Producer.send()` can be fulfilled before
we return it (`isDone()`), so `KafkaTemplate` short-circuits
with a `KafkaException`.
* Modify `KafkaOutboundAdapterParserTests.testSyncMode()`
to assert against `KafkaException` which may be thrown earlier
then previously expected `KafkaProducerException`
**Cherry-pick to `5.5.x`**
Fixes https://github.com/spring-projects/spring-integration/issues/3623
* `IntegrationFlow` refactoring
* Apply several code style improvements and good practices
* Code style: no empty lines for methods javadocs
* make deprecated implementation reuse actual one instead of the copy-paste approach
* add whats-new comments
* Fix whats-new page according to standards
* Upgrade Spring dependencies to the latest SNAPSHOTs
* Fix tests to verify against stack traces: the message
of the `NestedRuntimeException` does not include the nested exception information.
Related to https://github.com/spring-projects/spring-framework/issues/25162
* Fix `JdbcMessageStore` and `DefaultLockRepository` to rely on the `DataIntegrityViolationException`
instead of only its `DuplicateKeyException` extension.
This is the current behavior of the SQL errors translation
* Disable `WebFluxDslTests.testValidation()` - doesn't subscribe to the reply somehow...
* Refine `SimplePool.PoolSemaphore.reducePermits()`
Current `IntegrationRecordMessageListener.onMessage()`
send a conversion error to the `errorChannel`, but
it does not return if it was successful leading to
the NPE `enhanceHeadersAndSaveAttributes()` because the `message`
is null
* Change the `IntegrationRecordMessageListener.onMessage()` logic to check
the result of the `sendErrorMessageIfNecessary()` and return immediately
if success.
Rethrow an exception otherwise.
Fixes https://github.com/spring-projects/spring-integration/issues/3790
Some `KafkaHeaders` constants have been removed and replaced with new
more meaningful
* Fix removed constants everywhere in the code and docs in favor of
newly introduced, which replaces old
Fixes https://github.com/spring-projects/spring-integration/issues/3734
The `MessageHistory` is not in the trusted packages of the `DefaultKafkaHeaderMapper`
therefore it fails when `MessageHistory.read()` is performed.
* Configure default `DefaultKafkaHeaderMapper` in the `SubscribableKafkaChannel`,
`KafkaInboundGateway`, `KafkaMessageDrivenChannelAdapter` and `KafkaMessageSource`
to also trust packages exposed via `JacksonJsonUtils.DEFAULT_TRUSTED_PACKAGES`
which include a `MessageHistory`, too.
* Verify in some integration Kafka tests to be sure that `MessageHistory` is
deserialized properly in the transferred headers
* Rework some tests to use `@EmbeddedKafka` instead of `@BeforeAll/@AfterAll`
**Cherry-pick to `5.5.x`**
* Add `--add-opens` to `asciidoctor` Gradle task to avoid compilation warning
* Upgrade to Kotlin `1.6.10` and modify its Gradle task respective
* Add empty impl for new `AnnotationMetadataAdapter.getDeclaredMethods()`
* Parse HTTP methods to new `String methodNames` property of the `RequestMapping`
since `HttpMethod` is not an `enum` anymore.
* Ignore JDBC tests which rely on a map property resolution: the `[]` placeholder
is not handled in the latest SF anymore.
* Fix `WebFluxInboundEndpoint` for deprecations
* Upgrade to Java 17, SF-6.0, Gradle 7.2
* Upgrade to Jakarta dependencies and respective namespaces
* Fix some tests for Java 17 compatibility
* Fix wrong Javadocs
* Add some missed Javadocs
* Fix more `jakarta` namespace
* Fix WS & XML modules to use Jakarta EE
* `--add-opens` in some modules for their reflection-based tests
* Disable Kafka tests which does not work on Windows; see Apache Kafka `3.0.1`
* Upgrade to JUnit `5.8.1`
* Migrate JMS tests to Artemis
* Remove RMI module as it was deprecated before
* Fix `pr-build-workflow.yml` for Java 17
* Fix JavaDocs warnings using `Xdoclint:syntax` per module, not in the top-level `api` task
* Move docs for version `6.0`
Resolves https://github.com/spring-projects/spring-integration/issues/3637
If there are no records to receive, the poll blocked for `20*pollTimeout`.
Wake the consumer during partition assignment; however, this changes the behavior
when there are records present - the first poll always returns no records.
Detect that the wakeup was due to assignment and perform another poll.
Also add some debug logging, and don't create a new consumer if the source was
stopped.
**cherry-pick to 5.4.x**
* To align with the rest of Spring portfolio, based on Spring Framework 5.3.x generation,
it is better to upgrade to the Kotlin version supported by SF
* Fix `ClassUtils.isLambda()` to treat `$inlined$` classes as Kotlin lambdas and let
Java DSL to delegate to the `LambdaMessageProcessor` for better expected type handling.
Looks like Kotlin does not carry type info in its lambdas anymore (at least for functions).
* Add `-parameters` to the compiler args for possible method argument names discovery.
Essentially realign with the rest of portfolio
* Fix Kotlin tests for deprecated API
Spring for Apache Kafka 2.8 has introduced an new `CommonErrorHandler`
and deprecated its retying components including `RetryingMessageListenerAdapter`.
* For proper compatibility with `spring-kafka-2.8.0`, which is going to be
a foundation for upcoming Spring Boot 2.6, it is better to suppress deprecations
and don't raise such a concern to end-users.
In the future we will revise retrying logic to expected behavior from `spring-kafka-2.8.0`.
Or will do home-made one as it is now with `AmqpInboundChannelAdapter`, for example.
See https://github.com/spring-projects/spring-integration/issues/3605
Resolves https://github.com/spring-projects/spring-integration/issues/3600
Previously, the `KafkaProducerMessageHandler` did not synchronize a transaction
with some other transaction (e.g. JDBC); it published in a local transaction instead.
Also configure the Gradle Kotlin Daemon to work with JDK 16.
**cherry-pick to 5.4.x**
