Commit Graph

229 Commits

Author SHA1 Message Date
Artem Bilan
6f3fdc76c1 Some docs clean up 2021-11-15 10:00:14 -05:00
Artem Bilan
34d2b0904f Fix new Sonar smells 2021-10-07 10:59:43 -04:00
Gary Russell
e2557a42c2 GH-3617: KPMH - Option To Use Template's Converter
Resolves https://github.com/spring-projects/spring-integration/issues/3617

Optionally use the `KafkaTemplate` message converter instead of the default
(or supplied) `ProducerRecordCreator`.
2021-10-06 16:10:57 -04:00
Gary Russell
b8e414d4f6 GH-3637: Fix KafkaMessageSource First Poll
Resolves https://github.com/spring-projects/spring-integration/issues/3637

If there are no records to receive, the poll blocked for `20*pollTimeout`.

Wake the consumer during partition assignment; however, this changes the behavior
when there are records present - the first poll always returns no records.

Detect that the wakeup was due to assignment and perform another poll.

Also add some debug logging, and don't create a new consumer if the source was
stopped.

**cherry-pick to 5.4.x**
2021-09-30 16:34:33 -04:00
Artem Bilan
0fb64f1762 Upgrade to Kotlin 1.5
* To align with the rest of Spring portfolio, based on Spring Framework 5.3.x generation,
it is better to upgrade to the Kotlin version supported by SF
* Fix  `ClassUtils.isLambda()` to treat `$inlined$` classes as Kotlin lambdas and let
Java DSL to delegate to the `LambdaMessageProcessor` for better expected type handling.
Looks like Kotlin does not carry type info in its lambdas anymore (at least for functions).
* Add `-parameters` to the compiler args for possible method argument names discovery.
Essentially realign with the rest of portfolio
* Fix Kotlin tests for deprecated API
2021-09-13 16:46:05 -04:00
Artem Bilan
91d09c8ad3 Add deprecation suppression for spring-kafka-2.8
Spring for Apache Kafka 2.8 has introduced an new `CommonErrorHandler`
and deprecated its retying components including `RetryingMessageListenerAdapter`.

* For proper compatibility with `spring-kafka-2.8.0`, which is going to be
a foundation for upcoming Spring Boot 2.6, it is better to suppress deprecations
and don't raise such a concern to end-users.

In the future we will revise retrying logic to expected behavior from `spring-kafka-2.8.0`.
Or will do home-made one as it is now with `AmqpInboundChannelAdapter`, for example.

See https://github.com/spring-projects/spring-integration/issues/3605
2021-07-29 16:39:57 -04:00
Gary Russell
1b7d622777 GH-3600: Fix Kafka Tx Synchronization
Resolves https://github.com/spring-projects/spring-integration/issues/3600

Previously, the `KafkaProducerMessageHandler` did not synchronize a transaction
with some other transaction (e.g. JDBC); it published in a local transaction instead.

Also configure the Gradle Kotlin Daemon to work with JDK 16.

**cherry-pick to 5.4.x**
2021-07-28 11:54:31 -04:00
Gary Russell
0b223369ed Fix Sonar Issue 2021-06-04 11:03:14 -04:00
Gary Russell
95fce11b0a GH-3573: Add ConsumerRecord(s) to ConversionExcep.
Resolves https://github.com/spring-projects/spring-integration/issues/3573
2021-06-03 12:47:19 -04:00
Artem Bilan
c57634f085 Properly mention Apache Kafka trademark in docs (#3540) 2021-04-07 16:59:57 -04:00
Artem Bilan
df62147f5d Fix new Sonar smells 2021-04-07 11:37:05 -04:00
Artem Bilan
be34fb6866 Some test improvement; fix new Sonar smell 2021-02-26 14:20:49 -05:00
Artem Bilan
eef31d4b34 Fix new Sonar smells; optimize some tests 2021-02-24 11:34:29 -05:00
Artem Bilan
9c686ddcb7 More JVM memory for Gradle
* Fix race condition in the `KafkaOutboundAdapterParserTests`
2021-02-23 11:28:04 -05:00
Artem Bilan
ad57c6187a Fix MessageSourceTest for the latest spring-kafka
Related to: https://github.com/spring-projects/spring-kafka/issues/1659
2020-12-21 17:31:45 -05:00
Artem Bilan
bdb3ba8ebd Upgrade dependencies including Gradle
* Fix compatibility with upgraded dependencies
* Move `What's New` for `5.4` to `history`
* Add `XmlUnit` dependency since the latest AssertJ has deprecated its `isXmlEqualTo()`
* Remove `javax.annotation-api` and `apiguardian-api` as redundant dependencies
2020-12-02 17:20:46 -05:00
Artem Bilan
0331933c16 Rely on MProducerSupport.active for Flux (#3423)
* Rely on `MProducerSupport.active` for `Flux`

* Fix `MessageProducerSupport` to extract an `active` flag and set it before
`isRunning` - the `Flux` subscription relies on the `takeWhile()`
where in case of `autoStartup = false` we will never start consume because
it is set to `true` already after `doStart()`
* Refactor all the `MessageProducerSupport` implementation with similar
`active` state to use already one from the super class

**Cherry-pick to 5.3.x**

* * Remove `MessageProducerSupport.setActive()`
to not let to mutate it from the implementations
* Set `active` to `false` in the `destroy()`
* Clean up and fix typos in the affected `JmsMessageDrivenEndpoint`

* * Pull `active` flag down to the `AbstractEndpoint`
* Set `active = true` in the `start()` before calling `doStart()`
* Do same for `active = false` in the `stop()`
* Clean up `AbstractEndpoint` impls to not call `doStart/doStop` for nothing
* Refactor endpoints to rely on the `active` state from the `AbstractEndpoint`
not their own
2020-11-06 13:51:03 -05:00
Gary Russell
3cf66cc539 Fix previous commit 2020-10-19 11:48:18 -04:00
Gary Russell
b6d60bfd69 KPMH - Remove unused parameter 2020-10-19 11:32:33 -04:00
Gary Russell
75d14d32e7 Further Improve Kafka Test Run Time
Configurable `timeoutBuffer`.
2020-10-16 16:28:34 -04:00
Gary Russell
9c6000e453 Kafka Outbound Endpoints - Add futuresChannel
Currently, the only way to block is to set `sync=true` which waits for the future.
The problem with this is it only supports one-at-a-time publication.

Add an option to send the send futures to a channel, allowing the application to
send multiple records and then wait on the futures later.

Also fix long-running test.

* Add docs.
2020-10-16 14:37:37 -04:00
Artem Bilan
c7ff99a4e8 Use LogAccessor from SF
* Change main classes to use a `LogAccessor` API to simplify code flow
* Fix tests according `LogAccessor` property
* Fix some Sonar smells
2020-10-06 13:56:50 -04:00
Artem Bilan
f45206a013 Fix race condition in the MessageDrivenAdapterTests
The `pause` variable is set in the container already after a `consumer.pause()` call.
Therefore it's state might not be changed after acquiring the latch from the mock answer
2020-08-18 11:17:41 -04:00
Artem Bilan
9d557426b5 * Fix new Sonar smells
* Remove redundant `@SuppressWarnings("deprecation")`
* Add `Duration.ofSeconds(10)` to `StepVerifier.verify()`
to avoid infinite wait and lose failing text context on the hang CI build
2020-08-08 12:29:24 -04:00
Gary Russell
3de0445aaa GH-3358: Kafka IB Gateway - implement Pausable
Resolves https://github.com/spring-projects/spring-integration/issues/3358

Also add `isPaused()` to `Pausable`.

* Fix `@since` to the proper version
2020-08-07 15:26:57 -04:00
Gary Russell
1beb854fb4 Remove Legacy Metrics
- Simplify MBeans - instead of wrapping to expose lifecycle methods,
  implement `ManageableLifecycle`. Register an additional MBean for
  polled endpoints to control the lifecycle.

* Polishing

- Move `QueueChannel` `@ManagedAttribute`s to `QueueChannelOperations`
- Make all `AbstractEndpoints` `IntegrationManagedResource`s and remove `ManagedEndpoint`
  to allow exposure of any `@Managed*` methods (including those on `Pausable`)
- Revert to `Lifecycle` for classes that are not related to endpoints
- Remove legacy metrics from docs
2020-08-07 12:56:57 -04:00
Artem Bilan
094eb2e021 Fix new Sonar smells 2020-08-06 10:45:44 -04:00
Gary Russell
acdc25a172 Upgrade to spring-kafka 2.6.0 2020-08-04 12:14:16 -04:00
Artem Bilan
8b16aede89 Fix new Sonar smells 2020-07-16 11:36:04 -04:00
Gary Russell
9c252be028 Fix Kafka Send Timeout
Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/928

Kafka has a longer default send timeout; this means a send could be successful long
after Spring has timed out the send.

**I will backport to the 3.3.x extension after merge**

* Use setSendTimeout; make it final.
2020-07-15 10:35:33 -04:00
Artem Bilan
a19e37237b Fix new and some old Sonar smells 2020-07-14 10:49:28 -04:00
Artem Bilan
6e11d4cdf9 Fix new and some old Sonar smells 2020-07-13 14:07:07 -04:00
Gary Russell
9223613ee1 ConsumerRecord Logging Metadata Option 2020-07-10 15:14:49 -04:00
Gary Russell
b860a2e818 Merge Spring Integration Kafka into Main Repo
- merge master branch from old repo (`git-filter-repo`) for src branch
- update build.gradle
- copy docs from Spring for Apache Kafka Chapter

Doc polishing.

Fix http: references in tests.

* Add `What's New` note and fix `polling-consumer.adoc`
not linking to external project any more
2020-06-25 11:46:38 -04:00
Gary Russell
3ec74f380f KMSource - fix RebalanceListener (incremental)
Support cooperative rebalancing
 - incremental assignment/revocation
 - re-pause if a rebalance occurs while paused

**cheerry-pick to 3.2.x**
2020-06-25 11:29:09 -04:00
Artem Bilan
a28647259c Upgrade dependencies; prepare for release 2020-06-25 11:29:09 -04:00
Gary Russell
8771f8e18c Fix tests for latest SK snapshot 2020-06-25 11:29:08 -04:00
Artem Bilan
04019dbb96 Fix checkstyle.xml for Windows new line 2020-06-25 11:29:08 -04:00
Artem Bilan
bfef5a49cf Fix Kotlin tests according upstream changes
* Upgrade dependencies
2020-06-25 11:29:08 -04:00
Gary Russell
213772dcaf Fix S-K Deprecations 2020-06-25 11:29:08 -04:00
Gary Russell
1e484d0fad Support flush after send
* Add `flushExpression`.

* Add DSL, XML configuration.
2020-06-25 11:29:08 -04:00
Artem Bilan
e0be544a40 Add more XSD version into spring.schemas
* Fix some deprecation in `build.gradle`
2020-06-25 11:29:07 -04:00
Gary Russell
a58169f4e1 Fix Gradle Deprecations
* Rename schema

* Upgrade to Gradle 6.3-rc-3 for transitive dependency fix.

* Remove version from schema; polish `build.gradle` to remove unneeded features.
2020-06-25 11:29:07 -04:00
Gary Russell
81b9ae4a7a Use core Kotlin DSL 2020-06-25 11:29:07 -04:00
Gary Russell
0b7ea6d3c2 Remove deprecated methods 2020-06-25 11:29:07 -04:00
Gary Russell
bdd236052b Upgrade to SK 2.5.0
- add deliveryAttempt header when retry is not configured
- temporary work around for gradle bug, jar with test classifier missing from CP
2020-06-25 11:29:07 -04:00
Gary Russell
aaaaf86c07 GH-296: DSL: Support inline pub/sub subscriptions
Resolves https://github.com/spring-projects/spring-integration-kafka/issues/296

* * Add generic type to specsto avoid cast to `BroadcastCapableChannel`.

* * Remove unneeded fields in the abstract spec

* Restore log4j config
2020-06-25 11:29:06 -04:00
Gary Russell
2331eee642 GH-296: Add Kafka-backed MessageChannels
Resolves https://github.com/spring-projects/spring-integration-kafka/issues/296

* * Add DSL support

* * Polishing and XML support
2020-06-25 11:29:06 -04:00
Gary Russell
905a5155bc Master to 3.3; support non-Tx if template allows
* SI to 5.3.0.B-S
2020-06-25 11:29:06 -04:00
Artem Bilan
f596e49242 Upgrade to latest releases
* Upgrade to Gradle 6.1.1
* Fix for latest Checkstyle
* Fix for latest SI-Kotlin-DSL
2020-06-25 11:29:06 -04:00