Commit Graph

66 Commits

Author SHA1 Message Date
Chris Bono
0cb59496fe Make InteractiveQueryService only return used state stores
This restores the KafkaStreams 3.0 behavior in this area.

Fixes #2445
2022-06-25 22:09:19 -05:00
Pommerening, Nico
7554ff9bbd Kafka Streams binder health indicator improvements
Fix KafkaStreamsBinderHealthIndicator overriding HealthCheck Thread Details to report full details.

checkstyle fixes.
2022-06-23 14:20:04 -04:00
Chris Bono
06ffe76c01 Improve KafkaStreams ineligible-definitions property handling
* KafkaStreamsBinderEnvironmentPostProcessor respects existing ineligible-definitions
* Add tests for KafkaStreamsBinderEnvironmentPostProcessor
2022-06-17 13:42:25 -04:00
Chris Bono
6d1be361fe Simplify SerdeResolverUtilsTests setting of ineligible-definitions 2022-06-17 13:40:19 -04:00
yiliu6
acc75183e2 Enable multiple customizers for binder and provisioner.
Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2423
2022-06-17 12:43:41 -04:00
Soby Chacko
21171914ad SendToDlqAndContinue needs to be excluded
In Kafka Streams binder, we need to exclude the SendToDlqAndContinue
BiFunction by adding it to the ineligible function definitions.

See this commit for more context: 8ec15fa3ca
2022-06-16 18:12:11 -04:00
buildmaster
97975a0d3c Going back to snapshots 2022-06-15 11:35:09 +00:00
buildmaster
84d4cccdee Update SNAPSHOT to 4.0.0-M3 2022-06-15 11:32:24 +00:00
Oleg Zhurakousky
225d96c4fb Temporarily disable failing Kinesis test 2022-06-15 13:14:24 +02:00
Oleg Zhurakousky
da3157c6d1 Remove dependency on test-support-internal from kinesis binder 2022-06-15 12:52:19 +02:00
Oleg Zhurakousky
c4f516e72f Change from SCSt version to project.version in Kinesis binder 2022-06-15 12:26:41 +02:00
Oleg Zhurakousky
2b914cd0f7 Update SI Kinesis to 3.0.0-M1 2022-06-15 11:39:49 +02:00
Soby Chacko
7b9c66c4ac Adjust KafkaBinder DLQ test error messages 2022-06-14 10:49:25 -04:00
Soby Chacko
806a6500b5 Kafka Streams binder message conversion changes
Fixing boolean logic around non-native message conversion in Kafka Streams binder.
2022-06-07 17:50:36 -04:00
Soby Chacko
6dd6774280 Producer config customization and DLQ issues
When DLQ is enabled on a consumer binding, the internal producer properties
used by the DLQ mechanism to send to the topic is not populated with the
correct binding name. In the previous version, it was working fine since
we were relying on a ThreadLocal. This is revamped recently to properly
introduce a binding name as a top level property in ProducerProperties.
However, this was not set on the internal producer properties used by the
DLQ. This PR addresses this issue.

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2417
Resolves #2418
2022-06-07 14:29:46 +02:00
Soby Chacko
ce6a03ee59 Kafk Streams binder message conversion issues
When native decoding is disabled and message conversion
is used  in Kafka Streams binder, it doesn't currently
carry the original headers forward. Fixing this issue.

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2411
2022-06-06 19:19:14 -04:00
ferblaca
ae0332b7d4 add the topic name information to the binder exception if a problem occurs in the provisioning phase 2022-06-06 10:46:03 -04:00
Tomek Szmytka
0bf50cbf85 Clean up resources on close. Allow cleanly terminating application process on context shutdown
Prevent IDE from using star imports

Update authors

Checkstyle fixes
2022-06-01 17:12:48 -04:00
amseager
90e5d5ee8d add missing backoff multiplier 2022-05-31 15:24:19 -04:00
Soby Chacko
605a1b1eda Update kafka client versions
3.2.0
2022-05-31 15:12:03 -04:00
Chris Bono
9f8dc3eded Fix compile error in LocalstackContainerTest
* Improve docker reuse in `LocalstackContainerTest`
2022-05-24 12:05:17 -04:00
Gary Russell
fd3ef3e849 GH-2297: Polish Concurrency Test 2022-05-16 09:23:10 -04:00
Chris Bono
cc05160cc0 Cleanup SerdeResolverUtils
* Refactor SerdeResolverUtils logic
* Add more tests in SerdeResolverUtilsTests
* Fixed some random test warnings

Fixes #2370

Code review feedback: remove FQ ref to inner method

Fix checkstyles
2022-05-13 15:20:47 -04:00
Gary Russell
bf810fbd53 GH-2297: Concurrency in Reactor Kafka Binder (#2396)
* GH-2297: Concurrency in Reactor Kafka Binder

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2297

* Remove unnecessary local var.

* Remove subscription cancellation; already handled by the super class.
2022-05-12 16:00:44 -04:00
Gary Russell
e522450f8e GH-2298: Refactor Common Binder Code to Core
Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2298

Polishing per review comments; javadocs.
2022-05-12 12:14:47 -04:00
Soby Chacko
dae959999a Binding name as a consumer/producer property
Make binding names available through Consumer/Producer properties.
Currently, the binders use a ThreadLocal to store the binding name for internal use.
These changes introduce the binding name as a property for both producer/consumer bindings.

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2380
2022-05-10 17:22:51 -04:00
Soby Chacko
112a93c5f0 Address KafkaHeader constant name changes 2022-05-09 17:04:32 -04:00
Artem Bilan
d8fd4c2aab GH-2359: Enable other KCL initial offsets
Fixes https://github.com/spring-cloud/spring-cloud-stream/issues/2359

The KCL mode does not apply at the moment `AT_TIMESTAMP` and `TRIM_HORIZON` initial offsets

Resolves #2365
* Change the `KinesisMessageChannelBinder` binder logic to create an explicit
`KinesisClientLibConfiguration` based on the provided properties including
custom stream iterator type
* Adjust `KinesisTestBinder` to be able to create a KCL/KPL based bindings
2022-05-04 15:34:16 +02:00
Chris Bono
210eb7e9f5 Consider generics when resolving Serdes for KafkaStreams binder
Fixes #2318
2022-05-03 11:45:20 -04:00
Chris Bono
37abca427d Remove Kafka binder dependence on KafkaAutoConfiguration
Fixes #2349

Use single instance of EmbeddedKafka for KafkaBinderBootstrapTest
2022-05-03 10:45:51 -04:00
Artem Bilan
53e6e1a85c Migrate Kinesis binder docs to the common place 2022-05-02 15:36:02 -04:00
Oleg Zhurakousky
818553179a fixing docs
polish
2022-05-02 20:20:49 +02:00
Chris Bono
3de0ed44e2 Use testcontainers for Rabbit binder integration tests
Fixes #2341

Resolves #2358
2022-04-22 12:15:10 +02:00
Lars Bilger
7d3a846590 KafkaBinderMetrics Improvements
- Calculate Kafka lag metric correctly even if there are no committed offsets for the group
 - Update authors and copyright years
 - Checkstyle fixes
2022-04-20 11:16:18 -04:00
buildmaster
53604e703a Bumping versions 2022-04-16 17:25:40 +00:00
ben-enfuse-io
e983366b8b Change compression level to Integer and only set if not null.
Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2328
2022-04-14 10:53:55 -04:00
ben-enfuse-io
a3b9c2f68d Change initialQuorumSize to initialGroupSize in docs.
Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2329
2022-04-13 18:07:57 -04:00
matthieu.dupuy
8682417af4 Add setter for Container Customizer
Add setter for Container Customizer on AbstractMessageChannelBinder.

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2345
2022-04-08 10:00:21 -04:00
Sid Sam
ad89acf62d Update documentation for ListenerContainerWithDlqAndRetryCustomizer
Based on suggestions in https://github.com/spring-cloud/spring-cloud-stream/issues/2313. Hope the edit helps.
2022-04-04 10:14:43 -04:00
buildmaster
5408935158 Bumping versions 2022-04-02 17:31:25 +00:00
Artem Bilan
d351afe27a GH-2311: Migrate AWS Kinesis binder to core
Fixes https://github.com/spring-cloud/spring-cloud-stream/issues/2311

* Use Tescontainers for Localstack
* Fix tests according new `core` status quo
* Improve `KinesisStreamProvisioner` to wait for stream become active
before moving on to other logic

Disable Kinesis binder LocalstackContainerTest on mac
2022-04-01 16:30:53 -04:00
Soby Chacko
b4c6ec8332 Concurrency property issues in KStream binder
In Kafka Streams binder, when using a function with camelcase names,
it causes issues for parsing binding level concurrency properties.

Fixing this issue.

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2316
2022-03-31 16:43:17 -04:00
buildmaster
85ffd4d92d Going back to snapshots 2022-03-31 20:05:34 +00:00
buildmaster
a9bd88acce Update SNAPSHOT to 4.0.0-M2 2022-03-31 20:02:59 +00:00
Oleg Zhurakousky
0ff2a4ad6a Remove dependency on old test module 2022-03-31 21:27:45 +02:00
spencergibb
158e4b22cf Fixes dependency management now that stream is a monorepo 2022-03-31 14:00:09 -04:00
spencergibb
6b9f6a5163 Moves core/docs to root of project for compatibility
Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2312
2022-03-30 16:05:21 -04:00
spencergibb
ee50d722a5 formatting 2022-03-30 12:44:56 -04:00
Christian Tzolov
d57b445a62 Add connectionFactory getter
Resolves #2314
2022-03-30 12:09:34 -04:00
Gary Russell
8a8a440c73 GH-2307: Fix Problems with Rabbit Declarables
Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2307

- when creating `Declarables` beans for redeclaration after a connetion failure
  2 DLX/DLQ bindings used the same bean name so the second one was never recovered/
- removing consumer `Declarable` beans from the `autoDeclareContext` was incomplete
- removing producer `Declarable` beans was not performed at all

- fix bean names; fix cleanup; add tests for all config scenarios
- check cleanup in all `RabbitBinderTests`

**cherry-pick to 3.2.x when the mono repo is established**
Resolves #2309
2022-03-25 11:21:55 +01:00