Commit Graph

45 Commits

Author SHA1 Message Date
Gary Russell
fd3ef3e849 GH-2297: Polish Concurrency Test 2022-05-16 09:23:10 -04:00
Chris Bono
cc05160cc0 Cleanup SerdeResolverUtils
* Refactor SerdeResolverUtils logic
* Add more tests in SerdeResolverUtilsTests
* Fixed some random test warnings

Fixes #2370

Code review feedback: remove FQ ref to inner method

Fix checkstyles
2022-05-13 15:20:47 -04:00
Gary Russell
bf810fbd53 GH-2297: Concurrency in Reactor Kafka Binder (#2396)
* GH-2297: Concurrency in Reactor Kafka Binder

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2297

* Remove unnecessary local var.

* Remove subscription cancellation; already handled by the super class.
2022-05-12 16:00:44 -04:00
Gary Russell
e522450f8e GH-2298: Refactor Common Binder Code to Core
Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2298

Polishing per review comments; javadocs.
2022-05-12 12:14:47 -04:00
Soby Chacko
dae959999a Binding name as a consumer/producer property
Make binding names available through Consumer/Producer properties.
Currently, the binders use a ThreadLocal to store the binding name for internal use.
These changes introduce the binding name as a property for both producer/consumer bindings.

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2380
2022-05-10 17:22:51 -04:00
Soby Chacko
112a93c5f0 Address KafkaHeader constant name changes 2022-05-09 17:04:32 -04:00
Artem Bilan
d8fd4c2aab GH-2359: Enable other KCL initial offsets
Fixes https://github.com/spring-cloud/spring-cloud-stream/issues/2359

The KCL mode does not apply at the moment `AT_TIMESTAMP` and `TRIM_HORIZON` initial offsets

Resolves #2365
* Change the `KinesisMessageChannelBinder` binder logic to create an explicit
`KinesisClientLibConfiguration` based on the provided properties including
custom stream iterator type
* Adjust `KinesisTestBinder` to be able to create a KCL/KPL based bindings
2022-05-04 15:34:16 +02:00
Chris Bono
210eb7e9f5 Consider generics when resolving Serdes for KafkaStreams binder
Fixes #2318
2022-05-03 11:45:20 -04:00
Chris Bono
37abca427d Remove Kafka binder dependence on KafkaAutoConfiguration
Fixes #2349

Use single instance of EmbeddedKafka for KafkaBinderBootstrapTest
2022-05-03 10:45:51 -04:00
Artem Bilan
53e6e1a85c Migrate Kinesis binder docs to the common place 2022-05-02 15:36:02 -04:00
Oleg Zhurakousky
818553179a fixing docs
polish
2022-05-02 20:20:49 +02:00
Chris Bono
3de0ed44e2 Use testcontainers for Rabbit binder integration tests
Fixes #2341

Resolves #2358
2022-04-22 12:15:10 +02:00
Lars Bilger
7d3a846590 KafkaBinderMetrics Improvements
- Calculate Kafka lag metric correctly even if there are no committed offsets for the group
 - Update authors and copyright years
 - Checkstyle fixes
2022-04-20 11:16:18 -04:00
buildmaster
53604e703a Bumping versions 2022-04-16 17:25:40 +00:00
ben-enfuse-io
e983366b8b Change compression level to Integer and only set if not null.
Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2328
2022-04-14 10:53:55 -04:00
ben-enfuse-io
a3b9c2f68d Change initialQuorumSize to initialGroupSize in docs.
Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2329
2022-04-13 18:07:57 -04:00
matthieu.dupuy
8682417af4 Add setter for Container Customizer
Add setter for Container Customizer on AbstractMessageChannelBinder.

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2345
2022-04-08 10:00:21 -04:00
Sid Sam
ad89acf62d Update documentation for ListenerContainerWithDlqAndRetryCustomizer
Based on suggestions in https://github.com/spring-cloud/spring-cloud-stream/issues/2313. Hope the edit helps.
2022-04-04 10:14:43 -04:00
buildmaster
5408935158 Bumping versions 2022-04-02 17:31:25 +00:00
Artem Bilan
d351afe27a GH-2311: Migrate AWS Kinesis binder to core
Fixes https://github.com/spring-cloud/spring-cloud-stream/issues/2311

* Use Tescontainers for Localstack
* Fix tests according new `core` status quo
* Improve `KinesisStreamProvisioner` to wait for stream become active
before moving on to other logic

Disable Kinesis binder LocalstackContainerTest on mac
2022-04-01 16:30:53 -04:00
Soby Chacko
b4c6ec8332 Concurrency property issues in KStream binder
In Kafka Streams binder, when using a function with camelcase names,
it causes issues for parsing binding level concurrency properties.

Fixing this issue.

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2316
2022-03-31 16:43:17 -04:00
buildmaster
85ffd4d92d Going back to snapshots 2022-03-31 20:05:34 +00:00
buildmaster
a9bd88acce Update SNAPSHOT to 4.0.0-M2 2022-03-31 20:02:59 +00:00
Oleg Zhurakousky
0ff2a4ad6a Remove dependency on old test module 2022-03-31 21:27:45 +02:00
spencergibb
158e4b22cf Fixes dependency management now that stream is a monorepo 2022-03-31 14:00:09 -04:00
spencergibb
6b9f6a5163 Moves core/docs to root of project for compatibility
Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2312
2022-03-30 16:05:21 -04:00
spencergibb
ee50d722a5 formatting 2022-03-30 12:44:56 -04:00
Christian Tzolov
d57b445a62 Add connectionFactory getter
Resolves #2314
2022-03-30 12:09:34 -04:00
Gary Russell
8a8a440c73 GH-2307: Fix Problems with Rabbit Declarables
Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2307

- when creating `Declarables` beans for redeclaration after a connetion failure
  2 DLX/DLQ bindings used the same bean name so the second one was never recovered/
- removing consumer `Declarable` beans from the `autoDeclareContext` was incomplete
- removing producer `Declarable` beans was not performed at all

- fix bean names; fix cleanup; add tests for all config scenarios
- check cleanup in all `RabbitBinderTests`

**cherry-pick to 3.2.x when the mono repo is established**
Resolves #2309
2022-03-25 11:21:55 +01:00
Oleg Zhurakousky
bf8b963488 Make s-c-function deployer optioinal dependency in rabbit binder 2022-03-21 18:09:14 +01:00
Soby Chacko
840933d164 JUnit 5 migration for Kafka Streams binder tests 2022-03-15 18:45:05 -04:00
Gary Russell
6d8be51437 GH-2293: MessageConverter Bean Name
Add test to consume full `ReceiverRecord`.

Resolves #2295
2022-03-15 13:09:45 +01:00
Gary Russell
9ccf9ce48b GH-2293: Cancel Subscription on Stop 2022-03-12 13:16:40 -05:00
Soby Chacko
9b8ebe235e JUnit 5 migration for more Kafka binder tests 2022-03-11 16:50:54 -05:00
Soby Chacko
e51ade49db Kafka binder tests migration to JUnit 5 2022-03-11 15:07:37 -05:00
Soby Chacko
1c4f5e24dc Initial JUnit 5 migration for Kafka binder 2022-03-11 13:38:18 -05:00
Gary Russell
f3ccc7b72d GH-2293: Initial Commit Reactor-Kafka Binder
Note that it requires the message channel to be a `FluxMessageChannel`.

It currently only supports `Consumer<ReceiverRecord<?, ?>>` (no conversion).

The function binding should use the message channel directly instead of bridging
the direct channel to a reactive channel.

Producer Side PoC

Binder specific changes in ReactorKafkaBinder

Binder configuration

More changes to the reactive binder

Move KafkaBinderEnvironmentPostProcessor to the core module

Change binder key in spring.binders

Adding an e2e IT test for the reactive binder

Add lifecycle to integration adapters.

Remove unnecessary doStop().

Fix import order.
2022-03-10 17:22:20 -05:00
Soby Chacko
2f6de9263e Removed ununsed old release scripts 2022-03-09 13:36:32 -05:00
Soby Chacko
24035bad16 More README cleanup 2022-03-08 17:29:36 -05:00
Soby Chacko
3d85739251 README cleanup 2022-03-08 17:24:23 -05:00
Soby Chacko
5ddf93bd6d Version updates
Spring kafka - 3.0.0-SNAPSHOT
Kafka client - 3.1.0

Remove the temporarry changes to micrometer dependency version
2022-03-08 10:48:46 -05:00
Soby Chacko
21999e2660 Build cleanup 2022-03-07 18:17:41 -05:00
Soby Chacko
9fb42b65d9 Changing the micrometer version in Kafka binder
Temporarily changing this to match with Spring-Kafka's.
2022-03-07 11:34:12 -05:00
Soby Chacko
e1232ae731 Restructuring the repository 2022-02-23 19:42:29 -05:00
Soby Chacko
585f77e589 Relocate both Kafka and Rabbit binders within core 2022-02-23 15:41:13 -05:00