Commit Graph

4100 Commits

Author SHA1 Message Date
Oleg Zhurakousky
add6b0178e GH-2994, GH-2986 Add documentation explaining batch type conversion 2024-09-23 18:28:17 +02:00
Soby Chacko
e81976f7b5 Make MessageConverterHelper bean names distinct in binders. 2024-09-23 18:28:08 +02:00
Soby Chacko
dc38fd1199 Enhancements in Kafka binder FunctionBatchingConversionTests 2024-09-23 18:28:00 +02:00
Soby Chacko
e4a06ee686 Customizing batch message conversion behavior for Kafka binder
Continuation of the previous commit: 14c10462f8
2024-09-23 18:27:52 +02:00
Oleg Zhurakousky
2517760529 GH-2994, GH-2986 Add initial support for customizing Message Converter behavior
primarily during batch processing.
2024-09-23 18:27:42 +02:00
Oleg Zhurakousky
6c3fc4c350 Merge pull request #3001 from hightea/cherry-pick-avoid-duplicate-logs
Avoid duplicate error logging when multiple destinations per binding
2024-09-08 04:16:13 -07:00
Théophane Charbonnier
110061b04e Avoid duplicate error logging when multiple destinations per binding 2024-09-06 15:28:21 +02:00
Kurt Hong
87ed345d1a KafkaBinderMetrics' metrics should be unregistered before it's thread
* KafkaBinderMetrics' metrics should be unregistered before it's threadpool is shutdown.
* update authors and copyright years
2024-08-23 13:40:16 -04:00
Artem Bilan
f146ce3eac GH-2989 - Make SpEL converter stringify primitive values
Resolves #2989
2024-08-19 12:10:22 +02:00
buildmaster
3dd4a0747b Bumping versions 2024-08-02 10:15:33 +00:00
Artem Bilan
3b5bb2e0d9 Remove redundant delaySubscription from FunctionConfiguration
Related to: https://github.com/spring-projects/spring-integration/issues/9362

After the fix in Spring Integration: bdcb856a90
we ended up in a deadlock situation with a `beginPublishingTrigger` in the `FunctionConfiguration`
used for the `delaySubscription()` on an original `Publisher`.
The `FluxMessageChannel` uses its own `delaySubscription()` until the channel has its subscribers.
Apparently the logic before was with errors, so the `FluxMessageChannel` was marked as active
even if its subscriber is not ready yet, leading to famous `Dispatcher does not have subscribers` error.
So, looks like this `beginPublishingTrigger` was introduced back in days in Spring Cloud Stream
to mitigate that situation until we really emit a `BindingCreatedEvent`.

The deadlock (and the flaw, respectively) is with the `setupBindingTrigger()` method implementation
where `FluxMessageChannel` now "really" delays a subscription to the provided `Publisher`,
therefore not triggering that `Mono.create()` fulfilment immediately.
The `BindingCreatedEvent` arrives earlier, than we have a subscriber on the channel,
but `triggerRef.get()` is `null`, so we don't `success()` it and in the end don't subscribe
to an original `Publisher` since `delaySubscription()` on it is never completed.

Since `FunctionConfiguration` fully relies on `IntegrationFlow.from(Publisher)`,
which ends up with the mentioned  `FluxMessageChannel.subscribeTo()` and its own `delaySubscription()`
(which, in turn, apparently fixed now), we don't need our own `delaySubscription()` any more.
Therefore the fix in this PR is to propose to remove `beginPublishingTrigger` logic altogether.
2024-07-26 15:57:12 -04:00
Oleg Zhurakousky
f356599503 GH-1163-function
Added test for the fix provided in s-c-function
2024-07-17 17:20:32 +02:00
Oleg Zhurakousky
07e7d81e2d GH-2971 Properly handle Kafka tombstone payload conversion
See the corresponding commit in s-c-function
Resolves #2971
2024-07-17 16:19:12 +02:00
buildmaster
b5eccf6232 Bumping versions to 4.1.4-SNAPSHOT after release 2024-07-10 16:21:06 +00:00
buildmaster
2cbcd93232 Going back to snapshots 2024-07-10 16:21:05 +00:00
buildmaster
f459e2bfcd Update SNAPSHOT to 4.1.3 2024-07-10 16:17:09 +00:00
buildmaster
fc715b4e6e Bumping versions 2024-07-08 10:16:57 +00:00
Ryan Baxter
6f4498329e Upgrade antora 2024-06-25 14:28:48 -04:00
Soby Chacko
df3c467834 Checkstyle fixes 2024-06-18 19:24:09 -04:00
Soby Chacko
604649d083 GH-2961: Concurrent writes and partition header
- When concurrent threads publish to a binding, PartitionAwareFunctionWrapper
  resets to null between invocations. Addressing this issue by guarding this reset
  from occurring if the partiton header on the producer is found.

Fixes https://github.com/spring-cloud/spring-cloud-stream/issues/2961
2024-06-18 19:20:07 -04:00
Soby Chacko
4b2810f49a Fixing checkstyle issues 2024-06-18 19:08:56 -04:00
Théophane Charbonnier
2a5938e0fe GH-2961 : failing test for Header scst_partition randomly ommitted 2024-06-18 19:03:40 -04:00
Chanhyeong Cho
ae32191310 Minor cleanup
* Remove reference to @EnableBinding in DefaultBinding
* Docs cleanup
2024-06-17 15:20:50 -04:00
Oleg Zhurakousky
9e6caef371 GH-2955 Remove dynamicDestination property from BindingServiceProperties
Resolves #2950
2024-06-07 08:28:41 +02:00
Oleg Zhurakousky
d6550ed324 GH-2956 Fix assertion for multiple binders
Resolves #2956
2024-06-04 08:36:53 +02:00
buildmaster
7c206241cc Bumping versions to 4.1.3-SNAPSHOT after release 2024-05-30 10:53:13 +00:00
buildmaster
084242ac9e Going back to snapshots 2024-05-30 10:53:12 +00:00
buildmaster
79d093e984 Update SNAPSHOT to 4.1.2 2024-05-30 10:49:10 +00:00
Soby Chacko
244de0fcea Rabbit biner AOT/native runtime hints
* Add missing native runtime hints for the Rabbit binder properties classes

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2938
2024-05-29 16:59:46 -04:00
Soby Chacko
0e2954c01e GH-2941: Kafka Streams binder composition issues
* When composed functions are used on component functions in the Kafka Streams binder,
  there is an issue in which the first function in the composition is invoked twice.
  Fixing this issue by ensuring that the function execution path is only invoked once.

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2941
2024-05-29 11:57:59 -04:00
Oleg Zhurakousky
625f25130d GH-2930 Add configuration metadata
Resolves #2930
2024-05-27 09:10:53 +02:00
Oleg Zhurakousky
7f0021b394 GH-2728 Change the order of the refresh for multicontext 2024-05-27 09:05:50 +02:00
Soby Chacko
13f94d266f ReactorKafkaBinderTests race condition issue
* There seems to be a race condition in ReactorKafkaBinderTests that causes
  tests to fail on CI occasionaly. Trying to address this by single dedicated
  topic per test.
2024-05-17 17:38:53 -04:00
Soby Chacko
fd9fa80055 Test cleanup
* Use `EnableAutoConfiguration` instead of `SpringBootApplication` in tests
2024-05-16 15:28:54 -04:00
Soby Chacko
49b7a8028a Kafka binder test cleanup
* Use `EnableAutoConfiguration` instead of `SpringBootApplication` in tests
2024-05-16 14:25:56 -04:00
Soby Chacko
bd2bf09d1e GH-2951: Batch consumer and DLQ issues
Batch consumer with `ListenerContainerWithDlqAndRetryCustomizer` should disable binder-based DLQ.

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2951

* Addressing PR review comments

* Addressing review
2024-05-16 11:56:55 -04:00
Soby Chacko
a3c4364740 GH-2949: KafkaBinderHealthIndicator consumer group.id
* Add a new property in `KafkaBinderConfigurationProperties` to allow
  the users to specify a `group.id` for the metadata consumer used by the
  health indicator.

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2949
2024-05-13 20:43:30 -04:00
Soby Chacko
7e20c9efbe Adding docs for ReactorKafkaBinderHealthIndicator
See https://github.com/spring-cloud/spring-cloud-stream/issues/2948
2024-05-13 16:08:11 -04:00
Soby Chacko
a89331c095 GH-2948: Adding configuration for ReactorKafkaBinderHealthIndicator
Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2948
2024-05-13 13:49:25 -04:00
kutmk
1f41055917 Add @EnableTestBinder annotation (#2947)
* Add @EnableTestBinder annotation
* Use @EnableTestBinder in tests
* Update copyright and author tag
* Add javadoc
* Update explanation to use @EnableTestBinder

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2931
2024-05-13 11:29:00 -04:00
Oleg Zhurakousky
ee3faf605a GH-2946 Remove spring.binders from TestBinder
Resolves #2946
2024-05-08 18:28:14 +02:00
Soby Chacko
c2754de2ab GH-2943: EmbeddedKafka usage improvements in Kafka binder
* Currently, EmbeddedKafka is initialized as part of the class initialization in a few tests
  in the Kafka binder. This is preventing individual JUnit tests from being executed from an IDE (IntelliJ, for example).
  If we move this initialization to the JUnit  method, then that seems to be working.

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2943
2024-05-01 13:46:23 -04:00
Soby Chacko
d534e840dc GH-2942: EmbeddedKafka usage improvements in Kafka Streams binder
* Currently, EmbeddedKafka is initialized as part of the class initialization.
  This is preventing individual JUnit tests from being executed from an IDE (IntelliJ, for example).
  If we move this initialization to the JUnit `BeforeAll` method, then that seems to be working.

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2942
2024-05-01 11:52:47 -04:00
Soby Chacko
1d241583c3 GH-2941: Kafka Streams Component beans composition
Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2941

* Introduce function composition for Kafka Streams Component beans
2024-04-30 21:00:04 -04:00
Soby Chacko
f230d873c7 Adding Kafka docker-compose script for kraft 2024-04-30 10:53:43 -04:00
Soby Chacko
9cbd45910e GH-2926: Clarify docs on observablity when consuming in batches
Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2926

* Clarify docs on Observability when consuming records in batch mode in Kafka binder
2024-04-29 18:33:07 -04:00
Soby Chacko
7845be50ef GH-2932: Sanitize sensitive data on bindings endpoint
Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2932

Spring Boot provides `SanitizingFunction` to allow the applicaitons to clear out
sensitive data when using certain actuator endpoints. This feature can be
extended to custom endpoints as well. Enable the bindings actuator endpoint
to sanitze sensitive data based on user-provided logic in `SantizingFuction`
beans in the application.
2024-04-17 17:30:28 -04:00
Chris Bono
3d679c491c Fix custom test PulsarProducerFactory
Spring Pulsar 1.1.x introduced a new method `getPulsarClient`
on `PulsarProducerFactory` and the PulsarBinderIntegrationTests
has a custom factory that must be updated in order to run
on both Pulsar 1.0.x and 1.1.x (i.e. Spring Boot 3.2.x and 3.3.x).
2024-04-15 09:44:38 -04:00
Soby Chacko
3298d0d3c7 SBFM issues with afterSingletonsInstantiated
Related to GH-2935. Ensure that afterSingletonsInstantiated is called
even when autoStartup is not enabled.
2024-04-12 22:09:43 -04:00
Soby Chacko
5fb122e5a9 SBFM changes for Boot 3.2/3.3 compatibility 2024-04-12 17:40:52 -04:00