Commit Graph

303 Commits

Author SHA1 Message Date
Soby Chacko
e81976f7b5 Make MessageConverterHelper bean names distinct in binders. 2024-09-23 18:28:08 +02:00
Soby Chacko
dc38fd1199 Enhancements in Kafka binder FunctionBatchingConversionTests 2024-09-23 18:28:00 +02:00
Soby Chacko
e4a06ee686 Customizing batch message conversion behavior for Kafka binder
Continuation of the previous commit: 14c10462f8
2024-09-23 18:27:52 +02:00
Oleg Zhurakousky
2517760529 GH-2994, GH-2986 Add initial support for customizing Message Converter behavior
primarily during batch processing.
2024-09-23 18:27:42 +02:00
Kurt Hong
87ed345d1a KafkaBinderMetrics' metrics should be unregistered before it's thread
* KafkaBinderMetrics' metrics should be unregistered before it's threadpool is shutdown.
* update authors and copyright years
2024-08-23 13:40:16 -04:00
buildmaster
b5eccf6232 Bumping versions to 4.1.4-SNAPSHOT after release 2024-07-10 16:21:06 +00:00
buildmaster
2cbcd93232 Going back to snapshots 2024-07-10 16:21:05 +00:00
buildmaster
f459e2bfcd Update SNAPSHOT to 4.1.3 2024-07-10 16:17:09 +00:00
buildmaster
7c206241cc Bumping versions to 4.1.3-SNAPSHOT after release 2024-05-30 10:53:13 +00:00
buildmaster
084242ac9e Going back to snapshots 2024-05-30 10:53:12 +00:00
buildmaster
79d093e984 Update SNAPSHOT to 4.1.2 2024-05-30 10:49:10 +00:00
Soby Chacko
244de0fcea Rabbit biner AOT/native runtime hints
* Add missing native runtime hints for the Rabbit binder properties classes

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2938
2024-05-29 16:59:46 -04:00
Soby Chacko
0e2954c01e GH-2941: Kafka Streams binder composition issues
* When composed functions are used on component functions in the Kafka Streams binder,
  there is an issue in which the first function in the composition is invoked twice.
  Fixing this issue by ensuring that the function execution path is only invoked once.

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2941
2024-05-29 11:57:59 -04:00
Soby Chacko
13f94d266f ReactorKafkaBinderTests race condition issue
* There seems to be a race condition in ReactorKafkaBinderTests that causes
  tests to fail on CI occasionaly. Trying to address this by single dedicated
  topic per test.
2024-05-17 17:38:53 -04:00
Soby Chacko
fd9fa80055 Test cleanup
* Use `EnableAutoConfiguration` instead of `SpringBootApplication` in tests
2024-05-16 15:28:54 -04:00
Soby Chacko
49b7a8028a Kafka binder test cleanup
* Use `EnableAutoConfiguration` instead of `SpringBootApplication` in tests
2024-05-16 14:25:56 -04:00
Soby Chacko
bd2bf09d1e GH-2951: Batch consumer and DLQ issues
Batch consumer with `ListenerContainerWithDlqAndRetryCustomizer` should disable binder-based DLQ.

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2951

* Addressing PR review comments

* Addressing review
2024-05-16 11:56:55 -04:00
Soby Chacko
a3c4364740 GH-2949: KafkaBinderHealthIndicator consumer group.id
* Add a new property in `KafkaBinderConfigurationProperties` to allow
  the users to specify a `group.id` for the metadata consumer used by the
  health indicator.

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2949
2024-05-13 20:43:30 -04:00
Soby Chacko
a89331c095 GH-2948: Adding configuration for ReactorKafkaBinderHealthIndicator
Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2948
2024-05-13 13:49:25 -04:00
Soby Chacko
c2754de2ab GH-2943: EmbeddedKafka usage improvements in Kafka binder
* Currently, EmbeddedKafka is initialized as part of the class initialization in a few tests
  in the Kafka binder. This is preventing individual JUnit tests from being executed from an IDE (IntelliJ, for example).
  If we move this initialization to the JUnit  method, then that seems to be working.

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2943
2024-05-01 13:46:23 -04:00
Soby Chacko
d534e840dc GH-2942: EmbeddedKafka usage improvements in Kafka Streams binder
* Currently, EmbeddedKafka is initialized as part of the class initialization.
  This is preventing individual JUnit tests from being executed from an IDE (IntelliJ, for example).
  If we move this initialization to the JUnit `BeforeAll` method, then that seems to be working.

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2942
2024-05-01 11:52:47 -04:00
Soby Chacko
1d241583c3 GH-2941: Kafka Streams Component beans composition
Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2941

* Introduce function composition for Kafka Streams Component beans
2024-04-30 21:00:04 -04:00
Soby Chacko
7845be50ef GH-2932: Sanitize sensitive data on bindings endpoint
Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2932

Spring Boot provides `SanitizingFunction` to allow the applicaitons to clear out
sensitive data when using certain actuator endpoints. This feature can be
extended to custom endpoints as well. Enable the bindings actuator endpoint
to sanitze sensitive data based on user-provided logic in `SantizingFuction`
beans in the application.
2024-04-17 17:30:28 -04:00
Chris Bono
3d679c491c Fix custom test PulsarProducerFactory
Spring Pulsar 1.1.x introduced a new method `getPulsarClient`
on `PulsarProducerFactory` and the PulsarBinderIntegrationTests
has a custom factory that must be updated in order to run
on both Pulsar 1.0.x and 1.1.x (i.e. Spring Boot 3.2.x and 3.3.x).
2024-04-15 09:44:38 -04:00
Soby Chacko
3298d0d3c7 SBFM issues with afterSingletonsInstantiated
Related to GH-2935. Ensure that afterSingletonsInstantiated is called
even when autoStartup is not enabled.
2024-04-12 22:09:43 -04:00
Soby Chacko
5fb122e5a9 SBFM changes for Boot 3.2/3.3 compatibility 2024-04-12 17:40:52 -04:00
Soby Chacko
5ab62e1aa1 Fix Kafka binder tests for Boot 3.3.x compatibility 2024-04-12 16:42:17 -04:00
Soby Chacko
24556fd79e GH-2935: Call afterSingletonsInitialized() in SBFM
StreamsBuilderFactoryBean: Call afterSingletonsInitialized() programmatically in StreamsBuilderFactoryManager

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2935
2024-04-10 14:06:12 -04:00
buildmaster
99f1f1eb4a Bumping versions to 4.1.2-SNAPSHOT after release 2024-03-27 13:18:07 +00:00
buildmaster
3e7e65f259 Going back to snapshots 2024-03-27 13:18:06 +00:00
buildmaster
71727654ef Update SNAPSHOT to 4.1.1 2024-03-27 13:13:52 +00:00
WOODMARK\r.wiedmann.extern
eab86f81a8 GH-2922: Timestamp extractor - Kafka Streams 3.7.0
* Address immutability changes for the call to `Consumed#withTimestampExtractor`.

 * In 3.7.0, this call returns a new instance of `Consumed` as oppposed to
   mutating the existing instance in the previous versions. Address this
   change in behavior in the Kafka Streams binder.

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2922
2024-03-20 17:15:08 -04:00
Chris Bono
fcdaf2fdd8 Fix Pulsar binder for Java 21 (#2918)
Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2915

Running the Pulsar binder with Java 21 surfaced a bug where the layered
binder/binding config props are not properly applied to the targeted
consumer/producer builders.

The bug existed before Java 21 and is unrelated.

The binder/binding config props are converted to a Pulsar
ProducerConfigurationData object via Jackson. This conversion process iterates
over a map of config props. The iteration order of the map changed between Java
17 and 21. This results in object setters getting called in different orders, 
which in turn causes cross-field validation errors.

See #2915 for more details
2024-03-14 15:47:21 -04:00
Soby Chacko
c7b24a3d3c Pulsar binder integ test issues on JDK 21
* Disable the max-pending-messages in binderAndBindingPropsAreAppliedAndRespected test,
  since it is causing the test to fail on JDK 21. We will investigate why this propery has
  issues on JDK 21.

See this issue for details: https://github.com/spring-cloud/spring-cloud-stream/issues/2915
2024-03-07 11:50:49 -05:00
Soby Chacko
48dcc5dfd7 Fixing Pulsar binder tests
* Fixing Pulsar binder tests for Boot 3.2/3.3 compatibility
2024-03-04 16:18:54 -05:00
Soby Chacko
cabefa74fa Adjust Pulsar binder code for Boot 3.2/3.3 2024-03-04 15:35:49 -05:00
spencergibb
6e168d92e6 formatting 2024-03-04 15:06:58 -05:00
Soby Chacko
650edc180c Fixing Kafka Binder unit test issues 2024-03-04 14:57:04 -05:00
babubabu
432ce8fdd2 Fixing empty truststore-config in KBCP
* do not copy truststore, if "ssl.truststore.location" is set to empty string (e.g. ssl.truststore.location:{ENV_VARIABLE}) via KafkaBinderConfigurataionProperties.
2024-02-28 17:23:32 -05:00
LazroLeader
e7c1c7f769 Fixing concurrency issue in Kafka Streams binder
* Thread Safety Issue in serializeOnOutbound Method of KafkaStreamsMessageConversionDelegate 
* Wrapped perRecordContentTypeHolder with ThreadLocal
* update year and author
2024-02-15 15:58:07 -05:00
Artem Bilan
3e24d331db Rework observability for Rabbit Binder (#2903)
Fixes: #2901 

* Rework an observation for Rabbit Binder

The observation propagation doesn't work in multi-binder configuration

* Remove `ObservationAutoConfiguration` since it is not visible in case of multi-binder configuration
* Instead move `setObservationEnabled` flag setting to the `RabbitMessageChannelBinder`
* Add `io.micrometer.observation.ObservationRegistry` into `shared.beans` to make it visible for binder-specific application context
* Add `RabbitMultiBinderObservationTests` integration test where Rabbit Binder is in a multi-binder environment

As a side effect, this fixes an observation propagation for the Kafka binder as well in the multi-binder environment.
Its configuration is OK, but an `ObservationRegistry` must make it visible for the binder-specific application context.
See the mentioned `shared. beans`

Related to https://github.com/spring-cloud/spring-cloud-stream/issues/2901
Also see https://github.com/spring-cloud/spring-cloud-stream/issues/2902 for possible evolution

* Update the copyright of the affected classes
2024-02-14 18:53:06 -05:00
srinivasboini
4f69f05b82 GH-2882: Schema registry SSL config propagation
Fixes: #2882 

* Schema registry SSL config propagation to producer and consumer configurations
* Verify with tests
2024-02-08 16:43:29 -05:00
Soby Chacko
93cd52d29d KafkaTopicProvisioner - cleanup deprecation 2024-02-06 10:06:54 -05:00
Soby Chacko
3f9162b8b4 GH-2882: Schema registry SSL config propagation
Fixes: https://github.com/spring-cloud/spring-cloud-stream/issues/2882

When providing schema registry SSL truststore and keystore properties
as classpath URL's under Kafka binder configuration, the transformations
of them into corresponding filesystem URL's are not currenlty propagated
into producerProperties and consumerProperties under binder configuration.
This is ncessary for certain serializer/deserializer to work.
See the related GH issue for more context on this.
2024-02-01 13:44:53 -05:00
Soby Chacko
959c372c4b Address deprecations in Kafka Streams binder 2024-02-01 12:51:14 -05:00
Soby Chacko
4d1c5965ed KafkaBinderConfigurationProperties cleanup
- Address deprecation
 - Minor cleanup
2024-01-31 13:26:16 -05:00
Artem Bilan
bec9a1c996 Fix Checkstyle violation in KafkaConfigCustomizationTests 2024-01-18 15:52:55 -05:00
Artem Bilan
b098929048 Fix generics for BinderCustomizer contract
The expectation is like this `Binder<T, C extends ConsumerProperties, P extends ProducerProperties>`
so, the `BinderCustomizer` must be in extension bounds as well.

* Move Javadocs from the `BinderCustomizer.customize()` to the class level
* Add `BinderCustomizer` verification to the `KafkaConfigCustomizationTests`
2024-01-18 15:46:17 -05:00
Soby Chacko
adf6e2f781 Tests cleanup - Pulsar binder
- See previous commit for more details
2023-12-13 16:34:43 -05:00
Soby Chacko
e908de1dfd GH-2870: Issues when closing multi-binder contexts
- When closing multi-binder contexts, the application throws a StackOverflow error
   due to an unnecessary addition of the binder child context as a separate bean
   in the main context. This was done previously for testing purposes.
 - Disabling the affected tests in the Pulsar binder until we come up with a
   different solution.

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2870
2023-12-12 21:22:54 -05:00