Commit Graph

311 Commits

Author SHA1 Message Date
Oleg Zhurakousky
14c10462f8 GH-2994, GH-2986 Add initial support for customizing Message Converter behavior
primarily during batch processing.
2024-09-16 16:10:38 +02:00
Chris Bono
64dd18cc8e Use PulsarTopicBuilder in Pulsar binder (#2999)
Spring for Apache Pulsar introduced support for default tenant and namespace
for Pulsar topics in 6d23378fbb.
This ensures that all topic names are fully-qualified (using the default
tenant and namespace when not fully-qualified).
2024-09-03 19:20:18 -04:00
Kurt Hong
fe1392efb0 KafkaBinderMetrics' metrics should be unregistered before it's thread
* KafkaBinderMetrics' metrics should be unregistered before it's threadpool is shutdown.
* update authors and copyright years
2024-08-23 13:36:41 -04:00
Ömer Çelik
8e5f67a1a6 Code cleanup
Resolves #2990
2024-08-20 15:03:39 +02:00
Ömer Çelik
7fefaa38c0 Code cleanup 2024-08-20 12:53:03 +02:00
buildmaster
1de0e1c690 Going back to snapshots 2024-08-19 18:09:13 +00:00
buildmaster
b1507bdc01 Update SNAPSHOT to 4.2.0-M1 2024-08-19 18:06:14 +00:00
buildmaster
26dddfe1c1 Going back to snapshots 2024-08-19 17:29:31 +00:00
buildmaster
15f5791281 Update SNAPSHOT to 4.2.0-M1 2024-08-19 17:26:43 +00:00
buildmaster
71e0c92e7c Bumping versions 2024-08-06 18:45:33 +00:00
spencergibb
721b904ae2 Disables tests 2024-08-06 13:50:18 -04:00
spencergibb
8eda26210a Disables tests 2024-08-05 16:06:14 -04:00
spencergibb
42c42104ea Fixed compile errors with replacements for deprecations 2024-08-05 15:53:43 -04:00
buildmaster
b5eccf6232 Bumping versions to 4.1.4-SNAPSHOT after release 2024-07-10 16:21:06 +00:00
buildmaster
2cbcd93232 Going back to snapshots 2024-07-10 16:21:05 +00:00
buildmaster
f459e2bfcd Update SNAPSHOT to 4.1.3 2024-07-10 16:17:09 +00:00
buildmaster
7c206241cc Bumping versions to 4.1.3-SNAPSHOT after release 2024-05-30 10:53:13 +00:00
buildmaster
084242ac9e Going back to snapshots 2024-05-30 10:53:12 +00:00
buildmaster
79d093e984 Update SNAPSHOT to 4.1.2 2024-05-30 10:49:10 +00:00
Soby Chacko
244de0fcea Rabbit biner AOT/native runtime hints
* Add missing native runtime hints for the Rabbit binder properties classes

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2938
2024-05-29 16:59:46 -04:00
Soby Chacko
0e2954c01e GH-2941: Kafka Streams binder composition issues
* When composed functions are used on component functions in the Kafka Streams binder,
  there is an issue in which the first function in the composition is invoked twice.
  Fixing this issue by ensuring that the function execution path is only invoked once.

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2941
2024-05-29 11:57:59 -04:00
Soby Chacko
13f94d266f ReactorKafkaBinderTests race condition issue
* There seems to be a race condition in ReactorKafkaBinderTests that causes
  tests to fail on CI occasionaly. Trying to address this by single dedicated
  topic per test.
2024-05-17 17:38:53 -04:00
Soby Chacko
fd9fa80055 Test cleanup
* Use `EnableAutoConfiguration` instead of `SpringBootApplication` in tests
2024-05-16 15:28:54 -04:00
Soby Chacko
49b7a8028a Kafka binder test cleanup
* Use `EnableAutoConfiguration` instead of `SpringBootApplication` in tests
2024-05-16 14:25:56 -04:00
Soby Chacko
bd2bf09d1e GH-2951: Batch consumer and DLQ issues
Batch consumer with `ListenerContainerWithDlqAndRetryCustomizer` should disable binder-based DLQ.

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2951

* Addressing PR review comments

* Addressing review
2024-05-16 11:56:55 -04:00
Soby Chacko
a3c4364740 GH-2949: KafkaBinderHealthIndicator consumer group.id
* Add a new property in `KafkaBinderConfigurationProperties` to allow
  the users to specify a `group.id` for the metadata consumer used by the
  health indicator.

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2949
2024-05-13 20:43:30 -04:00
Soby Chacko
a89331c095 GH-2948: Adding configuration for ReactorKafkaBinderHealthIndicator
Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2948
2024-05-13 13:49:25 -04:00
Soby Chacko
c2754de2ab GH-2943: EmbeddedKafka usage improvements in Kafka binder
* Currently, EmbeddedKafka is initialized as part of the class initialization in a few tests
  in the Kafka binder. This is preventing individual JUnit tests from being executed from an IDE (IntelliJ, for example).
  If we move this initialization to the JUnit  method, then that seems to be working.

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2943
2024-05-01 13:46:23 -04:00
Soby Chacko
d534e840dc GH-2942: EmbeddedKafka usage improvements in Kafka Streams binder
* Currently, EmbeddedKafka is initialized as part of the class initialization.
  This is preventing individual JUnit tests from being executed from an IDE (IntelliJ, for example).
  If we move this initialization to the JUnit `BeforeAll` method, then that seems to be working.

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2942
2024-05-01 11:52:47 -04:00
Soby Chacko
1d241583c3 GH-2941: Kafka Streams Component beans composition
Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2941

* Introduce function composition for Kafka Streams Component beans
2024-04-30 21:00:04 -04:00
Soby Chacko
7845be50ef GH-2932: Sanitize sensitive data on bindings endpoint
Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2932

Spring Boot provides `SanitizingFunction` to allow the applicaitons to clear out
sensitive data when using certain actuator endpoints. This feature can be
extended to custom endpoints as well. Enable the bindings actuator endpoint
to sanitze sensitive data based on user-provided logic in `SantizingFuction`
beans in the application.
2024-04-17 17:30:28 -04:00
Chris Bono
3d679c491c Fix custom test PulsarProducerFactory
Spring Pulsar 1.1.x introduced a new method `getPulsarClient`
on `PulsarProducerFactory` and the PulsarBinderIntegrationTests
has a custom factory that must be updated in order to run
on both Pulsar 1.0.x and 1.1.x (i.e. Spring Boot 3.2.x and 3.3.x).
2024-04-15 09:44:38 -04:00
Soby Chacko
3298d0d3c7 SBFM issues with afterSingletonsInstantiated
Related to GH-2935. Ensure that afterSingletonsInstantiated is called
even when autoStartup is not enabled.
2024-04-12 22:09:43 -04:00
Soby Chacko
5fb122e5a9 SBFM changes for Boot 3.2/3.3 compatibility 2024-04-12 17:40:52 -04:00
Soby Chacko
5ab62e1aa1 Fix Kafka binder tests for Boot 3.3.x compatibility 2024-04-12 16:42:17 -04:00
Soby Chacko
24556fd79e GH-2935: Call afterSingletonsInitialized() in SBFM
StreamsBuilderFactoryBean: Call afterSingletonsInitialized() programmatically in StreamsBuilderFactoryManager

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2935
2024-04-10 14:06:12 -04:00
buildmaster
99f1f1eb4a Bumping versions to 4.1.2-SNAPSHOT after release 2024-03-27 13:18:07 +00:00
buildmaster
3e7e65f259 Going back to snapshots 2024-03-27 13:18:06 +00:00
buildmaster
71727654ef Update SNAPSHOT to 4.1.1 2024-03-27 13:13:52 +00:00
WOODMARK\r.wiedmann.extern
eab86f81a8 GH-2922: Timestamp extractor - Kafka Streams 3.7.0
* Address immutability changes for the call to `Consumed#withTimestampExtractor`.

 * In 3.7.0, this call returns a new instance of `Consumed` as oppposed to
   mutating the existing instance in the previous versions. Address this
   change in behavior in the Kafka Streams binder.

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2922
2024-03-20 17:15:08 -04:00
Chris Bono
fcdaf2fdd8 Fix Pulsar binder for Java 21 (#2918)
Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2915

Running the Pulsar binder with Java 21 surfaced a bug where the layered
binder/binding config props are not properly applied to the targeted
consumer/producer builders.

The bug existed before Java 21 and is unrelated.

The binder/binding config props are converted to a Pulsar
ProducerConfigurationData object via Jackson. This conversion process iterates
over a map of config props. The iteration order of the map changed between Java
17 and 21. This results in object setters getting called in different orders, 
which in turn causes cross-field validation errors.

See #2915 for more details
2024-03-14 15:47:21 -04:00
Soby Chacko
c7b24a3d3c Pulsar binder integ test issues on JDK 21
* Disable the max-pending-messages in binderAndBindingPropsAreAppliedAndRespected test,
  since it is causing the test to fail on JDK 21. We will investigate why this propery has
  issues on JDK 21.

See this issue for details: https://github.com/spring-cloud/spring-cloud-stream/issues/2915
2024-03-07 11:50:49 -05:00
Soby Chacko
48dcc5dfd7 Fixing Pulsar binder tests
* Fixing Pulsar binder tests for Boot 3.2/3.3 compatibility
2024-03-04 16:18:54 -05:00
Soby Chacko
cabefa74fa Adjust Pulsar binder code for Boot 3.2/3.3 2024-03-04 15:35:49 -05:00
spencergibb
6e168d92e6 formatting 2024-03-04 15:06:58 -05:00
Soby Chacko
650edc180c Fixing Kafka Binder unit test issues 2024-03-04 14:57:04 -05:00
babubabu
432ce8fdd2 Fixing empty truststore-config in KBCP
* do not copy truststore, if "ssl.truststore.location" is set to empty string (e.g. ssl.truststore.location:{ENV_VARIABLE}) via KafkaBinderConfigurataionProperties.
2024-02-28 17:23:32 -05:00
LazroLeader
e7c1c7f769 Fixing concurrency issue in Kafka Streams binder
* Thread Safety Issue in serializeOnOutbound Method of KafkaStreamsMessageConversionDelegate 
* Wrapped perRecordContentTypeHolder with ThreadLocal
* update year and author
2024-02-15 15:58:07 -05:00
Artem Bilan
3e24d331db Rework observability for Rabbit Binder (#2903)
Fixes: #2901 

* Rework an observation for Rabbit Binder

The observation propagation doesn't work in multi-binder configuration

* Remove `ObservationAutoConfiguration` since it is not visible in case of multi-binder configuration
* Instead move `setObservationEnabled` flag setting to the `RabbitMessageChannelBinder`
* Add `io.micrometer.observation.ObservationRegistry` into `shared.beans` to make it visible for binder-specific application context
* Add `RabbitMultiBinderObservationTests` integration test where Rabbit Binder is in a multi-binder environment

As a side effect, this fixes an observation propagation for the Kafka binder as well in the multi-binder environment.
Its configuration is OK, but an `ObservationRegistry` must make it visible for the binder-specific application context.
See the mentioned `shared. beans`

Related to https://github.com/spring-cloud/spring-cloud-stream/issues/2901
Also see https://github.com/spring-cloud/spring-cloud-stream/issues/2902 for possible evolution

* Update the copyright of the affected classes
2024-02-14 18:53:06 -05:00
srinivasboini
4f69f05b82 GH-2882: Schema registry SSL config propagation
Fixes: #2882 

* Schema registry SSL config propagation to producer and consumer configurations
* Verify with tests
2024-02-08 16:43:29 -05:00