Commit Graph

3928 Commits

Author SHA1 Message Date
buildmaster
ecaf757a38 Update SNAPSHOT to 4.1.0-M2 2023-09-27 10:29:34 +00:00
Soby Chacko
bca3dd5566 DltAwareProcessor improvements
- Instead of using a BiFunction as a delegate, use standard Function that takes the full record
 - Remove Supplier<Long> that was used to handle record time stamps since this is no longer needed
 - Docs cleanup
2023-09-26 21:34:22 -04:00
Chris Bono
ca2d3046b9 Remove Boot version from Pulsar binder pom.xml (#2819) 2023-09-26 09:33:44 -05:00
Chris Bono
b51e9e408b Update Pulsar binder to Spring Boot 3.2.0-M3 (#2818) 2023-09-26 09:07:20 -05:00
Oleg Zhurakousky
38ce1f79d9 Fix property value type 2023-09-26 07:09:46 +02:00
Chris Bono
e5c298e3bc Add Pulsar binder to the BOM 2023-09-25 10:35:49 -04:00
spencergibb
4c876b1a7f Adds antora extension and updates ui version 2023-09-22 16:13:10 -04:00
Oleg Zhurakousky
7ee4f61943 Updated observability doc 2023-09-22 15:16:00 +02:00
Soby Chacko
aeb88d01e9 KafkaBinderTests cleanup
- Remove deprecations
 - Other minor cleanup
2023-09-21 15:34:32 -04:00
Oleg Zhurakousky
2c7a883b8c Polish observability doc 2023-09-21 19:09:09 +02:00
Chris Bono
1ef65b340d Polish "GH-2806: Pulsar binder defaults properties config" 2023-09-21 10:01:40 -05:00
Soby Chacko
34aae4b62d GH-2806: Pulsar binder defaults properties config
- Ensure that the Pulsar binder default properties can be properly
   expressed via spring.cloud.stream.pulsar.default property prefix.
 - Add the binder child context bean with the name binderName_binderProducingContext
   into the parent application context so that individual beans from the binder context
   can be easily queried.

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2806
2023-09-21 09:37:54 -05:00
Soby Chacko
e06d4eedfd GH-2662: Kafka binder DLQ root cause message (#2812)
- Becasue NestedRuntimeException from Spring Framework core 6.x removed
   the getMessage method that included the detailMessage with cause in it,
   the Kafka binder DLQ records no longer include the cause message.
   Fix this issue by including the cause in the exception message.

See this comment for more details:
 - https://github.com/spring-cloud/spring-cloud-stream/issues/2662#issuecomment-1722849892
This fix is based on the following Spring Kafka commit.
 - 6f585058a6

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2662
2023-09-20 11:28:13 -04:00
Oleg Zhurakousky
ffabdb8bf9 GH-2782 Add initial observability documentation including reactive hooks
Resolves #2782
2023-09-20 17:23:49 +02:00
Soby Chacko
7e7688ce1b GH-2805: StreamBridge send and custom content-type
- When StreamBridge#send is called with binder-name and custom content-type,
   it does not honor the content-type value, but default to application/json.
   Fixing this issue for this call path by explicitly checking for any custom
   content-type provided on the binding.

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2805
Resolves #2813
2023-09-20 13:09:51 +02:00
Soby Chacko
316d393fe9 buildPulsarBinder script 2023-09-19 17:39:47 -04:00
omercelikceng
248fe01b9a Cloud Stream Core RuntimeHints
- Adding missing runtime hints for Stream Binding (InputBindingLifecycle and OutputBindingLifecycle)

Resolves spring-cloud#2793
2023-09-19 16:39:18 -04:00
Oleg Zhurakousky
32aae10ce3 Polish new antora doc 2023-09-18 16:38:29 +02:00
Soby Chacko
28ebf327fc Make AbstractBindingLifecycle public
See this discussion for details:
https://github.com/spring-cloud/spring-cloud-stream/pull/2793#issuecomment-1715420091
2023-09-14 10:57:29 -04:00
Chris Bono
db54f5e730 Update Pulsar binder to use new Spring Pulsar starter.
The PulsarBinder relies on the Spring Pulsar Spring Boot starter.
The starter moved out of the spring-pulsar core repo and into
Spring Boot proper. This commit updates the Pulsar binder to
use the new coordinates for the Spring Boot based starter.

* Additionally, the PulsarProperties were greatly reduced in the
  move to Spring Boot. As such, the binder exposes an extended
  set of config properties for producer/consumer (the initial
  set supported before the property reduction).
2023-09-14 10:33:06 -04:00
Soby Chacko
3c65104b1d GH-2796: Fix partition count related issues (#2807)
If a target middleware (Kafka for ex) topic has more partitions
than what is set on the partition-count producer property, the binder
never updates the original producer partition-count property to the
partition count from the middleware if it is higher than what was given
through the property. Because of this, te PartitionAwareFunctionWrapper
which evaluates and assigns the partition header in Spring Cloud Stream
does not compute the correct partition since the hash operation still
using the original partition-count from the producer binding property.
This commit is addressing this issue.

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2796
2023-09-14 08:52:14 -04:00
Oleg Zhurakousky
6a2ba83f6b Migrate RabbitMQ docs to antora 2023-09-14 11:48:01 +02:00
Soby Chacko
6d26ab75d2 Kafka binder antora docs cleanup 2023-09-14 11:48:01 +02:00
Oleg Zhurakousky
4ff968dd1b Finish doc organization for core and schema-registry 2023-09-14 11:48:01 +02:00
Oleg Zhurakousky
13c1afc351 Continue doc organization 2023-09-14 11:48:00 +02:00
Oleg Zhurakousky
f2085f7fba [Antora] Organize Intro and Main Concepts sections 2023-09-14 11:48:00 +02:00
Soby Chacko
6cfeb744d2 More Antora error fixing and cleanup 2023-09-14 11:48:00 +02:00
Oleg Zhurakousky
71e775a70f Initial antora error fixing 2023-09-14 11:48:00 +02:00
Marcin Grzejszczak
1084c3f67f WIP 2023-09-14 11:48:00 +02:00
Marcin Grzejszczak
c724c0d56e Enable Section Summary TOC for small pages 2023-09-14 11:48:00 +02:00
Marcin Grzejszczak
bb4186b856 Fix cross references 2023-09-14 11:48:00 +02:00
Marcin Grzejszczak
f2cff0c609 Remove includes 2023-09-14 11:48:00 +02:00
Marcin Grzejszczak
95192334fe Generate a default navigation 2023-09-14 11:48:00 +02:00
Marcin Grzejszczak
c2278acba9 Split files 2023-09-14 11:48:00 +02:00
Marcin Grzejszczak
73394a48ef Fix indentation for all pages 2023-09-14 11:48:00 +02:00
Marcin Grzejszczak
039893aadb Copy default antora files 2023-09-14 11:48:00 +02:00
Marcin Grzejszczak
ad39d8b9e6 Remove unnecessary asciidoc attributes 2023-09-14 11:48:00 +02:00
Marcin Grzejszczak
53dc8e151e Insert explicit ids for headers 2023-09-14 11:48:00 +02:00
Marcin Grzejszczak
4c404c6d51 Migrate Structure 2023-09-14 11:48:00 +02:00
Soby Chacko
9032e86068 Adding ci-pr.yml (#2808)
* Adding ci-pr.yml

* Add missing settings.xml

* [CI] Polish ci-pr.yml

- Remove extra profiles from settings.xml
- Simpify the workflow

---------

Co-authored-by: Chris Bono <cbono@vmware.com>
2023-09-13 18:16:52 -04:00
Soby Chacko
1a62bc3913 GH-2799: AOT processing and conversion service
When building in AOT mode, the BinderChildContextInitializer is unable to get access
to a SpEL expression conversion service in order to properly bind spring.cloud.stream
prefixed properties with values containing SpEL expressions such as headers[partition-key-expression].
However, we don't need to bind at this level of generality (spring.cloud.stream.*) in the
BinderChildContextInitializer since we are only looking for the user defined binders there.
To fix the issue, this commit explicitly binds only the spring.cloud.stream.binders.* properties
and ignores any other binding properties via spring.cloud.stream prefix. Any such conversions
will be done in later phases after the binder contexts are created.

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2799
2023-09-12 18:04:53 -04:00
Soby Chacko
0c256343c7 GH-2601: Kafka Streams Binder Docs Cleanup
Remove docs references to the deprecated/removed StreamsBuilderFactoryBeanCustomizer
in Spring for Apache Kafka in favor of StreamsBuilderFactoryBeanConfigurer.

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2601
2023-09-11 17:20:01 -04:00
Soby Chacko
b5f9c5093b Kafka binder RuntimeHints
- Adding missing runtime hints for Kafka binder properties

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2604
2023-09-08 18:27:24 -04:00
Soby Chacko
a1424fc4bf Schema Registry Property Migration
- Schema registry property migration to spring.cloud.stream prefix
 - Other cleanup in Schema Registry

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2687
2023-09-08 15:16:18 -04:00
Soby Chacko
26597db5b0 KafkaBinderHealthIndicator Changes
- Set considerDownWhenAnyPartitionHasNoLeader to true by default

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2760
2023-09-08 14:51:17 -04:00
Soby Chacko
bec40fba89 KafkaBinderConfigurationProperties native changes
- Earlier, we had to add Autowired on KafkaBinderConfigurationProperties constructor
   as there were some issues with runtime hints generation which caused issues when
   running an app in native mode. Spring Boot fixed these issues and we can remove
   this unnecessary Autowired from the constructor.

See the following issues from Spring Boot for more details.

https://github.com/spring-projects/spring-boot/issues/34507
https://github.com/spring-projects/spring-boot/issues/35564

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2640
2023-09-07 16:54:09 -04:00
Soby Chacko
00672ef5b6 Adding documentation for DltAwareProcessor 2023-09-07 12:37:19 -04:00
Soby Chacko
d822129d67 DltAwareProcessor enhancements
Cleaning up the code for the custom DltAwareProcessor
2023-09-06 20:36:35 -04:00
omercelik
149714062c GH-2787: NestedConfigurationProperties for native
Nested configurations(BindingProperties(ConsumerProperties and ProducerProperties))
in the BindingServiceProperties class do not work when built with native image.
For this reason, the consumer and producer properties under the Bindings configuration
do not work. Adding @NestedConfigurationProperty annotation on the property fields
in BindingProperties.

For more info see https://docs.spring.io/spring-boot/docs/current/reference/html/native-image.html#native-image.advanced

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2787
2023-09-05 16:37:03 -04:00
Gyurian Norbert
0949d4e145 GH-2761: Support non-CLASSPATH based certificates
If kafka truststore and keystore locations are not local files, then they
are converted to org.springframework.core.io.Resource resources, then copied
to local file system. This means that, paths can be defined as HTTP resources too.
Currently, the Kafka binder only supports CLASSPATH based resources. It would be
useful if we can support non-CLASSPATH like resources such as HTTP, so that if an
application uses config server for example to store certificates, then those will
be copied from it's HTTP endpoint to the local filesystem as Resources.

Checkstyle, documentation fixes.

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2761
2023-09-05 12:58:42 -04:00