Fixesspring-cloud/spring-cloud-stream-binder-kafka#318
* Use `ToDoubleFunction`-based `MeterRegistry.gauge()` variant to really
calculate a `lag` at runtime instead of statically defined before
* Add `KafkaBinderActuatorTests` integration test to demonstrate how
`Binder` is declared in the separate child context and how
`KafkaBinderMetrics` is not visible from the parent context.
This test also verify the real `gauge` value for the `consumer lag`
and should be used in the future to verify the `KafkaBinderMetrics`
exposure removing the code after TODO in the `KafkaMetricsTestConfig`
ConfigurationProperties bean provided by Kafka Streams binder
extends from `KafkaBinderConfigurationProperties` used by Kafka binder.
It creates a conflict when autowiring this bean from Kafka binder configuration.
This prevents an application to have both binders in the classpath.
Change the creation of this ConfigurationProperties bean so that it
avoids creating bean using EnbaleConfigurationProperties and then autowiring,
but directly create the Bean using `@Bean`. This prevents the conflict.
Resolves#244Resolves#315
- Rename spring-cloud-stream-binder-kstream to spring-cloud-stream-binder-kafka-streams
- Corresponding changes in maven pom.xml files
- Rename relevant classes to prefix with KafkaStreams instead of KStream
- Corresponding package changes from org.springframework.cloud.stream.kstream to
org.springframework.cloud.stream.kafka.streams
- Organize all the configuration property classes in a properties package
- Remove kstream from all the properties exposed by Apache Kafka Streams binder
- Classes that need not be public are now moved to package access level
- Test changes
- More javadocs to classes
Resolves#246
- In kafka streams applications, it is essential to have multiple bindings for
various target types such as KStream, KTable etc. These changes allow to have
more than one type of target type bindings on a single StreamListenerMethod.
- Currently support KStream and KTable target types
- Refactoring the KStream listener orchestrator strategy
- Input bindings are initiated through a proxy and later on wrapped with the
real target created from a common Kafka Streams StreamsBuilder
- Adding tests to verify multiple input bindings for KStream and KTable
Resolves#298Resolves#303
Polishing
Materializing KTables as state stores
Void return types on kafka streams StreamListener methods
Polishing
Fixes https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/301
For the `AdminClient`, arbitrary Kafka properties (set via the binder `configuration` property) should
supersede any boot properties. There was already special handling for the bootstrap servers, but
other arbitrary properties were ignored.
Add tests, including a test to verify the proper override of boot's broker list if appropriate.
- Add support for KIP-161: streams deserialization exception handlers
- Provide out of the box LogAndContinue and LogAndFail exception handlers
- Introduce a new exception handler that sends records in error on
deserialization to a DLQ
- Ensure that the exception handlers work in both when native decoding
is enabled (Kafka is doing deserialization) and when native decoding
is disabled (Binder is doing deserialization)
- Enhancements on the logic of native decoding in binder.
- General refactoring and cleanup
- Adding more tests
Resolves#275, #281
Introducing multiple StreamListener methods in Kafka Streams apps.
- Multiple stream listener methods now have individual backing
StreamBuilder and configuration.
- Register StreamsBuilderFactoryBean programmatically
Resolves#299, #300
Instead of relying on a property based approach, use proper
multiple output bindings to support the branching feature.
This is accomplished through a combination of using `SendTo`
annotation with multiple outputs and overriding the default
StreamListener method setup orchestration in the binder.
Resolves#265
- Ensure that branching can be done and any message conversions
done on all outbound data to multiple topics
- Refine MessageConversion logic in KStream binder
- Refactor message conversion code into a specific class
- Cleanup and refactoring
- Test to verify branching support
- More refinements in the way nativeEncoding/nativeDecoding logic works in KStream binder
- Doc changes for KStream binder
Aligning semantics of native encoding with core spring cloud stream.
With this change, if nativeEncoding is set, any message conversion is
skipped by the framework. It is up to the application to ensure that
the data sent on the outbound is of type byte[], otherwise, it fails.
If the nativeEncoding is false, which is the default, the binder does
the message conversion before sending the data on the outbound.
Corresponding changes in tests
Resolves#267
Resolves#259
- Remove the kafka server dependency from build/binder artifact
- Remove AdminUtilOperation and KafkaAdminUtilOperation
- Rely on AdminClient for provisioning operations
- Update KStream components with the new changes
- Test updates
- Polishing
Add timeout to all the blocking AdminClient calls
Addressing PR review comments
Addressing PR review comments
Addressing PR review
Update SK, SIK to 2.1.0.RELEASE and 3.0.0.RELEASE respectively
Update Kafka Streams class name changes
Fixes https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/251
When using native decoding, the message emitted by the binder has no `id` or `timestamp`
headers. This can cause problems if the first SI endpoint uses a JDBC message store,
for example.
Add the ability to configure whether these headers are populated as well as the ability
to inject a custom message converter.
