* Kafka Streams - DLQ control per consumer binding
Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/800
* Fine-grained DLQ control and deserialization exception handlers per input binding
* Deprecate KafkaStreamsBinderConfigurationProperties.SerdeError in preference to the
new enum `KafkaStreamsBinderConfigurationProperties.DeserializationExceptionHandler`
based properties
* Add tests, modifying docs
* Addressing PR review comments
Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/715
When using transactions, binder retry, dlq cannot be used because the retry
runs within the transaction, which is undesirable if there is another resource
involved; also publishing to the DLQ could be rolled back.
Use the retry properties to configure an `AfterRollbackProcessor` to perform the
retry and DLQ publishing after the transaction has rolled back.
Add docs; move the container customizer invocation to the end.
* Reintrouce BinderHeaderMapper
Provide a custom header mapper that is identical to the DefaultKafkaHeaderMapper in Spring Kafka.
This is to address some interoperability issues between Spring Cloud Stream 3.0.x and 2.x apps,
where mime types in the header are not de-serialized properly. This custom BinderHeaderMapper
will be eventually deprecated and removed once the fixes are in Spring Kafka.
Resolves#796
* Addressing review
* polishing
When the same metric name is repeated, there are some registry implementations such as the
micrometer Prometheus registry fail to register the duplicate entry. Fixing this issue by
restricting the duplicate metric names not to be registered.
Also, address an issue with multiple processors and metrics in the same application
by prepending the application ID of the Kafka Streams processor in the metric name itself.
Resolves#788
Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/628
Allow users to specify the number of partitions in the dead letter topic.
If the property is set to 1 and the `minPartitionCount` binder property is 1,
override the default behavior and always publish to partition 0.
Export Kafka Streams metrics available through KafkaStreams#metrics into a Micrometer MeterRegistry.
Add documentation for how to access metrics.
Modify test to verify metrics.
Resolves#543
Use DeadLetterPublishingRecoverer from Spring Kafka instead of custom DLQ components in the binder.
Remove code that is no longer needed for DLQ purposes.
In Kafka Streams, always set group to application id if the user doesn't set an explicit group.
Upgrade Spring Kafka to 2.3.0 and SIK to 3.2.0
Resolves#761
* Check HeaderMapper bean with a well known name
* If a custom bean name for header mapper is not provided through the binder property headerMapperBeanName,
then look for a header mapper bean with the name kafkaHeaderMapper.
* Add tests to verify
Resolves#749
* Addressing PR review comments
Resolves#744
* Address deprecations in the pollable consumer
* Introduce a property for pollable time out in KafkaConsuerProperties
* Fix tests
* Add docs
* Addressin PR review comments
When StreamListener based processors used in Kafka Streams
applications, it is not possible to register state store beans
using StateStoreBuilder. This is allowed in the functional model.
This method of providing custom state stores is desired as this
gives the user more flexibility in configuring Serde's and other
properties on the state store. Adding this feature to StreamListner
based Kafka Streams processors.
Adding test to verify.
Adding docs.
Resolves#676
Fixing an issue that causes a race condition when multiple functions
are present in a Kafka Streams application by isolating the responsible
proxy factory per function and not shared.
When multiple Kafka Streams functions are present in an application,
it should be possible to set the application id per function.
When an application provides a bean of type Serde, then the binder
should try to introspect that bean to see if it can be matched for
any inbound or outbound serialization.
Adding tests to verify the changes.
Adding docs.
Resolves#734, #735, #736
During application startup, there are cases when state stores
might take longer time to initialize. The call to find the
state store in the binder (InteractiveQueryService) will fail in
those situations. Providing a basic retry mechanism using which
applicaitons can opt-in for this retry.
Adding properties for retrying state store at the binder level.
Adding docs.
Polishing.
Resolves#706
