Fixes#542
- add option for `autoCommitOnError`
- make `autoCommitOnError` follow the state of `enableDlq` - no need to suppress commits if messages will be sent elsewhere
Honour the `SCS_KAFKA_TEST_EMBEDDED` flag when set as an environment variable
or Java property: when explicitly set to `false`, use an external broker and observe the
`SCS_EXTERNAL_SERVERS_REQUIRED` property (i.e. fail if server is missing). By default,
continue to run tests using an embedded broker.
Fixes#509
- prevent autoconfiguration from reducing the existing partition count and/or replication factor
Split `autoConfigureTopics` in two separate settings, one for `autoCreateTopics`
and one for `autoAddPartitions` (to existing topics)
Renamed `defaultMinPartitionCount` to just `minPartitionCount` on the Kafka binder
Add a flag for controlling topic autoconfiguration by the Kafka binder
Fixes#504
- introduces a new binder setting `autoConfigureTopics` that allows the user to disable the automatic creation of topics by the binder
Additional tests for partitioning
Adding missing documentation for `replicationFactor`
Fixes#495
Currently, `minPartitionCount` is available both as a binder default and as a consumer property.
It would be simpler if it only was a binder setting (seeing as it has effect only as a default).
Fixes#498
- Honour the retry settings from ConsumerProperties;
- Add an additional `enableDlq` option in KafkaConsumerProperties that enables forwarding failed messages to a DLQ topic.
Fix#500
- Initialize a fixed-size ExecutorService of the size of the concurrency setting of the binder
- Clean up the ExecutorService on unbind
- Remove unused `spyOn` support
- Both the producer and consumer properties have `HeaderMode`
- Handle the case of embeddedHeaders and raw for both the Sending/ReceivingHandlers in Redis binder
This resolves#408
Move message values extraction to superclass
This resolves#425
Address review comments
- Update AssertTrue conditional logic
- Modify the message
Changes after rebase
- Changes after extendedProperties addition
* Add parameterized ExtendedConsumerProperties and ExtendedProducerProperties;
* Added ExtendedPropertiesBinder with the ability of managing per-binding property extensions and interacting the core service;
* Add extensions for Kafka and Rabbit
Addressing PR comments
Changed binder-specific binding prefix to `spring.cloud.stream.binderType.bindings`
Removing kafka-binder.properties
- Change the Binder interface to support ConsumerProperties/ProducerProperties beans and subclasses
- Binders can subclass the property beans to add new supported properties that will be automatically populated
- Spring Cloud Stream will infer the target type and populate the beans from the environment based on a `spring.cloud.stream.bindings..<bindingName>` prefix
- Remove binder defaults and retain only general binder configurations
TODO: a) decide on instanceIndex/partitionIndex alignment (we do not need both)
b) support `defaultProducer`/`defaultConsumer` properties
c) add leniency control on binding (fail/ignore for unknown properties)
d) add a `requiredProperties` configuration for consumer/producer properties to finely tune the mandatory properties expected to be supported by a bound application
Changes made during review:
- Add support for consumer and producer defaults
- Remove partitionIndex, keeping only instanceIndex
- Fix default properties for Kafka binder
- Move batching properties to Rabbit only
- When deserialising the message, replace `contentType` with the `originalContentType` only if the `originalContentType` isn't null which means the original content type was set by the producer before serialising.
- Add and fix tests
This resolves#404
- When deserializing the payload at the consumer endpoint, the non-byte stream payload type requires to use `String` object when the underlying message content-type is of any `text` type contentType (text/plain, text/xml and text/html).
- This fix is only needed to support any non-SCSt applications that will have the 'text/*` contentType of the message that is being deserialized
- Add test
This resolves#403
