Commit Graph

139 Commits

Author SHA1 Message Date
buildmaster
097fb89d9e Update SNAPSHOT to 3.1.0-M4 2020-11-17 16:03:30 +00:00
gustavomonarin
676764b923 Fix documentation for kafka streams concurrency on binder level
The property spring.cloud.stream.binder.configuration.num.stream.threads does not work and is silently ignored.

The property spring.cloud.stream.kafka.streams.binder.configuration.num.stream.threads works fine and  is already covered by tests at MultipleFunctionsInSameAppTests#125

resolves #987
2020-11-11 13:55:35 -05:00
Soby Chacko
f1e3a0bdd6 Allow retries in Kafka Streams binder (#980)
* Allow retries in Kafka Streams binder

Provide applications the capability to retry critical sections of the
business logic. This is accomplished through a new API using which
critical path can be wrapped inside a Callable.

Adding tests and docs.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/945

* Reworking the PR

Remove the binder API that was added before for retrying.
Reuse binding provided RetryTemplate.

Tests and docs

* Cleanup

* Addressing PR review comments

* Fix typo
2020-10-30 16:39:49 -04:00
Soby Chacko
97e3b61d14 Adding improvements to InteractiveQueryService
New API additions.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/942
2020-10-28 18:58:18 -04:00
Soby Chacko
45f1927c6f Expand messageKeyExpression docs (#978)
* Expand messageKeyExpression docs

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/972

* Update docs/src/main/asciidoc/overview.adoc

Co-authored-by: Gary Russell <grussell@vmware.com>

Co-authored-by: Gary Russell <grussell@vmware.com>
2020-10-27 11:10:33 -04:00
Soby Chacko
0a0d3a1057 Remove duplicate KafkaStreams topology endpoint
Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/895
2020-10-22 12:31:40 -04:00
Soby Chacko
5cdd8a09f9 Custom DLQ Destination Resolver (#976)
* Custom DLQ Destination Resolver

Allow applications to provide a custom DLQ destination resolver
implementaiton by providing a new interface DlqDestinationResolver
as part of binder's public contract. This interface is a BiFunction
extension using which the applications can provide more fine grained
control over where to route records in error.

Adding test to verify.

Adding docs.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/966

* Add DlqDestinationResolver to MessageChannel based binder.

Tests and docs
2020-10-21 12:02:02 -04:00
buildmaster
1358bdfeec Going back to snapshots 2020-10-02 14:10:39 +00:00
buildmaster
f8dac888e4 Update SNAPSHOT to 3.1.0-M3 2020-10-02 14:09:20 +00:00
buildmaster
7bebe9b78f Bumping versions 2020-09-25 10:42:44 +00:00
Oleg Zhurakousky
c44c17008c Fix docs links 2020-09-24 16:22:55 +02:00
Soby Chacko
33fa713a9f Customizing producer/consumer factories (#963)
* Customizing producer/consumer factories

Adding hooks by providing Producer and Consumer config
customizers to perform advanced configuration on the producer
and consumer factories.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/960

* Addressing PR review comments

* Further PR updates
2020-09-23 13:21:31 -04:00
Soby Chacko
769ed56049 Deprecate autoCommitOffset in favor of ackMode (#957)
* Deprecate autoCommitOffset in favor or ackMode

Deprecate autoCommitOffset in favor of using a newly introduced consumer property ackMode.
If the consumer is not in batch mode and if ackEachRecord is enabled, then container
will use RECORD ackMode. Otherwise, use the provided ackMode using this property.
If none of these are true, then it will defer to the default setting of BATCH ackMode
set by the container.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/877

* Address PR review comments

* Addressing PR review comments
2020-09-21 11:16:51 -04:00
cleverchuk
1e9aa60c4e Fix Health indicator/partition leader issues
* Fix health indicator to properly indicate partition failure
 * Add new flag to control binder health indicator behavior
 * Regardless of the consumer that is reading from a partition, if the binder
   detects that a partition for the topic is without a leader, mark the binder
   health as DOWN (if the flag is set to true).
 * Remove synchronize block since only one thread executes the block
 * Add Docs for the new binder flag
 * Fix checkstyle issues
2020-09-11 12:06:28 -04:00
Soby Chacko
4161f875ed Change default replication factor to -1 (#956)
* Change default replication factor to -1

Binder now uses a default value of -1 for replication factor signaling the
broker to use defaults. Users who are on Kafka brokers older than 2.4,
need to set this to the previous default value of 1 used in the binder.

In either case, if there is an admin policy that requires replication factor > 1,
then that value must be used instead.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/808

* Addressing PR review comments
2020-08-28 12:18:53 -04:00
Aldo Sinanaj
5b3b92cdb9 GH-932: Added zstd compression type 2020-08-13 15:57:44 -04:00
buildmaster
ec52fbe2eb Going back to snapshots 2020-07-20 16:28:42 +00:00
buildmaster
5e128aafdc Update SNAPSHOT to 3.1.0-M2 2020-07-20 16:27:22 +00:00
Soby Chacko
4159217023 Kafka Streams metrics changes
* Kafka Streams metrics in the binder for Boot 2.2 users are streamlined to
  reflect the Micrometer native support added with version 1.4.0 which is
  available through Boot 2.3. While Boot 2.3 users will get this native support
  from  Micrometer, Boot 2.2 users will still rely on the custom implementation
  in the binder. This commit aligns that custom implemenation more with
  the native implementation.

* Disable the custom Kafka Streams metrics bean which is mentioned above
  (KafkaStreamsBinderMetrics) when the application is on Boot 2.3, as this
  implementation is only applicable for Boot 2.2.x.

* Update docs

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/880
2020-07-17 20:28:47 -04:00
Jay Bryant
087b09d193 Wording changes (#934)
* Wording changes

Replacing some terms

* Wording changes

One more change (missed a gerund form).
2020-07-14 11:34:54 -04:00
Soby Chacko
a549810899 Renaming Kafka Streams topology actuator endpoint
Kafka Streams topology actuator endpoint had a conflict with the JMX exporter
and was causing some IDE issues. Renming this endpoint to kafkastreamstopology.
Renaming the underlying methods in this actuator endpoint implentation.

Updating docs.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/895
2020-06-15 18:56:18 -04:00
Marcin Grzejszczak
38bea7e7da Added missing JAR packaging 2020-06-08 18:42:49 +02:00
Marcin Grzejszczak
201feca206 Migrated to new documentation generation 2020-06-08 18:41:25 +02:00
Soby Chacko
21a8f89f22 Update version to 3.1.0-SNAPSHOT 2020-06-02 13:58:54 -04:00
Soby Chacko
473ca53723 Configuring Kafka producer timeout
Allow setting timeout for closing the producer.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/891
Resolves #909
2020-06-02 12:54:42 -04:00
Soby Chacko
dbc00ffa92 Docs changes for consumer batch.mode
See https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/892\#issuecomment-631862705
2020-06-02 12:38:44 -04:00
Soby Chacko
fa0d6fe60e GH-899: num.stream.threads not taking effect
Binder is overriding the num.stream.threads property specified
through the binder configuration. Fixing this issue.

Adding tests to verify.

Doc changes

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/899
2020-06-02 12:34:50 -04:00
Gary Russell
1cfb63f4bf Reject improper settings for bootstrap.servers
Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/875

- the `bootstrap.servers` property cannot be overridden at the binding level.
2020-04-30 17:08:03 -04:00
buildmaster
36f00e5867 Going back to snapshots 2020-04-07 17:19:16 +00:00
buildmaster
e7487b9ada Update SNAPSHOT to 3.1.0.M1 2020-04-07 17:18:40 +00:00
Gary Russell
445eabc59a Transactional Producer Doc Polishing
- explain txId requirements for producer-initiated transactions
2020-04-07 11:16:14 -04:00
Soby Chacko
cc5d1b1aa6 GH:851 Kafka Streams topology visualization
Add Boot actuator endpoints for topology visualization.

Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/851
2020-03-25 13:02:57 +01:00
Gary Russell
07a740a5b5 GH-861: Add transaction manager bean override
Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/861

Add docs
2020-03-19 17:03:36 -04:00
Soby Chacko
90a47a675f Update Kafka Streams docs
Fix missing property prefix in StreamListener based applicationId settings.
2020-03-05 11:40:10 -05:00
Oleg Zhurakousky
e46bd1f844 Update POMs for Ivyland 2020-02-14 11:28:28 +01:00
buildmaster
5a8aad502f Bumping versions to 3.0.3.BUILD-SNAPSHOT after release 2020-02-13 08:31:42 +00:00
buildmaster
b84a0fdfc6 Going back to snapshots 2020-02-13 08:31:41 +00:00
buildmaster
e3fcddeab6 Update SNAPSHOT to 3.0.2.RELEASE 2020-02-13 08:30:55 +00:00
buildmaster
349fc85b4b Bumping versions to 3.0.2.BUILD-SNAPSHOT after release 2019-12-18 18:12:34 +00:00
buildmaster
a4762ad28b Going back to snapshots 2019-12-18 18:12:34 +00:00
buildmaster
bdd4b3e28b Update SNAPSHOT to 3.0.1.RELEASE 2019-12-18 18:11:50 +00:00
Soby Chacko
c7c05fe3c2 Docs for topic patterns in Kafka Streams binder 2019-12-17 18:43:45 -05:00
문찬용
4b50f7c2f5 Fix typo 2019-12-17 18:41:13 -05:00
Soby Chacko
70c1ce8970 Update image for spring.io kafka streams blog 2019-12-02 11:22:39 -05:00
Soby Chacko
1d8a4e67d2 Update image for spring.io kafka streams blog 2019-12-02 11:16:29 -05:00
buildmaster
bbfc776395 Bumping versions to 3.0.1.BUILD-SNAPSHOT after release 2019-11-22 14:51:01 +00:00
buildmaster
4e9ed30948 Going back to snapshots 2019-11-22 14:51:01 +00:00
buildmaster
34fd7a6a7a Update SNAPSHOT to 3.0.0.RELEASE 2019-11-22 14:49:08 +00:00
Soby Chacko
cf59cfcf12 Kafka Streams docs cleanup 2019-11-20 18:25:12 -05:00
Soby Chacko
637ec2e55d Kafka Streams binder docs improvements
Adding docs for production exception handlers.
Updating configuration options section.
2019-11-13 15:19:41 -05:00