Commit Graph

340 Commits

Author SHA1 Message Date
Oleg Zhurakousky
8dbed6b877 Going back to snapshots 2018-10-30 15:01:36 +01:00
Oleg Zhurakousky
46ee483023 Update SNAPSHOT to 2.1.0.RC1 2018-10-30 14:59:54 +01:00
Oleg Zhurakousky
3037b90fd2 Updated spring-kafka version 2018-10-30 14:51:55 +01:00
Alberto Manzaneque
cb2a074448 Fixing KafkaBinderMetric so that it reports correct lag of all topics, not just some
Resolves #478
2018-10-30 12:44:48 +01:00
Oleg Zhurakousky
d73ef9f243 Bumping versions 2018-10-30 09:28:07 +01:00
Oleg Zhurakousky
ef2b2db961 Merge pull request #480 from olegz/docs-update
Updated docs to be compliant with spring-cloud
2018-10-30 08:34:24 +01:00
Oleg Zhurakousky
f6c058f9b8 Updated docs to be compliant with spring-cloud
polishing
2018-10-29 13:18:09 +01:00
Soby Chacko
17c11fbb66 Docs update for generic binder/binding properties
Resolves #417
2018-10-22 17:23:01 -04:00
Oleg Zhurakousky
8ab289a715 polishing
Resolves #472

polishing
2018-10-22 11:40:15 +02:00
Soby Chacko
4c15cbc1d6 Support headers in Kafka Streams binder
* Use content type header from the record for binder provided inbound deserialization by
  making use of the header support added in Kafka Streams. If there is a content type set
  on the incoming record, that will get precedence.
* Introduce a new Composite Serde class for providing non-native Spring Cloud Stream specific collection
  of Serde implemenations. This is needed in order for things like avro converters that interact with
  the Spring Cloud Stream schema registry server.
* Adding tests
* Polishing

Resolves #456, #469
2018-10-22 09:57:05 +02:00
Soby Chacko
1b80cfcf4c Avoid unnecessary re-partitioning due to map calls.
Fixing streams get unnecessarily flagged for re-partitioning
from map calls on KStream.

Resolves #412
2018-10-20 18:23:32 -04:00
Soby Chacko
0d5d092f84 Fix merge issues in 40c1a8b8 2018-10-19 15:43:03 -04:00
Soby Chacko
99d739be21 Consumer concurrency settings in Kafka Streams binder (#475)
* Consumer concurrency settings in Kafka Streams binder

* If the consumer concurency settings are provided at the binding level,
  honor that before falling back to the defaults.
* Allow consumer binding specific broker configurations to be set from the application.
* Test changes.

Resolves #474

* Addressing PR review comments
2018-10-19 15:16:10 -04:00
Alberto Manzaneque
40c1a8b8f1 Setting group in TopicInformation correctly for anonymous consumers.
This makes metrics available for anonymous consumers as well.

A couple of tests to verify that TopicInformation is built correctly

Removing unneeded import
2018-10-19 14:52:09 -04:00
Soby Chacko
c43e45c7ad Addressing startOffset in Kafka Streams binder (#473)
* Addressing startOffset in Kafka Streams binder

Fixing the issue where auto.offset.reset is not honored through startOffset
provided by the individual consumer binding.

Adding integration test to override auto.offset.reset globally at the binder
level to latest and then set startOffset on individual binding to earliest.

Resolves #467

* Addressing PR review comments

* Addressing PR review comments
2018-10-19 13:29:15 -04:00
Oleg Zhurakousky
700c7a1af3 GH-470 Updated extended properties merge logic
Updated extended properties merge logic based on corresponding core updates (see GH-1503)
Resolves #470

polishing
2018-10-15 15:59:45 -04:00
Soby Chacko
9ec0cdd245 Clarify data conversion in Kafka Streams docs
Remove the usage of Messages in kafka streams binder docs.

Resolves #464
2018-10-11 15:03:33 -04:00
Soby Chacko
0227c9ecc9 Regex topics in health indicator
If the destination topic is pattern based, only do the basic up/down check
in the health indicator. In this case, the health indicator does not do any
partitions queries as it does for normal topics.

Resolves #430
2018-10-11 13:05:50 -04:00
Soby Chacko
a5cec9a25c Kafka Health Indicator changes
Disable Kafka health indicator check if management.health.binders.enabled
property is set to false. Currently, if this property is disabled, only the
core spring-cloud-stream mechanism of collecting the health checks are disabled.
The indivividual binders can still register the health check bean and boot health
actuator will still pick it up. If the user turns off health check by disabling
this property, then the stream app should completely disable any binder specific
health check.

Resolves #454
2018-10-11 09:39:31 -04:00
Soby Chacko
9443c621b0 Kafka binder JAAS config refactoring (#461)
* Kafka binder JAAS config refactoring

Refactor JaasLoginModuleConfiguraton to leverage ControlFlag enum from
spring-kafka, instead of directly using LoginModuleControlFlag from JDK.

Re-instate Jass configuration tests.

Resolves #459

* Fixing tests

* Addressing PR review comments
2018-10-10 14:53:13 -04:00
Soby Chacko
fe7bdd245d Default properties testing.
Remove camelCase in property names
2018-10-08 12:29:20 -04:00
Gary Russell
1d5fcca522 GH-455: Docs for tombstone records
Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/455
2018-10-04 16:23:11 -04:00
Soby Chacko
e75967351f Address deprecations in Kafka Streams binder
* Remove the use of deprecated constructors in StreamsBuilderFactoryBean.
* Remove direct usage of StreamsConfig in favor of KafkaStreamsConfiguration.
* Change the way DLQ sending objects are provided to StreamsConfig since
  the binder does not directly deal with StreamsConfig any longer.
* Refactoring and polishing.

Resolves #442
Resolves #457
2018-10-02 19:44:14 -04:00
Oleg Zhurakousky
d44f2348e6 Polishing, renamed method
Resolves #452
2018-10-02 14:43:07 -04:00
Vladimir Porshkevich
3b4bff959f Change KafkaBinderMetrics to a Gauge instead of TimeGauge
Kafka Offset just numerical value not Time

Resolves #446
2018-10-02 14:22:51 -04:00
Soby Chacko
acb5b47a4f Update spring-kafka version to 2.2.0.RC1 2018-10-02 09:53:51 -04:00
Oleg Zhurakousky
ea810bb9e1 Removed check-style dependencies from main pom 2018-10-02 09:30:08 -04:00
Soby Chacko
e2fc45c4ce Next update version: 2.1.0.BUILD-SNAPSHOT 2018-09-21 11:37:51 -04:00
Soby Chacko
74a044b0c0 2.1.0.M3 Release 2018-09-21 11:00:17 -04:00
Soby Chacko
70f385553a Application ID resolution for Kafka streams binder (#450)
* Application ID resolution for kafka streams binder

Avoid the need to rely on group property for application id.
First, check binding specific application id, if not look at defaults.
If nothing works, fall back to the default application id set by the
boot auto configuration.

Modify tests.
Update docs.

Resolves #448

* Addressing PR review comments

* Minor polishing

* Addressing PR review comments
2018-09-21 10:17:34 -04:00
Soby Chacko
f5739a9c7f Missing beans registration in Kafka Streams binder
There are duplicate code in various binder configurations where we register missing beans.
Consolidate them into a common class that implements ImportBeanDefinitionRegistrar and then
import this class in the binder configurations.

Resolves #445
2018-09-20 11:28:32 -04:00
Soby Chacko
39f5490488 Refactor bootstrap server configuration
There is a common piece of code repeated in both producer and consumer
configuration where it is populating the bootstrap server configuration.
Refactoring into a common method.

Resolves #208
2018-09-19 16:15:19 -04:00
Soby Chacko
d445a2bff9 Docs for GlobalKTable binding
Resolves #443
2018-09-19 15:05:48 -04:00
Oleg Zhurakousky
5446485cbf Updating with changes related to core GH-1484
Resolves #447
2018-09-19 10:45:04 +02:00
Soby Chacko
cc61d916b8 Extended default properties
Allow applications to configure default values for extended prducer and
consumer properties across multiple bindings in order to avoid repetition.

Address the changes in both Kafka and Kafka Streams binders.

Add integration test to verify both binding specific and default extended properties.

Resolves #444
Requires https://github.com/spring-cloud/spring-cloud-stream/pull/1477
2018-09-17 20:09:44 -04:00
Soby Chacko
41d9136812 GH-384: Binding support for GlobalKTable
Resolves spring-cloud/spring-cloud-stream-binder-kafka#384

* New binding targets and binder implementation for GlobalKTable within kafka-streams binder
* Refactoring existing structure to accommodate the new binder
* Adding integration test to verify the GlobalKTable behavior

Resolves #384

* Addressing PR review comments

* Update spring-kafka to 2.2.0.M3
Addressing PR review comments
Polishing

* Addressing PR review comments

* Addressing PR review comments
2018-09-14 10:51:15 -04:00
Soby Chacko
71a6a6cf28 Package structure changes for StreamBuilderFactoryBean 2018-09-12 12:05:52 -04:00
Gary Russell
36361d19bf GH-1459: Pollable Consumer and Requeue
Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/1459

Requires https://github.com/spring-cloud/spring-cloud-stream/pull/1467
2018-09-10 10:22:04 -04:00
Soby Chacko
e869516e3d Handling tombstones in kafka streams binder
Handle tombstones gracefully in the kafka streams binder
Modify tests to verify

Resolves spring-cloud/spring-cloud-stream-binder-kafka#294

* Addressing PR review comments

* Addressing PR review comments
2018-09-05 12:17:54 -04:00
Gary Russell
ea288c62eb GH-435: useNativeEncoding and Transactions
Fixes https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/435

All common properties should be settable on the transactional producer.
2018-09-04 15:56:26 -04:00
Soby Chacko
afa337f718 Remove deprecations in tests
Resolves #433
2018-09-04 15:28:36 -04:00
Soby Chacko
e4b08e888e Next update version: 2.1.0.BUILD-SNAPSHOT 2018-08-28 10:15:08 -04:00
Soby Chacko
de4b5a9443 2.1.0.M2 Release 2018-08-28 10:04:25 -04:00
Soby Chacko
a090614709 kafka Streams binder configuration fix
(to address core changes made in regards to Boot 2.1)

Ensure that KafkaStreamsBinderSupportAutoConfiguration runs
after BindingServiceConfiguration from core.
2018-08-14 10:33:51 -04:00
Gary Russell
a3f7ca9756 Fix mock to use poll(Duration) 2018-08-07 14:45:33 -04:00
Soby Chacko
82b07a0120 Fixing checkstyle issues 2018-08-07 14:23:47 -04:00
Soby Chacko
0d0cf8dcb7 Update to spring-cloud-build 2.1.0 snapshot
Spring Boot 2.1.0 snapshot
Spring Kafka 2.2.0/3.1.0 snapshots
Apache Kafka client 2.0.0
Fixing tests
Removing deprecations and removals
Polishing

Resolves #424
2018-08-07 14:02:52 -04:00
Soby Chacko
cbfe03be2f Kafka Streams binder test disabling
Temporarily disabling KafkaBinderBootstrapTest in Kafka Streams binder
due to builds taking much longer times due to this.
2018-08-03 08:11:52 -04:00
Soby Chacko
1f0c6cabc6 Revert "Kafka Streams binder test disabling"
This reverts commit 0c61cedc85.
2018-08-03 08:10:59 -04:00
Soby Chacko
0c61cedc85 Kafka Streams binder test disabling
Temporarily disabling KafkaBinderBootstrapTest in Kafka Streams binder
due to builds taking much longer times due to this.
2018-08-02 12:59:02 -04:00