Commit Graph

588 Commits

Author SHA1 Message Date
Soby Chacko
b4a2950acd KafkaStreamsStateStore with multiple input bindings
When KafkaStreamsStateStore annotation is used on a method with multiple input bindings,
it throws an exception. The reason is that each successive input binding after the first one
is trying to recreate the store that is already created. Fixing this issue.

Resolves #551
2019-02-28 16:31:38 -05:00
Gary Russell
c5c81f8148 SGH-1616: Add MessageSourceCustomizer
Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/1616
2019-02-28 13:44:14 -05:00
Arnaud Jardiné
d80e66d9b8 Actuator health for Kafka-streams binder
Add documentation about health indicator

Fix failing tests + add tests for multiple Kafka streams

Polishing

Resolves #544
2019-02-14 16:26:56 -05:00
Soby Chacko
86c9704ef4 Fix metrics with multi binders
Kafka binder metrics is broken with a multi-binder configuraiton.
Fixing the issues by propagating the MeterRegistry bean into the
binder context from parent.

Adding test to verify.

Removing the formatter plugin from the parent pom.

Resolves #546
Resolves #549
2019-02-14 09:25:11 +01:00
Gary Russell
63a4acda1b Fix test for SK 2.2.4
Override deserializer getters in test consumer factory because the
defaults incorrectly throw an `UnsupportedOperationException`.

See https://github.com/spring-projects/spring-kafka/pull/963
2019-02-13 16:28:13 -05:00
Gary Russell
31a6f5d7e3 GH-531: Fix test
- resolve conflict with another test that used `EnableBinding(Sink.class)`.
2019-02-07 12:44:32 -05:00
Aldo Sinanaj
4d02c38f70 GH-531: Support tombstones on input/output
GH-531: Set test timeout to 10 seconds

GH-531: Polishing - Fix @StreamListener

Added test for `@StreamListener`.
2019-02-07 09:01:54 -05:00
Oleg Zhurakousky
c22c08e259 GH-1601 satellite changes to the core 2019-02-05 07:08:48 +01:00
Soby Chacko
02e1fec3b4 Checkstyle upgrade
Upgrade checkstyle plugin to use the rules from spring-cloud-build.
Fix all the new checkstyle errors from the new rules.

Resolves #540
2019-02-04 18:11:10 -05:00
Soby Chacko
fb91b2aff9 Temporarily disabling the checkstyle plugin 2019-02-04 12:27:29 -05:00
Oleg Zhurakousky
a881b9a54a Created 2.2.x branch 2019-02-04 10:46:24 +01:00
Aldo Sinanaj
fc92a6b0af GH-389: Topic props: Use .topic instead of .admin
Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/389

GH-389: Deprecated [producer/consumer].admin in favor of topic property

GH-389: Fixed comments as suggest by Gary

Polishing - 2 more deprecation warning suppressions
2019-01-29 15:16:38 -05:00
Gary Russell
d65e8ff59d GH-529: Add lz4 and docs for zstd compression
Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/529

- Also log exception when getting partition information
2019-01-16 12:49:12 -05:00
buildmaster
9948674f30 Bumping versions to 2.1.1.BUILD-SNAPSHOT after release 2019-01-08 11:53:04 +00:00
buildmaster
cfc1e0e212 Going back to snapshots 2019-01-08 11:53:04 +00:00
buildmaster
4bd206f37f Update SNAPSHOT to 2.1.0.RELEASE 2019-01-08 11:51:00 +00:00
Oleg Zhurakousky
9d1d90e608 Upgraded to spring-kafka 2.2.2 2019-01-07 16:25:23 +01:00
Soby Chacko
0b8a760b5a Fix NPE in Kafka Streams binder
Fixing an NPE when type for the binder is not explicitly provided as a configuration.

Resolves #516
Resolves #524
Polishing
2019-01-03 20:26:10 +01:00
Gary Russell
d63f9e5fa6 GH-521: Fix pollable source client id
Fixes https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/521

Previously, all pollable message sources got the client id `message.source`.
MBean registration failed with a warning when multiple pollable sources were present.

Use the binding name as the client id by default, overridable using the `client.id`
consumer property.

**cherry-pick to 2.0.x**

Resolves #523
2019-01-02 20:09:49 +01:00
buildmaster
c51d7e0613 Going back to snapshots 2018-12-20 19:44:59 +00:00
buildmaster
7093551a86 Update SNAPSHOT to 2.1.0.RC4 2018-12-20 19:44:23 +00:00
Oleg Zhurakousky
e9f24b00c8 Adjusting for the type conversion fixes in GH-1564 core 2018-12-18 09:36:28 +01:00
buildmaster
ed1589f957 Going back to snapshots 2018-12-13 20:26:31 +00:00
buildmaster
c5f72250b5 Update SNAPSHOT to 2.1.0.RC3 2018-12-13 20:25:50 +00:00
buildmaster
0dc871aff0 Bumping versions 2018-12-13 16:51:07 +00:00
Soby Chacko
67bcd0749a Polishing 2018-12-12 21:34:22 -05:00
Soby Chacko
6001136ec0 Handle errors for non-existing topics (#514)
* Handle errors for non-existing topics

When topic creation is disabled both on the binder and the broker,
the binder currently throws an NPE. Catching this situation and
throw a more graceful error to the user.

Adding tests to verify.

Resolves #513

* Addressing PR review comments
2018-12-12 12:44:42 -05:00
Arnaud
e93904c238 Add logging for deserialization failures. 2018-12-12 10:44:11 -05:00
Soby Chacko
43c8f02bff User header mapper with MimeTypeJsonDeserializer
Temporarily provide a custom HeaderMaper as part of the binder that is
copied from Spring Kafka so that we can preserve backward compatibility
with older producers. When older producers send non String types,
the header mapper in Spring Kafka treats that as MimeType. This change will
use a HeaderMppaer that reinstates the MimeTypeJsonDeserializer.
When we can consume the Spring Kafka version that provides the HeaderMapper
with this fix in it, we will remove this custom version.

Resolves #509
2018-11-30 10:57:41 -05:00
Soby Chacko
64ea989b08 Kafka Streams environment properties changes
When Kafka Streams binder is used in multi binder environments, the properties defined under environment
is not propagated to the auto configuration class. The environment processing only takes place when the
actual binder configuration is instantiated (for example, KStreamConfiguration), and therefore the environment
properties are unavailable during the earlier autoconfiguration. This change makes the environment properties
availble during auto configuration.

Resolves #504
2018-11-28 14:25:15 -05:00
Soby Chacko
525adef3a6 Fixing checkstyle warnings 2018-11-27 16:57:03 -05:00
Soby Chacko
1866feb75f Enable KafkaStreams bootstrap tests
Remove the usage of `ImportBeanDefinitionRegistrar` in Kafka Streams binder
components since the regular use of getBean from the outer context is safe to do so.

Unignore tests

Resolves #501

* Addressing PR review comments
2018-11-27 16:54:31 -05:00
Gary Russell
bf1c366d9b GH-502: Add test for native partitioning
Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/502

Before applying the fix for,

https://github.com/spring-cloud/spring-cloud-stream/issues/1531

failed with:

```
org.springframework.messaging.MessageDeliveryException: failed to send Message to channel 'test.output'; nested exception is java.lang.IllegalArgumentException: Partition key cannot be null, failedMessage=GenericMessage [payload=byte[3], headers={kafka_partitionId=5, id=3350a823-c876-f7a9-f98b-fdbd2aaa4c12, timestamp=1542823925354}]
...
Caused by: java.lang.IllegalArgumentException: Partition key cannot be null
	at org.springframework.util.Assert.notNull(Assert.java:198)
	at org.springframework.cloud.stream.binder.PartitionHandler.extractKey(PartitionHandler.java:112)
	at org.springframework.cloud.stream.binder.PartitionHandler.determinePartition(PartitionHandler.java:93)
	at org.springframework.cloud.stream.binding.MessageConverterConfigurer$PartitioningInterceptor.preSend(MessageConverterConfigurer.java:381)
	at org.springframework.integration.channel.AbstractMessageChannel$ChannelInterceptorList.preSend(AbstractMessageChannel.java:589)
	at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:435)
	... 31 more

```

Resolves #503
2018-11-23 11:44:19 +01:00
Oleg Zhurakousky
81f4a861c5 Fixed Kafka Stream binder docs 2018-11-19 16:30:35 +01:00
buildmaster
662aa71159 Going back to snapshots 2018-11-19 14:49:11 +00:00
buildmaster
8683604004 Update SNAPSHOT to 2.1.0.RC2 2018-11-19 14:48:35 +00:00
Soby Chacko
7232839cdf Ignore JAAS tests as they randomly fail on CI 2018-11-17 12:13:29 -05:00
Oleg Zhurakousky
47443f8b70 Fixed tests related to GH-1527 change in core 2018-11-15 19:33:20 +01:00
Soby Chacko
b589f32933 Checkstyle fixes
Reuse checkstyle components from core spring-cloud-stream-tools module.
Addressing checkstyle warnings in kafka streams binder module.

Resolves #489
2018-11-09 18:11:51 -05:00
Soby Chacko
20af89bc00 Fixing health indicator issues
During startup if Kafka is down, binder health indicator check
erroneously reports that Kafka is up. Fixing this issue.

Resolves #495
2018-11-09 09:22:26 -05:00
Soby Chacko
0b50a6ce2f Address checkstyle warnings
Addressing checkstyle warnings in sprnig-cloud-stream-binder-kafka.

Resolves #487
2018-11-08 10:42:26 -05:00
Gary Russell
3a396c6d37 GH-491: Fix TODO in binder for initial seeking
Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/491

Use `seekToBeginning/End` instead of `0` and `Long.MAX_VALUE`.
2018-11-07 16:39:33 -05:00
rahulbats
3e39514003 Add log message for deserialization error handler
Add missing logging statement for the logAndContinue
deserialization error handler in Kafka Streams binder.

Polishing.

Resolves #494
2018-11-07 16:29:25 -05:00
Gary Russell
2144d3e26f GH-453: Add binding rebalance listener
Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/453

Fix initial value of initial variable
2018-11-07 14:54:02 -05:00
Soby Chacko
2d4e9a113b Use TopicExistsException in provisioner
Instead of checking for the text TopicExistsException in the exception
message, use strong type check for TopicExistsException through instanceof on
the cause of the exception.

Adding test to verify.

Resolves #209
2018-11-05 16:20:47 -05:00
Soby Chacko
84249727ae Fix boolean expression in resetOffsets
Resolves #481
Resolves #485
2018-11-05 17:43:44 +01:00
Soby Chacko
2fc158003f Address checksyle issues in core module
Address checkstyle errors and warnings in spring-cloud-stream-binder-kafka-core.
Remove a duplicate dependency declaration from parent pom.

Resolves #483
Resolves #484
2018-11-05 17:41:39 +01:00
Soby Chacko
9c20bb4cdc Remove unnecessary auto config import.
Remove the superfluous import of PropertyPlaceholderAutoConfiguration
in KafkaBinderConfiguration.

Resovles #211
2018-11-02 15:02:32 -04:00
Oleg Zhurakousky
8dbed6b877 Going back to snapshots 2018-10-30 15:01:36 +01:00
Oleg Zhurakousky
46ee483023 Update SNAPSHOT to 2.1.0.RC1 2018-10-30 14:59:54 +01:00