Commit Graph

90 Commits

Author SHA1 Message Date
Oleg Zhurakousky
c28873c870 Revert "GH-2510: Rabbit Binder Scale-out on Super Stream"
This reverts commit 16c8591ddd.
2022-10-05 21:31:01 +02:00
Oleg Zhurakousky
2e6eadefa3 Set more 'proxyBeanMethods' to false 2022-10-05 13:27:07 +02:00
Oleg Zhurakousky
3317d38779 Set 'proxyBeanMethods' to false 2022-10-05 09:30:04 +02:00
Ginkgo
b01c804008 Fix KafkaBinderHealthIndicator return Down Error
When we set enable.auto.commit to true, the actuator/health returns DOWN status.

For more details see this: https://github.com/spring-cloud/spring-cloud-stream/pull/2504#issue-1361385289

Update KafkaBinderConfigurationProperties.java

Polish commit
2022-10-03 19:31:31 -04:00
Soby Chacko
b85ee57d80 Kafka Streams Binder Default Package Beans
Currently, in Kafka Streams binder-based apps, processor beans
need to be declared public. This is unnecessary and caused by
some restrictions in the binder. This PR fixes this restriction.

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2516
2022-09-29 17:51:38 -04:00
Gary Russell
16c8591ddd GH-2510: Rabbit Binder Scale-out on Super Stream
Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2510

New feature on RabbitMQ (Super Stream with Single Active Consumer) enables
scaling out app instances when using this queue type/config.

`RabbitStreamMessageHandler` is now available in Spring Integration.
2022-09-27 12:51:42 -04:00
Oleg Zhurakousky
60ddac5560 Polish Kinesis binder following previous commit 2022-09-27 17:03:15 +02:00
Oleg Zhurakousky
5035a40990 GH-2512, 2507 Establish binder identity & change error channel binding scheme
This commit establishes the concept of binder identity (binder instance identity) to be used in cases where unique-to-the-binder-instance naming is required.
For example such naming is required to fix GH-2507 where error channel names could colide if two binders use the same destination name.

Resolves #2512
Resolves #2507
2022-09-27 16:02:24 +02:00
Oleg Zhurakousky
65c6274a11 GH-2511 Fix Rabbit environment tests/binder
Resolves #2511
2022-09-26 11:39:45 +02:00
spencergibb
63b96ba135 Updates testcontainers to 1.17.3 2022-09-22 12:53:48 -04:00
Oleg Zhurakousky
00d0c6bacf Add new spring-boot .imports files and move auto-configuration classes there 2022-09-22 17:53:47 +02:00
Soby Chacko
af022b2ee0 Cleanup Kafka Client Deps
Let the Kafka client versions be managed from Spring Boot
rather than the binder manages it's own versions.
2022-09-15 15:17:32 -04:00
Oleg Zhurakousky
fbdf2dc40b Fix Rabbit tests 2022-09-12 15:27:52 +02:00
Nico Heller
de0efcf0a1 GH-2485: Improvements in Kafka Binder Metrics
Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2485

These improvements are threading related

Initial implementation

Readd missing 'cache' for recent offset lags and general refactoring

Add a test case for the schedule-only variant and externalize scheduling interval

Consistency of configuration property naming and minor cleanup of existing code

Review feedback: documentation, author addition and copyright adjustments

Doc wording changes

Fix checkstyle issues

Adjust reference doc and improve new property names according to review

Move documentation to the correct file
2022-09-02 10:21:11 -04:00
Gary Russell
4fda96507c GH-2453: RMQ: Full Support for Alternate Exchange
Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2453

Previously, to configure the use of an alternative exchange (used to route
messages when no queue is bound), the user had to manually declare the
exchange and any bindings, and modify the destination exchange using a
`DeclarablesCustomizer` bean.

Add first class support to configure the destination exchange and, optionally,
provision the alternate exchange as well as optionally binding a specific
queue to it.

Resolves #2502
2022-09-01 16:44:15 +02:00
Soby Chacko
92ea029a8b AdminClient usage if autoCreateTopics is disabled (#2494)
* AdminClient usage if autoCreateTopics is disabled

Don't Create an AdminClient if autoCreateTopics is False

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2486

* Fix copyright year
2022-08-29 09:11:14 -04:00
Gary Russell
0daa58c48a GH-2483: Fix Tests 2022-08-18 10:19:19 -04:00
Gary Russell
f7146ad02e GH-2483: RMQ: Fix Redeclare Multi Routing Keys
Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2483

https://github.com/spring-cloud/spring-cloud-stream-binder-rabbit/issues/242
added support for binding a queue with multiple routing keys. However, only
one binding was added to the `autoRedeclarContext`.

Use the `Declarables` wrapper to support multiple instances with the same
name.

**cherry-pick to 3.2.x**
Resolves #2484
2022-08-18 15:47:17 +02:00
Chris Bono
f94da9d311 Update Spring Boot from 3.0.0-M4 -> 3.0.0-SNAPSHOT
Adapts to the following changes in upstreams libs:

- Spring Kafka removed ListenableFuture
- Spring AOT changed generator API

Fixes #2473 #2465

Checkstyle fixes
2022-08-09 10:32:25 -04:00
buildmaster
605344d5a5 Going back to snapshots 2022-07-29 19:05:12 +00:00
buildmaster
c69888ae68 Update SNAPSHOT to 4.0.0-M4 2022-07-29 19:02:38 +00:00
Oleg Zhurakousky
88e5f88270 GH-2455 Fix the AOT Binding initialization logic
This commit also includes partial work to ensure framework works in full native/AOT mode.

Also, PollableBean remains and will simply not work in AOT mode until https://github.com/spring-projects/spring-framework/issues/28748 is resolved. That said, i will be deprecating t and it will be removed in the next release given that we already have configurable alternative

Resolves #2456
2022-07-22 13:53:51 +02:00
Soby Chacko
ada56c4e0f Fixing Kafka binder build issues
Update maven surefire-plugin to 3.0.0-M7 to address an issue
with how the newly added GlobalEmbeddedKafkaTestExecutionListener
in spring-kafka-test loads configuration parameters. The current
versions (2.x) of the plugin are not properly loading these parameters.

https://maven.apache.org/surefire/download.cgi
2022-07-14 12:11:54 -04:00
Soby Chacko
89eb9a31d4 Code cleanup - Kafka Streams binder 2022-07-11 14:33:18 -04:00
Chris Bono
0cb59496fe Make InteractiveQueryService only return used state stores
This restores the KafkaStreams 3.0 behavior in this area.

Fixes #2445
2022-06-25 22:09:19 -05:00
Pommerening, Nico
7554ff9bbd Kafka Streams binder health indicator improvements
Fix KafkaStreamsBinderHealthIndicator overriding HealthCheck Thread Details to report full details.

checkstyle fixes.
2022-06-23 14:20:04 -04:00
Chris Bono
06ffe76c01 Improve KafkaStreams ineligible-definitions property handling
* KafkaStreamsBinderEnvironmentPostProcessor respects existing ineligible-definitions
* Add tests for KafkaStreamsBinderEnvironmentPostProcessor
2022-06-17 13:42:25 -04:00
Chris Bono
6d1be361fe Simplify SerdeResolverUtilsTests setting of ineligible-definitions 2022-06-17 13:40:19 -04:00
yiliu6
acc75183e2 Enable multiple customizers for binder and provisioner.
Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2423
2022-06-17 12:43:41 -04:00
Soby Chacko
21171914ad SendToDlqAndContinue needs to be excluded
In Kafka Streams binder, we need to exclude the SendToDlqAndContinue
BiFunction by adding it to the ineligible function definitions.

See this commit for more context: 8ec15fa3ca
2022-06-16 18:12:11 -04:00
buildmaster
97975a0d3c Going back to snapshots 2022-06-15 11:35:09 +00:00
buildmaster
84d4cccdee Update SNAPSHOT to 4.0.0-M3 2022-06-15 11:32:24 +00:00
Oleg Zhurakousky
225d96c4fb Temporarily disable failing Kinesis test 2022-06-15 13:14:24 +02:00
Oleg Zhurakousky
da3157c6d1 Remove dependency on test-support-internal from kinesis binder 2022-06-15 12:52:19 +02:00
Oleg Zhurakousky
c4f516e72f Change from SCSt version to project.version in Kinesis binder 2022-06-15 12:26:41 +02:00
Oleg Zhurakousky
2b914cd0f7 Update SI Kinesis to 3.0.0-M1 2022-06-15 11:39:49 +02:00
Soby Chacko
7b9c66c4ac Adjust KafkaBinder DLQ test error messages 2022-06-14 10:49:25 -04:00
Soby Chacko
806a6500b5 Kafka Streams binder message conversion changes
Fixing boolean logic around non-native message conversion in Kafka Streams binder.
2022-06-07 17:50:36 -04:00
Soby Chacko
6dd6774280 Producer config customization and DLQ issues
When DLQ is enabled on a consumer binding, the internal producer properties
used by the DLQ mechanism to send to the topic is not populated with the
correct binding name. In the previous version, it was working fine since
we were relying on a ThreadLocal. This is revamped recently to properly
introduce a binding name as a top level property in ProducerProperties.
However, this was not set on the internal producer properties used by the
DLQ. This PR addresses this issue.

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2417
Resolves #2418
2022-06-07 14:29:46 +02:00
Soby Chacko
ce6a03ee59 Kafk Streams binder message conversion issues
When native decoding is disabled and message conversion
is used  in Kafka Streams binder, it doesn't currently
carry the original headers forward. Fixing this issue.

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2411
2022-06-06 19:19:14 -04:00
ferblaca
ae0332b7d4 add the topic name information to the binder exception if a problem occurs in the provisioning phase 2022-06-06 10:46:03 -04:00
Tomek Szmytka
0bf50cbf85 Clean up resources on close. Allow cleanly terminating application process on context shutdown
Prevent IDE from using star imports

Update authors

Checkstyle fixes
2022-06-01 17:12:48 -04:00
amseager
90e5d5ee8d add missing backoff multiplier 2022-05-31 15:24:19 -04:00
Soby Chacko
605a1b1eda Update kafka client versions
3.2.0
2022-05-31 15:12:03 -04:00
Chris Bono
9f8dc3eded Fix compile error in LocalstackContainerTest
* Improve docker reuse in `LocalstackContainerTest`
2022-05-24 12:05:17 -04:00
Gary Russell
fd3ef3e849 GH-2297: Polish Concurrency Test 2022-05-16 09:23:10 -04:00
Chris Bono
cc05160cc0 Cleanup SerdeResolverUtils
* Refactor SerdeResolverUtils logic
* Add more tests in SerdeResolverUtilsTests
* Fixed some random test warnings

Fixes #2370

Code review feedback: remove FQ ref to inner method

Fix checkstyles
2022-05-13 15:20:47 -04:00
Gary Russell
bf810fbd53 GH-2297: Concurrency in Reactor Kafka Binder (#2396)
* GH-2297: Concurrency in Reactor Kafka Binder

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2297

* Remove unnecessary local var.

* Remove subscription cancellation; already handled by the super class.
2022-05-12 16:00:44 -04:00
Gary Russell
e522450f8e GH-2298: Refactor Common Binder Code to Core
Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2298

Polishing per review comments; javadocs.
2022-05-12 12:14:47 -04:00
Soby Chacko
dae959999a Binding name as a consumer/producer property
Make binding names available through Consumer/Producer properties.
Currently, the binders use a ThreadLocal to store the binding name for internal use.
These changes introduce the binding name as a property for both producer/consumer bindings.

Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/2380
2022-05-10 17:22:51 -04:00