Commit Graph

91 Commits

Author SHA1 Message Date
Mark Fisher
2bc59a184a AbstractBinder's afterPropertiesSet() is now final
- subclasses should implement the new onInit() method instead of overriding
- avoids the potential for a NPE in case an overriding subclass did not call the superclass method
2016-03-02 13:29:49 -05:00
Ilayaperumal Gopinathan
69e484163c Move spring-cloud-stream tuple
- The `spring-cloud-stream-tuple` is now split into `spring-tuple`, `spring-integration-tuple` along with `spring-batch-tuple`
  - Update dependencies to use the new ones

This resolves #326
2016-02-26 17:12:49 -05:00
Ilayaperumal Gopinathan
4dfd217db6 Maven configuration simplification
- Move kafka related dependencies from parent `dependencyManagement` to `kafka` binder project
 - Remove duplicate properties
 - Fix typo

This resolves #353
2016-02-25 12:44:27 -05:00
Patrick Peralta
d34aafb87b Partition handling refactoring
Moved partition calculation from AbstractBinder to
its own class PartitionHandler.

Updated per review feedback

Refactored bean creation
Added synchronization around bean creation
2016-02-24 16:42:51 -05:00
Ilayaperumal Gopinathan
047a3f8d4f Support sync producer option in Kafka binder
- Add a binding property `spring.cloud.stream.bindings.<outputChannelName>.syncProducer`
   When set to `true`, the Kafka producer metadata is set to use `sync` producer
 - Update DefaultPropertiesAccessor for Kafka to use the `syncProducer` property
 - Add test

This resolves #343

Move syncProducer as a binder level property

 - Add a property `syncProducer` in Kafka Binder configuration properties (prefix: spring.cloud.stream.binder.kafka)
  - Set Kafka ProducerMeatadata `sync` based on the value set in this property
 - Update test
2016-02-23 00:23:18 -05:00
Marius Bogoevici
2fa9cda89c Durability Configuration and Default Groups
Resolves #317

Remove the `durable` binder configuration property
Make subscriber groups durable by default
Introduce `requiredGroups` property
Kafka groups (non-anonymous) now start by default at EARLIEST, which is more appropriate for new stream consumers

Addressing PR comments
2016-02-22 17:33:18 -05:00
Mark Fisher
1b1ac1c07b use correct name for Redis consumer Binding 2016-02-22 17:14:38 -05:00
Marius Bogoevici
6c880726b8 Binder API simplification
Resolves #330

- removes `unbind` from the binder and moves it to the `Binding` instance itself;
- `Binding` is now an interface with a default implementation provided by SCS
- Removed all methods from Binder except for unbind()
- Removed old and unused code
- Removed circular reference between default Binder and Binding implementations
- Removed Binder type
2016-02-22 14:21:03 -05:00
Marius Bogoevici
73b4b6fb3e Fix race condition in multi-destination test 2016-02-19 17:10:08 -05:00
Marius Bogoevici
b1b0e960d7 Corrections for health indicator
Ensure that try-catch is applied around connecting to Zookeeper as well
Register topics in use for consumer
2016-02-19 16:49:54 -05:00
Ilayaperumal Gopinathan
0fb431db3a Add Kafka binder health indicator
- Health indicator fetches broker addresses using ZK configuration and matches it with the leaders of the paritions being used in the binder

This resolves #297

Use Set instead of List to store error messages per broker

Exception handling when connecting to ZK

Add ZK connect/session timeout values as configuration properties

 - Set it in the Kafka binder so that the same can be used for health indicator as well

Rename ZK properties and move them to binder configuration properties
2016-02-19 14:19:39 -05:00
Soby Chacko
76c64e3763 Allow binding to multiple input destinations
More integration tests in AbstractBinderTests
Use embedded Kafka in the tests
2016-02-17 19:11:00 +05:30
Ilayaperumal Gopinathan
71b26b0ed3 Remove unused kafka binder properties 2016-02-15 12:53:57 +05:30
Ilayaperumal Gopinathan
47d1d69402 Refactor Kafka binder configuration
- Move KafkaBinderConfigurationProperties out of the configuration class
 - Remove defaultProperties bean creation and use `locations` in `ConfigurationProperties` to load the default properties

Move properties to @PropertySource and add tests
2016-02-12 10:24:12 -05:00
Eric Bottard
412b1dd294 Use kafka on the travis infrastructure
Fixes spring-cloud/spring-cloud-stream#314

Give more time to redis on travis

Make multipliers more versatile, apply to kafka

Extract receive() method. Default is now 1s

bump multiplier for kafka on travis
2016-02-10 14:12:46 -05:00
Marius Bogoevici
c4a64b37fb Minor: formatting, imports, copyright 2016-02-09 21:50:34 -05:00
Soby Chacko
7138f125ee Upgrade Spring Integration Kafka to 1.3 in Kafka binder
Upgrade scala/SIK versions in mvn pom

Migrage to KafkaNativeOffsetManager from KafkaTopicOffsetManager

Introduce LoggingProducerListener for errors - See GH #151

Remove properties specific to the Topic offset manager
2016-02-09 21:13:59 -05:00
bamboo
0a6fc90f3a [artifactory-release] Next development version 2016-02-04 19:38:45 +00:00
bamboo
082e5e7330 [artifactory-release] Release version 1.0.0.M4 2016-02-04 19:38:44 +00:00
Marius Bogoevici
a854e986aa Revert to Rabbit version provided by Spring Boot/Cloud
At the time of the commit it is 1.5.3.RELEASE, but it will pull the version as per the Spring Cloud parent version.
2016-02-04 11:36:35 -05:00
Mark Fisher
7ffe9070e3 removed more lattice references 2016-02-04 08:55:07 -05:00
Ilayaperumal Gopinathan
32c727a5d4 Remove lattice profile usages/references 2016-02-04 13:40:53 +05:30
Ilayaperumal Gopinathan
9c381d3b26 Fix exception type in kafka topic validation test 2016-02-04 12:48:42 +05:30
Mark Fisher
252aee4079 polishing 2016-02-03 19:51:17 -05:00
Marius Bogoevici
8271478e27 GH-306 Add support for start offset and reset flag
Resolves #306

Fix configuration property passing

Adjust property names, fix environment merging from parent
2016-02-03 19:32:51 -05:00
Ilayaperumal Gopinathan
8f795074e9 Allow underscore in kafka topic name
- Instead of escaping the topic name, perform validation that throws `RuntimeException` when the given kafka topic name doesn't meet the criteria set by Kafka.

This resolves #217
2016-02-03 15:34:10 -05:00
Dave Syer
e9286c849a Add spring-rabbit explicitly to the dependencies BOM
It actually still has to be managed using a property in
spring-cloud-stream-binders-parent, but this change is for users.
Any user that wants an app using stream and rabbit needs to use the
spring-cloud-dependencies BOM. A gradle user has to do nothing else
with this change. A maven user still has to add an explicit
<spring-aqmp.version/> to the application POM.
2016-02-03 09:05:20 +00:00
Eric Bottard
7fb834981a GH-291 Provide an option to create a mutable Tuple
This fixes spring-cloud/spring-cloud-stream#290
2016-02-02 17:23:43 -05:00
Marius Bogoevici
7477fcbee9 Add health indicators infrastructure
- Adds a generic 'bindersHealthIndicator' bean controlled by `management.health.binders.enabled` that tallies results from binders
- Binders are expected to expose one or more health indicators based on the status of the middleware connection. If no health indicators are exposed, the status is deemed to be 'UNKNOWN'
- Add support for Redis and Rabbit based on existing health indicators

Moved binder health indicator to autoconfiguration

Also updated copyright
2016-02-02 22:15:33 +05:30
Marius Bogoevici
31961ae595 Add support for durable subscriptions in Redis 2016-01-29 19:04:12 -05:00
Dave Syer
dfe78f6d32 Fix Spring AMQP dependencies temporarily 2016-01-28 09:00:46 +00:00
Gary Russell
e2485e3290 Clean Up Auto-Declare Context on Unbind 2016-01-27 22:00:08 -05:00
Marius Bogoevici
192aa79baa Remove default groups, support default pubsub with empty group
Addressing PR comments
2016-01-27 21:59:58 -05:00
Mark Fisher
c3758b9dc0 Binder simplification
* removed pub/sub methods from binder
* consumer group is now a parameter of the remaining bindConsumer method

* remove DynamicProducer from Binder
* Move logic to create the dynamic channel to the channel resolver.
* Return bindings from bind methods and use them for unbinding
* Suffix for dlq

Move All Rabbit Binder CleanUp to Test Bindera

More RabbitMQ Binder Test Cleanup

Clean up declarations for remaining tests.

removed BinderUtils

use Redis ZSET for consumer groups

copyright dates

AutoBindDLQ: Single DLQ Per Group When Partitioned

Configure a single DLQ for each group for all partitions.

Add DLX Exchange binding for each original queue routing key, including the partition.

Fix DLQ Binding (Producer Side)

Option was not allowed and the routing key was wrong.

Add test to verify producers can be bound before consumers.

`autoBindDLQ` must be set (or reset) on both sides for success.
2016-01-27 21:59:58 -05:00
Ilayaperumal Gopinathan
e792b0e347 Use LocalizedQueueConnectionFactory from Spring AMQP
- Remove LocalizedQueueConnectionFactory and the unit tests
   - The unit tests are already covered in Spring AMQP and also, certain methods accessors require public access to run tests from SCS
 - Kept the integration tests as is (though this can be removed as well - it is also covered in Spring AMQP)

This resolves #279
2016-01-19 09:22:53 -05:00
Ilayaperumal Gopinathan
31ae416727 Remove Spring XD tuple dependency
- Use spring-cloud-stream-tuple instead

This resolves #280
2016-01-19 19:09:24 +05:30
Ilayaperumal Gopinathan
fa89e5bcb6 Handle binder config in BindingProperties
- Per-binding properties defined in `BindingProperties` need to hold the binder specific configurations needed when binding producer/consumers.
Previously, these were added in `ChannelBindingServiceProperties` as producer/consumer properties. Since these configuration properties are for
per-binding based on the input/output (consumer/producer) channel being bound it is better to move these properties into `BindingProperties`.
 - Add some of the supported producer/consumer properties into BindingProperties

- This fixes #256

Rename CommonBinderProperties -> BinderPropertyKeys

Rename class AbstractBinderPropertiesAccessor.java -> AbstractBindingPropertiesAccessor.java
2015-12-30 14:08:56 -05:00
bamboo
b869cb8c4b [artifactory-release] Next development version 2015-12-21 17:18:25 +00:00
bamboo
f3d46fc120 [artifactory-release] Release version 1.0.0.M3 2015-12-21 17:18:25 +00:00
Gary Russell
7ee5df4033 RabbitMQ Cleanup (New Tests).
Simple polishing.
2015-12-18 22:24:12 -05:00
Eric Bottard
286ba44ff1 Remove TODOs 2015-12-18 16:44:08 +01:00
Mark Fisher
df1817cc51 polishing 2015-12-17 18:38:35 -05:00
Marius Bogoevici
a4f97b364e Binders with default settings inherit from module's configuration
Resolve #229

* Optimizes connection factory creation for binders that can use Boot's autoconfiguration
* Make binder contexts a child of main context when creating a Binder with default configuration settings
* Inherit environment from main context when the Binder context is not a child, and make that optional
* Default environment inheritance to true
* Tests
2015-12-17 11:51:10 -05:00
Marius Bogoevici
bf178360ee Remove autoconfig
Ensure that the autoconfig and the cloud config don't overlap

Removed SeedConfiguration
2015-12-10 13:35:26 -05:00
Marius Bogoevici
96f96f7621 Initial implementation of multibinder support
- Addresses #138,#172,#173 with support for multiple binders in an application
- Binders can be of multiple types (e.g. Rabbit or Kafka), as well as multiple binders of the same type connecting to different system (e.g. multiple Rabbit binders connecting to different Rabbit clusters)
- Binders are now created in a separate context, allowing for multiple configurations
- Adds the `spring.cloud.spring.binders` namespace that allows for creating configuration properties for multiple binders
- Add support for default binder, modify Kafka tests to use multiple brokers when running embedded
2015-12-09 11:08:04 -05:00
Marius Bogoevici
bcff75b20b GH-213 Merge Binder SPI into Core
- Moved contents of `spring-cloud-stream-binder-spi` into `spring-cloud-stream`
- Removed `spring-cloud-stream-binder-local` and used local binder implementation for tests exclusively
2015-11-30 17:28:24 -05:00
Gary Russell
6f1f246dfb GH-211: PubSub Group Default to Random UUID
Resolves #211

Add a default UUID to the `BindingProperties.group` so a pubsub binding is
unique unless the group is specifically set.
2015-11-30 13:32:31 -05:00
Marius Bogoevici
770047871b Moved Rabbit management outside with the Rabbit bus 2015-11-19 17:15:02 -05:00
Gary Russell
d3c87daf64 GH-203: Support Late Binding with RabbitMQ
If the broker was down, the binding failed. Defer the queue/exchange/binding declarations
until the broker is available (already supported by `RabbitAdmin`.

Solves spring-cloud/spring-cloud-stream#203

Fix PubSub, Test

- Too many binder prefixes on pub sub queue name.
- Mismatched queue args in test case.
2015-11-19 10:28:58 -05:00
Marius Bogoevici
522881e879 Ignore RawModeKafkaBinderTests#createInboundPubSubBeforeOutboundPubSub 2015-11-11 13:58:11 -05:00