Commit Graph

378 Commits

Author SHA1 Message Date
Patrick Peralta
d34aafb87b Partition handling refactoring
Moved partition calculation from AbstractBinder to
its own class PartitionHandler.

Updated per review feedback

Refactored bean creation
Added synchronization around bean creation
2016-02-24 16:42:51 -05:00
Ilayaperumal Gopinathan
bc0723df89 Remove AggregateBuilder 2016-02-23 22:14:47 -05:00
Ilayaperumal Gopinathan
7d7fcc9184 Add AggregateApplicationBuilder
- Uses builder pattern to create aggregate application that supports specifying
  - args
  - profiles
  - config name for each child app and parent aggregate app separately.

An example config would look like this:

```
@SpringBootApplication
public class DoubleApplication {

	public static void main(String[] args) {
		new AggregateApplicationBuilder().
				from(SourceApplication.class).args("--fixedDelay=5000")
				.to(SinkApplication.class).args("--debug=true").run("--spring.application.name=aggregate-test");
	}
```

This resolves #362
2016-02-23 17:15:03 -05:00
Ilayaperumal Gopinathan
81e6769b74 Make history tracker properties configurable
- Add property `spring.cloud.stream.bindings.<channelName>.trackHistoryProperties` with comma separated string values that specify the exact
property names in `ChannelBindingServiceProperties` to be added in the message tracker header
 - Add tests

This resolves #252

Add test for empty trackedHistories

 - rename `trackHistoryProperties` to `trackedProperties`
 - updated Javadoc
2016-02-23 00:32:13 -05:00
Ilayaperumal Gopinathan
047a3f8d4f Support sync producer option in Kafka binder
- Add a binding property `spring.cloud.stream.bindings.<outputChannelName>.syncProducer`
   When set to `true`, the Kafka producer metadata is set to use `sync` producer
 - Update DefaultPropertiesAccessor for Kafka to use the `syncProducer` property
 - Add test

This resolves #343

Move syncProducer as a binder level property

 - Add a property `syncProducer` in Kafka Binder configuration properties (prefix: spring.cloud.stream.binder.kafka)
  - Set Kafka ProducerMeatadata `sync` based on the value set in this property
 - Update test
2016-02-23 00:23:18 -05:00
Ilayaperumal Gopinathan
b98d91da39 Support proxy settings for Aether Module resolver
- Add `AetherProxyProperties` as a ConfigurationProperties for the proxy settings needed for the ProxySelector in Aether system
 - Update proxy settings in two places:
   1) Where the remote repositories are setup
   2) WHere the repository system session is created to resolve the artifacts

This resolves #272

Set authentication only for proxy settings

Set port type to `int`
2016-02-22 21:38:30 -05:00
Mark Fisher
9c3eef49da TestBinding polishing 2016-02-22 17:45:09 -05:00
Marius Bogoevici
2fa9cda89c Durability Configuration and Default Groups
Resolves #317

Remove the `durable` binder configuration property
Make subscriber groups durable by default
Introduce `requiredGroups` property
Kafka groups (non-anonymous) now start by default at EARLIEST, which is more appropriate for new stream consumers

Addressing PR comments
2016-02-22 17:33:18 -05:00
Mark Fisher
1b1ac1c07b use correct name for Redis consumer Binding 2016-02-22 17:14:38 -05:00
Marius Bogoevici
492522f1f9 Added code of conduct 2016-02-22 16:30:36 -05:00
Marius Bogoevici
6c880726b8 Binder API simplification
Resolves #330

- removes `unbind` from the binder and moves it to the `Binding` instance itself;
- `Binding` is now an interface with a default implementation provided by SCS
- Removed all methods from Binder except for unbind()
- Removed old and unused code
- Removed circular reference between default Binder and Binding implementations
- Removed Binder type
2016-02-22 14:21:03 -05:00
Marius Bogoevici
73b4b6fb3e Fix race condition in multi-destination test 2016-02-19 17:10:08 -05:00
Marius Bogoevici
b1b0e960d7 Corrections for health indicator
Ensure that try-catch is applied around connecting to Zookeeper as well
Register topics in use for consumer
2016-02-19 16:49:54 -05:00
Ilayaperumal Gopinathan
0fb431db3a Add Kafka binder health indicator
- Health indicator fetches broker addresses using ZK configuration and matches it with the leaders of the paritions being used in the binder

This resolves #297

Use Set instead of List to store error messages per broker

Exception handling when connecting to ZK

Add ZK connect/session timeout values as configuration properties

 - Set it in the Kafka binder so that the same can be used for health indicator as well

Rename ZK properties and move them to binder configuration properties
2016-02-19 14:19:39 -05:00
Ilayaperumal Gopinathan
43c01f2545 Remove spring-cloud-stream-samples
- All the samples are moved into a separate repository

This resolves #321
2016-02-19 12:18:56 -05:00
Ilayaperumal Gopinathan
706711eaf8 Binder interface javadoc update
- `module` to `app`
 - Add more detail to the javadoc
2016-02-18 13:29:17 +05:30
Soby Chacko
76c64e3763 Allow binding to multiple input destinations
More integration tests in AbstractBinderTests
Use embedded Kafka in the tests
2016-02-17 19:11:00 +05:30
Ilayaperumal Gopinathan
1216b4b797 Polishing
- Fix the logger level for `onCompleted` method invocation
 - Add `running` check on stop method
 - Update copyright/author
2016-02-16 00:42:33 +05:30
Marius Bogoevici
3874620d2a Fix RxJava processor lifecycle
-  Make SubjectMessageHandler a SmartLifecycle and assign it to phase 0, allowing it to be started after the outputs are bound and before the inputs are bound;
- Setup Rx in the start() method;
- Set the ServiceActivator associated with the handler to phase 0 as well;
2016-02-16 00:18:39 +05:30
Ilayaperumal Gopinathan
71b26b0ed3 Remove unused kafka binder properties 2016-02-15 12:53:57 +05:30
Marius Bogoevici
6691e63427 GH-338 Add spring-cloud-stream-config-metadata to dependencies 2016-02-12 13:26:33 -05:00
Ilayaperumal Gopinathan
47d1d69402 Refactor Kafka binder configuration
- Move KafkaBinderConfigurationProperties out of the configuration class
 - Remove defaultProperties bean creation and use `locations` in `ConfigurationProperties` to load the default properties

Move properties to @PropertySource and add tests
2016-02-12 10:24:12 -05:00
Ilayaperumal Gopinathan
528b90b6b5 Fix autowiring CompositeHealthIndicator
- The setter method `setBindersHealthIndicator(CompositeHealthIndicator bindersHealthIndicator)` is sufficient enough to autowire the candidate bean with the name `bindersHealthIndicator` based on the hint at the method argument.
   This gets interesting when the health indicator is disabled via `management.health.binders.enabled` and thereby the `bindersHealthIndicator` bean is not instantiated. In this case, if there are any matching beans of the type
`CompositeHealthIndicator` then that competes for the autowiring in the setter.

 - Add `Qualifier` with the name `bindersHealthIndicator` which makes the autowiring happen only with the bean named `bindersHealthIndicator` and thereby no `type` matching
 - Add test that demonstrates this uses case

This resolves #339
2016-02-11 16:48:37 -05:00
Eric Bottard
412b1dd294 Use kafka on the travis infrastructure
Fixes spring-cloud/spring-cloud-stream#314

Give more time to redis on travis

Make multipliers more versatile, apply to kafka

Extract receive() method. Default is now 1s

bump multiplier for kafka on travis
2016-02-10 14:12:46 -05:00
Marius Bogoevici
c4a64b37fb Minor: formatting, imports, copyright 2016-02-09 21:50:34 -05:00
Soby Chacko
7138f125ee Upgrade Spring Integration Kafka to 1.3 in Kafka binder
Upgrade scala/SIK versions in mvn pom

Migrage to KafkaNativeOffsetManager from KafkaTopicOffsetManager

Introduce LoggingProducerListener for errors - See GH #151

Remove properties specific to the Topic offset manager
2016-02-09 21:13:59 -05:00
Donovan Muller
ccc6049adc Rabbit binder environment should be named rabbitmq
In the context of trying to configure Rabbit, the docs mention that the Rabbit config properties be added under `rabbit`:

```
      ...
      binders:
        rabbit1:
          type: rabbit
          environment:
            spring:
              rabbit:
                host: <host1>
      ...
```

however, the properties should be configured under `rabbitmq` as reference in the [Spring Boot Appendix A](https://docs.spring.io/spring-boot/docs/current/reference/html/common-application-properties.html)
2016-02-08 08:32:59 +02:00
Marius Bogoevici
bfa4c77e40 Post-release: Restore snapshot dependencies and module launcher default repo 2016-02-04 16:25:07 -05:00
bamboo
0a6fc90f3a [artifactory-release] Next development version 2016-02-04 19:38:45 +00:00
bamboo
082e5e7330 [artifactory-release] Release version 1.0.0.M4 2016-02-04 19:38:44 +00:00
Marius Bogoevici
62a60a4d47 Pre-release: update Spring Cloud versions and repository locations for Module Launcher 2016-02-04 14:06:14 -05:00
Marius Bogoevici
f88a49eaa7 GH-312 Add default constructor for TupleBuilder
Resolves #312

The TupleBuilder in XD provided a zero arg constructor. This restores backward compatibility by adding it back.
2016-02-04 14:04:41 -05:00
Marius Bogoevici
44c2192f36 Add instructions for running multibinder examples. 2016-02-04 13:58:09 -05:00
Marius Bogoevici
ebc0234a67 Remove direct references to snapshots and rely on project.version instead 2016-02-04 13:27:26 -05:00
Ilayaperumal Gopinathan
490baef8ac Update doc for multibinder sample apps
Update samples ascii doc
2016-02-04 12:14:24 -05:00
Marius Bogoevici
a854e986aa Revert to Rabbit version provided by Spring Boot/Cloud
At the time of the commit it is 1.5.3.RELEASE, but it will pull the version as per the Spring Cloud parent version.
2016-02-04 11:36:35 -05:00
Ilayaperumal Gopinathan
3e10eac8ac Update docbook copyright year 2016-02-04 21:55:30 +05:30
Mark Fisher
944c100576 doc polishing 2016-02-04 10:44:08 -05:00
Marius Bogoevici
b490b6b73d Adding binder docs + 1.0.0.M4 updates
Health indicators
2016-02-04 09:04:00 -05:00
Mark Fisher
7ffe9070e3 removed more lattice references 2016-02-04 08:55:07 -05:00
Ilayaperumal Gopinathan
32c727a5d4 Remove lattice profile usages/references 2016-02-04 13:40:53 +05:30
Ilayaperumal Gopinathan
9c381d3b26 Fix exception type in kafka topic validation test 2016-02-04 12:48:42 +05:30
Mark Fisher
252aee4079 polishing 2016-02-03 19:51:17 -05:00
Marius Bogoevici
8271478e27 GH-306 Add support for start offset and reset flag
Resolves #306

Fix configuration property passing

Adjust property names, fix environment merging from parent
2016-02-03 19:32:51 -05:00
Ilayaperumal Gopinathan
8f795074e9 Allow underscore in kafka topic name
- Instead of escaping the topic name, perform validation that throws `RuntimeException` when the given kafka topic name doesn't meet the criteria set by Kafka.

This resolves #217
2016-02-03 15:34:10 -05:00
Dave Syer
e9286c849a Add spring-rabbit explicitly to the dependencies BOM
It actually still has to be managed using a property in
spring-cloud-stream-binders-parent, but this change is for users.
Any user that wants an app using stream and rabbit needs to use the
spring-cloud-dependencies BOM. A gradle user has to do nothing else
with this change. A maven user still has to add an explicit
<spring-aqmp.version/> to the application POM.
2016-02-03 09:05:20 +00:00
Eric Bottard
7fb834981a GH-291 Provide an option to create a mutable Tuple
This fixes spring-cloud/spring-cloud-stream#290
2016-02-02 17:23:43 -05:00
Marius Bogoevici
7477fcbee9 Add health indicators infrastructure
- Adds a generic 'bindersHealthIndicator' bean controlled by `management.health.binders.enabled` that tallies results from binders
- Binders are expected to expose one or more health indicators based on the status of the middleware connection. If no health indicators are exposed, the status is deemed to be 'UNKNOWN'
- Add support for Redis and Rabbit based on existing health indicators

Moved binder health indicator to autoconfiguration

Also updated copyright
2016-02-02 22:15:33 +05:30
Ilayaperumal Gopinathan
a2486db905 Module launcher JMX domain name
- Module launcher needs to use its own jmx domain name so that it doesn't collide with any other default domains regisetered.

This resolves #286
2016-02-01 23:29:15 -05:00
Marius Bogoevici
2f330c7bc4 Minor polishing on dynamic channel binding 2016-02-01 22:48:48 -05:00