Commit Graph

44 Commits

Author SHA1 Message Date
Vinicius Carvalho
4422b21438 Add Avro serialization and schema management support
- Add schema server implementation
- Add schema client abstraction
- Add schema client implementation for own schema registry server
- Add schema client supporting Confluent schema registry
- Add Avro-based message converter supporting a static schema resource
- Add Avro-based message converter with schema evolution support, via
  schema registry client.
- On serialization, the converter register writer schemas with the schema
  registry server and augment the content type of outbound message with
  schema information.
  On deserialization, the reading converter will fetch the schema from the server
  if not available locally.

Use class information if schema is not specified

In the case of SpecificRecord and Reflective readers/writers, the class information can be used instead

Make subtype prefix configurable and shorten the subject

- Subtype prefix is now configurable and subject is the lowercase schema name
- Enhance/correct javadoc

Refine AbstractAvroMessageConverter

- distinguish between writer and reader schema when reader is created

Add schema registry and schema registry client docs
2016-07-31 21:26:36 -04:00
Marius Bogoevici
08d65a92bb Initial support for Reactive API
Fixes #520
Fixes #458 (for non-reactive binders)

Adds spring-cloud-stream-reactive module.
Introduces support for declarative @StreamListener and @Input and @Output annotated parameters.
Add StreamListenerArgumentAdapter and StreamListenerResultAdapter for wrapping bindable inputs
and outputs when passing arguments to declarative @StreamListener.
Adds support for using reactive types (Flux/Observable) with traditional binders.
Introduce FluxSender and ObservableSender for handling multiple streaming outputs per method.
Ensure that errors are caught and logged.

Fixing constructor assertions and Javadoc

Addressing PR comments

- rework @Input/@Output parameter validation
- renamed StreamListenerArgumentAdapter to StreamListenerParameterAdapter
- ensure that parameter direction is accounted for in the current adapters
2016-07-26 14:47:03 -04:00
Marius Bogoevici
55b9aa75dd Rework the converter system
Fixes #505

The goals are twofold: to simplify the registration of new converters and
to add some consistency to the conversion process, and align them with the
way converters are used in generic Spring Messaging listeners, by describing
two possible transformations: inbound, message (including contentType)->targetClass,
and outbound payload+headers -> message. The idea is for the two
transformations to match the input and output directions of the bound channels.

List of changes:

- Create and configure input/output channels distinctly - reflected
  in the definitions of `BindableChannelFactory` and `MessageChannelConfigurer`;
- Replace AbstractFromMessageConverter with bidirectional converters;
- Use the channel direction (input/output) to determine whether
  `toMessage` or `fromMessage` will be invoked;
- Use contentType support from `AbstractMessageConverter` to map
  converters to mime types instead;

Making converters more robust

Addressing comments

Convert content type headers to String before serializing

Addressing further PR comments
2016-07-22 17:02:38 -04:00
Marius Bogoevici
bd92af4784 Add AbstractMessageChannelBinder
Handle common aspects of message-channel binders:
- added generic, customizable ReceivingHandler and SendingHandler
- made doBindProducer delegate to a series of template methods
- made doBindConsumer delegate to a series of template methods
- moved partitioning to an interceptor, thus fixing #493

Removed unused manual ack handling
2016-07-18 17:15:49 -04:00
Marius Bogoevici
aef08a28a3 Move tests to AssertJ 2016-06-01 13:10:42 -04:00
Ilayaperumal Gopinathan
bcc45b8685 Use random port in StreamListenerTests 2016-05-13 11:44:35 +05:30
Marius Bogoevici
18dc0a707d Implement remainder of Spring Boot Checkstyle code rules
Fixes #532

Implements most of the remaining rules from Spring Boot, except JavaDoc + a few additional ones
2016-05-12 19:19:59 -04:00
Marius Bogoevici
a974ff7773 Checkstyle rules for imports and cleanup
Fixes #529

- enforced Spring Framework ordering rules
- disallowed static imports except for specific classes in test frameworks
2016-05-11 22:13:18 -04:00
Marius Bogoevici
296d710969 Update project version to 1.1.0.BUILD-SNAPSHOT 2016-05-10 19:41:15 -04:00
bamboo
e211a6340d [artifactory-release] Next development version 2016-05-10 13:33:46 +00:00
bamboo
53f1839a82 [artifactory-release] Release version 1.0.0.RELEASE 2016-05-10 13:33:46 +00:00
bamboo
450350f995 [artifactory-release] Next development version 2016-04-26 22:35:07 +00:00
bamboo
f11414230f [artifactory-release] Release version 1.0.0.RC3 2016-04-26 22:35:07 +00:00
Marius Bogoevici
147a0cb0bc Add checkstyle support
Fixes #481

Execute checkstyle validation at build time.
Add minimal set of rules.
Ensure that existing code conforms to the rules.
2016-04-15 10:44:37 +05:30
bamboo
f9330f1117 [artifactory-release] Next development version 2016-04-07 13:25:26 +00:00
bamboo
727434b732 [artifactory-release] Release version 1.0.0.RC2 2016-04-07 13:25:26 +00:00
Marius Bogoevici
d2e00610cd Make a globally defined ObjectMapper truly optional 2016-04-06 19:54:08 -04:00
Marius Bogoevici
0b4b916fe6 Minor fixes: formatting, cleanup 2016-04-06 11:29:38 -04:00
Ilayaperumal Gopinathan
277d896e1d Use Jackson ObjectMapper from AutoConfiguration
- For the message converters that use Jackson ObjectMapper, autowire the ObjectMapper from JacksonAutoConfiguration
   - This will allow configuration of ObjectMapper using boot configuration properties `JacksonProperties`

 - Add test

This resolves #466
2016-04-06 09:57:40 -04:00
bamboo
a9e432645d [artifactory-release] Next development version 2016-03-22 01:27:46 +00:00
bamboo
70a6abbc61 [artifactory-release] Release version 1.0.0.RC1 2016-03-22 01:27:46 +00:00
Marius Bogoevici
f681e2ce91 Setting version to 1.0.0.BUILD-SNAPSHOT 2016-03-21 21:11:25 -04:00
bamboo
98de668170 [artifactory-release] Next development version 2016-03-22 00:44:17 +00:00
bamboo
13daf68697 [artifactory-release] Release version 1.0.0.RC1 2016-03-22 00:44:17 +00:00
Marius Bogoevici
95b0197dd7 Removing message history support 2016-03-21 19:28:55 +05:30
Ilayaperumal Gopinathan
216149b78d Rename BindingListenerTests -> StreamListenerTests 2016-03-21 19:04:29 +05:30
Marius Bogoevici
4d2ef3c4d0 Introduce StreamListener annotation
Fixes #156

- Supports Spring Messaging infrastructure for argument mapping and type conversion, sharing the same configured converters as the ones configured on input/output channels;

- Also: refactored AbstractFromMessageConverter to return only payload - conversion to a message enriched with the content type of the channel is done by a separate wrapping converter;
2016-03-17 16:02:51 -04:00
Marius Bogoevici
3cd513d023 Minor polishing 2016-03-16 10:57:02 -04:00
Ilayaperumal Gopinathan
9abf36879d Set message header using BindingProperties' contentType
- Update MessageConverterConfigurer that uses a channel interceptor to check if the message contentType header is missing and if there is a binding property for `contentType` header is provided, then set that value as the message contentType header

This resolves #397

Add test
2016-03-16 12:41:29 +05:30
Marius Bogoevici
a7f21b542a Minor updates for custom content types
- renamed MessageConverterUtils.getJavaType to MessageConverterUtils.getJavaTypeForJavaObjectContentType and restricted usage only to 'application/x-java-object' types
2016-03-15 18:14:36 -04:00
Ilayaperumal Gopinathan
74c6223d3e Support custom message converters
- Autowire custom message converters into MessageConverterConfigurer
  - When configuring the message channel, set the `datatypes` of the channel based on the `supported` datatypes of all the matching message converters

This resolves #381
2016-03-15 17:19:31 -04:00
Marius Bogoevici
9b0c4bd627 Typesafe consumer and producer configurations
- Change the Binder interface to support ConsumerProperties/ProducerProperties beans and subclasses
- Binders can subclass the property beans to add new supported properties that will be automatically populated
- Spring Cloud Stream will  infer the target type and populate the beans from the environment based on a `spring.cloud.stream.bindings..<bindingName>` prefix
- Remove binder defaults and retain only general binder configurations

TODO: a) decide on instanceIndex/partitionIndex alignment (we do not need both)
b) support `defaultProducer`/`defaultConsumer` properties
c) add leniency control on binding (fail/ignore for unknown properties)
d) add a `requiredProperties` configuration for consumer/producer properties to finely tune the mandatory properties expected to be supported by a bound application

Changes made during review:

- Add support for consumer and producer defaults
- Remove partitionIndex, keeping only instanceIndex
- Fix default properties for Kafka binder
- Move batching properties to Rabbit only
2016-03-15 15:38:30 -04:00
Ilayaperumal Gopinathan
b184f67ee1 Use TestSupportBinder instead of redis binder in integration tests
- Modify the integration tests to verify the usecases per app basis (that uses the TestSupportBinder for its channel binding)
 - Remove redis binder usage in integration tests

This resolves #380
2016-03-04 11:27:27 -05:00
Marius Bogoevici
5670e29776 Test fixes
Class rule for Redis
Do not pass message collector for test input bindings
2016-03-02 11:28:23 -05:00
Ilayaperumal Gopinathan
cae56385e0 Binding error channel
- bind Spring Integration `errorChannel` as a message producer when the property `spring.cloud.stream.bindings.error.destination` is set
    - The error channel will use this property value as the destination name
  - Add test

This resolves #329

Rename NoopBindable -> BindableAdapter

Return error channel name for getOutputs

Fix review comments

  - Modified integration test to use `Sink` interface
  - Add unit test
2016-03-01 12:50:20 -05:00
Ilayaperumal Gopinathan
69e484163c Move spring-cloud-stream tuple
- The `spring-cloud-stream-tuple` is now split into `spring-tuple`, `spring-integration-tuple` along with `spring-batch-tuple`
  - Update dependencies to use the new ones

This resolves #326
2016-02-26 17:12:49 -05:00
Ilayaperumal Gopinathan
4dfd217db6 Maven configuration simplification
- Move kafka related dependencies from parent `dependencyManagement` to `kafka` binder project
 - Remove duplicate properties
 - Fix typo

This resolves #353
2016-02-25 12:44:27 -05:00
bamboo
0a6fc90f3a [artifactory-release] Next development version 2016-02-04 19:38:45 +00:00
bamboo
082e5e7330 [artifactory-release] Release version 1.0.0.M4 2016-02-04 19:38:44 +00:00
Marius Bogoevici
ebc0234a67 Remove direct references to snapshots and rely on project.version instead 2016-02-04 13:27:26 -05:00
Ilayaperumal Gopinathan
fa89e5bcb6 Handle binder config in BindingProperties
- Per-binding properties defined in `BindingProperties` need to hold the binder specific configurations needed when binding producer/consumers.
Previously, these were added in `ChannelBindingServiceProperties` as producer/consumer properties. Since these configuration properties are for
per-binding based on the input/output (consumer/producer) channel being bound it is better to move these properties into `BindingProperties`.
 - Add some of the supported producer/consumer properties into BindingProperties

- This fixes #256

Rename CommonBinderProperties -> BinderPropertyKeys

Rename class AbstractBinderPropertiesAccessor.java -> AbstractBindingPropertiesAccessor.java
2015-12-30 14:08:56 -05:00
bamboo
b869cb8c4b [artifactory-release] Next development version 2015-12-21 17:18:25 +00:00
bamboo
f3d46fc120 [artifactory-release] Release version 1.0.0.M3 2015-12-21 17:18:25 +00:00
Ilayaperumal Gopinathan
100a68c1bf GH-196 Add Message History Tracking
- Add `MessageChannelConfigurer` that any class that configures the message channel would implement
   - Have `message-converter` and `message-history-tracking` configurer implement this interface
  - Autowire all available configurers and use them to configure the message channel
  - For the message history tracking
    - add `channel-binding` properties as part of message header along with `timestamp`

 - Introduce CompositeMessageChannelConfigurer

Remove field injection of messageChannelConfigurer

Add tests for message track history configurer

Add content-type configurer test

Address review comments

 - Add `toString` to BindingProperties
 - Add test case to depict producer/consumer properties

Remove processor channels from tests
2015-12-18 20:13:40 -05:00