Commit Graph

828 Commits

Author SHA1 Message Date
Oleg Zhurakousky
5a87211237 Fixed warnings in spring-cloud-stream module
- Removed/modified unnecessary @SupressWarning annotations
 - Removed deprecated configuration for SharedChannelRegistry and fixed associated tests
 - Fixed and documehted other deprecations
2017-11-22 14:22:58 -05:00
Oleg Zhurakousky
c94f946c85 1136 Fixed type expectations in BinderProperties
- changed BinderProperties environment to be Map to support  appropriate semantics where keys and values may be non-Strings
- deprecated `setEnvironment(Properties environment)` operation

Resolves #1136

polishing
2017-11-22 12:25:19 -05:00
Gary Russell
54abf564fd KGH-12: Late binding for kafka etc
Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/12

Support late binding for binders that don't inherently support it.

Schedule attempts at 30 second intervals (default, configurable).

`IllegalState` and `IllegalArgument` exceptions are fatal.

Add test cases (mocks) to verify rebind after failure

Docs

Polishing - always use the setter on LateBinding

Remove volatile from the fields since they are only accessed from synchronized methods.
2017-11-22 10:50:34 -05:00
Oleg Zhurakousky
21a2a84d86 GH-1127: Add stub Binder backed by SI
Resolves: spring-cloud/spring-cloud-stream#1127

- Added initial implementation of SI-backed stub binder to facilitate
 more consistent testing
- Improved BinderAwareChannelResolverTests and ExtendedPropertiesBinderAwareChannelResolverTests
to make use of it
- Added sample Application that demonstrates the binder usage

Addressed PR comments
2017-11-15 09:49:02 -05:00
Oleg Zhurakousky
024e3cffcc Finished consolidating contentType Message conversion
- Removed MessageSerializationUtils
    - Consolidated Message contentType conversion refactoring that was started with d0be34f7cb commit

At this point Message conversion is consolidated in either MessageConverters or In/Out channel interceptors configured in MessageConverterConfigurer
2017-11-14 22:06:44 -05:00
Oleg Zhurakousky
464a98aa10 polishing
Resolves #1108
2017-11-14 10:29:29 -05:00
David Kalosi
08761c3de0 subject naming strategy POC
checkstyle fix

more checkstyle fixes
2017-11-14 10:23:41 -05:00
Oleg Zhurakousky
d0be34f7cb Refactored MessageConverterConfigurer
- Separated InboundContentTypeConvertingInterceptor and OutboundContentTypeConvertingInterceptor to isolate logic that needs to be performed by such interceptors for *inbound* and *outbound* messages.
- Documented their purpose via javadocs
- Moved InboundMessageConvertingInterceptor to TestSupportBinder (for now) to only support MessageCollector's 1.3 behavior
2017-11-14 10:12:43 -05:00
Gary Russell
83fa5f3858 GH-1009: Producer properties for dynamic bindings
Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/1009

Provide a callback mechanism to set properties on dynamically created bindings.

Polishing - callback name.

Resolves #1009
Resolves #1132
2017-11-14 09:25:27 -05:00
Soby Chacko
73f0fc38a6 Relax Consumer channel binding in AbstractTestBinder
When native decoding is enabled on the consumer, relax the requirement
for using AbstractBinderTest.createBindableChannel(..)

Fixes #1137
Resolves #1138
2017-11-14 08:52:01 -05:00
Oleg Zhurakousky
2275d40d7d polishing
Resolves #1133
2017-11-12 17:07:11 -05:00
Soby Chacko
53df5f5747 Native endoding/decoding changes for interceptors
Fixes #1109

- Disable outbound content type interceptor when nativeEndoding is enabled on the producer
- Disable inbound content type interceptor when nativeDecoding is enabled on the consumer

- Add a new flag in ConsumerProperties for useNativeDecoding

- Apply ContentType interceptor before the InboundInterceptor to ensure
  that content type set on the channel is applied on the message if content type
  is missing on the Message.

- Add tests for verifying interceptors are not applied when native encoding/decoding is enabled
- Fix a kryo integration test
- Remove Inbound interceptor from MessageCollector when native encoding is enabled on the producer
- Polishing
2017-11-12 17:01:20 -05:00
Gary Russell
b865f3cbe9 RGH-83: Allow binders to add interceptors
Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-rabbit/issues/83

The RabbitMQ binder needs to add an interceptor to evaluate expressions (if they
include `payload`) before the payload is serialized.

Add a hook to allow the binder to post-process the message channel.

Resolves #1134
2017-11-12 16:15:21 -05:00
Jacob Severson
61240fd0fa Adding hook for changing message target delimeter
Resolves #1120
2017-11-12 15:46:29 -05:00
Soby Chacko
9cc0b83a38 Update to next version: 2.0.0.BUILD-SNAPSHOT
Update spring-cloud-build parent to 2.0.0.BUILD-SNAPSHOT
2017-11-09 09:02:40 -05:00
Soby Chacko
03b7b0d612 Update to release version: 2.0.0.M3
spring-cloud-build parent to 2.0.0.M4
2017-11-09 08:50:15 -05:00
Oleg Zhurakousky
4e69ead7b4 GH-1126 Fixed default app port in Tests
Ensured AggregateApplicationTests, AggregateWithMainTest and AggregationTest arn't using default 8080 port
2017-11-08 13:10:47 -05:00
Gary Russell
b9884ae71a Add comment to embedded header decode error log 2017-11-08 10:18:01 -05:00
Oleg Zhurakousky
62c8513197 Initial cleanup of AbstractMessageChannelBinder
Removed extra handler for embeddedHeaders in favor of ChannelInterceptor to avoid additional dispatch

Addressed PR comments
2017-11-08 10:11:51 -05:00
Oleg Zhurakousky
a7fdf6dbb2 GH-1106 Addressed backward/forward contentType compatibility issues
Fixes #1106

- This PR builds on the previous PR with commit hash 6c259be6... (pr/1112)
- Added support in AbstractBinderTests to create bindable channel based on determining channel input type based on it's name (i.e., *input*)
- Added LegacyContentTypeHeaderInterceptor to TestSupportBinder to restor the previous behavior of MessageCollector for cases
  where Message's payload content type is a variant of 'text'.
- Restored tests that use MessageCollector to depend on proper payload type
- Added 'deserialize' routine back to MessageSerializationUtils
- Polished MessageConverterConfigurer.LegacyContentTypeHeaderInterceptor to remove conditional original-content-type header logic
- Removed default contentType from BindingProperties
- Restored tests that use MessageCollector to depend on proper payload type
- polishing
- fixed Kryo/Java serialization
- Fixed ser/de for "application/json" and contentType equals
- Fixed of ser/de of JSON strings to ensure that Strings are not re-quoted
- Fixed how we comparing contentTypes
- more polishing
- Fixed NPE in MessageSerializationUtils
- Make bindings and consumer groups in new tests added to AbstractBinderTests mutually exclusive
2017-11-07 11:21:13 -05:00
Soby Chacko
6c259be62b Legacy cotent-type changes for the consumer
- Remove binding config property `legacyContentTypeHeaderEnabled` that was
  introduced to enable legacy content type handling
- Enable LegacyContentTypeInterceptor in 2.0 always, but bypass any
  legacy content type handling if the message received is from
  a 2.0 producer by checking on the version header
- Introdce a new BinderHeader property for version
- Fix the LegacyContentType related tests
- Remove the check for originalContentType in ReceivingHandler when
  the payload received is a byte[]
- Remove unnecessary deserializePayloadIfNecessary calls in ReceivingHandler
- Remove deprecated deserializePayload methods in AbstractBinder
  and MessageSerializationUtils

Partly fixes #1106
Fixes #1110
2017-10-26 13:05:10 -04:00
Oleg Zhurakousky
4cc66ae0e0 fixed AbstractBinderTests to take into account previous commit '6eff89ca6c4dfc6ae9e0777e5f97e63c45615afc' 2017-10-26 10:59:23 -04:00
Oleg Zhurakousky
6eff89ca6c Added MessageChannel configuration assertion (#1113)
Added assertion to org.springframework.cloud.stream.binder.AbstractTestBinder to ensure that MessageChannels that are passed to any of the bind methods are configured with Message Interceptors
2017-10-26 10:16:57 -04:00
Oleg Zhurakousky
e45ebb3511 Polishing (Fixed tests) 2017-10-24 14:51:28 -04:00
Gary Russell
b5aa59e244 Prep work for Kafka Issue #236
See https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/236

- add `none` as a synonym for `HeaderMode.raw`
- add `headers` meaning "use native headers"
- change the default header mode - `null`
- remove `supportsHeadersNatively` - allow binders to chose header mode for each binding
  binders decide their own default when the property is null
- Rabbit will choose `headers` as default (but can be overridden to `none` for a binding)
  `embeddedHeaders` will not be allowed
- Kafka will choose `embeddedHeaders` as default for compatibility with 1.x peer apps
- remove deprecations on header embedding code

Polishing; support fallback

- if the consumer side is configured to support embedded headers,
  binders should look for native headers first and signal their presence
  using the `BinderHeaders.NATIVE_HEADERS_PRESENT` header;
  fall back to embedded headers if it appears the payload might contain
  them
- remove support for XD embedded headers (first byte < 0xff)
- add a quick check for embedded headers by examining the first byte for 0xff
- change the log from error to debug when decoding embedded headers fails

- see the kafka binder PR for a test case with a consumer that can handle all
  message flavors (native, embedded, and no headers).
2017-10-24 14:49:34 -04:00
Soby Chacko
a06231b20f Next build version - 2.0.0.BUILD-SNAPSHOT 2017-10-19 09:22:54 -04:00
Soby Chacko
061d6bed5c Update version for 2.0.0.M2 release
sprig cloud stream - 2.0.0.M2
spring cloud build - 2.0.0.M4

Closes #1105
2017-10-19 08:27:59 -04:00
Soby Chacko
66526d1176 Remove the hard-coded versions for Spring Integration in pom.xml
Spring Cloud Stream already gets it through Spring Boot

Fixes #1103
Fixes #1104
2017-10-18 18:44:58 -04:00
Laur Aliste
1e7150205f Fix typos in spring-cloud-stream-overview.adoc
- critiical -> critical;
- end 'etc' with a period; also consistently end etc in parentheses with
  'etc.).';
- typo in 'application';
- applicaiton -> application;
2017-10-16 10:43:27 +02:00
John Carey
598fc0b8e7 Fixing @Transformer for ProcessorApplication.
Without explicitly naming the `inputChannel` and `outputChannel`, well get this error on startup org.springframework.integration.MessageDispatchingException: Dispatcher has no subscribers
2017-10-10 13:43:05 -03:00
Vinicius Carvalho
081d8d8178 Disabled logStartupInfo on child context 2017-10-10 10:54:39 -04:00
Soby Chacko
08504952e6 Default interceptor for legacy contentType
Fixes #1057

Enable consumers to convert from originalContentType to contentType in the case of
legacy producers sending messages with originalContentType header.
By default, this conversion will not happen and only be acitvated by setting
`spring.cloud.stream.bindings.input.legacyContentTypeHeaderEnabled` to true.
2017-10-06 18:52:22 -04:00
Vinicius Carvalho
00b8f1aba1 Update to BOOT 2.0.0.BUILD-SNAPSHOT
- Fixes #1088
- Updates Actuator endpoints
- Removing metrics dependencies for now, we will revisit metrics and
  stream on a separate PR
2017-10-05 16:50:45 -04:00
Vinicius Carvalho
41d51144ef Fixing boot 2.0 tests lost on merge 2017-10-05 10:58:08 -04:00
Artem Bilan
a9d7257c80 GH-1083: Disallow reuse bean name for bindings
Resolves spring-cloud/spring-cloud-stream#1083

By default Spring Framework allows beans overriding via the same name.
The binding target definitions (`@Input` and `@Output`) populate beans as well
and when we use the same name for target we end up with unexpected behavior
but without errors.
Since it isn't so obvious via Spring Framework bean definition DSLs
(XML or Java & Annotations) how to override beans with the same name,
that is absolutely easy to use the same value for `@Input` and `@Output`
definitions even in different binding interfaces.
That's hard to analyze fro the target application since mostly
`@Input` and `@Output` produce `MessageChannel` beans.

* Fail fast with the `BeanDefinitionStoreException` when we meet existing
bean definition for the same name
* Add JavaDocs to the `@Input` and `@Output` to explain that their `value`
is a bean name, as well as destination by default

Since `@EnableBinding` is `@Inherited`, the inheritor picks up it from the
super class during configuration class parsing.
The parsing process logic is such that after the root class we go to parse its
super classes, and therefore come back to the `@EnableBinding` again.
In this case we process all the `@Import`s one more time and collect them to
the root `configurationClass`.
Essentially we get a duplication for the `ImportBeanDefinitionRegistrar`s
such as `BindingBeansRegistrar`.
The last one parsed `@EnableBinding` and registers appropriate beans for the
`@Input` and `@Output`, as well as for the binding interface - `BindableProxyFactory`.
But since we have it twice in the `configurationClass` we end up with
`BeanDefinitionStoreException` mentioned before.
That's how Spring Framework works with inheritance for configuration classes
and that's may be why it allows to override beans by default

* Skip parsing `@EnableBinding` one more time if the bean definition for
binding interface is already present in the `registry`
* Fix `AggregateWithMainTest` do not process `@ComponentScan` what causes
picking up the configuration classes for children contexts in the aggregation
* Fix `testBindableProxyFactoryCaching()` do not register `Source` and `Processor`
in the same application context because both of them cause registration for the
`Source.OUTPUT` bean

Conflicts:
	spring-cloud-stream/src/test/java/org/springframework/cloud/stream/aggregation/AggregationTest.java
2017-10-05 10:35:51 -04:00
Vinicius Carvalho
1b1dbe57d9 Fixes issue with originalContentTypeHeader
- Fixes #1072
- Added a new ContentTypeResolver that searches for originalContentType
  as well as contentType headers

Conflicts:
	spring-cloud-stream-schema/src/main/java/org/springframework/cloud/stream/schema/avro/AvroMessageConverterAutoConfiguration.java
2017-10-05 10:35:51 -04:00
Vinicius Carvalho
171f034a8c Content type redesign
Fixes #992, #1050, #1051, #1052

Adding custom jackson converter with some tests
Adds kryo message converter to replace codec
Checkstyle changes

Removing codec support
- Removed codec dependency from AbstractBinder
- MessageSerializationUtils is almost an empty shell for now, just to
  keep code compiling until we get EmbeddedHeaders interceptors
- Updated Kryo tests

Removing codec module from build
Added a new Annotation for custom converters '@StreamConverter'
Fixed some tests with new expected behavior
Moved broken tests to a temporary package to keep track of progress
Fixed KryoConverter to fail based on headers
Fixed a couple of more tests

Making converters strict to only convert their corresponding contentType
Bypassing conversion for ErrorMessages

* Configuring SI ConfigurableCompositeMessageConverter
 - Moved ContentType related beans into separate configuration
 - Configured SI ConfigurableCompositeMessageConverter to use same
   converters as Stream does (for ServiceActivator)
- TupleConverter should return byte[] as all other converters
- Fixed tests

* Fixes tests
 - Revert to Boot 2.0.0.M3. Snapshots breaking actuator
 - Checkstyle fixes
 - Disable JsonUnmarshalling as a catch all converter

Fixing Schema tests
Fixing Metrics tests
Fixing reactive tests
applying checkstyle fixes

 * Adding new content type tests
 - Fixed ContentTypeInterceptor misusage of default mimeType

Changing contentType doc section
Improving doc section
Last minute polish
Fixing BinderTests to use bytes to compare messages
Applied changes to Base Binders test to use the new contentType handling mechanism
PR review fixes

Renaming StreamConverter -> StreamMessageConverter
2017-10-05 10:35:51 -04:00
Vinicius Carvalho
24cf992301 Passing Conversion service to boot Binder 2017-10-05 10:35:14 -04:00
Vinicius Carvalho
d317638bef Minor fixes
- Upgraded versions of reactor and SI in poms
- added .jdk8 to enable java-8 in build
- fixed changes in mockito api
2017-10-05 10:35:14 -04:00
Marius Bogoevici
57d3ba3317 Set version to 2.0.0.BUILD-SNAPSHOT 2017-10-05 10:33:57 -04:00
Marius Bogoevici
0546c8ed7e Release 2.0.0.M1 2017-10-05 10:33:57 -04:00
Marius Bogoevici
ecfe94e7c5 Use Spring Boot version from Spring Cloud Build 2017-10-05 10:31:09 -04:00
Marius Bogoevici
7260eaacab Use Reactor version managed by Boot 2017-10-05 10:31:09 -04:00
Marius Bogoevici
58b8352d19 Remove SI Java DSL version (now part of SI 5) 2017-10-05 10:31:09 -04:00
Marius Bogoevici
ffab56b584 Add the ability to customize the Spring Boot version
Set Spring Boot version to 2.0.0.BUILD-SNAPSHOT
2017-10-05 10:31:09 -04:00
Janne Valkealahti
4f72a10bc6 Upgrade build to boot 2.x
- Pump up version to 2.0.0
- Some generic polish
- All changes around breakage with boot 2.x
- Some boot classes has been moved around
- You can't no longer have binding key ending with
  camelCase.
- New Binder now has illegal keys.
- Some changes to tests as we can directly do end-to-end
  testing with ENV_VAR_FORMAT as normal keys
- Spring data repo changes as now uses Optional
- Remove relaxed binder and its tests in favor of new Binder
- Some mockito api changes
- One Ingored test TextPlainToJsonConversionTest.testTextPlainToJsonConversionOnInput
- Relates to #935

Cache metric export properties

Add code formatting guidelines

Rearranged files
2017-10-05 10:31:09 -04:00
Gary Russell
e2c214b34e Update POMs to 1.3.1.BUILD-SNAPSHOT 2017-09-29 16:17:43 -04:00
Gary Russell
9e46ec00a0 Versions to 1.3.0.RELEASE, Bismuth-RELEASE 2017-09-29 11:31:06 -04:00
Artem Bilan
4c33e5eb58 GH-1083: Disallow reuse bean name for bindings
Resolves spring-cloud/spring-cloud-stream#1083

By default Spring Framework allows beans overriding via the same name.
The binding target definitions (`@Input` and `@Output`) populate beans as well
and when we use the same name for target we end up with unexpected behavior
but without errors.
Since it isn't so obvious via Spring Framework bean definition DSLs
(XML or Java & Annotations) how to override beans with the same name,
that is absolutely easy to use the same value for `@Input` and `@Output`
definitions even in different binding interfaces.
That's hard to analyze fro the target application since mostly
`@Input` and `@Output` produce `MessageChannel` beans.

* Fail fast with the `BeanDefinitionStoreException` when we meet existing
bean definition for the same name
* Add JavaDocs to the `@Input` and `@Output` to explain that their `value`
is a bean name, as well as destination by default

Since `@EnableBinding` is `@Inherited`, the inheritor picks up it from the
super class during configuration class parsing.
The parsing process logic is such that after the root class we go to parse its
super classes, and therefore come back to the `@EnableBinding` again.
In this case we process all the `@Import`s one more time and collect them to
the root `configurationClass`.
Essentially we get a duplication for the `ImportBeanDefinitionRegistrar`s
such as `BindingBeansRegistrar`.
The last one parsed `@EnableBinding` and registers appropriate beans for the
`@Input` and `@Output`, as well as for the binding interface - `BindableProxyFactory`.
But since we have it twice in the `configurationClass` we end up with
`BeanDefinitionStoreException` mentioned before.
That's how Spring Framework works with inheritance for configuration classes
and that's may be why it allows to override beans by default

* Skip parsing `@EnableBinding` one more time if the bean definition for
binding interface is already present in the `registry`
* Fix `AggregateWithMainTest` do not process `@ComponentScan` what causes
picking up the configuration classes for children contexts in the aggregation
* Fix `testBindableProxyFactoryCaching()` do not register `Source` and `Processor`
in the same application context because both of them cause registration for the
`Source.OUTPUT` bean
2017-09-28 13:41:18 -04:00
Soby Chacko
9fc51cb6e2 GH-1080: Fix StreamListener methods for proxies
Fix #1080

When class with `@StreamListener` method is proxied we end up
with the double target subscribers registration because we meet
the same method multiple times during `ReflectionUtils.doWithMethods()`

* Use `ReflectionUtils.getUniqueDeclaredMethods()` instead to extract
the list of method candidates

**Cherry-pick to 1.2.x**
2017-09-27 14:04:36 -04:00