Commit Graph

855 Commits

Author SHA1 Message Date
Oleg Zhurakousky
d5bfddc7fc polishing
Resolves #1190
2018-01-24 09:58:29 -05:00
Gary Russell
cef7f0b70c GH-1186: Polled Consumer Documentation
Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/1186
2018-01-24 09:55:51 -05:00
Gary Russell
e040cf02b4 Pollable Consumer : Start Source if Lifecycle
When binding a `MessageSource`, start it if it implements `Lifecycle`.

Resolves #1189
2018-01-23 13:48:45 -05:00
Soby Chacko
c1749ca866 New abstraction for StreamListener method setup invoker.
StreamListenerMethodSetupOrchestrator is an API hook that allows
downstram binder implementations or applications to inject custom
strategies to alter the default StreamListener adapter method invocations.

This PR primarily focuses on allowing customizations on the oubound side.
For the inbound, StreamListenerMethodSetupOrchestrator interface provides
a default implementation.

Corresponding refactoring in StreamListenerAnnotationBeanPostProcessor.
By default StreamListenerAnnotationBeanPostProcessor will use the current
strategies used for invoking the StreamListener adapters. If beans are provided
for orchestration, they take precedence however.

Test changes.

Further polishing in StreamListenerAnnotationBeanPostProcessor.

Resolves #1177
Resolves #1179
2018-01-18 11:59:43 -05:00
Oleg Zhurakousky
9ef1181f7a Merge pull request #1184 from garyrussell/docFix
Various Doc Fixes
2018-01-16 15:45:45 -05:00
Gary Russell
cb25d7d369 Various Doc Fixes
- missing `}` in one code snippet caused syntax-highlighting problems in asciidoc-aware editors such as atom
- use spaces, not tabs for code snippets
- Change markdown to asciidoc for one snippet
2018-01-16 15:34:44 -05:00
Oleg Zhurakousky
00748985d6 Support @Beans for Partitioning Properties
Fixed support for partition properties to be Spring configured

- Fixed support for 'partitionKeyExtractor' and 'partitionSelector' to be Spring configured
- Added new producer properties 'partitionKeyExtractorName' and 'partitionSelectorName'
- Deprecated 'partitionKeyExtractorClass' and 'partitionSelectorClass' properties
- Removed InitilaizingBean from MessageConverterConfigurer
- Updated documentation
- Added additional tests
- Removed PartitionedProducerTest  as redundant
- Fixed unrelated to this effort BinderPropertiesTests due to recent Boot changes

polishing
2018-01-16 15:24:16 -05:00
Oleg Zhurakousky
b3d5c4f518 Simplified contentType negotiation
In preparation to contentType TCK, simplified InboundContentTypeConvertingInterceptor's logic
2018-01-15 14:49:24 -05:00
Gary Russell
7267661d2e GH-1174: Add Pollable Consumer
Resolves: https://github.com/spring-cloud/spring-cloud-stream/issues/1174

Introduce `PollableMessageSource` for polled consumers.

The `@Input` annotation can now be applied to interface methods returning
`PollableMessageSource` and binders that can support polled consumers
will wire up an implementation `DefaultPollableMessageSource` which can
then be `poll()`ed.

The `poll()` method takes a `MessageHandler` callback to handle any message
returned by the poll; the method returns true if a message was found and
handled, false otherwise.

By default, acknowledging the message is deferred until the message handler
returns but that auto-acknowledging can be disabled if the user wishes to
defer the acknowledgment until later. Refer to the Spring Integration
documentation for more information (internally, a `MessageSourcePollingTemplate`
is used, which performs the ack/nack when the handler exits).

Usage:

```java
public interface PolledConsumer extends Processor {

	@Input("pollableSource")
	PollableMessageSource pollableSource();

}
```

and

```java
@Bean
public ApplicationRunner runner(PollableMessageSource pollableSource) {
	return args -> pollableSource.poll(message -> {
		System.out.println("Polled payload: " + message.getPayload());
	});
}
```

Polishing

Fix test; add support to test binder.

Add error channel handling and retry

More polishing driven by Rabbit implementation

- can't use the MessageSourcePollingTemplate within retry since it will fetch more messages
- populate the retry context with data for the error message strategy

Polishing - PR Comments

Fix NPE.

Revert finally in DefaultPollableMessageSource

Catch Throwable - JUnit throws Errors

Change method name to reflect it's only used for polled consumers.

Fix errors when no retry

Remove stack trace print

Fix error message strategy context with no retry

Add separate error MessageHandler for polled consumers.

Resolves #1174
Resolves #1176
2018-01-13 11:44:04 -05:00
Artem Bilan
8513e80c2a Replace propAccessorInjector with the SpelPAReg
After changing the `propertyAccessorInjector` `BeanPostProcessor`
just to a simple `InitializingBean` there is no guarantee which bean
will be accessed from the autowire phase.
There are some cases like `@PostConstruct` on the `@Configuration`
wich may be called before this new `propertyAccessorInjector` bean
and we end up with `EvaluationContext` not supplied with additional
`PropertyAccessors`

* For proper `IntegrationEvaluationContextFactoryBean` initialization
add `SpelPropertyAccessorRegistrar` for custom `PA` configuration
* Remove `propertyAccessorInjector` bean altogether
* Resolve deprecation in the `BindingServiceConfiguration`
* Modify `SpelExpressionConverterConfigurationTests` to confirm that
even `@PostConstruct` on `@Configuration` works as expected

Resolves #1178
2018-01-11 20:32:00 -05:00
Oleg Zhurakousky
b0356bdea9 1123 made CompositeMessageChannelConfigurer conditional
Annotated CompositeMessageChannelConfigurer wih @ConditionalOnMissingBean to allow it to be overriden

Resolves #1123
2018-01-10 22:30:45 -05:00
Oleg Zhurakousky
a8dfb2f833 GH-1135 Fixed argument resolvers consistency
- added SI provided argumentResolvers to MessageHandlerMethodFactory configuration to ensure that StreamListener annotated methods are consistent with equivalent SI config annotations (e.g., ServiceActivator)

Resolves #1135
2018-01-04 20:58:35 -05:00
Oleg Zhurakousky
71c295ef09 Removed PostProcessorConfiguration from BindingServiceConfiguration
- substituted BPPs for InitializingBeans removing the possibility of early initialization.

Resolves #1172
2018-01-04 13:53:56 -05:00
Oleg Zhurakousky
c76c29f249 Aligned StreamEmitterAnnotationBeanPostProcessor with StreamListenerAnnotationBeanPostProcessor
. . .as part of the effort to fix early initialization of beans inside BeanPostProcessors.

Resolves #1157
2018-01-03 21:46:40 -05:00
Oleg Zhurakousky
87bf2734e7 polishing BinderAwareRouterBeanPostProcessor
- removed BeanPostProcessor from BinderAwareRouterBeanPostProcessor
- simplified BinderAwareRouterBeanPostProcessor configuration in BindingServiceConfiguration

Resolves #1171
Resolves #1170
2018-01-03 21:07:35 -05:00
Artem Bilan
502bddd4e9 GH-1170: Fix early BeanFactory initialization
Fixes: spring-cloud/spring-cloud-stream#1170

The `messageHandlerHeaderPropagationBeanPostProcessor` causes early
`BeanFactory` initialization via its `SpringIntegrationProperties`
dependency.
This triggers an initialization all the `@ConfigurationProperties`
infrastructure in Spring Boot, when we might not have all the necessary
beans, e.g. `Converter` for some configuration property

* Eliminate the "early `BeanFactory` initialization syndrome" through
the `SmartInitializingSingleton`

**Cherry-pick to 1.3.x**
2018-01-03 14:07:30 -05:00
Artem Bilan
7d2e3a3815 Fix AbstractBinderTests to populate properties
Some tests in `AbstractBinderTests` are rely on the partition header,
but that one can be populated only by the `PartitionInterceptor`.
To allow it we need to provide appropriate producer properties, like
`partitionKeyExpression`.

* Populate test-based `ProducerProperties` to the
`createBindableChannel()` where `PartitionInterceptor` is populated
* Fix `checkstyle.xml` static imports rule for Mockito-2.x as the
current base line

Fix for the proper `BindingProperties` usage
2017-12-21 18:49:41 -05:00
Artem Bilan
6d22d039dd GH-1159 Extract EmbeddedHeaders first on consumer
Fixes: spring-cloud/spring-cloud-stream#1159

* Extract `EmbeddedHeadersChannelInterceptor` class for better navigation
* Use only one shared instance of this interceptor since it is stateless
* Add this `EmbeddedHeadersChannelInterceptor` in the position `0` to
extract embedded headers first

The fix confirmed by Kafka and Kinesis binders
2017-12-21 17:40:16 -05:00
Oleg Zhurakousky
80fee48b4a 1146 Fixed early initialization in StreamListenerAnnotationBeanPostProcessor
Removed autowiring from StreamListenerAnnotationBeanPostProcessor in favor of late-binding callbacks
Fixed tests

Resolves #1146
2017-12-13 11:14:45 -05:00
Oleg Zhurakousky
f59c85842b General cleanup of StreamListenerAnnotationBeanPostProcessor
- Removed BeanFactoryAware dependency
- Simplified `isDeclarativeMethodParameter` and related operations
2017-12-12 14:37:36 -05:00
Oleg Zhurakousky
7489c382ce General cleanup
- Deprecated BindableAdapter in favor of providing default operations in Bindable
- Removed 'postProcessBeforeInitialization(..)' from 'BinderAwareRouterBeanPostProcessor' as it is no longer required
2017-11-28 22:32:39 -05:00
Oleg Zhurakousky
7b5bb2f463 StreamListenerAnnotationBeanPostProcessor refactoring/polishing
Continue the work started in f53ff9585a
- simplified 'isDeclarativeMethodParameter(..) operation of StreamListenerAnnotationBeanPostProcessor
- aligned MessageChannelToInputFluxParameterAdapter and MessageChannelToInputObservableParameterAdapter around the actual Message
2017-11-28 14:58:15 -05:00
Oleg Zhurakousky
593b8c38ca Fixed ClassLoader discovery logic in BinderFactoryConfiguration 2017-11-28 10:07:07 -05:00
Oleg Zhurakousky
f53ff9585a 1146 Initial fix for eager initialization in StreamListenerAnnotationBPP
- Removed dependency and callbacks to 'BeanFactoryUtils.beansOfTypeIncludingAncestors' in
    StreamListenerAnnotationBeanPostProcessor in favor of more appropriate and lazy DI
    mechanism provided by Spring.
  - Polishing
2017-11-27 22:26:32 -05:00
Oleg Zhurakousky
5b6a5b38f4 Polishing DefaultBinderFactory
- Removed BinderInstanceHolder in favor of Entry
- Removed unnecessary List creation for 'configurationClasses'
- other minor polishing
2017-11-27 21:44:55 -05:00
Oleg Zhurakousky
acc4ecfccb polishing 2017-11-26 22:26:28 -05:00
Oleg Zhurakousky
2bac583d81 Refactoring and polishing
- Refactored Input/OutputBindingLifecycle to ensure they properly use Spring's dependency injection mechanisms
    - Refactored additional tests to use new SI-backed binder

polishing
2017-11-26 09:39:11 -05:00
Oleg Zhurakousky
5a87211237 Fixed warnings in spring-cloud-stream module
- Removed/modified unnecessary @SupressWarning annotations
 - Removed deprecated configuration for SharedChannelRegistry and fixed associated tests
 - Fixed and documehted other deprecations
2017-11-22 14:22:58 -05:00
Oleg Zhurakousky
c94f946c85 1136 Fixed type expectations in BinderProperties
- changed BinderProperties environment to be Map to support  appropriate semantics where keys and values may be non-Strings
- deprecated `setEnvironment(Properties environment)` operation

Resolves #1136

polishing
2017-11-22 12:25:19 -05:00
Gary Russell
54abf564fd KGH-12: Late binding for kafka etc
Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/12

Support late binding for binders that don't inherently support it.

Schedule attempts at 30 second intervals (default, configurable).

`IllegalState` and `IllegalArgument` exceptions are fatal.

Add test cases (mocks) to verify rebind after failure

Docs

Polishing - always use the setter on LateBinding

Remove volatile from the fields since they are only accessed from synchronized methods.
2017-11-22 10:50:34 -05:00
Oleg Zhurakousky
21a2a84d86 GH-1127: Add stub Binder backed by SI
Resolves: spring-cloud/spring-cloud-stream#1127

- Added initial implementation of SI-backed stub binder to facilitate
 more consistent testing
- Improved BinderAwareChannelResolverTests and ExtendedPropertiesBinderAwareChannelResolverTests
to make use of it
- Added sample Application that demonstrates the binder usage

Addressed PR comments
2017-11-15 09:49:02 -05:00
Oleg Zhurakousky
024e3cffcc Finished consolidating contentType Message conversion
- Removed MessageSerializationUtils
    - Consolidated Message contentType conversion refactoring that was started with d0be34f7cb commit

At this point Message conversion is consolidated in either MessageConverters or In/Out channel interceptors configured in MessageConverterConfigurer
2017-11-14 22:06:44 -05:00
Oleg Zhurakousky
464a98aa10 polishing
Resolves #1108
2017-11-14 10:29:29 -05:00
David Kalosi
08761c3de0 subject naming strategy POC
checkstyle fix

more checkstyle fixes
2017-11-14 10:23:41 -05:00
Oleg Zhurakousky
d0be34f7cb Refactored MessageConverterConfigurer
- Separated InboundContentTypeConvertingInterceptor and OutboundContentTypeConvertingInterceptor to isolate logic that needs to be performed by such interceptors for *inbound* and *outbound* messages.
- Documented their purpose via javadocs
- Moved InboundMessageConvertingInterceptor to TestSupportBinder (for now) to only support MessageCollector's 1.3 behavior
2017-11-14 10:12:43 -05:00
Gary Russell
83fa5f3858 GH-1009: Producer properties for dynamic bindings
Resolves https://github.com/spring-cloud/spring-cloud-stream/issues/1009

Provide a callback mechanism to set properties on dynamically created bindings.

Polishing - callback name.

Resolves #1009
Resolves #1132
2017-11-14 09:25:27 -05:00
Soby Chacko
73f0fc38a6 Relax Consumer channel binding in AbstractTestBinder
When native decoding is enabled on the consumer, relax the requirement
for using AbstractBinderTest.createBindableChannel(..)

Fixes #1137
Resolves #1138
2017-11-14 08:52:01 -05:00
Oleg Zhurakousky
2275d40d7d polishing
Resolves #1133
2017-11-12 17:07:11 -05:00
Soby Chacko
53df5f5747 Native endoding/decoding changes for interceptors
Fixes #1109

- Disable outbound content type interceptor when nativeEndoding is enabled on the producer
- Disable inbound content type interceptor when nativeDecoding is enabled on the consumer

- Add a new flag in ConsumerProperties for useNativeDecoding

- Apply ContentType interceptor before the InboundInterceptor to ensure
  that content type set on the channel is applied on the message if content type
  is missing on the Message.

- Add tests for verifying interceptors are not applied when native encoding/decoding is enabled
- Fix a kryo integration test
- Remove Inbound interceptor from MessageCollector when native encoding is enabled on the producer
- Polishing
2017-11-12 17:01:20 -05:00
Gary Russell
b865f3cbe9 RGH-83: Allow binders to add interceptors
Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-rabbit/issues/83

The RabbitMQ binder needs to add an interceptor to evaluate expressions (if they
include `payload`) before the payload is serialized.

Add a hook to allow the binder to post-process the message channel.

Resolves #1134
2017-11-12 16:15:21 -05:00
Jacob Severson
61240fd0fa Adding hook for changing message target delimeter
Resolves #1120
2017-11-12 15:46:29 -05:00
Soby Chacko
9cc0b83a38 Update to next version: 2.0.0.BUILD-SNAPSHOT
Update spring-cloud-build parent to 2.0.0.BUILD-SNAPSHOT
2017-11-09 09:02:40 -05:00
Soby Chacko
03b7b0d612 Update to release version: 2.0.0.M3
spring-cloud-build parent to 2.0.0.M4
2017-11-09 08:50:15 -05:00
Oleg Zhurakousky
4e69ead7b4 GH-1126 Fixed default app port in Tests
Ensured AggregateApplicationTests, AggregateWithMainTest and AggregationTest arn't using default 8080 port
2017-11-08 13:10:47 -05:00
Gary Russell
b9884ae71a Add comment to embedded header decode error log 2017-11-08 10:18:01 -05:00
Oleg Zhurakousky
62c8513197 Initial cleanup of AbstractMessageChannelBinder
Removed extra handler for embeddedHeaders in favor of ChannelInterceptor to avoid additional dispatch

Addressed PR comments
2017-11-08 10:11:51 -05:00
Oleg Zhurakousky
a7fdf6dbb2 GH-1106 Addressed backward/forward contentType compatibility issues
Fixes #1106

- This PR builds on the previous PR with commit hash 6c259be6... (pr/1112)
- Added support in AbstractBinderTests to create bindable channel based on determining channel input type based on it's name (i.e., *input*)
- Added LegacyContentTypeHeaderInterceptor to TestSupportBinder to restor the previous behavior of MessageCollector for cases
  where Message's payload content type is a variant of 'text'.
- Restored tests that use MessageCollector to depend on proper payload type
- Added 'deserialize' routine back to MessageSerializationUtils
- Polished MessageConverterConfigurer.LegacyContentTypeHeaderInterceptor to remove conditional original-content-type header logic
- Removed default contentType from BindingProperties
- Restored tests that use MessageCollector to depend on proper payload type
- polishing
- fixed Kryo/Java serialization
- Fixed ser/de for "application/json" and contentType equals
- Fixed of ser/de of JSON strings to ensure that Strings are not re-quoted
- Fixed how we comparing contentTypes
- more polishing
- Fixed NPE in MessageSerializationUtils
- Make bindings and consumer groups in new tests added to AbstractBinderTests mutually exclusive
2017-11-07 11:21:13 -05:00
Soby Chacko
6c259be62b Legacy cotent-type changes for the consumer
- Remove binding config property `legacyContentTypeHeaderEnabled` that was
  introduced to enable legacy content type handling
- Enable LegacyContentTypeInterceptor in 2.0 always, but bypass any
  legacy content type handling if the message received is from
  a 2.0 producer by checking on the version header
- Introdce a new BinderHeader property for version
- Fix the LegacyContentType related tests
- Remove the check for originalContentType in ReceivingHandler when
  the payload received is a byte[]
- Remove unnecessary deserializePayloadIfNecessary calls in ReceivingHandler
- Remove deprecated deserializePayload methods in AbstractBinder
  and MessageSerializationUtils

Partly fixes #1106
Fixes #1110
2017-10-26 13:05:10 -04:00
Oleg Zhurakousky
4cc66ae0e0 fixed AbstractBinderTests to take into account previous commit '6eff89ca6c4dfc6ae9e0777e5f97e63c45615afc' 2017-10-26 10:59:23 -04:00
Oleg Zhurakousky
6eff89ca6c Added MessageChannel configuration assertion (#1113)
Added assertion to org.springframework.cloud.stream.binder.AbstractTestBinder to ensure that MessageChannels that are passed to any of the bind methods are configured with Message Interceptors
2017-10-26 10:16:57 -04:00