Commit Graph

42 Commits

Author SHA1 Message Date
Oleg Zhurakousky
f566cb9749 Polishing GH-1517
Resolves #1517
Resolves #1519
2018-11-01 13:11:39 +01:00
Sercan Karaoglu
829604ae32 GH-1517 Avro Union Types import schema issue when we have multiple avro files 2018-11-01 12:58:21 +01:00
Vitali Zarembouski
7e6864e4fc support schema-type union
Resolves #1466
2018-10-24 10:58:12 +02:00
Soby Chacko
c85660fba2 Spring Boot 2.1 upgrade
* Removed MockBinderRegistryConfiguration
* Move necessary beans from `BindingServiceConfiguration` which is now auto configurable into
  `BinderFactoryConfiguration` that is invoked by `EnableBinding` in order to avoid certain
  cyclic dependency issues. The beans moved are binding target factories of type
  `MessageSourceBindingTargetFactory` and `SubscribableChannelBindingTargetFactory` and their dependencies.
  As a side effect, `ContentTypeConfiguration` is also brought back at the `EnableBinding` level through
  `BinderFactoryConfiguration`.
* Restore `ServerController`  bean in SchemaServerConfiguation as the removal of it
  earlier introduced  some connectivity issues with the schema registry server.
* Update copyrights
2018-08-14 13:42:02 +02:00
Soby Chacko
ae445e73ac Spring Boot 2.1 upgrade related changes
* Instead of redefining `BinderTypeRegistry` as a bean in several tests, properly use spring.binders
  to define mock binders and then choose a default binder in tests in case of multiple binders in
  same spring.binders file.
* Remove `BindingServiceConfiguration` in `EnableBinding` and introduce it as a proper
  Spring Boot autoconfiguration class. Added `BindingServiceConfiguration` to spring.factories.
* Move `BinderFactory` bean into `BindingServiceConfiguration` and add `ConditionalOnMissingBean`
  on it so that downstream users can define new `BinderFactory` beans as part of autoconfiguration.
* Remove `ConditionalOnMissingBean` from the `BinderTypeRegistry` bean in `BinderFactoryConfiguration`
  as we don't expect this bean to be overridden.
* Remove previously added property `spring.main.allow-bean-definition-overriding` in several tests.
* Since web/actuator is optional now, remove unncecessarily setting server.port to `0` in tests
* Ensure that `BindersHealthIndicatorAutoConfiguration` is autoconfigured after `BindingServiceConfiguration`
  so that it has a `BinderFactory` available.
* Remove redefining `ServerController` bean in `SchemaServerConfiguration` as this is already created through
  component scanning and causing the bean overriding exceptions.
* Tests cleanup and polishing.

Resolves #1429, #1430
2018-08-10 11:27:11 -04:00
Soby Chacko
cb105ab0ec Upgrade Spring Cloud Build to 2.1.0
This upgrades to Spring Boot 2.1.0
Fix tests where the beans need to get overridden
Minor cleaning up
Polishing

Resolves #1426
2018-08-07 19:47:13 +02:00
Oleg Zhurakousky
9f3f49eff0 Revert "Prevent SpringBootApplication starting to overlap with user defined Application."
This reverts commit 66b7d8dbe4.
2018-06-18 13:44:33 -04:00
Guilherme Blanco
66b7d8dbe4 Prevent SpringBootApplication starting to overlap with user defined Application.
Fixes #1391
Resolves #1393
2018-06-15 14:20:46 -04:00
Oleg Zhurakousky
b8711066e2 GH-1294 Fixed NPE in AvroSchemaRegistryClientMessageConverter
The NPE was a result of not following defensive programming practices and that is fixed.
Also, we already support both reader and writer schema being null where new ReflectDatumReader(type) is used in getDatumReader(..)  operation.
That said there may still be conditions which are not supported and that is okay since in the end it's a MessageConverter and as such it is one
of the extension points of the framework allowing user to provide a custom one follwing instructions in 'User-defined Message Converters' of the reference guide.

Resolves #1294
2018-04-02 16:47:46 -04:00
Soby Chacko
ae74c21720 Fixing compiler warnings
Fixing unchecked and deprecation warnings
Polishing

Resolves #1318
2018-03-21 14:55:22 -04:00
Soby Chacko
3d76fb40a2 Add ConditionalOnMissingBean on SchemaRegistryClient configuration
Resolves #1307
Resolves #1308
2018-03-16 17:38:31 -04:00
Gary Russell
12bd608908 Fix more trailing whitespace (tabs) 2018-03-12 10:26:46 -04:00
Gary Russell
e1fc583d1e Fix all trailing whitespace 2018-03-12 09:56:32 -04:00
jmaxwell
8876a0869b GH-1292 Remove Unnecessary ByteBuffer
Resolves #1292
Resolves #1293
2018-03-10 13:31:14 -05:00
Oleg Zhurakousky
09f4c8118f GH-1228 Type resolvers cleanup
- moved OriginalContentTypeResolver to avro module
- removed StringConvertingContentTypeResolver as it is no longer referenced anywhere
- deprecated KryoMessageConverter

Resolves #1228
2018-02-17 16:35:21 -05:00
jmaxwell
275c212d47 1217 Disable Pretty Print
Resolves #1224
2018-02-14 13:44:38 -05:00
Oleg Zhurakousky
21d1dbea83 Polishing DefaultSchemaRegistryClient
- made RestTemplate final
- removed setter

Resolves #1222 #1223
2018-02-14 10:40:10 -05:00
jmaxwell
c2b3194fb9 1222 DefaultSchemaRegistryClient should accept a custom RestTemplate.
Added protected accessors to allow subclasses access to RestTemplate and endpoint fields.
2018-02-14 10:21:17 -05:00
Oleg Zhurakousky
d8a9471012 Polishing AvroSchemaRegistry stuff
- removed deprecated constructor from AvroSchemaRegistryClientMessageConverter
- fixed test
- formatting

Resolves #1221 #1218
2018-02-14 10:15:17 -05:00
jmaxwell
f5d625e9de 1218 Inconsistent use of AvroSchemaRegistryClientMessageConverter.AVRO_FORMAT 2018-02-14 09:50:53 -05:00
jmaxwell
a160cd7ea8 1215 CachingCachingRegistryClient stores data by wrong keys resulting in fetch operations always missing.
Resolves #1216
2018-02-13 20:37:42 -05:00
Oleg Zhurakousky
9672a5b4df Fixed double conversion issue
- removed custom conversion from In/Out interceptors delegating everything to the available  MessageConverters
- added initial version of content-type TCK to validate various content-type conversion scenarious
- added initial content-type conversion matrix to the WIKI https://github.com/spring-cloud/spring-cloud-stream/wiki/Content-type-conversion-matrix

Resolves #1130
Resolves #1071
2018-02-03 07:33:27 -05:00
jmaxwell
6a23cf95f5 1202 Inconsistent use of this.prefix in AvroSchemaRegistryClientMessageConverter.resolveSchemaForWriting
Resolves #1202
Resolves #1203
2018-02-01 08:37:13 -06:00
Oleg Zhurakousky
464a98aa10 polishing
Resolves #1108
2017-11-14 10:29:29 -05:00
David Kalosi
08761c3de0 subject naming strategy POC
checkstyle fix

more checkstyle fixes
2017-11-14 10:23:41 -05:00
Vinicius Carvalho
1b1dbe57d9 Fixes issue with originalContentTypeHeader
- Fixes #1072
- Added a new ContentTypeResolver that searches for originalContentType
  as well as contentType headers

Conflicts:
	spring-cloud-stream-schema/src/main/java/org/springframework/cloud/stream/schema/avro/AvroMessageConverterAutoConfiguration.java
2017-10-05 10:35:51 -04:00
Vinicius Carvalho
171f034a8c Content type redesign
Fixes #992, #1050, #1051, #1052

Adding custom jackson converter with some tests
Adds kryo message converter to replace codec
Checkstyle changes

Removing codec support
- Removed codec dependency from AbstractBinder
- MessageSerializationUtils is almost an empty shell for now, just to
  keep code compiling until we get EmbeddedHeaders interceptors
- Updated Kryo tests

Removing codec module from build
Added a new Annotation for custom converters '@StreamConverter'
Fixed some tests with new expected behavior
Moved broken tests to a temporary package to keep track of progress
Fixed KryoConverter to fail based on headers
Fixed a couple of more tests

Making converters strict to only convert their corresponding contentType
Bypassing conversion for ErrorMessages

* Configuring SI ConfigurableCompositeMessageConverter
 - Moved ContentType related beans into separate configuration
 - Configured SI ConfigurableCompositeMessageConverter to use same
   converters as Stream does (for ServiceActivator)
- TupleConverter should return byte[] as all other converters
- Fixed tests

* Fixes tests
 - Revert to Boot 2.0.0.M3. Snapshots breaking actuator
 - Checkstyle fixes
 - Disable JsonUnmarshalling as a catch all converter

Fixing Schema tests
Fixing Metrics tests
Fixing reactive tests
applying checkstyle fixes

 * Adding new content type tests
 - Fixed ContentTypeInterceptor misusage of default mimeType

Changing contentType doc section
Improving doc section
Last minute polish
Fixing BinderTests to use bytes to compare messages
Applied changes to Base Binders test to use the new contentType handling mechanism
PR review fixes

Renaming StreamConverter -> StreamMessageConverter
2017-10-05 10:35:51 -04:00
Janne Valkealahti
4f72a10bc6 Upgrade build to boot 2.x
- Pump up version to 2.0.0
- Some generic polish
- All changes around breakage with boot 2.x
- Some boot classes has been moved around
- You can't no longer have binding key ending with
  camelCase.
- New Binder now has illegal keys.
- Some changes to tests as we can directly do end-to-end
  testing with ENV_VAR_FORMAT as normal keys
- Spring data repo changes as now uses Optional
- Remove relaxed binder and its tests in favor of new Binder
- Some mockito api changes
- One Ingored test TextPlainToJsonConversionTest.testTextPlainToJsonConversionOnInput
- Relates to #935

Cache metric export properties

Add code formatting guidelines

Rearranged files
2017-10-05 10:31:09 -04:00
Vinicius Carvalho
11b21d9eec Fixes issue with originalContentTypeHeader
- Fixes #1072
- Added a new ContentTypeResolver that searches for originalContentType
  as well as contentType headers
2017-09-11 09:55:07 -04:00
Vinicius Carvalho
007594cafd Fixes issue with originalContentType 2017-08-24 20:25:21 -04:00
Ilayaperumal Gopinathan
b10a014273 Minor refactoring 2017-07-17 07:51:31 +05:30
Vinicius Carvalho
661f8494a3 Fix Confluent Schema Registry registration
Changing internal registration logic

Adding some tests

Resolves #985 #1014
2017-07-17 07:51:31 +05:30
Jon Archer
b1285c3d4a Extract schema version not id in register method.
Fix #965

Original implementation was getting the schema id and using it as the
version. This made deserialization fail since it would endeavor to
retrieve the schema by version using the id value.
2017-05-15 22:01:29 -04:00
Ilayaperumal Gopinathan
bd002e4aaf Add code formatting guidelines
Add 'eclipse' folder containing Eclipse code
formatter configuration and instructions how to use
 it.

Update rule for join_wrapped_lines

 - Set to `false`

Resolves #930

Update README

Address review comments
2017-05-15 13:03:45 -04:00
Oleg Zhurakousky
8e77993fad GH-897 fixed the NPE issue with CacheManager
- Added a new constructor which takes CacheManager as an argument
- Deprecated the old constructor and setter for CacheManager
- Added fallback to NoOpsCacheManager to the deprecatd construtor to avoid the possiblity of NPE.
- Added warning to notify user that caching is _effectively disabled_ if NoOpCacheManager is used.

Fix #897
2017-05-10 22:53:58 -04:00
Julian Hanhart
d250022250 Fix URLs used by ConfluentSchemaRegistryClient
Fix #845

Fixed resource path for fetch by SchemaReference in the ConfluentSchemaRegistryClient.
2017-03-20 11:39:48 -04:00
Marius Bogoevici
21cc1e9b58 Polishing 2017-03-08 16:09:55 -05:00
Vinicius Carvalho
4cd9b0c3ec Introducing caching
Fix #832 #752

Adds caching capabilities to the converter. Remote calls to the
registry, Schema Parsing and toString were very expensive operations
that are now being cached

Adds a decorator to the registryClient that allows caching of remote
invocations.

Fixing review comments
2017-03-08 12:45:19 -05:00
Ilayaperumal Gopinathan
8792423abc Not enforcing EnableAutoConfiguration on aggregate parent
- The aggregate parent configuration now imports the required auto configuration classes such as ChannelBindingAutoConfiguration and EndpointAutoConfiguration
while `@EnableBinding` is still at the parent configuration to be able to share the binding configuration among the child applications
 - Add `ConditionalOnMissingBean` in spring cloud stream auto configuration classes so that if `EnableAutoConfiguration` is enabled at child application classes
they don't get instantiated again
 - Conditionally import EmbeddedServlet auto-configuration when web environment is enabled for parent
 - Add test

Resolves #737
2017-02-16 15:48:52 +00:00
Marius Bogoevici
f6073ade89 Fix id type on SchemaRegistrationResponse
Fixes #739

Use primitive type in `SchemaRegistryClient`
2016-12-19 21:17:39 +05:30
making
9f067a6907 Add spring-boot-configuration-processor to SCSt schema/server
Add spring.cloud.stream.schemaRegistryClient.enabled
2016-11-23 12:39:39 +05:30
Vinicius Carvalho
4422b21438 Add Avro serialization and schema management support
- Add schema server implementation
- Add schema client abstraction
- Add schema client implementation for own schema registry server
- Add schema client supporting Confluent schema registry
- Add Avro-based message converter supporting a static schema resource
- Add Avro-based message converter with schema evolution support, via
  schema registry client.
- On serialization, the converter register writer schemas with the schema
  registry server and augment the content type of outbound message with
  schema information.
  On deserialization, the reading converter will fetch the schema from the server
  if not available locally.

Use class information if schema is not specified

In the case of SpecificRecord and Reflective readers/writers, the class information can be used instead

Make subtype prefix configurable and shorten the subject

- Subtype prefix is now configurable and subject is the lowercase schema name
- Enhance/correct javadoc

Refine AbstractAvroMessageConverter

- distinguish between writer and reader schema when reader is created

Add schema registry and schema registry client docs
2016-07-31 21:26:36 -04:00