Content type redesign
Fixes #992, #1050, #1051, #1052 Adding custom jackson converter with some tests Adds kryo message converter to replace codec Checkstyle changes Removing codec support - Removed codec dependency from AbstractBinder - MessageSerializationUtils is almost an empty shell for now, just to keep code compiling until we get EmbeddedHeaders interceptors - Updated Kryo tests Removing codec module from build Added a new Annotation for custom converters '@StreamConverter' Fixed some tests with new expected behavior Moved broken tests to a temporary package to keep track of progress Fixed KryoConverter to fail based on headers Fixed a couple of more tests Making converters strict to only convert their corresponding contentType Bypassing conversion for ErrorMessages * Configuring SI ConfigurableCompositeMessageConverter - Moved ContentType related beans into separate configuration - Configured SI ConfigurableCompositeMessageConverter to use same converters as Stream does (for ServiceActivator) - TupleConverter should return byte[] as all other converters - Fixed tests * Fixes tests - Revert to Boot 2.0.0.M3. Snapshots breaking actuator - Checkstyle fixes - Disable JsonUnmarshalling as a catch all converter Fixing Schema tests Fixing Metrics tests Fixing reactive tests applying checkstyle fixes * Adding new content type tests - Fixed ContentTypeInterceptor misusage of default mimeType Changing contentType doc section Improving doc section Last minute polish Fixing BinderTests to use bytes to compare messages Applied changes to Base Binders test to use the new contentType handling mechanism PR review fixes Renaming StreamConverter -> StreamMessageConverter
This commit is contained in:
Vinicius Carvalho
committed by
Soby Chacko
Soby Chacko
parent
24cf992301
commit
171f034a8c
@@ -1361,122 +1361,187 @@ To allow you to propagate information about the content type of produced message
|
||||
For middleware that does not directly support headers, Spring Cloud Stream provides its own mechanism of automatically wrapping outbound messages in an envelope of its own.
|
||||
For middleware that does support headers, Spring Cloud Stream applications may receive messages with a given content type from non-Spring Cloud Stream applications.
|
||||
|
||||
Spring Cloud Stream can handle messages based on this information in two ways:
|
||||
The content type resolution process have been redesigned for Spring Cloud Stream 2.0.
|
||||
|
||||
* Through its `contentType` settings on inbound and outbound channels
|
||||
* Through its argument mapping performed for methods annotated with `@StreamListener`
|
||||
Please read the migrating from 1.3 section to understand the changes when interacting with applications using versions of the framework.
|
||||
|
||||
The framework depends on a `contentType` to be present as a header in order to know how serialize/deserialize a payload.
|
||||
|
||||
Spring Cloud Stream allows you to declaratively configure type conversion for inputs and outputs using the `spring.cloud.stream.bindings.<channelName>.content-type` property of a binding.
|
||||
Note that general type conversion may also be accomplished easily by using a transformer inside your application.
|
||||
Currently, Spring Cloud Stream natively supports the following type conversions commonly used in streams:
|
||||
|
||||
* *JSON* to/from *POJO*
|
||||
* *JSON* to/from https://github.com/spring-projects/spring-tuple/blob/master/spring-tuple/src/main/java/org/springframework/tuple/Tuple.java[org.springframework.tuple.Tuple]
|
||||
* *Object* to/from *byte[]* : Either the raw bytes serialized for remote transport, bytes emitted by an application, or converted to bytes using Java serialization(requires the object to be Serializable)
|
||||
* *String* to/from *byte[]*
|
||||
* *Object* to *plain text* (invokes the object's _toString()_ method)
|
||||
[NOTE]
|
||||
====
|
||||
For both input and output channel, setting a contentType via a property or via annotation only triggers the `default` converter if a message header with value `contentType` is not present.
|
||||
This is useful for cases where you just want to send a _POJO_ without sending any header information, or to consume messages that do not have a `contentType` header present.
|
||||
The framework will always override any default settings with the value found on the message headers.
|
||||
====
|
||||
|
||||
Where _JSON_ represents either a byte array or String payload containing JSON.
|
||||
Currently, Objects may be converted from a JSON byte array or String.
|
||||
Converting to JSON always produces a String.
|
||||
|
||||
If no `content-type` property is set on an outbound channel, Spring Cloud Stream will serialize the payload using a serializer based on the https://github.com/EsotericSoftware/kryo[Kryo] serialization framework.
|
||||
Deserializing messages at the destination requires the payload class to be present on the receiver's classpath.
|
||||
[TIP]
|
||||
====
|
||||
Although contentType became a required property, the framework will set a default value of `application/json` for all input/output channels if one is not
|
||||
provided by the user.
|
||||
====
|
||||
|
||||
[[mime-types]]
|
||||
=== MIME types
|
||||
`content-type` values are parsed as media types, e.g., `application/json` or `text/plain;charset=UTF-8`.
|
||||
The `content-type` values are parsed as media types, e.g., `application/json` or `text/plain;charset=UTF-8`.
|
||||
|
||||
MIME types are especially useful for indicating how to convert to String or byte[] content.
|
||||
Spring Cloud Stream also uses MIME type format to represent Java types, using the general type `application/x-java-object` with a `type` parameter.
|
||||
For example, `application/x-java-object;type=java.util.Map` or `application/x-java-object;type=com.bar.Foo` can be set as the `content-type` property of an input binding.
|
||||
In addition, Spring Cloud Stream provides custom MIME types, notably, `application/x-spring-tuple` to specify a Tuple.
|
||||
|
||||
[[mime-types-and-java-types]]
|
||||
=== MIME types and Java types
|
||||
|
||||
The type conversions Spring Cloud Stream provides out of the box are summarized in the following table:
|
||||
'Source Payload' means the payload before conversion and 'Target Payload' means the 'payload' after conversion.
|
||||
The type conversion can occur either on the 'producer' side (output) or at the 'consumer' side (input).
|
||||
=== Channel contentType and Message Headers
|
||||
|
||||
|===
|
||||
|Source Payload |Target Payload | `content-type` header (source message) | `content-type` header (after conversion) | Comments
|
||||
You can configure a message channel content type using `spring.cloud.stream.bindings.<channelName>.content-type` property, or using the `@Input` and `@Output` annotations.
|
||||
By doing so, even if you send a POJO with no `contentType` information, the framework will set the MessageHeader `contentType` to the specified value set for the channel.
|
||||
|
||||
|POJO
|
||||
|JSON String
|
||||
|ignored
|
||||
|application/json
|
||||
|
|
||||
However, if you send a `Message<T>` and sets the `contentType` manually, that takes precedence over the configured property value.
|
||||
This is valid for both input and output channels. The `MessageHeader` will always take precedence over the default configured `contentType` for the channel.
|
||||
|
||||
|Tuple
|
||||
|JSON String
|
||||
|ignored
|
||||
|application/json
|
||||
|JSON is tailored for Tuple
|
||||
=== ContentType handling for output channels
|
||||
|
||||
|POJO
|
||||
|String (toString())
|
||||
|ignored
|
||||
|text/plain, java.lang.String
|
||||
|
|
||||
Starting with version 2.0, the framework will no longer try to infer a contentType based on the payload `T` of a `Message<T>`.
|
||||
It will instead use the contentType header (or the default provided by the framework) to configure the right `MessageConverter` to serialize the payload into `byte[]`.
|
||||
|
||||
|POJO
|
||||
|byte[] (java.io serialized)
|
||||
|ignored
|
||||
|application/x-java-serialized-object
|
||||
|
|
||||
The `contentType` you set is a hint to activate the corresponding `MessageConverter`. The converter can then modify the contentType to augment the information, such as the case with `Kryo` and `Avro` conveters.
|
||||
|
||||
|JSON byte[] or String
|
||||
|POJO
|
||||
|application/json (or none)
|
||||
|application/x-java-object
|
||||
|
|
||||
For outbound messages, if your payload is of typ `byte[]`, the framework will skip the conversion logic, and just write those bytes to the wire.
|
||||
In this case, if `contentType` of the message is absent, it will set the default value specified to channel.
|
||||
|
||||
|byte[] or String
|
||||
|Serializable
|
||||
|application/x-java-serialized-object
|
||||
|application/x-java-object
|
||||
|
|
||||
|
||||
|JSON byte[] or String
|
||||
|Tuple
|
||||
|application/json (or none)
|
||||
|application/x-spring-tuple
|
||||
|
|
||||
|
||||
|byte[]
|
||||
|String
|
||||
|any
|
||||
|text/plain, java.lang.String
|
||||
|will apply any Charset specified in the content-type header
|
||||
|
||||
|String
|
||||
|byte[]
|
||||
|any
|
||||
|application/octet-stream
|
||||
|will apply any Charset specified in the content-type header
|
||||
|
||||
|===
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
Conversion applies to payloads that require type conversion.
|
||||
For example, if an application produces an XML string with outputType=application/json, the payload will not be converted from XML to JSON.
|
||||
This is because the payload send to the outbound channel is already a String so no conversion will be applied at runtime.
|
||||
It is also important to note that when using the default serialization mechanism, the payload class must be shared between the sending and receiving application, and compatible with the binary content.
|
||||
This can create issues when application code changes independently in the two applications, as the binary format and code may become incompatible.
|
||||
====
|
||||
|
||||
[TIP]
|
||||
====
|
||||
While conversion is supported for both inbound and outbound channels, it is especially recommended to be used for the conversion of outbound messages.
|
||||
For the conversion of inbound messages, especially when the target is a POJO, the `@StreamListener` support will perform the conversion automatically.
|
||||
If you intend to bypass conversion, just make sure you set the appropriate `contentType` header, otherwise you could be sending some arbitrary binary data, and the framework may set the header as `application/json` (default).
|
||||
====
|
||||
|
||||
The following snippet shows how you can bypass conversion and set the correct contentType header.
|
||||
|
||||
```java
|
||||
|
||||
@Autowired
|
||||
private Source source;
|
||||
|
||||
public void sendImageData(File f) throws Exception{
|
||||
byte[] data = Files.readAllBytes(f.toPath());
|
||||
MimeType mimeType = (f.getName().endsWith("gif")) ? MimeTypeUtils.IMAGE_GIF : MimeTypeUtils.IMAGE_JPEG;
|
||||
source.output().send(MessageBuilder.withPayload(data)
|
||||
.setHeader(MessageHeaders.CONTENT_TYPE, mimeType)
|
||||
.build());
|
||||
}
|
||||
|
||||
|
||||
```
|
||||
|
||||
Regardless of contentType used, the result is always a `Message<byte[]>` with a header `contentType` set. This is what gets passed to the binder to be sent over the wire.
|
||||
|
||||
|===
|
||||
|`content-type` header | MessageConverter | `content-type` augmented |Supported types | Comments
|
||||
|
||||
|application/json
|
||||
|CustomMappingJackson2MessageConverter
|
||||
|application/json
|
||||
| POJO, primitives and Strings that represent JSON data
|
||||
| It's the default converter if none is specified. Note that if you send a raw String it will be quoted
|
||||
|
||||
|text/plain
|
||||
|ObjectStringMessageConverter
|
||||
|text/plain
|
||||
|Invokes `toString()` of the object
|
||||
|
|
||||
|
||||
|application/x-spring-tuple
|
||||
|TupleJsonMessageConverter
|
||||
|application/x-spring-tuple
|
||||
|org.springframework.tuple.Tuple
|
||||
|
|
||||
|
||||
|application/x-java-serialized-object
|
||||
|JavaSerializationMessageConverter
|
||||
|application/x-java-serialized-object
|
||||
|Any Java type that implements `Serializable`
|
||||
|This converter uses java native serialization. Receivers of this data must have the same class on the classpath.
|
||||
|
||||
|application/x-java-object
|
||||
|KryoMessageConverter
|
||||
|application/x-java-object;type=<Class being serialized>
|
||||
|Any Java type that can be serialized using Kryo
|
||||
|Receivers of this data must have the same class on the classpath.
|
||||
|
||||
|application/avro
|
||||
|AvroMessageConverter
|
||||
|application/avro
|
||||
|A Generic or SpecificRecord from Avro types, a POJO if reflection is used
|
||||
|Avro needs an associated schema to write/read data. Please refer to the section on the docs on how to use it properly
|
||||
|
||||
|===
|
||||
|
||||
=== ContentType handling for input channels
|
||||
|
||||
For input channels, Spring Cloud Stream uses `@StreamListener` and `@ServiceActivator` content handling to support the conversion.
|
||||
It does so by checking either the channel `content-type` set via `@Input(contentType="text/plain")` annotation or via `spring.cloud.stream.bindings.<channel>.contentType` property, or the presense of a header `contentType`.
|
||||
|
||||
The framework will check the contentType set for the Message, select the appropriate `MessageConverter` and apply conversion passing the argument as the target type.
|
||||
|
||||
If the converter does not support the target type it will return `null`, if *all* configured converters return `null`, a `MessageConversionException` is thrown.
|
||||
|
||||
Just like output channels, if your method payload argument is of type `Message<byte[]>`, `byte[]` or `Message<?>` conversion is skipped and you get the raw bytes from the wire, plus the corresponding headers.
|
||||
|
||||
[TIP]
|
||||
====
|
||||
Remember, the MessageHeader always takes precedence over the annotation or property configuration.
|
||||
====
|
||||
|
||||
|===
|
||||
|`content-type` header | MessageConverter | Supported target type | Comments
|
||||
|
||||
|applicaiton/json
|
||||
|CustomMappingJackson2MessageConverter
|
||||
| POJO or String
|
||||
|
|
||||
|
||||
|text/plain
|
||||
|ObjectStringMessageConverter
|
||||
|String
|
||||
|
|
||||
|
||||
|application/x-spring-tuple
|
||||
|TupleJsonMessageConverter
|
||||
|org.springframework.tuple.Tuple
|
||||
|
|
||||
|
||||
|application/x-java-serialized-object
|
||||
|JavaSerializationMessageConverter
|
||||
|Any Java type that implements `Serializable`
|
||||
|
|
||||
|
||||
|application/x-java-object
|
||||
|KryoMessageConverter
|
||||
|Any Java type that can be serialized using Kryo
|
||||
|
|
||||
|
||||
|application/avro
|
||||
|AvroMessageConverter
|
||||
|A Generic or SpecificRecord from Avro types, a POJO if reflection is used
|
||||
|Avro needs an associated schema to write/read data. Please refer to the section on the docs on how to use it properly
|
||||
|
||||
|===
|
||||
|
||||
|
||||
=== Customizing message conversion
|
||||
|
||||
Besides the conversions that it supports out of the box, Spring Cloud Stream also supports registering your own message conversion implementations.
|
||||
This allows you to send and receive data in a variety of custom formats, including binary, and associate them with specific `contentTypes`.
|
||||
Spring Cloud Stream registers all the beans of type `org.springframework.messaging.converter.MessageConverter` as custom message converters along with the out of the box message converters.
|
||||
|
||||
Spring Cloud Stream registers all the beans of type `org.springframework.messaging.converter.MessageConverter` that are qualifeied using `@StreamConverter` annotation, as custom message converters along with the out of the box message converters.
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
The framework requires the `@StreamConverter` qualifier annotation to avoid picking up other converters that may be present on the `ApplicationContext` and could overlap with the default ones.
|
||||
====
|
||||
|
||||
If your message converter needs to work with a specific `content-type` and target class (for both input and output), then the message converter needs to extend `org.springframework.messaging.converter.AbstractMessageConverter`.
|
||||
For conversion when using `@StreamListener`, a message converter that implements `org.springframework.messaging.converter.MessageConverter` would suffice.
|
||||
@@ -1492,6 +1557,7 @@ public static class SinkApplication {
|
||||
...
|
||||
|
||||
@Bean
|
||||
@StreamConverter
|
||||
public MessageConverter customMessageConverter() {
|
||||
return new MyCustomMessageConverter();
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user