Spring/spring-kafka
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

GH-853: Type Safe ErrorHandlingDeserializer

Browse Source 
Resolves https://github.com/spring-projects/spring-kafka/issues/853

Since a `null` key is common, we only check for the exception header
if we detect that the error handling deserializer is configured.

* Polishing; add failedDeserializationFunction
This commit is contained in:
Gary Russell
2018-11-06 10:00:04 -05:00
committed by Artem Bilan
parent 7bf2c647ff
commit 074e9613e2
9 changed files with 554 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
src/checkstyle/checkstyle-suppressions.xml
View File

@@ -7,4 +7,5 @@
<suppress files="[\\/]test[\\/]" checks="RequireThis" />
<suppress files="[\\/]test[\\/]" checks="Javadoc*" />
<suppress files="KafkaMatchersTests" checks="RegexpSinglelineJava" />
<suppress files="DeserializationException" checks="MutableException" />
</suppressions>

61
src/reference/asciidoc/kafka.adoc
View File

@@ -1937,22 +1937,28 @@ Generally, the `BytesJsonMessageConverter` is more efficient because it avoids a
===== ErrorHandlingDeserializer
When a deserializer fails to deserialize a message, Spring has no way to handle the problem because it occurs before the `poll()` returns.
To solve this problem, version 2.2 introduced the `ErrorHandlingDeserializer`.
To solve this problem, version 2.2 introduced the `ErrorHandlingDeserializer2`.
This deserializer delegates to a real deserializer (key or value).
If the delegate fails to deserialize the record content, the `ErrorHandlingDeserializer` returns a `DeserializationException` instead, containing the cause and raw bytes.
When using a record-level `MessageListener`, if either the key or value contains a `DeserializationException`, the container's `ErrorHandler` is called with the failed `ConsumerRecord`.
When using a `BatchMessageListener`, the failed record is passed to the application along with the remaining records in the batch, so it is the responsibility of the application listener to check whether the key or value in a particular record is a `DeserializationException`.
If the delegate fails to deserialize the record content, the `ErrorHandlingDeserializer2` returns a `null` value and a `DeserializationException` in a header, containing the cause and raw bytes.
When using a record-level `MessageListener`, if either the key or value contains a `DeserializationException` header, the container's `ErrorHandler` is called with the failed `ConsumerRecord`; the record is not passed to the listener.
You can use the `DefaultKafkaConsumerFactory` constructor that takes key and value `Deserializer` objects and wire in appropriate `ErrorHandlingDeserializer` configured with the proper delegates.
Alternatively, you can configure a `failedDeserializationFunction` which is a `BiConsumer<byte[], Headers, T>`.
This function is invoked to create an instance of `T` which is passed to the listener, as normal.
The raw record value and headers are provided to the function.
The `DeserializationException` can be found (as a serialized Java object) in headers; see the javadocs for the `ErrorHandlingDeserializer2` for more information.
When using a `BatchMessageListener`, you **must** provide a `failedDeserializationFunction`, otherwise, the batch of records will not be type safe.
You can use the `DefaultKafkaConsumerFactory` constructor that takes key and value `Deserializer` objects and wire in appropriate `ErrorHandlingDeserializer2` configured with the proper delegates.
Alternatively, you can use consumer configuration properties which are used by the `ErrorHandlingDeserializer` to instantiate the delegates.
The property names are `ErrorHandlingDeserializer.KEY_DESERIALIZER_CLASS` and `ErrorHandlingDeserializer.VALUE_DESERIALIZER_CLASS`; the property value can be a class or class name.
The property names are `ErrorHandlingDeserializer2.KEY_DESERIALIZER_CLASS` and `ErrorHandlingDeserializer2.VALUE_DESERIALIZER_CLASS`; the property value can be a class or class name.
For example:
[source, java]
----
... // other props
props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, ErrorHandlingDeserializer.class);
props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, ErrorHandlingDeserializer.class);
props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, ErrorHandlingDeserializer2.class);
props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, ErrorHandlingDeserializer2.class);
props.put(ErrorHandlingDeserializer.KEY_DESERIALIZER_CLASS, JsonDeserializer.class);
props.put(JsonDeserializer.KEY_DEFAULT_TYPE, "com.example.MyKey")
props.put(ErrorHandlingDeserializer.VALUE_DESERIALIZER_CLASS, JsonDeserializer.class.getName());
@@ -1961,6 +1967,45 @@ props.put(JsonDeserializer.TRUSTED_PACKAGES, "com.example")
return new DefaultKafkaConsumerFactory<>(props);
----
The following is an example of using a `failedDeserializationFunction`.
[source, java]
----
public class BadFoo extends Foo {
private final byte[] failedDecode;
public BadFoo(byte[] failedDecode) {
this.failedDecode = failedDecode;
}
public byte[] getFailedDecode() {
return this.failedDecode;
}
}
public class FailedFooProvider implements BiFunction<byte[], Headers, Foo> {
@Override
public Foo apply(byte[] t, Headers u) {
return new BadFoo(t);
}
}
----
and config
[source, java]
----
...
consumerProps.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, ErrorHandlingDeserializer2.class);
consumerProps.put(ErrorHandlingDeserializer2.VALUE_DESERIALIZER_CLASS, JsonDeserializer.class);
consumerProps.put(ErrorHandlingDeserializer2.VALUE_FUNCTION, FailedFooProvider.class);
...
----
[[payload-conversion-with-batch]]
===== Payload Conversion with Batch Listeners