88 lines
35 KiB
HTML
88 lines
35 KiB
HTML
<html><head>
|
|
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
|
|
<title>33. Schema Evolution Support</title><link rel="stylesheet" type="text/css" href="css/manual-multipage.css"><meta name="generator" content="DocBook XSL Stylesheets V1.79.1"><link rel="home" href="multi_spring-cloud.html" title="Spring Cloud"><link rel="up" href="multi__spring_cloud_stream.html" title="Part V. Spring Cloud Stream"><link rel="prev" href="multi_content-type-management.html" title="32. Content Type Negotiation"><link rel="next" href="multi__inter_application_communication.html" title="34. Inter-Application Communication"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">33. Schema Evolution Support</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="multi_content-type-management.html">Prev</a> </td><th width="60%" align="center">Part V. Spring Cloud Stream</th><td width="20%" align="right"> <a accesskey="n" href="multi__inter_application_communication.html">Next</a></td></tr></table><hr></div><div class="chapter"><div class="titlepage"><div><div><h2 class="title"><a name="schema-evolution" href="#schema-evolution"></a>33. Schema Evolution Support</h2></div></div></div><p>Spring Cloud Stream provides support for schema evolution so that the data can be evolved over time and still work with older or newer producers and consumers and vice versa.
|
|
Most serialization models, especially the ones that aim for portability across different platforms and languages, rely on a schema that describes how the data is serialized in the binary payload.
|
|
In order to serialize the data and then to interpret it, both the sending and receiving sides must have access to a schema that describes the binary format.
|
|
In certain cases, the schema can be inferred from the payload type on serialization or from the target type on deserialization.
|
|
However, many applications benefit from having access to an explicit schema that describes the binary data format.
|
|
A schema registry lets you store schema information in a textual format (typically JSON) and makes that information accessible to various applications that need it to receive and send data in binary format.
|
|
A schema is referenceable as a tuple consisting of:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem">A subject that is the logical name of the schema</li><li class="listitem">The schema version</li><li class="listitem">The schema format, which describes the binary format of the data</li></ul></div><p>This following sections goes through the details of various components involved in schema evolution process.</p><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="_schema_registry_client" href="#_schema_registry_client"></a>33.1 Schema Registry Client</h2></div></div></div><p>The client-side abstraction for interacting with schema registry servers is the <code class="literal">SchemaRegistryClient</code> interface, which has the following structure:</p><pre class="programlisting"><span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">interface</span> SchemaRegistryClient {
|
|
|
|
SchemaRegistrationResponse register(String subject, String format, String schema);
|
|
|
|
String fetch(SchemaReference schemaReference);
|
|
|
|
String fetch(Integer id);
|
|
|
|
}</pre><p>Spring Cloud Stream provides out-of-the-box implementations for interacting with its own schema server and for interacting with the Confluent Schema Registry.</p><p>A client for the Spring Cloud Stream schema registry can be configured by using the <code class="literal">@EnableSchemaRegistryClient</code>, as follows:</p><pre class="programlisting"> <em><span class="hl-annotation" style="color: gray">@EnableBinding(Sink.class)</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@SpringBootApplication</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@EnableSchemaRegistryClient</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">static</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> AvroSinkApplication {
|
|
...
|
|
}</pre><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Note"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="images/note.png"></td><th align="left">Note</th></tr><tr><td align="left" valign="top"><p>The default converter is optimized to cache not only the schemas from the remote server but also the <code class="literal">parse()</code> and <code class="literal">toString()</code> methods, which are quite expensive.
|
|
Because of this, it uses a <code class="literal">DefaultSchemaRegistryClient</code> that does not cache responses.
|
|
If you intend to change the default behavior, you can use the client directly on your code and override it to the desired outcome.
|
|
To do so, you have to add the property <code class="literal">spring.cloud.stream.schemaRegistryClient.cached=true</code> to your application properties.</p></td></tr></table></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="_schema_registry_client_properties" href="#_schema_registry_client_properties"></a>33.1.1 Schema Registry Client Properties</h3></div></div></div><p>The Schema Registry Client supports the following properties:</p><div class="variablelist"><dl class="variablelist"><dt><span class="term"><code class="literal">spring.cloud.stream.schemaRegistryClient.endpoint</code></span></dt><dd>The location of the schema-server.
|
|
When setting this, use a full URL, including protocol (<code class="literal">http</code> or <code class="literal">https</code>) , port, and context path.</dd><dt><span class="term">Default</span></dt><dd><code class="literal"><a class="link" href="http://localhost:8990/" target="_top">http://localhost:8990/</a></code></dd><dt><span class="term"><code class="literal">spring.cloud.stream.schemaRegistryClient.cached</code></span></dt><dd>Whether the client should cache schema server responses.
|
|
Normally set to <code class="literal">false</code>, as the caching happens in the message converter.
|
|
Clients using the schema registry client should set this to <code class="literal">true</code>.</dd><dt><span class="term">Default</span></dt><dd><code class="literal">true</code></dd></dl></div></div></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="_avro_schema_registry_client_message_converters" href="#_avro_schema_registry_client_message_converters"></a>33.2 Avro Schema Registry Client Message Converters</h2></div></div></div><p>For applications that have a SchemaRegistryClient bean registered with the application context, Spring Cloud Stream auto configures an Apache Avro message converter for schema management.
|
|
This eases schema evolution, as applications that receive messages can get easy access to a writer schema that can be reconciled with their own reader schema.</p><p>For outbound messages, if the content type of the channel is set to <code class="literal">application/*+avro</code>, the <code class="literal">MessageConverter</code> is activated, as shown in the following example:</p><pre class="programlisting"><span xmlns:d="http://docbook.org/ns/docbook" class="hl-attribute">spring.cloud.stream.bindings.output.contentType</span>=application/*+avro</pre><p>During the outbound conversion, the message converter tries to infer the schema of each outbound messages (based on its type) and register it to a subject (based on the payload type) by using the <code class="literal">SchemaRegistryClient</code>.
|
|
If an identical schema is already found, then a reference to it is retrieved.
|
|
If not, the schema is registered, and a new version number is provided.
|
|
The message is sent with a <code class="literal">contentType</code> header by using the following scheme: <code class="literal">application/[prefix].[subject].v[version]+avro</code>, where <code class="literal">prefix</code> is configurable and <code class="literal">subject</code> is deduced from the payload type.</p><p>For example, a message of the type <code class="literal">User</code> might be sent as a binary payload with a content type of <code class="literal">application/vnd.user.v2+avro</code>, where <code class="literal">user</code> is the subject and <code class="literal">2</code> is the version number.</p><p>When receiving messages, the converter infers the schema reference from the header of the incoming message and tries to retrieve it. The schema is used as the writer schema in the deserialization process.</p><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="_avro_schema_registry_message_converter_properties" href="#_avro_schema_registry_message_converter_properties"></a>33.2.1 Avro Schema Registry Message Converter Properties</h3></div></div></div><p>If you have enabled Avro based schema registry client by setting <code class="literal">spring.cloud.stream.bindings.output.contentType=application/*+avro</code>, you can customize the behavior of the registration by setting the following properties.</p><div class="variablelist"><dl class="variablelist"><dt><span class="term">spring.cloud.stream.schema.avro.dynamicSchemaGenerationEnabled</span></dt><dd><p class="simpara">Enable if you want the converter to use reflection to infer a Schema from a POJO.</p><p class="simpara">Default: <code class="literal">false</code></p></dd><dt><span class="term">spring.cloud.stream.schema.avro.readerSchema</span></dt><dd>Avro compares schema versions by looking at a writer schema (origin payload) and a reader schema (your application payload). See the <a class="link" href="https://avro.apache.org/docs/1.7.6/spec.html" target="_top">Avro documentation</a> for more information. If set, this overrides any lookups at the schema server and uses the local schema as the reader schema.
|
|
Default: <code class="literal">null</code></dd><dt><span class="term">spring.cloud.stream.schema.avro.schemaLocations</span></dt><dd><p class="simpara">Registers any <code class="literal">.avsc</code> files listed in this property with the Schema Server.</p><p class="simpara">Default: <code class="literal">empty</code></p></dd><dt><span class="term">spring.cloud.stream.schema.avro.prefix</span></dt><dd><p class="simpara">The prefix to be used on the Content-Type header.</p><p class="simpara">Default: <code class="literal">vnd</code></p></dd></dl></div></div></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="_apache_avro_message_converters" href="#_apache_avro_message_converters"></a>33.3 Apache Avro Message Converters</h2></div></div></div><p>Spring Cloud Stream provides support for schema-based message converters through its <code class="literal">spring-cloud-stream-schema</code> module.
|
|
Currently, the only serialization format supported out of the box for schema-based message converters is Apache Avro, with more formats to be added in future versions.</p><p>The <code class="literal">spring-cloud-stream-schema</code> module contains two types of message converters that can be used for Apache Avro serialization:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem">Converters that use the class information of the serialized or deserialized objects or a schema with a location known at startup.</li><li class="listitem">Converters that use a schema registry. They locate the schemas at runtime and dynamically register new schemas as domain objects evolve.</li></ul></div></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="_converters_with_schema_support" href="#_converters_with_schema_support"></a>33.4 Converters with Schema Support</h2></div></div></div><p>The <code class="literal">AvroSchemaMessageConverter</code> supports serializing and deserializing messages either by using a predefined schema or by using the schema information available in the class (either reflectively or contained in the <code class="literal">SpecificRecord</code>).
|
|
If you provide a custom converter, then the default AvroSchemaMessageConverter bean is not created. The following example shows a custom converter:</p><p>To use custom converters, you can simply add it to the application context, optionally specifying one or more <code class="literal">MimeTypes</code> with which to associate it.
|
|
The default <code class="literal">MimeType</code> is <code class="literal">application/avro</code>.</p><p>If the target type of the conversion is a <code class="literal">GenericRecord</code>, a schema must be set.</p><p>The following example shows how to configure a converter in a sink application by registering the Apache Avro <code class="literal">MessageConverter</code> without a predefined schema.
|
|
In this example, note that the mime type value is <code class="literal">avro/bytes</code>, not the default <code class="literal">application/avro</code>.</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@EnableBinding(Sink.class)</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@SpringBootApplication</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">static</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> SinkApplication {
|
|
|
|
...
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Bean</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> MessageConverter userMessageConverter() {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">new</span> AvroSchemaMessageConverter(MimeType.valueOf(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"avro/bytes"</span>));
|
|
}
|
|
}</pre><p>Conversely, the following application registers a converter with a predefined schema (found on the classpath):</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@EnableBinding(Sink.class)</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@SpringBootApplication</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">static</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> SinkApplication {
|
|
|
|
...
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Bean</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> MessageConverter userMessageConverter() {
|
|
AvroSchemaMessageConverter converter = <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">new</span> AvroSchemaMessageConverter(MimeType.valueOf(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"avro/bytes"</span>));
|
|
converter.setSchemaLocation(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">new</span> ClassPathResource(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"schemas/User.avro"</span>));
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> converter;
|
|
}
|
|
}</pre></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="_schema_registry_server" href="#_schema_registry_server"></a>33.5 Schema Registry Server</h2></div></div></div><p>Spring Cloud Stream provides a schema registry server implementation.
|
|
To use it, you can add the <code class="literal">spring-cloud-stream-schema-server</code> artifact to your project and use the <code class="literal">@EnableSchemaRegistryServer</code> annotation, which adds the schema registry server REST controller to your application.
|
|
This annotation is intended to be used with Spring Boot web applications, and the listening port of the server is controlled by the <code class="literal">server.port</code> property.
|
|
The <code class="literal">spring.cloud.stream.schema.server.path</code> property can be used to control the root path of the schema server (especially when it is embedded in other applications).
|
|
The <code class="literal">spring.cloud.stream.schema.server.allowSchemaDeletion</code> boolean property enables the deletion of a schema. By default, this is disabled.</p><p>The schema registry server uses a relational database to store the schemas.
|
|
By default, it uses an embedded database.
|
|
You can customize the schema storage by using the <a class="link" href="http://docs.spring.io/spring-boot/docs/current-SNAPSHOT/reference/htmlsingle/#boot-features-sql" target="_top">Spring Boot SQL database and JDBC configuration options</a>.</p><p>The following example shows a Spring Boot application that enables the schema registry:</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@SpringBootApplication</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@EnableSchemaRegistryServer</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> SchemaRegistryServerApplication {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">static</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">void</span> main(String[] args) {
|
|
SpringApplication.run(SchemaRegistryServerApplication.<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span>, args);
|
|
}
|
|
}</pre><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="_schema_registry_server_api" href="#_schema_registry_server_api"></a>33.5.1 Schema Registry Server API</h3></div></div></div><p>The Schema Registry Server API consists of the following operations:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><code class="literal">POST /</code> — see <span class="quote">“<span class="quote"><a class="xref" href="multi_schema-evolution.html#spring-cloud-stream-overview-registering-new-schema" title="Registering a New Schema">the section called “Registering a New Schema”</a></span>”</span></li><li class="listitem">'GET /{subject}/{format}/{version}' — see <span class="quote">“<span class="quote"><a class="xref" href="multi_schema-evolution.html#spring-cloud-stream-overview-retrieve-schema-subject-format-version" title="Retrieving an Existing Schema by Subject, Format, and Version">the section called “Retrieving an Existing Schema by Subject, Format, and Version”</a></span>”</span></li><li class="listitem"><code class="literal">GET /{subject}/{format}</code> — see <span class="quote">“<span class="quote"><a class="xref" href="multi_schema-evolution.html#spring-cloud-stream-overview-retrieve-schema-subject-format" title="Retrieving an Existing Schema by Subject and Format">the section called “Retrieving an Existing Schema by Subject and Format”</a></span>”</span></li><li class="listitem"><code class="literal">GET /schemas/{id}</code> — see <span class="quote">“<span class="quote"><a class="xref" href="multi_schema-evolution.html#spring-cloud-stream-overview-retrieve-schema-id" title="Retrieving an Existing Schema by ID">the section called “Retrieving an Existing Schema by ID”</a></span>”</span></li><li class="listitem"><code class="literal">DELETE /{subject}/{format}/{version}</code> — see <span class="quote">“<span class="quote"><a class="xref" href="multi_schema-evolution.html#spring-cloud-stream-overview-deleting-schema-subject-format-version" title="Deleting a Schema by Subject, Format, and Version">the section called “Deleting a Schema by Subject, Format, and Version”</a></span>”</span></li><li class="listitem"><code class="literal">DELETE /schemas/{id}</code> — see <span class="quote">“<span class="quote"><a class="xref" href="multi_schema-evolution.html#spring-cloud-stream-overview-deleting-schema-id" title="Deleting a Schema by ID">the section called “Deleting a Schema by ID”</a></span>”</span></li><li class="listitem"><code class="literal">DELETE /{subject}</code> — see <span class="quote">“<span class="quote"><a class="xref" href="multi_schema-evolution.html#spring-cloud-stream-overview-deleting-schema-subject" title="Deleting a Schema by Subject">the section called “Deleting a Schema by Subject”</a></span>”</span></li></ul></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a name="spring-cloud-stream-overview-registering-new-schema" href="#spring-cloud-stream-overview-registering-new-schema"></a>Registering a New Schema</h4></div></div></div><p>To register a new schema, send a <code class="literal">POST</code> request to the <code class="literal">/</code> endpoint.</p><p>The <code class="literal">/</code> accepts a JSON payload with the following fields:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><code class="literal">subject</code>: The schema subject</li><li class="listitem"><code class="literal">format</code>: The schema format</li><li class="listitem"><code class="literal">definition</code>: The schema definition</li></ul></div><p>Its response is a schema object in JSON, with the following fields:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><code class="literal">id</code>: The schema ID</li><li class="listitem"><code class="literal">subject</code>: The schema subject</li><li class="listitem"><code class="literal">format</code>: The schema format</li><li class="listitem"><code class="literal">version</code>: The schema version</li><li class="listitem"><code class="literal">definition</code>: The schema definition</li></ul></div></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a name="spring-cloud-stream-overview-retrieve-schema-subject-format-version" href="#spring-cloud-stream-overview-retrieve-schema-subject-format-version"></a>Retrieving an Existing Schema by Subject, Format, and Version</h4></div></div></div><p>To retrieve an existing schema by subject, format, and version, send <code class="literal">GET</code> request to the <code class="literal">/{subject}/{format}/{version}</code> endpoint.</p><p>Its response is a schema object in JSON, with the following fields:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><code class="literal">id</code>: The schema ID</li><li class="listitem"><code class="literal">subject</code>: The schema subject</li><li class="listitem"><code class="literal">format</code>: The schema format</li><li class="listitem"><code class="literal">version</code>: The schema version</li><li class="listitem"><code class="literal">definition</code>: The schema definition</li></ul></div></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a name="spring-cloud-stream-overview-retrieve-schema-subject-format" href="#spring-cloud-stream-overview-retrieve-schema-subject-format"></a>Retrieving an Existing Schema by Subject and Format</h4></div></div></div><p>To retrieve an existing schema by subject and format, send a <code class="literal">GET</code> request to the <code class="literal">/subject/format</code> endpoint.</p><p>Its response is a list of schemas with each schema object in JSON, with the following fields:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><code class="literal">id</code>: The schema ID</li><li class="listitem"><code class="literal">subject</code>: The schema subject</li><li class="listitem"><code class="literal">format</code>: The schema format</li><li class="listitem"><code class="literal">version</code>: The schema version</li><li class="listitem"><code class="literal">definition</code>: The schema definition</li></ul></div></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a name="spring-cloud-stream-overview-retrieve-schema-id" href="#spring-cloud-stream-overview-retrieve-schema-id"></a>Retrieving an Existing Schema by ID</h4></div></div></div><p>To retrieve a schema by its ID, send a <code class="literal">GET</code> request to the <code class="literal">/schemas/{id}</code> endpoint.</p><p>Its response is a schema object in JSON, with the following fields:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><code class="literal">id</code>: The schema ID</li><li class="listitem"><code class="literal">subject</code>: The schema subject</li><li class="listitem"><code class="literal">format</code>: The schema format</li><li class="listitem"><code class="literal">version</code>: The schema version</li><li class="listitem"><code class="literal">definition</code>: The schema definition</li></ul></div></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a name="spring-cloud-stream-overview-deleting-schema-subject-format-version" href="#spring-cloud-stream-overview-deleting-schema-subject-format-version"></a>Deleting a Schema by Subject, Format, and Version</h4></div></div></div><p>To delete a schema identified by its subject, format, and version, send a <code class="literal">DELETE</code> request to the <code class="literal">/{subject}/{format}/{version}</code> endpoint.</p></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a name="spring-cloud-stream-overview-deleting-schema-id" href="#spring-cloud-stream-overview-deleting-schema-id"></a>Deleting a Schema by ID</h4></div></div></div><p>To delete a schema by its ID, send a <code class="literal">DELETE</code> request to the <code class="literal">/schemas/{id}</code> endpoint.</p></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a name="spring-cloud-stream-overview-deleting-schema-subject" href="#spring-cloud-stream-overview-deleting-schema-subject"></a>Deleting a Schema by Subject</h4></div></div></div><p><code class="literal">DELETE /{subject}</code></p><p>Delete existing schemas by their subject.</p><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Note"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="images/note.png"></td><th align="left">Note</th></tr><tr><td align="left" valign="top"><p>This note applies to users of Spring Cloud Stream 1.1.0.RELEASE only.
|
|
Spring Cloud Stream 1.1.0.RELEASE used the table name, <code class="literal">schema</code>, for storing <code class="literal">Schema</code> objects. <code class="literal">Schema</code> is a keyword in a number of database implementations.
|
|
To avoid any conflicts in the future, starting with 1.1.1.RELEASE, we have opted for the name <code class="literal">SCHEMA_REPOSITORY</code> for the storage table.
|
|
Any Spring Cloud Stream 1.1.0.RELEASE users who upgrade should migrate their existing schemas to the new table before upgrading.</p></td></tr></table></div></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="_using_confluents_schema_registry" href="#_using_confluents_schema_registry"></a>33.5.2 Using Confluent’s Schema Registry</h3></div></div></div><p>The default configuration creates a <code class="literal">DefaultSchemaRegistryClient</code> bean.
|
|
If you want to use the Confluent schema registry, you need to create a bean of type <code class="literal">ConfluentSchemaRegistryClient</code>, which supersedes the one configured by default by the framework. The following example shows how to create such a bean:</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@Bean</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> SchemaRegistryClient schemaRegistryClient(<em><span class="hl-annotation" style="color: gray">@Value("${spring.cloud.stream.schemaRegistryClient.endpoint}")</span></em> String endpoint){
|
|
ConfluentSchemaRegistryClient client = <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">new</span> ConfluentSchemaRegistryClient();
|
|
client.setEndpoint(endpoint);
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> client;
|
|
}</pre><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Note"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="images/note.png"></td><th align="left">Note</th></tr><tr><td align="left" valign="top"><p>The ConfluentSchemaRegistryClient is tested against Confluent platform version 4.0.0.</p></td></tr></table></div></div></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="_schema_registration_and_resolution" href="#_schema_registration_and_resolution"></a>33.6 Schema Registration and Resolution</h2></div></div></div><p>To better understand how Spring Cloud Stream registers and resolves new schemas and its use of Avro schema comparison features, we provide two separate subsections:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><span class="quote">“<span class="quote"><a class="xref" href="multi_schema-evolution.html#spring-cloud-stream-overview-schema-registration-process" title="33.6.1 Schema Registration Process (Serialization)">Section 33.6.1, “Schema Registration Process (Serialization)”</a></span>”</span></li><li class="listitem"><span class="quote">“<span class="quote"><a class="xref" href="multi_schema-evolution.html#spring-cloud-stream-overview-schema-resolution-process" title="33.6.2 Schema Resolution Process (Deserialization)">Section 33.6.2, “Schema Resolution Process (Deserialization)”</a></span>”</span></li></ul></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="spring-cloud-stream-overview-schema-registration-process" href="#spring-cloud-stream-overview-schema-registration-process"></a>33.6.1 Schema Registration Process (Serialization)</h3></div></div></div><p>The first part of the registration process is extracting a schema from the payload that is being sent over a channel.
|
|
Avro types such as <code class="literal">SpecificRecord</code> or <code class="literal">GenericRecord</code> already contain a schema, which can be retrieved immediately from the instance.
|
|
In the case of POJOs, a schema is inferred if the <code class="literal">spring.cloud.stream.schema.avro.dynamicSchemaGenerationEnabled</code> property is set to <code class="literal">true</code> (the default).</p><div class="figure"><a name="d0e11681" href="#d0e11681"></a><p class="title"><b>Figure 33.1. Schema Writer Resolution Process</b></p><div class="figure-contents"><div class="mediaobject" align="center"><img src="images/schema_resolution.png" align="middle" alt="schema resolution"></div></div></div><br class="figure-break"><p>Ones a schema is obtained, the converter loads its metadata (version) from the remote server.
|
|
First, it queries a local cache. If no result is found, it submits the data to the server, which replies with versioning information.
|
|
The converter always caches the results to avoid the overhead of querying the Schema Server for every new message that needs to be serialized.</p><div class="figure"><a name="d0e11692" href="#d0e11692"></a><p class="title"><b>Figure 33.2. Schema Registration Process</b></p><div class="figure-contents"><div class="mediaobject" align="center"><img src="images/registration.png" align="middle" alt="registration"></div></div></div><br class="figure-break"><p>With the schema version information, the converter sets the <code class="literal">contentType</code> header of the message to carry the version information — for example: <code class="literal">application/vnd.user.v1+avro</code>.</p></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="spring-cloud-stream-overview-schema-resolution-process" href="#spring-cloud-stream-overview-schema-resolution-process"></a>33.6.2 Schema Resolution Process (Deserialization)</h3></div></div></div><p>When reading messages that contain version information (that is, a <code class="literal">contentType</code> header with a scheme like the one described under <span class="quote">“<span class="quote"><a class="xref" href="multi_schema-evolution.html#spring-cloud-stream-overview-schema-registration-process" title="33.6.1 Schema Registration Process (Serialization)">Section 33.6.1, “Schema Registration Process (Serialization)”</a></span>”</span>), the converter queries the Schema server to fetch the writer schema of the message.
|
|
Once it has found the correct schema of the incoming message, it retrieves the reader schema and, by using Avro’s schema resolution support, reads it into the reader definition (setting defaults and any missing properties).</p><div class="figure"><a name="d0e11720" href="#d0e11720"></a><p class="title"><b>Figure 33.3. Schema Reading Resolution Process</b></p><div class="figure-contents"><div class="mediaobject" align="center"><img src="images/schema_reading.png" align="middle" alt="schema reading"></div></div></div><br class="figure-break"><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Note"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="images/note.png"></td><th align="left">Note</th></tr><tr><td align="left" valign="top"><p>You should understand the difference between a writer schema (the application that wrote the message) and a reader schema (the receiving application).
|
|
We suggest taking a moment to read <a class="link" href="https://avro.apache.org/docs/1.7.6/spec.html" target="_top">the Avro terminology</a> and understand the process.
|
|
Spring Cloud Stream always fetches the writer schema to determine how to read a message.
|
|
If you want to get Avro’s schema evolution support working, you need to make sure that a <code class="literal">readerSchema</code> was properly set for your application.</p></td></tr></table></div></div></div></div><div class="navfooter"><hr><table width="100%" summary="Navigation footer"><tr><td width="40%" align="left"><a accesskey="p" href="multi_content-type-management.html">Prev</a> </td><td width="20%" align="center"><a accesskey="u" href="multi__spring_cloud_stream.html">Up</a></td><td width="40%" align="right"> <a accesskey="n" href="multi__inter_application_communication.html">Next</a></td></tr><tr><td width="40%" align="left" valign="top">32. Content Type Negotiation </td><td width="20%" align="center"><a accesskey="h" href="multi_spring-cloud.html">Home</a></td><td width="40%" align="right" valign="top"> 34. Inter-Application Communication</td></tr></table></div></body></html> |