1052 lines
272 KiB
HTML
1052 lines
272 KiB
HTML
<html><head>
|
|
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
|
|
<title>Spring Cloud Stream Reference Guide</title><link rel="stylesheet" type="text/css" href="css/manual-singlepage.css"><meta name="generator" content="DocBook XSL Stylesheets V1.79.1"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div lang="en" class="book"><div class="titlepage"><div><div><h1 class="title"><a name="d0e3"></a>Spring Cloud Stream Reference Guide</h1></div><div><div xmlns:d="http://docbook.org/ns/docbook" class="authorgroup"><h2>Authors</h2><span class="author"><span class="firstname">Sabby</span> <span class="surname">Anandan</span></span>, <span class="author"><span class="firstname">Marius</span> <span class="surname">Bogoevici</span></span>, <span class="author"><span class="firstname">Eric</span> <span class="surname">Bottard</span></span>, <span class="author"><span class="firstname">Mark</span> <span class="surname">Fisher</span></span>, <span class="author"><span class="firstname">Ilayaperumal</span> <span class="surname">Gopinathan</span></span>, <span class="author"><span class="firstname">Gunnar</span> <span class="surname">Hillert</span></span>, <span class="author"><span class="firstname">Mark</span> <span class="surname">Pollack</span></span>, <span class="author"><span class="firstname">Patrick</span> <span class="surname">Peralta</span></span>, <span class="author"><span class="firstname">Glenn</span> <span class="surname">Renfro</span></span>, <span class="author"><span class="firstname">Thomas</span> <span class="surname">Risberg</span></span>, <span class="author"><span class="firstname">Dave</span> <span class="surname">Syer</span></span>, <span class="author"><span class="firstname">David</span> <span class="surname">Turanski</span></span>, <span class="author"><span class="firstname">Janne</span> <span class="surname">Valkealahti</span></span>, <span class="author"><span class="firstname">Benjamin</span> <span class="surname">Klein</span></span>, <span class="author"><span class="firstname">Vinicius</span> <span class="surname">Carvalho</span></span>, <span class="author"><span class="firstname">Gary</span> <span class="surname">Russell</span></span>, <span class="author"><span class="firstname">Oleg</span> <span class="surname">Zhurakousky</span></span>, <span class="author"><span class="firstname">Jay</span> <span class="surname">Bryant</span></span></div></div></div><hr></div><div class="toc"><p><b>Table of Contents</b></p><dl class="toc"><dt><span class="part"><a href="#_preface">I. Preface</a></span></dt><dd><dl><dt><span class="chapter"><a href="#_a_brief_history_of_springs_data_integration_journey">1. A Brief History of Spring’s Data Integration Journey</a></span></dt><dt><span class="chapter"><a href="#_quick_start">2. Quick Start</a></span></dt><dd><dl><dt><span class="section"><a href="#spring-cloud-stream-preface-creating-sample-application">2.1. Creating a Sample Application by Using Spring Initializr</a></span></dt><dt><span class="section"><a href="#spring-cloud-stream-preface-importing-project">2.2. Importing the Project into Your IDE</a></span></dt><dt><span class="section"><a href="#spring-cloud-stream-preface-adding-message-handler">2.3. Adding a Message Handler, Building, and Running</a></span></dt></dl></dd><dt><span class="chapter"><a href="#_whats_new_in_2_0">3. What’s New in 2.0?</a></span></dt><dd><dl><dt><span class="section"><a href="#spring-cloud-stream-preface-new-features">3.1. New Features and Components</a></span></dt><dt><span class="section"><a href="#spring-cloud-stream-preface-notable-enhancements">3.2. Notable Enhancements</a></span></dt><dd><dl><dt><span class="section"><a href="#spring-cloud-stream-preface-actuator-web-dependencies">3.2.1. Both Actuator and Web Dependencies Are Now Optional</a></span></dt><dt><span class="section"><a href="#spring-cloud-stream-preface-content-type-negotiation-improvements">3.2.2. Content-type Negotiation Improvements</a></span></dt></dl></dd><dt><span class="section"><a href="#spring-cloud-stream-preface-notable-deprecations">3.3. Notable Deprecations</a></span></dt><dd><dl><dt><span class="section"><a href="#spring-cloud-stream-preface-deprecation-java-serialization">3.3.1. Java Serialization (Java Native and Kryo)</a></span></dt><dt><span class="section"><a href="#spring-cloud-stream-preface-deprecation-classes-methods">3.3.2. Deprecated Classes and Methods</a></span></dt></dl></dd></dl></dd><dt><span class="chapter"><a href="#spring-cloud-stream-overview-introducing">4. Introducing Spring Cloud Stream</a></span></dt><dt><span class="chapter"><a href="#_main_concepts">5. Main Concepts</a></span></dt><dd><dl><dt><span class="section"><a href="#spring-cloud-stream-overview-application-model">5.1. Application Model</a></span></dt><dd><dl><dt><span class="section"><a href="#_fat_jar">5.1.1. Fat JAR</a></span></dt></dl></dd><dt><span class="section"><a href="#spring-cloud-stream-overview-binder-abstraction">5.2. The Binder Abstraction</a></span></dt><dt><span class="section"><a href="#spring-cloud-stream-overview-persistent-publish-subscribe-support">5.3. Persistent Publish-Subscribe Support</a></span></dt><dt><span class="section"><a href="#consumer-groups">5.4. Consumer Groups</a></span></dt><dt><span class="section"><a href="#consumer-types">5.5. Consumer Types</a></span></dt><dd><dl><dt><span class="section"><a href="#durability">5.5.1. Durability</a></span></dt></dl></dd><dt><span class="section"><a href="#partitioning">5.6. Partitioning Support</a></span></dt></dl></dd><dt><span class="chapter"><a href="#_programming_model">6. Programming Model</a></span></dt><dd><dl><dt><span class="section"><a href="#_destination_binders">6.1. Destination Binders</a></span></dt><dt><span class="section"><a href="#_destination_bindings">6.2. Destination Bindings</a></span></dt><dt><span class="section"><a href="#spring-cloud-stream-overview-producing-consuming-messages">6.3. Producing and Consuming Messages</a></span></dt><dd><dl><dt><span class="section"><a href="#_spring_integration_support">6.3.1. Spring Integration Support</a></span></dt><dt><span class="section"><a href="#_using_streamlistener_annotation">6.3.2. Using @StreamListener Annotation</a></span></dt><dt><span class="section"><a href="#_using_streamlistener_for_content_based_routing">6.3.3. Using @StreamListener for Content-based routing</a></span></dt><dt><span class="section"><a href="#_spring_cloud_function">6.3.4. Spring Cloud Function support</a></span></dt><dd><dl><dt><span class="section"><a href="#_functional_composition">Functional Composition</a></span></dt></dl></dd><dt><span class="section"><a href="#spring-cloud-streams-overview-using-polled-consumers">6.3.5. Using Polled Consumers</a></span></dt><dd><dl><dt><span class="section"><a href="#_overview">Overview</a></span></dt><dt><span class="section"><a href="#polled-errors">Handling Errors</a></span></dt></dl></dd></dl></dd><dt><span class="section"><a href="#spring-cloud-stream-overview-error-handling">6.4. Error Handling</a></span></dt><dd><dl><dt><span class="section"><a href="#_application_error_handling">6.4.1. Application Error Handling</a></span></dt><dt><span class="section"><a href="#_system_error_handling">6.4.2. System Error Handling</a></span></dt><dd><dl><dt><span class="section"><a href="#_drop_failed_messages">Drop Failed Messages</a></span></dt><dt><span class="section"><a href="#_dlq_dead_letter_queue">DLQ - Dead Letter Queue</a></span></dt><dt><span class="section"><a href="#_re_queue_failed_messages">Re-queue Failed Messages</a></span></dt></dl></dd><dt><span class="section"><a href="#_retry_template">6.4.3. Retry Template</a></span></dt></dl></dd><dt><span class="section"><a href="#spring-cloud-stream-overview-reactive-programming-support">6.5. Reactive Programming Support</a></span></dt><dd><dl><dt><span class="section"><a href="#_reactor_based_handlers">6.5.1. Reactor-based Handlers</a></span></dt><dt><span class="section"><a href="#_reactive_sources">6.5.2. Reactive Sources</a></span></dt></dl></dd></dl></dd><dt><span class="chapter"><a href="#spring-cloud-stream-overview-binders">7. Binders</a></span></dt><dd><dl><dt><span class="section"><a href="#_producers_and_consumers">7.1. Producers and Consumers</a></span></dt><dt><span class="section"><a href="#spring-cloud-stream-overview-binder-api">7.2. Binder SPI</a></span></dt><dt><span class="section"><a href="#_binder_detection">7.3. Binder Detection</a></span></dt><dd><dl><dt><span class="section"><a href="#_classpath_detection">7.3.1. Classpath Detection</a></span></dt></dl></dd><dt><span class="section"><a href="#multiple-binders">7.4. Multiple Binders on the Classpath</a></span></dt><dt><span class="section"><a href="#multiple-systems">7.5. Connecting to Multiple Systems</a></span></dt><dt><span class="section"><a href="#_binding_visualization_and_control">7.6. Binding visualization and control</a></span></dt><dt><span class="section"><a href="#_binder_configuration_properties">7.7. Binder Configuration Properties</a></span></dt></dl></dd><dt><span class="chapter"><a href="#_configuration_options">8. Configuration Options</a></span></dt><dd><dl><dt><span class="section"><a href="#_binding_service_properties">8.1. Binding Service Properties</a></span></dt><dt><span class="section"><a href="#binding-properties">8.2. Binding Properties</a></span></dt><dd><dl><dt><span class="section"><a href="#_common_binding_properties">8.2.1. Common Binding Properties</a></span></dt><dt><span class="section"><a href="#_consumer_properties">8.2.2. Consumer Properties</a></span></dt><dt><span class="section"><a href="#_producer_properties">8.2.3. Producer Properties</a></span></dt></dl></dd><dt><span class="section"><a href="#dynamicdestination">8.3. Using Dynamically Bound Destinations</a></span></dt></dl></dd><dt><span class="chapter"><a href="#content-type-management">9. Content Type Negotiation</a></span></dt><dd><dl><dt><span class="section"><a href="#_mechanics">9.1. Mechanics</a></span></dt><dd><dl><dt><span class="section"><a href="#_content_type_versus_argument_type">9.1.1. Content Type versus Argument Type</a></span></dt><dt><span class="section"><a href="#_message_converters">9.1.2. Message Converters</a></span></dt></dl></dd><dt><span class="section"><a href="#_provided_messageconverters">9.2. Provided MessageConverters</a></span></dt><dt><span class="section"><a href="#spring-cloud-stream-overview-user-defined-message-converters">9.3. User-defined Message Converters</a></span></dt></dl></dd><dt><span class="chapter"><a href="#schema-evolution">10. Schema Evolution Support</a></span></dt><dd><dl><dt><span class="section"><a href="#_schema_registry_client">10.1. Schema Registry Client</a></span></dt><dd><dl><dt><span class="section"><a href="#_schema_registry_client_properties">10.1.1. Schema Registry Client Properties</a></span></dt></dl></dd><dt><span class="section"><a href="#_avro_schema_registry_client_message_converters">10.2. Avro Schema Registry Client Message Converters</a></span></dt><dd><dl><dt><span class="section"><a href="#_avro_schema_registry_message_converter_properties">10.2.1. Avro Schema Registry Message Converter Properties</a></span></dt></dl></dd><dt><span class="section"><a href="#_apache_avro_message_converters">10.3. Apache Avro Message Converters</a></span></dt><dt><span class="section"><a href="#_converters_with_schema_support">10.4. Converters with Schema Support</a></span></dt><dt><span class="section"><a href="#_schema_registry_server">10.5. Schema Registry Server</a></span></dt><dd><dl><dt><span class="section"><a href="#_schema_registry_server_api">10.5.1. Schema Registry Server API</a></span></dt><dd><dl><dt><span class="section"><a href="#spring-cloud-stream-overview-registering-new-schema">Registering a New Schema</a></span></dt><dt><span class="section"><a href="#spring-cloud-stream-overview-retrieve-schema-subject-format-version">Retrieving an Existing Schema by Subject, Format, and Version</a></span></dt><dt><span class="section"><a href="#spring-cloud-stream-overview-retrieve-schema-subject-format">Retrieving an Existing Schema by Subject and Format</a></span></dt><dt><span class="section"><a href="#spring-cloud-stream-overview-retrieve-schema-id">Retrieving an Existing Schema by ID</a></span></dt><dt><span class="section"><a href="#spring-cloud-stream-overview-deleting-schema-subject-format-version">Deleting a Schema by Subject, Format, and Version</a></span></dt><dt><span class="section"><a href="#spring-cloud-stream-overview-deleting-schema-id">Deleting a Schema by ID</a></span></dt><dt><span class="section"><a href="#spring-cloud-stream-overview-deleting-schema-subject">Deleting a Schema by Subject</a></span></dt></dl></dd><dt><span class="section"><a href="#_using_confluents_schema_registry">10.5.2. Using Confluent’s Schema Registry</a></span></dt></dl></dd><dt><span class="section"><a href="#_schema_registration_and_resolution">10.6. Schema Registration and Resolution</a></span></dt><dd><dl><dt><span class="section"><a href="#spring-cloud-stream-overview-schema-registration-process">10.6.1. Schema Registration Process (Serialization)</a></span></dt><dt><span class="section"><a href="#spring-cloud-stream-overview-schema-resolution-process">10.6.2. Schema Resolution Process (Deserialization)</a></span></dt></dl></dd></dl></dd><dt><span class="chapter"><a href="#_inter_application_communication">11. Inter-Application Communication</a></span></dt><dd><dl><dt><span class="section"><a href="#spring-cloud-stream-overview-connecting-multiple-application-instances">11.1. Connecting Multiple Application Instances</a></span></dt><dt><span class="section"><a href="#spring-cloud-stream-overview-instance-index-instance-count">11.2. Instance Index and Instance Count</a></span></dt><dt><span class="section"><a href="#spring-cloud-stream-overview-partitioning">11.3. Partitioning</a></span></dt><dd><dl><dt><span class="section"><a href="#spring-cloud-stream-overview-configuring-output-bindings-partitioning">11.3.1. Configuring Output Bindings for Partitioning</a></span></dt><dt><span class="section"><a href="#spring-cloud-stream-overview-configuring-input-bindings-partitioning">11.3.2. Configuring Input Bindings for Partitioning</a></span></dt></dl></dd></dl></dd><dt><span class="chapter"><a href="#_testing">12. Testing</a></span></dt><dd><dl><dt><span class="section"><a href="#_disabling_the_test_binder_autoconfiguration">12.1. Disabling the Test Binder Autoconfiguration</a></span></dt></dl></dd><dt><span class="chapter"><a href="#_health_indicator">13. Health Indicator</a></span></dt><dt><span class="chapter"><a href="#spring-cloud-stream-overview-metrics-emitter">14. Metrics Emitter</a></span></dt><dt><span class="chapter"><a href="#_samples">15. Samples</a></span></dt><dd><dl><dt><span class="section"><a href="#_deploying_stream_applications_on_cloudfoundry">15.1. Deploying Stream Applications on CloudFoundry</a></span></dt></dl></dd><dt><span class="chapter"><a href="#_binder_implementations">16. Binder Implementations</a></span></dt></dl></dd></dl></div><div class="part"><div class="titlepage"><div><div><h1 class="title"><a name="_preface" href="#_preface"></a>Part I. Preface</h1></div></div></div><div class="chapter"><div class="titlepage"><div><div><h2 class="title"><a name="_a_brief_history_of_springs_data_integration_journey" href="#_a_brief_history_of_springs_data_integration_journey"></a>1. A Brief History of Spring’s Data Integration Journey</h2></div></div></div><p>Spring’s journey on Data Integration started with <a class="link" href="https://projects.spring.io/spring-integration/" target="_top">Spring Integration</a>. With its programming model, it provided a consistent developer experience to build applications that can embrace <a class="link" href="http://www.enterpriseintegrationpatterns.com/" target="_top">Enterprise Integration Patterns</a> to connect with external systems such as, databases, message brokers, and among others.</p><p>Fast forward to the cloud-era, where microservices have become prominent in the enterprise setting. <a class="link" href="https://projects.spring.io/spring-boot/" target="_top">Spring Boot</a> transformed the way how developers built Applications. With Spring’s programming model and the runtime responsibilities handled by Spring Boot, it became seamless to develop stand-alone, production-grade Spring-based microservices.</p><p>To extend this to Data Integration workloads, Spring Integration and Spring Boot were put together into a new project. Spring Cloud Stream was born.</p><p>With Spring Cloud Stream, developers can:
|
|
* Build, test, iterate, and deploy data-centric applications in isolation.
|
|
* Apply modern microservices architecture patterns, including composition through messaging.
|
|
* Decouple application responsibilities with event-centric thinking. An event can represent something that has happened in time, to which the downstream consumer applications can react without knowing where it originated or the producer’s identity.
|
|
* Port the business logic onto message brokers (such as RabbitMQ, Apache Kafka, Amazon Kinesis).
|
|
* Interoperate between channel-based and non-channel-based application binding scenarios to support stateless and stateful computations by using Project Reactor’s Flux and Kafka Streams APIs.
|
|
* Rely on the framework’s automatic content-type support for common use-cases. Extending to different data conversion types is possible.</p></div><div class="chapter"><div class="titlepage"><div><div><h2 class="title"><a name="_quick_start" href="#_quick_start"></a>2. Quick Start</h2></div></div></div><p>You can try Spring Cloud Stream in less then 5 min even before you jump into any details by following this three-step guide.</p><p>We show you how to create a Spring Cloud Stream application that receives messages coming from the messaging middleware of your choice (more on this later) and logs received messages to the console.
|
|
We call it <code class="literal">LoggingConsumer</code>.
|
|
While not very practical, it provides a good introduction to some of the main concepts
|
|
and abstractions, making it easier to digest the rest of this user guide.</p><p>The three steps are as follows:</p><div class="orderedlist"><ol class="orderedlist" type="1"><li class="listitem"><a class="xref" href="#spring-cloud-stream-preface-creating-sample-application" title="2.1 Creating a Sample Application by Using Spring Initializr">Section 2.1, “Creating a Sample Application by Using Spring Initializr”</a></li><li class="listitem"><a class="xref" href="#spring-cloud-stream-preface-importing-project" title="2.2 Importing the Project into Your IDE">Section 2.2, “Importing the Project into Your IDE”</a></li><li class="listitem"><a class="xref" href="#spring-cloud-stream-preface-adding-message-handler" title="2.3 Adding a Message Handler, Building, and Running">Section 2.3, “Adding a Message Handler, Building, and Running”</a></li></ol></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="spring-cloud-stream-preface-creating-sample-application" href="#spring-cloud-stream-preface-creating-sample-application"></a>2.1 Creating a Sample Application by Using Spring Initializr</h2></div></div></div><p>To get started, visit the <a class="link" href="https://start.spring.io" target="_top">Spring Initializr</a>. From there, you can generate our <code class="literal">LoggingConsumer</code> application. To do so:</p><div class="orderedlist"><ol class="orderedlist" type="1"><li class="listitem">In the <span class="strong"><strong>Dependencies</strong></span> section, start typing <code class="literal">stream</code>.
|
|
When the <span class="quote">“<span class="quote">Cloud Stream</span>”</span> option should appears, select it.</li><li class="listitem">Start typing either 'kafka' or 'rabbit'.</li><li class="listitem"><p class="simpara">Select <span class="quote">“<span class="quote">Kafka</span>”</span> or <span class="quote">“<span class="quote">RabbitMQ</span>”</span>.</p><p class="simpara">Basically, you choose the messaging middleware to which your application binds.
|
|
We recommend using the one you have already installed or feel more comfortable with installing and running.
|
|
Also, as you can see from the Initilaizer screen, there are a few other options you can choose.
|
|
For example, you can choose Gradle as your build tool instead of Maven (the default).</p></li><li class="listitem"><p class="simpara">In the <span class="strong"><strong>Artifact</strong></span> field, type 'logging-consumer'.</p><p class="simpara">The value of the <span class="strong"><strong>Artifact</strong></span> field becomes the application name.
|
|
If you chose RabbitMQ for the middleware, your Spring Initializr should now be as follows:</p></li></ol></div><div class="informalfigure"><div class="mediaobject" align="center"><img src="https://raw.githubusercontent.com/spring-cloud/spring-cloud-stream/master/docs/src/main/asciidoc/images/spring-initializr.png" align="middle" alt="spring initializr"></div></div><div class="orderedlist"><ol class="orderedlist" type="1"><li class="listitem"><p class="simpara">Click the <span class="strong"><strong>Generate Project</strong></span> button.</p><p class="simpara">Doing so downloads the zipped version of the generated project to your hard drive.</p></li><li class="listitem">Unzip the file into the folder you want to use as your project directory.</li></ol></div><div class="tip" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Tip"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Tip]" src="images/tip.png"></td><th align="left">Tip</th></tr><tr><td align="left" valign="top"><p>We encourage you to explore the many possibilities available in the Spring Initializr.
|
|
It lets you create many different kinds of Spring applications.</p></td></tr></table></div></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="spring-cloud-stream-preface-importing-project" href="#spring-cloud-stream-preface-importing-project"></a>2.2 Importing the Project into Your IDE</h2></div></div></div><p>Now you can import the project into your IDE.
|
|
Keep in mind that, depending on the IDE, you may need to follow a specific import procedure.
|
|
For example, depending on how the project was generated (Maven or Gradle), you may need to follow specific import procedure (for example, in Eclipse or STS, you need to use File → Import → Maven → Existing Maven Project).</p><p>Once imported, the project must have no errors of any kind. Also, <code class="literal">src/main/java</code> should contain <code class="literal">com.example.loggingconsumer.LoggingConsumerApplication</code>.</p><p>Technically, at this point, you can run the application’s main class.
|
|
It is already a valid Spring Boot application.
|
|
However, it does not do anything, so we want to add some code.</p></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="spring-cloud-stream-preface-adding-message-handler" href="#spring-cloud-stream-preface-adding-message-handler"></a>2.3 Adding a Message Handler, Building, and Running</h2></div></div></div><p>Modify the <code class="literal">com.example.loggingconsumer.LoggingConsumerApplication</code> class to look as follows:</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@SpringBootApplication</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@EnableBinding(Sink.class)</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> LoggingConsumerApplication {
|
|
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">static</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">void</span> main(String[] args) {
|
|
SpringApplication.run(LoggingConsumerApplication.<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span>, args);
|
|
}
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@StreamListener(Sink.INPUT)</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">void</span> handle(Person person) {
|
|
System.out.println(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"Received: "</span> + person);
|
|
}
|
|
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">static</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> Person {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">private</span> String name;
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> String getName() {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> name;
|
|
}
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">void</span> setName(String name) {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">this</span>.name = name;
|
|
}
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> String toString() {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">this</span>.name;
|
|
}
|
|
}
|
|
}</pre><p>As you can see from the preceding listing:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem">We have enabled <code class="literal">Sink</code> binding (input-no-output) by using <code class="literal">@EnableBinding(Sink.class)</code>.
|
|
Doing so signals to the framework to initiate binding to the messaging middleware, where it automatically creates the destination (that is, queue, topic, and others) that are bound to the <code class="literal">Sink.INPUT</code> channel.</li><li class="listitem">We have added a <code class="literal">handler</code> method to receive incoming messages of type <code class="literal">Person</code>.
|
|
Doing so lets you see one of the core features of the framework: It tries to automatically convert incoming message payloads to type <code class="literal">Person</code>.</li></ul></div><p>You now have a fully functional Spring Cloud Stream application that does listens for messages.
|
|
From here, for simplicity, we assume you selected RabbitMQ in <a class="link" href="#spring-cloud-stream-preface-creating-sample-application" title="2.1 Creating a Sample Application by Using Spring Initializr">step one</a>.
|
|
Assuming you have RabbitMQ installed and running, you can start the application by running its <code class="literal">main</code> method in your IDE.</p><p>You should see following output:</p><pre class="screen"> --- [ main] c.s.b.r.p.RabbitExchangeQueueProvisioner : declaring queue for inbound: input.anonymous.CbMIwdkJSBO1ZoPDOtHtCg, bound to: input
|
|
--- [ main] o.s.a.r.c.CachingConnectionFactory : Attempting to connect to: [localhost:5672]
|
|
--- [ main] o.s.a.r.c.CachingConnectionFactory : Created new connection: rabbitConnectionFactory#2a3a299:0/SimpleConnection@66c83fc8. . .
|
|
. . .
|
|
--- [ main] o.s.i.a.i.AmqpInboundChannelAdapter : started inbound.input.anonymous.CbMIwdkJSBO1ZoPDOtHtCg
|
|
. . .
|
|
--- [ main] c.e.l.LoggingConsumerApplication : Started LoggingConsumerApplication in 2.531 seconds (JVM running for 2.897)</pre><p>Go to the RabbitMQ management console or any other RabbitMQ client and send a message to <code class="literal">input.anonymous.CbMIwdkJSBO1ZoPDOtHtCg</code>.
|
|
The <code class="literal">anonymous.CbMIwdkJSBO1ZoPDOtHtCg</code> part represents the group name and is generated, so it is bound to be different in your environment.
|
|
For something more predictable, you can use an explicit group name by setting <code class="literal">spring.cloud.stream.bindings.input.group=hello</code> (or whatever name you like).</p><p>The contents of the message should be a JSON representation of the <code class="literal">Person</code> class, as follows:</p><pre class="literallayout">{"name":"Sam Spade"}</pre><p>Then, in your console, you should see:</p><p><code class="literal">Received: Sam Spade</code></p><p>You can also build and package your application into a boot jar (by using <code class="literal">./mvnw clean install</code>) and run the built JAR by using the <code class="literal">java -jar</code> command.</p><p>Now you have a working (albeit very basic) Spring Cloud Stream application.</p></div></div><div class="chapter"><div class="titlepage"><div><div><h2 class="title"><a name="_whats_new_in_2_0" href="#_whats_new_in_2_0"></a>3. What’s New in 2.0?</h2></div></div></div><p>Spring Cloud Stream introduces a number of new features, enhancements, and changes. The following sections outline the most notable ones:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><a class="xref" href="#spring-cloud-stream-preface-new-features" title="3.1 New Features and Components">Section 3.1, “New Features and Components”</a></li><li class="listitem"><a class="xref" href="#spring-cloud-stream-preface-notable-enhancements" title="3.2 Notable Enhancements">Section 3.2, “Notable Enhancements”</a></li></ul></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="spring-cloud-stream-preface-new-features" href="#spring-cloud-stream-preface-new-features"></a>3.1 New Features and Components</h2></div></div></div><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><span class="strong"><strong>Polling Consumers</strong></span>: Introduction of polled consumers, which lets the application control message processing rates.
|
|
See <span class="quote">“<span class="quote"><a class="xref" href="#spring-cloud-streams-overview-using-polled-consumers" title="6.3.5 Using Polled Consumers">Section 6.3.5, “Using Polled Consumers”</a></span>”</span> for more details.
|
|
You can also read <a class="link" href="https://spring.io/blog/2018/02/27/spring-cloud-stream-2-0-polled-consumers" target="_top">this blog post</a> for more details.</li><li class="listitem"><span class="strong"><strong>Micrometer Support</strong></span>: Metrics has been switched to use <a class="link" href="https://micrometer.io/" target="_top">Micrometer</a>.
|
|
<code class="literal">MeterRegistry</code> is also provided as a bean so that custom applications can autowire it to capture custom metrics.
|
|
See <span class="quote">“<span class="quote"><a class="xref" href="#spring-cloud-stream-overview-metrics-emitter" title="14. Metrics Emitter">Chapter 14, <i>Metrics Emitter</i></a></span>”</span> for more details.</li><li class="listitem"><span class="strong"><strong>New Actuator Binding Controls</strong></span>: New actuator binding controls let you both visualize and control the Bindings lifecycle.
|
|
For more details, see <a class="xref" href="#_binding_visualization_and_control" title="7.6 Binding visualization and control">Section 7.6, “Binding visualization and control”</a>.</li><li class="listitem"><span class="strong"><strong>Configurable RetryTemplate</strong></span>: Aside from providing properties to configure <code class="literal">RetryTemplate</code>, we now let you provide your own template, effectively overriding the one provided by the framework.
|
|
To use it, configure it as a <code class="literal">@Bean</code> in your application.</li></ul></div></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="spring-cloud-stream-preface-notable-enhancements" href="#spring-cloud-stream-preface-notable-enhancements"></a>3.2 Notable Enhancements</h2></div></div></div><p>This version includes the following notable enhancements:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><a class="xref" href="#spring-cloud-stream-preface-actuator-web-dependencies" title="3.2.1 Both Actuator and Web Dependencies Are Now Optional">Section 3.2.1, “Both Actuator and Web Dependencies Are Now Optional”</a></li><li class="listitem"><a class="xref" href="#spring-cloud-stream-preface-content-type-negotiation-improvements" title="3.2.2 Content-type Negotiation Improvements">Section 3.2.2, “Content-type Negotiation Improvements”</a></li><li class="listitem"><a class="xref" href="#spring-cloud-stream-preface-notable-deprecations" title="3.3 Notable Deprecations">Section 3.3, “Notable Deprecations”</a></li></ul></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="spring-cloud-stream-preface-actuator-web-dependencies" href="#spring-cloud-stream-preface-actuator-web-dependencies"></a>3.2.1 Both Actuator and Web Dependencies Are Now Optional</h3></div></div></div><p>This change slims down the footprint of the deployed application in the event neither actuator nor web dependencies required.
|
|
It also lets you switch between the reactive and conventional web paradigms by manually adding one of the following dependencies.</p><p>The following listing shows how to add the conventional web framework:</p><pre class="programlisting"><span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"><dependency></span>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"><groupId></span>org.springframework.boot<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"></groupId></span>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"><artifactId></span>spring-boot-starter-web<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"></artifactId></span>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"></dependency></span></pre><p>The following listing shows how to add the reactive web framework:</p><pre class="programlisting"><span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"><dependency></span>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"><groupId></span>org.springframework.boot<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"></groupId></span>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"><artifactId></span>spring-boot-starter-webflux<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"></artifactId></span>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"></dependency></span></pre><p>The following list shows how to add the actuator dependency:</p><pre class="programlisting"><span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"><dependency></span>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"><groupId></span>org.springframework.boot<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"></groupId></span>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"><artifactId></span>spring-boot-starter-actuator<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"></artifactId></span>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"></dependency></span></pre></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="spring-cloud-stream-preface-content-type-negotiation-improvements" href="#spring-cloud-stream-preface-content-type-negotiation-improvements"></a>3.2.2 Content-type Negotiation Improvements</h3></div></div></div><p>One of the core themes for verion 2.0 is improvements (in both consistency and performance) around content-type negotiation and message conversion.
|
|
The following summary outlines the notable changes and improvements in this area.
|
|
See the <span class="quote">“<span class="quote"><a class="xref" href="#content-type-management" title="9. Content Type Negotiation">Chapter 9, <i>Content Type Negotiation</i></a></span>”</span> section for more details.
|
|
Also <a class="link" href="https://spring.io/blog/2018/02/26/spring-cloud-stream-2-0-content-type-negotiation-and-transformation" target="_top">this blog post</a> contains more detail.</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem">All message conversion is now handled <span class="strong"><strong>only</strong></span> by <code class="literal">MessageConverter</code> objects.</li><li class="listitem">We introduced the <code class="literal">@StreamMessageConverter</code> annotation to provide custom <code class="literal">MessageConverter</code> objects.</li><li class="listitem">We introduced the default <code class="literal">Content Type</code> as <code class="literal">application/json</code>, which needs to be taken into consideration when migrating 1.3 application or operating in the mixed mode (that is, 1.3 producer → 2.0 consumer).</li><li class="listitem">Messages with textual payloads and a <code class="literal">contentType</code> of <code class="literal">text/…​</code> or <code class="literal">…​/json</code> are no longer converted to <code class="literal">Message<String></code> for cases where the argument type of the provided <code class="literal">MessageHandler</code> can not be determined (that is, <code class="literal">public void handle(Message<?> message)</code> or <code class="literal">public void handle(Object payload)</code>).
|
|
Furthermore, a strong argument type may not be enough to properly convert messages, so the <code class="literal">contentType</code> header may be used as a supplement by some <code class="literal">MessageConverters</code>.</li></ul></div></div></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="spring-cloud-stream-preface-notable-deprecations" href="#spring-cloud-stream-preface-notable-deprecations"></a>3.3 Notable Deprecations</h2></div></div></div><p>As of version 2.0, the following items have been deprecated:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><a class="xref" href="#spring-cloud-stream-preface-deprecation-java-serialization" title="3.3.1 Java Serialization (Java Native and Kryo)">Section 3.3.1, “Java Serialization (Java Native and Kryo)”</a></li><li class="listitem"><a class="xref" href="#spring-cloud-stream-preface-deprecation-classes-methods" title="3.3.2 Deprecated Classes and Methods">Section 3.3.2, “Deprecated Classes and Methods”</a></li></ul></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="spring-cloud-stream-preface-deprecation-java-serialization" href="#spring-cloud-stream-preface-deprecation-java-serialization"></a>3.3.1 Java Serialization (Java Native and Kryo)</h3></div></div></div><p><code class="literal">JavaSerializationMessageConverter</code> and <code class="literal">KryoMessageConverter</code> remain for now. However, we plan to move them out of the core packages and support in the future.
|
|
The main reason for this deprecation is to flag the issue that type-based, language-specific serialization could cause in distributed environments, where Producers and Consumers may depend on different JVM versions or have different versions of supporting libraries (that is, Kryo).
|
|
We also wanted to draw the attention to the fact that Consumers and Producers may not even be Java-based, so polyglot style serialization (i.e., JSON) is better suited.</p></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="spring-cloud-stream-preface-deprecation-classes-methods" href="#spring-cloud-stream-preface-deprecation-classes-methods"></a>3.3.2 Deprecated Classes and Methods</h3></div></div></div><p>The following is a quick summary of notable deprecations. See the corresponding {spring-cloud-stream-javadoc-current}[javadoc] for more details.</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><code class="literal">SharedChannelRegistry</code>. Use <code class="literal">SharedBindingTargetRegistry</code>.</li><li class="listitem"><code class="literal">Bindings</code>.
|
|
Beans qualified by it are already uniquely identified by their type — for example, provided <code class="literal">Source</code>, <code class="literal">Processor</code>, or custom bindings:</li></ul></div><pre class="screen">public interface Sample {
|
|
String OUTPUT = "sampleOutput";
|
|
|
|
@Output(Sample.OUTPUT)
|
|
MessageChannel output();
|
|
}</pre><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><code class="literal">HeaderMode.raw</code>. Use <code class="literal">none</code>, <code class="literal">headers</code> or <code class="literal">embeddedHeaders</code></li><li class="listitem"><code class="literal">ProducerProperties.partitionKeyExtractorClass</code> in favor of <code class="literal">partitionKeyExtractorName</code> and <code class="literal">ProducerProperties.partitionSelectorClass</code> in favor of <code class="literal">partitionSelectorName</code>.
|
|
This change ensures that both components are Spring configured and managed and are referenced in a Spring-friendly way.</li><li class="listitem"><code class="literal">BinderAwareRouterBeanPostProcessor</code>. While the component remains, it is no longer a <code class="literal">BeanPostProcessor</code> and will be renamed in the future.</li><li class="listitem"><code class="literal">BinderProperties.setEnvironment(Properties environment)</code>. Use <code class="literal">BinderProperties.setEnvironment(Map<String, Object> environment)</code>.</li></ul></div></div></div></div><div class="chapter"><div class="titlepage"><div><div><h2 class="title"><a name="spring-cloud-stream-overview-introducing" href="#spring-cloud-stream-overview-introducing"></a>4. Introducing Spring Cloud Stream</h2></div></div></div><p>Spring Cloud Stream is a framework for building message-driven microservice applications.
|
|
Spring Cloud Stream builds upon Spring Boot to create standalone, production-grade Spring applications and uses Spring Integration to provide connectivity to message brokers.
|
|
It provides opinionated configuration of middleware from several vendors, introducing the concepts of persistent publish-subscribe semantics, consumer groups, and partitions.</p><p>You can add the <code class="literal">@EnableBinding</code> annotation to your application to get immediate connectivity to a message broker, and you can add <code class="literal">@StreamListener</code> to a method to cause it to receive events for stream processing.
|
|
The following example shows a sink application that receives external messages:</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@SpringBootApplication</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@EnableBinding(Sink.class)</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> VoteRecordingSinkApplication {
|
|
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">static</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">void</span> main(String[] args) {
|
|
SpringApplication.run(VoteRecordingSinkApplication.<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span>, args);
|
|
}
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@StreamListener(Sink.INPUT)</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">void</span> processVote(Vote vote) {
|
|
votingService.recordVote(vote);
|
|
}
|
|
}</pre><p>The <code class="literal">@EnableBinding</code> annotation takes one or more interfaces as parameters (in this case, the parameter is a single <code class="literal">Sink</code> interface).
|
|
An interface declares input and output channels.
|
|
Spring Cloud Stream provides the <code class="literal">Source</code>, <code class="literal">Sink</code>, and <code class="literal">Processor</code> interfaces. You can also define your own interfaces.</p><p>The following listing shows the definition of the <code class="literal">Sink</code> interface:</p><pre class="programlisting"><span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">interface</span> Sink {
|
|
String INPUT = <span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"input"</span>;
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Input(Sink.INPUT)</span></em>
|
|
SubscribableChannel input();
|
|
}</pre><p>The <code class="literal">@Input</code> annotation identifies an input channel, through which received messages enter the application.
|
|
The <code class="literal">@Output</code> annotation identifies an output channel, through which published messages leave the application.
|
|
The <code class="literal">@Input</code> and <code class="literal">@Output</code> annotations can take a channel name as a parameter.
|
|
If a name is not provided, the name of the annotated method is used.</p><p>Spring Cloud Stream creates an implementation of the interface for you.
|
|
You can use this in the application by autowiring it, as shown in the following example (from a test case):</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@RunWith(SpringJUnit4ClassRunner.class)</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@SpringApplicationConfiguration(classes = VoteRecordingSinkApplication.class)</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@WebAppConfiguration</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@DirtiesContext</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> StreamApplicationTests {
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Autowired</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">private</span> Sink sink;
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Test</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">void</span> contextLoads() {
|
|
assertNotNull(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">this</span>.sink.input());
|
|
}
|
|
}</pre></div><div class="chapter"><div class="titlepage"><div><div><h2 class="title"><a name="_main_concepts" href="#_main_concepts"></a>5. Main Concepts</h2></div></div></div><p>Spring Cloud Stream provides a number of abstractions and primitives that simplify the writing of message-driven microservice applications.
|
|
This section gives an overview of the following:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><a class="link" href="#spring-cloud-stream-overview-application-model" title="5.1 Application Model">Spring Cloud Stream’s application model</a></li><li class="listitem"><a class="xref" href="#spring-cloud-stream-overview-binder-abstraction" title="5.2 The Binder Abstraction">Section 5.2, “The Binder Abstraction”</a></li><li class="listitem"><a class="link" href="#spring-cloud-stream-overview-persistent-publish-subscribe-support" title="5.3 Persistent Publish-Subscribe Support">Persistent publish-subscribe support</a></li><li class="listitem"><a class="link" href="#consumer-groups" title="5.4 Consumer Groups">Consumer group support</a></li><li class="listitem"><a class="link" href="#partitioning" title="5.6 Partitioning Support">Partitioning support</a></li><li class="listitem"><a class="link" href="#spring-cloud-stream-overview-binder-api" title="7.2 Binder SPI">A pluggable Binder SPI</a></li></ul></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="spring-cloud-stream-overview-application-model" href="#spring-cloud-stream-overview-application-model"></a>5.1 Application Model</h2></div></div></div><p>A Spring Cloud Stream application consists of a middleware-neutral core.
|
|
The application communicates with the outside world through input and output channels injected into it by Spring Cloud Stream.
|
|
Channels are connected to external brokers through middleware-specific Binder implementations.</p><div class="figure"><a name="d0e729" href="#d0e729"></a><p class="title"><b>Figure 5.1. Spring Cloud Stream Application</b></p><div class="figure-contents"><div class="mediaobject" align="center"><img src="https://raw.githubusercontent.com/spring-cloud/spring-cloud-stream/master/docs/src/main/asciidoc/images/SCSt-with-binder.png" align="middle" alt="SCSt with binder"></div></div></div><br class="figure-break"><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="_fat_jar" href="#_fat_jar"></a>5.1.1 Fat JAR</h3></div></div></div><p>Spring Cloud Stream applications can be run in stand-alone mode from your IDE for testing.
|
|
To run a Spring Cloud Stream application in production, you can create an executable (or <span class="quote">“<span class="quote">fat</span>”</span>) JAR by using the standard Spring Boot tooling provided for Maven or Gradle. See the <a class="link" href="https://docs.spring.io/spring-boot/docs/current/reference/html/howto-build.html#howto-create-an-executable-jar-with-maven" target="_top">Spring Boot Reference Guide</a> for more details.</p></div></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="spring-cloud-stream-overview-binder-abstraction" href="#spring-cloud-stream-overview-binder-abstraction"></a>5.2 The Binder Abstraction</h2></div></div></div><p>Spring Cloud Stream provides Binder implementations for <a class="link" href="https://github.com/spring-cloud/spring-cloud-stream-binder-kafka" target="_top">Kafka</a> and <a class="link" href="https://github.com/spring-cloud/spring-cloud-stream-binder-rabbit" target="_top">Rabbit MQ</a>.
|
|
Spring Cloud Stream also includes a <a class="link" href="https://github.com/spring-cloud/spring-cloud-stream/blob/master/spring-cloud-stream-test-support/src/main/java/org/springframework/cloud/stream/test/binder/TestSupportBinder.java" target="_top">TestSupportBinder</a>, which leaves a channel unmodified so that tests can interact with channels directly and reliably assert on what is received.
|
|
You can also use the extensible API to write your own Binder.</p><p>Spring Cloud Stream uses Spring Boot for configuration, and the Binder abstraction makes it possible for a Spring Cloud Stream application to be flexible in how it connects to middleware.
|
|
For example, deployers can dynamically choose, at runtime, the destinations (such as the Kafka topics or RabbitMQ exchanges) to which channels connect.
|
|
Such configuration can be provided through external configuration properties and in any form supported by Spring Boot (including application arguments, environment variables, and <code class="literal">application.yml</code> or <code class="literal">application.properties</code> files).
|
|
In the sink example from the <a class="xref" href="#spring-cloud-stream-overview-introducing" title="4. Introducing Spring Cloud Stream">Chapter 4, <i>Introducing Spring Cloud Stream</i></a> section, setting the <code class="literal">spring.cloud.stream.bindings.input.destination</code> application property to <code class="literal">raw-sensor-data</code> causes it to read from the <code class="literal">raw-sensor-data</code> Kafka topic or from a queue bound to the <code class="literal">raw-sensor-data</code> RabbitMQ exchange.</p><p>Spring Cloud Stream automatically detects and uses a binder found on the classpath.
|
|
You can use different types of middleware with the same code.
|
|
To do so, include a different binder at build time.
|
|
For more complex use cases, you can also package multiple binders with your application and have it choose the binder( and even whether to use different binders for different channels) at runtime.</p></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="spring-cloud-stream-overview-persistent-publish-subscribe-support" href="#spring-cloud-stream-overview-persistent-publish-subscribe-support"></a>5.3 Persistent Publish-Subscribe Support</h2></div></div></div><p>Communication between applications follows a publish-subscribe model, where data is broadcast through shared topics.
|
|
This can be seen in the following figure, which shows a typical deployment for a set of interacting Spring Cloud Stream applications.</p><div class="figure"><a name="d0e792" href="#d0e792"></a><p class="title"><b>Figure 5.2. Spring Cloud Stream Publish-Subscribe</b></p><div class="figure-contents"><div class="mediaobject" align="center"><img src="https://raw.githubusercontent.com/spring-cloud/spring-cloud-stream/master/docs/src/main/asciidoc/images/SCSt-sensors.png" align="middle" alt="SCSt sensors"></div></div></div><br class="figure-break"><p>Data reported by sensors to an HTTP endpoint is sent to a common destination named <code class="literal">raw-sensor-data</code>.
|
|
From the destination, it is independently processed by a microservice application that computes time-windowed averages and by another microservice application that ingests the raw data into HDFS (Hadoop Distributed File System).
|
|
In order to process the data, both applications declare the topic as their input at runtime.</p><p>The publish-subscribe communication model reduces the complexity of both the producer and the consumer and lets new applications be added to the topology without disruption of the existing flow.
|
|
For example, downstream from the average-calculating application, you can add an application that calculates the highest temperature values for display and monitoring.
|
|
You can then add another application that interprets the same flow of averages for fault detection.
|
|
Doing all communication through shared topics rather than point-to-point queues reduces coupling between microservices.</p><p>While the concept of publish-subscribe messaging is not new, Spring Cloud Stream takes the extra step of making it an opinionated choice for its application model.
|
|
By using native middleware support, Spring Cloud Stream also simplifies use of the publish-subscribe model across different platforms.</p></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="consumer-groups" href="#consumer-groups"></a>5.4 Consumer Groups</h2></div></div></div><p>While the publish-subscribe model makes it easy to connect applications through shared topics, the ability to scale up by creating multiple instances of a given application is equally important.
|
|
When doing so, different instances of an application are placed in a competing consumer relationship, where only one of the instances is expected to handle a given message.</p><p>Spring Cloud Stream models this behavior through the concept of a consumer group.
|
|
(Spring Cloud Stream consumer groups are similar to and inspired by Kafka consumer groups.)
|
|
Each consumer binding can use the <code class="literal">spring.cloud.stream.bindings.<channelName>.group</code> property to specify a group name.
|
|
For the consumers shown in the following figure, this property would be set as <code class="literal">spring.cloud.stream.bindings.<channelName>.group=hdfsWrite</code> or <code class="literal">spring.cloud.stream.bindings.<channelName>.group=average</code>.</p><div class="figure"><a name="d0e826" href="#d0e826"></a><p class="title"><b>Figure 5.3. Spring Cloud Stream Consumer Groups</b></p><div class="figure-contents"><div class="mediaobject" align="center"><img src="https://raw.githubusercontent.com/spring-cloud/spring-cloud-stream/master/docs/src/main/asciidoc/images/SCSt-groups.png" align="middle" alt="SCSt groups"></div></div></div><br class="figure-break"><p>All groups that subscribe to a given destination receive a copy of published data, but only one member of each group receives a given message from that destination.
|
|
By default, when a group is not specified, Spring Cloud Stream assigns the application to an anonymous and independent single-member consumer group that is in a publish-subscribe relationship with all other consumer groups.</p></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="consumer-types" href="#consumer-types"></a>5.5 Consumer Types</h2></div></div></div><p>Two types of consumer are supported:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem">Message-driven (sometimes referred to as Asynchronous)</li><li class="listitem">Polled (sometimes referred to as Synchronous)</li></ul></div><p>Prior to version 2.0, only asynchronous consumers were supported. A message is delivered as soon as it is available and a thread is available to process it.</p><p>When you wish to control the rate at which messages are processed, you might want to use a synchronous consumer.</p><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="durability" href="#durability"></a>5.5.1 Durability</h3></div></div></div><p>Consistent with the opinionated application model of Spring Cloud Stream, consumer group subscriptions are durable.
|
|
That is, a binder implementation ensures that group subscriptions are persistent and that, once at least one subscription for a group has been created, the group receives messages, even if they are sent while all applications in the group are stopped.</p><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Note"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="images/note.png"></td><th align="left">Note</th></tr><tr><td align="left" valign="top"><p>Anonymous subscriptions are non-durable by nature.
|
|
For some binder implementations (such as RabbitMQ), it is possible to have non-durable group subscriptions.</p></td></tr></table></div><p>In general, it is preferable to always specify a consumer group when binding an application to a given destination.
|
|
When scaling up a Spring Cloud Stream application, you must specify a consumer group for each of its input bindings.
|
|
Doing so prevents the application’s instances from receiving duplicate messages (unless that behavior is desired, which is unusual).</p></div></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="partitioning" href="#partitioning"></a>5.6 Partitioning Support</h2></div></div></div><p>Spring Cloud Stream provides support for partitioning data between multiple instances of a given application.
|
|
In a partitioned scenario, the physical communication medium (such as the broker topic) is viewed as being structured into multiple partitions.
|
|
One or more producer application instances send data to multiple consumer application instances and ensure that data identified by common characteristics are processed by the same consumer instance.</p><p>Spring Cloud Stream provides a common abstraction for implementing partitioned processing use cases in a uniform fashion.
|
|
Partitioning can thus be used whether the broker itself is naturally partitioned (for example, Kafka) or not (for example, RabbitMQ).</p><div class="figure"><a name="d0e870" href="#d0e870"></a><p class="title"><b>Figure 5.4. Spring Cloud Stream Partitioning</b></p><div class="figure-contents"><div class="mediaobject" align="center"><img src="https://raw.githubusercontent.com/spring-cloud/spring-cloud-stream/master/docs/src/main/asciidoc/images/SCSt-partitioning.png" align="middle" alt="SCSt partitioning"></div></div></div><br class="figure-break"><p>Partitioning is a critical concept in stateful processing, where it is critical (for either performance or consistency reasons) to ensure that all related data is processed together.
|
|
For example, in the time-windowed average calculation example, it is important that all measurements from any given sensor are processed by the same application instance.</p><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Note"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="images/note.png"></td><th align="left">Note</th></tr><tr><td align="left" valign="top"><p>To set up a partitioned processing scenario, you must configure both the data-producing and the data-consuming ends.</p></td></tr></table></div></div></div><div class="chapter"><div class="titlepage"><div><div><h2 class="title"><a name="_programming_model" href="#_programming_model"></a>6. Programming Model</h2></div></div></div><p>To understand the programming model, you should be familiar with the following core concepts:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><span class="strong"><strong>Destination Binders:</strong></span> Components responsible to provide integration with the external messaging systems.</li><li class="listitem"><span class="strong"><strong>Destination Bindings:</strong></span> Bridge between the external messaging systems and application provided <span class="emphasis"><em>Producers</em></span> and <span class="emphasis"><em>Consumers</em></span> of messages (created by the Destination Binders).</li><li class="listitem"><span class="strong"><strong>Message:</strong></span> The canonical data structure used by producers and consumers to communicate with Destination Binders (and thus other applications via external messaging systems).</li></ul></div><div class="informalfigure"><div class="mediaobject" align="center"><img src="https://raw.githubusercontent.com/spring-cloud/spring-cloud-stream/master/docs/src/main/asciidoc/images/SCSt-overview.png" align="middle" alt="SCSt overview"></div></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="_destination_binders" href="#_destination_binders"></a>6.1 Destination Binders</h2></div></div></div><p>Destination Binders are extension components of Spring Cloud Stream responsible for providing the necessary configuration and implementation to facilitate
|
|
integration with external messaging systems.
|
|
This integration is responsible for connectivity, delegation, and routing of messages to and from producers and consumers, data type conversion,
|
|
invocation of the user code, and more.</p><p>Binders handle a lot of the boiler plate responsibilities that would otherwise fall on your shoulders. However, to accomplish that, the binder still needs
|
|
some help in the form of minimalistic yet required set of instructions from the user, which typically come in the form of some type of configuration.</p><p>While it is out of scope of this section to discuss all of the available binder and binding configuration options (the rest of the manual covers them extensively),
|
|
<span class="emphasis"><em>Destination Binding</em></span> does require special attention. The next section discusses it in detail.</p></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="_destination_bindings" href="#_destination_bindings"></a>6.2 Destination Bindings</h2></div></div></div><p>As stated earlier, <span class="emphasis"><em>Destination Bindings</em></span> provide a bridge between the external messaging system and application-provided <span class="emphasis"><em>Producers</em></span> and <span class="emphasis"><em>Consumers</em></span>.</p><p>Applying the @EnableBinding annotation to one of the application’s configuration classes defines a destination binding.
|
|
The <code class="literal">@EnableBinding</code> annotation itself is meta-annotated with <code class="literal">@Configuration</code> and triggers the configuration of the Spring Cloud Stream infrastructure.</p><p>The following example shows a fully configured and functioning Spring Cloud Stream application that receives the payload of the message from the <code class="literal">INPUT</code>
|
|
destination as a <code class="literal">String</code> type (see <a class="xref" href="#content-type-management" title="9. Content Type Negotiation">Chapter 9, <i>Content Type Negotiation</i></a> section), logs it to the console and sends it to the <code class="literal">OUTPUT</code> destination after converting it to upper case.</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@SpringBootApplication</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@EnableBinding(Processor.class)</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> MyApplication {
|
|
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">static</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">void</span> main(String[] args) {
|
|
SpringApplication.run(MyApplication.<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span>, args);
|
|
}
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@StreamListener(Processor.INPUT)</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@SendTo(Processor.OUTPUT)</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> String handle(String value) {
|
|
System.out.println(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"Received: "</span> + value);
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> value.toUpperCase();
|
|
}
|
|
}</pre><p>As you can see the <code class="literal">@EnableBinding</code> annotation can take one or more interface classes as parameters. The parameters are referred to as <span class="emphasis"><em>bindings</em></span>,
|
|
and they contain methods representing <span class="emphasis"><em>bindable components</em></span>.
|
|
These components are typically message channels (see <a class="link" href="https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-messaging.html" target="_top">Spring Messaging</a>)
|
|
for channel-based binders (such as Rabbit, Kafka, and others). However other types of bindings can
|
|
provide support for the native features of the corresponding technology. For example Kafka Streams binder (formerly known as KStream) allows native bindings directly to Kafka Streams
|
|
(see <a class="link" href="https://docs.spring.io/autorepo/docs/spring-cloud-stream-binder-kafka-docs/1.1.0.M1/reference/htmlsingle/" target="_top">Kafka Streams</a> for more details).</p><p>Spring Cloud Stream already provides <span class="emphasis"><em>binding</em></span> interfaces for typical message exchange contracts, which include:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><span class="strong"><strong>Sink:</strong></span> Identifies the contract for the message consumer by providing the destination from which the message is consumed.</li><li class="listitem"><span class="strong"><strong>Source:</strong></span> Identifies the contract for the message producer by providing the destination to which the produced message is sent.</li><li class="listitem"><span class="strong"><strong>Processor:</strong></span> Encapsulates both the sink and the source contracts by exposing two destinations that allow consumption and production of messages.</li></ul></div><pre class="programlisting"><span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">interface</span> Sink {
|
|
|
|
String INPUT = <span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"input"</span>;
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Input(Sink.INPUT)</span></em>
|
|
SubscribableChannel input();
|
|
}</pre><pre class="programlisting"><span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">interface</span> Source {
|
|
|
|
String OUTPUT = <span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"output"</span>;
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Output(Source.OUTPUT)</span></em>
|
|
MessageChannel output();
|
|
}</pre><pre class="programlisting"><span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">interface</span> Processor <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">extends</span> Source, Sink {}</pre><p>While the preceding example satisfies the majority of cases, you can also define your own contracts by defining your own bindings interfaces and use <code class="literal">@Input</code> and <code class="literal">@Output</code>
|
|
annotations to identify the actual <span class="emphasis"><em>bindable components</em></span>.</p><p>For example:</p><pre class="programlisting"><span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">interface</span> Barista {
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Input</span></em>
|
|
SubscribableChannel orders();
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Output</span></em>
|
|
MessageChannel hotDrinks();
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Output</span></em>
|
|
MessageChannel coldDrinks();
|
|
}</pre><p>Using the interface shown in the preceding example as a parameter to <code class="literal">@EnableBinding</code> triggers the creation of the three bound channels named <code class="literal">orders</code>, <code class="literal">hotDrinks</code>, and <code class="literal">coldDrinks</code>,
|
|
respectively.</p><p>You can provide as many binding interfaces as you need, as arguments to the <code class="literal">@EnableBinding</code> annotation, as shown in the following example:</p><pre class="programlisting">@EnableBinding(value = { Orders.<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span>, Payment.<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> })</pre><p>In Spring Cloud Stream, the bindable <code class="literal">MessageChannel</code> components are the Spring Messaging <code class="literal">MessageChannel</code> (for outbound) and its extension, <code class="literal">SubscribableChannel</code>,
|
|
(for inbound).</p><p><span class="strong"><strong>Pollable Destination Binding</strong></span></p><p>While the previously described bindings support event-based message consumption, sometimes you need more control, such as rate of consumption.</p><p>Starting with version 2.0, you can now bind a pollable consumer:</p><p>The following example shows how to bind a pollable consumer:</p><pre class="programlisting"><span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">interface</span> PolledBarista {
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Input</span></em>
|
|
PollableMessageSource orders();
|
|
. . .
|
|
}</pre><p>In this case, an implementation of <code class="literal">PollableMessageSource</code> is bound to the <code class="literal">orders</code> “channel”. See <a class="xref" href="#spring-cloud-streams-overview-using-polled-consumers" title="6.3.5 Using Polled Consumers">Section 6.3.5, “Using Polled Consumers”</a> for more details.</p><p><span class="strong"><strong>Customizing Channel Names</strong></span></p><p>By using the <code class="literal">@Input</code> and <code class="literal">@Output</code> annotations, you can specify a customized channel name for the channel, as shown in the following example:</p><pre class="programlisting"><span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">interface</span> Barista {
|
|
<em><span class="hl-annotation" style="color: gray">@Input("inboundOrders")</span></em>
|
|
SubscribableChannel orders();
|
|
}</pre><p>In the preceding example, the created bound channel is named <code class="literal">inboundOrders</code>.</p><p>Normally, you need not access individual channels or bindings directly (other then configuring them via <code class="literal">@EnableBinding</code> annotation). However there may be
|
|
times, such as testing or other corner cases, when you do.</p><p>Aside from generating channels for each binding and registering them as Spring beans, for each bound interface, Spring Cloud Stream generates a bean that implements the interface.
|
|
That means you can have access to the interfaces representing the bindings or individual channels by auto-wiring either in your application, as shown in the following two examples:</p><p><span class="emphasis"><em>Autowire Binding interface</em></span></p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@Autowire</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">private</span> Source source
|
|
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">void</span> sayHello(String name) {
|
|
source.output().send(MessageBuilder.withPayload(name).build());
|
|
}</pre><p><span class="emphasis"><em>Autowire individual channel</em></span></p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@Autowire</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">private</span> MessageChannel output;
|
|
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">void</span> sayHello(String name) {
|
|
output.send(MessageBuilder.withPayload(name).build());
|
|
}</pre><p>You can also use standard Spring’s <code class="literal">@Qualifier</code> annotation for cases when channel names are customized or in multiple-channel scenarios that require specifically named channels.</p><p>The following example shows how to use the @Qualifier annotation in this way:</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@Autowire</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@Qualifier("myChannel")</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">private</span> MessageChannel output;</pre></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="spring-cloud-stream-overview-producing-consuming-messages" href="#spring-cloud-stream-overview-producing-consuming-messages"></a>6.3 Producing and Consuming Messages</h2></div></div></div><p>You can write a Spring Cloud Stream application by using either Spring Integration annotations or Spring Cloud Stream native annotation.</p><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="_spring_integration_support" href="#_spring_integration_support"></a>6.3.1 Spring Integration Support</h3></div></div></div><p>Spring Cloud Stream is built on the concepts and patterns defined by <a class="link" href="http://www.enterpriseintegrationpatterns.com/" target="_top">Enterprise Integration Patterns</a> and relies
|
|
in its internal implementation on an already established and popular implementation of Enterprise Integration Patterns within the Spring portfolio of projects:
|
|
<a class="link" href="https://projects.spring.io/spring-integration/" target="_top">Spring Integration</a> framework.</p><p>So its only natural for it to support the foundation, semantics, and configuration options that are already established by Spring Integration</p><p>For example, you can attach the output channel of a <code class="literal">Source</code> to a <code class="literal">MessageSource</code> and use the familiar <code class="literal">@InboundChannelAdapter</code> annotation, as follows:</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@EnableBinding(Source.class)</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> TimerSource {
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Bean</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@InboundChannelAdapter(value = Source.OUTPUT, poller = @Poller(fixedDelay = "10", maxMessagesPerPoll = "1"))</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> MessageSource<String> timerMessageSource() {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> () -> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">new</span> GenericMessage<>(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"Hello Spring Cloud Stream"</span>);
|
|
}
|
|
}</pre><p>Similarly, you can use @Transformer or @ServiceActivator while providing an implementation of a message handler method for a <span class="emphasis"><em>Processor</em></span> binding contract, as shown in the following example:</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@EnableBinding(Processor.class)</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> TransformProcessor {
|
|
<em><span class="hl-annotation" style="color: gray">@Transformer(inputChannel = Processor.INPUT, outputChannel = Processor.OUTPUT)</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> Object transform(String message) {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> message.toUpperCase();
|
|
}
|
|
}</pre><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Note"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="images/note.png"></td><th align="left">Note</th></tr><tr><td align="left" valign="top"><p>While this may be skipping ahead a bit, it is important to understand that, when you consume from the same binding using <code class="literal">@StreamListener</code> annotation, a pub-sub model is used.
|
|
Each method annotated with <code class="literal">@StreamListener</code> receives its own copy of a message, and each one has its own consumer group.
|
|
However, if you consume from the same binding by using one of the Spring Integration annotation (such as <code class="literal">@Aggregator</code>, <code class="literal">@Transformer</code>, or <code class="literal">@ServiceActivator</code>), those consume in a competing model.
|
|
No individual consumer group is created for each subscription.</p></td></tr></table></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="_using_streamlistener_annotation" href="#_using_streamlistener_annotation"></a>6.3.2 Using @StreamListener Annotation</h3></div></div></div><p>Complementary to its Spring Integration support, Spring Cloud Stream provides its own <code class="literal">@StreamListener</code> annotation, modeled after other Spring Messaging annotations
|
|
(<code class="literal">@MessageMapping</code>, <code class="literal">@JmsListener</code>, <code class="literal">@RabbitListener</code>, and others) and provides conviniences, such as content-based routing and others.</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@EnableBinding(Sink.class)</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> VoteHandler {
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Autowired</span></em>
|
|
VotingService votingService;
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@StreamListener(Sink.INPUT)</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">void</span> handle(Vote vote) {
|
|
votingService.record(vote);
|
|
}
|
|
}</pre><p>As with other Spring Messaging methods, method arguments can be annotated with <code class="literal">@Payload</code>, <code class="literal">@Headers</code>, and <code class="literal">@Header</code>.</p><p>For methods that return data, you must use the <code class="literal">@SendTo</code> annotation to specify the output binding destination for data returned by the method, as shown in the following example:</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@EnableBinding(Processor.class)</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> TransformProcessor {
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Autowired</span></em>
|
|
VotingService votingService;
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@StreamListener(Processor.INPUT)</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@SendTo(Processor.OUTPUT)</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> VoteResult handle(Vote vote) {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> votingService.record(vote);
|
|
}
|
|
}</pre></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="_using_streamlistener_for_content_based_routing" href="#_using_streamlistener_for_content_based_routing"></a>6.3.3 Using @StreamListener for Content-based routing</h3></div></div></div><p>Spring Cloud Stream supports dispatching messages to multiple handler methods annotated with <code class="literal">@StreamListener</code> based on conditions.</p><p>In order to be eligible to support conditional dispatching, a method must satisfy the follow conditions:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem">It must not return a value.</li><li class="listitem">It must be an individual message handling method (reactive API methods are not supported).</li></ul></div><p>The condition is specified by a SpEL expression in the <code class="literal">condition</code> argument of the annotation and is evaluated for each message.
|
|
All the handlers that match the condition are invoked in the same thread, and no assumption must be made about the order in which the invocations take place.</p><p>In the following example of a <code class="literal">@StreamListener</code> with dispatching conditions, all the messages bearing a header <code class="literal">type</code> with the value <code class="literal">bogey</code> are dispatched to the
|
|
<code class="literal">receiveBogey</code> method, and all the messages bearing a header <code class="literal">type</code> with the value <code class="literal">bacall</code> are dispatched to the <code class="literal">receiveBacall</code> method.</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@EnableBinding(Sink.class)</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@EnableAutoConfiguration</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">static</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> TestPojoWithAnnotatedArguments {
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@StreamListener(target = Sink.INPUT, condition = "headers['type']=='bogey'")</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">void</span> receiveBogey(<em><span class="hl-annotation" style="color: gray">@Payload</span></em> BogeyPojo bogeyPojo) {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-comment">// handle the message</span>
|
|
}
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@StreamListener(target = Sink.INPUT, condition = "headers['type']=='bacall'")</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">void</span> receiveBacall(<em><span class="hl-annotation" style="color: gray">@Payload</span></em> BacallPojo bacallPojo) {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-comment">// handle the message</span>
|
|
}
|
|
}</pre><p><span class="strong"><strong>Content Type Negotiation in the Context of <code class="literal">condition</code></strong></span></p><p>It is important to understand some of the mechanics behind content-based routing using the <code class="literal">condition</code> argument of <code class="literal">@StreamListener</code>, especially in the context of the type of the message as a whole.
|
|
It may also help if you familiarize yourself with the <a class="xref" href="#content-type-management" title="9. Content Type Negotiation">Chapter 9, <i>Content Type Negotiation</i></a> before you proceed.</p><p>Consider the following scenario:</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@EnableBinding(Sink.class)</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@EnableAutoConfiguration</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">static</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> CatsAndDogs {
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@StreamListener(target = Sink.INPUT, condition = "payload.class.simpleName=='Dog'")</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">void</span> bark(Dog dog) {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-comment">// handle the message</span>
|
|
}
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@StreamListener(target = Sink.INPUT, condition = "payload.class.simpleName=='Cat'")</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">void</span> purr(Cat cat) {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-comment">// handle the message</span>
|
|
}
|
|
}</pre><p>The preceding code is perfectly valid. It compiles and deploys without any issues, yet it never produces the result you expect.</p><p>That is because you are testing something that does not yet exist in a state you expect. That is because the payload of the message is not yet converted from the
|
|
wire format (<code class="literal">byte[]</code>) to the desired type.
|
|
In other words, it has not yet gone through the type conversion process described in the <a class="xref" href="#content-type-management" title="9. Content Type Negotiation">Chapter 9, <i>Content Type Negotiation</i></a>.</p><p>So, unless you use a SPeL expression that evaluates raw data (for example, the value of the first byte in the byte array), use message header-based expressions
|
|
(such as <code class="literal">condition = "headers['type']=='dog'"</code>).</p><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Note"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="images/note.png"></td><th align="left">Note</th></tr><tr><td align="left" valign="top"><p>At the moment, dispatching through <code class="literal">@StreamListener</code> conditions is supported only for channel-based binders (not for reactive programming)
|
|
support.</p></td></tr></table></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="_spring_cloud_function" href="#_spring_cloud_function"></a>6.3.4 Spring Cloud Function support</h3></div></div></div><p>Since Spring Cloud Stream v2.1, another alternative for defining <span class="emphasis"><em>stream handlers</em></span> and <span class="emphasis"><em>sources</em></span> is to use build-in
|
|
support for <a class="link" href="https://cloud.spring.io/spring-cloud-function/" target="_top">Spring Cloud Function</a> where they can be expressed as beans of
|
|
type <code class="literal">java.util.function.[Supplier/Function/Consumer]</code>.</p><p>To specify which functional bean to bind to the external destination(s) exposed by the bindings, you must provide <code class="literal">spring.cloud.stream.function.definition</code> property.</p><p>Here is the example of the Processor application exposing message handler as <code class="literal">java.util.function.Function</code></p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@SpringBootApplication</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@EnableBinding(Processor.class)</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> MyFunctionBootApp {
|
|
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">static</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">void</span> main(String[] args) {
|
|
SpringApplication.run(MyFunctionBootApp.<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span>, <span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"--spring.cloud.stream.function.definition=toUpperCase"</span>);
|
|
}
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Bean</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> Function<String, String> toUpperCase() {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> s -> s.toUpperCase();
|
|
}
|
|
}</pre><p>In the above you we simply define a bean of type <code class="literal">java.util.function.Function</code> called <span class="emphasis"><em>toUpperCase</em></span> and identify it as a bean to be used as message handler
|
|
whose 'input' and 'output' must be bound to the external destinations exposed by the Processor binding.</p><p>Below are the examples of simple functional applications to support Source, Processor and Sink.</p><p>Here is the example of a Source application defined as <code class="literal">java.util.function.Supplier</code></p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@SpringBootApplication</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@EnableBinding(Source.class)</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">static</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> SourceFromSupplier {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">static</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">void</span> main(String[] args) {
|
|
SpringApplication.run(SourceFromSupplier.<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span>, <span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"--spring.cloud.stream.function.definition=date"</span>);
|
|
}
|
|
<em><span class="hl-annotation" style="color: gray">@Bean</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> Supplier<Date> date() {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> () -> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">new</span> Date(<span class="hl-number">12345L</span>);
|
|
}
|
|
}</pre><p>Here is the example of a Processor application defined as <code class="literal">java.util.function.Function</code></p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@SpringBootApplication</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@EnableBinding(Processor.class)</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">static</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> ProcessorFromFunction {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">static</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">void</span> main(String[] args) {
|
|
SpringApplication.run(ProcessorFromFunction.<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span>, <span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"--spring.cloud.stream.function.definition=toUpperCase"</span>);
|
|
}
|
|
<em><span class="hl-annotation" style="color: gray">@Bean</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> Function<String, String> toUpperCase() {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> s -> s.toUpperCase();
|
|
}
|
|
}</pre><p>Here is the example of a Sink application defined as <code class="literal">java.util.function.Consumer</code></p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@EnableAutoConfiguration</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@EnableBinding(Sink.class)</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">static</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> SinkFromConsumer {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">static</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">void</span> main(String[] args) {
|
|
SpringApplication.run(SinkFromConsumer.<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span>, <span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"--spring.cloud.stream.function.definition=sink"</span>);
|
|
}
|
|
<em><span class="hl-annotation" style="color: gray">@Bean</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> Consumer<String> sink() {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> System.out::println;
|
|
}
|
|
}</pre><div class="section"><div class="titlepage"><div><div><h4 class="title"><a name="_functional_composition" href="#_functional_composition"></a>Functional Composition</h4></div></div></div><p>Using this programming model you can also benefit from functional composition where you can dynamically compose complex handlers from a set of simple functions.
|
|
As an example let’s add the following function bean to the application defined above</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@Bean</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> Function<String, String> wrapInQuotes() {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> s -> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"\""</span> + s + <span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"\""</span>;
|
|
}</pre><p>and modify the <code class="literal">spring.cloud.stream.function.definition</code> property to reflect your intention to compose a new function from both ‘toUpperCase’ and ‘wrapInQuotes’.
|
|
To do that Spring Cloud Function allows you to use <code class="literal">|</code> (pipe) symbol. So to finish our example our property will now look like this:</p><pre class="programlisting">—spring.cloud.stream.function.definition=toUpperCase|wrapInQuotes</pre></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="spring-cloud-streams-overview-using-polled-consumers" href="#spring-cloud-streams-overview-using-polled-consumers"></a>6.3.5 Using Polled Consumers</h3></div></div></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a name="_overview" href="#_overview"></a>Overview</h4></div></div></div><p>When using polled consumers, you poll the <code class="literal">PollableMessageSource</code> on demand.
|
|
Consider the following example of a polled consumer:</p><pre class="programlisting"><span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">interface</span> PolledConsumer {
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Input</span></em>
|
|
PollableMessageSource destIn();
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Output</span></em>
|
|
MessageChannel destOut();
|
|
|
|
}</pre><p>Given the polled consumer in the preceding example, you might use it as follows:</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@Bean</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> ApplicationRunner poller(PollableMessageSource destIn, MessageChannel destOut) {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> args -> {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">while</span> (someCondition()) {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">try</span> {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">if</span> (!destIn.poll(m -> {
|
|
String newPayload = ((String) m.getPayload()).toUpperCase();
|
|
destOut.send(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">new</span> GenericMessage<>(newPayload));
|
|
})) {
|
|
Thread.sleep(<span class="hl-number">1000</span>);
|
|
}
|
|
}
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">catch</span> (Exception e) {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-comment">// handle failure</span>
|
|
}
|
|
}
|
|
};
|
|
}</pre><p>The <code class="literal">PollableMessageSource.poll()</code> method takes a <code class="literal">MessageHandler</code> argument (often a lambda expression, as shown here).
|
|
It returns <code class="literal">true</code> if the message was received and successfully processed.</p><p>As with message-driven consumers, if the <code class="literal">MessageHandler</code> throws an exception, messages are published to error channels,
|
|
as discussed in <code class="literal"><a class="xref" href="#spring-cloud-stream-overview-error-handling" title="6.4 Error Handling">Section 6.4, “Error Handling”</a></code>.</p><p>Normally, the <code class="literal">poll()</code> method acknowledges the message when the <code class="literal">MessageHandler</code> exits.
|
|
If the method exits abnormally, the message is rejected (not re-queued), but see <a class="xref" href="#polled-errors" title="Handling Errors">the section called “Handling Errors”</a>.
|
|
You can override that behavior by taking responsibility for the acknowledgment, as shown in the following example:</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@Bean</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> ApplicationRunner poller(PollableMessageSource dest1In, MessageChannel dest2Out) {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> args -> {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">while</span> (someCondition()) {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">if</span> (!dest1In.poll(m -> {
|
|
StaticMessageHeaderAccessor.getAcknowledgmentCallback(m).noAutoAck();
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-comment">// e.g. hand off to another thread which can perform the ack</span>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-comment">// or acknowledge(Status.REQUEUE)</span>
|
|
|
|
})) {
|
|
Thread.sleep(<span class="hl-number">1000</span>);
|
|
}
|
|
}
|
|
};
|
|
}</pre><div class="important" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Important"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Important]" src="images/important.png"></td><th align="left">Important</th></tr><tr><td align="left" valign="top"><p>You must <code class="literal">ack</code> (or <code class="literal">nack</code>) the message at some point, to avoid resource leaks.</p></td></tr></table></div><div class="important" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Important"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Important]" src="images/important.png"></td><th align="left">Important</th></tr><tr><td align="left" valign="top"><p>Some messaging systems (such as Apache Kafka) maintain a simple offset in a log. If a delivery fails and is re-queued with <code class="literal">StaticMessageHeaderAccessor.getAcknowledgmentCallback(m).acknowledge(Status.REQUEUE);</code>, any later successfully ack’d messages are redelivered.</p></td></tr></table></div><p>There is also an overloaded <code class="literal">poll</code> method, for which the definition is as follows:</p><pre class="programlisting">poll(MessageHandler handler, ParameterizedTypeReference<?> type)</pre><p>The <code class="literal">type</code> is a conversion hint that allows the incoming message payload to be converted, as shown in the following example:</p><pre class="programlisting"><span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">boolean</span> result = pollableSource.poll(received -> {
|
|
Map<String, Foo> payload = (Map<String, Foo>) received.getPayload();
|
|
...
|
|
|
|
}, <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">new</span> ParameterizedTypeReference<Map<String, Foo>>() {});</pre></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a name="polled-errors" href="#polled-errors"></a>Handling Errors</h4></div></div></div><p>By default, an error channel is configured for the pollable source; if the callback throws an exception, an <code class="literal">ErrorMessage</code> is sent to the error channel (<code class="literal"><destination>.<group>.errors</code>); this error channel is also bridged to the global Spring Integration <code class="literal">errorChannel</code>.</p><p>You can subscribe to either error channel with a <code class="literal">@ServiceActivator</code> to handle errors; without a subscription, the error will simply be logged and the message will be acknowledged as successful.
|
|
If the error channel service activator throws an exception, the message will be rejected (by default) and won’t be redelivered.
|
|
If the service activator throws a <code class="literal">RequeueCurrentMessageException</code>, the message will be requeued at the broker and will be again retrieved on a subsequent poll.</p><p>If the listener throws a <code class="literal">RequeueCurrentMessageException</code> directly, the message will be requeued, as discussed above, and will not be sent to the error channels.</p></div></div></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="spring-cloud-stream-overview-error-handling" href="#spring-cloud-stream-overview-error-handling"></a>6.4 Error Handling</h2></div></div></div><p>Errors happen, and Spring Cloud Stream provides several flexible mechanisms to handle them.
|
|
The error handling comes in two flavors:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><span class="strong"><strong>application:</strong></span> The error handling is done within the application (custom error handler).</li><li class="listitem"><span class="strong"><strong>system:</strong></span> The error handling is delegated to the binder (re-queue, DL, and others). Note that the techniques are dependent on binder implementation and the
|
|
capability of the underlying messaging middleware.</li></ul></div><p>Spring Cloud Stream uses the <a class="link" href="https://github.com/spring-projects/spring-retry" target="_top">Spring Retry</a> library to facilitate successful message processing. See <a class="xref" href="#_retry_template" title="6.4.3 Retry Template">Section 6.4.3, “Retry Template”</a> for more details.
|
|
However, when all fails, the exceptions thrown by the message handlers are propagated back to the binder. At that point, binder invokes custom error handler or communicates
|
|
the error back to the messaging system (re-queue, DLQ, and others).</p><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="_application_error_handling" href="#_application_error_handling"></a>6.4.1 Application Error Handling</h3></div></div></div><p>There are two types of application-level error handling. Errors can be handled at each binding subscription or a global handler can handle all the binding subscription errors. Let’s review the details.</p><div class="figure"><a name="d0e1507" href="#d0e1507"></a><p class="title"><b>Figure 6.1. A Spring Cloud Stream Sink Application with Custom and Global Error Handlers</b></p><div class="figure-contents"><div class="mediaobject" align="center"><img src="https://raw.githubusercontent.com/spring-cloud/spring-cloud-stream/master/docs/src/main/asciidoc/images/custom_vs_global_error_channels.png" align="middle" alt="custom vs global error channels"></div></div></div><br class="figure-break"><p>For each input binding, Spring Cloud Stream creates a dedicated error channel with the following semantics <code class="literal"><destinationName>.errors</code>.</p><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Note"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="images/note.png"></td><th align="left">Note</th></tr><tr><td align="left" valign="top"><p>The <code class="literal"><destinationName></code> consists of the name of the binding (such as <code class="literal">input</code>) and the name of the group (such as <code class="literal">myGroup</code>).</p></td></tr></table></div><p>Consider the following:</p><pre class="programlisting">spring.cloud.stream.bindings.input.group=myGroup</pre><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@StreamListener(Sink.INPUT)</span></em> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-comment">// destination name 'input.myGroup'</span>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">void</span> handle(Person value) {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">throw</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">new</span> RuntimeException(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"BOOM!"</span>);
|
|
}
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@ServiceActivator(inputChannel = Processor.INPUT + ".myGroup.errors")</span></em> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-comment">//channel name 'input.myGroup.errors'</span>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">void</span> error(Message<?> message) {
|
|
System.out.println(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"Handling ERROR: "</span> + message);
|
|
}</pre><p>In the preceding example the destination name is <code class="literal">input.myGroup</code> and the dedicated error channel name is <code class="literal">input.myGroup.errors</code>.</p><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Note"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="images/note.png"></td><th align="left">Note</th></tr><tr><td align="left" valign="top"><p>The use of @StreamListener annotation is intended specifically to define bindings that bridge internal channels and external destinations. Given that the destination
|
|
specific error channel does NOT have an associated external destination, such channel is a prerogative of Spring Integration (SI). This means that the handler
|
|
for such destination must be defined using one of the SI handler annotations (i.e., @ServiceActivator, @Transformer etc.).</p></td></tr></table></div><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Note"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="images/note.png"></td><th align="left">Note</th></tr><tr><td align="left" valign="top"><p>If <code class="literal">group</code> is not specified anonymous group is used (something like <code class="literal">input.anonymous.2K37rb06Q6m2r51-SPIDDQ</code>), which is not suitable for error
|
|
handling scenarious, since you don’t know what it’s going to be until the destination is created.</p></td></tr></table></div><p>Also, in the event you are binding to the existing destination such as:</p><pre class="programlisting">spring.cloud.stream.bindings.input.destination=myFooDestination
|
|
spring.cloud.stream.bindings.input.group=myGroup</pre><p>the full destination name is <code class="literal">myFooDestination.myGroup</code> and then the dedicated error channel name is <code class="literal">myFooDestination.myGroup.errors</code>.</p><p>Back to the example…​</p><p>The <code class="literal">handle(..)</code> method, which subscribes to the channel named <code class="literal">input</code>, throws an exception. Given there is also a subscriber to the error channel <code class="literal">input.myGroup.errors</code>
|
|
all error messages are handled by this subscriber.</p><p>If you have multiple bindings, you may want to have a single error handler. Spring Cloud Stream automatically provides support for
|
|
a <span class="emphasis"><em>global error channel</em></span> by bridging each individual error channel to the channel named <code class="literal">errorChannel</code>, allowing a single subscriber to handle all errors,
|
|
as shown in the following example:</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@StreamListener("errorChannel")</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">void</span> error(Message<?> message) {
|
|
System.out.println(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"Handling ERROR: "</span> + message);
|
|
}</pre><p>This may be a convenient option if error handling logic is the same regardless of which handler produced the error.</p></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="_system_error_handling" href="#_system_error_handling"></a>6.4.2 System Error Handling</h3></div></div></div><p>System-level error handling implies that the errors are communicated back to the messaging system and, given that not every messaging system
|
|
is the same, the capabilities may differ from binder to binder.</p><p>That said, in this section we explain the general idea behind system level error handling and use Rabbit binder as an example. NOTE: Kafka binder provides similar
|
|
support, although some configuration properties do differ. Also, for more details and configuration options, see the individual binder’s documentation.</p><p>If no internal error handlers are configured, the errors propagate to the binders, and the binders subsequently propagate those errors back to the messaging system.
|
|
Depending on the capabilities of the messaging system such a system may <span class="emphasis"><em>drop</em></span> the message, <span class="emphasis"><em>re-queue</em></span> the message for re-processing or <span class="emphasis"><em>send the failed message to DLQ</em></span>.
|
|
Both Rabbit and Kafka support these concepts. However, other binders may not, so refer to your individual binder’s documentation for details on supported system-level
|
|
error-handling options.</p><div class="section"><div class="titlepage"><div><div><h4 class="title"><a name="_drop_failed_messages" href="#_drop_failed_messages"></a>Drop Failed Messages</h4></div></div></div><p>By default, if no additional system-level configuration is provided, the messaging system drops the failed message.
|
|
While acceptable in some cases, for most cases, it is not, and we need some recovery mechanism to avoid message loss.</p></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a name="_dlq_dead_letter_queue" href="#_dlq_dead_letter_queue"></a>DLQ - Dead Letter Queue</h4></div></div></div><p>DLQ allows failed messages to be sent to a special destination: - <span class="emphasis"><em>Dead Letter Queue</em></span>.</p><p>When configured, failed messages are sent to this destination for subsequent re-processing or auditing and reconciliation.</p><p>For example, continuing on the previous example and to set up the DLQ with Rabbit binder, you need to set the following property:</p><pre class="programlisting">spring.cloud.stream.rabbit.bindings.input.consumer.auto-bind-dlq=true</pre><p>Keep in mind that, in the above property, <code class="literal">input</code> corresponds to the name of the input destination binding.
|
|
The <code class="literal">consumer</code> indicates that it is a consumer property and <code class="literal">auto-bind-dlq</code> instructs the binder to configure DLQ for <code class="literal">input</code>
|
|
destination, which results in an additional Rabbit queue named <code class="literal">input.myGroup.dlq</code>.</p><p>Once configured, all failed messages are routed to this queue with an error message similar to the following:</p><pre class="programlisting">delivery_mode: 1
|
|
headers:
|
|
x-death:
|
|
count: 1
|
|
reason: rejected
|
|
queue: input.hello
|
|
time: 1522328151
|
|
exchange:
|
|
routing-keys: input.myGroup
|
|
Payload {"name”:"Bob"}</pre><p>As you can see from the above, your original message is preserved for further actions.</p><p>However, one thing you may have noticed is that there is limited information on the original issue with the message processing. For example, you do not see a stack
|
|
trace corresponding to the original error.
|
|
To get more relevant information about the original error, you must set an additional property:</p><pre class="programlisting">spring.cloud.stream.rabbit.bindings.input.consumer.republish-to-dlq=true</pre><p>Doing so forces the internal error handler to intercept the error message and add additional information to it before publishing it to DLQ.
|
|
Once configured, you can see that the error message contains more information relevant to the original error, as follows:</p><pre class="programlisting">delivery_mode: 2
|
|
headers:
|
|
x-original-exchange:
|
|
x-exception-message: has an error
|
|
x-original-routingKey: input.myGroup
|
|
x-exception-stacktrace: org.springframework.messaging.MessageHandlingException: nested exception is
|
|
org.springframework.messaging.MessagingException: has an error, failedMessage=GenericMessage [payload=byte[15],
|
|
headers={amqp_receivedDeliveryMode=NON_PERSISTENT, amqp_receivedRoutingKey=input.hello, amqp_deliveryTag=1,
|
|
deliveryAttempt=3, amqp_consumerQueue=input.hello, amqp_redelivered=false, id=a15231e6-3f80-677b-5ad7-d4b1e61e486e,
|
|
amqp_consumerTag=amq.ctag-skBFapilvtZhDsn0k3ZmQg, contentType=application/json, timestamp=1522327846136}]
|
|
at org.spring...integ...han...MethodInvokingMessageProcessor.processMessage(MethodInvokingMessageProcessor.java:107)
|
|
at. . . . .
|
|
Payload {"name”:"Bob"}</pre><p>This effectively combines application-level and system-level error handling to further assist with downstream troubleshooting mechanics.</p></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a name="_re_queue_failed_messages" href="#_re_queue_failed_messages"></a>Re-queue Failed Messages</h4></div></div></div><p>As mentioned earlier, the currently supported binders (Rabbit and Kafka) rely on <code class="literal">RetryTemplate</code> to facilitate successful message processing. See <a class="xref" href="#_retry_template" title="6.4.3 Retry Template">Section 6.4.3, “Retry Template”</a> for details.
|
|
However, for cases when <code class="literal">max-attempts</code> property is set to 1, internal reprocessing of the message is disabled. At this point, you can facilitate message re-processing (re-tries)
|
|
by instructing the messaging system to re-queue the failed message. Once re-queued, the failed message is sent back to the original handler, essentially creating a retry loop.</p><p>This option may be feasible for cases where the nature of the error is related to some sporadic yet short-term unavailability of some resource.</p><p>To accomplish that, you must set the following properties:</p><pre class="programlisting">spring.cloud.stream.bindings.input.consumer.max-attempts=1
|
|
spring.cloud.stream.rabbit.bindings.input.consumer.requeue-rejected=true</pre><p>In the preceding example, the <code class="literal">max-attempts</code> set to 1 essentially disabling internal re-tries and <code class="literal">requeue-rejected</code> (short for <span class="emphasis"><em>requeue rejected messages</em></span>) is set to <code class="literal">true</code>.
|
|
Once set, the failed message is resubmitted to the same handler and loops continuously or until the handler throws <code class="literal">AmqpRejectAndDontRequeueException</code>
|
|
essentially allowing you to build your own re-try logic within the handler itself.</p></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="_retry_template" href="#_retry_template"></a>6.4.3 Retry Template</h3></div></div></div><p>The <code class="literal">RetryTemplate</code> is part of the <a class="link" href="https://github.com/spring-projects/spring-retry" target="_top">Spring Retry</a> library.
|
|
While it is out of scope of this document to cover all of the capabilities of the <code class="literal">RetryTemplate</code>, we will mention the following consumer properties that are specifically related to
|
|
the <code class="literal">RetryTemplate</code>:</p><div class="variablelist"><dl class="variablelist"><dt><span class="term">maxAttempts</span></dt><dd><p class="simpara">The number of attempts to process the message.</p><p class="simpara">Default: 3.</p></dd><dt><span class="term">backOffInitialInterval</span></dt><dd><p class="simpara">The backoff initial interval on retry.</p><p class="simpara">Default 1000 milliseconds.</p></dd><dt><span class="term">backOffMaxInterval</span></dt><dd><p class="simpara">The maximum backoff interval.</p><p class="simpara">Default 10000 milliseconds.</p></dd><dt><span class="term">backOffMultiplier</span></dt><dd><p class="simpara">The backoff multiplier.</p><p class="simpara">Default 2.0.</p></dd><dt><span class="term">defaultRetryable</span></dt><dd><p class="simpara">Whether exceptions thrown by the listener that are not listed in the <code class="literal">retryableExceptions</code> are retryable.</p><p class="simpara">Default: <code class="literal">true</code>.</p></dd><dt><span class="term">retryableExceptions</span></dt><dd><p class="simpara">A map of Throwable class names in the key and a boolean in the value.
|
|
Specify those exceptions (and subclasses) that will or won’t be retried.
|
|
Also see <code class="literal">defaultRetriable</code>.
|
|
Example: <code class="literal">spring.cloud.stream.bindings.input.consumer.retryable-exceptions.java.lang.IllegalStateException=false</code>.</p><p class="simpara">Default: empty.</p></dd></dl></div><p>While the preceding settings are sufficient for majority of the customization requirements, they may not satisfy certain complex requirements at, which
|
|
point you may want to provide your own instance of the <code class="literal">RetryTemplate</code>. To do so configure it as a bean in your application configuration. The application provided
|
|
instance will override the one provided by the framework. Also, to avoid conflicts you must qualify the instance of the <code class="literal">RetryTemplate</code> you want to be used by the binder
|
|
as <code class="literal">@StreamRetryTemplate</code>. For example,</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@StreamRetryTemplate</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> RetryTemplate myRetryTemplate() {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">new</span> RetryTemplate();
|
|
}</pre><p>As you can see from the above example you don’t need to annotate it with <code class="literal">@Bean</code> since <code class="literal">@StreamRetryTemplate</code> is a qualified <code class="literal">@Bean</code>.</p></div></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="spring-cloud-stream-overview-reactive-programming-support" href="#spring-cloud-stream-overview-reactive-programming-support"></a>6.5 Reactive Programming Support</h2></div></div></div><p>Spring Cloud Stream also supports the use of reactive APIs where incoming and outgoing data is handled as continuous data flows.
|
|
Support for reactive APIs is available through <code class="literal">spring-cloud-stream-reactive</code>, which needs to be added explicitly to your project.</p><p>The programming model with reactive APIs is declarative. Instead of specifying how each individual message should be handled, you can use operators that describe functional transformations from inbound to outbound data flows.</p><p>At present Spring Cloud Stream supports the only the <a class="link" href="https://projectreactor.io/" target="_top">Reactor API</a>.
|
|
In the future, we intend to support a more generic model based on Reactive Streams.</p><p>The reactive programming model also uses the <code class="literal">@StreamListener</code> annotation for setting up reactive handlers.
|
|
The differences are that:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem">The <code class="literal">@StreamListener</code> annotation must not specify an input or output, as they are provided as arguments and return values from the method.</li><li class="listitem">The arguments of the method must be annotated with <code class="literal">@Input</code> and <code class="literal">@Output</code>, indicating which input or output the incoming and outgoing data flows connect to, respectively.</li><li class="listitem">The return value of the method, if any, is annotated with <code class="literal">@Output</code>, indicating the input where data should be sent.</li></ul></div><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Note"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="images/note.png"></td><th align="left">Note</th></tr><tr><td align="left" valign="top"><p>Reactive programming support requires Java 1.8.</p></td></tr></table></div><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Note"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="images/note.png"></td><th align="left">Note</th></tr><tr><td align="left" valign="top"><p>As of Spring Cloud Stream 1.1.1 and later (starting with release train Brooklyn.SR2), reactive programming support requires the use of Reactor 3.0.4.RELEASE and higher.
|
|
Earlier Reactor versions (including 3.0.1.RELEASE, 3.0.2.RELEASE and 3.0.3.RELEASE) are not supported.
|
|
<code class="literal">spring-cloud-stream-reactive</code> transitively retrieves the proper version, but it is possible for the project structure to manage the version of the <code class="literal">io.projectreactor:reactor-core</code> to an earlier release, especially when using Maven.
|
|
This is the case for projects generated by using Spring Initializr with Spring Boot 1.x, which overrides the Reactor version to <code class="literal">2.0.8.RELEASE</code>.
|
|
In such cases, you must ensure that the proper version of the artifact is released.
|
|
You can do so by adding a direct dependency on <code class="literal">io.projectreactor:reactor-core</code> with a version of <code class="literal">3.0.4.RELEASE</code> or later to your project.</p></td></tr></table></div><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Note"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="images/note.png"></td><th align="left">Note</th></tr><tr><td align="left" valign="top"><p>The use of term, <span class="quote">“<span class="quote">reactive</span>”</span>, currently refers to the reactive APIs being used and not to the execution model being reactive (that is, the bound endpoints still use a 'push' rather than a 'pull' model). While some backpressure support is provided by the use of Reactor, we do intend, in a future release, to support entirely reactive pipelines by the use of native reactive clients for the connected middleware.</p></td></tr></table></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="_reactor_based_handlers" href="#_reactor_based_handlers"></a>6.5.1 Reactor-based Handlers</h3></div></div></div><p>A Reactor-based handler can have the following argument types:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem">For arguments annotated with <code class="literal">@Input</code>, it supports the Reactor <code class="literal">Flux</code> type.
|
|
The parameterization of the inbound Flux follows the same rules as in the case of individual message handling: It can be the entire <code class="literal">Message</code>, a POJO that can be the <code class="literal">Message</code> payload, or a POJO that is the result of a transformation based on the <code class="literal">Message</code> content-type header. Multiple inputs are provided.</li><li class="listitem">For arguments annotated with <code class="literal">Output</code>, it supports the <code class="literal">FluxSender</code> type, which connects a <code class="literal">Flux</code> produced by the method with an output. Generally speaking, specifying outputs as arguments is only recommended when the method can have multiple outputs.</li></ul></div><p>A Reactor-based handler supports a return type of <code class="literal">Flux</code>. In that case, it must be annotated with <code class="literal">@Output</code>. We recommend using the return value of the method when a single output <code class="literal">Flux</code> is available.</p><p>The following example shows a Reactor-based <code class="literal">Processor</code>:</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@EnableBinding(Processor.class)</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@EnableAutoConfiguration</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">static</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> UppercaseTransformer {
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@StreamListener</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@Output(Processor.OUTPUT)</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> Flux<String> receive(<em><span class="hl-annotation" style="color: gray">@Input(Processor.INPUT)</span></em> Flux<String> input) {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> input.map(s -> s.toUpperCase());
|
|
}
|
|
}</pre><p>The same processor using output arguments looks like the following example:</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@EnableBinding(Processor.class)</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@EnableAutoConfiguration</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">static</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> UppercaseTransformer {
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@StreamListener</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">void</span> receive(<em><span class="hl-annotation" style="color: gray">@Input(Processor.INPUT)</span></em> Flux<String> input,
|
|
<em><span class="hl-annotation" style="color: gray">@Output(Processor.OUTPUT)</span></em> FluxSender output) {
|
|
output.send(input.map(s -> s.toUpperCase()));
|
|
}
|
|
}</pre></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="_reactive_sources" href="#_reactive_sources"></a>6.5.2 Reactive Sources</h3></div></div></div><p>Spring Cloud Stream reactive support also provides the ability for creating reactive sources through the <code class="literal">@StreamEmitter</code> annotation.
|
|
By using the <code class="literal">@StreamEmitter</code> annotation, a regular source may be converted to a reactive one.
|
|
<code class="literal">@StreamEmitter</code> is a method level annotation that marks a method to be an emitter to outputs declared with <code class="literal">@EnableBinding</code>.
|
|
You cannot use the <code class="literal">@Input</code> annotation along with <code class="literal">@StreamEmitter</code>, as the methods marked with this annotation are not listening for any input. Rather, methods marked with <code class="literal">@StreamEmitter</code> generate output.
|
|
Following the same programming model used in <code class="literal">@StreamListener</code>, <code class="literal">@StreamEmitter</code> also allows flexible ways of using the <code class="literal">@Output</code> annotation, depending on whether the method has any arguments, a return type, and other considerations.</p><p>The remainder of this section contains examples of using the <code class="literal">@StreamEmitter</code> annotation in various styles.</p><p>The following example emits the <code class="literal">Hello, World</code> message every millisecond and publishes to a Reactor <code class="literal">Flux</code>:</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@EnableBinding(Source.class)</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@EnableAutoConfiguration</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">static</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> HelloWorldEmitter {
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@StreamEmitter</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@Output(Source.OUTPUT)</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> Flux<String> emit() {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> Flux.intervalMillis(<span class="hl-number">1</span>)
|
|
.map(l -> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"Hello World"</span>);
|
|
}
|
|
}</pre><p>In the preceding example, the resulting messages in the <code class="literal">Flux</code> are sent to the output channel of the <code class="literal">Source</code>.</p><p>The next example is another flavor of an <code class="literal">@StreamEmmitter</code> that sends a Reactor <code class="literal">Flux</code>.
|
|
Instead of returning a <code class="literal">Flux</code>, the following method uses a <code class="literal">FluxSender</code> to programmatically send a <code class="literal">Flux</code> from a source:</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@EnableBinding(Source.class)</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@EnableAutoConfiguration</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">static</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> HelloWorldEmitter {
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@StreamEmitter</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@Output(Source.OUTPUT)</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">void</span> emit(FluxSender output) {
|
|
output.send(Flux.intervalMillis(<span class="hl-number">1</span>)
|
|
.map(l -> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"Hello World"</span>));
|
|
}
|
|
}</pre><p>The next example is exactly same as the above snippet in functionality and style.
|
|
However, instead of using an explicit <code class="literal">@Output</code> annotation on the method, it uses the annotation on the method parameter.</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@EnableBinding(Source.class)</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@EnableAutoConfiguration</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">static</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> HelloWorldEmitter {
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@StreamEmitter</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">void</span> emit(<em><span class="hl-annotation" style="color: gray">@Output(Source.OUTPUT)</span></em> FluxSender output) {
|
|
output.send(Flux.intervalMillis(<span class="hl-number">1</span>)
|
|
.map(l -> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"Hello World"</span>));
|
|
}
|
|
}</pre><p>The last example in this section is yet another flavor of writing reacting sources by using the Reactive Streams Publisher API and taking advantage of the support for it in <a class="link" href="https://github.com/spring-projects/spring-integration-java-dsl/wiki/Spring-Integration-Java-DSL-Reference" target="_top">Spring Integration Java DSL</a>.
|
|
The <code class="literal">Publisher</code> in the following example still uses Reactor <code class="literal">Flux</code> under the hood, but, from an application perspective, that is transparent to the user and only needs Reactive Streams and Java DSL for Spring Integration:</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@EnableBinding(Source.class)</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@EnableAutoConfiguration</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">static</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> HelloWorldEmitter {
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@StreamEmitter</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@Output(Source.OUTPUT)</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@Bean</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> Publisher<Message<String>> emit() {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> IntegrationFlows.from(() ->
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">new</span> GenericMessage<>(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"Hello World"</span>),
|
|
e -> e.poller(p -> p.fixedDelay(<span class="hl-number">1</span>)))
|
|
.toReactivePublisher();
|
|
}
|
|
}</pre></div></div></div><div class="chapter"><div class="titlepage"><div><div><h2 class="title"><a name="spring-cloud-stream-overview-binders" href="#spring-cloud-stream-overview-binders"></a>7. Binders</h2></div></div></div><p>Spring Cloud Stream provides a Binder abstraction for use in connecting to physical destinations at the external middleware.
|
|
This section provides information about the main concepts behind the Binder SPI, its main components, and implementation-specific details.</p><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="_producers_and_consumers" href="#_producers_and_consumers"></a>7.1 Producers and Consumers</h2></div></div></div><p>The following image shows the general relationship of producers and consumers:</p><div class="figure"><a name="d0e2038" href="#d0e2038"></a><p class="title"><b>Figure 7.1. Producers and Consumers</b></p><div class="figure-contents"><div class="mediaobject" align="center"><img src="https://raw.githubusercontent.com/spring-cloud/spring-cloud-stream/master/docs/src/main/asciidoc/images/producers-consumers.png" align="middle" alt="producers consumers"></div></div></div><br class="figure-break"><p>A producer is any component that sends messages to a channel.
|
|
The channel can be bound to an external message broker with a <code class="literal">Binder</code> implementation for that broker.
|
|
When invoking the <code class="literal">bindProducer()</code> method, the first parameter is the name of the destination within the broker, the second parameter is the local channel instance to which the producer sends messages, and the third parameter contains properties (such as a partition key expression) to be used within the adapter that is created for that channel.</p><p>A consumer is any component that receives messages from a channel.
|
|
As with a producer, the consumer’s channel can be bound to an external message broker.
|
|
When invoking the <code class="literal">bindConsumer()</code> method, the first parameter is the destination name, and a second parameter provides the name of a logical group of consumers.
|
|
Each group that is represented by consumer bindings for a given destination receives a copy of each message that a producer sends to that destination (that is, it follows normal publish-subscribe semantics).
|
|
If there are multiple consumer instances bound with the same group name, then messages are load-balanced across those consumer instances so that each message sent by a producer is consumed by only a single consumer instance within each group (that is, it follows normal queueing semantics).</p></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="spring-cloud-stream-overview-binder-api" href="#spring-cloud-stream-overview-binder-api"></a>7.2 Binder SPI</h2></div></div></div><p>The Binder SPI consists of a number of interfaces, out-of-the box utility classes, and discovery strategies that provide a pluggable mechanism for connecting to external middleware.</p><p>The key point of the SPI is the <code class="literal">Binder</code> interface, which is a strategy for connecting inputs and outputs to external middleware. The following listing shows the definnition of the <code class="literal">Binder</code> interface:</p><pre class="programlisting"><span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">interface</span> Binder<T, C <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">extends</span> ConsumerProperties, P <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">extends</span> ProducerProperties> {
|
|
Binding<T> bindConsumer(String name, String group, T inboundBindTarget, C consumerProperties);
|
|
|
|
Binding<T> bindProducer(String name, T outboundBindTarget, P producerProperties);
|
|
}</pre><p>The interface is parameterized, offering a number of extension points:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem">Input and output bind targets. As of version 1.0, only <code class="literal">MessageChannel</code> is supported, but this is intended to be used as an extension point in the future.</li><li class="listitem">Extended consumer and producer properties, allowing specific Binder implementations to add supplemental properties that can be supported in a type-safe manner.</li></ul></div><p>A typical binder implementation consists of the following:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem">A class that implements the <code class="literal">Binder</code> interface;</li><li class="listitem">A Spring <code class="literal">@Configuration</code> class that creates a bean of type <code class="literal">Binder</code> along with the middleware connection infrastructure.</li><li class="listitem"><p class="simpara">A <code class="literal">META-INF/spring.binders</code> file found on the classpath containing one or more binder definitions, as shown in the following example:</p><pre class="screen">kafka:\
|
|
org.springframework.cloud.stream.binder.kafka.config.KafkaBinderConfiguration</pre></li></ul></div></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="_binder_detection" href="#_binder_detection"></a>7.3 Binder Detection</h2></div></div></div><p>Spring Cloud Stream relies on implementations of the Binder SPI to perform the task of connecting channels to message brokers.
|
|
Each Binder implementation typically connects to one type of messaging system.</p><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="_classpath_detection" href="#_classpath_detection"></a>7.3.1 Classpath Detection</h3></div></div></div><p>By default, Spring Cloud Stream relies on Spring Boot’s auto-configuration to configure the binding process.
|
|
If a single Binder implementation is found on the classpath, Spring Cloud Stream automatically uses it.
|
|
For example, a Spring Cloud Stream project that aims to bind only to RabbitMQ can add the following dependency:</p><pre class="programlisting"><span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"><dependency></span>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"><groupId></span>org.springframework.cloud<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"></groupId></span>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"><artifactId></span>spring-cloud-stream-binder-rabbit<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"></artifactId></span>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"></dependency></span></pre><p>For the specific Maven coordinates of other binder dependencies, see the documentation of that binder implementation.</p></div></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="multiple-binders" href="#multiple-binders"></a>7.4 Multiple Binders on the Classpath</h2></div></div></div><p>When multiple binders are present on the classpath, the application must indicate which binder is to be used for each channel binding.
|
|
Each binder configuration contains a <code class="literal">META-INF/spring.binders</code> file, which is a simple properties file, as shown in the following example:</p><pre class="screen">rabbit:\
|
|
org.springframework.cloud.stream.binder.rabbit.config.RabbitServiceAutoConfiguration</pre><p>Similar files exist for the other provided binder implementations (such as Kafka), and custom binder implementations are expected to provide them as well.
|
|
The key represents an identifying name for the binder implementation, whereas the value is a comma-separated list of configuration classes that each contain one and only one bean definition of type <code class="literal">org.springframework.cloud.stream.binder.Binder</code>.</p><p>Binder selection can either be performed globally, using the <code class="literal">spring.cloud.stream.defaultBinder</code> property (for example, <code class="literal">spring.cloud.stream.defaultBinder=rabbit</code>) or individually, by configuring the binder on each channel binding.
|
|
For instance, a processor application (that has channels named <code class="literal">input</code> and <code class="literal">output</code> for read and write respectively) that reads from Kafka and writes to RabbitMQ can specify the following configuration:</p><pre class="screen">spring.cloud.stream.bindings.input.binder=kafka
|
|
spring.cloud.stream.bindings.output.binder=rabbit</pre></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="multiple-systems" href="#multiple-systems"></a>7.5 Connecting to Multiple Systems</h2></div></div></div><p>By default, binders share the application’s Spring Boot auto-configuration, so that one instance of each binder found on the classpath is created.
|
|
If your application should connect to more than one broker of the same type, you can specify multiple binder configurations, each with different environment settings.</p><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Note"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="images/note.png"></td><th align="left">Note</th></tr><tr><td align="left" valign="top"><p>Turning on explicit binder configuration disables the default binder configuration process altogether.
|
|
If you do so, all binders in use must be included in the configuration.
|
|
Frameworks that intend to use Spring Cloud Stream transparently may create binder configurations that can be referenced by name, but they do not affect the default binder configuration.
|
|
In order to do so, a binder configuration may have its <code class="literal">defaultCandidate</code> flag set to false (for example, <code class="literal">spring.cloud.stream.binders.<configurationName>.defaultCandidate=false</code>).
|
|
This denotes a configuration that exists independently of the default binder configuration process.</p></td></tr></table></div><p>The following example shows a typical configuration for a processor application that connects to two RabbitMQ broker instances:</p><pre class="programlisting">spring:
|
|
cloud:
|
|
stream:
|
|
bindings:
|
|
input:
|
|
destination: thing1
|
|
binder: rabbit1
|
|
output:
|
|
destination: thing2
|
|
binder: rabbit2
|
|
binders:
|
|
rabbit1:
|
|
type: rabbit
|
|
environment:
|
|
spring:
|
|
rabbitmq:
|
|
host: <host1>
|
|
rabbit2:
|
|
type: rabbit
|
|
environment:
|
|
spring:
|
|
rabbitmq:
|
|
host: <host2></pre></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="_binding_visualization_and_control" href="#_binding_visualization_and_control"></a>7.6 Binding visualization and control</h2></div></div></div><p>Since version 2.0, Spring Cloud Stream supports visualization and control of the Bindings through Actuator endpoints.</p><p>Starting with version 2.0 actuator and web are optional, you must first add one of the web dependencies as well as add the actuator dependency manually.
|
|
The following example shows how to add the dependency for the Web framework:</p><pre class="programlisting"><span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"><dependency></span>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"><groupId></span>org.springframework.boot<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"></groupId></span>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"><artifactId></span>spring-boot-starter-web<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"></artifactId></span>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"></dependency></span></pre><p>The following example shows how to add the dependency for the WebFlux framework:</p><pre class="programlisting"><span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"><dependency></span>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"><groupId></span>org.springframework.boot<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"></groupId></span>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"><artifactId></span>spring-boot-starter-webflux<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"></artifactId></span>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"></dependency></span></pre><p>You can add the Actuator dependency as follows:</p><pre class="programlisting"><span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"><dependency></span>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"><groupId></span>org.springframework.boot<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"></groupId></span>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"><artifactId></span>spring-boot-starter-actuator<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"></artifactId></span>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"></dependency></span></pre><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Note"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="images/note.png"></td><th align="left">Note</th></tr><tr><td align="left" valign="top"><p>To run Spring Cloud Stream 2.0 apps in Cloud Foundry, you must add <code class="literal">spring-boot-starter-web</code> and <code class="literal">spring-boot-starter-actuator</code> to the classpath. Otherwise, the
|
|
application will not start due to health check failures.</p></td></tr></table></div><p>You must also enable the <code class="literal">bindings</code> actuator endpoints by setting the following property: <code class="literal">--management.endpoints.web.exposure.include=bindings</code>.</p><p>Once those prerequisites are satisfied. you should see the following in the logs when application start:</p><pre class="literallayout">: Mapped "{[/actuator/bindings/{name}],methods=[POST]. . .
|
|
: Mapped "{[/actuator/bindings],methods=[GET]. . .
|
|
: Mapped "{[/actuator/bindings/{name}],methods=[GET]. . .</pre><p>To visualize the current bindings, access the following URL:
|
|
<code class="literal"><a class="link" href="http://<host>:<port>/actuator/bindings" target="_top">http://<host>:<port>/actuator/bindings</a></code></p><p>Alternative, to see a single binding, access one of the URLs similar to the following:
|
|
<code class="literal"><a class="link" href="http://<host>:<port>/actuator/bindings/myBindingName" target="_top">http://<host>:<port>/actuator/bindings/myBindingName</a></code></p><p>You can also stop, start, pause, and resume individual bindings by posting to the same URL while providing a <code class="literal">state</code> argument as JSON, as shown in the following examples:</p><p>curl -d '{"state":"STOPPED"}' -H "Content-Type: application/json" -X POST <a class="link" href="http://<host>:<port>/actuator/bindings/myBindingName" target="_top">http://<host>:<port>/actuator/bindings/myBindingName</a>
|
|
curl -d '{"state":"STARTED"}' -H "Content-Type: application/json" -X POST <a class="link" href="http://<host>:<port>/actuator/bindings/myBindingName" target="_top">http://<host>:<port>/actuator/bindings/myBindingName</a>
|
|
curl -d '{"state":"PAUSED"}' -H "Content-Type: application/json" -X POST <a class="link" href="http://<host>:<port>/actuator/bindings/myBindingName" target="_top">http://<host>:<port>/actuator/bindings/myBindingName</a>
|
|
curl -d '{"state":"RESUMED"}' -H "Content-Type: application/json" -X POST <a class="link" href="http://<host>:<port>/actuator/bindings/myBindingName" target="_top">http://<host>:<port>/actuator/bindings/myBindingName</a></p><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Note"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="images/note.png"></td><th align="left">Note</th></tr><tr><td align="left" valign="top"><p><code class="literal">PAUSED</code> and <code class="literal">RESUMED</code> work only when the corresponding binder and its underlying technology supports it. Otherwise, you see the warning message in the logs.
|
|
Currently, only Kafka binder supports the <code class="literal">PAUSED</code> and <code class="literal">RESUMED</code> states.</p></td></tr></table></div></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="_binder_configuration_properties" href="#_binder_configuration_properties"></a>7.7 Binder Configuration Properties</h2></div></div></div><p>The following properties are available when customizing binder configurations. These properties exposed via <code class="literal">org.springframework.cloud.stream.config.BinderProperties</code></p><p>They must be prefixed with <code class="literal">spring.cloud.stream.binders.<configurationName></code>.</p><div class="variablelist"><dl class="variablelist"><dt><span class="term">type</span></dt><dd><p class="simpara">The binder type.
|
|
It typically references one of the binders found on the classpath — in particular, a key in a <code class="literal">META-INF/spring.binders</code> file.</p><p class="simpara">By default, it has the same value as the configuration name.</p></dd><dt><span class="term">inheritEnvironment</span></dt><dd><p class="simpara">Whether the configuration inherits the environment of the application itself.</p><p class="simpara">Default: <code class="literal">true</code>.</p></dd><dt><span class="term">environment</span></dt><dd><p class="simpara">Root for a set of properties that can be used to customize the environment of the binder.
|
|
When this property is set, the context in which the binder is being created is not a child of the application context.
|
|
This setting allows for complete separation between the binder components and the application components.</p><p class="simpara">Default: <code class="literal">empty</code>.</p></dd><dt><span class="term">defaultCandidate</span></dt><dd><p class="simpara">Whether the binder configuration is a candidate for being considered a default binder or can be used only when explicitly referenced.
|
|
This setting allows adding binder configurations without interfering with the default processing.</p><p class="simpara">Default: <code class="literal">true</code>.</p></dd></dl></div></div></div><div class="chapter"><div class="titlepage"><div><div><h2 class="title"><a name="_configuration_options" href="#_configuration_options"></a>8. Configuration Options</h2></div></div></div><p>Spring Cloud Stream supports general configuration options as well as configuration for bindings and binders.
|
|
Some binders let additional binding properties support middleware-specific features.</p><p>Configuration options can be provided to Spring Cloud Stream applications through any mechanism supported by Spring Boot.
|
|
This includes application arguments, environment variables, and YAML or .properties files.</p><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="_binding_service_properties" href="#_binding_service_properties"></a>8.1 Binding Service Properties</h2></div></div></div><p>These properties are exposed via <code class="literal">org.springframework.cloud.stream.config.BindingServiceProperties</code></p><div class="variablelist"><dl class="variablelist"><dt><span class="term">spring.cloud.stream.instanceCount</span></dt><dd><p class="simpara">The number of deployed instances of an application.
|
|
Must be set for partitioning on the producer side. Must be set on the consumer side when using RabbitMQ and with Kafka if <code class="literal">autoRebalanceEnabled=false</code>.</p><p class="simpara">Default: <code class="literal">1</code>.</p></dd><dt><span class="term">spring.cloud.stream.instanceIndex</span></dt><dd>The instance index of the application: A number from <code class="literal">0</code> to <code class="literal">instanceCount - 1</code>.
|
|
Used for partitioning with RabbitMQ and with Kafka if <code class="literal">autoRebalanceEnabled=false</code>.
|
|
Automatically set in Cloud Foundry to match the application’s instance index.</dd><dt><span class="term">spring.cloud.stream.dynamicDestinations</span></dt><dd><p class="simpara">A list of destinations that can be bound dynamically (for example, in a dynamic routing scenario).
|
|
If set, only listed destinations can be bound.</p><p class="simpara">Default: empty (letting any destination be bound).</p></dd><dt><span class="term">spring.cloud.stream.defaultBinder</span></dt><dd><p class="simpara">The default binder to use, if multiple binders are configured.
|
|
See <a class="link" href="#multiple-binders" title="7.4 Multiple Binders on the Classpath">Multiple Binders on the Classpath</a>.</p><p class="simpara">Default: empty.</p></dd><dt><span class="term">spring.cloud.stream.overrideCloudConnectors</span></dt><dd><p class="simpara">This property is only applicable when the <code class="literal">cloud</code> profile is active and Spring Cloud Connectors are provided with the application.
|
|
If the property is <code class="literal">false</code> (the default), the binder detects a suitable bound service (for example, a RabbitMQ service bound in Cloud Foundry for the RabbitMQ binder) and uses it for creating connections (usually through Spring Cloud Connectors).
|
|
When set to <code class="literal">true</code>, this property instructs binders to completely ignore the bound services and rely on Spring Boot properties (for example, relying on the <code class="literal">spring.rabbitmq.*</code> properties provided in the environment for the RabbitMQ binder).
|
|
The typical usage of this property is to be nested in a customized environment <a class="link" href="#multiple-systems" title="7.5 Connecting to Multiple Systems">when connecting to multiple systems</a>.</p><p class="simpara">Default: <code class="literal">false</code>.</p></dd><dt><span class="term">spring.cloud.stream.bindingRetryInterval</span></dt><dd><p class="simpara">The interval (in seconds) between retrying binding creation when, for example, the binder does not support late binding and the broker (for example, Apache Kafka) is down.
|
|
Set it to zero to treat such conditions as fatal, preventing the application from starting.</p><p class="simpara">Default: <code class="literal">30</code></p></dd></dl></div></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="binding-properties" href="#binding-properties"></a>8.2 Binding Properties</h2></div></div></div><p>Binding properties are supplied by using the format of <code class="literal">spring.cloud.stream.bindings.<channelName>.<property>=<value></code>.
|
|
The <code class="literal"><channelName></code> represents the name of the channel being configured (for example, <code class="literal">output</code> for a <code class="literal">Source</code>).</p><p>To avoid repetition, Spring Cloud Stream supports setting values for all channels, in the format of <code class="literal">spring.cloud.stream.default.<property>=<value></code>.</p><p>When it comes to avoiding repetitions for extended binding properties, this format should be used - <code class="literal">spring.cloud.stream.<binder-type>.default.<producer|consumer>.<property>=<value></code>.</p><p>In what follows, we indicate where we have omitted the <code class="literal">spring.cloud.stream.bindings.<channelName>.</code> prefix and focus just on the property name, with the understanding that the prefix ise included at runtime.</p><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="_common_binding_properties" href="#_common_binding_properties"></a>8.2.1 Common Binding Properties</h3></div></div></div><p>These properties are exposed via <code class="literal">org.springframework.cloud.stream.config.BindingProperties</code></p><p>The following binding properties are available for both input and output bindings and must be prefixed with <code class="literal">spring.cloud.stream.bindings.<channelName>.</code> (for example, <code class="literal">spring.cloud.stream.bindings.input.destination=ticktock</code>).</p><p>Default values can be set by using the <code class="literal">spring.cloud.stream.default</code> prefix (for example`spring.cloud.stream.default.contentType=application/json`).</p><div class="variablelist"><dl class="variablelist"><dt><span class="term">destination</span></dt><dd>The target destination of a channel on the bound middleware (for example, the RabbitMQ exchange or Kafka topic).
|
|
If the channel is bound as a consumer, it could be bound to multiple destinations, and the destination names can be specified as comma-separated <code class="literal">String</code> values.
|
|
If not set, the channel name is used instead.
|
|
The default value of this property cannot be overridden.</dd><dt><span class="term">group</span></dt><dd><p class="simpara">The consumer group of the channel.
|
|
Applies only to inbound bindings.
|
|
See <a class="link" href="#consumer-groups" title="5.4 Consumer Groups">Consumer Groups</a>.</p><p class="simpara">Default: <code class="literal">null</code> (indicating an anonymous consumer).</p></dd><dt><span class="term">contentType</span></dt><dd><p class="simpara">The content type of the channel.
|
|
See <span class="quote">“<span class="quote"><a class="xref" href="#content-type-management" title="9. Content Type Negotiation">Chapter 9, <i>Content Type Negotiation</i></a></span>”</span>.</p><p class="simpara">Default: <code class="literal">application/json</code>.</p></dd><dt><span class="term">binder</span></dt><dd><p class="simpara">The binder used by this binding.
|
|
See <span class="quote">“<span class="quote"><a class="xref" href="#multiple-binders" title="7.4 Multiple Binders on the Classpath">Section 7.4, “Multiple Binders on the Classpath”</a></span>”</span> for details.</p><p class="simpara">Default: <code class="literal">null</code> (the default binder is used, if it exists).</p></dd></dl></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="_consumer_properties" href="#_consumer_properties"></a>8.2.2 Consumer Properties</h3></div></div></div><p>These properties are exposed via <code class="literal">org.springframework.cloud.stream.binder.ConsumerProperties</code></p><p>The following binding properties are available for input bindings only and must be prefixed with <code class="literal">spring.cloud.stream.bindings.<channelName>.consumer.</code> (for example, <code class="literal">spring.cloud.stream.bindings.input.consumer.concurrency=3</code>).</p><p>Default values can be set by using the <code class="literal">spring.cloud.stream.default.consumer</code> prefix (for example, <code class="literal">spring.cloud.stream.default.consumer.headerMode=none</code>).</p><div class="variablelist"><dl class="variablelist"><dt><span class="term">concurrency</span></dt><dd><p class="simpara">The concurrency of the inbound consumer.</p><p class="simpara">Default: <code class="literal">1</code>.</p></dd><dt><span class="term">partitioned</span></dt><dd><p class="simpara">Whether the consumer receives data from a partitioned producer.</p><p class="simpara">Default: <code class="literal">false</code>.</p></dd><dt><span class="term">headerMode</span></dt><dd><p class="simpara">When set to <code class="literal">none</code>, disables header parsing on input.
|
|
Effective only for messaging middleware that does not support message headers natively and requires header embedding.
|
|
This option is useful when consuming data from non-Spring Cloud Stream applications when native headers are not supported.
|
|
When set to <code class="literal">headers</code>, it uses the middleware’s native header mechanism.
|
|
When set to <code class="literal">embeddedHeaders</code>, it embeds headers into the message payload.</p><p class="simpara">Default: depends on the binder implementation.</p></dd><dt><span class="term">maxAttempts</span></dt><dd><p class="simpara">If processing fails, the number of attempts to process the message (including the first).
|
|
Set to <code class="literal">1</code> to disable retry.</p><p class="simpara">Default: <code class="literal">3</code>.</p></dd><dt><span class="term">backOffInitialInterval</span></dt><dd><p class="simpara">The backoff initial interval on retry.</p><p class="simpara">Default: <code class="literal">1000</code>.</p></dd><dt><span class="term">backOffMaxInterval</span></dt><dd><p class="simpara">The maximum backoff interval.</p><p class="simpara">Default: <code class="literal">10000</code>.</p></dd><dt><span class="term">backOffMultiplier</span></dt><dd><p class="simpara">The backoff multiplier.</p><p class="simpara">Default: <code class="literal">2.0</code>.</p></dd><dt><span class="term">defaultRetryable</span></dt><dd><p class="simpara">Whether exceptions thrown by the listener that are not listed in the <code class="literal">retryableExceptions</code> are retryable.</p><p class="simpara">Default: <code class="literal">true</code>.</p></dd><dt><span class="term">instanceIndex</span></dt><dd><p class="simpara">When set to a value greater than equal to zero, it allows customizing the instance index of this consumer (if different from <code class="literal">spring.cloud.stream.instanceIndex</code>).
|
|
When set to a negative value, it defaults to <code class="literal">spring.cloud.stream.instanceIndex</code>.
|
|
See <span class="quote">“<span class="quote"><a class="xref" href="#spring-cloud-stream-overview-instance-index-instance-count" title="11.2 Instance Index and Instance Count">Section 11.2, “Instance Index and Instance Count”</a></span>”</span> for more information.</p><p class="simpara">Default: <code class="literal">-1</code>.</p></dd><dt><span class="term">instanceCount</span></dt><dd><p class="simpara">When set to a value greater than equal to zero, it allows customizing the instance count of this consumer (if different from <code class="literal">spring.cloud.stream.instanceCount</code>).
|
|
When set to a negative value, it defaults to <code class="literal">spring.cloud.stream.instanceCount</code>.
|
|
See <span class="quote">“<span class="quote"><a class="xref" href="#spring-cloud-stream-overview-instance-index-instance-count" title="11.2 Instance Index and Instance Count">Section 11.2, “Instance Index and Instance Count”</a></span>”</span> for more information.</p><p class="simpara">Default: <code class="literal">-1</code>.</p></dd><dt><span class="term">retryableExceptions</span></dt><dd><p class="simpara">A map of Throwable class names in the key and a boolean in the value.
|
|
Specify those exceptions (and subclasses) that will or won’t be retried.
|
|
Also see <code class="literal">defaultRetriable</code>.
|
|
Example: <code class="literal">spring.cloud.stream.bindings.input.consumer.retryable-exceptions.java.lang.IllegalStateException=false</code>.</p><p class="simpara">Default: empty.</p></dd><dt><span class="term">useNativeDecoding</span></dt><dd><p class="simpara">When set to <code class="literal">true</code>, the inbound message is deserialized directly by the client library, which must be configured correspondingly (for example, setting an appropriate Kafka producer value deserializer).
|
|
When this configuration is being used, the inbound message unmarshalling is not based on the <code class="literal">contentType</code> of the binding.
|
|
When native decoding is used, it is the responsibility of the producer to use an appropriate encoder (for example, the Kafka producer value serializer) to serialize the outbound message.
|
|
Also, when native encoding and decoding is used, the <code class="literal">headerMode=embeddedHeaders</code> property is ignored and headers are not embedded in the message.
|
|
See the producer property <code class="literal">useNativeEncoding</code>.</p><p class="simpara">Default: <code class="literal">false</code>.</p></dd></dl></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="_producer_properties" href="#_producer_properties"></a>8.2.3 Producer Properties</h3></div></div></div><p>These properties are exposed via <code class="literal">org.springframework.cloud.stream.binder.ProducerProperties</code></p><p>The following binding properties are available for output bindings only and must be prefixed with <code class="literal">spring.cloud.stream.bindings.<channelName>.producer.</code> (for example, <code class="literal">spring.cloud.stream.bindings.input.producer.partitionKeyExpression=payload.id</code>).</p><p>Default values can be set by using the prefix <code class="literal">spring.cloud.stream.default.producer</code> (for example, <code class="literal">spring.cloud.stream.default.producer.partitionKeyExpression=payload.id</code>).</p><div class="variablelist"><dl class="variablelist"><dt><span class="term">partitionKeyExpression</span></dt><dd><p class="simpara">A SpEL expression that determines how to partition outbound data.
|
|
If set, or if <code class="literal">partitionKeyExtractorClass</code> is set, outbound data on this channel is partitioned. <code class="literal">partitionCount</code> must be set to a value greater than 1 to be effective.
|
|
Mutually exclusive with <code class="literal">partitionKeyExtractorClass</code>.
|
|
See <span class="quote">“<span class="quote"><a class="xref" href="#partitioning" title="5.6 Partitioning Support">Section 5.6, “Partitioning Support”</a></span>”</span>.</p><p class="simpara">Default: null.</p></dd><dt><span class="term">partitionKeyExtractorClass</span></dt><dd><p class="simpara">A <code class="literal">PartitionKeyExtractorStrategy</code> implementation.
|
|
If set, or if <code class="literal">partitionKeyExpression</code> is set, outbound data on this channel is partitioned. <code class="literal">partitionCount</code> must be set to a value greater than 1 to be effective.
|
|
Mutually exclusive with <code class="literal">partitionKeyExpression</code>.
|
|
See <span class="quote">“<span class="quote"><a class="xref" href="#partitioning" title="5.6 Partitioning Support">Section 5.6, “Partitioning Support”</a></span>”</span>.</p><p class="simpara">Default: <code class="literal">null</code>.</p></dd><dt><span class="term">partitionSelectorClass</span></dt><dd><p class="simpara"> A <code class="literal">PartitionSelectorStrategy</code> implementation.
|
|
Mutually exclusive with <code class="literal">partitionSelectorExpression</code>.
|
|
If neither is set, the partition is selected as the <code class="literal">hashCode(key) % partitionCount</code>, where <code class="literal">key</code> is computed through either <code class="literal">partitionKeyExpression</code> or <code class="literal">partitionKeyExtractorClass</code>.</p><p class="simpara">Default: <code class="literal">null</code>.</p></dd><dt><span class="term">partitionSelectorExpression</span></dt><dd><p class="simpara">A SpEL expression for customizing partition selection.
|
|
Mutually exclusive with <code class="literal">partitionSelectorClass</code>.
|
|
If neither is set, the partition is selected as the <code class="literal">hashCode(key) % partitionCount</code>, where <code class="literal">key</code> is computed through either <code class="literal">partitionKeyExpression</code> or <code class="literal">partitionKeyExtractorClass</code>.</p><p class="simpara">Default: <code class="literal">null</code>.</p></dd><dt><span class="term">partitionCount</span></dt><dd><p class="simpara">The number of target partitions for the data, if partitioning is enabled.
|
|
Must be set to a value greater than 1 if the producer is partitioned.
|
|
On Kafka, it is interpreted as a hint. The larger of this and the partition count of the target topic is used instead.</p><p class="simpara">Default: <code class="literal">1</code>.</p></dd><dt><span class="term">requiredGroups</span></dt><dd>A comma-separated list of groups to which the producer must ensure message delivery even if they start after it has been created (for example, by pre-creating durable queues in RabbitMQ).</dd><dt><span class="term">headerMode</span></dt><dd><p class="simpara">When set to <code class="literal">none</code>, it disables header embedding on output.
|
|
It is effective only for messaging middleware that does not support message headers natively and requires header embedding.
|
|
This option is useful when producing data for non-Spring Cloud Stream applications when native headers are not supported.
|
|
When set to <code class="literal">headers</code>, it uses the middleware’s native header mechanism.
|
|
When set to <code class="literal">embeddedHeaders</code>, it embeds headers into the message payload.</p><p class="simpara">Default: Depends on the binder implementation.</p></dd><dt><span class="term">useNativeEncoding</span></dt><dd><p class="simpara">When set to <code class="literal">true</code>, the outbound message is serialized directly by the client library, which must be configured correspondingly (for example, setting an appropriate Kafka producer value serializer).
|
|
When this configuration is being used, the outbound message marshalling is not based on the <code class="literal">contentType</code> of the binding.
|
|
When native encoding is used, it is the responsibility of the consumer to use an appropriate decoder (for example, the Kafka consumer value de-serializer) to deserialize the inbound message.
|
|
Also, when native encoding and decoding is used, the <code class="literal">headerMode=embeddedHeaders</code> property is ignored and headers are not embedded in the message.
|
|
See the consumer property <code class="literal">useNativeDecoding</code>.</p><p class="simpara">Default: <code class="literal">false</code>.</p></dd><dt><span class="term">errorChannelEnabled</span></dt><dd><p class="simpara">When set to <code class="literal">true</code>, if the binder supports asynchroous send results, send failures are sent to an error channel for the destination.
|
|
See <code class="literal"><a class="xref" href="#spring-cloud-stream-overview-error-handling" title="6.4 Error Handling">Section 6.4, “Error Handling”</a></code> for more information.</p><p class="simpara">Default: <code class="literal">false</code>.</p></dd></dl></div></div></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="dynamicdestination" href="#dynamicdestination"></a>8.3 Using Dynamically Bound Destinations</h2></div></div></div><p>Besides the channels defined by using <code class="literal">@EnableBinding</code>, Spring Cloud Stream lets applications send messages to dynamically bound destinations.
|
|
This is useful, for example, when the target destination needs to be determined at runtime.
|
|
Applications can do so by using the <code class="literal">BinderAwareChannelResolver</code> bean, registered automatically by the <code class="literal">@EnableBinding</code> annotation.</p><p>The 'spring.cloud.stream.dynamicDestinations' property can be used for restricting the dynamic destination names to a known set (whitelisting).
|
|
If this property is not set, any destination can be bound dynamically.</p><p>The <code class="literal">BinderAwareChannelResolver</code> can be used directly, as shown in the following example of a REST controller using a path variable to decide the target channel:</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@EnableBinding</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@Controller</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> SourceWithDynamicDestination {
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Autowired</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">private</span> BinderAwareChannelResolver resolver;
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@RequestMapping(path = "/{target}", method = POST, consumes = "*/*")</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@ResponseStatus(HttpStatus.ACCEPTED)</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">void</span> handleRequest(<em><span class="hl-annotation" style="color: gray">@RequestBody</span></em> String body, <em><span class="hl-annotation" style="color: gray">@PathVariable("target")</span></em> target,
|
|
<em><span class="hl-annotation" style="color: gray">@RequestHeader(HttpHeaders.CONTENT_TYPE)</span></em> Object contentType) {
|
|
sendMessage(body, target, contentType);
|
|
}
|
|
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">private</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">void</span> sendMessage(String body, String target, Object contentType) {
|
|
resolver.resolveDestination(target).send(MessageBuilder.createMessage(body,
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">new</span> MessageHeaders(Collections.singletonMap(MessageHeaders.CONTENT_TYPE, contentType))));
|
|
}
|
|
}</pre><p>Now consider what happens when we start the application on the default port (8080) and make the following requests with CURL:</p><pre class="screen">curl -H "Content-Type: application/json" -X POST -d "customer-1" http://localhost:8080/customers
|
|
|
|
curl -H "Content-Type: application/json" -X POST -d "order-1" http://localhost:8080/orders</pre><p>The destinations, 'customers' and 'orders', are created in the broker (in the exchange for Rabbit or in the topic for Kafka) with names of 'customers' and 'orders', and the data is published to the appropriate destinations.</p><p>The <code class="literal">BinderAwareChannelResolver</code> is a general-purpose Spring Integration <code class="literal">DestinationResolver</code> and can be injected in other components — for example, in a router using a SpEL expression based on the <code class="literal">target</code> field of an incoming JSON message. The following example includes a router that reads SpEL expressions:</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@EnableBinding</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@Controller</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> SourceWithDynamicDestination {
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Autowired</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">private</span> BinderAwareChannelResolver resolver;
|
|
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@RequestMapping(path = "/", method = POST, consumes = "application/json")</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@ResponseStatus(HttpStatus.ACCEPTED)</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">void</span> handleRequest(<em><span class="hl-annotation" style="color: gray">@RequestBody</span></em> String body, <em><span class="hl-annotation" style="color: gray">@RequestHeader(HttpHeaders.CONTENT_TYPE)</span></em> Object contentType) {
|
|
sendMessage(body, contentType);
|
|
}
|
|
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">private</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">void</span> sendMessage(Object body, Object contentType) {
|
|
routerChannel().send(MessageBuilder.createMessage(body,
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">new</span> MessageHeaders(Collections.singletonMap(MessageHeaders.CONTENT_TYPE, contentType))));
|
|
}
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Bean(name = "routerChannel")</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> MessageChannel routerChannel() {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">new</span> DirectChannel();
|
|
}
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Bean</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@ServiceActivator(inputChannel = "routerChannel")</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> ExpressionEvaluatingRouter router() {
|
|
ExpressionEvaluatingRouter router =
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">new</span> ExpressionEvaluatingRouter(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">new</span> SpelExpressionParser().parseExpression(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"payload.target"</span>));
|
|
router.setDefaultOutputChannelName(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"default-output"</span>);
|
|
router.setChannelResolver(resolver);
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> router;
|
|
}
|
|
}</pre><p>The <a class="link" href="https://github.com/spring-cloud-stream-app-starters/router" target="_top">Router Sink Application</a> uses this technique to create the destinations on-demand.</p><p>If the channel names are known in advance, you can configure the producer properties as with any other destination.
|
|
Alternatively, if you register a <code class="literal">NewDestinationBindingCallback<></code> bean, it is invoked just before the binding is created.
|
|
The callback takes the generic type of the extended producer properties used by the binder.
|
|
It has one method:</p><pre class="programlisting"><span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">void</span> configure(String channelName, MessageChannel channel, ProducerProperties producerProperties,
|
|
T extendedProducerProperties);</pre><p>The following example shows how to use the RabbitMQ binder:</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@Bean</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> NewDestinationBindingCallback<RabbitProducerProperties> dynamicConfigurer() {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> (name, channel, props, extended) -> {
|
|
props.setRequiredGroups(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"bindThisQueue"</span>);
|
|
extended.setQueueNameGroupOnly(true);
|
|
extended.setAutoBindDlq(true);
|
|
extended.setDeadLetterQueueName(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"myDLQ"</span>);
|
|
};
|
|
}</pre><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Note"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="images/note.png"></td><th align="left">Note</th></tr><tr><td align="left" valign="top"><p>If you need to support dynamic destinations with multiple binder types, use <code class="literal">Object</code> for the generic type and cast the <code class="literal">extended</code> argument as needed.</p></td></tr></table></div></div></div><div class="chapter"><div class="titlepage"><div><div><h2 class="title"><a name="content-type-management" href="#content-type-management"></a>9. Content Type Negotiation</h2></div></div></div><p>Data transformation is one of the core features of any message-driven microservice architecture. Given that, in Spring Cloud Stream, such data
|
|
is represented as a Spring <code class="literal">Message</code>, a message may have to be transformed to a desired shape or size before reaching its destination. This is required for two reasons:</p><div class="orderedlist"><ol class="orderedlist" type="1"><li class="listitem">To convert the contents of the incoming message to match the signature of the application-provided handler.</li><li class="listitem">To convert the contents of the outgoing message to the wire format.</li></ol></div><p>The wire format is typically <code class="literal">byte[]</code> (that is true for the Kafka and Rabbit binders), but it is governed by the binder implementation.</p><p>In Spring Cloud Stream, message transformation is accomplished with an <code class="literal">org.springframework.messaging.converter.MessageConverter</code>.</p><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Note"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="images/note.png"></td><th align="left">Note</th></tr><tr><td align="left" valign="top"><p>As a supplement to the details to follow, you may also want to read the following <a class="link" href="https://spring.io/blog/2018/02/26/spring-cloud-stream-2-0-content-type-negotiation-and-transformation" target="_top">blog post</a>.</p></td></tr></table></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="_mechanics" href="#_mechanics"></a>9.1 Mechanics</h2></div></div></div><p>To better understand the mechanics and the necessity behind content-type negotiation, we take a look at a very simple use case by using the following message handler as an example:</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@StreamListener(Processor.INPUT)</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@SendTo(Processor.OUTPUT)</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> String handle(Person person) {..}</pre><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Note"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="images/note.png"></td><th align="left">Note</th></tr><tr><td align="left" valign="top"><p>For simplicity, we assume that this is the only handler in the application (we assume there is no internal pipeline).</p></td></tr></table></div><p>The handler shown in the preceding example expects a <code class="literal">Person</code> object as an argument and produces a <code class="literal">String</code> type as an output.
|
|
In order for the framework to succeed in passing the incoming <code class="literal">Message</code> as an argument to this handler, it has to somehow transform the payload of the <code class="literal">Message</code> type from the wire format to a <code class="literal">Person</code> type.
|
|
In other words, the framework must locate and apply the appropriate <code class="literal">MessageConverter</code>.
|
|
To accomplish that, the framework needs some instructions from the user.
|
|
One of these instructions is already provided by the signature of the handler method itself (<code class="literal">Person</code> type).
|
|
Consequently, in theory, that should be (and, in some cases, is) enough.
|
|
However, for the majority of use cases, in order to select the appropriate <code class="literal">MessageConverter</code>, the framework needs an additional piece of information.
|
|
That missing piece is <code class="literal">contentType</code>.</p><p>Spring Cloud Stream provides three mechanisms to define <code class="literal">contentType</code> (in order of precedence):</p><div class="orderedlist"><ol class="orderedlist" type="1"><li class="listitem"><span class="strong"><strong>HEADER</strong></span>: The <code class="literal">contentType</code> can be communicated through the Message itself. By providing a <code class="literal">contentType</code> header, you declare the content type to use to locate and apply the appropriate <code class="literal">MessageConverter</code>.</li><li class="listitem"><p class="simpara"><span class="strong"><strong>BINDING</strong></span>: The <code class="literal">contentType</code> can be set per destination binding by setting the <code class="literal">spring.cloud.stream.bindings.input.content-type</code> property.</p><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Note"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="images/note.png"></td><th align="left">Note</th></tr><tr><td align="left" valign="top"><p>The <code class="literal">input</code> segment in the property name corresponds to the actual name of the destination (which is “input” in our case). This approach lets you declare, on a per-binding basis, the content type to use to locate and apply the appropriate <code class="literal">MessageConverter</code>.</p></td></tr></table></div></li><li class="listitem"><span class="strong"><strong>DEFAULT</strong></span>: If <code class="literal">contentType</code> is not present in the <code class="literal">Message</code> header or the binding, the default <code class="literal">application/json</code> content type is used to
|
|
locate and apply the appropriate <code class="literal">MessageConverter</code>.</li></ol></div><p>As mentioned earlier, the preceding list also demonstrates the order of precedence in case of a tie. For example, a header-provided content type takes precedence over any other content type.
|
|
The same applies for a content type set on a per-binding basis, which essentially lets you override the default content type.
|
|
However, it also provides a sensible default (which was determined from community feedback).</p><p>Another reason for making <code class="literal">application/json</code> the default stems from the interoperability requirements driven by distributed microservices architectures, where producer and consumer not only run in different JVMs but can also run on different non-JVM platforms.</p><p>When the non-void handler method returns, if the the return value is already a <code class="literal">Message</code>, that <code class="literal">Message</code> becomes the payload. However, when the return value is not a <code class="literal">Message</code>, the new <code class="literal">Message</code> is constructed with the return value as the payload while inheriting
|
|
headers from the input <code class="literal">Message</code> minus the headers defined or filtered by <code class="literal">SpringIntegrationProperties.messageHandlerNotPropagatedHeaders</code>.
|
|
By default, there is only one header set there: <code class="literal">contentType</code>. This means that the new <code class="literal">Message</code> does not have <code class="literal">contentType</code> header set, thus ensuring that the <code class="literal">contentType</code> can evolve.
|
|
You can always opt out of returning a <code class="literal">Message</code> from the handler method where you can inject any header you wish.</p><p>If there is an internal pipeline, the <code class="literal">Message</code> is sent to the next handler by going through the same process of conversion. However, if there is no internal pipeline or you have reached the end of it, the <code class="literal">Message</code> is sent back to the output destination.</p><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="_content_type_versus_argument_type" href="#_content_type_versus_argument_type"></a>9.1.1 Content Type versus Argument Type</h3></div></div></div><p>As mentioned earlier, for the framework to select the appropriate <code class="literal">MessageConverter</code>, it requires argument type and, optionally, content type information.
|
|
The logic for selecting the appropriate <code class="literal">MessageConverter</code> resides with the argument resolvers (<code class="literal">HandlerMethodArgumentResolvers</code>), which trigger right before the invocation of the user-defined handler method (which is when the actual argument type is known to the framework).
|
|
If the argument type does not match the type of the current payload, the framework delegates to the stack of the
|
|
pre-configured <code class="literal">MessageConverters</code> to see if any one of them can convert the payload.
|
|
As you can see, the <code class="literal">Object fromMessage(Message<?> message, Class<?> targetClass);</code>
|
|
operation of the MessageConverter takes <code class="literal">targetClass</code> as one of its arguments.
|
|
The framework also ensures that the provided <code class="literal">Message</code> always contains a <code class="literal">contentType</code> header.
|
|
When no contentType header was already present, it injects either the per-binding <code class="literal">contentType</code> header or the default <code class="literal">contentType</code> header.
|
|
The combination of <code class="literal">contentType</code> argument type is the mechanism by which framework determines if message can be converted to a target type.
|
|
If no appropriate <code class="literal">MessageConverter</code> is found, an exception is thrown, which you can handle by adding a custom <code class="literal">MessageConverter</code> (see <span class="quote">“<span class="quote"><a class="xref" href="#spring-cloud-stream-overview-user-defined-message-converters" title="9.3 User-defined Message Converters">Section 9.3, “User-defined Message Converters”</a></span>”</span>).</p><p>But what if the payload type matches the target type declared by the handler method? In this case, there is nothing to convert, and the
|
|
payload is passed unmodified. While this sounds pretty straightforward and logical, keep in mind handler methods that take a <code class="literal">Message<?></code> or <code class="literal">Object</code> as an argument.
|
|
By declaring the target type to be <code class="literal">Object</code> (which is an <code class="literal">instanceof</code> everything in Java), you essentially forfeit the conversion process.</p><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Note"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="images/note.png"></td><th align="left">Note</th></tr><tr><td align="left" valign="top"><p>Do not expect <code class="literal">Message</code> to be converted into some other type based only on the <code class="literal">contentType</code>.
|
|
Remember that the <code class="literal">contentType</code> is complementary to the target type.
|
|
If you wish, you can provide a hint, which <code class="literal">MessageConverter</code> may or may not take into consideration.</p></td></tr></table></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="_message_converters" href="#_message_converters"></a>9.1.2 Message Converters</h3></div></div></div><p><code class="literal">MessageConverters</code> define two methods:</p><pre class="programlisting">Object fromMessage(Message<?> message, Class<?> targetClass);
|
|
|
|
Message<?> toMessage(Object payload, <em><span class="hl-annotation" style="color: gray">@Nullable</span></em> MessageHeaders headers);</pre><p>It is important to understand the contract of these methods and their usage, specifically in the context of Spring Cloud Stream.</p><p>The <code class="literal">fromMessage</code> method converts an incoming <code class="literal">Message</code> to an argument type.
|
|
The payload of the <code class="literal">Message</code> could be any type, and it is
|
|
up to the actual implementation of the <code class="literal">MessageConverter</code> to support multiple types.
|
|
For example, some JSON converter may support the payload type as <code class="literal">byte[]</code>, <code class="literal">String</code>, and others.
|
|
This is important when the application contains an internal pipeline (that is, input → handler1 → handler2 →. . . → output) and the output of the upstream handler results in a <code class="literal">Message</code> which may not be in the initial wire format.</p><p>However, the <code class="literal">toMessage</code> method has a more strict contract and must always convert <code class="literal">Message</code> to the wire format: <code class="literal">byte[]</code>.</p><p>So, for all intents and purposes (and especially when implementing your own converter) you regard the two methods as having the following signatures:</p><pre class="programlisting">Object fromMessage(Message<?> message, Class<?> targetClass);
|
|
|
|
Message<<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">byte</span>[]> toMessage(Object payload, <em><span class="hl-annotation" style="color: gray">@Nullable</span></em> MessageHeaders headers);</pre></div></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="_provided_messageconverters" href="#_provided_messageconverters"></a>9.2 Provided MessageConverters</h2></div></div></div><p>As mentioned earlier, the framework already provides a stack of <code class="literal">MessageConverters</code> to handle most common use cases.
|
|
The following list describes the provided <code class="literal">MessageConverters</code>, in order of precedence (the first <code class="literal">MessageConverter</code> that works is used):</p><div class="orderedlist"><ol class="orderedlist" type="1"><li class="listitem"><code class="literal">ApplicationJsonMessageMarshallingConverter</code>: Variation of the <code class="literal">org.springframework.messaging.converter.MappingJackson2MessageConverter</code>. Supports conversion of the payload of the <code class="literal">Message</code> to/from POJO for cases when <code class="literal">contentType</code> is <code class="literal">application/json</code> (DEFAULT).</li><li class="listitem"><code class="literal">TupleJsonMessageConverter</code>: <span class="strong"><strong>DEPRECATED</strong></span> Supports conversion of the payload of the <code class="literal">Message</code> to/from <code class="literal">org.springframework.tuple.Tuple</code>.</li><li class="listitem"><code class="literal">ByteArrayMessageConverter</code>: Supports conversion of the payload of the <code class="literal">Message</code> from <code class="literal">byte[]</code> to <code class="literal">byte[]</code> for cases when <code class="literal">contentType</code> is <code class="literal">application/octet-stream</code>. It is essentially a pass through and exists primarily for backward compatibility.</li><li class="listitem"><code class="literal">ObjectStringMessageConverter</code>: Supports conversion of any type to a <code class="literal">String</code> when <code class="literal">contentType</code> is <code class="literal">text/plain</code>.
|
|
It invokes Object’s <code class="literal">toString()</code> method or, if the payload is <code class="literal">byte[]</code>, a new <code class="literal">String(byte[])</code>.</li><li class="listitem"><code class="literal">JavaSerializationMessageConverter</code>: <span class="strong"><strong>DEPRECATED</strong></span> Supports conversion based on java serialization when <code class="literal">contentType</code> is <code class="literal">application/x-java-serialized-object</code>.</li><li class="listitem"><code class="literal">KryoMessageConverter</code>: <span class="strong"><strong>DEPRECATED</strong></span> Supports conversion based on Kryo serialization when <code class="literal">contentType</code> is <code class="literal">application/x-java-object</code>.</li><li class="listitem"><code class="literal">JsonUnmarshallingConverter</code>: Similar to the <code class="literal">ApplicationJsonMessageMarshallingConverter</code>. It supports conversion of any type when <code class="literal">contentType</code> is <code class="literal">application/x-java-object</code>.
|
|
It expects the actual type information to be embedded in the <code class="literal">contentType</code> as an attribute (for example, <code class="literal">application/x-java-object;type=foo.bar.Cat</code>).</li></ol></div><p>When no appropriate converter is found, the framework throws an exception. When that happens, you should check your code and configuration and ensure you did not miss anything (that is, ensure that you provided a <code class="literal">contentType</code> by using a binding or a header).
|
|
However, most likely, you found some uncommon case (such as a custom <code class="literal">contentType</code> perhaps) and the current stack of provided <code class="literal">MessageConverters</code>
|
|
does not know how to convert. If that is the case, you can add custom <code class="literal">MessageConverter</code>. See <a class="xref" href="#spring-cloud-stream-overview-user-defined-message-converters" title="9.3 User-defined Message Converters">Section 9.3, “User-defined Message Converters”</a>.</p></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="spring-cloud-stream-overview-user-defined-message-converters" href="#spring-cloud-stream-overview-user-defined-message-converters"></a>9.3 User-defined Message Converters</h2></div></div></div><p>Spring Cloud Stream exposes a mechanism to define and register additional <code class="literal">MessageConverters</code>.
|
|
To use it, implement <code class="literal">org.springframework.messaging.converter.MessageConverter</code>, configure it as a <code class="literal">@Bean</code>, and annotate it with <code class="literal">@StreamMessageConverter</code>.
|
|
It is then apended to the existing stack of `MessageConverter`s.</p><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Note"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="images/note.png"></td><th align="left">Note</th></tr><tr><td align="left" valign="top"><p>It is important to understand that custom <code class="literal">MessageConverter</code> implementations are added to the head of the existing stack.
|
|
Consequently, custom <code class="literal">MessageConverter</code> implementations take precedence over the existing ones, which lets you override as well as add to the existing converters.</p></td></tr></table></div><p>The following example shows how to create a message converter bean to support a new content type called <code class="literal">application/bar</code>:</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@EnableBinding(Sink.class)</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@SpringBootApplication</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">static</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> SinkApplication {
|
|
|
|
...
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Bean</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@StreamMessageConverter</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> MessageConverter customMessageConverter() {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">new</span> MyCustomMessageConverter();
|
|
}
|
|
}
|
|
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> MyCustomMessageConverter <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">extends</span> AbstractMessageConverter {
|
|
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> MyCustomMessageConverter() {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">super</span>(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">new</span> MimeType(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"application"</span>, <span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"bar"</span>));
|
|
}
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Override</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">protected</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">boolean</span> supports(Class<?> clazz) {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> (Bar.<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span>.equals(clazz));
|
|
}
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Override</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">protected</span> Object convertFromInternal(Message<?> message, Class<?> targetClass, Object conversionHint) {
|
|
Object payload = message.getPayload();
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> (payload <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">instanceof</span> Bar ? payload : <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">new</span> Bar((<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">byte</span>[]) payload));
|
|
}
|
|
}</pre><p>Spring Cloud Stream also provides support for Avro-based converters and schema evolution.
|
|
See <span class="quote">“<span class="quote"><a class="xref" href="#schema-evolution" title="10. Schema Evolution Support">Chapter 10, <i>Schema Evolution Support</i></a></span>”</span> for details.</p></div></div><div class="chapter"><div class="titlepage"><div><div><h2 class="title"><a name="schema-evolution" href="#schema-evolution"></a>10. Schema Evolution Support</h2></div></div></div><p>Spring Cloud Stream provides support for schema evolution so that the data can be evolved over time and still work with older or newer producers and consumers and vice versa.
|
|
Most serialization models, especially the ones that aim for portability across different platforms and languages, rely on a schema that describes how the data is serialized in the binary payload.
|
|
In order to serialize the data and then to interpret it, both the sending and receiving sides must have access to a schema that describes the binary format.
|
|
In certain cases, the schema can be inferred from the payload type on serialization or from the target type on deserialization.
|
|
However, many applications benefit from having access to an explicit schema that describes the binary data format.
|
|
A schema registry lets you store schema information in a textual format (typically JSON) and makes that information accessible to various applications that need it to receive and send data in binary format.
|
|
A schema is referenceable as a tuple consisting of:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem">A subject that is the logical name of the schema</li><li class="listitem">The schema version</li><li class="listitem">The schema format, which describes the binary format of the data</li></ul></div><p>This following sections goes through the details of various components involved in schema evolution process.</p><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="_schema_registry_client" href="#_schema_registry_client"></a>10.1 Schema Registry Client</h2></div></div></div><p>The client-side abstraction for interacting with schema registry servers is the <code class="literal">SchemaRegistryClient</code> interface, which has the following structure:</p><pre class="programlisting"><span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">interface</span> SchemaRegistryClient {
|
|
|
|
SchemaRegistrationResponse register(String subject, String format, String schema);
|
|
|
|
String fetch(SchemaReference schemaReference);
|
|
|
|
String fetch(Integer id);
|
|
|
|
}</pre><p>Spring Cloud Stream provides out-of-the-box implementations for interacting with its own schema server and for interacting with the Confluent Schema Registry.</p><p>A client for the Spring Cloud Stream schema registry can be configured by using the <code class="literal">@EnableSchemaRegistryClient</code>, as follows:</p><pre class="programlisting"> <em><span class="hl-annotation" style="color: gray">@EnableBinding(Sink.class)</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@SpringBootApplication</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@EnableSchemaRegistryClient</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">static</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> AvroSinkApplication {
|
|
...
|
|
}</pre><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Note"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="images/note.png"></td><th align="left">Note</th></tr><tr><td align="left" valign="top"><p>The default converter is optimized to cache not only the schemas from the remote server but also the <code class="literal">parse()</code> and <code class="literal">toString()</code> methods, which are quite expensive.
|
|
Because of this, it uses a <code class="literal">DefaultSchemaRegistryClient</code> that does not cache responses.
|
|
If you intend to change the default behavior, you can use the client directly on your code and override it to the desired outcome.
|
|
To do so, you have to add the property <code class="literal">spring.cloud.stream.schemaRegistryClient.cached=true</code> to your application properties.</p></td></tr></table></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="_schema_registry_client_properties" href="#_schema_registry_client_properties"></a>10.1.1 Schema Registry Client Properties</h3></div></div></div><p>The Schema Registry Client supports the following properties:</p><div class="variablelist"><dl class="variablelist"><dt><span class="term"><code class="literal">spring.cloud.stream.schemaRegistryClient.endpoint</code></span></dt><dd>The location of the schema-server.
|
|
When setting this, use a full URL, including protocol (<code class="literal">http</code> or <code class="literal">https</code>) , port, and context path.</dd><dt><span class="term">Default</span></dt><dd><code class="literal"><a class="link" href="http://localhost:8990/" target="_top">http://localhost:8990/</a></code></dd><dt><span class="term"><code class="literal">spring.cloud.stream.schemaRegistryClient.cached</code></span></dt><dd>Whether the client should cache schema server responses.
|
|
Normally set to <code class="literal">false</code>, as the caching happens in the message converter.
|
|
Clients using the schema registry client should set this to <code class="literal">true</code>.</dd><dt><span class="term">Default</span></dt><dd><code class="literal">false</code></dd></dl></div></div></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="_avro_schema_registry_client_message_converters" href="#_avro_schema_registry_client_message_converters"></a>10.2 Avro Schema Registry Client Message Converters</h2></div></div></div><p>For applications that have a SchemaRegistryClient bean registered with the application context, Spring Cloud Stream auto configures an Apache Avro message converter for schema management.
|
|
This eases schema evolution, as applications that receive messages can get easy access to a writer schema that can be reconciled with their own reader schema.</p><p>For outbound messages, if the content type of the channel is set to <code class="literal">application/*+avro</code>, the <code class="literal">MessageConverter</code> is activated, as shown in the following example:</p><pre class="programlisting"><span xmlns:d="http://docbook.org/ns/docbook" class="hl-attribute">spring.cloud.stream.bindings.output.contentType</span>=application/*+avro</pre><p>During the outbound conversion, the message converter tries to infer the schema of each outbound messages (based on its type) and register it to a subject (based on the payload type) by using the <code class="literal">SchemaRegistryClient</code>.
|
|
If an identical schema is already found, then a reference to it is retrieved.
|
|
If not, the schema is registered, and a new version number is provided.
|
|
The message is sent with a <code class="literal">contentType</code> header by using the following scheme: <code class="literal">application/[prefix].[subject].v[version]+avro</code>, where <code class="literal">prefix</code> is configurable and <code class="literal">subject</code> is deduced from the payload type.</p><p>For example, a message of the type <code class="literal">User</code> might be sent as a binary payload with a content type of <code class="literal">application/vnd.user.v2+avro</code>, where <code class="literal">user</code> is the subject and <code class="literal">2</code> is the version number.</p><p>When receiving messages, the converter infers the schema reference from the header of the incoming message and tries to retrieve it. The schema is used as the writer schema in the deserialization process.</p><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="_avro_schema_registry_message_converter_properties" href="#_avro_schema_registry_message_converter_properties"></a>10.2.1 Avro Schema Registry Message Converter Properties</h3></div></div></div><p>If you have enabled Avro based schema registry client by setting <code class="literal">spring.cloud.stream.bindings.output.contentType=application/*+avro</code>, you can customize the behavior of the registration by setting the following properties.</p><div class="variablelist"><dl class="variablelist"><dt><span class="term">spring.cloud.stream.schema.avro.dynamicSchemaGenerationEnabled</span></dt><dd><p class="simpara">Enable if you want the converter to use reflection to infer a Schema from a POJO.</p><p class="simpara">Default: <code class="literal">false</code></p></dd><dt><span class="term">spring.cloud.stream.schema.avro.readerSchema</span></dt><dd>Avro compares schema versions by looking at a writer schema (origin payload) and a reader schema (your application payload). See the <a class="link" href="https://avro.apache.org/docs/1.7.6/spec.html" target="_top">Avro documentation</a> for more information. If set, this overrides any lookups at the schema server and uses the local schema as the reader schema.
|
|
Default: <code class="literal">null</code></dd><dt><span class="term">spring.cloud.stream.schema.avro.schemaLocations</span></dt><dd><p class="simpara">Registers any <code class="literal">.avsc</code> files listed in this property with the Schema Server.</p><p class="simpara">Default: <code class="literal">empty</code></p></dd><dt><span class="term">spring.cloud.stream.schema.avro.prefix</span></dt><dd><p class="simpara">The prefix to be used on the Content-Type header.</p><p class="simpara">Default: <code class="literal">vnd</code></p></dd></dl></div></div></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="_apache_avro_message_converters" href="#_apache_avro_message_converters"></a>10.3 Apache Avro Message Converters</h2></div></div></div><p>Spring Cloud Stream provides support for schema-based message converters through its <code class="literal">spring-cloud-stream-schema</code> module.
|
|
Currently, the only serialization format supported out of the box for schema-based message converters is Apache Avro, with more formats to be added in future versions.</p><p>The <code class="literal">spring-cloud-stream-schema</code> module contains two types of message converters that can be used for Apache Avro serialization:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem">Converters that use the class information of the serialized or deserialized objects or a schema with a location known at startup.</li><li class="listitem">Converters that use a schema registry. They locate the schemas at runtime and dynamically register new schemas as domain objects evolve.</li></ul></div></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="_converters_with_schema_support" href="#_converters_with_schema_support"></a>10.4 Converters with Schema Support</h2></div></div></div><p>The <code class="literal">AvroSchemaMessageConverter</code> supports serializing and deserializing messages either by using a predefined schema or by using the schema information available in the class (either reflectively or contained in the <code class="literal">SpecificRecord</code>).
|
|
If you provide a custom converter, then the default AvroSchemaMessageConverter bean is not created. The following example shows a custom converter:</p><p>To use custom converters, you can simply add it to the application context, optionally specifying one or more <code class="literal">MimeTypes</code> with which to associate it.
|
|
The default <code class="literal">MimeType</code> is <code class="literal">application/avro</code>.</p><p>If the target type of the conversion is a <code class="literal">GenericRecord</code>, a schema must be set.</p><p>The following example shows how to configure a converter in a sink application by registering the Apache Avro <code class="literal">MessageConverter</code> without a predefined schema.
|
|
In this example, note that the mime type value is <code class="literal">avro/bytes</code>, not the default <code class="literal">application/avro</code>.</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@EnableBinding(Sink.class)</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@SpringBootApplication</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">static</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> SinkApplication {
|
|
|
|
...
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Bean</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> MessageConverter userMessageConverter() {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">new</span> AvroSchemaMessageConverter(MimeType.valueOf(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"avro/bytes"</span>));
|
|
}
|
|
}</pre><p>Conversely, the following application registers a converter with a predefined schema (found on the classpath):</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@EnableBinding(Sink.class)</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@SpringBootApplication</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">static</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> SinkApplication {
|
|
|
|
...
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Bean</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> MessageConverter userMessageConverter() {
|
|
AvroSchemaMessageConverter converter = <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">new</span> AvroSchemaMessageConverter(MimeType.valueOf(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"avro/bytes"</span>));
|
|
converter.setSchemaLocation(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">new</span> ClassPathResource(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"schemas/User.avro"</span>));
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> converter;
|
|
}
|
|
}</pre></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="_schema_registry_server" href="#_schema_registry_server"></a>10.5 Schema Registry Server</h2></div></div></div><p>Spring Cloud Stream provides a schema registry server implementation.
|
|
To use it, you can add the <code class="literal">spring-cloud-stream-schema-server</code> artifact to your project and use the <code class="literal">@EnableSchemaRegistryServer</code> annotation, which adds the schema registry server REST controller to your application.
|
|
This annotation is intended to be used with Spring Boot web applications, and the listening port of the server is controlled by the <code class="literal">server.port</code> property.
|
|
The <code class="literal">spring.cloud.stream.schema.server.path</code> property can be used to control the root path of the schema server (especially when it is embedded in other applications).
|
|
The <code class="literal">spring.cloud.stream.schema.server.allowSchemaDeletion</code> boolean property enables the deletion of a schema. By default, this is disabled.</p><p>The schema registry server uses a relational database to store the schemas.
|
|
By default, it uses an embedded database.
|
|
You can customize the schema storage by using the <a class="link" href="http://docs.spring.io/spring-boot/docs/current-SNAPSHOT/reference/htmlsingle/#boot-features-sql" target="_top">Spring Boot SQL database and JDBC configuration options</a>.</p><p>The following example shows a Spring Boot application that enables the schema registry:</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@SpringBootApplication</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@EnableSchemaRegistryServer</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> SchemaRegistryServerApplication {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">static</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">void</span> main(String[] args) {
|
|
SpringApplication.run(SchemaRegistryServerApplication.<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span>, args);
|
|
}
|
|
}</pre><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="_schema_registry_server_api" href="#_schema_registry_server_api"></a>10.5.1 Schema Registry Server API</h3></div></div></div><p>The Schema Registry Server API consists of the following operations:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><code class="literal">POST /</code> — see <span class="quote">“<span class="quote"><a class="xref" href="#spring-cloud-stream-overview-registering-new-schema" title="Registering a New Schema">the section called “Registering a New Schema”</a></span>”</span></li><li class="listitem">'GET /{subject}/{format}/{version}' — see <span class="quote">“<span class="quote"><a class="xref" href="#spring-cloud-stream-overview-retrieve-schema-subject-format-version" title="Retrieving an Existing Schema by Subject, Format, and Version">the section called “Retrieving an Existing Schema by Subject, Format, and Version”</a></span>”</span></li><li class="listitem"><code class="literal">GET /{subject}/{format}</code> — see <span class="quote">“<span class="quote"><a class="xref" href="#spring-cloud-stream-overview-retrieve-schema-subject-format" title="Retrieving an Existing Schema by Subject and Format">the section called “Retrieving an Existing Schema by Subject and Format”</a></span>”</span></li><li class="listitem"><code class="literal">GET /schemas/{id}</code> — see <span class="quote">“<span class="quote"><a class="xref" href="#spring-cloud-stream-overview-retrieve-schema-id" title="Retrieving an Existing Schema by ID">the section called “Retrieving an Existing Schema by ID”</a></span>”</span></li><li class="listitem"><code class="literal">DELETE /{subject}/{format}/{version}</code> — see <span class="quote">“<span class="quote"><a class="xref" href="#spring-cloud-stream-overview-deleting-schema-subject-format-version" title="Deleting a Schema by Subject, Format, and Version">the section called “Deleting a Schema by Subject, Format, and Version”</a></span>”</span></li><li class="listitem"><code class="literal">DELETE /schemas/{id}</code> — see <span class="quote">“<span class="quote"><a class="xref" href="#spring-cloud-stream-overview-deleting-schema-id" title="Deleting a Schema by ID">the section called “Deleting a Schema by ID”</a></span>”</span></li><li class="listitem"><code class="literal">DELETE /{subject}</code> — see <span class="quote">“<span class="quote"><a class="xref" href="#spring-cloud-stream-overview-deleting-schema-subject" title="Deleting a Schema by Subject">the section called “Deleting a Schema by Subject”</a></span>”</span></li></ul></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a name="spring-cloud-stream-overview-registering-new-schema" href="#spring-cloud-stream-overview-registering-new-schema"></a>Registering a New Schema</h4></div></div></div><p>To register a new schema, send a <code class="literal">POST</code> request to the <code class="literal">/</code> endpoint.</p><p>The <code class="literal">/</code> accepts a JSON payload with the following fields:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><code class="literal">subject</code>: The schema subject</li><li class="listitem"><code class="literal">format</code>: The schema format</li><li class="listitem"><code class="literal">definition</code>: The schema definition</li></ul></div><p>Its response is a schema object in JSON, with the following fields:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><code class="literal">id</code>: The schema ID</li><li class="listitem"><code class="literal">subject</code>: The schema subject</li><li class="listitem"><code class="literal">format</code>: The schema format</li><li class="listitem"><code class="literal">version</code>: The schema version</li><li class="listitem"><code class="literal">definition</code>: The schema definition</li></ul></div></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a name="spring-cloud-stream-overview-retrieve-schema-subject-format-version" href="#spring-cloud-stream-overview-retrieve-schema-subject-format-version"></a>Retrieving an Existing Schema by Subject, Format, and Version</h4></div></div></div><p>To retrieve an existing schema by subject, format, and version, send <code class="literal">GET</code> request to the <code class="literal">/{subject}/{format}/{version}</code> endpoint.</p><p>Its response is a schema object in JSON, with the following fields:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><code class="literal">id</code>: The schema ID</li><li class="listitem"><code class="literal">subject</code>: The schema subject</li><li class="listitem"><code class="literal">format</code>: The schema format</li><li class="listitem"><code class="literal">version</code>: The schema version</li><li class="listitem"><code class="literal">definition</code>: The schema definition</li></ul></div></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a name="spring-cloud-stream-overview-retrieve-schema-subject-format" href="#spring-cloud-stream-overview-retrieve-schema-subject-format"></a>Retrieving an Existing Schema by Subject and Format</h4></div></div></div><p>To retrieve an existing schema by subject and format, send a <code class="literal">GET</code> request to the <code class="literal">/subject/format</code> endpoint.</p><p>Its response is a list of schemas with each schema object in JSON, with the following fields:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><code class="literal">id</code>: The schema ID</li><li class="listitem"><code class="literal">subject</code>: The schema subject</li><li class="listitem"><code class="literal">format</code>: The schema format</li><li class="listitem"><code class="literal">version</code>: The schema version</li><li class="listitem"><code class="literal">definition</code>: The schema definition</li></ul></div></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a name="spring-cloud-stream-overview-retrieve-schema-id" href="#spring-cloud-stream-overview-retrieve-schema-id"></a>Retrieving an Existing Schema by ID</h4></div></div></div><p>To retrieve a schema by its ID, send a <code class="literal">GET</code> request to the <code class="literal">/schemas/{id}</code> endpoint.</p><p>Its response is a schema object in JSON, with the following fields:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><code class="literal">id</code>: The schema ID</li><li class="listitem"><code class="literal">subject</code>: The schema subject</li><li class="listitem"><code class="literal">format</code>: The schema format</li><li class="listitem"><code class="literal">version</code>: The schema version</li><li class="listitem"><code class="literal">definition</code>: The schema definition</li></ul></div></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a name="spring-cloud-stream-overview-deleting-schema-subject-format-version" href="#spring-cloud-stream-overview-deleting-schema-subject-format-version"></a>Deleting a Schema by Subject, Format, and Version</h4></div></div></div><p>To delete a schema identified by its subject, format, and version, send a <code class="literal">DELETE</code> request to the <code class="literal">/{subject}/{format}/{version}</code> endpoint.</p></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a name="spring-cloud-stream-overview-deleting-schema-id" href="#spring-cloud-stream-overview-deleting-schema-id"></a>Deleting a Schema by ID</h4></div></div></div><p>To delete a schema by its ID, send a <code class="literal">DELETE</code> request to the <code class="literal">/schemas/{id}</code> endpoint.</p></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a name="spring-cloud-stream-overview-deleting-schema-subject" href="#spring-cloud-stream-overview-deleting-schema-subject"></a>Deleting a Schema by Subject</h4></div></div></div><p><code class="literal">DELETE /{subject}</code></p><p>Delete existing schemas by their subject.</p><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Note"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="images/note.png"></td><th align="left">Note</th></tr><tr><td align="left" valign="top"><p>This note applies to users of Spring Cloud Stream 1.1.0.RELEASE only.
|
|
Spring Cloud Stream 1.1.0.RELEASE used the table name, <code class="literal">schema</code>, for storing <code class="literal">Schema</code> objects. <code class="literal">Schema</code> is a keyword in a number of database implementations.
|
|
To avoid any conflicts in the future, starting with 1.1.1.RELEASE, we have opted for the name <code class="literal">SCHEMA_REPOSITORY</code> for the storage table.
|
|
Any Spring Cloud Stream 1.1.0.RELEASE users who upgrade should migrate their existing schemas to the new table before upgrading.</p></td></tr></table></div></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="_using_confluents_schema_registry" href="#_using_confluents_schema_registry"></a>10.5.2 Using Confluent’s Schema Registry</h3></div></div></div><p>The default configuration creates a <code class="literal">DefaultSchemaRegistryClient</code> bean.
|
|
If you want to use the Confluent schema registry, you need to create a bean of type <code class="literal">ConfluentSchemaRegistryClient</code>, which supersedes the one configured by default by the framework. The following example shows how to create such a bean:</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@Bean</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> SchemaRegistryClient schemaRegistryClient(<em><span class="hl-annotation" style="color: gray">@Value("${spring.cloud.stream.schemaRegistryClient.endpoint}")</span></em> String endpoint){
|
|
ConfluentSchemaRegistryClient client = <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">new</span> ConfluentSchemaRegistryClient();
|
|
client.setEndpoint(endpoint);
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> client;
|
|
}</pre><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Note"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="images/note.png"></td><th align="left">Note</th></tr><tr><td align="left" valign="top"><p>The ConfluentSchemaRegistryClient is tested against Confluent platform version 4.0.0.</p></td></tr></table></div></div></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="_schema_registration_and_resolution" href="#_schema_registration_and_resolution"></a>10.6 Schema Registration and Resolution</h2></div></div></div><p>To better understand how Spring Cloud Stream registers and resolves new schemas and its use of Avro schema comparison features, we provide two separate subsections:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><span class="quote">“<span class="quote"><a class="xref" href="#spring-cloud-stream-overview-schema-registration-process" title="10.6.1 Schema Registration Process (Serialization)">Section 10.6.1, “Schema Registration Process (Serialization)”</a></span>”</span></li><li class="listitem"><span class="quote">“<span class="quote"><a class="xref" href="#spring-cloud-stream-overview-schema-resolution-process" title="10.6.2 Schema Resolution Process (Deserialization)">Section 10.6.2, “Schema Resolution Process (Deserialization)”</a></span>”</span></li></ul></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="spring-cloud-stream-overview-schema-registration-process" href="#spring-cloud-stream-overview-schema-registration-process"></a>10.6.1 Schema Registration Process (Serialization)</h3></div></div></div><p>The first part of the registration process is extracting a schema from the payload that is being sent over a channel.
|
|
Avro types such as <code class="literal">SpecificRecord</code> or <code class="literal">GenericRecord</code> already contain a schema, which can be retrieved immediately from the instance.
|
|
In the case of POJOs, a schema is inferred if the <code class="literal">spring.cloud.stream.schema.avro.dynamicSchemaGenerationEnabled</code> property is set to <code class="literal">true</code> (the default).</p><div class="figure"><a name="d0e4090" href="#d0e4090"></a><p class="title"><b>Figure 10.1. Schema Writer Resolution Process</b></p><div class="figure-contents"><div class="mediaobject" align="center"><img src="https://raw.githubusercontent.com/spring-cloud/spring-cloud-stream/master/docs/src/main/asciidoc/images/schema_resolution.png" align="middle" alt="schema resolution"></div></div></div><br class="figure-break"><p>Ones a schema is obtained, the converter loads its metadata (version) from the remote server.
|
|
First, it queries a local cache. If no result is found, it submits the data to the server, which replies with versioning information.
|
|
The converter always caches the results to avoid the overhead of querying the Schema Server for every new message that needs to be serialized.</p><div class="figure"><a name="d0e4101" href="#d0e4101"></a><p class="title"><b>Figure 10.2. Schema Registration Process</b></p><div class="figure-contents"><div class="mediaobject" align="center"><img src="https://raw.githubusercontent.com/spring-cloud/spring-cloud-stream/master/docs/src/main/asciidoc/images/registration.png" align="middle" alt="registration"></div></div></div><br class="figure-break"><p>With the schema version information, the converter sets the <code class="literal">contentType</code> header of the message to carry the version information — for example: <code class="literal">application/vnd.user.v1+avro</code>.</p></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="spring-cloud-stream-overview-schema-resolution-process" href="#spring-cloud-stream-overview-schema-resolution-process"></a>10.6.2 Schema Resolution Process (Deserialization)</h3></div></div></div><p>When reading messages that contain version information (that is, a <code class="literal">contentType</code> header with a scheme like the one described under <span class="quote">“<span class="quote"><a class="xref" href="#spring-cloud-stream-overview-schema-registration-process" title="10.6.1 Schema Registration Process (Serialization)">Section 10.6.1, “Schema Registration Process (Serialization)”</a></span>”</span>), the converter queries the Schema server to fetch the writer schema of the message.
|
|
Once it has found the correct schema of the incoming message, it retrieves the reader schema and, by using Avro’s schema resolution support, reads it into the reader definition (setting defaults and any missing properties).</p><div class="figure"><a name="d0e4129" href="#d0e4129"></a><p class="title"><b>Figure 10.3. Schema Reading Resolution Process</b></p><div class="figure-contents"><div class="mediaobject" align="center"><img src="https://raw.githubusercontent.com/spring-cloud/spring-cloud-stream/master/docs/src/main/asciidoc/images/schema_reading.png" align="middle" alt="schema reading"></div></div></div><br class="figure-break"><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Note"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="images/note.png"></td><th align="left">Note</th></tr><tr><td align="left" valign="top"><p>You should understand the difference between a writer schema (the application that wrote the message) and a reader schema (the receiving application).
|
|
We suggest taking a moment to read <a class="link" href="https://avro.apache.org/docs/1.7.6/spec.html" target="_top">the Avro terminology</a> and understand the process.
|
|
Spring Cloud Stream always fetches the writer schema to determine how to read a message.
|
|
If you want to get Avro’s schema evolution support working, you need to make sure that a <code class="literal">readerSchema</code> was properly set for your application.</p></td></tr></table></div></div></div></div><div class="chapter"><div class="titlepage"><div><div><h2 class="title"><a name="_inter_application_communication" href="#_inter_application_communication"></a>11. Inter-Application Communication</h2></div></div></div><p>Spring Cloud Stream enables communication between applications. Inter-application communication is a complex issue spanning several concerns, as described in the following topics:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><span class="quote">“<span class="quote"><a class="xref" href="#spring-cloud-stream-overview-connecting-multiple-application-instances" title="11.1 Connecting Multiple Application Instances">Section 11.1, “Connecting Multiple Application Instances”</a></span>”</span></li><li class="listitem"><span class="quote">“<span class="quote"><a class="xref" href="#spring-cloud-stream-overview-instance-index-instance-count" title="11.2 Instance Index and Instance Count">Section 11.2, “Instance Index and Instance Count”</a></span>”</span></li><li class="listitem"><span class="quote">“<span class="quote"><a class="xref" href="#spring-cloud-stream-overview-partitioning" title="11.3 Partitioning">Section 11.3, “Partitioning”</a></span>”</span></li></ul></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="spring-cloud-stream-overview-connecting-multiple-application-instances" href="#spring-cloud-stream-overview-connecting-multiple-application-instances"></a>11.1 Connecting Multiple Application Instances</h2></div></div></div><p>While Spring Cloud Stream makes it easy for individual Spring Boot applications to connect to messaging systems, the typical scenario for Spring Cloud Stream is the creation of multi-application pipelines, where microservice applications send data to each other.
|
|
You can achieve this scenario by correlating the input and output destinations of <span class="quote">“<span class="quote">adjacent</span>”</span> applications.</p><p>Suppose a design calls for the Time Source application to send data to the Log Sink application. You could use a common destination named <code class="literal">ticktock</code> for bindings within both applications.</p><p>Time Source (that has the channel name <code class="literal">output</code>) would set the following property:</p><pre class="screen">spring.cloud.stream.bindings.output.destination=ticktock</pre><p>Log Sink (that has the channel name <code class="literal">input</code>) would set the following property:</p><pre class="screen">spring.cloud.stream.bindings.input.destination=ticktock</pre></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="spring-cloud-stream-overview-instance-index-instance-count" href="#spring-cloud-stream-overview-instance-index-instance-count"></a>11.2 Instance Index and Instance Count</h2></div></div></div><p>When scaling up Spring Cloud Stream applications, each instance can receive information about how many other instances of the same application exist and what its own instance index is.
|
|
Spring Cloud Stream does this through the <code class="literal">spring.cloud.stream.instanceCount</code> and <code class="literal">spring.cloud.stream.instanceIndex</code> properties.
|
|
For example, if there are three instances of a HDFS sink application, all three instances have <code class="literal">spring.cloud.stream.instanceCount</code> set to <code class="literal">3</code>, and the individual applications have <code class="literal">spring.cloud.stream.instanceIndex</code> set to <code class="literal">0</code>, <code class="literal">1</code>, and <code class="literal">2</code>, respectively.</p><p>When Spring Cloud Stream applications are deployed through Spring Cloud Data Flow, these properties are configured automatically; when Spring Cloud Stream applications are launched independently, these properties must be set correctly.
|
|
By default, <code class="literal">spring.cloud.stream.instanceCount</code> is <code class="literal">1</code>, and <code class="literal">spring.cloud.stream.instanceIndex</code> is <code class="literal">0</code>.</p><p>In a scaled-up scenario, correct configuration of these two properties is important for addressing partitioning behavior (see below) in general, and the two properties are always required by certain binders (for example, the Kafka binder) in order to ensure that data are split correctly across multiple consumer instances.</p></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="spring-cloud-stream-overview-partitioning" href="#spring-cloud-stream-overview-partitioning"></a>11.3 Partitioning</h2></div></div></div><p>Partitioning in Spring Cloud Stream consists of two tasks:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><span class="quote">“<span class="quote"><a class="xref" href="#spring-cloud-stream-overview-configuring-output-bindings-partitioning" title="11.3.1 Configuring Output Bindings for Partitioning">Section 11.3.1, “Configuring Output Bindings for Partitioning”</a></span>”</span></li><li class="listitem"><span class="quote">“<span class="quote"><a class="xref" href="#spring-cloud-stream-overview-configuring-input-bindings-partitioning" title="11.3.2 Configuring Input Bindings for Partitioning">Section 11.3.2, “Configuring Input Bindings for Partitioning”</a></span>”</span></li></ul></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="spring-cloud-stream-overview-configuring-output-bindings-partitioning" href="#spring-cloud-stream-overview-configuring-output-bindings-partitioning"></a>11.3.1 Configuring Output Bindings for Partitioning</h3></div></div></div><p>You can configure an output binding to send partitioned data by setting one and only one of its <code class="literal">partitionKeyExpression</code> or <code class="literal">partitionKeyExtractorName</code> properties, as well as its <code class="literal">partitionCount</code> property.</p><p>For example, the following is a valid and typical configuration:</p><pre class="screen">spring.cloud.stream.bindings.output.producer.partitionKeyExpression=payload.id
|
|
spring.cloud.stream.bindings.output.producer.partitionCount=5</pre><p>Based on that example configuration, data is sent to the target partition by using the following logic.</p><p>A partition key’s value is calculated for each message sent to a partitioned output channel based on the <code class="literal">partitionKeyExpression</code>.
|
|
The <code class="literal">partitionKeyExpression</code> is a SpEL expression that is evaluated against the outbound message for extracting the partitioning key.</p><p>If a SpEL expression is not sufficient for your needs, you can instead calculate the partition key value by providing an implementation of <code class="literal">org.springframework.cloud.stream.binder.PartitionKeyExtractorStrategy</code> and configuring it as a bean (by using the <code class="literal">@Bean</code> annotation).
|
|
If you have more then one bean of type <code class="literal">org.springframework.cloud.stream.binder.PartitionKeyExtractorStrategy</code> available in the Application Context, you can further filter it by specifying its name with the <code class="literal">partitionKeyExtractorName</code> property, as shown in the following example:</p><pre class="screen">--spring.cloud.stream.bindings.output.producer.partitionKeyExtractorName=customPartitionKeyExtractor
|
|
--spring.cloud.stream.bindings.output.producer.partitionCount=5
|
|
. . .
|
|
@Bean
|
|
public CustomPartitionKeyExtractorClass customPartitionKeyExtractor() {
|
|
return new CustomPartitionKeyExtractorClass();
|
|
}</pre><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Note"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="images/note.png"></td><th align="left">Note</th></tr><tr><td align="left" valign="top"><p>In previous versions of Spring Cloud Stream, you could specify the implementation of <code class="literal">org.springframework.cloud.stream.binder.PartitionKeyExtractorStrategy</code> by setting the <code class="literal">spring.cloud.stream.bindings.output.producer.partitionKeyExtractorClass</code> property.
|
|
Since version 2.0, this property is deprecated, and support for it will be removed in a future version.</p></td></tr></table></div><p>Once the message key is calculated, the partition selection process determines the target partition as a value between <code class="literal">0</code> and <code class="literal">partitionCount - 1</code>.
|
|
The default calculation, applicable in most scenarios, is based on the following formula: <code class="literal">key.hashCode() % partitionCount</code>.
|
|
This can be customized on the binding, either by setting a SpEL expression to be evaluated against the 'key' (through the <code class="literal">partitionSelectorExpression</code> property) or by configuring an implementation of <code class="literal">org.springframework.cloud.stream.binder.PartitionSelectorStrategy</code> as a bean (by using the @Bean annotation).
|
|
Similar to the <code class="literal">PartitionKeyExtractorStrategy</code>, you can further filter it by using the <code class="literal">spring.cloud.stream.bindings.output.producer.partitionSelectorName</code> property when more than one bean of this type is available in the Application Context, as shown in the following example:</p><pre class="screen">--spring.cloud.stream.bindings.output.producer.partitionSelectorName=customPartitionSelector
|
|
. . .
|
|
@Bean
|
|
public CustomPartitionSelectorClass customPartitionSelector() {
|
|
return new CustomPartitionSelectorClass();
|
|
}</pre><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Note"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="images/note.png"></td><th align="left">Note</th></tr><tr><td align="left" valign="top"><p>In previous versions of Spring Cloud Stream you could specify the implementation of <code class="literal">org.springframework.cloud.stream.binder.PartitionSelectorStrategy</code> by setting the <code class="literal">spring.cloud.stream.bindings.output.producer.partitionSelectorClass</code> property.
|
|
Since version 2.0, this property is deprecated and support for it will be removed in a future version.</p></td></tr></table></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="spring-cloud-stream-overview-configuring-input-bindings-partitioning" href="#spring-cloud-stream-overview-configuring-input-bindings-partitioning"></a>11.3.2 Configuring Input Bindings for Partitioning</h3></div></div></div><p>An input binding (with the channel name <code class="literal">input</code>) is configured to receive partitioned data by setting its <code class="literal">partitioned</code> property, as well as the <code class="literal">instanceIndex</code> and <code class="literal">instanceCount</code> properties on the application itself, as shown in the following example:</p><pre class="screen">spring.cloud.stream.bindings.input.consumer.partitioned=true
|
|
spring.cloud.stream.instanceIndex=3
|
|
spring.cloud.stream.instanceCount=5</pre><p>The <code class="literal">instanceCount</code> value represents the total number of application instances between which the data should be partitioned.
|
|
The <code class="literal">instanceIndex</code> must be a unique value across the multiple instances, with a value between <code class="literal">0</code> and <code class="literal">instanceCount - 1</code>.
|
|
The instance index helps each application instance to identify the unique partition(s) from which it receives data.
|
|
It is required by binders using technology that does not support partitioning natively.
|
|
For example, with RabbitMQ, there is a queue for each partition, with the queue name containing the instance index.
|
|
With Kafka, if <code class="literal">autoRebalanceEnabled</code> is <code class="literal">true</code> (default), Kafka takes care of distributing partitions across instances, and these properties are not required.
|
|
If <code class="literal">autoRebalanceEnabled</code> is set to false, the <code class="literal">instanceCount</code> and <code class="literal">instanceIndex</code> are used by the binder to determine which partition(s) the instance subscribes to (you must have at least as many partitions as there are instances).
|
|
The binder allocates the partitions instead of Kafka.
|
|
This might be useful if you want messages for a particular partition to always go to the same instance.
|
|
When a binder configuration requires them, it is important to set both values correctly in order to ensure that all of the data is consumed and that the application instances receive mutually exclusive datasets.</p><p>While a scenario in which using multiple instances for partitioned data processing may be complex to set up in a standalone case, Spring Cloud Dataflow can simplify the process significantly by populating both the input and output values correctly and by letting you rely on the runtime infrastructure to provide information about the instance index and instance count.</p></div></div></div><div class="chapter"><div class="titlepage"><div><div><h2 class="title"><a name="_testing" href="#_testing"></a>12. Testing</h2></div></div></div><p>Spring Cloud Stream provides support for testing your microservice applications without connecting to a messaging system.
|
|
You can do that by using the <code class="literal">TestSupportBinder</code> provided by the <code class="literal">spring-cloud-stream-test-support</code> library, which can be added as a test dependency to the application, as shown in the following example:</p><pre class="programlisting"> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"><dependency></span>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"><groupId></span>org.springframework.cloud<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"></groupId></span>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"><artifactId></span>spring-cloud-stream-test-support<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"></artifactId></span>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"><scope></span>test<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"></scope></span>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-tag"></dependency></span></pre><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Note"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="images/note.png"></td><th align="left">Note</th></tr><tr><td align="left" valign="top"><p>The <code class="literal">TestSupportBinder</code> uses the Spring Boot autoconfiguration mechanism to supersede the other binders found on the classpath.
|
|
Therefore, when adding a binder as a dependency, you must make sure that the <code class="literal">test</code> scope is being used.</p></td></tr></table></div><p>The <code class="literal">TestSupportBinder</code> lets you interact with the bound channels and inspect any messages sent and received by the application.</p><p>For outbound message channels, the <code class="literal">TestSupportBinder</code> registers a single subscriber and retains the messages emitted by the application in a <code class="literal">MessageCollector</code>.
|
|
They can be retrieved during tests and have assertions made against them.</p><p>You can also send messages to inbound message channels so that the consumer application can consume the messages.
|
|
The following example shows how to test both input and output channels on a processor:</p><pre class="programlisting"><em><span class="hl-annotation" style="color: gray">@RunWith(SpringRunner.class)</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@SpringBootTest(webEnvironment= SpringBootTest.WebEnvironment.RANDOM_PORT)</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> ExampleTest {
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Autowired</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">private</span> Processor processor;
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Autowired</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">private</span> MessageCollector messageCollector;
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Test</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@SuppressWarnings("unchecked")</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">void</span> testWiring() {
|
|
Message<String> message = <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">new</span> GenericMessage<>(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"hello"</span>);
|
|
processor.input().send(message);
|
|
Message<String> received = (Message<String>) messageCollector.forChannel(processor.output()).poll();
|
|
assertThat(received.getPayload(), equalTo(<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"hello world"</span>));
|
|
}
|
|
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@SpringBootApplication</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@EnableBinding(Processor.class)</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">static</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> MyProcessor {
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Autowired</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">private</span> Processor channels;
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Transformer(inputChannel = Processor.INPUT, outputChannel = Processor.OUTPUT)</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> String transform(String in) {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> in + <span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">" world"</span>;
|
|
}
|
|
}
|
|
}</pre><p>In the preceding example, we create an application that has an input channel and an output channel, both bound through the <code class="literal">Processor</code> interface.
|
|
The bound interface is injected into the test so that we can have access to both channels.
|
|
We send a message on the input channel, and we use the <code class="literal">MessageCollector</code> provided by Spring Cloud Stream’s test support to capture that the message has been sent to the output channel as a result.
|
|
Once we have received the message, we can validate that the component functions correctly.</p><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="_disabling_the_test_binder_autoconfiguration" href="#_disabling_the_test_binder_autoconfiguration"></a>12.1 Disabling the Test Binder Autoconfiguration</h2></div></div></div><p>The intent behind the test binder superseding all the other binders on the classpath is to make it easy to test your applications without making changes to your production dependencies.
|
|
In some cases (for example, integration tests) it is useful to use the actual production binders instead, and that requires disabling the test binder autoconfiguration.
|
|
To do so, you can exclude the <code class="literal">org.springframework.cloud.stream.test.binder.TestSupportBinderAutoConfiguration</code> class by using one of the Spring Boot autoconfiguration exclusion mechanisms, as shown in the following example:</p><pre class="programlisting"> <em><span class="hl-annotation" style="color: gray">@SpringBootApplication(exclude = TestSupportBinderAutoConfiguration.class)</span></em>
|
|
<em><span class="hl-annotation" style="color: gray">@EnableBinding(Processor.class)</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">static</span> <span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">class</span> MyProcessor {
|
|
|
|
<em><span class="hl-annotation" style="color: gray">@Transformer(inputChannel = Processor.INPUT, outputChannel = Processor.OUTPUT)</span></em>
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">public</span> String transform(String in) {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-keyword">return</span> in + <span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">" world"</span>;
|
|
}
|
|
}</pre><p>When autoconfiguration is disabled, the test binder is available on the classpath, and its <code class="literal">defaultCandidate</code> property is set to <code class="literal">false</code> so that it does not interfere with the regular user configuration. It can be referenced under the name, <code class="literal">test</code>, as shown in the following example:</p><p><code class="literal">spring.cloud.stream.defaultBinder=test</code></p></div></div><div class="chapter"><div class="titlepage"><div><div><h2 class="title"><a name="_health_indicator" href="#_health_indicator"></a>13. Health Indicator</h2></div></div></div><p>Spring Cloud Stream provides a health indicator for binders.
|
|
It is registered under the name <code class="literal">binders</code> and can be enabled or disabled by setting the <code class="literal">management.health.binders.enabled</code> property.</p><p>To enable health check you first need to enable both "web" and "actuator" by including its dependencies (see <a class="xref" href="#spring-cloud-stream-preface-actuator-web-dependencies" title="3.2.1 Both Actuator and Web Dependencies Are Now Optional">Section 3.2.1, “Both Actuator and Web Dependencies Are Now Optional”</a>)</p><p>If <code class="literal">management.health.binders.enabled</code> is not set explicitly by the application, then <code class="literal">management.health.defaults.enabled</code> is matched as <code class="literal">true</code> and the binder health indicators are enabled.
|
|
If you want to disable health indicator completely, then you have to set <code class="literal">management.health.binders.enabled</code> to <code class="literal">false</code>.</p><p>You can use Spring Boot actuator health endpoint to access the health indicator - <code class="literal">/actuator/health</code>.
|
|
By default, you will only receive the top level application status when you hit the above endpoint.
|
|
In order to receive the full details from the binder specific health indicators, you need to include the property <code class="literal">management.endpoint.health.show-details</code> with the value <code class="literal">ALWAYS</code> in your application.</p><p>Health indicators are binder-specific and certain binder implementations may not necessarily provide a health indicator.</p><p>If you want to completely disable all health indicators available out of the box and instead provide your own health indicators,
|
|
you can do so by setting property <code class="literal">management.health.binders.enabled</code> to <code class="literal">false</code> and then provide your own <code class="literal">HealthIndicator</code> beans in your application.
|
|
In this case, the health indicator infrastructure from Spring Boot will still pick up these custom beans.
|
|
Even if you are not disabling the binder health indicators, you can still enhance the health checks by providing your own <code class="literal">HealthIndicator</code> beans in addition to the out of the box health checks.</p><p>When you have multiple binders in the same application, health indicators are enabled by default unless the application turns them off by setting <code class="literal">management.health.binders.enabled</code> to <code class="literal">false</code>.
|
|
In this case, if the user wants to disable health check for a subset of the binders, then that should be done by setting <code class="literal">management.health.binders.enabled</code> to <code class="literal">false</code> in the multi binder configurations’s environment.
|
|
See <a class="link" href="#multiple-systems" title="7.5 Connecting to Multiple Systems">Connecting to Multiple Systems</a> for details on how environment specific properties can be provided.</p></div><div class="chapter"><div class="titlepage"><div><div><h2 class="title"><a name="spring-cloud-stream-overview-metrics-emitter" href="#spring-cloud-stream-overview-metrics-emitter"></a>14. Metrics Emitter</h2></div></div></div><p>Spring Boot Actuator provides dependency management and auto-configuration for <a class="link" href="https://micrometer.io/" target="_top">Micrometer</a>, an application metrics
|
|
facade that supports numerous <a class="link" href="https://docs.spring.io/spring-boot/docs/2.0.0.RELEASE/reference/htmlsingle/#production-ready-metrics" target="_top">monitoring systems</a>.</p><p>Spring Cloud Stream provides support for emitting any available micrometer-based metrics to a binding destination, allowing for periodic
|
|
collection of metric data from stream applications without relying on polling individual endpoints.</p><p>Metrics Emitter is activated by defining the <code class="literal">spring.cloud.stream.bindings.applicationMetrics.destination</code> property,
|
|
which specifies the name of the binding destination used by the current binder to publish metric messages.</p><p>For example:</p><pre class="programlisting">spring.cloud.stream.bindings.applicationMetrics.destination=myMetricDestination</pre><p>The preceding example instructs the binder to bind to <code class="literal">myMetricDestination</code> (that is, Rabbit exchange, Kafka topic, and others).</p><p>The following properties can be used for customizing the emission of metrics:</p><div class="variablelist"><dl class="variablelist"><dt><span class="term">spring.cloud.stream.metrics.key</span></dt><dd><p class="simpara">The name of the metric being emitted. Should be a unique value per application.</p><p class="simpara">Default: <code class="literal">${spring.application.name:${vcap.application.name:${spring.config.name:application}}}</code></p></dd><dt><span class="term">spring.cloud.stream.metrics.properties</span></dt><dd><p class="simpara">Allows white listing application properties that are added to the metrics payload</p><p class="simpara">Default: null.</p></dd><dt><span class="term">spring.cloud.stream.metrics.meter-filter</span></dt><dd><p class="simpara">Pattern to control the 'meters' one wants to capture.
|
|
For example, specifying <code class="literal">spring.integration.*</code> captures metric information for meters whose name starts with <code class="literal">spring.integration.</code></p><p class="simpara">Default: all 'meters' are captured.</p></dd><dt><span class="term">spring.cloud.stream.metrics.schedule-interval</span></dt><dd><p class="simpara">Interval to control the rate of publishing metric data.</p><p class="simpara">Default: 1 min</p></dd></dl></div><p>Consider the following:</p><pre class="programlisting">java -jar time-source.jar \
|
|
--spring.cloud.stream.bindings.applicationMetrics.destination=someMetrics \
|
|
--spring.cloud.stream.metrics.properties=spring.application** \
|
|
--spring.cloud.stream.metrics.meter-filter=spring.integration.*</pre><p>The following example shows the payload of the data published to the binding destination as a result of the preceding command:</p><pre class="programlisting">{
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"name"</span>: <span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"application"</span>,
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"createdTime"</span>: <span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"2018-03-23T14:48:12.700Z"</span>,
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"properties"</span>: {
|
|
},
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"metrics"</span>: [
|
|
{
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"id"</span>: {
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"name"</span>: <span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"spring.integration.send"</span>,
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"tags"</span>: [
|
|
{
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"key"</span>: <span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"exception"</span>,
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"value"</span>: <span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"none"</span>
|
|
},
|
|
{
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"key"</span>: <span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"name"</span>,
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"value"</span>: <span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"input"</span>
|
|
},
|
|
{
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"key"</span>: <span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"result"</span>,
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"value"</span>: <span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"success"</span>
|
|
},
|
|
{
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"key"</span>: <span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"type"</span>,
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"value"</span>: <span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"channel"</span>
|
|
}
|
|
],
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"type"</span>: <span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"TIMER"</span>,
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"description"</span>: <span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"Send processing time"</span>,
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"baseUnit"</span>: <span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"milliseconds"</span>
|
|
},
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"timestamp"</span>: <span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"2018-03-23T14:48:12.697Z"</span>,
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"sum"</span>: <span class="hl-number">130.340546</span>,
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"count"</span>: <span class="hl-number">6</span>,
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"mean"</span>: <span class="hl-number">21.72342433333333</span>,
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"upper"</span>: <span class="hl-number">116.176299</span>,
|
|
<span xmlns:d="http://docbook.org/ns/docbook" class="hl-string">"total"</span>: <span class="hl-number">130.340546</span>
|
|
}
|
|
]
|
|
}</pre><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><table border="0" summary="Note"><tr><td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="images/note.png"></td><th align="left">Note</th></tr><tr><td align="left" valign="top"><p>Given that the format of the Metric message has slightly changed after migrating to Micrometer, the published message will also have
|
|
a <code class="literal">STREAM_CLOUD_STREAM_VERSION</code> header set to <code class="literal">2.x</code> to help distinguish between Metric messages from the older versions of the Spring Cloud Stream.</p></td></tr></table></div></div><div class="chapter"><div class="titlepage"><div><div><h2 class="title"><a name="_samples" href="#_samples"></a>15. Samples</h2></div></div></div><p>For Spring Cloud Stream samples, see the <a class="link" href="https://github.com/spring-cloud/spring-cloud-stream-samples" target="_top">spring-cloud-stream-samples</a> repository on GitHub.</p><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="_deploying_stream_applications_on_cloudfoundry" href="#_deploying_stream_applications_on_cloudfoundry"></a>15.1 Deploying Stream Applications on CloudFoundry</h2></div></div></div><p>On CloudFoundry, services are usually exposed through a special environment variable called <a class="link" href="https://docs.cloudfoundry.org/devguide/deploy-apps/environment-variable.html#VCAP-SERVICES" target="_top">VCAP_SERVICES</a>.</p><p>When configuring your binder connections, you can use the values from an environment variable as explained on the <a class="link" href="http://docs.spring.io/spring-cloud-dataflow-server-cloudfoundry/docs/current-SNAPSHOT/reference/htmlsingle/#getting-started-ups" target="_top">dataflow Cloud Foundry Server</a> docs.</p></div></div><div class="chapter"><div class="titlepage"><div><div><h2 class="title"><a name="_binder_implementations" href="#_binder_implementations"></a>16. Binder Implementations</h2></div></div></div><p>The following is the list of available binder implementations</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><a class="link" href="https://cloud.spring.io/spring-cloud-stream-binder-rabbit/" target="_top">RabbitMQ</a></li><li class="listitem"><a class="link" href="https://cloud.spring.io/spring-cloud-stream-binder-kafka/" target="_top">Apache Kafka</a></li><li class="listitem"><a class="link" href="https://github.com/spring-cloud/spring-cloud-stream-binder-aws-kinesis" target="_top">Amazon Kinesis</a></li><li class="listitem"><a class="link" href="https://github.com/spring-cloud/spring-cloud-gcp/tree/master/spring-cloud-gcp-pubsub-stream-binder" target="_top">Google PubSub <span class="emphasis"><em>(partner maintained)</em></span></a></li><li class="listitem"><a class="link" href="https://github.com/SolaceProducts/spring-cloud-stream-binder-solace" target="_top">Solace PubSub+ <span class="emphasis"><em>(partner maintained)</em></span></a></li><li class="listitem"><a class="link" href="https://github.com/Microsoft/spring-cloud-azure/tree/master/spring-cloud-azure-eventhub-stream-binder" target="_top">Azure Event Hubs <span class="emphasis"><em>(partner maintained)</em></span></a></li></ul></div></div></div></div></body></html> |