4868 lines
225 KiB
HTML
4868 lines
225 KiB
HTML
<!DOCTYPE html>
|
||
<html lang="en">
|
||
<head>
|
||
<meta charset="UTF-8">
|
||
<!--[if IE]><meta http-equiv="X-UA-Compatible" content="IE=edge"><![endif]-->
|
||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||
<meta name="generator" content="Asciidoctor 1.5.8">
|
||
<title>Preface</title>
|
||
<link rel="stylesheet" href="css/spring.css">
|
||
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.7.0/css/font-awesome.min.css">
|
||
|
||
<style>
|
||
.hidden {
|
||
display: none;
|
||
}
|
||
|
||
.switch {
|
||
border-width: 1px 1px 0 1px;
|
||
border-style: solid;
|
||
border-color: #7a2518;
|
||
display: inline-block;
|
||
}
|
||
|
||
.switch--item {
|
||
padding: 10px;
|
||
background-color: #ffffff;
|
||
color: #7a2518;
|
||
display: inline-block;
|
||
cursor: pointer;
|
||
}
|
||
|
||
.switch--item:not(:first-child) {
|
||
border-width: 0 0 0 1px;
|
||
border-style: solid;
|
||
border-color: #7a2518;
|
||
}
|
||
|
||
.switch--item.selected {
|
||
background-color: #7a2519;
|
||
color: #ffffff;
|
||
}
|
||
</style>
|
||
<script src="https://cdnjs.cloudflare.com/ajax/libs/zepto/1.2.0/zepto.min.js"></script>
|
||
<script type="text/javascript">
|
||
function addBlockSwitches() {
|
||
$('.primary').each(function() {
|
||
primary = $(this);
|
||
createSwitchItem(primary, createBlockSwitch(primary)).item.addClass("selected");
|
||
primary.children('.title').remove();
|
||
});
|
||
$('.secondary').each(function(idx, node) {
|
||
secondary = $(node);
|
||
primary = findPrimary(secondary);
|
||
switchItem = createSwitchItem(secondary, primary.children('.switch'));
|
||
switchItem.content.addClass('hidden');
|
||
findPrimary(secondary).append(switchItem.content);
|
||
secondary.remove();
|
||
});
|
||
}
|
||
|
||
function createBlockSwitch(primary) {
|
||
blockSwitch = $('<div class="switch"></div>');
|
||
primary.prepend(blockSwitch);
|
||
return blockSwitch;
|
||
}
|
||
|
||
function findPrimary(secondary) {
|
||
candidate = secondary.prev();
|
||
while (!candidate.is('.primary')) {
|
||
candidate = candidate.prev();
|
||
}
|
||
return candidate;
|
||
}
|
||
|
||
function createSwitchItem(block, blockSwitch) {
|
||
blockName = block.children('.title').text();
|
||
content = block.children('.content').first().append(block.next('.colist'));
|
||
item = $('<div class="switch--item">' + blockName + '</div>');
|
||
item.on('click', '', content, function(e) {
|
||
$(this).addClass('selected');
|
||
$(this).siblings().removeClass('selected');
|
||
e.data.siblings('.content').addClass('hidden');
|
||
e.data.removeClass('hidden');
|
||
});
|
||
blockSwitch.append(item);
|
||
return {'item': item, 'content': content};
|
||
}
|
||
|
||
$(addBlockSwitches);
|
||
</script>
|
||
|
||
</head>
|
||
<body class="book toc2 toc-left">
|
||
<div id="header">
|
||
<div id="toc" class="toc2">
|
||
<div id="toctitle">Table of Contents</div>
|
||
<ul class="sectlevel1">
|
||
<li><a href="#spring-cloud-stream-reference">Preface</a>
|
||
<ul class="sectlevel2">
|
||
<li><a href="#_a_brief_history_of_springs_data_integration_journey">A Brief History of Spring’s Data Integration Journey</a></li>
|
||
<li><a href="#_quick_start">Quick Start</a>
|
||
<ul class="sectlevel3">
|
||
<li><a href="#spring-cloud-stream-preface-creating-sample-application">Creating a Sample Application by Using Spring Initializr</a></li>
|
||
<li><a href="#spring-cloud-stream-preface-importing-project">Importing the Project into Your IDE</a></li>
|
||
<li><a href="#spring-cloud-stream-preface-adding-message-handler">Adding a Message Handler, Building, and Running</a></li>
|
||
</ul>
|
||
</li>
|
||
</ul>
|
||
</li>
|
||
<li><a href="#_whats_new_in_3_0">What’s New in 3.0?</a>
|
||
<ul class="sectlevel2">
|
||
<li><a href="#spring-cloud-stream-preface-new-features">New Features and Enhancements</a></li>
|
||
<li><a href="#spring-cloud-stream-preface-notable-deprecations">Notable Deprecations</a></li>
|
||
</ul>
|
||
</li>
|
||
<li><a href="#spring-cloud-stream-overview-introducing">Introducing Spring Cloud Stream</a></li>
|
||
<li><a href="#_main_concepts">Main Concepts</a>
|
||
<ul class="sectlevel2">
|
||
<li><a href="#spring-cloud-stream-overview-application-model">Application Model</a>
|
||
<ul class="sectlevel3">
|
||
<li><a href="#_fat_jar">Fat JAR</a></li>
|
||
</ul>
|
||
</li>
|
||
<li><a href="#spring-cloud-stream-overview-binder-abstraction">The Binder Abstraction</a></li>
|
||
<li><a href="#spring-cloud-stream-overview-persistent-publish-subscribe-support">Persistent Publish-Subscribe Support</a></li>
|
||
<li><a href="#consumer-groups">Consumer Groups</a></li>
|
||
<li><a href="#consumer-types">Consumer Types</a>
|
||
<ul class="sectlevel3">
|
||
<li><a href="#durability">Durability</a></li>
|
||
</ul>
|
||
</li>
|
||
<li><a href="#partitioning">Partitioning Support</a></li>
|
||
</ul>
|
||
</li>
|
||
<li><a href="#_programming_model">Programming Model</a>
|
||
<ul class="sectlevel2">
|
||
<li><a href="#_destination_binders">Destination Binders</a></li>
|
||
<li><a href="#_bindings">Bindings</a>
|
||
<ul class="sectlevel3">
|
||
<li><a href="#_binding_and_binding_names">Binding and Binding names</a>
|
||
<ul class="sectlevel4">
|
||
<li><a href="#_functional_binding_names">Functional binding names</a></li>
|
||
<li><a href="#_annotation_based_binding_names_legacy">Annotation-based binding names (legacy)</a></li>
|
||
</ul>
|
||
</li>
|
||
</ul>
|
||
</li>
|
||
<li><a href="#spring-cloud-stream-overview-producing-consuming-messages">Producing and Consuming Messages</a>
|
||
<ul class="sectlevel3">
|
||
<li><a href="#spring_cloud_function">Spring Cloud Function support</a>
|
||
<ul class="sectlevel4">
|
||
<li><a href="#_overview">Overview</a></li>
|
||
<li><a href="#_suppliers_sources">Suppliers (Sources)</a></li>
|
||
<li><a href="#_consumer_reactive">Consumer (Reactive)</a>
|
||
<ul class="sectlevel5">
|
||
<li><a href="#_polling_configuration_properties">Polling Configuration Properties</a></li>
|
||
</ul>
|
||
</li>
|
||
<li><a href="#_sending_arbitrary_data_to_an_output_e_g_foreign_event_driven_sources">Sending arbitrary data to an output (e.g. Foreign event-driven sources)</a>
|
||
<ul class="sectlevel5">
|
||
<li><a href="#_using_streambridge">Using StreamBridge</a></li>
|
||
<li><a href="#_streambridge_and_dynamic_destinations">StreamBridge and Dynamic Destinations</a></li>
|
||
<li><a href="#_using_reactor_api">Using reactor API</a></li>
|
||
</ul>
|
||
</li>
|
||
<li><a href="#_reactive_functions_support">Reactive Functions support</a></li>
|
||
<li><a href="#_functional_composition">Functional Composition</a></li>
|
||
<li><a href="#_functions_with_multiple_input_and_output_arguments">Functions with multiple input and output arguments</a></li>
|
||
<li><a href="#_multiple_functions_in_a_single_application">Multiple functions in a single application</a></li>
|
||
<li><a href="#_batch_consumers">Batch Consumers</a></li>
|
||
<li><a href="#_spring_integration_flow_as_functions">Spring Integration flow as functions</a></li>
|
||
</ul>
|
||
</li>
|
||
<li><a href="#_annotation_based_support_legacy">Annotation-based support (legacy)</a>
|
||
<ul class="sectlevel4">
|
||
<li><a href="#_spring_integration_support">Spring Integration Support</a></li>
|
||
<li><a href="#_using_streamlistener_annotation">Using @StreamListener Annotation</a></li>
|
||
<li><a href="#_using_streamlistener_for_content_based_routing">Using @StreamListener for Content-based routing</a></li>
|
||
</ul>
|
||
</li>
|
||
<li><a href="#spring-cloud-streams-overview-using-polled-consumers">Using Polled Consumers</a>
|
||
<ul class="sectlevel4">
|
||
<li><a href="#_overview_2">Overview</a></li>
|
||
<li><a href="#polled-errors">Handling Errors</a></li>
|
||
</ul>
|
||
</li>
|
||
</ul>
|
||
</li>
|
||
<li><a href="#_event_routing">Event Routing</a>
|
||
<ul class="sectlevel3">
|
||
<li><a href="#_routing_to_consumer">Routing TO Consumer</a>
|
||
<ul class="sectlevel4">
|
||
<li><a href="#_using_message_headers">Using message headers</a></li>
|
||
<li><a href="#_using_application_properties">Using application properties</a></li>
|
||
</ul>
|
||
</li>
|
||
<li><a href="#_routing_from_consumer">Routing FROM Consumer</a>
|
||
<ul class="sectlevel4">
|
||
<li><a href="#_binderawarechannelresolver">BinderAwareChannelResolver</a></li>
|
||
<li><a href="#_spring_cloud_stream_sendto_destination">spring.cloud.stream.sendto.destination</a></li>
|
||
</ul>
|
||
</li>
|
||
</ul>
|
||
</li>
|
||
<li><a href="#spring-cloud-stream-overview-error-handling">Error Handling</a>
|
||
<ul class="sectlevel3">
|
||
<li><a href="#_drop_failed_messages">Drop Failed Messages</a></li>
|
||
<li><a href="#_dlq_dead_letter_queue">DLQ - Dead Letter Queue</a></li>
|
||
<li><a href="#_retry_template_and_retrybackoff">Retry Template and retryBackoff</a></li>
|
||
</ul>
|
||
</li>
|
||
</ul>
|
||
</li>
|
||
<li><a href="#spring-cloud-stream-overview-binders">Binders</a>
|
||
<ul class="sectlevel2">
|
||
<li><a href="#_producers_and_consumers">Producers and Consumers</a></li>
|
||
<li><a href="#spring-cloud-stream-overview-binder-api">Binder SPI</a></li>
|
||
<li><a href="#_binder_detection">Binder Detection</a>
|
||
<ul class="sectlevel3">
|
||
<li><a href="#_classpath_detection">Classpath Detection</a></li>
|
||
</ul>
|
||
</li>
|
||
<li><a href="#multiple-binders">Multiple Binders on the Classpath</a></li>
|
||
<li><a href="#multiple-systems">Connecting to Multiple Systems</a></li>
|
||
<li><a href="#binding_visualization_control">Binding visualization and control</a></li>
|
||
<li><a href="#_binder_configuration_properties">Binder Configuration Properties</a></li>
|
||
<li><a href="#spring-cloud-stream-overview-custom-binder-impl">Implementing Custom Binders</a></li>
|
||
</ul>
|
||
</li>
|
||
<li><a href="#_configuration_options">Configuration Options</a>
|
||
<ul class="sectlevel2">
|
||
<li><a href="#_binding_service_properties">Binding Service Properties</a></li>
|
||
<li><a href="#binding-properties">Binding Properties</a>
|
||
<ul class="sectlevel3">
|
||
<li><a href="#_common_binding_properties">Common Binding Properties</a></li>
|
||
<li><a href="#_consumer_properties">Consumer Properties</a></li>
|
||
<li><a href="#_advanced_consumer_configuration">Advanced Consumer Configuration</a></li>
|
||
<li><a href="#_producer_properties">Producer Properties</a></li>
|
||
</ul>
|
||
</li>
|
||
</ul>
|
||
</li>
|
||
<li><a href="#content-type-management">Content Type Negotiation</a>
|
||
<ul class="sectlevel2">
|
||
<li><a href="#_mechanics">Mechanics</a>
|
||
<ul class="sectlevel3">
|
||
<li><a href="#_content_type_versus_argument_type">Content Type versus Argument Type</a></li>
|
||
<li><a href="#_message_converters">Message Converters</a></li>
|
||
</ul>
|
||
</li>
|
||
<li><a href="#_provided_messageconverters">Provided MessageConverters</a></li>
|
||
<li><a href="#spring-cloud-stream-overview-user-defined-message-converters">User-defined Message Converters</a></li>
|
||
<li><a href="#spring-cloud-stream-overview-connecting-multiple-application-instances">Connecting Multiple Application Instances</a></li>
|
||
<li><a href="#spring-cloud-stream-overview-instance-index-instance-count">Instance Index and Instance Count</a></li>
|
||
<li><a href="#spring-cloud-stream-overview-partitioning">Partitioning</a>
|
||
<ul class="sectlevel3">
|
||
<li><a href="#spring-cloud-stream-overview-configuring-output-bindings-partitioning">Configuring Output Bindings for Partitioning</a></li>
|
||
<li><a href="#spring-cloud-stream-overview-configuring-input-bindings-partitioning">Configuring Input Bindings for Partitioning</a></li>
|
||
</ul>
|
||
</li>
|
||
</ul>
|
||
</li>
|
||
<li><a href="#_testing">Testing</a>
|
||
<ul class="sectlevel2">
|
||
<li><a href="#spring_integration_test_binder">Spring Integration Test Binder</a>
|
||
<ul class="sectlevel3">
|
||
<li><a href="#_test_binder_configuration">Test Binder configuration</a></li>
|
||
<li><a href="#_test_binder_usage">Test Binder usage</a></li>
|
||
<li><a href="#_test_binder_and_pollablemessagesource">Test Binder and PollableMessageSource</a></li>
|
||
</ul>
|
||
</li>
|
||
</ul>
|
||
</li>
|
||
<li><a href="#_health_indicator">Health Indicator</a></li>
|
||
<li><a href="#_samples">Samples</a>
|
||
<ul class="sectlevel2">
|
||
<li><a href="#_deploying_stream_applications_on_cloudfoundry">Deploying Stream Applications on CloudFoundry</a></li>
|
||
</ul>
|
||
</li>
|
||
<li><a href="#_binder_implementations">Binder Implementations</a></li>
|
||
</ul>
|
||
</div>
|
||
</div>
|
||
<div id="content">
|
||
<div id="preamble">
|
||
<div class="sectionbody">
|
||
<div class="paragraph">
|
||
<p><strong>3.0.5.RELEASE</strong></p>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sect1">
|
||
<h2 id="spring-cloud-stream-reference"><a class="link" href="#spring-cloud-stream-reference">Preface</a></h2>
|
||
<div class="sectionbody">
|
||
<div class="sect2">
|
||
<h3 id="_a_brief_history_of_springs_data_integration_journey"><a class="link" href="#_a_brief_history_of_springs_data_integration_journey">A Brief History of Spring’s Data Integration Journey</a></h3>
|
||
<div class="paragraph">
|
||
<p>Spring’s journey on Data Integration started with <a href="https://projects.spring.io/spring-integration/">Spring Integration</a>. With its programming model, it provided a consistent developer experience to build applications that can embrace <a href="http://www.enterpriseintegrationpatterns.com/">Enterprise Integration Patterns</a> to connect with external systems such as, databases, message brokers, and among others.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Fast forward to the cloud-era, where microservices have become prominent in the enterprise setting. <a href="https://projects.spring.io/spring-boot/">Spring Boot</a> transformed the way how developers built Applications. With Spring’s programming model and the runtime responsibilities handled by Spring Boot, it became seamless to develop stand-alone, production-grade Spring-based microservices.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>To extend this to Data Integration workloads, Spring Integration and Spring Boot were put together into a new project. Spring Cloud Stream was born.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>With Spring Cloud Stream, developers can:</p>
|
||
</div>
|
||
<div class="ulist">
|
||
<ul>
|
||
<li>
|
||
<p>Build, test and deploy data-centric applications in isolation.</p>
|
||
</li>
|
||
<li>
|
||
<p>Apply modern microservices architecture patterns, including composition through messaging.</p>
|
||
</li>
|
||
<li>
|
||
<p>Decouple application responsibilities with event-centric thinking. An event can represent something that has happened in time, to which the downstream consumer applications can react without knowing where it originated or the producer’s identity.</p>
|
||
</li>
|
||
<li>
|
||
<p>Port the business logic onto message brokers (such as RabbitMQ, Apache Kafka, Amazon Kinesis).</p>
|
||
</li>
|
||
<li>
|
||
<p>Rely on the framework’s automatic content-type support for common use-cases. Extending to different data conversion types is possible.</p>
|
||
</li>
|
||
<li>
|
||
<p>and many more. . .</p>
|
||
</li>
|
||
</ul>
|
||
</div>
|
||
</div>
|
||
<div class="sect2">
|
||
<h3 id="_quick_start"><a class="link" href="#_quick_start">Quick Start</a></h3>
|
||
<div class="paragraph">
|
||
<p>You can try Spring Cloud Stream in less then 5 min even before you jump into any details by following this three-step guide.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>We show you how to create a Spring Cloud Stream application that receives messages coming from the messaging middleware of your choice (more on this later) and logs received messages to the console.
|
||
We call it <code>LoggingConsumer</code>.
|
||
While not very practical, it provides a good introduction to some of the main concepts
|
||
and abstractions, making it easier to digest the rest of this user guide.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The three steps are as follows:</p>
|
||
</div>
|
||
<div class="olist arabic">
|
||
<ol class="arabic">
|
||
<li>
|
||
<p><a href="#spring-cloud-stream-preface-creating-sample-application">Creating a Sample Application by Using Spring Initializr</a></p>
|
||
</li>
|
||
<li>
|
||
<p><a href="#spring-cloud-stream-preface-importing-project">Importing the Project into Your IDE</a></p>
|
||
</li>
|
||
<li>
|
||
<p><a href="#spring-cloud-stream-preface-adding-message-handler">Adding a Message Handler, Building, and Running</a></p>
|
||
</li>
|
||
</ol>
|
||
</div>
|
||
<div class="sect3">
|
||
<h4 id="spring-cloud-stream-preface-creating-sample-application"><a class="link" href="#spring-cloud-stream-preface-creating-sample-application">Creating a Sample Application by Using Spring Initializr</a></h4>
|
||
<div class="paragraph">
|
||
<p>To get started, visit the <a href="https://start.spring.io">Spring Initializr</a>. From there, you can generate our <code>LoggingConsumer</code> application. To do so:</p>
|
||
</div>
|
||
<div class="olist arabic">
|
||
<ol class="arabic">
|
||
<li>
|
||
<p>In the <strong>Dependencies</strong> section, start typing <code>stream</code>.
|
||
When the “Cloud Stream” option should appears, select it.</p>
|
||
</li>
|
||
<li>
|
||
<p>Start typing either 'kafka' or 'rabbit'.</p>
|
||
</li>
|
||
<li>
|
||
<p>Select “Kafka” or “RabbitMQ”.</p>
|
||
<div class="paragraph">
|
||
<p>Basically, you choose the messaging middleware to which your application binds.
|
||
We recommend using the one you have already installed or feel more comfortable with installing and running.
|
||
Also, as you can see from the Initilaizer screen, there are a few other options you can choose.
|
||
For example, you can choose Gradle as your build tool instead of Maven (the default).</p>
|
||
</div>
|
||
</li>
|
||
<li>
|
||
<p>In the <strong>Artifact</strong> field, type 'logging-consumer'.</p>
|
||
<div class="paragraph">
|
||
<p>The value of the <strong>Artifact</strong> field becomes the application name.
|
||
If you chose RabbitMQ for the middleware, your Spring Initializr should now be as follows:</p>
|
||
</div>
|
||
</li>
|
||
</ol>
|
||
</div>
|
||
<div class="imageblock text-center">
|
||
<div class="content">
|
||
<img src="https://raw.githubusercontent.com/spring-cloud/spring-cloud-stream/master/docs/src/main/asciidoc/images/spring-initializr.png" alt="spring initializr">
|
||
</div>
|
||
</div>
|
||
<div class="olist arabic">
|
||
<ol class="arabic">
|
||
<li>
|
||
<p>Click the <strong>Generate Project</strong> button.</p>
|
||
<div class="paragraph">
|
||
<p>Doing so downloads the zipped version of the generated project to your hard drive.</p>
|
||
</div>
|
||
</li>
|
||
<li>
|
||
<p>Unzip the file into the folder you want to use as your project directory.</p>
|
||
</li>
|
||
</ol>
|
||
</div>
|
||
<div class="admonitionblock tip">
|
||
<table>
|
||
<tr>
|
||
<td class="icon">
|
||
<i class="fa icon-tip" title="Tip"></i>
|
||
</td>
|
||
<td class="content">
|
||
We encourage you to explore the many possibilities available in the Spring Initializr.
|
||
It lets you create many different kinds of Spring applications.
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
</div>
|
||
<div class="sect3">
|
||
<h4 id="spring-cloud-stream-preface-importing-project"><a class="link" href="#spring-cloud-stream-preface-importing-project">Importing the Project into Your IDE</a></h4>
|
||
<div class="paragraph">
|
||
<p>Now you can import the project into your IDE.
|
||
Keep in mind that, depending on the IDE, you may need to follow a specific import procedure.
|
||
For example, depending on how the project was generated (Maven or Gradle), you may need to follow specific import procedure (for example, in Eclipse or STS, you need to use File → Import → Maven → Existing Maven Project).</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Once imported, the project must have no errors of any kind. Also, <code>src/main/java</code> should contain <code>com.example.loggingconsumer.LoggingConsumerApplication</code>.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Technically, at this point, you can run the application’s main class.
|
||
It is already a valid Spring Boot application.
|
||
However, it does not do anything, so we want to add some code.</p>
|
||
</div>
|
||
</div>
|
||
<div class="sect3">
|
||
<h4 id="spring-cloud-stream-preface-adding-message-handler"><a class="link" href="#spring-cloud-stream-preface-adding-message-handler">Adding a Message Handler, Building, and Running</a></h4>
|
||
<div class="paragraph">
|
||
<p>Modify the <code>com.example.loggingconsumer.LoggingConsumerApplication</code> class to look as follows:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@SpringBootApplication
|
||
public class LoggingConsumerApplication {
|
||
|
||
public static void main(String[] args) {
|
||
SpringApplication.run(LoggingConsumerApplication.class, args);
|
||
}
|
||
|
||
@Bean
|
||
public Consumer<Person> log() {
|
||
return person -> {
|
||
System.out.println("Received: " + person);
|
||
};
|
||
}
|
||
|
||
public static class Person {
|
||
private String name;
|
||
public String getName() {
|
||
return name;
|
||
}
|
||
public void setName(String name) {
|
||
this.name = name;
|
||
}
|
||
public String toString() {
|
||
return this.name;
|
||
}
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>As you can see from the preceding listing:</p>
|
||
</div>
|
||
<div class="ulist">
|
||
<ul>
|
||
<li>
|
||
<p>We are using functional programming model (see <a href="#spring_cloud_function">Spring Cloud Function support</a>) to define a single message handler as <code>Consumer</code>.</p>
|
||
</li>
|
||
<li>
|
||
<p>We are relying on framework conventions to bind such handler to the input destination binding exposed by the binder.</p>
|
||
</li>
|
||
</ul>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Doing so also lets you see one of the core features of the framework: It tries to automatically convert incoming message payloads to type <code>Person</code>.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>You now have a fully functional Spring Cloud Stream application that does listens for messages.
|
||
From here, for simplicity, we assume you selected RabbitMQ in <a href="#spring-cloud-stream-preface-creating-sample-application">step one</a>.
|
||
Assuming you have RabbitMQ installed and running, you can start the application by running its <code>main</code> method in your IDE.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>You should see following output:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code> --- [ main] c.s.b.r.p.RabbitExchangeQueueProvisioner : declaring queue for inbound: input.anonymous.CbMIwdkJSBO1ZoPDOtHtCg, bound to: input
|
||
--- [ main] o.s.a.r.c.CachingConnectionFactory : Attempting to connect to: [localhost:5672]
|
||
--- [ main] o.s.a.r.c.CachingConnectionFactory : Created new connection: rabbitConnectionFactory#2a3a299:0/SimpleConnection@66c83fc8. . .
|
||
. . .
|
||
--- [ main] o.s.i.a.i.AmqpInboundChannelAdapter : started inbound.input.anonymous.CbMIwdkJSBO1ZoPDOtHtCg
|
||
. . .
|
||
--- [ main] c.e.l.LoggingConsumerApplication : Started LoggingConsumerApplication in 2.531 seconds (JVM running for 2.897)</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Go to the RabbitMQ management console or any other RabbitMQ client and send a message to <code>input.anonymous.CbMIwdkJSBO1ZoPDOtHtCg</code>.
|
||
The <code>anonymous.CbMIwdkJSBO1ZoPDOtHtCg</code> part represents the group name and is generated, so it is bound to be different in your environment.
|
||
For something more predictable, you can use an explicit group name by setting <code>spring.cloud.stream.bindings.input.group=hello</code> (or whatever name you like).</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The contents of the message should be a JSON representation of the <code>Person</code> class, as follows:</p>
|
||
</div>
|
||
<div class="literalblock">
|
||
<div class="content">
|
||
<pre>{"name":"Sam Spade"}</pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Then, in your console, you should see:</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p><code>Received: Sam Spade</code></p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>You can also build and package your application into a boot jar (by using <code>./mvnw clean install</code>) and run the built JAR by using the <code>java -jar</code> command.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Now you have a working (albeit very basic) Spring Cloud Stream application.</p>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sect1">
|
||
<h2 id="_whats_new_in_3_0"><a class="link" href="#_whats_new_in_3_0">What’s New in 3.0?</a></h2>
|
||
<div class="sectionbody">
|
||
<div class="sect2">
|
||
<h3 id="spring-cloud-stream-preface-new-features"><a class="link" href="#spring-cloud-stream-preface-new-features">New Features and Enhancements</a></h3>
|
||
<div class="ulist">
|
||
<ul>
|
||
<li>
|
||
<p><strong>Routing Function</strong> - see <a href="#Routing with functions">[Routing with functions]</a> for more details.</p>
|
||
</li>
|
||
<li>
|
||
<p><strong>Multiple bindings with functions</strong> (multiple message handlers) - see <a href="#_multiple_functions_in_a_single_application">Multiple functions in a single application</a> for more details.</p>
|
||
</li>
|
||
<li>
|
||
<p><strong>Functions with multiple inputs/outputs</strong> (single function that can subscribe or target multiple destinations) - see <a href="#_functions_with_multiple_input_and_output_arguments">Functions with multiple input and output arguments</a> for more details.</p>
|
||
</li>
|
||
<li>
|
||
<p><strong>Native support for reactive programming</strong> - since v3.0.0 we no longer distribute spring-cloud-stream-reactive modules and instead
|
||
relying on native reactive support provided by spring cloud function. For backward
|
||
compatibility you can still bring <code>spring-cloud-stream-reactive</code> from previous versions.</p>
|
||
</li>
|
||
</ul>
|
||
</div>
|
||
</div>
|
||
<div class="sect2">
|
||
<h3 id="spring-cloud-stream-preface-notable-deprecations"><a class="link" href="#spring-cloud-stream-preface-notable-deprecations">Notable Deprecations</a></h3>
|
||
<div class="ulist">
|
||
<ul>
|
||
<li>
|
||
<p><em>Reactive module</em> (<code>spring-cloud-stream-reactive</code>) is discontinued and no longer distributed in favor of native support via spring-cloud-function.
|
||
For backward
|
||
compatibility you can still bring <code>spring-cloud-stream-reactive</code> from previous versions.</p>
|
||
</li>
|
||
<li>
|
||
<p><em>Test support binder</em> <code>spring-cloud-stream-test-support</code> with MessageCollector in favor of a new test binder. See <a href="#_testing">Testing</a> for more details.</p>
|
||
</li>
|
||
<li>
|
||
<p><em>@StreamMessageConverter</em> - deprecated as it is no longer required.</p>
|
||
</li>
|
||
<li>
|
||
<p>The <code>original-content-type</code> header references have been removed after it’s been deprecated in v2.0.</p>
|
||
</li>
|
||
<li>
|
||
<p>The <code>BinderAwareChannelResolver</code> is deprecated in favor if providing <code>spring.cloud.stream.sendto.destination</code> property.
|
||
This is primarily for function-based programming model. For StreamListener it would still be required and thus will stay until we deprecate and eventually discontinue StreamListener
|
||
and annotation-based programming model.</p>
|
||
</li>
|
||
</ul>
|
||
</div>
|
||
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sect1">
|
||
<h2 id="spring-cloud-stream-overview-introducing"><a class="link" href="#spring-cloud-stream-overview-introducing">Introducing Spring Cloud Stream</a></h2>
|
||
<div class="sectionbody">
|
||
<div class="paragraph">
|
||
<p>Spring Cloud Stream is a framework for building message-driven microservice applications.
|
||
Spring Cloud Stream builds upon Spring Boot to create standalone, production-grade Spring applications and uses Spring Integration to provide connectivity to message brokers.
|
||
It provides opinionated configuration of middleware from several vendors, introducing the concepts of persistent publish-subscribe semantics, consumer groups, and partitions.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>By simply adding spring-cloud-stream dependencies to the classpath of your application you’ll get immediate connectivity
|
||
to a message broker exposed via provided spring-cloud-stream binder (more on hat later), and you can implement your functional
|
||
requirement that will be executed based on the incoming message using simple <code>java.util.function.Function</code></p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Here is the quick example:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@SpringBootApplication
|
||
public class SampleApplication {
|
||
|
||
public static void main(String[] args) {
|
||
SpringApplication.run(SampleApplication.class, args);
|
||
}
|
||
|
||
@Bean
|
||
public Function<String, String> uppercase() {
|
||
return value -> value.toUpperCase();
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>and corresponding test</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@SpringBootTest(classes = SampleApplication.class)
|
||
@Import({TestChannelBinderConfiguration.class})
|
||
class BootTestStreamApplicationTests {
|
||
|
||
@Autowired
|
||
private InputDestination input;
|
||
|
||
@Autowired
|
||
private OutputDestination output;
|
||
|
||
@Test
|
||
void contextLoads() {
|
||
input.send(new GenericMessage<byte[]>("hello".getBytes()));
|
||
assertThat(output.receive().getPayload()).isEqualTo("HELLO".getBytes());
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sect1">
|
||
<h2 id="_main_concepts"><a class="link" href="#_main_concepts">Main Concepts</a></h2>
|
||
<div class="sectionbody">
|
||
<div class="paragraph">
|
||
<p>Spring Cloud Stream provides a number of abstractions and primitives that simplify the writing of message-driven microservice applications.
|
||
This section gives an overview of the following:</p>
|
||
</div>
|
||
<div class="ulist">
|
||
<ul>
|
||
<li>
|
||
<p><a href="#spring-cloud-stream-overview-application-model">Spring Cloud Stream’s application model</a></p>
|
||
</li>
|
||
<li>
|
||
<p><a href="#spring-cloud-stream-overview-binder-abstraction">The Binder Abstraction</a></p>
|
||
</li>
|
||
<li>
|
||
<p><a href="#spring-cloud-stream-overview-persistent-publish-subscribe-support">Persistent publish-subscribe support</a></p>
|
||
</li>
|
||
<li>
|
||
<p><a href="#consumer-groups">Consumer group support</a></p>
|
||
</li>
|
||
<li>
|
||
<p><a href="#partitioning">Partitioning support</a></p>
|
||
</li>
|
||
<li>
|
||
<p><a href="#spring-cloud-stream-overview-binder-api">A pluggable Binder SPI</a></p>
|
||
</li>
|
||
</ul>
|
||
</div>
|
||
<div class="sect2">
|
||
<h3 id="spring-cloud-stream-overview-application-model"><a class="link" href="#spring-cloud-stream-overview-application-model">Application Model</a></h3>
|
||
<div class="paragraph">
|
||
<p>A Spring Cloud Stream application consists of a middleware-neutral core.
|
||
The application communicates with the outside world by establishing <em>bindings</em> between destinations
|
||
exposed by the external brokers and input/output arguments in your code. Broker specific details
|
||
necessary to establish bindings are handled by middleware-specific <em>Binder</em> implementations.</p>
|
||
</div>
|
||
<div class="imageblock text-center">
|
||
<div class="content">
|
||
<img src="https://raw.githubusercontent.com/spring-cloud/spring-cloud-stream/master/docs/src/main/asciidoc/images/SCSt-with-binder.png" alt="SCSt with binder" width="800">
|
||
</div>
|
||
<div class="title">Figure 1. Spring Cloud Stream Application</div>
|
||
</div>
|
||
<div class="sect3">
|
||
<h4 id="_fat_jar"><a class="link" href="#_fat_jar">Fat JAR</a></h4>
|
||
<div class="paragraph">
|
||
<p>Spring Cloud Stream applications can be run in stand-alone mode from your IDE for testing.
|
||
To run a Spring Cloud Stream application in production, you can create an executable (or “fat”) JAR by using the standard Spring Boot tooling provided for Maven or Gradle. See the <a href="https://docs.spring.io/spring-boot/docs/current/reference/html/howto-build.html#howto-create-an-executable-jar-with-maven">Spring Boot Reference Guide</a> for more details.</p>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sect2">
|
||
<h3 id="spring-cloud-stream-overview-binder-abstraction"><a class="link" href="#spring-cloud-stream-overview-binder-abstraction">The Binder Abstraction</a></h3>
|
||
<div class="paragraph">
|
||
<p>Spring Cloud Stream provides Binder implementations for <a href="https://github.com/spring-cloud/spring-cloud-stream-binder-kafka">Kafka</a> and <a href="https://github.com/spring-cloud/spring-cloud-stream-binder-rabbit">Rabbit MQ</a>.
|
||
The framework also includes a test binder for integration testing of your applications as spring-cloud-stream application. See <a href="#_testing">Testing</a> section for more details.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Binder abstraction is also one of the extension points of the framework, which means you can implement your own binder on top of Spring Cloud Stream.
|
||
In the <a href="https://medium.com/@domenicosibilio/how-to-create-a-spring-cloud-stream-binder-from-scratch-ab8b29ee931b">How to create a Spring Cloud Stream Binder from scratch</a> post a community member documents
|
||
in details, with an example, a set of steps necessary to implement a custom binder.
|
||
The steps are also highlighted in the <code><a href="#spring-cloud-stream-overview-custom-binder-impl">Implementing Custom Binders</a></code> section.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Spring Cloud Stream uses Spring Boot for configuration, and the Binder abstraction makes it possible for a Spring Cloud Stream application to be flexible in how it connects to middleware.
|
||
For example, deployers can dynamically choose, at runtime, the mapping between the external destinations (such as the Kafka topics or RabbitMQ exchanges) and inputs
|
||
and outputs of the message handler (such as input parameter of the function and its return argument).
|
||
Such configuration can be provided through external configuration properties and in any form supported by Spring Boot (including application arguments, environment variables, and <code>application.yml</code> or <code>application.properties</code> files).
|
||
In the sink example from the <a href="#spring-cloud-stream-overview-introducing">Introducing Spring Cloud Stream</a> section, setting the <code>spring.cloud.stream.bindings.input.destination</code> application property to <code>raw-sensor-data</code> causes it to read from the <code>raw-sensor-data</code> Kafka topic or from a queue bound to the <code>raw-sensor-data</code> RabbitMQ exchange.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Spring Cloud Stream automatically detects and uses a binder found on the classpath.
|
||
You can use different types of middleware with the same code.
|
||
To do so, include a different binder at build time.
|
||
For more complex use cases, you can also package multiple binders with your application and have it choose the binder( and even whether to use different binders for different bindings) at runtime.</p>
|
||
</div>
|
||
</div>
|
||
<div class="sect2">
|
||
<h3 id="spring-cloud-stream-overview-persistent-publish-subscribe-support"><a class="link" href="#spring-cloud-stream-overview-persistent-publish-subscribe-support">Persistent Publish-Subscribe Support</a></h3>
|
||
<div class="paragraph">
|
||
<p>Communication between applications follows a publish-subscribe model, where data is broadcast through shared topics.
|
||
This can be seen in the following figure, which shows a typical deployment for a set of interacting Spring Cloud Stream applications.</p>
|
||
</div>
|
||
<div class="imageblock text-center">
|
||
<div class="content">
|
||
<img src="https://raw.githubusercontent.com/spring-cloud/spring-cloud-stream/master/docs/src/main/asciidoc/images/SCSt-sensors.png" alt="SCSt sensors" width="800">
|
||
</div>
|
||
<div class="title">Figure 2. Spring Cloud Stream Publish-Subscribe</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Data reported by sensors to an HTTP endpoint is sent to a common destination named <code>raw-sensor-data</code>.
|
||
From the destination, it is independently processed by a microservice application that computes time-windowed averages and by another microservice application that ingests the raw data into HDFS (Hadoop Distributed File System).
|
||
In order to process the data, both applications declare the topic as their input at runtime.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The publish-subscribe communication model reduces the complexity of both the producer and the consumer and lets new applications be added to the topology without disruption of the existing flow.
|
||
For example, downstream from the average-calculating application, you can add an application that calculates the highest temperature values for display and monitoring.
|
||
You can then add another application that interprets the same flow of averages for fault detection.
|
||
Doing all communication through shared topics rather than point-to-point queues reduces coupling between microservices.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>While the concept of publish-subscribe messaging is not new, Spring Cloud Stream takes the extra step of making it an opinionated choice for its application model.
|
||
By using native middleware support, Spring Cloud Stream also simplifies use of the publish-subscribe model across different platforms.</p>
|
||
</div>
|
||
</div>
|
||
<div class="sect2">
|
||
<h3 id="consumer-groups"><a class="link" href="#consumer-groups">Consumer Groups</a></h3>
|
||
<div class="paragraph">
|
||
<p>While the publish-subscribe model makes it easy to connect applications through shared topics, the ability to scale up by creating multiple instances of a given application is equally important.
|
||
When doing so, different instances of an application are placed in a competing consumer relationship, where only one of the instances is expected to handle a given message.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Spring Cloud Stream models this behavior through the concept of a consumer group.
|
||
(Spring Cloud Stream consumer groups are similar to and inspired by Kafka consumer groups.)
|
||
Each consumer binding can use the <code>spring.cloud.stream.bindings.<bindingName>.group</code> property to specify a group name.
|
||
For the consumers shown in the following figure, this property would be set as <code>spring.cloud.stream.bindings.<bindingName>.group=hdfsWrite</code> or <code>spring.cloud.stream.bindings.<bindingName>.group=average</code>.</p>
|
||
</div>
|
||
<div class="imageblock text-center">
|
||
<div class="content">
|
||
<img src="https://raw.githubusercontent.com/spring-cloud/spring-cloud-stream/master/docs/src/main/asciidoc/images/SCSt-groups.png" alt="SCSt groups" width="800">
|
||
</div>
|
||
<div class="title">Figure 3. Spring Cloud Stream Consumer Groups</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>All groups that subscribe to a given destination receive a copy of published data, but only one member of each group receives a given message from that destination.
|
||
By default, when a group is not specified, Spring Cloud Stream assigns the application to an anonymous and independent single-member consumer group that is in a publish-subscribe relationship with all other consumer groups.</p>
|
||
</div>
|
||
</div>
|
||
<div class="sect2">
|
||
<h3 id="consumer-types"><a class="link" href="#consumer-types">Consumer Types</a></h3>
|
||
<div class="paragraph">
|
||
<p>Two types of consumer are supported:</p>
|
||
</div>
|
||
<div class="ulist">
|
||
<ul>
|
||
<li>
|
||
<p>Message-driven (sometimes referred to as Asynchronous)</p>
|
||
</li>
|
||
<li>
|
||
<p>Polled (sometimes referred to as Synchronous)</p>
|
||
</li>
|
||
</ul>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Prior to version 2.0, only asynchronous consumers were supported. A message is delivered as soon as it is available and a thread is available to process it.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>When you wish to control the rate at which messages are processed, you might want to use a synchronous consumer.</p>
|
||
</div>
|
||
<div class="sect3">
|
||
<h4 id="durability"><a class="link" href="#durability">Durability</a></h4>
|
||
<div class="paragraph">
|
||
<p>Consistent with the opinionated application model of Spring Cloud Stream, consumer group subscriptions are durable.
|
||
That is, a binder implementation ensures that group subscriptions are persistent and that, once at least one subscription for a group has been created, the group receives messages, even if they are sent while all applications in the group are stopped.</p>
|
||
</div>
|
||
<div class="admonitionblock note">
|
||
<table>
|
||
<tr>
|
||
<td class="icon">
|
||
<i class="fa icon-note" title="Note"></i>
|
||
</td>
|
||
<td class="content">
|
||
<div class="paragraph">
|
||
<p>Anonymous subscriptions are non-durable by nature.
|
||
For some binder implementations (such as RabbitMQ), it is possible to have non-durable group subscriptions.</p>
|
||
</div>
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>In general, it is preferable to always specify a consumer group when binding an application to a given destination.
|
||
When scaling up a Spring Cloud Stream application, you must specify a consumer group for each of its input bindings.
|
||
Doing so prevents the application’s instances from receiving duplicate messages (unless that behavior is desired, which is unusual).</p>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sect2">
|
||
<h3 id="partitioning"><a class="link" href="#partitioning">Partitioning Support</a></h3>
|
||
<div class="paragraph">
|
||
<p>Spring Cloud Stream provides support for partitioning data between multiple instances of a given application.
|
||
In a partitioned scenario, the physical communication medium (such as the broker topic) is viewed as being structured into multiple partitions.
|
||
One or more producer application instances send data to multiple consumer application instances and ensure that data identified by common characteristics are processed by the same consumer instance.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Spring Cloud Stream provides a common abstraction for implementing partitioned processing use cases in a uniform fashion.
|
||
Partitioning can thus be used whether the broker itself is naturally partitioned (for example, Kafka) or not (for example, RabbitMQ).</p>
|
||
</div>
|
||
<div class="imageblock text-center">
|
||
<div class="content">
|
||
<img src="https://raw.githubusercontent.com/spring-cloud/spring-cloud-stream/master/docs/src/main/asciidoc/images/SCSt-partitioning.png" alt="SCSt partitioning" width="800">
|
||
</div>
|
||
<div class="title">Figure 4. Spring Cloud Stream Partitioning</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Partitioning is a critical concept in stateful processing, where it is critical (for either performance or consistency reasons) to ensure that all related data is processed together.
|
||
For example, in the time-windowed average calculation example, it is important that all measurements from any given sensor are processed by the same application instance.</p>
|
||
</div>
|
||
<div class="admonitionblock note">
|
||
<table>
|
||
<tr>
|
||
<td class="icon">
|
||
<i class="fa icon-note" title="Note"></i>
|
||
</td>
|
||
<td class="content">
|
||
To set up a partitioned processing scenario, you must configure both the data-producing and the data-consuming ends.
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sect1">
|
||
<h2 id="_programming_model"><a class="link" href="#_programming_model">Programming Model</a></h2>
|
||
<div class="sectionbody">
|
||
<div class="paragraph">
|
||
<p>To understand the programming model, you should be familiar with the following core concepts:</p>
|
||
</div>
|
||
<div class="ulist">
|
||
<ul>
|
||
<li>
|
||
<p><strong>Destination Binders:</strong> Components responsible to provide integration with the external messaging systems.</p>
|
||
</li>
|
||
<li>
|
||
<p><strong>Bindings:</strong> Bridge between the external messaging systems and application provided <em>Producers</em> and <em>Consumers</em> of messages (created by the Destination Binders).</p>
|
||
</li>
|
||
<li>
|
||
<p><strong>Message:</strong> The canonical data structure used by producers and consumers to communicate with Destination Binders (and thus other applications via external messaging systems).</p>
|
||
</li>
|
||
</ul>
|
||
</div>
|
||
<div class="imageblock text-center">
|
||
<div class="content">
|
||
<img src="https://raw.githubusercontent.com/spring-cloud/spring-cloud-stream/master/docs/src/main/asciidoc/images/SCSt-overview.png" alt="SCSt overview" width="800">
|
||
</div>
|
||
</div>
|
||
<div class="sect2">
|
||
<h3 id="_destination_binders"><a class="link" href="#_destination_binders">Destination Binders</a></h3>
|
||
<div class="paragraph">
|
||
<p>Destination Binders are extension components of Spring Cloud Stream responsible for providing the necessary configuration and implementation to facilitate
|
||
integration with external messaging systems.
|
||
This integration is responsible for connectivity, delegation, and routing of messages to and from producers and consumers, data type conversion,
|
||
invocation of the user code, and more.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Binders handle a lot of the boiler plate responsibilities that would otherwise fall on your shoulders. However, to accomplish that, the binder still needs
|
||
some help in the form of minimalistic yet required set of instructions from the user, which typically come in the form of some type of <em>binding</em> configuration.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>While it is out of scope of this section to discuss all of the available binder and binding configuration options (the rest of the manual covers them extensively),
|
||
<em>Binding</em> as a concept, does require special attention. The next section discusses it in detail.</p>
|
||
</div>
|
||
</div>
|
||
<div class="sect2">
|
||
<h3 id="_bindings"><a class="link" href="#_bindings">Bindings</a></h3>
|
||
<div class="paragraph">
|
||
<p>As stated earlier, <em>Bindings</em> provide a bridge between the external messaging system (e.g., queue, topic etc.) and application-provided <em>Producers</em> and <em>Consumers</em>.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The following example shows a fully configured and functioning Spring Cloud Stream application that receives the payload of the message
|
||
as a <code>String</code> type (see <a href="#content-type-management">Content Type Negotiation</a> section), logs it to the console and sends it down stream after converting it to upper case.</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@SpringBootApplication
|
||
public class SampleApplication {
|
||
|
||
public static void main(String[] args) {
|
||
SpringApplication.run(SampleApplication.class, args);
|
||
}
|
||
|
||
@Bean
|
||
public Function<String, String> uppercase() {
|
||
return value -> {
|
||
System.out.println("Received: " + value);
|
||
return value.toUpperCase()
|
||
};
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Unlike previous versions of spring-cloud-stream which relied on <code>@EnableBinding</code> and <code>@StreamListener</code> annotations,
|
||
the above example looks no different then any vanilla spring-boot application. It defines a single bean of type <code>Function</code>
|
||
and that it is. So, how does it became spring-cloud-stream application?
|
||
It becomes spring-cloud-stream application simply based on the presence of spring-cloud-stream and binder dependencies
|
||
and auto-configuration classes on the classpath effectively setting the context for your boot application as spring-cloud-stream application.
|
||
And in this context beans of type <code>Supplier</code>, <code>Function</code> or <code>Consumer</code> are treated as defacto message handlers triggering
|
||
binding of to destinations exposed by the provided binder following certain naming conventions and
|
||
rules to avoid extra configuration.</p>
|
||
</div>
|
||
<div class="sect3">
|
||
<h4 id="_binding_and_binding_names"><a class="link" href="#_binding_and_binding_names">Binding and Binding names</a></h4>
|
||
<div class="paragraph">
|
||
<p>Binding is an abstraction that represents a bridge between sources and targets exposed by the binder and user code,
|
||
This abstraction has a name and while we try to do our best to limit configuration required to run spring-cloud-stream applications,
|
||
being aware of such name(s) is necessary for cases where additional per-binding configuration is required.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Throughout this manual you will see examples of configuration properties such as <code>spring.cloud.stream.bindings.input.destination=myQueue</code>.
|
||
The <code>input</code> segment in this property name is what we refer to as <em>binding name</em> and it could derive via several mechanisms.
|
||
The following sub-sections will describe the naming conventions and configuration elements used by spring-cloud-stream to control binding names.</p>
|
||
</div>
|
||
<div class="sect4">
|
||
<h5 id="_functional_binding_names"><a class="link" href="#_functional_binding_names">Functional binding names</a></h5>
|
||
<div class="paragraph">
|
||
<p>Unlike the explicit naming required by annotation-based support (legacy) used in the previous versions of spring-cloud-stream, the functional
|
||
programming model defaults to a simple convention when it comes to binding names, thus greatly simplifying application configuration.
|
||
Let’s look at the first example:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@SpringBootApplication
|
||
public class SampleApplication {
|
||
|
||
@Bean
|
||
public Function<String, String> uppercase() {
|
||
return value -> value.toUpperCase();
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>In the preceding example we have an application with a single function which acts as message handler. As a <code>Function</code> it has an
|
||
input and output.
|
||
The naming convention used to name input and output bindings is as follows:</p>
|
||
</div>
|
||
<div class="ulist">
|
||
<ul>
|
||
<li>
|
||
<p>input - <code><functionName> + -in- + <index></code></p>
|
||
</li>
|
||
<li>
|
||
<p>output - <code><functionName> + -out- + <index></code></p>
|
||
</li>
|
||
</ul>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The <code>in</code> and <code>out</code> corresponds to the type of binding (such as <em>input</em> or <em>output</em>).
|
||
The <code>index</code> is the index of the input or output binding. It is always 0 for typical single input/output function,
|
||
so it’s only relevant for <a href="#_functions_with_multiple_input_and_output_arguments">Functions with multiple input and output arguments</a>.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>So if for example you would want to map the input of this function to a remote
|
||
destination (e.g., topic, queue etc) called "my-topic" you would do so with the following property:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre>--spring.cloud.stream.bindings.uppercase-in-0.destination=my-topic</pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Note how <code>uppercase-in-0</code> is used as a segment in property name. The same goes for <code>uppercase-out-0</code>.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p><strong><strong>Descriptive Binding Names</strong></strong></p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Some times to improve readability you may want to give your binding a more descriptive name (such as 'account', 'orders` etc).
|
||
Another way of looking at it is you can map an <em>implicit binding name</em> to an <em>explicit binding name</em>. And you can do it with
|
||
<code>spring.cloud.stream.function.bindings.<binding-name></code> property.
|
||
This property also provides a migration path for existing applications that rely on custom interface-based
|
||
bindings that require explicit names.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>For example,</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre>--spring.cloud.stream.function.bindings.uppercase-in-0=input`</pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>In the preceding example you mapped and effectively renamed <code>uppercase-in-0</code> binding name to <code>input</code>. Now all configuration
|
||
properties can refer to <code>input</code> binding name instead (e.g., <code>--spring.cloud.stream.bindings.input.destination=my-topic</code>).</p>
|
||
</div>
|
||
<div class="admonitionblock note">
|
||
<table>
|
||
<tr>
|
||
<td class="icon">
|
||
<i class="fa icon-note" title="Note"></i>
|
||
</td>
|
||
<td class="content">
|
||
While descriptive binding names may enhance the readability aspect of the configuration, they also create
|
||
another level of misdirection by mapping an implicit binding name to an explicit binding name. And since all subsequent
|
||
configuration properties will use the explicit binding name you must always refer to this 'bindings' property to
|
||
correlate which function it actually corresponds to. We believe that for most cases (with the exception of <a href="#_functional_composition">Functional Composition</a>)
|
||
it may be an overkill, so, it is our recommendation to avoid using it all together, especially
|
||
since not using it provides a clear path between binder destination and binding name, such as <code>spring.cloud.stream.bindings.uppercase-in-0.destination=sample-topic</code>,
|
||
where you are clearly correlating the input of <code>uppercase</code> function to <code>sample-topic</code> destination.
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>For more on properties and other configuration options please see <a href="#_configuration_options">Configuration Options</a> section.</p>
|
||
</div>
|
||
</div>
|
||
<div class="sect4">
|
||
<h5 id="_annotation_based_binding_names_legacy"><a class="link" href="#_annotation_based_binding_names_legacy">Annotation-based binding names (legacy)</a></h5>
|
||
<div class="paragraph">
|
||
<p>In previous versions of spring-cloud-stream <em>binding</em> names and in fact implementations, derived from the <code>@EnableBinding</code>
|
||
annotation which typically would take one or more interface classes as parameters. The parameters are referred to
|
||
as <em>bindings</em>, and they contain methods representing <em>bindable components</em>.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>For compliance with legacy style applications we still support this annotation-based programming model and you can get more information about it in
|
||
<a href="#_annotation_based_support_legacy">Annotation-based support (legacy)</a> section (sub-section of the <a href="#_programming_model">Programming Model</a> section).</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Spring Cloud Stream already provides <em>binding</em> interfaces for typical message exchange contracts, which include:</p>
|
||
</div>
|
||
<div class="ulist">
|
||
<ul>
|
||
<li>
|
||
<p><strong>Sink:</strong> Identifies the contract for the message consumer by providing the destination from which the message is consumed.</p>
|
||
</li>
|
||
<li>
|
||
<p><strong>Source:</strong> Identifies the contract for the message producer by providing the destination to which the produced message is sent.</p>
|
||
</li>
|
||
<li>
|
||
<p><strong>Processor:</strong> Encapsulates both the sink and the source contracts by exposing two destinations that allow consumption and production of messages.</p>
|
||
</li>
|
||
</ul>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">public interface Sink {
|
||
|
||
String INPUT = "input";
|
||
|
||
@Input(Sink.INPUT)
|
||
SubscribableChannel input();
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">public interface Source {
|
||
|
||
String OUTPUT = "output";
|
||
|
||
@Output(Source.OUTPUT)
|
||
MessageChannel output();
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">public interface Processor extends Source, Sink {}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>And you can define your own interfaces as well</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">public interface MyBinding {
|
||
|
||
String FOO = "foo";
|
||
|
||
@Output(MyBinding.FOO)
|
||
MessageChannel foo();
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="admonitionblock note">
|
||
<table>
|
||
<tr>
|
||
<td class="icon">
|
||
<i class="fa icon-note" title="Note"></i>
|
||
</td>
|
||
<td class="content">
|
||
The reason why <code>@EnableBinding</code> and binding interfaces are not required with functional programming model is because
|
||
they could be derived from the type of functional interface itself. For example, <em>Processor = Function</em>, <em>Source = Supplier</em>
|
||
and so on.
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p><strong>Pollable Destination Binding</strong></p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>While the previously described bindings support event-based message consumption, sometimes you need more control, such as rate of consumption.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Starting with version 2.0, you can now bind a pollable consumer:</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The following example shows how to bind a pollable consumer:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">public interface PolledBarista {
|
||
|
||
@Input
|
||
PollableMessageSource orders();
|
||
. . .
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>In this case, an implementation of <code>PollableMessageSource</code> is bound to the <code>orders</code> “channel”. See <a href="#spring-cloud-streams-overview-using-polled-consumers">Using Polled Consumers</a> for more details.</p>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sect2">
|
||
<h3 id="spring-cloud-stream-overview-producing-consuming-messages"><a class="link" href="#spring-cloud-stream-overview-producing-consuming-messages">Producing and Consuming Messages</a></h3>
|
||
<div class="paragraph">
|
||
<p>You can write a Spring Cloud Stream application by simply writing functions and exposing them as `@Bean`s.
|
||
You can also use Spring Integration annotations based configuration or
|
||
Spring Cloud Stream annotation based configuration, although starting with spring-cloud-stream 3.x
|
||
we recommend using functional implementations.</p>
|
||
</div>
|
||
<div class="sect3">
|
||
<h4 id="spring_cloud_function"><a class="link" href="#spring_cloud_function">Spring Cloud Function support</a></h4>
|
||
<div class="sect4">
|
||
<h5 id="_overview"><a class="link" href="#_overview">Overview</a></h5>
|
||
<div class="paragraph">
|
||
<p>Since Spring Cloud Stream v2.1, another alternative for defining <em>stream handlers</em> and <em>sources</em> is to use build-in
|
||
support for <a href="https://cloud.spring.io/spring-cloud-function/">Spring Cloud Function</a> where they can be expressed as beans of
|
||
type <code>java.util.function.[Supplier/Function/Consumer]</code>.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>To specify which functional bean to bind to the external destination(s) exposed by the bindings,
|
||
you must provide <code>spring.cloud.function.definition</code> property.</p>
|
||
</div>
|
||
<div class="admonitionblock note">
|
||
<table>
|
||
<tr>
|
||
<td class="icon">
|
||
<i class="fa icon-note" title="Note"></i>
|
||
</td>
|
||
<td class="content">
|
||
In the event you only have single bean of type <code>java.util.function.[Supplier/Function/Consumer]</code>, you can
|
||
skip the <code>spring.cloud.function.definition</code> property, since such functional bean will be auto-discovered. However,
|
||
it is considered best practice to use such property to avoid any confusion.
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Here is the example of the application exposing message handler as <code>java.util.function.Function</code> effectively supporting
|
||
<em>pass-thru</em> semantics by acting as consumer and producer of data.</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@SpringBootApplication
|
||
public class MyFunctionBootApp {
|
||
|
||
public static void main(String[] args) {
|
||
SpringApplication.run(MyFunctionBootApp.class);
|
||
}
|
||
|
||
@Bean
|
||
public Function<String, String> toUpperCase() {
|
||
return s -> s.toUpperCase();
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>In the preceding example, we define a bean of type <code>java.util.function.Function</code> called <em>toUpperCase</em> to be acting as message handler
|
||
whose 'input' and 'output' must be bound to the external destinations exposed by the provided destination binder.
|
||
By default the 'input' and 'output' binding names will be <code>toUpperCase-in-0</code> and <code>toUpperCase-out-0</code>.
|
||
Please see <a href="#_functional_binding_names">Functional binding names</a> section for details on naming convention used to establish binding names.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Below are the examples of simple functional applications to support other semantics:</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Here is the example of a <em>source</em> semantics exposed as <code>java.util.function.Supplier</code></p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@SpringBootApplication
|
||
public static class SourceFromSupplier {
|
||
|
||
@Bean
|
||
public Supplier<Date> date() {
|
||
return () -> new Date(12345L);
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Here is the example of a <em>sink semantics</em> exposed as <code>java.util.function.Consumer</code></p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@SpringBootApplication
|
||
public static class SinkFromConsumer {
|
||
|
||
@Bean
|
||
public Consumer<String> sink() {
|
||
return System.out::println;
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sect4">
|
||
<h5 id="_suppliers_sources"><a class="link" href="#_suppliers_sources">Suppliers (Sources)</a></h5>
|
||
<div class="paragraph">
|
||
<p><code>Function</code> and <code>Consumer</code> are pretty straightforward when it comes to how their invocation is triggered. They are triggered based
|
||
on data (events) sent to the destination they are bound to. In other words, they are classic event-driven components.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>However, <code>Supplier</code> is in its own category when it comes to triggering. Since it is, by definition, the source (the origin) of the data, it does not
|
||
subscribe to any in-bound destination and, therefore, has to be triggered by some other mechanism(s).
|
||
There is also a question of <code>Supplier</code> implementation, which could be <em>imperative</em> or <em>reactive</em> and which directly relates to the triggering of such suppliers.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Consider the following sample:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@SpringBootApplication
|
||
public static class SupplierConfiguration {
|
||
|
||
@Bean
|
||
public Supplier<String> stringSupplier() {
|
||
return () -> "Hello from Supplier";
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The preceding <code>Supplier</code> bean produces a string whenever its <code>get()</code> method is invoked. However, who invokes this method and how often?
|
||
The framework provides a default polling mechanism (answering the question of "Who?") that will trigger the invocation of the supplier and by default it will do so
|
||
every second (answering the question of "How often?").
|
||
In other words, the above configuration produces a single message every second and each message is sent to an <code>output</code> destination that is exposed by the binder.
|
||
To learn how to customize the polling mechanism, see <a href="#_polling_configuration_properties">Polling Configuration Properties</a> section.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Consider a different sample:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@SpringBootApplication
|
||
public static class SupplierConfiguration {
|
||
|
||
@Bean
|
||
public Supplier<Flux<String>> stringSupplier() {
|
||
return () -> Flux.fromStream(Stream.generate(new Supplier<String>() {
|
||
@Override
|
||
public String get() {
|
||
try {
|
||
Thread.sleep(1000);
|
||
return "Hello from Supplier";
|
||
} catch (Exception e) {
|
||
// ignore
|
||
}
|
||
}
|
||
})).subscribeOn(Schedulers.elastic()).share();
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The preceding <code>Supplier</code> bean adopts the reactive programming style. Typically, and unlike the imperative supplier,
|
||
it should be triggered only once, given that the invocation of its <code>get()</code> method produces (supplies) the continuous stream of messages and not an
|
||
individual message.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The framework recognizes the difference in the programming style and guarantees that such a supplier is triggered only once.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>However, imagine the use case where you want to poll some data source and return a finite stream of data representing the result set.
|
||
The reactive programming style is a perfect mechanism for such a Supplier. However, given the finite nature of the produced stream,
|
||
such Supplier still needs to be invoked periodically.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Consider the following sample, which emulates such use case by producing a finite stream of data:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@SpringBootApplication
|
||
public static class SupplierConfiguration {
|
||
|
||
@PollableBean
|
||
public Supplier<Flux<String>> stringSupplier() {
|
||
return () -> Flux.just("hello", "bye");
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The bean itself is annotated with <code>PollableBean</code> annotation (sub-set of <code>@Bean</code>), thus signaling to the framework that although the implementation
|
||
of such a supplier is reactive, it still needs to be polled.</p>
|
||
</div>
|
||
<div class="admonitionblock note">
|
||
<table>
|
||
<tr>
|
||
<td class="icon">
|
||
<i class="fa icon-note" title="Note"></i>
|
||
</td>
|
||
<td class="content">
|
||
There is a <code>splittable</code> attribute defined in <code>PollableBean</code> which signals to the post processors of this annotation
|
||
that the result produced by the annotated component has to be split and is set to <code>true</code> by default. It means that
|
||
the framework will split the returning sending out each item as an individual message. If this is not
|
||
he desired behavior you can set it to <code>false</code> at which point such supplier will simply return
|
||
the produced Flux without splitting it.
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
</div>
|
||
<div class="sect4">
|
||
<h5 id="_consumer_reactive"><a class="link" href="#_consumer_reactive">Consumer (Reactive)</a></h5>
|
||
<div class="paragraph">
|
||
<p>Reactive <code>Consumer</code> is a little bit special because it has a void return type, leaving framework with no reference to subscribe to.
|
||
Most likely you will not need to write <code>Consumer<Flux<?>></code>, and instead write it as a <code>Function<Flux<?>, Mono<Void>></code> invoking <code>then</code>
|
||
operator as the last operator on your stream.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>For example:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">public Function<Flux<?>, Mono<Void>>`consumer() {
|
||
return flux -> flux.map(..).filter(..).then();
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>But if you do need to write an explicit <code>Consumer<Flux<?>></code>, remember to subscribe to the incoming Flux.</p>
|
||
</div>
|
||
<div class="sect5">
|
||
<h6 id="_polling_configuration_properties"><a class="link" href="#_polling_configuration_properties">Polling Configuration Properties</a></h6>
|
||
<div class="paragraph">
|
||
<p>The following properties are exposed by <code>org.springframework.cloud.stream.config.DefaultPollerProperties</code> and are prefixed with
|
||
<code>spring.cloud.stream.poller</code>:</p>
|
||
</div>
|
||
<div class="dlist">
|
||
<dl>
|
||
<dt class="hdlist1">fixedDelay</dt>
|
||
<dd>
|
||
<p>Fixed delay for default poller in milliseconds.</p>
|
||
<div class="paragraph">
|
||
<p>Default: 1000L.</p>
|
||
</div>
|
||
</dd>
|
||
<dt class="hdlist1">maxMessagesPerPoll</dt>
|
||
<dd>
|
||
<p>Maximum messages for each polling event of the default poller.</p>
|
||
<div class="paragraph">
|
||
<p>Default: 1L.</p>
|
||
</div>
|
||
</dd>
|
||
</dl>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>For example <code>--spring.cloud.stream.poller.fixed-delay=2000</code> sets the poller interval to poll every two seconds.</p>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sect4">
|
||
<h5 id="_sending_arbitrary_data_to_an_output_e_g_foreign_event_driven_sources"><a class="link" href="#_sending_arbitrary_data_to_an_output_e_g_foreign_event_driven_sources">Sending arbitrary data to an output (e.g. Foreign event-driven sources)</a></h5>
|
||
<div class="paragraph">
|
||
<p>There are cases where the actual source of data may be coming from the external (foreign) system that is not a binder. For example, the
|
||
source of the data may be a classic REST endpoint. How do we bridge such source with the functional mechanism used by spring-cloud-stream?</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Spring Cloud Stream provides two mechanisms, so let’s look at them in more details</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Here, for both samples we’ll use a standard MVC endpoint method called <code>delegateToSupplier</code> bound to the root web context,
|
||
delegating incoming requests to stream via two different mechanisms -
|
||
imperative (via StreamBridge) and reactive (via EmitterProcessor).</p>
|
||
</div>
|
||
<div class="sect5">
|
||
<h6 id="_using_streambridge"><a class="link" href="#_using_streambridge">Using StreamBridge</a></h6>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@SpringBootApplication
|
||
@Controller
|
||
public class WebSourceApplication {
|
||
|
||
public static void main(String[] args) {
|
||
SpringApplication.run(WebSourceApplication.class, "--spring.cloud.stream.source=toStream");
|
||
}
|
||
|
||
@Autowired
|
||
private StreamBridge streamBridge;
|
||
|
||
@RequestMapping
|
||
@ResponseStatus(HttpStatus.ACCEPTED)
|
||
public void delegateToSupplier(@RequestBody String body) {
|
||
System.out.println("Sending " + body);
|
||
streamBridge.send("toStream-out-0", body);
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Here we autowire a <code>StreamBridge</code> bean which allows us to send data to an output binding effectively
|
||
bridging non-stream application with spring-cloud-stream. Note that preceding example does not have any
|
||
source functions defined (e.g., Supplier bean) leaving the framework with no trigger to create source bindings, which would be typical for cases where
|
||
configuration contains function beans.
|
||
So to trigger the creation of source binding we use <code>spring.cloud.stream.source</code> property where you can declare the name of your sources.
|
||
The provided name will be used as a trigger to create a source binding.
|
||
So in the preceding example the name of the output binding will be <code>toStream-out-0</code> which is consistent with the binding naming
|
||
convention used by functions (see <a href="#_binding_and_binding_names">Binding and Binding names</a>). You can use <code>;</code> to signify multiple sources
|
||
(e.g., <code>--spring.cloud.stream.source=foo;bar</code>)</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Also, note that <code>streamBridge.send(..)</code> method takes an <code>Object</code> for data. This means you can send POJO or <code>Message</code> to it and it
|
||
will go through the same routine when sending output as if it was from any Function or Supplier providing the same level
|
||
of consistency as with functions. This means the output type conversion, partitioning etc are honored as if it was from the output produced by functions.</p>
|
||
</div>
|
||
</div>
|
||
<div class="sect5">
|
||
<h6 id="_streambridge_and_dynamic_destinations"><a class="link" href="#_streambridge_and_dynamic_destinations">StreamBridge and Dynamic Destinations</a></h6>
|
||
<div class="paragraph">
|
||
<p><code>StreamBridge</code> can also be used for cases when output destination(s) are not known ahead of time similar to the use cases
|
||
described in <a href="#_routing_from_consumer">Routing FROM Consumer</a> section.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Let’s look at the example</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@SpringBootApplication
|
||
@Controller
|
||
public class WebSourceApplication {
|
||
|
||
public static void main(String[] args) {
|
||
SpringApplication.run(WebSourceApplication.class, args);
|
||
}
|
||
|
||
@Autowired
|
||
private StreamBridge streamBridge;
|
||
|
||
@RequestMapping
|
||
@ResponseStatus(HttpStatus.ACCEPTED)
|
||
public void delegateToSupplier(@RequestBody String body) {
|
||
System.out.println("Sending " + body);
|
||
streamBridge.send("myDestiniation", body);
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>As you can see the preceding example is very similar to the previous one with the exception of explicit binding instruction provided via
|
||
<code>spring.cloud.stream.source</code> property (which is not provided).
|
||
Here we’re sending data to <code>myDestiniation</code> name which does not exist as a binding. Therefore such name will be treated as dynamic destination
|
||
as described in <a href="#_routing_from_consumer">Routing FROM Consumer</a> section.</p>
|
||
</div>
|
||
</div>
|
||
<div class="sect5">
|
||
<h6 id="_using_reactor_api"><a class="link" href="#_using_reactor_api">Using reactor API</a></h6>
|
||
<div class="paragraph">
|
||
<p>Another approach that can be used to send arbitrary data to the output is using Reactor API.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>All we need to do is declare a <code>Supplier<Flux<whatever>></code> which returns <a href="https://projectreactor.io/docs/core/release/api/reactor/core/publisher/EmitterProcessor.html">EmitterProcessor</a>
|
||
from the reactor API (see <a href="#_reactive_functions_support">Reactive Functions support</a> for more details) to effectively provide a
|
||
bridge between the actual event source (<em>foreign source</em>) and spring-cloud-stream.
|
||
Now, all you need to do now is feed the <code>EmitterProcessor</code> with data via <code>EmitterProcessor#onNext(data)</code> operation.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>For example,</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">public class SampleApplication {
|
||
|
||
public static void main(String[] args) {
|
||
SpringApplication.run(SampleApplication.class);
|
||
}
|
||
|
||
EmitterProcessor<String> processor = EmitterProcessor.create();
|
||
|
||
@Bean
|
||
public ApplicationRunner runner() {
|
||
Message<String> msg1 = MessageBuilder.withPayload("foo")
|
||
.setHeader("*.events", "test1.events.billing")
|
||
.build();
|
||
Message<String> msg2 = MessageBuilder.withPayload("bar")
|
||
.setHeader("*.events", "test2.events.messages")
|
||
.build();
|
||
return args -> {
|
||
this.processor.onNext(msg1);
|
||
this.processor.onNext(msg2);
|
||
};
|
||
}
|
||
|
||
@Bean
|
||
public Supplier<Flux<String>> supplier() {
|
||
return () -> this.processor;
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>In the preceding example, we are using <code>ApplicationRunner</code> as a <em>foreign source</em> to feed the stream.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>A more practical example, where the foreign source is REST endpoint.</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@SpringBootApplication
|
||
@Controller
|
||
public class WebSourceApplication {
|
||
|
||
public static void main(String[] args) {
|
||
SpringApplication.run(WebSourceApplication.class);
|
||
}
|
||
|
||
EmitterProcessor<String> processor = EmitterProcessor.create();
|
||
|
||
@RequestMapping
|
||
@ResponseStatus(HttpStatus.ACCEPTED)
|
||
public void delegateToSupplier(@RequestBody String body) {
|
||
processor.onNext(body);
|
||
}
|
||
|
||
@Bean
|
||
public Supplier<Flux<String>> supplier() {
|
||
return () -> this.processor;
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Same as before we declare a <code>Supplier</code> bean which returns <code>Flux<String></code>. But given that this is a REST endpoint
|
||
we send messages by simply posting to this REST endpoint.</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre>curl -H "Content-Type: text/plain" -X POST -d "hello from the other side" http://localhost:8080/</pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>By showing two example we want to emphasize the approach will work with any type of foreign sources.</p>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sect4">
|
||
<h5 id="_reactive_functions_support"><a class="link" href="#_reactive_functions_support">Reactive Functions support</a></h5>
|
||
<div class="paragraph">
|
||
<p>Since <em>Spring Cloud Function</em> is build on top of <a href="https://projectreactor.io/">Project Reactor</a> there isn’t much you need to do
|
||
to benefit from reactive programming model while implementing <code>Supplier</code>, <code>Function</code> or <code>Consumer</code>.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>For example:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@SpringBootApplication
|
||
public static class SinkFromConsumer {
|
||
|
||
@Bean
|
||
public Function<Flux<String>, Flux<String>> reactiveUpperCase() {
|
||
return flux -> flux.map(val -> val.toUpperCase());
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sect4">
|
||
<h5 id="_functional_composition"><a class="link" href="#_functional_composition">Functional Composition</a></h5>
|
||
<div class="paragraph">
|
||
<p>Using functional programming model you can also benefit from functional composition where you can dynamically compose complex handlers from a set of simple functions.
|
||
As an example let’s add the following function bean to the application defined above</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@Bean
|
||
public Function<String, String> wrapInQuotes() {
|
||
return s -> "\"" + s + "\"";
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>and modify the <code>spring.cloud.function.definition</code> property to reflect your intention to compose a new function from both ‘toUpperCase’ and ‘wrapInQuotes’.
|
||
To do so Spring Cloud Function relies on <code>|</code> (pipe) symbol. So, to finish our example our property will now look like this:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">--spring.cloud.function.definition=toUpperCase|wrapInQuotes</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="admonitionblock note">
|
||
<table>
|
||
<tr>
|
||
<td class="icon">
|
||
<i class="fa icon-note" title="Note"></i>
|
||
</td>
|
||
<td class="content">
|
||
One of the great benefits of functional composition support provided by <em>Spring Cloud Function</em> is
|
||
the fact that you can compose <em>reactive</em> and <em>imperative</em> functions.
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The result of a composition is a single function which, as you may guess, could have a very long and rather cryptic name (e.g., <code>foo|bar|baz|xyz. . .</code>)
|
||
presenting a great deal of inconvenience when it comes to other configuration properties. This is where <em>descriptive binding names</em>
|
||
feature described in <a href="#_functional_binding_names">Functional binding names</a> section can help.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>For example, if we want to give our <code>toUpperCase|wrapInQuotes</code> a more descriptive name we can do so
|
||
with the following property <code>spring.cloud.stream.function.bindings.toUpperCase|wrapInQuotes=quotedUpperCase</code> allowing
|
||
other configuration properties to refer to that binding name (e.g., <code>spring.cloud.stream.bindings.quotedUpperCase.destination=myDestination</code>).</p>
|
||
</div>
|
||
</div>
|
||
<div class="sect4">
|
||
<h5 id="_functions_with_multiple_input_and_output_arguments"><a class="link" href="#_functions_with_multiple_input_and_output_arguments">Functions with multiple input and output arguments</a></h5>
|
||
<div class="paragraph">
|
||
<p>Starting with version 3.0 spring-cloud-stream provides support for functions that
|
||
have multiple inputs and/or multiple outputs (return values). What does this actually mean and
|
||
what type of use cases it is targeting?</p>
|
||
</div>
|
||
<div class="ulist">
|
||
<ul>
|
||
<li>
|
||
<p><em>Big Data: Imagine the source of data you’re dealing with is highly un-organized and contains various types of data elements
|
||
(e.g., orders, transactions etc) and you effectively need to sort it out.</em></p>
|
||
</li>
|
||
<li>
|
||
<p><em>Data aggregation: Another use case may require you to merge data elements from 2+ incoming _streams</em>.</p>
|
||
</li>
|
||
</ul>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The above describes just a few use cases where you may need to use a single function to accept and/or produce
|
||
multiple <em>streams</em> of data. And that is the type of use cases we are targeting here.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Also, note a slightly different emphasis on the concept of <em>streams</em> here. The assumption is that such functions are only valuable
|
||
if they are given access to the actual streams of data (not the individual elements). So for that we are relying on
|
||
abstractions provided by <a href="https://projectreactor.io/">Project Reactor</a> (i.e., <code>Flux</code> and <code>Mono</code>) which is already available on the
|
||
classpath as part of the dependencies brought in by spring-cloud-functions.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Another important aspect is representation of multiple input and outputs. While java provides
|
||
variety of different abstractions to represent <em>multiple of something</em> those abstractions
|
||
are <em>a) unbounded</em>, <em>b) lack arity</em> and <em>c) lack type information</em> which are all important in this context.
|
||
As an example, let’s look at <code>Collection</code> or an array which only allows us to
|
||
describe <em>multiple</em> of a single type or up-cast everything to an <code>Object</code>, affecting transparent type conversion feature of
|
||
spring-cloud-stream and so on.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>So to accommodate all these requirements the initial support is relying on he signature which utilizes another abstraction
|
||
provided by <em>Project Reactor</em> - Tuples. However, we are working on allowing a more flexible signatures.</p>
|
||
</div>
|
||
<div class="admonitionblock important">
|
||
<table>
|
||
<tr>
|
||
<td class="icon">
|
||
<i class="fa icon-important" title="Important"></i>
|
||
</td>
|
||
<td class="content">
|
||
Please refer to <a href="#_binding_and_binding_names">Binding and Binding names</a> section to understand the naming convention used to establish <em>binding names</em>
|
||
used by such application.
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Let’s look at the few samples:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@SpringBootApplication
|
||
public class SampleApplication {
|
||
|
||
@Bean
|
||
public Function<Tuple2<Flux<String>, Flux<Integer>>, Flux<String>> gather() {
|
||
return tuple -> {
|
||
Flux<String> stringStream = tuple.getT1();
|
||
Flux<String> intStream = tuple.getT2().map(i -> String.valueOf(i));
|
||
return Flux.merge(stringStream, intStream);
|
||
};
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The above example demonstrates function which takes two inputs (first of type <code>String</code> and second of type <code>Integer</code>)
|
||
and produces a single output of type <code>String</code>.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>So, for the above example the two input bindings will be <code>gather-in-0</code> and <code>gather-in-1</code> and for consistency the
|
||
output binding also follows the same convention and is named <code>gather-out-0</code>.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Knowing that will allow you to set binding specific properties the same way you did with <code>@StreamListener</code>.
|
||
For example, the following will override content-type for <code>gather-in-0</code> binding:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre>--spring.cloud.stream.bindings.gather-in-0.content-type=text/plain</pre>
|
||
</div>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@SpringBootApplication
|
||
public class SampleApplication {
|
||
|
||
@Bean
|
||
public static Function<Flux<Integer>, Tuple2<Flux<String>, Flux<String>>> scatter() {
|
||
return flux -> {
|
||
Flux<Integer> connectedFlux = flux.publish().autoConnect(2);
|
||
UnicastProcessor even = UnicastProcessor.create();
|
||
UnicastProcessor odd = UnicastProcessor.create();
|
||
Flux<Integer> evenFlux = connectedFlux.filter(number -> number % 2 == 0).doOnNext(number -> even.onNext("EVEN: " + number));
|
||
Flux<Integer> oddFlux = connectedFlux.filter(number -> number % 2 != 0).doOnNext(number -> odd.onNext("ODD: " + number));
|
||
|
||
return Tuples.of(Flux.from(even).doOnSubscribe(x -> evenFlux.subscribe()), Flux.from(odd).doOnSubscribe(x -> oddFlux.subscribe()));
|
||
};
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The above example is somewhat of a the opposite from the previous sample and demonstrates function which
|
||
takes single input of type <code>Integer</code> and produces two outputs (both of type <code>String</code>).</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>So, for the above example the input binding is <code>gather-in-0</code> and the
|
||
output bindings are <code>gather-out-0</code> and <code>gather-out-1</code>.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>And you test it with the following code:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@Test
|
||
public void testSingleInputMultiOutput() {
|
||
try (ConfigurableApplicationContext context = new SpringApplicationBuilder(
|
||
TestChannelBinderConfiguration.getCompleteConfiguration(
|
||
SampleApplication.class))
|
||
.run("--spring.cloud.function.definition=scatter")) {
|
||
|
||
InputDestination inputDestination = context.getBean(InputDestination.class);
|
||
OutputDestination outputDestination = context.getBean(OutputDestination.class);
|
||
|
||
for (int i = 0; i < 10; i++) {
|
||
inputDestination.send(MessageBuilder.withPayload(String.valueOf(i).getBytes()).build());
|
||
}
|
||
|
||
int counter = 0;
|
||
for (int i = 0; i < 5; i++) {
|
||
Message<byte[]> even = outputDestination.receive(0, 0);
|
||
assertThat(even.getPayload()).isEqualTo(("EVEN: " + String.valueOf(counter++)).getBytes());
|
||
Message<byte[]> odd = outputDestination.receive(0, 1);
|
||
assertThat(odd.getPayload()).isEqualTo(("ODD: " + String.valueOf(counter++)).getBytes());
|
||
}
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sect4">
|
||
<h5 id="_multiple_functions_in_a_single_application"><a class="link" href="#_multiple_functions_in_a_single_application">Multiple functions in a single application</a></h5>
|
||
<div class="paragraph">
|
||
<p>There may also be a need for grouping several message handlers in a single application. You would do so by
|
||
defining several functions.</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@SpringBootApplication
|
||
public class SampleApplication {
|
||
|
||
@Bean
|
||
public Function<String, String> uppercase() {
|
||
return value -> value.toUpperCase();
|
||
}
|
||
|
||
@Bean
|
||
public Function<String, String> reverse() {
|
||
return value -> new StringBuilder(value).reverse().toString();
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>In the above example we have configuration which defines two functions <code>uppercase</code> and <code>reverse</code>.
|
||
So first, as mentioned before, we need to notice that there is a a conflict (more then one function) and therefore
|
||
we need to resolve it by providing <code>spring.cloud.function.definition</code> property pointing to the actual function
|
||
we want to bind. Except here we will use <code>;</code> delimiter to point to both functions (see test case below).</p>
|
||
</div>
|
||
<div class="admonitionblock important">
|
||
<table>
|
||
<tr>
|
||
<td class="icon">
|
||
<i class="fa icon-important" title="Important"></i>
|
||
</td>
|
||
<td class="content">
|
||
As with functions with multiple inputs/outputs, please refer to <a href="#_binding_and_binding_names">Binding and Binding names</a> section to understand the naming
|
||
convention used to establish <em>binding names</em> used by such application.
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>And you test it with the following code:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@Test
|
||
public void testMultipleFunctions() {
|
||
try (ConfigurableApplicationContext context = new SpringApplicationBuilder(
|
||
TestChannelBinderConfiguration.getCompleteConfiguration(
|
||
ReactiveFunctionConfiguration.class))
|
||
.run("--spring.cloud.function.definition=uppercase;reverse")) {
|
||
|
||
InputDestination inputDestination = context.getBean(InputDestination.class);
|
||
OutputDestination outputDestination = context.getBean(OutputDestination.class);
|
||
|
||
Message<byte[]> inputMessage = MessageBuilder.withPayload("Hello".getBytes()).build();
|
||
inputDestination.send(inputMessage, "uppercase-in-0");
|
||
inputDestination.send(inputMessage, "reverse-in-0");
|
||
|
||
Message<byte[]> outputMessage = outputDestination.receive(0, "uppercase-out-0");
|
||
assertThat(outputMessage.getPayload()).isEqualTo("HELLO".getBytes());
|
||
|
||
outputMessage = outputDestination.receive(0, "uppercase-out-1");
|
||
assertThat(outputMessage.getPayload()).isEqualTo("olleH".getBytes());
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sect4">
|
||
<h5 id="_batch_consumers"><a class="link" href="#_batch_consumers">Batch Consumers</a></h5>
|
||
<div class="paragraph">
|
||
<p>When using a <code>MessageChannelBinder</code> that supports batch listeners, and the feature is enabled for the consumer binding, you can set <code>spring.cloud.stream.bindings.<binding-name>.consumer.batch-mode</code> to <code>true</code> to enable the
|
||
entire batch of messages to be passed to the function in a <code>List</code>.</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@Bean
|
||
public Function<List<Person>, Person> findFirstPerson() {
|
||
return persons -> persons.get(0);
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sect4">
|
||
<h5 id="_spring_integration_flow_as_functions"><a class="link" href="#_spring_integration_flow_as_functions">Spring Integration flow as functions</a></h5>
|
||
<div class="paragraph">
|
||
<p>When you implement a function, you may have complex requirements that fit the category
|
||
of <a href="https://www.enterpriseintegrationpatterns.com">Enterprise Integration Patterns</a> (EIP). These are best handled by using a
|
||
framework such as <a href="https://spring.io/projects/spring-integration">Spring Integration</a> (SI), which is a reference implementation of EIP.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Thankfully SI already provides support for exposing integration flows as functions via
|
||
<a href="https://docs.spring.io/spring-integration/docs/current/reference/html/#java-dsl-gateway">Integration flow as gateway</a>
|
||
Consider the following sample:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@SpringBootApplication
|
||
public class FunctionSampleSpringIntegrationApplication {
|
||
|
||
public static void main(String[] args) {
|
||
SpringApplication.run(FunctionSampleSpringIntegrationApplication.class, args);
|
||
}
|
||
|
||
@Bean
|
||
public IntegrationFlow uppercaseFlow() {
|
||
return IntegrationFlows.from(MessageFunction.class, "uppercase")
|
||
.<String, String>transform(String::toUpperCase)
|
||
.logAndReply(LoggingHandler.Level.WARN);
|
||
}
|
||
|
||
public interface MessageFunction extends Function<Message<String>, Message<String>> {
|
||
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>For those who are familiar with SI you can see we define a bean of type <code>IntegrationFlow</code> where we
|
||
declare an integration flow that we want to expose as a <code>Function<String, String></code> (using SI DSL) called <code>uppercase</code>.
|
||
The <code>MessageFunction</code> interface lets us explicitly declare the type of the inputs and outputs for proper type conversion.
|
||
See <a href="#content-type-management">Content Type Negotiation</a> section for more on type conversion.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>To receive raw input you can use <code>from(Function.class, …​)</code>.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The resulting function is bound to the input and output destinations exposed by the target binder.</p>
|
||
</div>
|
||
<div class="admonitionblock important">
|
||
<table>
|
||
<tr>
|
||
<td class="icon">
|
||
<i class="fa icon-important" title="Important"></i>
|
||
</td>
|
||
<td class="content">
|
||
Please refer to <a href="#_binding_and_binding_names">Binding and Binding names</a> section to understand the naming
|
||
convention used to establish <em>binding names</em> used by such application.
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>For more details on interoperability of Spring Integration and Spring Cloud Stream specifically around functional programming model
|
||
you may find <a href="https://spring.io/blog/2019/10/25/spring-cloud-stream-and-spring-integration">this post</a> very interesting, as it dives a bit deeper
|
||
into various patterns you can apply by merging the best of Spring Integration and Spring Cloud Stream/Functions.</p>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sect3">
|
||
<h4 id="_annotation_based_support_legacy"><a class="link" href="#_annotation_based_support_legacy">Annotation-based support (legacy)</a></h4>
|
||
<div class="paragraph">
|
||
<p>As mentioned earlier you can also use Spring Integration annotations based configuration or
|
||
Spring Cloud Stream annotation based configuration.</p>
|
||
</div>
|
||
<div class="sect4">
|
||
<h5 id="_spring_integration_support"><a class="link" href="#_spring_integration_support">Spring Integration Support</a></h5>
|
||
<div class="paragraph">
|
||
<p>Spring Cloud Stream is built on the concepts and patterns defined by <a href="http://www.enterpriseintegrationpatterns.com/">Enterprise Integration Patterns</a> and relies
|
||
in its internal implementation on an already established and popular implementation of Enterprise Integration Patterns within the Spring portfolio of projects:
|
||
<a href="https://projects.spring.io/spring-integration/">Spring Integration</a> framework.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>So its only natural for it to support the foundation, semantics, and configuration options that are already established by Spring Integration</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>For example, you can attach the output channel of a <code>Source</code> to a <code>MessageSource</code> and use the familiar <code>@InboundChannelAdapter</code> annotation, as follows:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@EnableBinding(Source.class)
|
||
public class TimerSource {
|
||
|
||
@Bean
|
||
@InboundChannelAdapter(value = Source.OUTPUT, poller = @Poller(fixedDelay = "10", maxMessagesPerPoll = "1"))
|
||
public MessageSource<String> timerMessageSource() {
|
||
return () -> new GenericMessage<>("Hello Spring Cloud Stream");
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Similarly, you can use @Transformer or @ServiceActivator while providing an implementation of a message handler method for a <em>Processor</em> binding contract, as shown in the following example:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@EnableBinding(Processor.class)
|
||
public class TransformProcessor {
|
||
@Transformer(inputChannel = Processor.INPUT, outputChannel = Processor.OUTPUT)
|
||
public Object transform(String message) {
|
||
return message.toUpperCase();
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="admonitionblock note">
|
||
<table>
|
||
<tr>
|
||
<td class="icon">
|
||
<i class="fa icon-note" title="Note"></i>
|
||
</td>
|
||
<td class="content">
|
||
While this may be skipping ahead a bit, it is important to understand that, when you consume from the same binding using <code>@StreamListener</code> annotation, a pub-sub model is used.
|
||
Each method annotated with <code>@StreamListener</code> receives its own copy of a message, and each one has its own consumer group.
|
||
However, if you consume from the same binding by using one of the Spring Integration annotation (such as <code>@Aggregator</code>, <code>@Transformer</code>, or <code>@ServiceActivator</code>), those consume in a competing model.
|
||
No individual consumer group is created for each subscription.
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
</div>
|
||
<div class="sect4">
|
||
<h5 id="_using_streamlistener_annotation"><a class="link" href="#_using_streamlistener_annotation">Using @StreamListener Annotation</a></h5>
|
||
<div class="paragraph">
|
||
<p>Complementary to its Spring Integration support, Spring Cloud Stream provides its own <code>@StreamListener</code> annotation, modeled after other Spring Messaging annotations
|
||
(<code>@MessageMapping</code>, <code>@JmsListener</code>, <code>@RabbitListener</code>, and others) and provides conviniences, such as content-based routing and others.</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@EnableBinding(Sink.class)
|
||
public class VoteHandler {
|
||
|
||
@Autowired
|
||
VotingService votingService;
|
||
|
||
@StreamListener(Sink.INPUT)
|
||
public void handle(Vote vote) {
|
||
votingService.record(vote);
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>As with other Spring Messaging methods, method arguments can be annotated with <code>@Payload</code>, <code>@Headers</code>, and <code>@Header</code>.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>For methods that return data, you must use the <code>@SendTo</code> annotation to specify the output binding destination for data returned by the method, as shown in the following example:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@EnableBinding(Processor.class)
|
||
public class TransformProcessor {
|
||
|
||
@Autowired
|
||
VotingService votingService;
|
||
|
||
@StreamListener(Processor.INPUT)
|
||
@SendTo(Processor.OUTPUT)
|
||
public VoteResult handle(Vote vote) {
|
||
return votingService.record(vote);
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Similar to Spring MVC you can also benefit from JSR-303/309 compliant validation by annotating your arguments with <code>@Valid</code>.</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@StreamListener(Processor.INPUT)
|
||
@SendTo(Processor.OUTPUT)
|
||
public VoteResult handle(@Valid Vote vote) {
|
||
return votingService.record(vote);
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>In the above example the <code>Vote</code> object and its individual fields will be validated according to the rules set by you (e.g., <code>@NotBlank</code>, <code>@Min</code>/<code>@Max</code> etc.).</p>
|
||
</div>
|
||
<div class="admonitionblock note">
|
||
<table>
|
||
<tr>
|
||
<td class="icon">
|
||
<i class="fa icon-note" title="Note"></i>
|
||
</td>
|
||
<td class="content">
|
||
Spring Cloud Stream does NOT provide a default <code>org.springframework.validation.Validator</code> to avoid potential
|
||
conflicts with validators provided by other frameworks that may be part of your application (e.g., MVC),
|
||
therefore you may need to provide your own validator by configuring a bean of type <code>org.springframework.validation.Validator</code>.
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
</div>
|
||
<div class="sect4">
|
||
<h5 id="_using_streamlistener_for_content_based_routing"><a class="link" href="#_using_streamlistener_for_content_based_routing">Using @StreamListener for Content-based routing</a></h5>
|
||
<div class="paragraph">
|
||
<p>Spring Cloud Stream supports dispatching messages to multiple handler methods annotated with <code>@StreamListener</code> based on conditions.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>In order to be eligible to support conditional dispatching, a method must satisfy the follow conditions:</p>
|
||
</div>
|
||
<div class="ulist">
|
||
<ul>
|
||
<li>
|
||
<p>It must not return a value.</p>
|
||
</li>
|
||
<li>
|
||
<p>It must be an individual message handling method (reactive API methods are not supported).</p>
|
||
</li>
|
||
</ul>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The condition is specified by a SpEL expression in the <code>condition</code> argument of the annotation and is evaluated for each message.
|
||
All the handlers that match the condition are invoked in the same thread, and no assumption must be made about the order in which the invocations take place.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>In the following example of a <code>@StreamListener</code> with dispatching conditions, all the messages bearing a header <code>type</code> with the value <code>bogey</code> are dispatched to the
|
||
<code>receiveBogey</code> method, and all the messages bearing a header <code>type</code> with the value <code>bacall</code> are dispatched to the <code>receiveBacall</code> method.</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@EnableBinding(Sink.class)
|
||
@EnableAutoConfiguration
|
||
public static class TestPojoWithAnnotatedArguments {
|
||
|
||
@StreamListener(target = Sink.INPUT, condition = "headers['type']=='bogey'")
|
||
public void receiveBogey(@Payload BogeyPojo bogeyPojo) {
|
||
// handle the message
|
||
}
|
||
|
||
@StreamListener(target = Sink.INPUT, condition = "headers['type']=='bacall'")
|
||
public void receiveBacall(@Payload BacallPojo bacallPojo) {
|
||
// handle the message
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p><strong>Content Type Negotiation in the Context of <code>condition</code></strong></p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>It is important to understand some of the mechanics behind content-based routing using the <code>condition</code> argument of <code>@StreamListener</code>, especially in the context of the type of the message as a whole.
|
||
It may also help if you familiarize yourself with the <a href="#content-type-management">Content Type Negotiation</a> before you proceed.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Consider the following scenario:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@EnableBinding(Sink.class)
|
||
@EnableAutoConfiguration
|
||
public static class CatsAndDogs {
|
||
|
||
@StreamListener(target = Sink.INPUT, condition = "payload.class.simpleName=='Dog'")
|
||
public void bark(Dog dog) {
|
||
// handle the message
|
||
}
|
||
|
||
@StreamListener(target = Sink.INPUT, condition = "payload.class.simpleName=='Cat'")
|
||
public void purr(Cat cat) {
|
||
// handle the message
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The preceding code is perfectly valid. It compiles and deploys without any issues, yet it never produces the result you expect.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>That is because you are testing something that does not yet exist in a state you expect. That is because the payload of the message is not yet converted from the
|
||
wire format (<code>byte[]</code>) to the desired type.
|
||
In other words, it has not yet gone through the type conversion process described in the <a href="#content-type-management">Content Type Negotiation</a>.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>So, unless you use a SPeL expression that evaluates raw data (for example, the value of the first byte in the byte array), use message header-based expressions
|
||
(such as <code>condition = "headers['type']=='dog'"</code>).</p>
|
||
</div>
|
||
<div class="admonitionblock note">
|
||
<table>
|
||
<tr>
|
||
<td class="icon">
|
||
<i class="fa icon-note" title="Note"></i>
|
||
</td>
|
||
<td class="content">
|
||
At the moment, dispatching through <code>@StreamListener</code> conditions is supported only for channel-based binders (not for reactive programming)
|
||
support.
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sect3">
|
||
<h4 id="spring-cloud-streams-overview-using-polled-consumers"><a class="link" href="#spring-cloud-streams-overview-using-polled-consumers">Using Polled Consumers</a></h4>
|
||
<div class="sect4">
|
||
<h5 id="_overview_2"><a class="link" href="#_overview_2">Overview</a></h5>
|
||
<div class="paragraph">
|
||
<p>When using polled consumers, you poll the <code>PollableMessageSource</code> on demand.
|
||
Consider the following example of a polled consumer:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">public interface PolledConsumer {
|
||
|
||
@Input
|
||
PollableMessageSource destIn();
|
||
|
||
@Output
|
||
MessageChannel destOut();
|
||
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Given the polled consumer in the preceding example, you might use it as follows:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@Bean
|
||
public ApplicationRunner poller(PollableMessageSource destIn, MessageChannel destOut) {
|
||
return args -> {
|
||
while (someCondition()) {
|
||
try {
|
||
if (!destIn.poll(m -> {
|
||
String newPayload = ((String) m.getPayload()).toUpperCase();
|
||
destOut.send(new GenericMessage<>(newPayload));
|
||
})) {
|
||
Thread.sleep(1000);
|
||
}
|
||
}
|
||
catch (Exception e) {
|
||
// handle failure
|
||
}
|
||
}
|
||
};
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>A less manual and more Spring-like alternative would be to configure a scheduled task bean. For example,</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@Scheduled(fixedDelay = 5_000)
|
||
public void poll() {
|
||
System.out.println("Polling...");
|
||
this.source.poll(m -> {
|
||
System.out.println(m.getPayload());
|
||
|
||
}, new ParameterizedTypeReference<Foo>() { });
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The <code>PollableMessageSource.poll()</code> method takes a <code>MessageHandler</code> argument (often a lambda expression, as shown here).
|
||
It returns <code>true</code> if the message was received and successfully processed.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>As with message-driven consumers, if the <code>MessageHandler</code> throws an exception, messages are published to error channels,
|
||
as discussed in <code><a href="#spring-cloud-stream-overview-error-handling">Error Handling</a></code>.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Normally, the <code>poll()</code> method acknowledges the message when the <code>MessageHandler</code> exits.
|
||
If the method exits abnormally, the message is rejected (not re-queued), but see <a href="#polled-errors">Handling Errors</a>.
|
||
You can override that behavior by taking responsibility for the acknowledgment, as shown in the following example:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@Bean
|
||
public ApplicationRunner poller(PollableMessageSource dest1In, MessageChannel dest2Out) {
|
||
return args -> {
|
||
while (someCondition()) {
|
||
if (!dest1In.poll(m -> {
|
||
StaticMessageHeaderAccessor.getAcknowledgmentCallback(m).noAutoAck();
|
||
// e.g. hand off to another thread which can perform the ack
|
||
// or acknowledge(Status.REQUEUE)
|
||
|
||
})) {
|
||
Thread.sleep(1000);
|
||
}
|
||
}
|
||
};
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="admonitionblock important">
|
||
<table>
|
||
<tr>
|
||
<td class="icon">
|
||
<i class="fa icon-important" title="Important"></i>
|
||
</td>
|
||
<td class="content">
|
||
You must <code>ack</code> (or <code>nack</code>) the message at some point, to avoid resource leaks.
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
<div class="admonitionblock important">
|
||
<table>
|
||
<tr>
|
||
<td class="icon">
|
||
<i class="fa icon-important" title="Important"></i>
|
||
</td>
|
||
<td class="content">
|
||
Some messaging systems (such as Apache Kafka) maintain a simple offset in a log. If a delivery fails and is re-queued with <code>StaticMessageHeaderAccessor.getAcknowledgmentCallback(m).acknowledge(Status.REQUEUE);</code>, any later successfully ack’d messages are redelivered.
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>There is also an overloaded <code>poll</code> method, for which the definition is as follows:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">poll(MessageHandler handler, ParameterizedTypeReference<?> type)</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The <code>type</code> is a conversion hint that allows the incoming message payload to be converted, as shown in the following example:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">boolean result = pollableSource.poll(received -> {
|
||
Map<String, Foo> payload = (Map<String, Foo>) received.getPayload();
|
||
...
|
||
|
||
}, new ParameterizedTypeReference<Map<String, Foo>>() {});</code></pre>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sect4">
|
||
<h5 id="polled-errors"><a class="link" href="#polled-errors">Handling Errors</a></h5>
|
||
<div class="paragraph">
|
||
<p>By default, an error channel is configured for the pollable source; if the callback throws an exception, an <code>ErrorMessage</code> is sent to the error channel (<code><destination>.<group>.errors</code>); this error channel is also bridged to the global Spring Integration <code>errorChannel</code>.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>You can subscribe to either error channel with a <code>@ServiceActivator</code> to handle errors; without a subscription, the error will simply be logged and the message will be acknowledged as successful.
|
||
If the error channel service activator throws an exception, the message will be rejected (by default) and won’t be redelivered.
|
||
If the service activator throws a <code>RequeueCurrentMessageException</code>, the message will be requeued at the broker and will be again retrieved on a subsequent poll.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>If the listener throws a <code>RequeueCurrentMessageException</code> directly, the message will be requeued, as discussed above, and will not be sent to the error channels.</p>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sect2">
|
||
<h3 id="_event_routing"><a class="link" href="#_event_routing">Event Routing</a></h3>
|
||
<div class="paragraph">
|
||
<p>Event Routing, in the context of Spring Cloud Stream, is the ability to either
|
||
<em>a) route evens to a particular even subscriber</em> or
|
||
<em>b) route event produced by an event subscriber to a particular destination</em>.
|
||
Here we’ll refer to it as route ‘TO’ and route ‘FROM’.</p>
|
||
</div>
|
||
<div class="sect3">
|
||
<h4 id="_routing_to_consumer"><a class="link" href="#_routing_to_consumer">Routing TO Consumer</a></h4>
|
||
<div class="paragraph">
|
||
<p>Routing can be achieved by relying on <code>RoutingFunction</code> available in Spring Cloud Function 3.0. All you need to do is enable it via
|
||
<code>--spring.cloud.stream.function.routing.enabled=true</code> application property or provide <code>spring.cloud.function.routing-expression</code> property.
|
||
Once enabled <code>RoutingFunction</code> will be bound to input destination
|
||
receiving all the messages and route them to other functions based on the provided instruction.</p>
|
||
</div>
|
||
<div class="admonitionblock important">
|
||
<table>
|
||
<tr>
|
||
<td class="icon">
|
||
<i class="fa icon-important" title="Important"></i>
|
||
</td>
|
||
<td class="content">
|
||
For the purposes of binding the name of the routing destination is <code>functionRouter-in-0</code>
|
||
(see RoutingFunction.FUNCTION_NAME and binding naming convention <a href="#_functional_binding_names">Functional binding names</a>).
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Instruction could be provided with individual messages as well as application properties.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Here are couple of samples:</p>
|
||
</div>
|
||
<div class="sect4">
|
||
<h5 id="_using_message_headers"><a class="link" href="#_using_message_headers">Using message headers</a></h5>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@SpringBootApplication
|
||
public class SampleApplication {
|
||
|
||
public static void main(String[] args) {
|
||
SpringApplication.run(SampleApplication.class,
|
||
"--spring.cloud.stream.function.routing.enabled=true");
|
||
}
|
||
|
||
@Bean
|
||
public Consumer<String> even() {
|
||
return value -> {
|
||
System.out.println("EVEN: " + value);
|
||
};
|
||
}
|
||
|
||
@Bean
|
||
public Consumer<String> odd() {
|
||
return value -> {
|
||
System.out.println("ODD: " + value);
|
||
};
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>By sending message to the <code>functionRouter-in-0</code> destination exposed by the binder (i.e., rabbit, kafka),
|
||
such message will be routed to the appropriate (‘even’ or ‘odd’) Consumer.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>By default <code>RoutingFunction</code> will look for <code>spring.cloud.function.definition</code> or <code>spring.cloud.function.routing-expression</code> (for more dynamic scenarios with SpEL)
|
||
header and if it is found its value will be treated as routing instruction.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>For example,
|
||
setting <code>spring.cloud.function.routing-expression</code> header to value <code>T(java.lang.System).currentTimeMillis() % 2 == 0 ? 'even' : 'odd'</code> will end up semi-randomly routing request to either <code>odd</code> or <code>even</code> functions.
|
||
Also, for SpEL, the <em>root object</em> of the evaluation context is <code>Message</code> so you can do evaluation on individual headers (or message) as well <code>…​.routing-expression=headers['type']</code></p>
|
||
</div>
|
||
</div>
|
||
<div class="sect4">
|
||
<h5 id="_using_application_properties"><a class="link" href="#_using_application_properties">Using application properties</a></h5>
|
||
<div class="paragraph">
|
||
<p>The <code>spring.cloud.function.routing-expression</code> and/or <code>spring.cloud.function.definition</code>
|
||
can be passed as application properties (e.g., <code>spring.cloud.function.routing-expression=headers['type']</code>.</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@SpringBootApplication
|
||
public class RoutingStreamApplication {
|
||
|
||
public static void main(String[] args) {
|
||
SpringApplication.run(RoutingStreamApplication.class,
|
||
"--spring.cloud.function.routing-expression="
|
||
+ "T(java.lang.System).nanoTime() % 2 == 0 ? 'even' : 'odd'");
|
||
}
|
||
@Bean
|
||
public Consumer<Integer> even() {
|
||
return value -> System.out.println("EVEN: " + value);
|
||
}
|
||
|
||
@Bean
|
||
public Consumer<Integer> odd() {
|
||
return value -> System.out.println("ODD: " + value);
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="admonitionblock important">
|
||
<table>
|
||
<tr>
|
||
<td class="icon">
|
||
<i class="fa icon-important" title="Important"></i>
|
||
</td>
|
||
<td class="content">
|
||
Passing instructions via application properties is especially important for reactive functions since given that reactive
|
||
function is only invoked once to pass the Publisher, so access to the individual items is limited.
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sect3">
|
||
<h4 id="_routing_from_consumer"><a class="link" href="#_routing_from_consumer">Routing FROM Consumer</a></h4>
|
||
<div class="paragraph">
|
||
<p>Aside from static destinations, Spring Cloud Stream lets applications send messages to dynamically bound destinations.
|
||
This is useful, for example, when the target destination needs to be determined at runtime.
|
||
Applications can do so in one of two ways</p>
|
||
</div>
|
||
<div class="sect4">
|
||
<h5 id="_binderawarechannelresolver"><a class="link" href="#_binderawarechannelresolver">BinderAwareChannelResolver</a></h5>
|
||
<div class="paragraph">
|
||
<p>The <code>BinderAwareChannelResolver</code> is a special bean registered automatically by the framework.
|
||
You can autowire this bean into your application and use it to resolve output destination at runtime</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The 'spring.cloud.stream.dynamicDestinations' property can be used for restricting the dynamic destination names to a known set (whitelisting).
|
||
If this property is not set, any destination can be bound dynamically.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The following example demonstrates one of the common scenarios where REST controller uses a path variable to determine target destination:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@SpringBootApplication
|
||
@Controller
|
||
public class SourceWithDynamicDestination {
|
||
|
||
@Autowired
|
||
private BinderAwareChannelResolver resolver;
|
||
|
||
@RequestMapping(value="/{target}")
|
||
@ResponseStatus(HttpStatus.ACCEPTED)
|
||
public void send(@RequestBody String body, @PathVariable("target") String target){
|
||
resolver.resolveDestination(target).send(new GenericMessage<String>(body));
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Now consider what happens when we start the application on the default port (8080) and make the following requests with CURL:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre>curl -H "Content-Type: application/json" -X POST -d "customer-1" http://localhost:8080/customers
|
||
|
||
curl -H "Content-Type: application/json" -X POST -d "order-1" http://localhost:8080/orders</pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The destinations, 'customers' and 'orders', are created in the broker (in the exchange for Rabbit or in the topic for Kafka)
|
||
with names of 'customers' and 'orders', and the data is published to the appropriate destinations.</p>
|
||
</div>
|
||
</div>
|
||
<div class="sect4">
|
||
<h5 id="_spring_cloud_stream_sendto_destination"><a class="link" href="#_spring_cloud_stream_sendto_destination">spring.cloud.stream.sendto.destination</a></h5>
|
||
<div class="paragraph">
|
||
<p>You can also delegate to the framework to dynamically resolve the output destination by specifying <code>spring.cloud.stream.sendto.destination</code> header
|
||
set to the name of the destination to be resolved.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Consider the following example:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@SpringBootApplication
|
||
@Controller
|
||
public class SourceWithDynamicDestination {
|
||
|
||
@Bean
|
||
public Function<String, Message<String>> destinationAsPayload() {
|
||
return value -> {
|
||
return MessageBuilder.withPayload(value)
|
||
.setHeader("spring.cloud.stream.sendto.destination", value).build();};
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Albeit trivial you can clearly see in this example, our output is a Message with <code>spring.cloud.stream.sendto.destination</code> header
|
||
set to the value of he input argument. The framework will consult this header and will attempt to create or discover
|
||
destination with that name and send output to it.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>If destination names are known in advance, you can configure the producer properties as with any other destination.
|
||
Alternatively, if you register a <code>NewDestinationBindingCallback<></code> bean, it is invoked just before the binding is created.
|
||
The callback takes the generic type of the extended producer properties used by the binder.
|
||
It has one method:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">void configure(String destinationName, MessageChannel channel, ProducerProperties producerProperties,
|
||
T extendedProducerProperties);</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The following example shows how to use the RabbitMQ binder:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@Bean
|
||
public NewDestinationBindingCallback<RabbitProducerProperties> dynamicConfigurer() {
|
||
return (name, channel, props, extended) -> {
|
||
props.setRequiredGroups("bindThisQueue");
|
||
extended.setQueueNameGroupOnly(true);
|
||
extended.setAutoBindDlq(true);
|
||
extended.setDeadLetterQueueName("myDLQ");
|
||
};
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="admonitionblock note">
|
||
<table>
|
||
<tr>
|
||
<td class="icon">
|
||
<i class="fa icon-note" title="Note"></i>
|
||
</td>
|
||
<td class="content">
|
||
If you need to support dynamic destinations with multiple binder types, use <code>Object</code> for the generic type and cast the <code>extended</code> argument as needed.
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Also, please see <a href="#_using_streambridge">Using StreamBridge</a> section to see how yet another option (StreamBridge) can be utilized for similar cases.</p>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sect2">
|
||
<h3 id="spring-cloud-stream-overview-error-handling"><a class="link" href="#spring-cloud-stream-overview-error-handling">Error Handling</a></h3>
|
||
<div class="paragraph">
|
||
<p>In this section we’ll explain the general idea behind error handling mechanisms provided by the framework.
|
||
We’ll be using Rabbit binder as an example, since individual binders define different set
|
||
of properties for certain supported mechanisms specific to underlying broker capabilities (such as Kafka binder).</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Errors happen, and Spring Cloud Stream provides several flexible mechanisms to deal with them. Note that the techniques are dependent on binder implementation and the
|
||
capability of the underlying messaging middleware.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Whenever there is an exception during message processing, the framework will make several attempts at re-trying
|
||
the same message (3 by default). For that, the framework uses <a href="https://github.com/spring-projects/spring-retry">Spring Retry</a> library
|
||
(for imperative functions and standard message handlers) and <code>retryBackoff</code> capabilities of the reactive API (for reactive
|
||
functions).</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Whenever handler (function) throws and exception, it is propagated back to the binder, and the binder subsequently propagates
|
||
the error back to the messaging system.
|
||
Depending on the capabilities of the messaging system such system may <em>drop</em> the message, <em>re-queue</em> the message for re-processing or <em>send the failed message to DLQ</em>.
|
||
Both Rabbit and Kafka support these concepts. However, other binders may not, so refer to your individual binder’s documentation for details on supported
|
||
error-handling options.</p>
|
||
</div>
|
||
<div class="sect3">
|
||
<h4 id="_drop_failed_messages"><a class="link" href="#_drop_failed_messages">Drop Failed Messages</a></h4>
|
||
<div class="paragraph">
|
||
<p>By default, if no additional system-level configuration is provided, the messaging system drops the failed message.
|
||
While acceptable in some cases, for most cases, it is not, and we need some recovery mechanism to avoid message loss.</p>
|
||
</div>
|
||
</div>
|
||
<div class="sect3">
|
||
<h4 id="_dlq_dead_letter_queue"><a class="link" href="#_dlq_dead_letter_queue">DLQ - Dead Letter Queue</a></h4>
|
||
<div class="paragraph">
|
||
<p>Perhaps the most common mechanism, DLQ allows failed messages to be sent to a special destination: - <em>Dead Letter Queue</em>.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>When configured, failed messages are sent to this destination for subsequent re-processing or auditing and reconciliation.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Consider the following example:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@SpringBootApplication
|
||
public class SimpleStreamApplication {
|
||
|
||
public static void main(String[] args) throws Exception {
|
||
SpringApplication.run(SimpleStreamApplication.class,
|
||
"--spring.cloud.function.definition=uppercase",
|
||
"--spring.cloud.stream.bindings.uppercase-in-0.destination=uppercase",
|
||
"--spring.cloud.stream.bindings.uppercase-in-0.group=myGroup",
|
||
"--spring.cloud.stream.rabbit.bindings.uppercase-in-0.consumer.auto-bind-dlq=true"
|
||
);
|
||
}
|
||
|
||
@Bean
|
||
public Function<Person, Person> uppercase() {
|
||
return personIn -> {
|
||
throw new RuntimeException("intentional");
|
||
});
|
||
};
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>As a reminder, in this example <code>uppercase-in-0</code> segment of the property corresponds to the name of the input destination binding.
|
||
The <code>consumer</code> segment indicates that it is a consumer property.</p>
|
||
</div>
|
||
<div class="admonitionblock note">
|
||
<table>
|
||
<tr>
|
||
<td class="icon">
|
||
<i class="fa icon-note" title="Note"></i>
|
||
</td>
|
||
<td class="content">
|
||
When using DLQ, at least <code>group</code> property must be provided for proper naming of the DLQ destination. However <code>group</code> often used together
|
||
with <code>destination</code> property, as in our example.
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Aside from some standard properties we also set the <code>auto-bind-dlq</code> to instruct the binder to create and configure DLQ destination for
|
||
<code>uppercase-in-0</code> binding which corresponds to <code>uppercase</code> destination (see corresponding property), which results in an additional Rabbit queue named <code>uppercase.myGroup.dlq</code> (see Kafka documentation for Kafka specific DLQ properties).</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Once configured, all failed messages are routed to this destination preserving the original message for further actions.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>And you can see that the error message contains more information relevant to the original error, as follows:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-text hljs" data-lang="text">. . . .
|
||
x-exception-stacktrace: org.springframework.messaging.MessageHandlingException: nested exception is
|
||
org.springframework.messaging.MessagingException: has an error, failedMessage=GenericMessage [payload=byte[15],
|
||
headers={amqp_receivedDeliveryMode=NON_PERSISTENT, amqp_receivedRoutingKey=input.hello, amqp_deliveryTag=1,
|
||
deliveryAttempt=3, amqp_consumerQueue=input.hello, amqp_redelivered=false, id=a15231e6-3f80-677b-5ad7-d4b1e61e486e,
|
||
amqp_consumerTag=amq.ctag-skBFapilvtZhDsn0k3ZmQg, contentType=application/json, timestamp=1522327846136}]
|
||
at org.spring...integ...han...MethodInvokingMessageProcessor.processMessage(MethodInvokingMessageProcessor.java:107)
|
||
at. . . . .
|
||
Payload: blah</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>You can also facilitate immediate dispatch to DLQ (without re-tries) by setting <code>max-attempts</code> to '1'. For example,</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-text hljs" data-lang="text">--spring.cloud.stream.bindings.uppercase-in-0.consumer.max-attempts=1</code></pre>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sect3">
|
||
<h4 id="_retry_template_and_retrybackoff"><a class="link" href="#_retry_template_and_retrybackoff">Retry Template and retryBackoff</a></h4>
|
||
<div class="paragraph">
|
||
<p>In this section we cover configuration properties relevant to configuration of retry capabilities.
|
||
Given that we use two different mechanisms for imperative and reactive handlers (RetryTemplate and retryBackoff), properties that corresponds to both will be identified as such.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The <code>RetryTemplate</code> is part of the <a href="https://github.com/spring-projects/spring-retry">Spring Retry</a> library.
|
||
While it is out of scope of this document to cover all of the capabilities of the <code>RetryTemplate</code>, we will mention the following consumer properties that are specifically related to
|
||
the <code>RetryTemplate</code>:</p>
|
||
</div>
|
||
<div class="dlist">
|
||
<dl>
|
||
<dt class="hdlist1">maxAttempts</dt>
|
||
<dd>
|
||
<p>The number of attempts to process the message.</p>
|
||
<div class="paragraph">
|
||
<p>Default: 3. - Applies to 'retryBackoff'</p>
|
||
</div>
|
||
</dd>
|
||
<dt class="hdlist1">backOffInitialInterval</dt>
|
||
<dd>
|
||
<p>The backoff initial interval on retry.</p>
|
||
<div class="paragraph">
|
||
<p>Default 1000 milliseconds. - Applies to 'retryBackoff'</p>
|
||
</div>
|
||
</dd>
|
||
<dt class="hdlist1">backOffMaxInterval</dt>
|
||
<dd>
|
||
<p>The maximum backoff interval.</p>
|
||
<div class="paragraph">
|
||
<p>Default 10000 milliseconds. - Applies to 'retryBackoff'</p>
|
||
</div>
|
||
</dd>
|
||
<dt class="hdlist1">backOffMultiplier</dt>
|
||
<dd>
|
||
<p>The backoff multiplier.</p>
|
||
<div class="paragraph">
|
||
<p>Default 2.0.</p>
|
||
</div>
|
||
</dd>
|
||
<dt class="hdlist1">defaultRetryable</dt>
|
||
<dd>
|
||
<p>Whether exceptions thrown by the listener that are not listed in the <code>retryableExceptions</code> are retryable.</p>
|
||
<div class="paragraph">
|
||
<p>Default: <code>true</code>.</p>
|
||
</div>
|
||
</dd>
|
||
<dt class="hdlist1">retryableExceptions</dt>
|
||
<dd>
|
||
<p>A map of Throwable class names in the key and a boolean in the value.
|
||
Specify those exceptions (and subclasses) that will or won’t be retried.
|
||
Also see <code>defaultRetriable</code>.
|
||
Example: <code>spring.cloud.stream.bindings.input.consumer.retryable-exceptions.java.lang.IllegalStateException=false</code>.</p>
|
||
<div class="paragraph">
|
||
<p>Default: empty.</p>
|
||
</div>
|
||
</dd>
|
||
</dl>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>While the preceding settings are sufficient for majority of the customization requirements, they may not satisfy certain complex requirements at, which
|
||
point you may want to provide your own instance of the <code>RetryTemplate</code>. To do so configure it as a bean in your application configuration. The application provided
|
||
instance will override the one provided by the framework. Also, to avoid conflicts you must qualify the instance of the <code>RetryTemplate</code> you want to be used by the binder
|
||
as <code>@StreamRetryTemplate</code>. For example,</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@StreamRetryTemplate
|
||
public RetryTemplate myRetryTemplate() {
|
||
return new RetryTemplate();
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>As you can see from the above example you don’t need to annotate it with <code>@Bean</code> since <code>@StreamRetryTemplate</code> is a qualified <code>@Bean</code>.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>If you need to be more precise with your <code>RetryTemplate</code>, you can specify the bean by name in your <code>ConsumerProperties</code> to associate
|
||
the specific retry bean per binding.</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code>spring.cloud.stream.bindings.<foo>.consumer.retry-template-name=<your-retry-template-bean-name></code></pre>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sect1">
|
||
<h2 id="spring-cloud-stream-overview-binders"><a class="link" href="#spring-cloud-stream-overview-binders">Binders</a></h2>
|
||
<div class="sectionbody">
|
||
<div class="paragraph">
|
||
<p>Spring Cloud Stream provides a Binder abstraction for use in connecting to physical destinations at the external middleware.
|
||
This section provides information about the main concepts behind the Binder SPI, its main components, and implementation-specific details.</p>
|
||
</div>
|
||
<div class="sect2">
|
||
<h3 id="_producers_and_consumers"><a class="link" href="#_producers_and_consumers">Producers and Consumers</a></h3>
|
||
<div class="paragraph">
|
||
<p>The following image shows the general relationship of producers and consumers:</p>
|
||
</div>
|
||
<div class="imageblock text-center">
|
||
<div class="content">
|
||
<img src="https://raw.githubusercontent.com/spring-cloud/spring-cloud-stream/master/docs/src/main/asciidoc/images/producers-consumers.png" alt="producers consumers" width="800">
|
||
</div>
|
||
<div class="title">Figure 5. Producers and Consumers</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>A producer is any component that sends messages to a binding destination.
|
||
The binding destination can be bound to an external message broker with a <code>Binder</code> implementation for that broker.
|
||
When invoking the <code>bindProducer()</code> method, the first parameter is the name of the destination within the broker, the second parameter is the instance if local destination to which the producer sends messages, and the third parameter contains properties (such as a partition key expression) to be used within the adapter that is created for that binding destination.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>A consumer is any component that receives messages from the binding destination.
|
||
As with a producer, the consumer can be bound to an external message broker.
|
||
When invoking the <code>bindConsumer()</code> method, the first parameter is the destination name, and a second parameter provides the name of a logical group of consumers.
|
||
Each group that is represented by consumer bindings for a given destination receives a copy of each message that a producer sends to that destination (that is, it follows normal publish-subscribe semantics).
|
||
If there are multiple consumer instances bound with the same group name, then messages are load-balanced across those consumer instances so that each message sent by a producer is consumed by only a single consumer instance within each group (that is, it follows normal queueing semantics).</p>
|
||
</div>
|
||
</div>
|
||
<div class="sect2">
|
||
<h3 id="spring-cloud-stream-overview-binder-api"><a class="link" href="#spring-cloud-stream-overview-binder-api">Binder SPI</a></h3>
|
||
<div class="paragraph">
|
||
<p>The Binder SPI consists of a number of interfaces, out-of-the box utility classes, and discovery strategies that provide a pluggable mechanism for connecting to external middleware.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The key point of the SPI is the <code>Binder</code> interface, which is a strategy for connecting inputs and outputs to external middleware. The following listing shows the definition of the <code>Binder</code> interface:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">public interface Binder<T, C extends ConsumerProperties, P extends ProducerProperties> {
|
||
Binding<T> bindConsumer(String bindingName, String group, T inboundBindTarget, C consumerProperties);
|
||
|
||
Binding<T> bindProducer(String bindingName, T outboundBindTarget, P producerProperties);
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The interface is parameterized, offering a number of extension points:</p>
|
||
</div>
|
||
<div class="ulist">
|
||
<ul>
|
||
<li>
|
||
<p>Input and output bind targets.</p>
|
||
</li>
|
||
<li>
|
||
<p>Extended consumer and producer properties, allowing specific Binder implementations to add supplemental properties that can be supported in a type-safe manner.</p>
|
||
</li>
|
||
</ul>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>A typical binder implementation consists of the following:</p>
|
||
</div>
|
||
<div class="ulist">
|
||
<ul>
|
||
<li>
|
||
<p>A class that implements the <code>Binder</code> interface;</p>
|
||
</li>
|
||
<li>
|
||
<p>A Spring <code>@Configuration</code> class that creates a bean of type <code>Binder</code> along with the middleware connection infrastructure.</p>
|
||
</li>
|
||
<li>
|
||
<p>A <code>META-INF/spring.binders</code> file found on the classpath containing one or more binder definitions, as shown in the following example:</p>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code>kafka:\
|
||
org.springframework.cloud.stream.binder.kafka.config.KafkaBinderConfiguration</code></pre>
|
||
</div>
|
||
</div>
|
||
</li>
|
||
</ul>
|
||
</div>
|
||
<div class="admonitionblock note">
|
||
<table>
|
||
<tr>
|
||
<td class="icon">
|
||
<i class="fa icon-note" title="Note"></i>
|
||
</td>
|
||
<td class="content">
|
||
As it was mentioned earlier Binder abstraction is also one of the extension points of the framework. So if you can’t find a suitable binder in the preceding list you can implement your own binder on top of Spring Cloud Stream.
|
||
In the <a href="https://medium.com/@domenicosibilio/how-to-create-a-spring-cloud-stream-binder-from-scratch-ab8b29ee931b">How to create a Spring Cloud Stream Binder from scratch</a> post a community member documents
|
||
in details, with an example, a set of steps necessary to implement a custom binder.
|
||
The steps are also highlighted in the <code><a href="#spring-cloud-stream-overview-custom-binder-impl">Implementing Custom Binders</a></code> section.
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
</div>
|
||
<div class="sect2">
|
||
<h3 id="_binder_detection"><a class="link" href="#_binder_detection">Binder Detection</a></h3>
|
||
<div class="paragraph">
|
||
<p>Spring Cloud Stream relies on implementations of the Binder SPI to perform the task of connecting (binding) user code to message brokers.
|
||
Each Binder implementation typically connects to one type of messaging system.</p>
|
||
</div>
|
||
<div class="sect3">
|
||
<h4 id="_classpath_detection"><a class="link" href="#_classpath_detection">Classpath Detection</a></h4>
|
||
<div class="paragraph">
|
||
<p>By default, Spring Cloud Stream relies on Spring Boot’s auto-configuration to configure the binding process.
|
||
If a single Binder implementation is found on the classpath, Spring Cloud Stream automatically uses it.
|
||
For example, a Spring Cloud Stream project that aims to bind only to RabbitMQ can add the following dependency:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-xml hljs" data-lang="xml"><dependency>
|
||
<groupId>org.springframework.cloud</groupId>
|
||
<artifactId>spring-cloud-stream-binder-rabbit</artifactId>
|
||
</dependency></code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>For the specific Maven coordinates of other binder dependencies, see the documentation of that binder implementation.</p>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sect2">
|
||
<h3 id="multiple-binders"><a class="link" href="#multiple-binders">Multiple Binders on the Classpath</a></h3>
|
||
<div class="paragraph">
|
||
<p>When multiple binders are present on the classpath, the application must indicate which binder is to be used for each destination binding.
|
||
Each binder configuration contains a <code>META-INF/spring.binders</code> file, which is a simple properties file, as shown in the following example:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code>rabbit:\
|
||
org.springframework.cloud.stream.binder.rabbit.config.RabbitServiceAutoConfiguration</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Similar files exist for the other provided binder implementations (such as Kafka), and custom binder implementations are expected to provide them as well.
|
||
The key represents an identifying name for the binder implementation, whereas the value is a comma-separated list of configuration classes that each contain one and only one bean definition of type <code>org.springframework.cloud.stream.binder.Binder</code>.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Binder selection can either be performed globally, using the <code>spring.cloud.stream.defaultBinder</code> property (for example, <code>spring.cloud.stream.defaultBinder=rabbit</code>) or individually, by configuring the binder on each binding.
|
||
For instance, a processor application (that has bindings named <code>input</code> and <code>output</code> for read and write respectively) that reads from Kafka and writes to RabbitMQ can specify the following configuration:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code>spring.cloud.stream.bindings.input.binder=kafka
|
||
spring.cloud.stream.bindings.output.binder=rabbit</code></pre>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sect2">
|
||
<h3 id="multiple-systems"><a class="link" href="#multiple-systems">Connecting to Multiple Systems</a></h3>
|
||
<div class="paragraph">
|
||
<p>By default, binders share the application’s Spring Boot auto-configuration, so that one instance of each binder found on the classpath is created.
|
||
If your application should connect to more than one broker of the same type, you can specify multiple binder configurations, each with different environment settings.</p>
|
||
</div>
|
||
<div class="admonitionblock note">
|
||
<table>
|
||
<tr>
|
||
<td class="icon">
|
||
<i class="fa icon-note" title="Note"></i>
|
||
</td>
|
||
<td class="content">
|
||
Turning on explicit binder configuration disables the default binder configuration process altogether.
|
||
If you do so, all binders in use must be included in the configuration.
|
||
Frameworks that intend to use Spring Cloud Stream transparently may create binder configurations that can be referenced by name, but they do not affect the default binder configuration.
|
||
In order to do so, a binder configuration may have its <code>defaultCandidate</code> flag set to false (for example, <code>spring.cloud.stream.binders.<configurationName>.defaultCandidate=false</code>).
|
||
This denotes a configuration that exists independently of the default binder configuration process.
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The following example shows a typical configuration for a processor application that connects to two RabbitMQ broker instances:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-yml hljs" data-lang="yml">spring:
|
||
cloud:
|
||
stream:
|
||
bindings:
|
||
input:
|
||
destination: thing1
|
||
binder: rabbit1
|
||
output:
|
||
destination: thing2
|
||
binder: rabbit2
|
||
binders:
|
||
rabbit1:
|
||
type: rabbit
|
||
environment:
|
||
spring:
|
||
rabbitmq:
|
||
host: <host1>
|
||
rabbit2:
|
||
type: rabbit
|
||
environment:
|
||
spring:
|
||
rabbitmq:
|
||
host: <host2></code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="admonitionblock note">
|
||
<table>
|
||
<tr>
|
||
<td class="icon">
|
||
<i class="fa icon-note" title="Note"></i>
|
||
</td>
|
||
<td class="content">
|
||
The <code>environment</code> property of the particular binder can also be used for any Spring Boot property,
|
||
including this <code>spring.main.sources</code> which can be useful for adding additional configurations for the
|
||
particular binders, e.g. overriding auto-configured beans.
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>For example;</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-yaml hljs" data-lang="yaml">environment:
|
||
spring:
|
||
main:
|
||
sources: com.acme.config.MyCustomBinderConfiguration</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>To activate a specific profile for the particular binder environment, you should use a <code>spring.profiles.active</code> property:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-yaml hljs" data-lang="yaml">environment:
|
||
spring:
|
||
profiles:
|
||
active: myBinderProfile</code></pre>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sect2">
|
||
<h3 id="binding_visualization_control"><a class="link" href="#binding_visualization_control">Binding visualization and control</a></h3>
|
||
<div class="paragraph">
|
||
<p>Since version 2.0, Spring Cloud Stream supports visualization and control of the Bindings through Actuator endpoints.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Starting with version 2.0 actuator and web are optional, you must first add one of the web dependencies as well as add the actuator dependency manually.
|
||
The following example shows how to add the dependency for the Web framework:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-xml hljs" data-lang="xml"><dependency>
|
||
<groupId>org.springframework.boot</groupId>
|
||
<artifactId>spring-boot-starter-web</artifactId>
|
||
</dependency></code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The following example shows how to add the dependency for the WebFlux framework:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-xml hljs" data-lang="xml"><dependency>
|
||
<groupId>org.springframework.boot</groupId>
|
||
<artifactId>spring-boot-starter-webflux</artifactId>
|
||
</dependency></code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>You can add the Actuator dependency as follows:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-xml hljs" data-lang="xml"><dependency>
|
||
<groupId>org.springframework.boot</groupId>
|
||
<artifactId>spring-boot-starter-actuator</artifactId>
|
||
</dependency></code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="admonitionblock note">
|
||
<table>
|
||
<tr>
|
||
<td class="icon">
|
||
<i class="fa icon-note" title="Note"></i>
|
||
</td>
|
||
<td class="content">
|
||
To run Spring Cloud Stream 2.0 apps in Cloud Foundry, you must add <code>spring-boot-starter-web</code> and <code>spring-boot-starter-actuator</code> to the classpath. Otherwise, the
|
||
application will not start due to health check failures.
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>You must also enable the <code>bindings</code> actuator endpoints by setting the following property: <code>--management.endpoints.web.exposure.include=bindings</code>.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Once those prerequisites are satisfied. you should see the following in the logs when application start:</p>
|
||
</div>
|
||
<div class="literalblock">
|
||
<div class="content">
|
||
<pre>: Mapped "{[/actuator/bindings/{name}],methods=[POST]. . .
|
||
: Mapped "{[/actuator/bindings],methods=[GET]. . .
|
||
: Mapped "{[/actuator/bindings/{name}],methods=[GET]. . .</pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>To visualize the current bindings, access the following URL:
|
||
<code><a href="http://<host>:<port>/actuator/bindings" class="bare">http://<host>:<port>/actuator/bindings</a></code></p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Alternative, to see a single binding, access one of the URLs similar to the following:
|
||
<code><a href="http://<host>:<port>/actuator/bindings/<bindingName>" class="bare">http://<host>:<port>/actuator/bindings/<bindingName></a>;</code></p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>You can also stop, start, pause, and resume individual bindings by posting to the same URL while providing a <code>state</code> argument as JSON, as shown in the following examples:</p>
|
||
</div>
|
||
<div class="literalblock">
|
||
<div class="content">
|
||
<pre>curl -d '{"state":"STOPPED"}' -H "Content-Type: application/json" -X POST http://<host>:<port>/actuator/bindings/myBindingName
|
||
curl -d '{"state":"STARTED"}' -H "Content-Type: application/json" -X POST http://<host>:<port>/actuator/bindings/myBindingName
|
||
curl -d '{"state":"PAUSED"}' -H "Content-Type: application/json" -X POST http://<host>:<port>/actuator/bindings/myBindingName
|
||
curl -d '{"state":"RESUMED"}' -H "Content-Type: application/json" -X POST http://<host>:<port>/actuator/bindings/myBindingName</pre>
|
||
</div>
|
||
</div>
|
||
<div class="admonitionblock note">
|
||
<table>
|
||
<tr>
|
||
<td class="icon">
|
||
<i class="fa icon-note" title="Note"></i>
|
||
</td>
|
||
<td class="content">
|
||
<code>PAUSED</code> and <code>RESUMED</code> work only when the corresponding binder and its underlying technology supports it. Otherwise, you see the warning message in the logs.
|
||
Currently, only Kafka binder supports the <code>PAUSED</code> and <code>RESUMED</code> states.
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
</div>
|
||
<div class="sect2">
|
||
<h3 id="_binder_configuration_properties"><a class="link" href="#_binder_configuration_properties">Binder Configuration Properties</a></h3>
|
||
<div class="paragraph">
|
||
<p>The following properties are available when customizing binder configurations. These properties exposed via <code>org.springframework.cloud.stream.config.BinderProperties</code></p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>They must be prefixed with <code>spring.cloud.stream.binders.<configurationName></code>.</p>
|
||
</div>
|
||
<div class="dlist">
|
||
<dl>
|
||
<dt class="hdlist1">type</dt>
|
||
<dd>
|
||
<p>The binder type.
|
||
It typically references one of the binders found on the classpath — in particular, a key in a <code>META-INF/spring.binders</code> file.</p>
|
||
<div class="paragraph">
|
||
<p>By default, it has the same value as the configuration name.</p>
|
||
</div>
|
||
</dd>
|
||
<dt class="hdlist1">inheritEnvironment</dt>
|
||
<dd>
|
||
<p>Whether the configuration inherits the environment of the application itself.</p>
|
||
<div class="paragraph">
|
||
<p>Default: <code>true</code>.</p>
|
||
</div>
|
||
</dd>
|
||
<dt class="hdlist1">environment</dt>
|
||
<dd>
|
||
<p>Root for a set of properties that can be used to customize the environment of the binder.
|
||
When this property is set, the context in which the binder is being created is not a child of the application context.
|
||
This setting allows for complete separation between the binder components and the application components.</p>
|
||
<div class="paragraph">
|
||
<p>Default: <code>empty</code>.</p>
|
||
</div>
|
||
</dd>
|
||
<dt class="hdlist1">defaultCandidate</dt>
|
||
<dd>
|
||
<p>Whether the binder configuration is a candidate for being considered a default binder or can be used only when explicitly referenced.
|
||
This setting allows adding binder configurations without interfering with the default processing.</p>
|
||
<div class="paragraph">
|
||
<p>Default: <code>true</code>.</p>
|
||
</div>
|
||
</dd>
|
||
</dl>
|
||
</div>
|
||
</div>
|
||
<div class="sect2">
|
||
<h3 id="spring-cloud-stream-overview-custom-binder-impl"><a class="link" href="#spring-cloud-stream-overview-custom-binder-impl">Implementing Custom Binders</a></h3>
|
||
<div class="paragraph">
|
||
<p>In order to implement a custom <code>Binder</code>, all you need is to:</p>
|
||
</div>
|
||
<div class="ulist">
|
||
<ul>
|
||
<li>
|
||
<p>Add the required dependencies</p>
|
||
</li>
|
||
<li>
|
||
<p>Provide a ProvisioningProvider implementation</p>
|
||
</li>
|
||
<li>
|
||
<p>Provide a MessageProducer implementation</p>
|
||
</li>
|
||
<li>
|
||
<p>Provide a MessageHandler implementation</p>
|
||
</li>
|
||
<li>
|
||
<p>Provide a Binder implementation</p>
|
||
</li>
|
||
<li>
|
||
<p>Create a Binder Configuration</p>
|
||
</li>
|
||
<li>
|
||
<p>Define your binder in META-INF/spring.binders</p>
|
||
</li>
|
||
</ul>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p><strong><strong>Add the required dependencies</strong></strong></p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Add the <code>spring-cloud-stream</code> dependency to your project <em>(eg. for Maven)</em>:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-xml hljs" data-lang="xml"><dependency>
|
||
<groupId>org.springframework.cloud</groupId>
|
||
<artifactId>spring-cloud-stream</artifactId>
|
||
<version>${spring.cloud.stream.version}</version>
|
||
</dependency></code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p><strong><strong>Provide a ProvisioningProvider implementation</strong></strong></p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The <code>ProvisioningProvider</code> is responsible for the provisioning of consumer and producer destinations, and is required to convert the logical destinations included in the application.yml or application.properties file in physical destination references.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Below an example of ProvisioningProvider implementation that simply trims the destinations provided via input/output bindings configuration:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">public class FileMessageBinderProvisioner implements ProvisioningProvider<ConsumerProperties, ProducerProperties> {
|
||
|
||
@Override
|
||
public ProducerDestination provisionProducerDestination(
|
||
final String name,
|
||
final ProducerProperties properties) {
|
||
|
||
return new FileMessageDestination(name);
|
||
}
|
||
|
||
@Override
|
||
public ConsumerDestination provisionConsumerDestination(
|
||
final String name,
|
||
final String group,
|
||
final ConsumerProperties properties) {
|
||
|
||
return new FileMessageDestination(name);
|
||
}
|
||
|
||
private class FileMessageDestination implements ProducerDestination, ConsumerDestination {
|
||
|
||
private final String destination;
|
||
|
||
private FileMessageDestination(final String destination) {
|
||
this.destination = destination;
|
||
}
|
||
|
||
@Override
|
||
public String getName() {
|
||
return destination.trim();
|
||
}
|
||
|
||
@Override
|
||
public String getNameForPartition(int partition) {
|
||
throw new UnsupportedOperationException("Partitioning is not implemented for file messaging.");
|
||
}
|
||
|
||
}
|
||
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p><strong><strong>Provide a MessageProducer implementation</strong></strong></p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The <code>MessageProducer</code> is responsible for consuming events and handling them as messages to the client application that is configured to consume such events.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Here is an example of MessageProducer implementation that extends the <code>MessageProducerSupport</code> abstraction in order to poll on a file that matches the trimmed destination name and is located in the project path, while also archiving read messages and discarding consequent identical messages:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">public class FileMessageProducer extends MessageProducerSupport {
|
||
|
||
public static final String ARCHIVE = "archive.txt";
|
||
private final ConsumerDestination destination;
|
||
private String previousPayload;
|
||
|
||
public FileMessageProducer(ConsumerDestination destination) {
|
||
this.destination = destination;
|
||
}
|
||
|
||
@Override
|
||
public void doStart() {
|
||
receive();
|
||
}
|
||
|
||
private void receive() {
|
||
ScheduledExecutorService executorService = Executors.newScheduledThreadPool(1);
|
||
|
||
executorService.scheduleWithFixedDelay(() -> {
|
||
String payload = getPayload();
|
||
|
||
if(payload != null) {
|
||
Message<String> receivedMessage = MessageBuilder.withPayload(payload).build();
|
||
archiveMessage(payload);
|
||
sendMessage(receivedMessage);
|
||
}
|
||
|
||
}, 0, 50, MILLISECONDS);
|
||
}
|
||
|
||
private String getPayload() {
|
||
try {
|
||
List<String> allLines = Files.readAllLines(Paths.get(destination.getName()));
|
||
String currentPayload = allLines.get(allLines.size() - 1);
|
||
|
||
if(!currentPayload.equals(previousPayload)) {
|
||
previousPayload = currentPayload;
|
||
return currentPayload;
|
||
}
|
||
} catch (IOException e) {
|
||
throw new RuntimeException(e);
|
||
}
|
||
|
||
return null;
|
||
}
|
||
|
||
private void archiveMessage(String payload) {
|
||
try {
|
||
Files.write(Paths.get(ARCHIVE), (payload + "\n").getBytes(), CREATE, APPEND);
|
||
} catch (IOException e) {
|
||
throw new RuntimeException(e);
|
||
}
|
||
}
|
||
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="admonitionblock note">
|
||
<table>
|
||
<tr>
|
||
<td class="icon">
|
||
<i class="fa icon-note" title="Note"></i>
|
||
</td>
|
||
<td class="content">
|
||
When implementing a custom binder, this step is not strictly mandatory as you could always resort to using an already existing MessageProducer implementation!
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p><strong><strong>Provide a MessageHandler implementation</strong></strong></p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The <code>MessageHandler</code> provides the logic required to produce an event.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Here is an example of MessageHandler implementation:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">public class FileMessageHandler implements MessageHandler{
|
||
|
||
@Override
|
||
public void handleMessage(Message<?> message) throws MessagingException {
|
||
//write message to file
|
||
}
|
||
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="admonitionblock note">
|
||
<table>
|
||
<tr>
|
||
<td class="icon">
|
||
<i class="fa icon-note" title="Note"></i>
|
||
</td>
|
||
<td class="content">
|
||
When implementing a custom binder, this step is not strictly mandatory as you could always resort to using an already existing MessageHandler implementation!
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p><strong><strong>Provide a Binder implementation</strong></strong></p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>You are now able to provide your own implementation of the <code>Binder</code> abstraction. This can be easily done by:</p>
|
||
</div>
|
||
<div class="ulist">
|
||
<ul>
|
||
<li>
|
||
<p>extending the <code>AbstractMessageChannelBinder</code> class</p>
|
||
</li>
|
||
<li>
|
||
<p>specifying your ProvisioningProvider as a generic argument of the AbstractMessageChannelBinder</p>
|
||
</li>
|
||
<li>
|
||
<p>overriding the <code>createProducerMessageHandler</code> and <code>createConsumerEndpoint</code> methods</p>
|
||
</li>
|
||
</ul>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p><em>eg.:</em></p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">public class FileMessageBinder extends AbstractMessageChannelBinder<ConsumerProperties, ProducerProperties, FileMessageBinderProvisioner> {
|
||
|
||
public FileMessageBinder(
|
||
String[] headersToEmbed,
|
||
FileMessageBinderProvisioner provisioningProvider) {
|
||
|
||
super(headersToEmbed, provisioningProvider);
|
||
}
|
||
|
||
@Override
|
||
protected MessageHandler createProducerMessageHandler(
|
||
final ProducerDestination destination,
|
||
final ProducerProperties producerProperties,
|
||
final MessageChannel errorChannel) throws Exception {
|
||
|
||
return message -> {
|
||
String fileName = destination.getName();
|
||
String payload = new String((byte[])message.getPayload()) + "\n";
|
||
|
||
try {
|
||
Files.write(Paths.get(fileName), payload.getBytes(), CREATE, APPEND);
|
||
} catch (IOException e) {
|
||
throw new RuntimeException(e);
|
||
}
|
||
};
|
||
}
|
||
|
||
@Override
|
||
protected MessageProducer createConsumerEndpoint(
|
||
final ConsumerDestination destination,
|
||
final String group,
|
||
final ConsumerProperties properties) throws Exception {
|
||
|
||
return new FileMessageProducer(destination);
|
||
}
|
||
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p><strong><strong>Create a Binder Configuration</strong></strong></p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>It is strictly required that you create a Spring Configuration to initialize the bean for your binder implementation <em>(and all other beans that you might need)</em>:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@Configuration
|
||
public class FileMessageBinderConfiguration {
|
||
|
||
@Bean
|
||
@ConditionalOnMissingBean
|
||
public FileMessageBinderProvisioner fileMessageBinderProvisioner() {
|
||
return new FileMessageBinderProvisioner();
|
||
}
|
||
|
||
@Bean
|
||
@ConditionalOnMissingBean
|
||
public FileMessageBinder fileMessageBinder(FileMessageBinderProvisioner fileMessageBinderProvisioner) {
|
||
return new FileMessageBinder(null, fileMessageBinderProvisioner);
|
||
}
|
||
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p><strong><strong>Define your binder in META-INF/spring.binders</strong></strong></p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Finally, you must define your binder in a <code>META-INF/spring.binders</code> file on the classpath, specifying both the name of the binder and the full qualified name of your Binder Configuration class:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code>myFileBinder:\
|
||
com.example.springcloudstreamcustombinder.config.FileMessageBinderConfiguration</code></pre>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sect1">
|
||
<h2 id="_configuration_options"><a class="link" href="#_configuration_options">Configuration Options</a></h2>
|
||
<div class="sectionbody">
|
||
<div class="paragraph">
|
||
<p>Spring Cloud Stream supports general configuration options as well as configuration for bindings and binders.
|
||
Some binders let additional binding properties support middleware-specific features.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Configuration options can be provided to Spring Cloud Stream applications through any mechanism supported by Spring Boot.
|
||
This includes application arguments, environment variables, and YAML or .properties files.</p>
|
||
</div>
|
||
<div class="sect2">
|
||
<h3 id="_binding_service_properties"><a class="link" href="#_binding_service_properties">Binding Service Properties</a></h3>
|
||
<div class="paragraph">
|
||
<p>These properties are exposed via <code>org.springframework.cloud.stream.config.BindingServiceProperties</code></p>
|
||
</div>
|
||
<div class="dlist">
|
||
<dl>
|
||
<dt class="hdlist1">spring.cloud.stream.instanceCount</dt>
|
||
<dd>
|
||
<p>The number of deployed instances of an application.
|
||
Must be set for partitioning on the producer side. Must be set on the consumer side when using RabbitMQ and with Kafka if <code>autoRebalanceEnabled=false</code>.</p>
|
||
<div class="paragraph">
|
||
<p>Default: <code>1</code>.</p>
|
||
</div>
|
||
</dd>
|
||
<dt class="hdlist1">spring.cloud.stream.instanceIndex</dt>
|
||
<dd>
|
||
<p>The instance index of the application: A number from <code>0</code> to <code>instanceCount - 1</code>.
|
||
Used for partitioning with RabbitMQ and with Kafka if <code>autoRebalanceEnabled=false</code>.
|
||
Automatically set in Cloud Foundry to match the application’s instance index.</p>
|
||
</dd>
|
||
<dt class="hdlist1">spring.cloud.stream.dynamicDestinations</dt>
|
||
<dd>
|
||
<p>A list of destinations that can be bound dynamically (for example, in a dynamic routing scenario).
|
||
If set, only listed destinations can be bound.</p>
|
||
<div class="paragraph">
|
||
<p>Default: empty (letting any destination be bound).</p>
|
||
</div>
|
||
</dd>
|
||
<dt class="hdlist1">spring.cloud.stream.defaultBinder</dt>
|
||
<dd>
|
||
<p>The default binder to use, if multiple binders are configured.
|
||
See <a href="#multiple-binders">Multiple Binders on the Classpath</a>.</p>
|
||
<div class="paragraph">
|
||
<p>Default: empty.</p>
|
||
</div>
|
||
</dd>
|
||
<dt class="hdlist1">spring.cloud.stream.overrideCloudConnectors</dt>
|
||
<dd>
|
||
<p>This property is only applicable when the <code>cloud</code> profile is active and Spring Cloud Connectors are provided with the application.
|
||
If the property is <code>false</code> (the default), the binder detects a suitable bound service (for example, a RabbitMQ service bound in Cloud Foundry for the RabbitMQ binder) and uses it for creating connections (usually through Spring Cloud Connectors).
|
||
When set to <code>true</code>, this property instructs binders to completely ignore the bound services and rely on Spring Boot properties (for example, relying on the <code>spring.rabbitmq.*</code> properties provided in the environment for the RabbitMQ binder).
|
||
The typical usage of this property is to be nested in a customized environment <a href="#multiple-systems">when connecting to multiple systems</a>.</p>
|
||
<div class="paragraph">
|
||
<p>Default: <code>false</code>.</p>
|
||
</div>
|
||
</dd>
|
||
<dt class="hdlist1">spring.cloud.stream.bindingRetryInterval</dt>
|
||
<dd>
|
||
<p>The interval (in seconds) between retrying binding creation when, for example, the binder does not support late binding and the broker (for example, Apache Kafka) is down.
|
||
Set it to zero to treat such conditions as fatal, preventing the application from starting.</p>
|
||
<div class="paragraph">
|
||
<p>Default: <code>30</code></p>
|
||
</div>
|
||
</dd>
|
||
</dl>
|
||
</div>
|
||
</div>
|
||
<div class="sect2">
|
||
<h3 id="binding-properties"><a class="link" href="#binding-properties">Binding Properties</a></h3>
|
||
<div class="paragraph">
|
||
<p>Binding properties are supplied by using the format of <code>spring.cloud.stream.bindings.<bindingName>.<property>=<value></code>.
|
||
The <code><bindingName></code> represents the name of the binding being configured.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>For example, for the following function</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@Bean
|
||
public Function<String, String> uppercase() {
|
||
return v -> v.toUpperCase();
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>there are two bindings named <code>uppercase-in-0</code> for input and <code>uppercase-out-0</code> for output. See <a href="#_binding_and_binding_names">Binding and Binding names</a> for more details.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>To avoid repetition, Spring Cloud Stream supports setting values for all bindings, in the format of <code>spring.cloud.stream.default.<property>=<value></code>
|
||
and <code>spring.cloud.stream.default.<producer|consumer>.<property>=<value></code> for common binding properties.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>When it comes to avoiding repetitions for extended binding properties, this format should be used - <code>spring.cloud.stream.<binder-type>.default.<producer|consumer>.<property>=<value></code>.</p>
|
||
</div>
|
||
<div class="sect3">
|
||
<h4 id="_common_binding_properties"><a class="link" href="#_common_binding_properties">Common Binding Properties</a></h4>
|
||
<div class="paragraph">
|
||
<p>These properties are exposed via <code>org.springframework.cloud.stream.config.BindingProperties</code></p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The following binding properties are available for both input and output bindings and must be prefixed with <code>spring.cloud.stream.bindings.<bindingName>.</code>
|
||
(for example, <code>spring.cloud.stream.bindings.uppercase-in-0.destination=ticktock</code>).</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Default values can be set by using the <code>spring.cloud.stream.default</code> prefix (for example`spring.cloud.stream.default.contentType=application/json`).</p>
|
||
</div>
|
||
<div class="dlist">
|
||
<dl>
|
||
<dt class="hdlist1">destination</dt>
|
||
<dd>
|
||
<p>The target destination of a binding on the bound middleware (for example, the RabbitMQ exchange or Kafka topic).
|
||
If binding represents a consumer binding (input), it could be bound to multiple destinations, and the destination names can be specified as comma-separated <code>String</code> values.
|
||
If not, he actual binding name is used instead.
|
||
The default value of this property cannot be overridden.</p>
|
||
</dd>
|
||
<dt class="hdlist1">group</dt>
|
||
<dd>
|
||
<p>The consumer group of the binding.
|
||
Applies only to inbound bindings.
|
||
See <a href="#consumer-groups">Consumer Groups</a>.</p>
|
||
<div class="paragraph">
|
||
<p>Default: <code>null</code> (indicating an anonymous consumer).</p>
|
||
</div>
|
||
</dd>
|
||
<dt class="hdlist1">contentType</dt>
|
||
<dd>
|
||
<p>The content type of this binding.
|
||
See <code><a href="#content-type-management">Content Type Negotiation</a></code>.</p>
|
||
<div class="paragraph">
|
||
<p>Default: <code>application/json</code>.</p>
|
||
</div>
|
||
</dd>
|
||
<dt class="hdlist1">binder</dt>
|
||
<dd>
|
||
<p>The binder used by this binding.
|
||
See <code><a href="#multiple-binders">Multiple Binders on the Classpath</a></code> for details.</p>
|
||
<div class="paragraph">
|
||
<p>Default: <code>null</code> (the default binder is used, if it exists).</p>
|
||
</div>
|
||
</dd>
|
||
</dl>
|
||
</div>
|
||
</div>
|
||
<div class="sect3">
|
||
<h4 id="_consumer_properties"><a class="link" href="#_consumer_properties">Consumer Properties</a></h4>
|
||
<div class="paragraph">
|
||
<p>These properties are exposed via <code>org.springframework.cloud.stream.binder.ConsumerProperties</code></p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The following binding properties are available for input bindings only and must be prefixed with <code>spring.cloud.stream.bindings.<bindingName>.consumer.</code> (for example, <code>spring.cloud.stream.bindings.input.consumer.concurrency=3</code>).</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Default values can be set by using the <code>spring.cloud.stream.default.consumer</code> prefix (for example, <code>spring.cloud.stream.default.consumer.headerMode=none</code>).</p>
|
||
</div>
|
||
<div class="dlist">
|
||
<dl>
|
||
<dt class="hdlist1">autoStartup</dt>
|
||
<dd>
|
||
<p>Signals if this consumer needs to be started automatically</p>
|
||
<div class="paragraph">
|
||
<p>Default: <code>true</code>.</p>
|
||
</div>
|
||
</dd>
|
||
<dt class="hdlist1">concurrency</dt>
|
||
<dd>
|
||
<p>The concurrency of the inbound consumer.</p>
|
||
<div class="paragraph">
|
||
<p>Default: <code>1</code>.</p>
|
||
</div>
|
||
</dd>
|
||
<dt class="hdlist1">partitioned</dt>
|
||
<dd>
|
||
<p>Whether the consumer receives data from a partitioned producer.</p>
|
||
<div class="paragraph">
|
||
<p>Default: <code>false</code>.</p>
|
||
</div>
|
||
</dd>
|
||
<dt class="hdlist1">headerMode</dt>
|
||
<dd>
|
||
<p>When set to <code>none</code>, disables header parsing on input.
|
||
Effective only for messaging middleware that does not support message headers natively and requires header embedding.
|
||
This option is useful when consuming data from non-Spring Cloud Stream applications when native headers are not supported.
|
||
When set to <code>headers</code>, it uses the middleware’s native header mechanism.
|
||
When set to <code>embeddedHeaders</code>, it embeds headers into the message payload.</p>
|
||
<div class="paragraph">
|
||
<p>Default: depends on the binder implementation.</p>
|
||
</div>
|
||
</dd>
|
||
<dt class="hdlist1">maxAttempts</dt>
|
||
<dd>
|
||
<p>If processing fails, the number of attempts to process the message (including the first).
|
||
Set to <code>1</code> to disable retry.</p>
|
||
<div class="paragraph">
|
||
<p>Default: <code>3</code>.</p>
|
||
</div>
|
||
</dd>
|
||
<dt class="hdlist1">backOffInitialInterval</dt>
|
||
<dd>
|
||
<p>The backoff initial interval on retry.</p>
|
||
<div class="paragraph">
|
||
<p>Default: <code>1000</code>.</p>
|
||
</div>
|
||
</dd>
|
||
<dt class="hdlist1">backOffMaxInterval</dt>
|
||
<dd>
|
||
<p>The maximum backoff interval.</p>
|
||
<div class="paragraph">
|
||
<p>Default: <code>10000</code>.</p>
|
||
</div>
|
||
</dd>
|
||
<dt class="hdlist1">backOffMultiplier</dt>
|
||
<dd>
|
||
<p>The backoff multiplier.</p>
|
||
<div class="paragraph">
|
||
<p>Default: <code>2.0</code>.</p>
|
||
</div>
|
||
</dd>
|
||
<dt class="hdlist1">defaultRetryable</dt>
|
||
<dd>
|
||
<p>Whether exceptions thrown by the listener that are not listed in the <code>retryableExceptions</code> are retryable.</p>
|
||
<div class="paragraph">
|
||
<p>Default: <code>true</code>.</p>
|
||
</div>
|
||
</dd>
|
||
<dt class="hdlist1">instanceCount</dt>
|
||
<dd>
|
||
<p>When set to a value greater than equal to zero, it allows customizing the instance count of this consumer (if different from <code>spring.cloud.stream.instanceCount</code>).
|
||
When set to a negative value, it defaults to <code>spring.cloud.stream.instanceCount</code>.
|
||
See <code><a href="#spring-cloud-stream-overview-instance-index-instance-count">Instance Index and Instance Count</a></code> for more information.</p>
|
||
<div class="paragraph">
|
||
<p>Default: <code>-1</code>.</p>
|
||
</div>
|
||
</dd>
|
||
<dt class="hdlist1">instanceIndex</dt>
|
||
<dd>
|
||
<p>When set to a value greater than equal to zero, it allows customizing the instance index of this consumer (if different from <code>spring.cloud.stream.instanceIndex</code>).
|
||
When set to a negative value, it defaults to <code>spring.cloud.stream.instanceIndex</code>.
|
||
Ignored if <code>instanceIndexList</code> is provided.
|
||
See <code><a href="#spring-cloud-stream-overview-instance-index-instance-count">Instance Index and Instance Count</a></code> for more information.</p>
|
||
<div class="paragraph">
|
||
<p>Default: <code>-1</code>.</p>
|
||
</div>
|
||
</dd>
|
||
<dt class="hdlist1">instanceIndexList</dt>
|
||
<dd>
|
||
<p>Used with binders that do not support native partitioning (such as RabbitMQ); allows an application instance to consume from more than one partition.</p>
|
||
<div class="paragraph">
|
||
<p>Default: empty.</p>
|
||
</div>
|
||
</dd>
|
||
<dt class="hdlist1">retryableExceptions</dt>
|
||
<dd>
|
||
<p>A map of Throwable class names in the key and a boolean in the value.
|
||
Specify those exceptions (and subclasses) that will or won’t be retried.
|
||
Also see <code>defaultRetriable</code>.
|
||
Example: <code>spring.cloud.stream.bindings.input.consumer.retryable-exceptions.java.lang.IllegalStateException=false</code>.</p>
|
||
<div class="paragraph">
|
||
<p>Default: empty.</p>
|
||
</div>
|
||
</dd>
|
||
<dt class="hdlist1">useNativeDecoding</dt>
|
||
<dd>
|
||
<p>When set to <code>true</code>, the inbound message is deserialized directly by the client library, which must be configured correspondingly (for example, setting an appropriate Kafka producer value deserializer).
|
||
When this configuration is being used, the inbound message unmarshalling is not based on the <code>contentType</code> of the binding.
|
||
When native decoding is used, it is the responsibility of the producer to use an appropriate encoder (for example, the Kafka producer value serializer) to serialize the outbound message.
|
||
Also, when native encoding and decoding is used, the <code>headerMode=embeddedHeaders</code> property is ignored and headers are not embedded in the message.
|
||
See the producer property <code>useNativeEncoding</code>.</p>
|
||
<div class="paragraph">
|
||
<p>Default: <code>false</code>.</p>
|
||
</div>
|
||
</dd>
|
||
<dt class="hdlist1">multiplex</dt>
|
||
<dd>
|
||
<p>When set to true, the underlying binder will natively multiplex destinations on the same input binding.</p>
|
||
<div class="paragraph">
|
||
<p>Default: <code>false</code>.</p>
|
||
</div>
|
||
</dd>
|
||
</dl>
|
||
</div>
|
||
</div>
|
||
<div class="sect3">
|
||
<h4 id="_advanced_consumer_configuration"><a class="link" href="#_advanced_consumer_configuration">Advanced Consumer Configuration</a></h4>
|
||
<div class="paragraph">
|
||
<p>For advanced configuration of the underlying message listener container for message-driven consumers, add a single <code>ListenerContainerCustomizer</code> bean to the application context.
|
||
It will be invoked after the above properties have been applied and can be used to set additional properties.
|
||
Similarly, for polled consumers, add a <code>MessageSourceCustomizer</code> bean.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The following is an example for the RabbitMQ binder:</p>
|
||
</div>
|
||
<div class="exampleblock">
|
||
<div class="content">
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@Bean
|
||
public ListenerContainerCustomizer<AbstractMessageListenerContainer> containerCustomizer() {
|
||
return (container, dest, group) -> container.setAdviceChain(advice1, advice2);
|
||
}
|
||
|
||
@Bean
|
||
public MessageSourceCustomizer<AmqpMessageSource> sourceCustomizer() {
|
||
return (source, dest, group) -> source.setPropertiesConverter(customPropertiesConverter);
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sect3">
|
||
<h4 id="_producer_properties"><a class="link" href="#_producer_properties">Producer Properties</a></h4>
|
||
<div class="paragraph">
|
||
<p>These properties are exposed via <code>org.springframework.cloud.stream.binder.ProducerProperties</code></p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The following binding properties are available for output bindings only and must be prefixed with <code>spring.cloud.stream.bindings.<bindingName>.producer.</code>
|
||
(for example, <code>spring.cloud.stream.bindings.func-out-0.producer.partitionKeyExpression=payload.id</code>).</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Default values can be set by using the prefix <code>spring.cloud.stream.default.producer</code> (for example, <code>spring.cloud.stream.default.producer.partitionKeyExpression=payload.id</code>).</p>
|
||
</div>
|
||
<div class="dlist">
|
||
<dl>
|
||
<dt class="hdlist1">autoStartup</dt>
|
||
<dd>
|
||
<p>Signals if this consumer needs to be started automatically</p>
|
||
<div class="paragraph">
|
||
<p>Default: <code>true</code>.</p>
|
||
</div>
|
||
</dd>
|
||
<dt class="hdlist1">partitionKeyExpression</dt>
|
||
<dd>
|
||
<p>A SpEL expression that determines how to partition outbound data.
|
||
If set, outbound data on this binding is partitioned. <code>partitionCount</code> must be set to a value greater than 1 to be effective.
|
||
See <code><a href="#partitioning">Partitioning Support</a></code>.</p>
|
||
<div class="paragraph">
|
||
<p>Default: null.</p>
|
||
</div>
|
||
</dd>
|
||
<dt class="hdlist1">partitionKeyExtractorName</dt>
|
||
<dd>
|
||
<p>The name of the bean that implements <code>PartitionKeyExtractorStrategy</code>. Used to extract a key used to compute
|
||
the partition id (see 'partitionSelector*'). Mutually exclusive with 'partitionKeyExpression'.</p>
|
||
<div class="paragraph">
|
||
<p>Default: null.</p>
|
||
</div>
|
||
</dd>
|
||
<dt class="hdlist1">partitionSelectorName</dt>
|
||
<dd>
|
||
<p>The name of the bean that implements <code>PartitionSelectorStrategy</code>. Used to determine partition id based
|
||
on partition key (see 'partitionKeyExtractor*'). Mutually exclusive with 'partitionSelectorExpression'.</p>
|
||
<div class="paragraph">
|
||
<p>Default: null.</p>
|
||
</div>
|
||
</dd>
|
||
<dt class="hdlist1">partitionSelectorExpression</dt>
|
||
<dd>
|
||
<p>A SpEL expression for customizing partition selection.
|
||
If neither is set, the partition is selected as the <code>hashCode(key) % partitionCount</code>, where <code>key</code> is computed through either <code>partitionKeyExpression</code>.</p>
|
||
<div class="paragraph">
|
||
<p>Default: <code>null</code>.</p>
|
||
</div>
|
||
</dd>
|
||
<dt class="hdlist1">partitionCount</dt>
|
||
<dd>
|
||
<p>The number of target partitions for the data, if partitioning is enabled.
|
||
Must be set to a value greater than 1 if the producer is partitioned.
|
||
On Kafka, it is interpreted as a hint. The larger of this and the partition count of the target topic is used instead.</p>
|
||
<div class="paragraph">
|
||
<p>Default: <code>1</code>.</p>
|
||
</div>
|
||
</dd>
|
||
<dt class="hdlist1">requiredGroups</dt>
|
||
<dd>
|
||
<p>A comma-separated list of groups to which the producer must ensure message delivery even if they start after it has been created (for example, by pre-creating durable queues in RabbitMQ).</p>
|
||
</dd>
|
||
<dt class="hdlist1">headerMode</dt>
|
||
<dd>
|
||
<p>When set to <code>none</code>, it disables header embedding on output.
|
||
It is effective only for messaging middleware that does not support message headers natively and requires header embedding.
|
||
This option is useful when producing data for non-Spring Cloud Stream applications when native headers are not supported.
|
||
When set to <code>headers</code>, it uses the middleware’s native header mechanism.
|
||
When set to <code>embeddedHeaders</code>, it embeds headers into the message payload.</p>
|
||
<div class="paragraph">
|
||
<p>Default: Depends on the binder implementation.</p>
|
||
</div>
|
||
</dd>
|
||
<dt class="hdlist1">useNativeEncoding</dt>
|
||
<dd>
|
||
<p>When set to <code>true</code>, the outbound message is serialized directly by the client library, which must be configured correspondingly (for example, setting an appropriate Kafka producer value serializer).
|
||
When this configuration is being used, the outbound message marshalling is not based on the <code>contentType</code> of the binding.
|
||
When native encoding is used, it is the responsibility of the consumer to use an appropriate decoder (for example, the Kafka consumer value de-serializer) to deserialize the inbound message.
|
||
Also, when native encoding and decoding is used, the <code>headerMode=embeddedHeaders</code> property is ignored and headers are not embedded in the message.
|
||
See the consumer property <code>useNativeDecoding</code>.</p>
|
||
<div class="paragraph">
|
||
<p>Default: <code>false</code>.</p>
|
||
</div>
|
||
</dd>
|
||
<dt class="hdlist1">errorChannelEnabled</dt>
|
||
<dd>
|
||
<p>When set to true, if the binder supports asynchroous send results, send failures are sent to an error channel for the destination. See Error Handling for more information.</p>
|
||
<div class="paragraph">
|
||
<p>Default: false.</p>
|
||
</div>
|
||
</dd>
|
||
</dl>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sect1">
|
||
<h2 id="content-type-management"><a class="link" href="#content-type-management">Content Type Negotiation</a></h2>
|
||
<div class="sectionbody">
|
||
<div class="paragraph">
|
||
<p>Data transformation is one of the core features of any message-driven microservice architecture. Given that, in Spring Cloud Stream, such data
|
||
is represented as a Spring <code>Message</code>, a message may have to be transformed to a desired shape or size before reaching its destination. This is required for two reasons:</p>
|
||
</div>
|
||
<div class="olist arabic">
|
||
<ol class="arabic">
|
||
<li>
|
||
<p>To convert the contents of the incoming message to match the signature of the application-provided handler.</p>
|
||
</li>
|
||
<li>
|
||
<p>To convert the contents of the outgoing message to the wire format.</p>
|
||
</li>
|
||
</ol>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The wire format is typically <code>byte[]</code> (that is true for the Kafka and Rabbit binders), but it is governed by the binder implementation.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>In Spring Cloud Stream, message transformation is accomplished with an <code>org.springframework.messaging.converter.MessageConverter</code>.</p>
|
||
</div>
|
||
<div class="admonitionblock note">
|
||
<table>
|
||
<tr>
|
||
<td class="icon">
|
||
<i class="fa icon-note" title="Note"></i>
|
||
</td>
|
||
<td class="content">
|
||
As a supplement to the details to follow, you may also want to read the following <a href="https://spring.io/blog/2018/02/26/spring-cloud-stream-2-0-content-type-negotiation-and-transformation">blog post</a>.
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
<div class="sect2">
|
||
<h3 id="_mechanics"><a class="link" href="#_mechanics">Mechanics</a></h3>
|
||
<div class="paragraph">
|
||
<p>To better understand the mechanics and the necessity behind content-type negotiation, we take a look at a very simple use case by using the following message handler as an example:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">public Function<Person, Person> personFunction {..}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="admonitionblock note">
|
||
<table>
|
||
<tr>
|
||
<td class="icon">
|
||
<i class="fa icon-note" title="Note"></i>
|
||
</td>
|
||
<td class="content">
|
||
For simplicity, we assume that this is the only handler function in the application (we assume there is no internal pipeline).
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The handler shown in the preceding example expects a <code>Person</code> object as an argument and produces a <code>String</code> type as an output.
|
||
In order for the framework to succeed in passing the incoming <code>Message</code> as an argument to this handler, it has to somehow transform the payload of the <code>Message</code> type from the wire format to a <code>Person</code> type.
|
||
In other words, the framework must locate and apply the appropriate <code>MessageConverter</code>.
|
||
To accomplish that, the framework needs some instructions from the user.
|
||
One of these instructions is already provided by the signature of the handler method itself (<code>Person</code> type).
|
||
Consequently, in theory, that should be (and, in some cases, is) enough.
|
||
However, for the majority of use cases, in order to select the appropriate <code>MessageConverter</code>, the framework needs an additional piece of information.
|
||
That missing piece is <code>contentType</code>.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Spring Cloud Stream provides three mechanisms to define <code>contentType</code> (in order of precedence):</p>
|
||
</div>
|
||
<div class="olist arabic">
|
||
<ol class="arabic">
|
||
<li>
|
||
<p><strong>HEADER</strong>: The <code>contentType</code> can be communicated through the Message itself. By providing a <code>contentType</code> header, you declare the content type to use to locate and apply the appropriate <code>MessageConverter</code>.</p>
|
||
</li>
|
||
<li>
|
||
<p><strong>BINDING</strong>: The <code>contentType</code> can be set per destination binding by setting the <code>spring.cloud.stream.bindings.input.content-type</code> property.</p>
|
||
<div class="admonitionblock note">
|
||
<table>
|
||
<tr>
|
||
<td class="icon">
|
||
<i class="fa icon-note" title="Note"></i>
|
||
</td>
|
||
<td class="content">
|
||
The <code>input</code> segment in the property name corresponds to the actual name of the destination (which is “input” in our case). This approach lets you declare, on a per-binding basis, the content type to use to locate and apply the appropriate <code>MessageConverter</code>.
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
</li>
|
||
<li>
|
||
<p><strong>DEFAULT</strong>: If <code>contentType</code> is not present in the <code>Message</code> header or the binding, the default <code>application/json</code> content type is used to
|
||
locate and apply the appropriate <code>MessageConverter</code>.</p>
|
||
</li>
|
||
</ol>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>As mentioned earlier, the preceding list also demonstrates the order of precedence in case of a tie. For example, a header-provided content type takes precedence over any other content type.
|
||
The same applies for a content type set on a per-binding basis, which essentially lets you override the default content type.
|
||
However, it also provides a sensible default (which was determined from community feedback).</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Another reason for making <code>application/json</code> the default stems from the interoperability requirements driven by distributed microservices architectures, where producer and consumer not only run in different JVMs but can also run on different non-JVM platforms.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>When the non-void handler method returns, if the return value is already a <code>Message</code>, that <code>Message</code> becomes the payload. However, when the return value is not a <code>Message</code>, the new <code>Message</code> is constructed with the return value as the payload while inheriting
|
||
headers from the input <code>Message</code> minus the headers defined or filtered by <code>SpringIntegrationProperties.messageHandlerNotPropagatedHeaders</code>.
|
||
By default, there is only one header set there: <code>contentType</code>. This means that the new <code>Message</code> does not have <code>contentType</code> header set, thus ensuring that the <code>contentType</code> can evolve.
|
||
You can always opt out of returning a <code>Message</code> from the handler method where you can inject any header you wish.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>If there is an internal pipeline, the <code>Message</code> is sent to the next handler by going through the same process of conversion. However, if there is no internal pipeline or you have reached the end of it, the <code>Message</code> is sent back to the output destination.</p>
|
||
</div>
|
||
<div class="sect3">
|
||
<h4 id="_content_type_versus_argument_type"><a class="link" href="#_content_type_versus_argument_type">Content Type versus Argument Type</a></h4>
|
||
<div class="paragraph">
|
||
<p>As mentioned earlier, for the framework to select the appropriate <code>MessageConverter</code>, it requires argument type and, optionally, content type information.
|
||
The logic for selecting the appropriate <code>MessageConverter</code> resides with the argument resolvers (<code>HandlerMethodArgumentResolvers</code>), which trigger right before the invocation of the user-defined handler method (which is when the actual argument type is known to the framework).
|
||
If the argument type does not match the type of the current payload, the framework delegates to the stack of the
|
||
pre-configured <code>MessageConverters</code> to see if any one of them can convert the payload.
|
||
As you can see, the <code>Object fromMessage(Message<?> message, Class<?> targetClass);</code>
|
||
operation of the MessageConverter takes <code>targetClass</code> as one of its arguments.
|
||
The framework also ensures that the provided <code>Message</code> always contains a <code>contentType</code> header.
|
||
When no contentType header was already present, it injects either the per-binding <code>contentType</code> header or the default <code>contentType</code> header.
|
||
The combination of <code>contentType</code> argument type is the mechanism by which framework determines if message can be converted to a target type.
|
||
If no appropriate <code>MessageConverter</code> is found, an exception is thrown, which you can handle by adding a custom <code>MessageConverter</code> (see <code><a href="#spring-cloud-stream-overview-user-defined-message-converters">User-defined Message Converters</a></code>).</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>But what if the payload type matches the target type declared by the handler method? In this case, there is nothing to convert, and the
|
||
payload is passed unmodified. While this sounds pretty straightforward and logical, keep in mind handler methods that take a <code>Message<?></code> or <code>Object</code> as an argument.
|
||
By declaring the target type to be <code>Object</code> (which is an <code>instanceof</code> everything in Java), you essentially forfeit the conversion process.</p>
|
||
</div>
|
||
<div class="admonitionblock note">
|
||
<table>
|
||
<tr>
|
||
<td class="icon">
|
||
<i class="fa icon-note" title="Note"></i>
|
||
</td>
|
||
<td class="content">
|
||
Do not expect <code>Message</code> to be converted into some other type based only on the <code>contentType</code>.
|
||
Remember that the <code>contentType</code> is complementary to the target type.
|
||
If you wish, you can provide a hint, which <code>MessageConverter</code> may or may not take into consideration.
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
</div>
|
||
<div class="sect3">
|
||
<h4 id="_message_converters"><a class="link" href="#_message_converters">Message Converters</a></h4>
|
||
<div class="paragraph">
|
||
<p><code>MessageConverters</code> define two methods:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">Object fromMessage(Message<?> message, Class<?> targetClass);
|
||
|
||
Message<?> toMessage(Object payload, @Nullable MessageHeaders headers);</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>It is important to understand the contract of these methods and their usage, specifically in the context of Spring Cloud Stream.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The <code>fromMessage</code> method converts an incoming <code>Message</code> to an argument type.
|
||
The payload of the <code>Message</code> could be any type, and it is
|
||
up to the actual implementation of the <code>MessageConverter</code> to support multiple types.
|
||
For example, some JSON converter may support the payload type as <code>byte[]</code>, <code>String</code>, and others.
|
||
This is important when the application contains an internal pipeline (that is, input → handler1 → handler2 →. . . → output) and the output of the upstream handler results in a <code>Message</code> which may not be in the initial wire format.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>However, the <code>toMessage</code> method has a more strict contract and must always convert <code>Message</code> to the wire format: <code>byte[]</code>.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>So, for all intents and purposes (and especially when implementing your own converter) you regard the two methods as having the following signatures:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">Object fromMessage(Message<?> message, Class<?> targetClass);
|
||
|
||
Message<byte[]> toMessage(Object payload, @Nullable MessageHeaders headers);</code></pre>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sect2">
|
||
<h3 id="_provided_messageconverters"><a class="link" href="#_provided_messageconverters">Provided MessageConverters</a></h3>
|
||
<div class="paragraph">
|
||
<p>As mentioned earlier, the framework already provides a stack of <code>MessageConverters</code> to handle most common use cases.
|
||
The following list describes the provided <code>MessageConverters</code>, in order of precedence (the first <code>MessageConverter</code> that works is used):</p>
|
||
</div>
|
||
<div class="olist arabic">
|
||
<ol class="arabic">
|
||
<li>
|
||
<p><code>ApplicationJsonMessageMarshallingConverter</code>: Variation of the <code>org.springframework.messaging.converter.MappingJackson2MessageConverter</code>. Supports conversion of the payload of the <code>Message</code> to/from POJO for cases when <code>contentType</code> is <code>application/json</code> (DEFAULT).</p>
|
||
</li>
|
||
<li>
|
||
<p><code>ByteArrayMessageConverter</code>: Supports conversion of the payload of the <code>Message</code> from <code>byte[]</code> to <code>byte[]</code> for cases when <code>contentType</code> is <code>application/octet-stream</code>. It is essentially a pass through and exists primarily for backward compatibility.</p>
|
||
</li>
|
||
<li>
|
||
<p><code>ObjectStringMessageConverter</code>: Supports conversion of any type to a <code>String</code> when <code>contentType</code> is <code>text/plain</code>.
|
||
It invokes Object’s <code>toString()</code> method or, if the payload is <code>byte[]</code>, a new <code>String(byte[])</code>.</p>
|
||
</li>
|
||
<li>
|
||
<p><code>JsonUnmarshallingConverter</code>: Similar to the <code>ApplicationJsonMessageMarshallingConverter</code>. It supports conversion of any type when <code>contentType</code> is <code>application/x-java-object</code>.
|
||
It expects the actual type information to be embedded in the <code>contentType</code> as an attribute (for example, <code>application/x-java-object;type=foo.bar.Cat</code>).</p>
|
||
</li>
|
||
</ol>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>When no appropriate converter is found, the framework throws an exception. When that happens, you should check your code and configuration and ensure you did not miss anything (that is, ensure that you provided a <code>contentType</code> by using a binding or a header).
|
||
However, most likely, you found some uncommon case (such as a custom <code>contentType</code> perhaps) and the current stack of provided <code>MessageConverters</code>
|
||
does not know how to convert. If that is the case, you can add custom <code>MessageConverter</code>. See <a href="#spring-cloud-stream-overview-user-defined-message-converters">User-defined Message Converters</a>.</p>
|
||
</div>
|
||
</div>
|
||
<div class="sect2">
|
||
<h3 id="spring-cloud-stream-overview-user-defined-message-converters"><a class="link" href="#spring-cloud-stream-overview-user-defined-message-converters">User-defined Message Converters</a></h3>
|
||
<div class="paragraph">
|
||
<p>Spring Cloud Stream exposes a mechanism to define and register additional <code>MessageConverters</code>.
|
||
To use it, implement <code>org.springframework.messaging.converter.MessageConverter</code>, configure it as a <code>@Bean</code>.
|
||
It is then appended to the existing stack of `MessageConverter`s.</p>
|
||
</div>
|
||
<div class="admonitionblock note">
|
||
<table>
|
||
<tr>
|
||
<td class="icon">
|
||
<i class="fa icon-note" title="Note"></i>
|
||
</td>
|
||
<td class="content">
|
||
It is important to understand that custom <code>MessageConverter</code> implementations are added to the head of the existing stack.
|
||
Consequently, custom <code>MessageConverter</code> implementations take precedence over the existing ones, which lets you override as well as add to the existing converters.
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The following example shows how to create a message converter bean to support a new content type called <code>application/bar</code>:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@SpringBootApplication
|
||
public static class SinkApplication {
|
||
|
||
...
|
||
|
||
@Bean
|
||
public MessageConverter customMessageConverter() {
|
||
return new MyCustomMessageConverter();
|
||
}
|
||
}
|
||
|
||
public class MyCustomMessageConverter extends AbstractMessageConverter {
|
||
|
||
public MyCustomMessageConverter() {
|
||
super(new MimeType("application", "bar"));
|
||
}
|
||
|
||
@Override
|
||
protected boolean supports(Class<?> clazz) {
|
||
return (Bar.class.equals(clazz));
|
||
}
|
||
|
||
@Override
|
||
protected Object convertFromInternal(Message<?> message, Class<?> targetClass, Object conversionHint) {
|
||
Object payload = message.getPayload();
|
||
return (payload instanceof Bar ? payload : new Bar((byte[]) payload));
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Spring Cloud Stream also provides support for Avro-based converters and schema evolution.
|
||
See <code><a href="#schema-evolution">[schema-evolution]</a></code> for details.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>[
|
||
== Inter-Application Communication</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Spring Cloud Stream enables communication between applications. Inter-application communication is a complex issue spanning several concerns, as described in the following topics:</p>
|
||
</div>
|
||
<div class="ulist">
|
||
<ul>
|
||
<li>
|
||
<p><code><a href="#spring-cloud-stream-overview-connecting-multiple-application-instances">Connecting Multiple Application Instances</a></code></p>
|
||
</li>
|
||
<li>
|
||
<p><code><a href="#spring-cloud-stream-overview-instance-index-instance-count">Instance Index and Instance Count</a></code></p>
|
||
</li>
|
||
<li>
|
||
<p><code><a href="#spring-cloud-stream-overview-partitioning">Partitioning</a></code></p>
|
||
</li>
|
||
</ul>
|
||
</div>
|
||
</div>
|
||
<div class="sect2">
|
||
<h3 id="spring-cloud-stream-overview-connecting-multiple-application-instances"><a class="link" href="#spring-cloud-stream-overview-connecting-multiple-application-instances">Connecting Multiple Application Instances</a></h3>
|
||
<div class="paragraph">
|
||
<p>While Spring Cloud Stream makes it easy for individual Spring Boot applications to connect to messaging systems, the typical scenario for Spring Cloud Stream is the creation of multi-application pipelines, where microservice applications send data to each other.
|
||
You can achieve this scenario by correlating the input and output destinations of “adjacent” applications.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Suppose a design calls for the Time Source application to send data to the Log Sink application. You could use a common destination named <code>ticktock</code> for bindings within both applications.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Time Source (that has the binding named <code>output</code>) would set the following property:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre>spring.cloud.stream.bindings.output.destination=ticktock</pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Log Sink (that has the binding named <code>input</code>) would set the following property:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre>spring.cloud.stream.bindings.input.destination=ticktock</pre>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sect2">
|
||
<h3 id="spring-cloud-stream-overview-instance-index-instance-count"><a class="link" href="#spring-cloud-stream-overview-instance-index-instance-count">Instance Index and Instance Count</a></h3>
|
||
<div class="paragraph">
|
||
<p>When scaling up Spring Cloud Stream applications, each instance can receive information about how many other instances of the same application exist and what its own instance index is.
|
||
Spring Cloud Stream does this through the <code>spring.cloud.stream.instanceCount</code> and <code>spring.cloud.stream.instanceIndex</code> properties.
|
||
For example, if there are three instances of a HDFS sink application, all three instances have <code>spring.cloud.stream.instanceCount</code> set to <code>3</code>, and the individual applications have <code>spring.cloud.stream.instanceIndex</code> set to <code>0</code>, <code>1</code>, and <code>2</code>, respectively.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>When Spring Cloud Stream applications are deployed through Spring Cloud Data Flow, these properties are configured automatically; when Spring Cloud Stream applications are launched independently, these properties must be set correctly.
|
||
By default, <code>spring.cloud.stream.instanceCount</code> is <code>1</code>, and <code>spring.cloud.stream.instanceIndex</code> is <code>0</code>.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>In a scaled-up scenario, correct configuration of these two properties is important for addressing partitioning behavior (see below) in general, and the two properties are always required by certain binders (for example, the Kafka binder) in order to ensure that data are split correctly across multiple consumer instances.</p>
|
||
</div>
|
||
</div>
|
||
<div class="sect2">
|
||
<h3 id="spring-cloud-stream-overview-partitioning"><a class="link" href="#spring-cloud-stream-overview-partitioning">Partitioning</a></h3>
|
||
<div class="paragraph">
|
||
<p>Partitioning in Spring Cloud Stream consists of two tasks:</p>
|
||
</div>
|
||
<div class="ulist">
|
||
<ul>
|
||
<li>
|
||
<p><code><a href="#spring-cloud-stream-overview-configuring-output-bindings-partitioning">Configuring Output Bindings for Partitioning</a></code></p>
|
||
</li>
|
||
<li>
|
||
<p><code><a href="#spring-cloud-stream-overview-configuring-input-bindings-partitioning">Configuring Input Bindings for Partitioning</a></code></p>
|
||
</li>
|
||
</ul>
|
||
</div>
|
||
<div class="sect3">
|
||
<h4 id="spring-cloud-stream-overview-configuring-output-bindings-partitioning"><a class="link" href="#spring-cloud-stream-overview-configuring-output-bindings-partitioning">Configuring Output Bindings for Partitioning</a></h4>
|
||
<div class="paragraph">
|
||
<p>You can configure an output binding to send partitioned data by setting one and only one of its <code>partitionKeyExpression</code> or <code>partitionKeyExtractorName</code> properties, as well as its <code>partitionCount</code> property.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>For example, the following is a valid and typical configuration:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre>spring.cloud.stream.bindings.func-out-0.producer.partitionKeyExpression=payload.id
|
||
spring.cloud.stream.bindings.func-out-0.producer.partitionCount=5</pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Based on that example configuration, data is sent to the target partition by using the following logic.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>A partition key’s value is calculated for each message sent to a partitioned output binding based on the <code>partitionKeyExpression</code>.
|
||
The <code>partitionKeyExpression</code> is a SpEL expression that is evaluated against the outbound message for extracting the partitioning key.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>If a SpEL expression is not sufficient for your needs, you can instead calculate the partition key value by providing an implementation of <code>org.springframework.cloud.stream.binder.PartitionKeyExtractorStrategy</code> and configuring it as a bean (by using the <code>@Bean</code> annotation).
|
||
If you have more then one bean of type <code>org.springframework.cloud.stream.binder.PartitionKeyExtractorStrategy</code> available in the Application Context, you can further filter it by specifying its name with the <code>partitionKeyExtractorName</code> property, as shown in the following example:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code>--spring.cloud.stream.bindings.func-out-0.producer.partitionKeyExtractorName=customPartitionKeyExtractor
|
||
--spring.cloud.stream.bindings.func-out-0.producer.partitionCount=5
|
||
. . .
|
||
@Bean
|
||
public CustomPartitionKeyExtractorClass customPartitionKeyExtractor() {
|
||
return new CustomPartitionKeyExtractorClass();
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="admonitionblock note">
|
||
<table>
|
||
<tr>
|
||
<td class="icon">
|
||
<i class="fa icon-note" title="Note"></i>
|
||
</td>
|
||
<td class="content">
|
||
In previous versions of Spring Cloud Stream, you could specify the implementation of <code>org.springframework.cloud.stream.binder.PartitionKeyExtractorStrategy</code> by setting the <code>spring.cloud.stream.bindings.output.producer.partitionKeyExtractorClass</code> property.
|
||
Since version 3.0, this property is removed.
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Once the message key is calculated, the partition selection process determines the target partition as a value between <code>0</code> and <code>partitionCount - 1</code>.
|
||
The default calculation, applicable in most scenarios, is based on the following formula: <code>key.hashCode() % partitionCount</code>.
|
||
This can be customized on the binding, either by setting a SpEL expression to be evaluated against the 'key' (through the <code>partitionSelectorExpression</code> property) or by configuring an implementation of <code>org.springframework.cloud.stream.binder.PartitionSelectorStrategy</code> as a bean (by using the @Bean annotation).
|
||
Similar to the <code>PartitionKeyExtractorStrategy</code>, you can further filter it by using the <code>spring.cloud.stream.bindings.output.producer.partitionSelectorName</code> property when more than one bean of this type is available in the Application Context, as shown in the following example:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code>--spring.cloud.stream.bindings.func-out-0.producer.partitionSelectorName=customPartitionSelector
|
||
. . .
|
||
@Bean
|
||
public CustomPartitionSelectorClass customPartitionSelector() {
|
||
return new CustomPartitionSelectorClass();
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="admonitionblock note">
|
||
<table>
|
||
<tr>
|
||
<td class="icon">
|
||
<i class="fa icon-note" title="Note"></i>
|
||
</td>
|
||
<td class="content">
|
||
In previous versions of Spring Cloud Stream you could specify the implementation of <code>org.springframework.cloud.stream.binder.PartitionSelectorStrategy</code> by setting the <code>spring.cloud.stream.bindings.output.producer.partitionSelectorClass</code> property.
|
||
Since version 3.0, this property is removed.
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
</div>
|
||
<div class="sect3">
|
||
<h4 id="spring-cloud-stream-overview-configuring-input-bindings-partitioning"><a class="link" href="#spring-cloud-stream-overview-configuring-input-bindings-partitioning">Configuring Input Bindings for Partitioning</a></h4>
|
||
<div class="paragraph">
|
||
<p>An input binding (with the binding name <code>uppercase-in-0</code>) is configured to receive partitioned data by setting its <code>partitioned</code>
|
||
property, as well as the <code>instanceIndex</code> and <code>instanceCount</code> properties on the application itself, as shown in the following example:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre>spring.cloud.stream.bindings.uppercase-in-0.consumer.partitioned=true
|
||
spring.cloud.stream.instanceIndex=3
|
||
spring.cloud.stream.instanceCount=5</pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The <code>instanceCount</code> value represents the total number of application instances between which the data should be partitioned.
|
||
The <code>instanceIndex</code> must be a unique value across the multiple instances, with a value between <code>0</code> and <code>instanceCount - 1</code>.
|
||
The instance index helps each application instance to identify the unique partition(s) from which it receives data.
|
||
It is required by binders using technology that does not support partitioning natively.
|
||
For example, with RabbitMQ, there is a queue for each partition, with the queue name containing the instance index.
|
||
With Kafka, if <code>autoRebalanceEnabled</code> is <code>true</code> (default), Kafka takes care of distributing partitions across instances, and these properties are not required.
|
||
If <code>autoRebalanceEnabled</code> is set to false, the <code>instanceCount</code> and <code>instanceIndex</code> are used by the binder to determine which partition(s) the instance subscribes to (you must have at least as many partitions as there are instances).
|
||
The binder allocates the partitions instead of Kafka.
|
||
This might be useful if you want messages for a particular partition to always go to the same instance.
|
||
When a binder configuration requires them, it is important to set both values correctly in order to ensure that all of the data is consumed and that the application instances receive mutually exclusive datasets.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>While a scenario in which using multiple instances for partitioned data processing may be complex to set up in a standalone case, Spring Cloud Dataflow can simplify the process significantly by populating both the input and output values correctly and by letting you rely on the runtime infrastructure to provide information about the instance index and instance count.</p>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sect1">
|
||
<h2 id="_testing"><a class="link" href="#_testing">Testing</a></h2>
|
||
<div class="sectionbody">
|
||
<div class="paragraph">
|
||
<p>Spring Cloud Stream provides support for testing your microservice applications without connecting to a messaging system.</p>
|
||
</div>
|
||
<div class="sect2">
|
||
<h3 id="spring_integration_test_binder"><a class="link" href="#spring_integration_test_binder">Spring Integration Test Binder</a></h3>
|
||
<div class="paragraph">
|
||
<p>The old test binder defined in <code>spring-cloud-stream-test-support</code> module was specifically designed to facilitate <em>unit testing</em> of the actual messaging components and thus bypasses some of the core functionality of the binder API.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>While such light-weight approach is sufficient for a lot of cases, it usually requires additional <em>integration testing</em> with real binders (e.g., Rabbit, Kafka etc). So we are effectively deprecating it.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>To begin bridging the gap between <em>unit</em> and <em>integration</em> testing we’ve developed a new test binder which uses <a href="https://spring.io/projects/spring-integration">Spring Integration</a> framework
|
||
as an in-JVM Message Broker essentially giving you the best of both worlds - a real binder without the networking.</p>
|
||
</div>
|
||
<div class="sect3">
|
||
<h4 id="_test_binder_configuration"><a class="link" href="#_test_binder_configuration">Test Binder configuration</a></h4>
|
||
<div class="paragraph">
|
||
<p>To enable Spring Integration Test Binder all you need is:</p>
|
||
</div>
|
||
<div class="ulist">
|
||
<ul>
|
||
<li>
|
||
<p>Add required dependencies</p>
|
||
</li>
|
||
<li>
|
||
<p>Remove the dependency for <code>spring-cloud-stream-test-support</code></p>
|
||
</li>
|
||
</ul>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p><strong><strong>Add required dependencies</strong></strong></p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Below is the example of the required Maven POM entries which could be easily retrofitted into Gradle.</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-xml hljs" data-lang="xml"><dependency>
|
||
<groupId>org.springframework.cloud</groupId>
|
||
<artifactId>spring-cloud-stream</artifactId>
|
||
<version>${spring.cloud.stream.version}</version>
|
||
<type>test-jar</type>
|
||
<scope>test</scope>
|
||
<classifier>test-binder</classifier>
|
||
</dependency></code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p><strong><strong>Remove the dependency for <code>spring-cloud-stream-test-support</code></strong></strong></p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>To avoid conflicts with the existing test binder you must remove the following entry</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-xml hljs" data-lang="xml"><dependency>
|
||
<groupId>org.springframework.cloud</groupId>
|
||
<artifactId>spring-cloud-stream-test-support</artifactId>
|
||
<scope>test</scope>
|
||
</dependency></code></pre>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sect3">
|
||
<h4 id="_test_binder_usage"><a class="link" href="#_test_binder_usage">Test Binder usage</a></h4>
|
||
<div class="paragraph">
|
||
<p>Now you can test your microservice as a simple unit test</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@SpringBootTest
|
||
@RunWith(SpringRunner.class)
|
||
public class SampleStreamTests {
|
||
|
||
@Autowired
|
||
private InputDestination input;
|
||
|
||
@Autowired
|
||
private OutputDestination output;
|
||
|
||
@Test
|
||
public void testEmptyConfiguration() {
|
||
this.input.send(new GenericMessage<byte[]>("hello".getBytes()));
|
||
assertThat(output.receive().getPayload()).isEqualTo("HELLO".getBytes());
|
||
}
|
||
|
||
@SpringBootApplication
|
||
@Import(TestChannelBinderConfiguration.class)
|
||
public static class SampleConfiguration {
|
||
@Bean
|
||
public Function<String, String> uppercase() {
|
||
return v -> v.toUpperCase();
|
||
}
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>And if you need more control or want to test several configurations in the same test suite
|
||
you can also do the following:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@EnableAutoConfiguration
|
||
public static class MyTestConfiguration {
|
||
@Bean
|
||
public Function<String, String> uppercase() {
|
||
return v -> v.toUpperCase();
|
||
}
|
||
}
|
||
|
||
. . .
|
||
|
||
@Test
|
||
public void sampleTest() {
|
||
try (ConfigurableApplicationContext context = new SpringApplicationBuilder(
|
||
TestChannelBinderConfiguration.getCompleteConfiguration(
|
||
MyTestConfiguration.class))
|
||
.run("--spring.cloud.function.definition=uppercase")) {
|
||
InputDestination source = context.getBean(InputDestination.class);
|
||
OutputDestination target = context.getBean(OutputDestination.class);
|
||
source.send(new GenericMessage<byte[]>("hello".getBytes()));
|
||
assertThat(target.receive().getPayload()).isEqualTo("HELLO".getBytes());
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>For cases where you have multiple bindings and/or multiple inputs and outputs, or simply want to be explicit about names of
|
||
the destination you are sending to or receiving from, the <code>send()</code> and <code>receive()</code>
|
||
methods of <code>InputDestination</code> and <code>OutputDestination</code> are overridden to allow you to provide the name of the input and output destination.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Consider the following sample:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@EnableAutoConfiguration
|
||
public static class SampleFunctionConfiguration {
|
||
|
||
@Bean
|
||
public Function<String, String> uppercase() {
|
||
return value -> value.toUpperCase();
|
||
}
|
||
|
||
@Bean
|
||
public Function<String, String> reverse() {
|
||
return value -> new StringBuilder(value).reverse().toString();
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>and the actual test</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@Test
|
||
public void testMultipleFunctions() {
|
||
try (ConfigurableApplicationContext context = new SpringApplicationBuilder(
|
||
TestChannelBinderConfiguration.getCompleteConfiguration(
|
||
SampleFunctionConfiguration.class))
|
||
.run("--spring.cloud.function.definition=uppercase;reverse")) {
|
||
|
||
InputDestination inputDestination = context.getBean(InputDestination.class);
|
||
OutputDestination outputDestination = context.getBean(OutputDestination.class);
|
||
|
||
Message<byte[]> inputMessage = MessageBuilder.withPayload("Hello".getBytes()).build();
|
||
inputDestination.send(inputMessage, "uppercase-in-0");
|
||
inputDestination.send(inputMessage, "uppercase-in-0");
|
||
|
||
Message<byte[]> outputMessage = outputDestination.receive(0, "uppercase-out-0");
|
||
assertThat(outputMessage.getPayload()).isEqualTo("HELLO".getBytes());
|
||
|
||
outputMessage = outputDestination.receive(0, "uppercase-out-0");
|
||
assertThat(outputMessage.getPayload()).isEqualTo("olleH".getBytes());
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>For cases where you have additional mapping properties such as <code>destination</code> you should use those names. For example, consider a different version of the
|
||
preceding test where we explicitly map inputs and outputs of the <code>uppercase</code> function to <code>myInput</code> and <code>myOutput</code> binding names:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@Test
|
||
public void testMultipleFunctions() {
|
||
try (ConfigurableApplicationContext context = new SpringApplicationBuilder(
|
||
TestChannelBinderConfiguration.getCompleteConfiguration(
|
||
SampleFunctionConfiguration.class))
|
||
.run(
|
||
"--spring.cloud.function.definition=uppercase;reverse",
|
||
"--spring.cloud.stream.bindings.uppercase-in-0.destination=myInput",
|
||
"--spring.cloud.stream.bindings.uppercase-out-0.destination=myOutput",
|
||
)) {
|
||
|
||
InputDestination inputDestination = context.getBean(InputDestination.class);
|
||
OutputDestination outputDestination = context.getBean(OutputDestination.class);
|
||
|
||
Message<byte[]> inputMessage = MessageBuilder.withPayload("Hello".getBytes()).build();
|
||
inputDestination.send(inputMessage, "myInput");
|
||
inputDestination.send(inputMessage, "myInput");
|
||
|
||
Message<byte[]> outputMessage = outputDestination.receive(0, "myOutput");
|
||
assertThat(outputMessage.getPayload()).isEqualTo("HELLO".getBytes());
|
||
|
||
outputMessage = outputDestination.receive(0, "myOutput");
|
||
assertThat(outputMessage.getPayload()).isEqualTo("olleH".getBytes());
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>You can also use this binder with legacy annotation-based configuration:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@SpringBootApplication
|
||
@EnableBinding(Processor.class)
|
||
public class LegacyStreamApplication {
|
||
|
||
@StreamListener(Processor.INPUT)
|
||
@SendTo(Processor.OUTPUT)
|
||
public String echo(String value) {
|
||
return value;
|
||
}
|
||
}
|
||
|
||
. . .
|
||
|
||
@Test
|
||
public void sampleTest() {
|
||
try (ConfigurableApplicationContext context = new SpringApplicationBuilder(
|
||
TestChannelBinderConfiguration.getCompleteConfiguration(
|
||
LegacyStreamApplication.class)).run()) {
|
||
InputDestination source = context.getBean(InputDestination.class);
|
||
OutputDestination target = context.getBean(OutputDestination.class);
|
||
source.send(new GenericMessage<byte[]>("hello".getBytes()));
|
||
assertThat(target.receive().getPayload()).isEqualTo("hello".getBytes());
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>In the above you simply create an ApplicationContext with your configuration (your application) while additionally supplying <code>TestChannelBinderConfiguration</code>
|
||
provided by the framework. Then you access <code>InputDestination</code> and <code>OutputDestination</code> beans to send/receive messages. In the context of this binder
|
||
<code>InputDestination</code> and <code>OutputDestination</code> emulate remote destinations such as Rabbit <em>exchange/queue</em> or Kafka <em>topic</em>.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>In the future we plan to simplify the API.</p>
|
||
</div>
|
||
<div class="admonitionblock note">
|
||
<table>
|
||
<tr>
|
||
<td class="icon">
|
||
<i class="fa icon-note" title="Note"></i>
|
||
</td>
|
||
<td class="content">
|
||
In its current state Spring Integration Test Binder only supports the three bindings provided by the framework (Source, Processor, Sink) specifically to promote
|
||
light-weight microservices architectures rather then general purpose messaging applications.
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
</div>
|
||
<div class="sect3">
|
||
<h4 id="_test_binder_and_pollablemessagesource"><a class="link" href="#_test_binder_and_pollablemessagesource">Test Binder and PollableMessageSource</a></h4>
|
||
<div class="paragraph">
|
||
<p>Spring Integration Test Binder also allows you to write tests when working with <code>PollableMessageSource</code> (see <a href="#spring-cloud-streams-overview-using-polled-consumers">Using Polled Consumers</a> for more details).</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The important thing that needs to be understood though is that polling is not event-driven, and that <code>PollableMessageSource</code> is a strategy which exposes operation to produce (poll for) a Message (singular).
|
||
How often you poll or how many threads you use or where you’re polling from (message queue or file system) is entirely up to you;
|
||
In other words it is your responsibility to configure Poller or Threads or the actual source of Message. Luckily Spring has plenty of abstractions to configure exactly that.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Let’s look at the example:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@Test
|
||
public void samplePollingTest() {
|
||
ApplicationContext context = new SpringApplicationBuilder(SamplePolledConfiguration.class)
|
||
.web(WebApplicationType.NONE)
|
||
.run("--spring.jmx.enabled=false");
|
||
OutputDestination destination = context.getBean(OutputDestination.class);
|
||
System.out.println("Message 1: " + new String(destination.receive().getPayload()));
|
||
System.out.println("Message 2: " + new String(destination.receive().getPayload()));
|
||
System.out.println("Message 3: " + new String(destination.receive().getPayload()));
|
||
}
|
||
|
||
@EnableBinding(SamplePolledConfiguration.PolledConsumer.class)
|
||
@Import(TestChannelBinderConfiguration.class)
|
||
@EnableAutoConfiguration
|
||
public static class SamplePolledConfiguration {
|
||
@Bean
|
||
public ApplicationRunner poller(PollableMessageSource polledMessageSource, MessageChannel output, TaskExecutor taskScheduler) {
|
||
return args -> {
|
||
taskScheduler.execute(() -> {
|
||
for (int i = 0; i < 3; i++) {
|
||
try {
|
||
if (!polledMessageSource.poll(m -> {
|
||
String newPayload = ((String) m.getPayload()).toUpperCase();
|
||
output.send(new GenericMessage<>(newPayload));
|
||
})) {
|
||
Thread.sleep(2000);
|
||
}
|
||
}
|
||
catch (Exception e) {
|
||
// handle failure
|
||
}
|
||
}
|
||
});
|
||
};
|
||
}
|
||
|
||
public static interface PolledConsumer extends Source {
|
||
@Input
|
||
PollableMessageSource pollableSource();
|
||
}
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>The above (very rudimentary) example will produce 3 messages in 2 second intervals sending them to the output destination of <code>Source</code>
|
||
which this binder sends to <code>OutputDestination</code> where we retrieve them (for any assertions).
|
||
Currently it prints the following:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-text hljs" data-lang="text">Message 1: POLLED DATA
|
||
Message 2: POLLED DATA
|
||
Message 3: POLLED DATA</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>As you can see the data is the same. That is because this binder defines a default implementation of the actual <code>MessageSource</code> - the source
|
||
from which the Messages are polled using <code>poll()</code> operation. While sufficient for most testing scenarios, there are cases where you may want
|
||
to define your own <code>MessageSource</code>. To do so simply configure a bean of type <code>MessageSource</code> in your test configuration providing your own
|
||
implementation of Message sourcing.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Here is the example:</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@Bean
|
||
public MessageSource<?> source() {
|
||
return () -> new GenericMessage<>("My Own Data " + UUID.randomUUID());
|
||
}</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>rendering the following output;</p>
|
||
</div>
|
||
<div class="listingblock">
|
||
<div class="content">
|
||
<pre class="highlightjs highlight"><code class="language-text hljs" data-lang="text">Message 1: MY OWN DATA 1C180A91-E79F-494F-ABF4-BA3F993710DA
|
||
Message 2: MY OWN DATA D8F3A477-5547-41B4-9434-E69DA7616FEE
|
||
Message 3: MY OWN DATA 20BF2E64-7FF4-4CB6-A823-4053D30B5C74</code></pre>
|
||
</div>
|
||
</div>
|
||
<div class="admonitionblock note">
|
||
<table>
|
||
<tr>
|
||
<td class="icon">
|
||
<i class="fa icon-note" title="Note"></i>
|
||
</td>
|
||
<td class="content">
|
||
DO NOT name this bean <code>messageSource</code> as it is going to be in conflict with the bean of the same name (different type)
|
||
provided by Spring Boot for unrelated reasons.
|
||
</td>
|
||
</tr>
|
||
</table>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sect1">
|
||
<h2 id="_health_indicator"><a class="link" href="#_health_indicator">Health Indicator</a></h2>
|
||
<div class="sectionbody">
|
||
<div class="paragraph">
|
||
<p>Spring Cloud Stream provides a health indicator for binders.
|
||
It is registered under the name <code>binders</code> and can be enabled or disabled by setting the <code>management.health.binders.enabled</code> property.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>To enable health check you first need to enable both "web" and "actuator" by including its dependencies (see <a href="#binding_visualization_control">Binding visualization and control</a>)</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>If <code>management.health.binders.enabled</code> is not set explicitly by the application, then <code>management.health.defaults.enabled</code> is matched as <code>true</code> and the binder health indicators are enabled.
|
||
If you want to disable health indicator completely, then you have to set <code>management.health.binders.enabled</code> to <code>false</code>.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>You can use Spring Boot actuator health endpoint to access the health indicator - <code>/actuator/health</code>.
|
||
By default, you will only receive the top level application status when you hit the above endpoint.
|
||
In order to receive the full details from the binder specific health indicators, you need to include the property <code>management.endpoint.health.show-details</code> with the value <code>ALWAYS</code> in your application.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Health indicators are binder-specific and certain binder implementations may not necessarily provide a health indicator.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>If you want to completely disable all health indicators available out of the box and instead provide your own health indicators,
|
||
you can do so by setting property <code>management.health.binders.enabled</code> to <code>false</code> and then provide your own <code>HealthIndicator</code> beans in your application.
|
||
In this case, the health indicator infrastructure from Spring Boot will still pick up these custom beans.
|
||
Even if you are not disabling the binder health indicators, you can still enhance the health checks by providing your own <code>HealthIndicator</code> beans in addition to the out of the box health checks.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>When you have multiple binders in the same application, health indicators are enabled by default unless the application turns them off by setting <code>management.health.binders.enabled</code> to <code>false</code>.
|
||
In this case, if the user wants to disable health check for a subset of the binders, then that should be done by setting <code>management.health.binders.enabled</code> to <code>false</code> in the multi binder configurations’s environment.
|
||
See <a href="#multiple-systems">Connecting to Multiple Systems</a> for details on how environment specific properties can be provided.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>If there are multiple binders present in the classpath but not all of them are used in the application, this may cause some issues in the context of health indicators.
|
||
There may be implementation specific details as to how the health checks are performed. For example, a Kafka binder may decide the status as <code>DOWN</code> if there are no destinations registered by the binder.
|
||
For this reason, if you include a binder in the classpath, it is advised to use that binder by providing at least one binding (for E.g. through <code>EnableBinding</code>).
|
||
If you don’t have any bindings to provide for this binder, then that is an indication that you don’t need to include that binder in the classpath.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>Lets take a concrete situation. Imagine you have both Kafka and Kafka Streams binders present in the classpath, but only use the Kafka Streams binder in the application code, i.e. only provide bindings using the Kafka Streams binder.
|
||
Since Kafka binder is not used and it has specific checks to see if any destinations are registered, the binder health heck will fail.
|
||
The top level application health check status will be reported as <code>DOWN</code>.
|
||
In this situation, you can simply remove the dependency for kafka binder from your application since you are not using it.</p>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sect1">
|
||
<h2 id="_samples"><a class="link" href="#_samples">Samples</a></h2>
|
||
<div class="sectionbody">
|
||
<div class="paragraph">
|
||
<p>For Spring Cloud Stream samples, see the <a href="https://github.com/spring-cloud/spring-cloud-stream-samples">spring-cloud-stream-samples</a> repository on GitHub.</p>
|
||
</div>
|
||
<div class="sect2">
|
||
<h3 id="_deploying_stream_applications_on_cloudfoundry"><a class="link" href="#_deploying_stream_applications_on_cloudfoundry">Deploying Stream Applications on CloudFoundry</a></h3>
|
||
<div class="paragraph">
|
||
<p>On CloudFoundry, services are usually exposed through a special environment variable called <a href="https://docs.cloudfoundry.org/devguide/deploy-apps/environment-variable.html#VCAP-SERVICES">VCAP_SERVICES</a>.</p>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>When configuring your binder connections, you can use the values from an environment variable as explained on the <a href="http://docs.spring.io/spring-cloud-dataflow-server-cloudfoundry/docs/current-SNAPSHOT/reference/htmlsingle/#getting-started-ups">dataflow Cloud Foundry Server</a> docs.</p>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sect1">
|
||
<h2 id="_binder_implementations"><a class="link" href="#_binder_implementations">Binder Implementations</a></h2>
|
||
<div class="sectionbody">
|
||
<div class="paragraph">
|
||
<p>The following is the list of available binder implementations</p>
|
||
</div>
|
||
<div class="ulist">
|
||
<ul>
|
||
<li>
|
||
<p><a href="https://cloud.spring.io/spring-cloud-stream-binder-rabbit/">RabbitMQ</a></p>
|
||
</li>
|
||
<li>
|
||
<p><a href="https://cloud.spring.io/spring-cloud-stream-binder-kafka/">Apache Kafka</a></p>
|
||
</li>
|
||
<li>
|
||
<p><a href="https://github.com/spring-cloud/spring-cloud-stream-binder-aws-kinesis">Amazon Kinesis</a></p>
|
||
</li>
|
||
<li>
|
||
<p><a href="https://github.com/spring-cloud/spring-cloud-gcp/tree/master/spring-cloud-gcp-pubsub-stream-binder">Google PubSub <em>(partner maintained)</em></a></p>
|
||
</li>
|
||
<li>
|
||
<p><a href="https://github.com/SolaceProducts/spring-cloud-stream-binder-solace">Solace PubSub+ <em>(partner maintained)</em></a></p>
|
||
</li>
|
||
<li>
|
||
<p><a href="https://github.com/Microsoft/spring-cloud-azure/tree/master/spring-cloud-azure-stream-binder/spring-cloud-azure-eventhubs-stream-binder">Azure Event Hubs <em>(partner maintained)</em></a></p>
|
||
</li>
|
||
<li>
|
||
<p><a href="https://github.com/alibaba/spring-cloud-alibaba/wiki/RocketMQ-en">Apache RocketMQ <em>(partner maintained)</em></a></p>
|
||
</li>
|
||
</ul>
|
||
</div>
|
||
<div class="paragraph">
|
||
<p>As it was mentioned earlier Binder abstraction is also one of the extension points of the framework. So if you can’t find a suitable binder in the preceding list you can implement your own binder on top of Spring Cloud Stream.
|
||
In the <a href="https://medium.com/@domenicosibilio/how-to-create-a-spring-cloud-stream-binder-from-scratch-ab8b29ee931b">How to create a Spring Cloud Stream Binder from scratch</a> post a community member documents
|
||
in details, with an example, a set of steps necessary to implement a custom binder.
|
||
The steps are also highlighted in the <code><a href="#spring-cloud-stream-overview-custom-binder-impl">Implementing Custom Binders</a></code> section.</p>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<script type="text/javascript" src="js/tocbot/tocbot.min.js"></script>
|
||
<script type="text/javascript" src="js/toc.js"></script>
|
||
<link rel="stylesheet" href="js/highlight/styles/atom-one-dark-reasonable.min.css">
|
||
<script src="js/highlight/highlight.min.js"></script>
|
||
<script>hljs.initHighlighting()</script>
|
||
</body>
|
||
</html> |