907 lines
46 KiB
Plaintext
907 lines
46 KiB
Plaintext
= Spring Cloud Function
|
||
|
||
Mark Fisher, Dave Syer, Oleg Zhurakousky, Anshul Mehra, Dan Dobrin
|
||
|
||
*{project-version}*
|
||
|
||
---
|
||
|
||
:github: https://github.com/spring-cloud/spring-cloud-function
|
||
:githubmaster: {github}/tree/master
|
||
:docslink: {githubmaster}/docs/src/main/asciidoc
|
||
:nofooter:
|
||
:branch: master
|
||
|
||
== Introduction
|
||
|
||
include::_intro.adoc[]
|
||
|
||
include::https://raw.githubusercontent.com/spring-cloud/spring-cloud-build/master/docs/src/main/asciidoc/contributing-docs.adoc[]
|
||
|
||
== Getting Started
|
||
|
||
include::getting-started.adoc[]
|
||
|
||
== Programming model
|
||
|
||
|
||
=== Function Catalog and Flexible Function Signatures
|
||
|
||
One of the main features of Spring Cloud Function is to adapt and support a range of type signatures for user-defined functions,
|
||
while providing a consistent execution model.
|
||
That's why all user defined functions are transformed into a canonical representation by `FunctionCatalog`.
|
||
|
||
While users don't normally have to care about the `FunctionCatalog` at all, it is useful to know what
|
||
kind of functions are supported in user code.
|
||
|
||
It is also important to understand that Spring Cloud Function provides first class support for reactive API
|
||
provided by https://projectreactor.io/[Project Reactor] allowing reactive primitives such as `Mono` and `Flux`
|
||
to be used as types in user defined functions providing greater flexibility when choosing programming model for
|
||
your function implementation.
|
||
Reactive programming model also enables functional support for features that would be otherwise difficult to impossible to implement
|
||
using imperative programming style. For more on this please read <<Function Arity>> section.
|
||
|
||
=== Java 8 function support
|
||
|
||
Spring Cloud Function embraces and builds on top of the 3 core functional interfaces defined by Java
|
||
and available to us since Java 8.
|
||
|
||
- Supplier<O>
|
||
- Function<I, O>
|
||
- Consumer<I>
|
||
|
||
==== Supplier
|
||
Supplier can be _reactive_ - `Supplier<Flux<T>>`
|
||
or _imperative_ - `Supplier<T>`. From the invocation standpoint this should make no difference
|
||
to the implementor of such Supplier. However, when used within frameworks
|
||
(e.g., https://spring.io/projects/spring-cloud-stream[Spring Cloud Stream]), Suppliers, especially reactive,
|
||
often used to represent the source of the stream, therefore they are invoked once to get the stream (e.g., Flux)
|
||
to which consumers can subscribe to. In other words such suppliers represent an equivalent of an _infinite stream_.
|
||
However, the same reactive suppliers can also represent _finite_ stream(s) (e.g., result set on the polled JDBC data).
|
||
In those cases such reactive suppliers must be hooked up to some polling mechanism of the underlying framework.
|
||
|
||
To assist with that Spring Cloud Function provides a marker annotation
|
||
`org.springframework.cloud.function.context.PollableSupplier` to signal that such supplier produces a
|
||
finite stream and may need to be polled again. That said, it is important to understand that Spring Cloud Function itself
|
||
provides no behavior for this annotation.
|
||
|
||
In addition `PollableSupplier` annotation exposes a _splittable_ attribute to signal that produced stream
|
||
needs to be split (see https://www.enterpriseintegrationpatterns.com/patterns/messaging/Sequencer.html[Splitter EIP])
|
||
|
||
Here is the example:
|
||
|
||
[source, java]
|
||
----
|
||
@PollableSupplier(splittable = true)
|
||
public Supplier<Flux<String>> someSupplier() {
|
||
return () -> {
|
||
String v1 = String.valueOf(System.nanoTime());
|
||
String v2 = String.valueOf(System.nanoTime());
|
||
String v3 = String.valueOf(System.nanoTime());
|
||
return Flux.just(v1, v2, v3);
|
||
};
|
||
}
|
||
----
|
||
|
||
==== Function
|
||
Function can also be written in imperative or reactive way, yet unlike Supplier and Consumer there are
|
||
no special considerations for the implementor other then understanding that when used within frameworks
|
||
such as https://spring.io/projects/spring-cloud-stream[Spring Cloud Stream] and others, reactive function is
|
||
invoked only once to pass a reference to the stream (Flux or Mono) and imperative is invoked once per event.
|
||
|
||
==== Consumer
|
||
Consumer is a little bit special because it has a `void` return type,
|
||
which implies blocking, at least potentially. Most likely you will not
|
||
need to write `Consumer<Flux<?>>`, but if you do need to do that,
|
||
remember to subscribe to the input flux.
|
||
|
||
=== Function Composition
|
||
|
||
Function Composition is a feature that allows one to compose several functions into one.
|
||
The core support is based on function composition feature available with https://docs.oracle.com/javase/8/docs/api/java/util/function/Function.html#andThen-java.util.function.Function-[Function.andThen(..)]
|
||
support available since Java 8. However on top of it, we provide few additional features.
|
||
|
||
==== Declarative Function Composition
|
||
|
||
This feature allows you to provide composition instruction in a declarative way using `|` (pipe) or `,` (comma) delimiter
|
||
when providing `spring.cloud.function.definition` property.
|
||
|
||
For example
|
||
----
|
||
--spring.cloud.function.definition=uppercase|reverse
|
||
----
|
||
Here we effectively provided a definition of a single function which itself is a composition of
|
||
function `uppercase` and function `reverse`. In fact that is one of the reasons why the property name is _definition_ and not _name_,
|
||
since the definition of a function can be a composition of several named functions.
|
||
And as mentioned you can use `,` instead of pipe (such as `...definition=uppercase,reverse`).
|
||
|
||
==== Composing non-Functions
|
||
Spring Cloud Function also supports composing Supplier with `Consumer` or `Function` as well as `Function` with `Consumer`.
|
||
What's important here is to understand the end product of such definitions.
|
||
Composing Supplier with Function still results in Supplier while composing Supplier with Consumer will effectively render Runnable.
|
||
Following the same logic composing Function with Consumer will result in Consumer.
|
||
|
||
And of course you can't compose uncomposable such as Consumer and Function, Consumer and Supplier etc.
|
||
|
||
|
||
=== Function Routing and Filtering
|
||
|
||
Since version 2.2 Spring Cloud Function provides routing feature allowing
|
||
you to invoke a single function which acts as a router to an actual function you wish to invoke
|
||
This feature is very useful in certain FAAS environments where maintaining configurations
|
||
for several functions could be cumbersome or exposing more then one function is not possible.
|
||
|
||
The `RoutingFunction` is registered in _FunctionCatalog_ under the name `functionRouter`. For simplicity
|
||
and consistency you can also refer to `RoutingFunction.FUNCTION_NAME` constant.
|
||
|
||
This function has the following signature:
|
||
|
||
[source, java]
|
||
----
|
||
public class RoutingFunction implements Function<Object, Object> {
|
||
. . .
|
||
}
|
||
----
|
||
The routing instructions could be communicated in several ways. We support providing instructions via Message headers, System
|
||
properties as well as pluggable strategy. So let's look at some of the details
|
||
|
||
==== MessageRoutingCallback
|
||
|
||
The `MessageRoutingCallback` is a strategy to assist with determining the name of the route-to function definition.
|
||
|
||
[source, java]
|
||
----
|
||
public interface MessageRoutingCallback {
|
||
FunctionRoutingResult routingResult(Message<?> message);
|
||
. . .
|
||
}
|
||
----
|
||
|
||
All you need to do is implement and register it as a bean to be picked up by the `RoutingFunction`.
|
||
For example:
|
||
|
||
[source, java]
|
||
----
|
||
@Bean
|
||
public MessageRoutingCallback customRouter() {
|
||
return new MessageRoutingCallback() {
|
||
@Override
|
||
FunctionRoutingResult routingResult(Message<?> message) {
|
||
return new FunctionRoutingResult((String) message.getHeaders().get("func_name"));
|
||
}
|
||
};
|
||
}
|
||
----
|
||
|
||
In the preceding example you can see a very simple implementation of `MessageRoutingCallback` which determines the function definition from
|
||
`func_name` Message header of the incoming Message and returns the instance of `FunctionRoutingResult` containing the definition of function to invoke.
|
||
|
||
Additionally, the `FunctionRoutingResult` provides another constructor allowing you to provide an instance of `Message` as second argument to be used down stream.
|
||
This is primarily for runtime optimizations. To better understand this case let's look at the following scenario.
|
||
You need to route based on the payoload type. However, an input Message typically comes in as let's say JSON payload (as `byte[]`) . In order
|
||
to determine the route-to function definition you need to first process such JSON and potentially create an instance of the target type.
|
||
Once that determination is done you can pass it to `RoutingFunction` which still has a reference to the original Message with un-processed payload
|
||
This means that somewhere downstream, type conversion/transformation would need to be repeated.
|
||
|
||
Allowing you to create a new `Message` with converted payload as part of the `FunctionRoutingResult` will instruct `RoutingFunction` to use such `Message`
|
||
downstream. So effectively you letting the framework to benefit from the work you already did.
|
||
|
||
|
||
*Message Headers*
|
||
|
||
If the input argument is of type `Message<?>`, you can communicate routing instruction by setting one of
|
||
`spring.cloud.function.definition` or `spring.cloud.function.routing-expression` Message headers.
|
||
For more static cases you can use `spring.cloud.function.definition` header which allows you to provide
|
||
the name of a single function (e.g., `...definition=foo`) or a composition instruction (e.g., `...definition=foo|bar|baz`).
|
||
For more dynamic cases you can use `spring.cloud.function.routing-expression` header which allows
|
||
you to use Spring Expression Language (SpEL) and provide SpEL expression that should resolve
|
||
into definition of a function (as described above).
|
||
|
||
NOTE: SpEL evaluation context's root object is the
|
||
actual input argument, so in the case of `Message<?>` you can construct expression that has access
|
||
to both `payload` and `headers` (e.g., `spring.cloud.function.routing-expression=headers.function_name`).
|
||
|
||
In specific execution environments/models the adapters are responsible to translate and communicate
|
||
`spring.cloud.function.definition` and/or `spring.cloud.function.routing-expression` via Message header.
|
||
For example, when using _spring-cloud-function-web_ you can provide `spring.cloud.function.definition` as an HTTP
|
||
header and the framework will propagate it as well as other HTTP headers as Message headers.
|
||
|
||
*Application Properties*
|
||
|
||
Routing instruction can also be communicated via `spring.cloud.function.definition`
|
||
or `spring.cloud.function.routing-expression` as application properties. The rules described in the
|
||
previous section apply here as well. The only difference is you provide these instructions as
|
||
application properties (e.g., `--spring.cloud.function.definition=foo`).
|
||
|
||
NOTE: It is important to understand that providing `spring.cloud.function.definition`
|
||
or `spring.cloud.function.routing-expression` as Message headers will only work for imperative functions (e.g., `Function<Foo, Bar>`).
|
||
That is to say that we can _only_ route ***per-message*** with imperative functions. With reactive functions we can not route
|
||
***per-message***. Therefore you can only provide your routing instructions as Application Properties.
|
||
It's all about unit-of-work. In imperative function unit of work is Message so we can route based on such unit-of-work.
|
||
With reactive function unit-of-work is the entire stream, so we'll act only on the instruction provided via application
|
||
properties and route the entire stream.
|
||
|
||
*Order of priority for routing instructions*
|
||
|
||
Given that we have several mechanisms of providing routing instructions it is important to understand the priorities for
|
||
conflict resolutions in the event multiple mechanisms are used at the same time, so here is the order:
|
||
|
||
1. `MessageRoutingCallback` (If function is imperative will take over regardless if anything else is defined)
|
||
2. Message Headers (If function is imperative and no `MessageRoutingCallback` provided)
|
||
3. Application Properties (Any function)
|
||
|
||
|
||
==== Function Filtering
|
||
Filtering is the type of routing where there are only two paths - 'go' or 'discard'. In terms of functions it mean
|
||
you only want to invoke a certain function if some condition returns 'true', otherwise you want to discard input.
|
||
However, when it comes to discarding input there are many interpretation of what it could mean in the context of your application.
|
||
For example, you may want to log it, or you may want to maintain the counter of discarded messages. you may also want to do nothing at all.
|
||
Because of these different paths, we do not provide a general configuration option for how to deal with discarded messages.
|
||
Instead we simply recommend to define a simple Consumer which would signify the 'discard' path:
|
||
|
||
[source, java]
|
||
----
|
||
@Bean
|
||
public Consumer<?> devNull() {
|
||
// log, count or whatever
|
||
}
|
||
----
|
||
Now you can have routing expression that really only has two paths effectively becoming a filter. For example:
|
||
|
||
[source, text]
|
||
----
|
||
--spring.cloud.function.routing-expression=headers.contentType.toString().equals('text/plain') ? 'echo' : 'devNull'
|
||
----
|
||
Every message that does not fit criteria to go to 'echo' function will go to 'devNull' where you can simply do nothing with it.
|
||
The signature `Consumer<?>` will also ensure that no type conversion will be attempted resulting in almost no execution overhead.
|
||
|
||
|
||
IMPORTANT: When dealing with reactive inputs (e.g., Publisher), routing instructions must only be provided via Function properties. This is
|
||
due to the nature of the reactive functions which are invoked only once to pass a Publisher and the rest
|
||
is handled by the reactor, hence we can not access and/or rely on the routing instructions communicated via individual
|
||
values (e.g., Message).
|
||
|
||
==== Multiple Routers
|
||
|
||
By default the framework will always have a single routing function configured as described in previous sections. However, there are times when you may need more then one routing function.
|
||
In that case you can create your own instance of the `RoutingFunction` bean in addition to the existing one as long as you give it a name other than `functionRouter`.
|
||
|
||
You can pass `spring.cloud.function.routing-expression` or `spring.cloud.function.definition` to RoutinFunction as key/value pairs in the map.
|
||
|
||
Here is a simple example
|
||
|
||
----
|
||
@Configuration
|
||
protected static class MultipleRouterConfiguration {
|
||
|
||
@Bean
|
||
RoutingFunction mySpecialRouter(FunctionCatalog functionCatalog, BeanFactory beanFactory, @Nullable MessageRoutingCallback routingCallback) {
|
||
Map<String, String> propertiesMap = new HashMap<>();
|
||
propertiesMap.put(FunctionProperties.PREFIX + ".routing-expression", "'reverse'");
|
||
return new RoutingFunction(functionCatalog, propertiesMap, new BeanFactoryResolver(beanFactory), routingCallback);
|
||
}
|
||
|
||
@Bean
|
||
public Function<String, String> reverse() {
|
||
return v -> new StringBuilder(v).reverse().toString();
|
||
}
|
||
|
||
@Bean
|
||
public Function<String, String> uppercase() {
|
||
return String::toUpperCase;
|
||
}
|
||
}
|
||
----
|
||
|
||
and a test that demonstrates how it works
|
||
|
||
`
|
||
----
|
||
@Test
|
||
public void testMultipleRouters() {
|
||
System.setProperty(FunctionProperties.PREFIX + ".routing-expression", "'uppercase'");
|
||
FunctionCatalog functionCatalog = this.configureCatalog(MultipleRouterConfiguration.class);
|
||
Function function = functionCatalog.lookup(RoutingFunction.FUNCTION_NAME);
|
||
assertThat(function).isNotNull();
|
||
Message<String> message = MessageBuilder.withPayload("hello").build();
|
||
assertThat(function.apply(message)).isEqualTo("HELLO");
|
||
|
||
function = functionCatalog.lookup("mySpecialRouter");
|
||
assertThat(function).isNotNull();
|
||
message = MessageBuilder.withPayload("hello").build();
|
||
assertThat(function.apply(message)).isEqualTo("olleh");
|
||
}
|
||
----
|
||
|
||
=== Input/Output Enrichment
|
||
|
||
There are often times when you need to modify or refine an incoming or outgoing Message and to keep your code clean of non-functional concerns. You don’t want to do it inside of your business logic.
|
||
|
||
You can always accomplish it via <<Function Composition>>. Such approach provides several benefits:
|
||
|
||
- It allows you to isolate this non-functional concern into a separate function which you can compose with the business function as function definition.
|
||
- It provides you with complete freedom (and danger) as to what you can modify before incoming message reaches the actual business function.
|
||
|
||
[source, java]
|
||
----
|
||
@Bean
|
||
public Function<Message<?>, Message<?>> enrich() {
|
||
return message -> MessageBuilder.fromMessage(message).setHeader("foo", "bar").build();
|
||
}
|
||
|
||
@Bean
|
||
public Function<Message<?>, Message<?>> myBusinessFunction() {
|
||
// do whatever
|
||
}
|
||
----
|
||
|
||
And then compose your function by providing the following function definition `enrich|myBusinessFunction`.
|
||
|
||
While the described approach is the most flexible, it is also the most involved as it requires you to write some code, make it a bean or
|
||
manually register it as a function before you can compose it with the business function as you can see from the preceding example.
|
||
|
||
But what if modifications (enrichments) you are trying to make are trivial as they are in the preceding example? Is there a simpler and more dynamic and configurable
|
||
mechanism to accomplish the same?
|
||
|
||
Since version 3.1.3, the framework allows you to provide SpEL expression to enrich individual message headers for both input going into function and
|
||
and output coming out of it. Let’s look at one of the tests as the example.
|
||
|
||
[source, java]
|
||
----
|
||
@Test
|
||
public void testMixedInputOutputHeaderMapping() throws Exception {
|
||
try (ConfigurableApplicationContext context = new SpringApplicationBuilder(
|
||
SampleFunctionConfiguration.class).web(WebApplicationType.NONE).run(
|
||
"--logging.level.org.springframework.cloud.function=DEBUG",
|
||
"--spring.main.lazy-initialization=true",
|
||
"--spring.cloud.function.configuration.split.output-header-mapping-expression.keyOut1='hello1'",
|
||
"--spring.cloud.function.configuration.split.output-header-mapping-expression.keyOut2=headers.contentType",
|
||
"--spring.cloud.function.configuration.split.input-header-mapping-expression.key1=headers.path.split('/')[0]",
|
||
"--spring.cloud.function.configuration.split.input-header-mapping-expression.key2=headers.path.split('/')[1]",
|
||
"--spring.cloud.function.configuration.split.input-header-mapping-expression.key3=headers.path")) {
|
||
|
||
FunctionCatalog functionCatalog = context.getBean(FunctionCatalog.class);
|
||
FunctionInvocationWrapper function = functionCatalog.lookup("split");
|
||
Message<byte[]> result = (Message<byte[]>) function.apply(MessageBuilder.withPayload("helo")
|
||
.setHeader(MessageHeaders.CONTENT_TYPE, "application/json")
|
||
.setHeader("path", "foo/bar/baz").build());
|
||
assertThat(result.getHeaders().containsKey("keyOut1")).isTrue();
|
||
assertThat(result.getHeaders().get("keyOut1")).isEqualTo("hello1");
|
||
assertThat(result.getHeaders().containsKey("keyOut2")).isTrue();
|
||
assertThat(result.getHeaders().get("keyOut2")).isEqualTo("application/json");
|
||
}
|
||
}
|
||
----
|
||
|
||
Here you see a properties called `input-header-mapping-expression` and `output-header-mapping-expression` preceded by the name of the function (i.e., `split`) and followed by the name of the message header key you want to set and the value as SpEL expression. The first expression (for 'keyOut1') is literal SpEL expressions enclosed in single quotes, effectively setting 'keyOut1' to value `hello1`. The `keyOut2` is set to the value of existing 'contentType' header.
|
||
|
||
You can also observe some interesting features in the input header mapping where we actually splitting a value of the existing header 'path', setting individual values of key1 and key2 to the values of split elements based on the index.
|
||
|
||
NOTE: if for whatever reason the provided expression evaluation fails, the execution of the function will proceed as if nothing ever happen.
|
||
However you will see the WARN message in your logs informing you about it
|
||
|
||
[source, text]
|
||
----
|
||
o.s.c.f.context.catalog.InputEnricher : Failed while evaluating expression "hello1" on incoming message. . .
|
||
----
|
||
|
||
In the event you are dealing with functions that have multiple inputs (next section), you can use index immediately after `input-header-mapping-expression`
|
||
|
||
[source, text]
|
||
----
|
||
--spring.cloud.function.configuration.echo.input-header-mapping-expression[0].key1=‘hello1'
|
||
--spring.cloud.function.configuration.echo.input-header-mapping-expression[1].key2='hello2'
|
||
----
|
||
|
||
=== Function Arity
|
||
|
||
There are times when a stream of data needs to be categorized and organized. For example,
|
||
consider a classic big-data use case of dealing with unorganized data containing, let’s say,
|
||
‘orders’ and ‘invoices’, and you want each to go into a separate data store.
|
||
This is where function arity (functions with multiple inputs and outputs) support
|
||
comes to play.
|
||
|
||
Let’s look at an example of such a function (full implementation details are available
|
||
https://github.com/spring-cloud/spring-cloud-stream/blob/master/spring-cloud-stream/src/test/java/org/springframework/cloud/stream/function/MultipleInputOutputFunctionTests.java#L342[here]),
|
||
|
||
[source, java]
|
||
----
|
||
@Bean
|
||
public Function<Flux<Integer>, Tuple2<Flux<String>, Flux<String>>> organise() {
|
||
return flux -> ...;
|
||
}
|
||
----
|
||
|
||
Given that Project Reactor is a core dependency of SCF, we are using its Tuple library.
|
||
Tuples give us a unique advantage by communicating to us both _cardinality_ and _type_ information.
|
||
Both are extremely important in the context of SCSt. Cardinality lets us know
|
||
how many input and output bindings need to be created and bound to the corresponding
|
||
inputs and outputs of a function. Awareness of the type information ensures proper type
|
||
conversion.
|
||
|
||
Also, this is where the ‘index’ part of the naming convention for binding
|
||
names comes into play, since, in this function, the two output binding
|
||
names are `organise-out-0` and `organise-out-1`.
|
||
|
||
IMPORTANT: IMPORTANT: At the moment, function arity is *only* supported for reactive functions
|
||
(`Function<TupleN<Flux<?>...>, TupleN<Flux<?>...>>`) centered on Complex event processing
|
||
where evaluation and computation on confluence of events typically requires view into a
|
||
stream of events rather than single event.
|
||
|
||
=== Type conversion (Content-Type negotiation)
|
||
|
||
Content-Type negotiation is one of the core features of Spring Cloud Function as it allows to not only transform the incoming data to the types declared
|
||
by the function signature, but to do the same transformation during function composition making otherwise un-composable (by type) functions composable.
|
||
|
||
To better understand the mechanics and the necessity behind content-type negotiation, we take a look at a very simple use case by
|
||
using the following function as an example:
|
||
|
||
[source, java]
|
||
----
|
||
@Bean
|
||
public Function<Person, String> personFunction {..}
|
||
----
|
||
|
||
The function shown in the preceding example expects a `Person` object as an argument and produces a String type as an output. If such function is
|
||
invoked with the type `Person`, than all works fine. But typically function plays a role of a handler for the incoming data which most often comes
|
||
in the raw format such as `byte[]`, `JSON String` etc. In order for the framework to succeed in passing the incoming data as an argument to
|
||
this function, it has to somehow transform the incoming data to a `Person` type.
|
||
|
||
Spring Cloud Function relies on two native to Spring mechanisms to accomplish that.
|
||
|
||
. _MessageConverter_ - to convert from incoming Message data to a type declared by the function.
|
||
. _ConversionService_ - to convert from incoming non-Message data to a type declared by the function.
|
||
|
||
This means that depending on the type of the raw data (Message or non-Message) Spring Cloud Function will apply one or the other mechanisms.
|
||
|
||
For most cases when dealing with functions that are invoked as part of some other request (e.g., HTTP, Messaging etc) the framework relies on `MessageConverters`,
|
||
since such requests already converted to Spring `Message`. In other words, the framework locates and applies the appropriate `MessageConverter`.
|
||
To accomplish that, the framework needs some instructions from the user. One of these instructions is already provided by the signature of the function
|
||
itself (Person type). Consequently, in theory, that should be (and, in some cases, is) enough. However, for the majority of use cases, in order to
|
||
select the appropriate `MessageConverter`, the framework needs an additional piece of information. That missing piece is `contentType` header.
|
||
|
||
Such header usually comes as part of the Message where it is injected by the corresponding adapter that created such Message in the first place.
|
||
For example, HTTP POST request will have its content-type HTTP header copied to `contentType` header of the Message.
|
||
|
||
For cases when such header does not exist framework relies on the default content type as `application/json`.
|
||
|
||
|
||
==== Content Type versus Argument Type
|
||
|
||
As mentioned earlier, for the framework to select the appropriate `MessageConverter`, it requires argument type and, optionally, content type information.
|
||
The logic for selecting the appropriate `MessageConverter` resides with the argument resolvers which trigger right before the invocation of the user-defined
|
||
function (which is when the actual argument type is known to the framework).
|
||
If the argument type does not match the type of the current payload, the framework delegates to the stack of the
|
||
pre-configured `MessageConverters` to see if any one of them can convert the payload.
|
||
|
||
The combination of `contentType` and argument type is the mechanism by which framework determines if message can be converted to a target type by locating
|
||
the appropriate `MessageConverter`.
|
||
If no appropriate `MessageConverter` is found, an exception is thrown, which you can handle by adding a custom `MessageConverter`
|
||
(see `<<user-defined-message-converters>>`).
|
||
|
||
NOTE: Do not expect `Message` to be converted into some other type based only on the `contentType`.
|
||
Remember that the `contentType` is complementary to the target type.
|
||
It is a hint, which `MessageConverter` may or may not take into consideration.
|
||
|
||
==== Message Converters
|
||
|
||
`MessageConverters` define two methods:
|
||
|
||
[source, java]
|
||
----
|
||
Object fromMessage(Message<?> message, Class<?> targetClass);
|
||
|
||
Message<?> toMessage(Object payload, @Nullable MessageHeaders headers);
|
||
----
|
||
|
||
It is important to understand the contract of these methods and their usage, specifically in the context of Spring Cloud Stream.
|
||
|
||
The `fromMessage` method converts an incoming `Message` to an argument type.
|
||
The payload of the `Message` could be any type, and it is
|
||
up to the actual implementation of the `MessageConverter` to support multiple types.
|
||
|
||
|
||
==== Provided MessageConverters
|
||
|
||
As mentioned earlier, the framework already provides a stack of `MessageConverters` to handle most common use cases.
|
||
The following list describes the provided `MessageConverters`, in order of precedence (the first `MessageConverter` that works is used):
|
||
|
||
. `JsonMessageConverter`: Supports conversion of the payload of the `Message` to/from POJO for cases when `contentType` is `application/json` using Jackson or Gson libraries (DEFAULT).
|
||
. `ByteArrayMessageConverter`: Supports conversion of the payload of the `Message` from `byte[]` to `byte[]` for cases when `contentType` is `application/octet-stream`. It is essentially a pass through and exists primarily for backward compatibility.
|
||
. `StringMessageConverter`: Supports conversion of any type to a `String` when `contentType` is `text/plain`.
|
||
|
||
When no appropriate converter is found, the framework throws an exception. When that happens, you should check your code and configuration and ensure you did
|
||
not miss anything (that is, ensure that you provided a `contentType` by using a binding or a header).
|
||
However, most likely, you found some uncommon case (such as a custom `contentType` perhaps) and the current stack of provided `MessageConverters`
|
||
does not know how to convert. If that is the case, you can add custom `MessageConverter`. See <<user-defined-message-converters>>.
|
||
|
||
[[user-defined-message-converters]]
|
||
==== User-defined Message Converters
|
||
|
||
Spring Cloud Function exposes a mechanism to define and register additional `MessageConverters`.
|
||
To use it, implement `org.springframework.messaging.converter.MessageConverter`, configure it as a `@Bean`.
|
||
It is then appended to the existing stack of `MessageConverter`s.
|
||
|
||
NOTE: It is important to understand that custom `MessageConverter` implementations are added to the head of the existing stack.
|
||
Consequently, custom `MessageConverter` implementations take precedence over the existing ones, which lets you override as well as add to the existing converters.
|
||
|
||
The following example shows how to create a message converter bean to support a new content type called `application/bar`:
|
||
|
||
[source,java]
|
||
----
|
||
@SpringBootApplication
|
||
public static class SinkApplication {
|
||
|
||
...
|
||
|
||
@Bean
|
||
public MessageConverter customMessageConverter() {
|
||
return new MyCustomMessageConverter();
|
||
}
|
||
}
|
||
|
||
public class MyCustomMessageConverter extends AbstractMessageConverter {
|
||
|
||
public MyCustomMessageConverter() {
|
||
super(new MimeType("application", "bar"));
|
||
}
|
||
|
||
@Override
|
||
protected boolean supports(Class<?> clazz) {
|
||
return (Bar.class.equals(clazz));
|
||
}
|
||
|
||
@Override
|
||
protected Object convertFromInternal(Message<?> message, Class<?> targetClass, Object conversionHint) {
|
||
Object payload = message.getPayload();
|
||
return (payload instanceof Bar ? payload : new Bar((byte[]) payload));
|
||
}
|
||
}
|
||
----
|
||
|
||
==== Note on JSON options
|
||
|
||
In Spring Cloud Function we support Jackson and Gson mechanisms to deal with JSON.
|
||
And for your benefit have abstracted it under `org.springframework.cloud.function.json.JsonMapper` which itself is aware of two mechanisms and will use the one selected
|
||
by you or following the default rule.
|
||
The default rules are as follows:
|
||
|
||
* Whichever library is on the classpath that is the mechanism that is going to be used. So if you have `com.fasterxml.jackson.*` to the classpath, Jackson is going to be used and if you have `com.google.code.gson`, then Gson will be used.
|
||
* If you have both, then Gson will be the default, or you can set `spring.cloud.function.preferred-json-mapper` property with either of two values: `gson` or `jackson`.
|
||
|
||
|
||
That said, the type conversion is usually transparent to the developer, however given that `org.springframework.cloud.function.json.JsonMapper` is also registered as a bean
|
||
you can easily inject it into your code if needed.
|
||
|
||
|
||
=== Kotlin Lambda support
|
||
|
||
We also provide support for Kotlin lambdas (since v2.0).
|
||
Consider the following:
|
||
|
||
[source, java]
|
||
----
|
||
@Bean
|
||
open fun kotlinSupplier(): () -> String {
|
||
return { "Hello from Kotlin" }
|
||
}
|
||
|
||
@Bean
|
||
open fun kotlinFunction(): (String) -> String {
|
||
return { it.toUpperCase() }
|
||
}
|
||
|
||
@Bean
|
||
open fun kotlinConsumer(): (String) -> Unit {
|
||
return { println(it) }
|
||
}
|
||
|
||
----
|
||
The above represents Kotlin lambdas configured as Spring beans. The signature of each maps to a Java equivalent of
|
||
`Supplier`, `Function` and `Consumer`, and thus supported/recognized signatures by the framework.
|
||
While mechanics of Kotlin-to-Java mapping are outside of the scope of this documentation, it is important to understand that the
|
||
same rules for signature transformation outlined in "Java 8 function support" section are applied here as well.
|
||
|
||
To enable Kotlin support all you need is to add Kotlin SDK libraries on the classpath which will trigger appropriate
|
||
autoconfiguration and supporting classes.
|
||
|
||
=== Function Component Scan
|
||
|
||
Spring Cloud Function will scan for implementations of `Function`, `Consumer` and `Supplier` in a package called `functions` if it exists. Using this
|
||
feature you can write functions that have no dependencies on Spring - not even the `@Component` annotation is needed. If you want to use a different
|
||
package, you can set `spring.cloud.function.scan.packages`. You can also use `spring.cloud.function.scan.enabled=false` to switch off the scan completely.
|
||
|
||
== Standalone Web Applications
|
||
|
||
Functions could be automatically exported as HTTP endpoints.
|
||
|
||
The `spring-cloud-function-web` module has autoconfiguration that
|
||
activates when it is included in a Spring Boot web application (with
|
||
MVC support). There is also a `spring-cloud-starter-function-web` to
|
||
collect all the optional dependencies in case you just want a simple
|
||
getting started experience.
|
||
|
||
With the web configurations activated your app will have an MVC
|
||
endpoint (on "/" by default, but configurable with
|
||
`spring.cloud.function.web.path`) that can be used to access the
|
||
functions in the application context where function name becomes part of the URL path. The supported content types are
|
||
plain text and JSON.
|
||
|
||
|===
|
||
| Method | Path | Request | Response | Status
|
||
|
||
| GET | /{supplier} | - | Items from the named supplier | 200 OK
|
||
| POST | /{consumer} | JSON object or text | Mirrors input and pushes request body into consumer | 202 Accepted
|
||
| POST | /{consumer} | JSON array or text with new lines | Mirrors input and pushes body into consumer one by one | 202 Accepted
|
||
| POST | /{function} | JSON object or text | The result of applying the named function | 200 OK
|
||
| POST | /{function} | JSON array or text with new lines | The result of applying the named function | 200 OK
|
||
| GET | /{function}/{item} | - | Convert the item into an object and return the result of applying the function | 200 OK
|
||
|
||
|===
|
||
|
||
As the table above shows the behaviour of the endpoint depends on the method and also the type of incoming request data. When the incoming data
|
||
is single valued, and the target function is declared as obviously single valued (i.e. not returning a collection or `Flux`), then the response
|
||
will also contain a single value.
|
||
For multi-valued responses the client can ask for a server-sent event stream by sending `Accept: text/event-stream".
|
||
|
||
Functions and consumers that are declared with input and output in `Message<?>` will see the request headers on the input messages, and the output message headers will be converted to HTTP headers.
|
||
|
||
When POSTing text the response format might be different with Spring Boot 2.0 and older versions, depending on the content negotiation (provide content type and accept headers for the best results).
|
||
|
||
See <<Testing Functional Applications>> to see the details and example on how to test such application.
|
||
|
||
|
||
=== Function Mapping rules
|
||
|
||
If there is only a single function (consumer etc.) in the catalog, the name in the path is optional.
|
||
In other words, providing you only have `uppercase` function in catalog
|
||
`curl -H "Content-Type: text/plain" localhost:8080/uppercase -d hello` and `curl -H "Content-Type: text/plain" localhost:8080/ -d hello` calls are identical.
|
||
|
||
Composite functions can be addressed using pipes or commas to separate function names (pipes are legal in URL paths, but a bit awkward to type on the command line).
|
||
For example, `curl -H "Content-Type: text/plain" localhost:8080/uppercase,reverse -d hello`.
|
||
|
||
For cases where there is more then a single function in catalog, each function will be exported and mapped with function name being
|
||
part of the path (e.g., `localhost:8080/uppercase`).
|
||
In this scenario you can still map specific function or function composition to the root path by providing
|
||
`spring.cloud.function.definition` property
|
||
|
||
For example,
|
||
----
|
||
--spring.cloud.function.definition=foo|bar
|
||
----
|
||
|
||
The above property will compose 'foo' and 'bar' function and map the composed function to the "/" path.
|
||
|
||
The same property will also work for cases where function can not be resolved via URL. For example, your URL may be `localhost:8080/uppercase`, but there is no `uppercase` function.
|
||
However there are function `foo` and `bar`. So, in this case `localhost:8080/uppercase` will resolve to `foo|bar`.
|
||
This could be useful especially for cases when URL is used to communicate certain information since there will be Message header called `uri` with the value
|
||
of the actual URL, giving user ability to use it for evaluation and computation.
|
||
|
||
=== Function Filtering rules
|
||
|
||
In situations where there are more then one function in catalog there may be a need to only export certain functions or function compositions. In that case you can use
|
||
the same `spring.cloud.function.definition` property listing functions you intend to export delimited by `;`.
|
||
Note that in this case nothing will be mapped to the root path and functions that are not listed (including compositions) are not going to be exported
|
||
|
||
For example,
|
||
----
|
||
--spring.cloud.function.definition=foo;bar
|
||
----
|
||
|
||
This will only export function `foo` and function `bar` regardless how many functions are available in catalog (e.g., `localhost:8080/foo`).
|
||
|
||
----
|
||
--spring.cloud.function.definition=foo|bar;baz
|
||
----
|
||
|
||
This will only export function composition `foo|bar` and function `baz` regardless how many functions are available in catalog (e.g., `localhost:8080/foo,bar`).
|
||
|
||
|
||
== Standalone Streaming Applications
|
||
|
||
To send or receive messages from a broker (such as RabbitMQ or Kafka) you can leverage `spring-cloud-stream` project and it's integration with Spring Cloud Function.
|
||
Please refer to https://cloud.spring.io/spring-cloud-static/spring-cloud-stream/current/reference/html/spring-cloud-stream.html#spring_cloud_function[Spring Cloud Function] section of the Spring Cloud Stream reference manual for more details and examples.
|
||
|
||
== Deploying a Packaged Function
|
||
|
||
Spring Cloud Function provides a "deployer" library that allows you to launch a jar file (or exploded archive, or set of jar files) with an isolated class loader and expose the functions defined in it. This is quite a powerful tool that would allow you to, for instance, adapt a function to a range of different input-output adapters without changing the target jar file. Serverless platforms often have this kind of feature built in, so you could see it as a building block for a function invoker in such a platform (indeed the https://projectriff.io[Riff] Java function invoker uses this library).
|
||
|
||
The standard entry point is to add `spring-cloud-function-deployer` to the classpath, the deployer kicks in and looks for some configuration to tell it where to find the function jar.
|
||
|
||
```xml
|
||
<dependency>
|
||
<groupId>org.springframework.cloud</groupId>
|
||
<artifactId>spring-cloud-function-deployer</artifactId>
|
||
<version>${spring.cloud.function.version}</version>
|
||
</dependency>
|
||
```
|
||
|
||
|
||
At a minimum the user has to provide a `spring.cloud.function.location` which is a URL or resource location for the archive containing
|
||
the functions. It can optionally use a `maven:` prefix to locate the artifact via a dependency lookup (see `FunctionProperties`
|
||
for complete details). A Spring Boot application is bootstrapped from the jar file, using the `MANIFEST.MF` to locate a start class, so
|
||
that a standard Spring Boot fat jar works well, for example. If the target jar can be launched successfully then the result is a function
|
||
registered in the main application's `FunctionCatalog`. The registered function can be applied by code in the main application, even though
|
||
it was created in an isolated class loader (by deault).
|
||
|
||
Here is the example of deploying a JAR which contains an 'uppercase' function and invoking it .
|
||
|
||
```java
|
||
@SpringBootApplication
|
||
public class DeployFunctionDemo {
|
||
|
||
public static void main(String[] args) {
|
||
ApplicationContext context = SpringApplication.run(DeployFunctionDemo.class,
|
||
"--spring.cloud.function.location=..../target/uppercase-0.0.1-SNAPSHOT.jar",
|
||
"--spring.cloud.function.definition=uppercase");
|
||
|
||
FunctionCatalog catalog = context.getBean(FunctionCatalog.class);
|
||
Function<String, String> function = catalog.lookup("uppercase");
|
||
System.out.println(function.apply("hello"));
|
||
}
|
||
}
|
||
```
|
||
|
||
And here is the example using Maven URI (taken from one of the tests in `FunctionDeployerTests`):
|
||
|
||
```java
|
||
@SpringBootApplication
|
||
public class DeployFunctionDemo {
|
||
|
||
public static void main(String[] args) {
|
||
String[] args = new String[] {
|
||
"--spring.cloud.function.location=maven://oz.demo:demo-uppercase:0.0.1-SNAPSHOT",
|
||
"--spring.cloud.function.function-class=oz.demo.uppercase.MyFunction" };
|
||
|
||
ApplicationContext context = SpringApplication.run(DeployerApplication.class, args);
|
||
FunctionCatalog catalog = context.getBean(FunctionCatalog.class);
|
||
Function<String, String> function = catalog.lookup("myFunction");
|
||
|
||
assertThat(function.apply("bob")).isEqualTo("BOB");
|
||
}
|
||
}
|
||
```
|
||
|
||
Keep in mind that Maven resource such as local and remote repositories, user, password and more are resolved using default MavenProperties which
|
||
effectively use local defaults and will work for majority of cases. However if you need to customize you can simply provide a bean of type
|
||
`MavenProperties` where you can set additional properties (see example below).
|
||
|
||
```java
|
||
@Bean
|
||
public MavenProperties mavenProperties() {
|
||
MavenProperties properties = new MavenProperties();
|
||
properties.setLocalRepository("target/it/");
|
||
return properties;
|
||
}
|
||
```
|
||
|
||
=== Supported Packaging Scenarios
|
||
|
||
Currently Spring Cloud Function supports several packaging scenarios to give you the most flexibility when it comes to deploying functions.
|
||
|
||
==== Simple JAR
|
||
|
||
This packaging option implies no dependency on anything related to Spring.
|
||
For example; Consider that such JAR contains the following class:
|
||
```java
|
||
package function.example;
|
||
. . .
|
||
public class UpperCaseFunction implements Function<String, String> {
|
||
@Override
|
||
public String apply(String value) {
|
||
return value.toUpperCase();
|
||
}
|
||
}
|
||
```
|
||
All you need to do is specify `location` and `function-class` properties when deploying such package:
|
||
|
||
```
|
||
--spring.cloud.function.location=target/it/simplestjar/target/simplestjar-1.0.0.RELEASE.jar
|
||
--spring.cloud.function.function-class=function.example.UpperCaseFunction
|
||
```
|
||
|
||
It's conceivable in some cases that you might want to package multiple functions together. For such scenarios you can use
|
||
`spring.cloud.function.function-class` property to list several classes delimiting them by `;`.
|
||
|
||
For example,
|
||
|
||
```
|
||
--spring.cloud.function.function-class=function.example.UpperCaseFunction;function.example.ReverseFunction
|
||
```
|
||
|
||
Here we are identifying two functions to deploy, which we can now access in function catalog by name (e.g., `catalog.lookup("reverseFunction");`).
|
||
|
||
|
||
For more details please reference the complete sample available https://github.com/spring-cloud/spring-cloud-function/tree/master/spring-cloud-function-deployer/src/it/simplestjar[here].
|
||
You can also find a corresponding test in https://github.com/spring-cloud/spring-cloud-function/blob/master/spring-cloud-function-deployer/src/test/java/org/springframework/cloud/function/deployer/FunctionDeployerTests.java#L70[FunctionDeployerTests].
|
||
|
||
*** Component Scanning ***
|
||
|
||
Since version 3.1.4 you can simplify your configuration thru component scanning feature described in <<Function Component Scan>>. If you place your functional class in
|
||
package named `functions`, you can omit `spring.cloud.function.function-class` property as framework will auto-discover functional classes loading them in function catalog.
|
||
Keep in mind the naming convention to follow when doing function lookup. For example function class `functions.UpperCaseFunction` will be available in `FunctionCatalog`
|
||
under the name `upperCaseFunction`.
|
||
|
||
==== Spring Boot JAR
|
||
|
||
This packaging option implies there is a dependency on Spring Boot and that the JAR was generated as Spring Boot JAR. That said, given that the deployed JAR
|
||
runs in the isolated class loader, there will not be any version conflict with the Spring Boot version used by the actual deployer.
|
||
For example; Consider that such JAR contains the following class (which could have some additional Spring dependencies providing Spring/Spring Boot is on the classpath):
|
||
```java
|
||
package function.example;
|
||
. . .
|
||
public class UpperCaseFunction implements Function<String, String> {
|
||
@Override
|
||
public String apply(String value) {
|
||
return value.toUpperCase();
|
||
}
|
||
}
|
||
```
|
||
As before all you need to do is specify `location` and `function-class` properties when deploying such package:
|
||
|
||
```
|
||
--spring.cloud.function.location=target/it/simplestjar/target/simplestjar-1.0.0.RELEASE.jar
|
||
--spring.cloud.function.function-class=function.example.UpperCaseFunction
|
||
```
|
||
For more details please reference the complete sample available https://github.com/spring-cloud/spring-cloud-function/tree/master/spring-cloud-function-deployer/src/it/bootjar[here].
|
||
You can also find a corresponding test in https://github.com/spring-cloud/spring-cloud-function/blob/master/spring-cloud-function-deployer/src/test/java/org/springframework/cloud/function/deployer/FunctionDeployerTests.java#L50[FunctionDeployerTests].
|
||
|
||
==== Spring Boot Application
|
||
|
||
This packaging option implies your JAR is complete stand alone Spring Boot application with functions as managed Spring beans.
|
||
As before there is an obvious assumption that there is a dependency on Spring Boot and that the JAR was generated as Spring Boot JAR. That said, given that the deployed JAR
|
||
runs in the isolated class loader, there will not be any version conflict with the Spring Boot version used by the actual deployer.
|
||
For example; Consider that such JAR contains the following class:
|
||
```java
|
||
package function.example;
|
||
. . .
|
||
@SpringBootApplication
|
||
public class SimpleFunctionAppApplication {
|
||
|
||
public static void main(String[] args) {
|
||
SpringApplication.run(SimpleFunctionAppApplication.class, args);
|
||
}
|
||
|
||
@Bean
|
||
public Function<String, String> uppercase() {
|
||
return value -> value.toUpperCase();
|
||
}
|
||
}
|
||
```
|
||
Given that we're effectively dealing with another Spring Application context and that functions are spring managed beans,
|
||
in addition to the `location` property we also specify `definition` property instead of `function-class`.
|
||
|
||
```
|
||
--spring.cloud.function.location=target/it/bootapp/target/bootapp-1.0.0.RELEASE-exec.jar
|
||
--spring.cloud.function.definition=uppercase
|
||
```
|
||
For more details please reference the complete sample available https://github.com/spring-cloud/spring-cloud-function/tree/master/spring-cloud-function-deployer/src/it/bootapp[here].
|
||
You can also find a corresponding test in https://github.com/spring-cloud/spring-cloud-function/blob/master/spring-cloud-function-deployer/src/test/java/org/springframework/cloud/function/deployer/FunctionDeployerTests.java#L164[FunctionDeployerTests].
|
||
|
||
NOTE: This particular deployment option may or may not have Spring Cloud Function on it's classpath. From the deployer perspective this doesn't matter.
|
||
|
||
== Functional Bean Definitions
|
||
|
||
include::functional.adoc[leveloffset=+1]
|
||
|
||
== Serverless Platform Adapters
|
||
|
||
As well as being able to run as a standalone process, a Spring Cloud
|
||
Function application can be adapted to run one of the existing
|
||
serverless platforms. In the project there are adapters for
|
||
https://github.com/spring-cloud/spring-cloud-function/tree/{branch}/spring-cloud-function-adapters/spring-cloud-function-adapter-aws[AWS
|
||
Lambda],
|
||
https://github.com/spring-cloud/spring-cloud-function/tree/{branch}/spring-cloud-function-adapters/spring-cloud-function-adapter-azure[Azure],
|
||
and
|
||
https://github.com/spring-cloud/spring-cloud-function/tree/{branch}/spring-cloud-function-adapters/spring-cloud-function-adapter-openwhisk[Apache
|
||
OpenWhisk]. The https://github.com/fnproject/fn[Oracle Fn platform]
|
||
has its own Spring Cloud Function adapter. And
|
||
https://projectriff.io[Riff] supports Java functions and its
|
||
https://github.com/projectriff/java-function-invoker[Java Function
|
||
Invoker] acts natively is an adapter for Spring Cloud Function jars.
|
||
|
||
include::adapters/aws-intro.adoc[]
|
||
include::adapters/azure-intro.adoc[]
|
||
include::adapters/gcp-intro.adoc[]
|
||
|