SpringCloud/spring-cloud-function
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Add docs and updates for background function support

Browse Source 
rename fuction-sample-gcp to function-sample-gcp-http

refdoc polish

background sample polish

Resolves #525
Update pub/sub bg function to use base64 encoding
This commit is contained in:
dzou
2020-05-19 18:11:26 -04:00
committed by Oleg Zhurakousky
parent f279010aab
commit a4788aba08
13 changed files with 474 additions and 66 deletions
Download Patch File Download Diff File Expand all files Collapse all files

259
docs/src/main/asciidoc/adapters/gcp-intro.adoc
View File

@@ -5,7 +5,68 @@
The Google Cloud Functions adapter enables Spring Cloud Function apps to run on the https://cloud.google.com/functions[Google Cloud Functions] serverless platform.
You can either run the function locally using the open source https://github.com/GoogleCloudPlatform/functions-framework-java[Google Functions Framework for Java] or on GCP.
==== Getting Started
==== Project Dependencies
Start by adding the `spring-cloud-function-adapter-gcp` dependency to your project.
[source, xml]
----
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-function-adapter-gcp</artifactId>
</dependency>
...
</dependencies>
----
In addition, add the `spring-boot-maven-plugin` which will build the JAR of the function to deploy.
NOTE: Notice that we also reference `spring-cloud-function-adapter-gcp` as a dependency of the `spring-boot-maven-plugin`. This is necessary because it modifies the plugin to package your function in the correct JAR format for deployment on Google Cloud Functions.
[source, xml]
----
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<outputDirectory>target/deploy</outputDirectory>
</configuration>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-function-adapter-gcp</artifactId>
</dependency>
</dependencies>
</plugin>
----
Finally, add the Maven plugin provided as part of the Google Functions Framework for Java.
This allows you to test your functions locally via `mvn function:run`.
NOTE: The function target should always be set to `org.springframework.cloud.function.adapter.gcp.GcfJarLauncher`; this is an adapter class which acts as the entry point to your Spring Cloud Function from the Google Cloud Functions platform.
[source,xml]
----
<plugin>
<groupId>com.google.cloud.functions</groupId>
<artifactId>function-maven-plugin</artifactId>
<version>0.9.1</version>
<configuration>
<functionTarget>org.springframework.cloud.function.adapter.gcp.GcfJarLauncher</functionTarget>
<port>8080</port>
</configuration>
</plugin>
----
A full example of a working `pom.xml` can be found in the https://github.com/spring-cloud/spring-cloud-function/blob/master/spring-cloud-function-samples/function-sample-gcp-http/pom.xml[Spring Cloud Functions GCP sample].
==== HTTP Functions
Google Cloud Functions supports deploying https://cloud.google.com/functions/docs/writing/http[HTTP Functions], which are functions that are invoked by HTTP request. The sections below describe instructions for deploying a Spring Cloud Function as an HTTP Function.
===== Getting Started
Let’s start with a simple Spring Cloud Function example:
@@ -25,23 +86,6 @@ public class CloudFunctionMain {
}
----
===== Test locally
Start by adding the Maven plugin provided as part of the Google Functions Framework for Java.
[source,xml]
----
<plugin>
<groupId>com.google.cloud.functions</groupId>
<artifactId>function-maven-plugin</artifactId>
<version>0.9.1</version>
<configuration>
<functionTarget>org.springframework.cloud.function.adapter.gcp.GcfJarLauncher</functionTarget>
<port>8080</port>
</configuration>
</plugin>
----
Specify your configuration main class in `resources/META-INF/MANIFEST.MF`.
[source]
@@ -49,7 +93,8 @@ Specify your configuration main class in `resources/META-INF/MANIFEST.MF`.
Main-Class: com.example.CloudFunctionMain
----
Then run the function:
Then run the function locally.
This is provided by the Google Cloud Functions `function-maven-plugin` described in the project dependencies section.
----
mvn function:run
@@ -66,51 +111,21 @@ curl http://localhost:8080/ -d "hello"
As of March 2020, Google Cloud Functions for Java is in Alpha.
You can get on the https://docs.google.com/forms/d/e/1FAIpQLScC98jGi7CfG0n3UYlj7Xad8XScvZC8-BBOg7Pk3uSZx_2cdQ/viewform[whitelist] to try it out.
In order to use the adapter, first add the dependency to your pom.xml:
[source, xml]
----
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-function-adapter-gcp</artifactId>
</dependency>
----
Then, add the `spring-boot-maven-plugin` with `spring-cloud-function-adapter-gcp` as a dependency.
The extra dependency is used for `spring-boot-maven-plugin` to package your function in the correct JAR format for deployment on Google Cloud Functions.
[source, xml]
----
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<outputDirectory>target/deploy</outputDirectory>
</configuration>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-function-adapter-gcp</artifactId>
</dependency>
</dependencies>
</plugin>
----
Package the application.
Start by packaging your application.
----
mvn package
----
You should see the resulting JAR in `target/deploy` directory.
If you added the custom `spring-boot-maven-plugin` plugin defined above, you should see the resulting JAR in `target/deploy` directory.
This JAR is correctly formatted for deployment to Google Cloud Functions.
Make sure that you have the https://cloud.google.com/sdk/install[Cloud SDK CLI] installed.
Next, make sure that you have the https://cloud.google.com/sdk/install[Cloud SDK CLI] installed.
From the project base directory run the following command to deploy.
----
gcloud alpha functions deploy function-sample-gcp \
gcloud alpha functions deploy function-sample-gcp-http \
--entry-point org.springframework.cloud.function.adapter.gcp.GcfJarLauncher \
--runtime java11 \
--trigger-http \
@@ -121,9 +136,141 @@ gcloud alpha functions deploy function-sample-gcp \
Invoke the HTTP function:
----
curl https://REGION-PROJECT_ID.cloudfunctions.net/function-sample-gcp -d "hello"
curl https://REGION-PROJECT_ID.cloudfunctions.net/function-sample-gcp-http -d "hello"
----
==== Sample Function
==== Background Functions
Go to the https://github.com/spring-cloud/spring-cloud-function/tree/master/spring-cloud-function-samples/function-sample-gcp/[function-sample-gcp] to try out a sample function that you can test locally or deploy to GCP.
Google Cloud Functions also supports deploying https://cloud.google.com/functions/docs/writing/background[Background Functions] which are invoked indirectly in response to an event, such as a message on a https://cloud.google.com/pubsub[Cloud Pub/Sub] topic, a change in a https://cloud.google.com/storage[Cloud Storage] bucket, or a https://firebase.google.com/[Firebase] event.
The `spring-cloud-function-adapter-gcp` allows for functions to be deployed as background functions as well.
The sections below describe the process for writing a Cloud Pub/Sub topic background function.
However, there are a number of different event types that can trigger a background function to execute which are not discussed here; these are described in the https://cloud.google.com/functions/docs/calling[Background Function triggers documentation].
===== Getting Started
Let’s start with a simple Spring Cloud Function which will run as a GCF background function:
[source, java]
----
@SpringBootApplication
public class BackgroundFunctionMain {
public static void main(String[] args) {
SpringApplication.run(BackgroundFunctionMain.class, args);
}
@Bean
public Consumer<PubSubMessage> pubSubFunction() {
return message -> System.out.println("The Pub/Sub message data: " + message.getData());
}
}
----
In addition, create `PubSubMessage` class in the project with the below definition.
This class represents the https://cloud.google.com/functions/docs/calling/pubsub#event_structure[Pub/Sub event structure] which gets passed to your function on a Pub/Sub topic event.
[source, java]
----
public class PubSubMessage {
private String data;
private Map<String, String> attributes;
private String messageId;
private String publishTime;
public String getData() {
return data;
}
public void setData(String data) {
this.data = data;
}
public Map<String, String> getAttributes() {
return attributes;
}
public void setAttributes(Map<String, String> attributes) {
this.attributes = attributes;
}
public String getMessageId() {
return messageId;
}
public void setMessageId(String messageId) {
this.messageId = messageId;
}
public String getPublishTime() {
return publishTime;
}
public void setPublishTime(String publishTime) {
this.publishTime = publishTime;
}
}
----
Specify your configuration main class in `resources/META-INF/MANIFEST.MF`.
[source]
----
Main-Class: com.example.BackgroundFunctionMain
----
Then run the function locally.
This is provided by the Google Cloud Functions `function-maven-plugin` described in the project dependencies section.
----
mvn function:run
----
Invoke the HTTP function:
----
curl localhost:8080 -H "Content-Type: application/json" -d '{"data":"hello"}'
----
Verify that the function was invoked by viewing the logs.
===== Deploy to GCP
In order to deploy your background function to GCP, first package your application.
----
mvn package
----
If you added the custom `spring-boot-maven-plugin` plugin defined above, you should see the resulting JAR in `target/deploy` directory.
This JAR is correctly formatted for deployment to Google Cloud Functions.
Next, make sure that you have the https://cloud.google.com/sdk/install[Cloud SDK CLI] installed.
From the project base directory run the following command to deploy.
----
gcloud alpha functions deploy function-sample-gcp-background \
--entry-point org.springframework.cloud.function.adapter.gcp.GcfJarLauncher \
--runtime java11 \
--trigger-topic my-functions-topic \
--source target/deploy \
--memory 512MB
----
Google Cloud Function will now invoke the function every time a message is published to the topic specified by `--trigger-topic`.
For a walkthrough on testing and verifying your background function, see the instructions for running the https://github.com/spring-cloud/spring-cloud-function/tree/master/spring-cloud-function-samples/function-sample-gcp-background/[GCF Background Function sample].
==== Sample Functions
The project provides the following sample functions as reference:
* The https://github.com/spring-cloud/spring-cloud-function/tree/master/spring-cloud-function-samples/function-sample-gcp-http/[function-sample-gcp-http] is an HTTP Function which you can test locally and try deploying.
* The https://github.com/spring-cloud/spring-cloud-function/tree/master/spring-cloud-function-samples/function-sample-gcp-background/[function-sample-gcp-background] shows an example of a background function that is triggered by a message being published to a specified Pub/Sub topic.

16
spring-cloud-function-adapters/spring-cloud-function-adapter-gcp/src/main/java/org/springframework/cloud/function/adapter/gcp/GcfJarLauncher.java
View File

@@ -16,9 +16,11 @@
package org.springframework.cloud.function.adapter.gcp;
import com.google.cloud.functions.Context;
import com.google.cloud.functions.HttpFunction;
import com.google.cloud.functions.HttpRequest;
import com.google.cloud.functions.HttpResponse;
import com.google.cloud.functions.RawBackgroundFunction;
import org.springframework.boot.loader.JarLauncher;
import org.springframework.boot.loader.jar.JarFile;
@@ -30,11 +32,11 @@ import org.springframework.boot.loader.jar.JarFile;
* @author Ray Tsang
* @author Daniel Zou
*/
public class GcfJarLauncher extends JarLauncher implements HttpFunction {
public class GcfJarLauncher extends JarLauncher implements HttpFunction, RawBackgroundFunction {
private final ClassLoader loader;
private final HttpFunction delegate;
private final Object delegate;
public GcfJarLauncher() throws Exception {
JarFile.registerUrlProtocolHandler();
@@ -43,12 +45,18 @@ public class GcfJarLauncher extends JarLauncher implements HttpFunction {
Class<?> clazz = this.loader
.loadClass("org.springframework.cloud.function.adapter.gcp.FunctionInvoker");
this.delegate = (HttpFunction) clazz.getConstructor().newInstance();
this.delegate = clazz.getConstructor().newInstance();
}
@Override
public void service(HttpRequest httpRequest, HttpResponse httpResponse) throws Exception {
Thread.currentThread().setContextClassLoader(this.loader);
delegate.service(httpRequest, httpResponse);
((HttpFunction) delegate).service(httpRequest, httpResponse);
}
@Override
public void accept(String json, Context context) {
Thread.currentThread().setContextClassLoader(this.loader);
((RawBackgroundFunction) delegate).accept(json, context);
}
}

107
spring-cloud-function-samples/function-sample-gcp-background/README.adoc Normal file
View File

@@ -0,0 +1,107 @@
:branch: master
=== Google Cloud Background Functions Sample Application
Google Cloud Functions supports deploying https://cloud.google.com/functions/docs/writing/background[Background Functions] which are invoked indirectly in response to an event, such as a message on a https://cloud.google.com/pubsub[Pub/Sub] topic, a change in a https://cloud.google.com/storage[Cloud Storage] bucket, or a https://firebase.google.com/[Firebase] event.
This sample demonstrates a simple background function which triggers from a Pub/Sub event.
===== Test locally
In a terminal window, run:
----
mvn function:run
----
In a separate window, invoke the background function by issuing the `curl` command below.
NOTE: In this sample, we demonstrate a background function which is triggered when a message is https://cloud.google.com/functions/docs/calling/pubsub[published to a specified Cloud Pub/Sub topic].
The `curl` request simulates sending the message that would be received by your function when a Pub/Sub event occurs.
In order to simulate how Pub/Sub will send you this message, we use a base64-encoded string as the input because Pub/Sub encodes the message data in base64 in the Pub/Sub event.
See the notes section below for a more information on the Pub/Sub event structure.
----
curl localhost:8080 -H "Content-Type: application/json" -d '{"data":"aGVsbG8="}'
----
In the original terminal window where the `mvn function:run` was invoked, you should see a message printed.
----
Received Pub/Sub message with data: hello
----
===== Deploy to GCP
As of March 2020, Google Cloud Functions for Java is in Alpha.
You can get on the https://docs.google.com/forms/d/e/1FAIpQLScC98jGi7CfG0n3UYlj7Xad8XScvZC8-BBOg7Pk3uSZx_2cdQ/viewform[whitelist] to try it out.
The next steps will demonstrate how to deploy your background function to GCP and have it be invoked by a Pub/Sub event.
To complete the next steps, make sure that you have the https://cloud.google.com/sdk/install[Cloud SDK CLI] installed.
1. Create a new Pub/Sub topic. This topic will be used as the trigger for the background function.
+
----
gcloud pubsub topics create my-functions-topic
----
2. In this sample directory, package the application by running:
+
----
mvn package
----
+
You should see the JAR to deploy in the `target/deploy` directory.
3. Deploy the JAR that you created by running
+
----
gcloud alpha functions deploy function-sample-gcp-background \
--entry-point org.springframework.cloud.function.adapter.gcp.GcfJarLauncher \
--runtime java11 \
--trigger-topic my-functions-topic \
--source target/deploy \
--memory 512MB
----
+
Notice the parameter `--trigger-topic` which topic will trigger the function invocation when new messages are published to it.
4. Invoke the background function by publishing a message to your topic.
+
----
gcloud pubsub topics publish my-functions-topic --message="Hello world"
----
5. To verify that this sample background function was invoked, check the logs of the background function on GCF by running:
+
----
gcloud functions logs read function-sample-gcp-background --filter=Received
----
+
You should see a log message that looks like below.
It might take a minute or two for the log to appear.
+
----
function-sample-gcp-background-2 h8q1jt46069r 2020-05-19 19:48:27.960 Received Pub/Sub message with data: Hello world
----
+
This log output is produced by the sample background function, so this entry in the logs means that the background function was successfully invoked.
===== Notes
One important note about the Pub/Sub background function provided in `BackgroundFunctionMain` is that the `data` field must be decoded from base64 encoding because Pub/Sub will encode the published message in base64 according to the https://cloud.google.com/functions/docs/calling/pubsub#event_structure[PubSubMessage event structure].
[source, java]
----
@Bean
public Consumer<PubSubMessage> pubSubFunction() {
return message -> {
// The PubSubMessage data field arrives as a base-64 encoded string and must be decoded.
// See: https://cloud.google.com/functions/docs/calling/pubsub#event_structure
String decodedMessage = new String(Base64.decode(message.getData()), StandardCharsets.UTF_8);
System.out.println("Received Pub/Sub message with data: " + decodedMessage);
};
}
----

58
spring-cloud-function-samples/function-sample-gcp-background/pom.xml Normal file
View File

@@ -0,0 +1,58 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>io.spring.sample</groupId>
<artifactId>function-sample-gcp-background</artifactId>
<version>2.0.0.RELEASE</version>
<packaging>jar</packaging>
<name>function-sample-gcp-background</name>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.3.0.BUILD-SNAPSHOT</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-function-adapter-gcp</artifactId>
<version>3.1.0.BUILD-SNAPSHOT</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<outputDirectory>target/deploy</outputDirectory>
</configuration>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-function-adapter-gcp</artifactId>
<version>3.1.0.BUILD-SNAPSHOT</version>
</dependency>
</dependencies>
</plugin>
<plugin>
<groupId>com.google.cloud.functions</groupId>
<artifactId>function-maven-plugin</artifactId>
<version>0.9.1</version>
<configuration>
<functionTarget>org.springframework.cloud.function.adapter.gcp.GcfJarLauncher</functionTarget>
<port>8080</port>
</configuration>
</plugin>
</plugins>
</build>
</project>

31
spring-cloud-function-samples/function-sample-gcp-background/src/main/java/com/example/BackgroundFunctionMain.java Normal file
View File

@@ -0,0 +1,31 @@
package com.example;
import java.nio.charset.StandardCharsets;
import java.util.Base64;
import java.util.function.Consumer;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
@SpringBootApplication
public class BackgroundFunctionMain {
public static void main(String[] args) {
SpringApplication.run(BackgroundFunctionMain.class, args);
}
/**
* The background function which triggers on an event from Pub/Sub and consumes the Pub/Sub
* event message.
*/
@Bean
public Consumer<PubSubMessage> pubSubFunction() {
return message -> {
// The PubSubMessage data field arrives as a base-64 encoded string and must be decoded.
// See: https://cloud.google.com/functions/docs/calling/pubsub#event_structure
String decodedMessage = new String(Base64.getDecoder().decode(message.getData()), StandardCharsets.UTF_8);
System.out.println("Received Pub/Sub message with data: " + decodedMessage);
};
}
}

56
spring-cloud-function-samples/function-sample-gcp-background/src/main/java/com/example/PubSubMessage.java Normal file
View File

@@ -0,0 +1,56 @@
package com.example;
import java.util.Map;
/**
* A class that can be mapped to the GCF Pub/Sub Message event type. This is for use in
* the background functions.
*
* <p>See the PubSubMessage definition for reference:
* https://cloud.google.com/pubsub/docs/reference/rest/v1/PubsubMessage
*
* @author Mike Eltsufin
*/
public class PubSubMessage {
private String data;
private Map<String, String> attributes;
private String messageId;
private String publishTime;
public String getData() {
return data;
}
public void setData(String data) {
this.data = data;
}
public Map<String, String> getAttributes() {
return attributes;
}
public void setAttributes(Map<String, String> attributes) {
this.attributes = attributes;
}
public String getMessageId() {
return messageId;
}
public void setMessageId(String messageId) {
this.messageId = messageId;
}
public String getPublishTime() {
return publishTime;
}
public void setPublishTime(String publishTime) {
this.publishTime = publishTime;
}
}

1
spring-cloud-function-samples/function-sample-gcp-background/src/main/resources/META-INF/MANIFEST.MF Normal file
View File

@@ -0,0 +1 @@
Main-Class: com.example.BackgroundFunctionMain

4
spring-cloud-function-samples/function-sample-gcp/README.adoc → spring-cloud-function-samples/function-sample-gcp-http/README.adoc
View File

@@ -34,7 +34,7 @@ Make sure that you have the https://cloud.google.com/sdk/install[Cloud SDK CLI]
Run the following command from the project root to deploy.
----
gcloud alpha functions deploy function-sample-gcp \
gcloud alpha functions deploy function-sample-gcp-http \
--entry-point org.springframework.cloud.function.adapter.gcp.GcfJarLauncher \
--runtime java11 \
--trigger-http \
@@ -45,5 +45,5 @@ gcloud alpha functions deploy function-sample-gcp \
Invoke the HTTP function:
----
curl https://REGION-PROJECT_ID.cloudfunctions.net/function-sample-gcp -d "hello"
curl https://REGION-PROJECT_ID.cloudfunctions.net/function-sample-gcp-http -d "hello"
----

5
spring-cloud-function-samples/function-sample-gcp/pom.xml → spring-cloud-function-samples/function-sample-gcp-http/pom.xml
View File

@@ -6,11 +6,11 @@
<modelVersion>4.0.0</modelVersion>
<groupId>io.spring.sample</groupId>
<artifactId>function-sample-gcp</artifactId>
<artifactId>function-sample-gcp-http</artifactId>
<version>2.0.0.RELEASE</version>
<packaging>jar</packaging>
<name>function-sample-gcp</name>
<name>function-sample-gcp-http</name>
<parent>
<groupId>org.springframework.boot</groupId>
@@ -77,7 +77,6 @@
</configuration>
</plugin>
</plugins>
</build>
<repositories>

0
spring-cloud-function-samples/function-sample-gcp/src/main/java/com/example/CloudFunctionMain.java → spring-cloud-function-samples/function-sample-gcp-http/src/main/java/com/example/CloudFunctionMain.java
View File

0
spring-cloud-function-samples/function-sample-gcp/src/main/resources/META-INF/MANIFEST.MF → spring-cloud-function-samples/function-sample-gcp-http/src/main/resources/META-INF/MANIFEST.MF
View File

0
spring-cloud-function-samples/function-sample-gcp/src/test/java/com/example/FunctionSampleGcpIntegrationTest.java → spring-cloud-function-samples/function-sample-gcp-http/src/test/java/com/example/FunctionSampleGcpIntegrationTest.java
View File

3
spring-cloud-function-samples/pom.xml
View File

@@ -22,7 +22,8 @@
<module>function-sample-supplier-exporter</module>
<module>function-sample-azure</module>
<!--><module>function-sample-spring-integration</module>-->
<module>function-sample-gcp</module>
<module>function-sample-gcp-http</module>
<module>function-sample-gcp-background</module>
</modules>
<build>