diff --git a/docs/src/main/asciidoc/adapters/gcp-intro.adoc b/docs/src/main/asciidoc/adapters/gcp-intro.adoc index c656366ad..8b1eb3542 100644 --- a/docs/src/main/asciidoc/adapters/gcp-intro.adoc +++ b/docs/src/main/asciidoc/adapters/gcp-intro.adoc @@ -5,7 +5,68 @@ The Google Cloud Functions adapter enables Spring Cloud Function apps to run on the https://cloud.google.com/functions[Google Cloud Functions] serverless platform. You can either run the function locally using the open source https://github.com/GoogleCloudPlatform/functions-framework-java[Google Functions Framework for Java] or on GCP. -==== Getting Started +==== Project Dependencies + +Start by adding the `spring-cloud-function-adapter-gcp` dependency to your project. + +[source, xml] +---- + + + org.springframework.cloud + spring-cloud-function-adapter-gcp + + + ... + +---- + +In addition, add the `spring-boot-maven-plugin` which will build the JAR of the function to deploy. + +NOTE: Notice that we also reference `spring-cloud-function-adapter-gcp` as a dependency of the `spring-boot-maven-plugin`. This is necessary because it modifies the plugin to package your function in the correct JAR format for deployment on Google Cloud Functions. + +[source, xml] +---- + + org.springframework.boot + spring-boot-maven-plugin + + target/deploy + + + + org.springframework.cloud + spring-cloud-function-adapter-gcp + + + +---- + +Finally, add the Maven plugin provided as part of the Google Functions Framework for Java. +This allows you to test your functions locally via `mvn function:run`. + +NOTE: The function target should always be set to `org.springframework.cloud.function.adapter.gcp.GcfJarLauncher`; this is an adapter class which acts as the entry point to your Spring Cloud Function from the Google Cloud Functions platform. + +[source,xml] +---- + + com.google.cloud.functions + function-maven-plugin + 0.9.1 + + org.springframework.cloud.function.adapter.gcp.GcfJarLauncher + 8080 + + +---- + +A full example of a working `pom.xml` can be found in the https://github.com/spring-cloud/spring-cloud-function/blob/master/spring-cloud-function-samples/function-sample-gcp-http/pom.xml[Spring Cloud Functions GCP sample]. + +==== HTTP Functions + +Google Cloud Functions supports deploying https://cloud.google.com/functions/docs/writing/http[HTTP Functions], which are functions that are invoked by HTTP request. The sections below describe instructions for deploying a Spring Cloud Function as an HTTP Function. + +===== Getting Started Let’s start with a simple Spring Cloud Function example: @@ -25,23 +86,6 @@ public class CloudFunctionMain { } ---- -===== Test locally - -Start by adding the Maven plugin provided as part of the Google Functions Framework for Java. - -[source,xml] ----- - - com.google.cloud.functions - function-maven-plugin - 0.9.1 - - org.springframework.cloud.function.adapter.gcp.GcfJarLauncher - 8080 - - ----- - Specify your configuration main class in `resources/META-INF/MANIFEST.MF`. [source] @@ -49,7 +93,8 @@ Specify your configuration main class in `resources/META-INF/MANIFEST.MF`. Main-Class: com.example.CloudFunctionMain ---- -Then run the function: +Then run the function locally. +This is provided by the Google Cloud Functions `function-maven-plugin` described in the project dependencies section. ---- mvn function:run @@ -66,51 +111,21 @@ curl http://localhost:8080/ -d "hello" As of March 2020, Google Cloud Functions for Java is in Alpha. You can get on the https://docs.google.com/forms/d/e/1FAIpQLScC98jGi7CfG0n3UYlj7Xad8XScvZC8-BBOg7Pk3uSZx_2cdQ/viewform[whitelist] to try it out. -In order to use the adapter, first add the dependency to your pom.xml: - -[source, xml] ----- - - org.springframework.cloud - spring-cloud-function-adapter-gcp - ----- - -Then, add the `spring-boot-maven-plugin` with `spring-cloud-function-adapter-gcp` as a dependency. -The extra dependency is used for `spring-boot-maven-plugin` to package your function in the correct JAR format for deployment on Google Cloud Functions. - -[source, xml] ----- - - org.springframework.boot - spring-boot-maven-plugin - - target/deploy - - - - org.springframework.cloud - spring-cloud-function-adapter-gcp - - - ----- - -Package the application. +Start by packaging your application. ---- mvn package ---- -You should see the resulting JAR in `target/deploy` directory. +If you added the custom `spring-boot-maven-plugin` plugin defined above, you should see the resulting JAR in `target/deploy` directory. This JAR is correctly formatted for deployment to Google Cloud Functions. -Make sure that you have the https://cloud.google.com/sdk/install[Cloud SDK CLI] installed. +Next, make sure that you have the https://cloud.google.com/sdk/install[Cloud SDK CLI] installed. From the project base directory run the following command to deploy. ---- -gcloud alpha functions deploy function-sample-gcp \ +gcloud alpha functions deploy function-sample-gcp-http \ --entry-point org.springframework.cloud.function.adapter.gcp.GcfJarLauncher \ --runtime java11 \ --trigger-http \ @@ -121,9 +136,141 @@ gcloud alpha functions deploy function-sample-gcp \ Invoke the HTTP function: ---- -curl https://REGION-PROJECT_ID.cloudfunctions.net/function-sample-gcp -d "hello" +curl https://REGION-PROJECT_ID.cloudfunctions.net/function-sample-gcp-http -d "hello" ---- -==== Sample Function +==== Background Functions -Go to the https://github.com/spring-cloud/spring-cloud-function/tree/master/spring-cloud-function-samples/function-sample-gcp/[function-sample-gcp] to try out a sample function that you can test locally or deploy to GCP. +Google Cloud Functions also supports deploying https://cloud.google.com/functions/docs/writing/background[Background Functions] which are invoked indirectly in response to an event, such as a message on a https://cloud.google.com/pubsub[Cloud Pub/Sub] topic, a change in a https://cloud.google.com/storage[Cloud Storage] bucket, or a https://firebase.google.com/[Firebase] event. + +The `spring-cloud-function-adapter-gcp` allows for functions to be deployed as background functions as well. + +The sections below describe the process for writing a Cloud Pub/Sub topic background function. +However, there are a number of different event types that can trigger a background function to execute which are not discussed here; these are described in the https://cloud.google.com/functions/docs/calling[Background Function triggers documentation]. + +===== Getting Started + +Let’s start with a simple Spring Cloud Function which will run as a GCF background function: + +[source, java] +---- +@SpringBootApplication +public class BackgroundFunctionMain { + + public static void main(String[] args) { + SpringApplication.run(BackgroundFunctionMain.class, args); + } + + @Bean + public Consumer pubSubFunction() { + return message -> System.out.println("The Pub/Sub message data: " + message.getData()); + } +} +---- + +In addition, create `PubSubMessage` class in the project with the below definition. +This class represents the https://cloud.google.com/functions/docs/calling/pubsub#event_structure[Pub/Sub event structure] which gets passed to your function on a Pub/Sub topic event. + +[source, java] +---- +public class PubSubMessage { + + private String data; + + private Map attributes; + + private String messageId; + + private String publishTime; + + public String getData() { + return data; + } + + public void setData(String data) { + this.data = data; + } + + public Map getAttributes() { + return attributes; + } + + public void setAttributes(Map attributes) { + this.attributes = attributes; + } + + public String getMessageId() { + return messageId; + } + + public void setMessageId(String messageId) { + this.messageId = messageId; + } + + public String getPublishTime() { + return publishTime; + } + + public void setPublishTime(String publishTime) { + this.publishTime = publishTime; + } + +} +---- + +Specify your configuration main class in `resources/META-INF/MANIFEST.MF`. + +[source] +---- +Main-Class: com.example.BackgroundFunctionMain +---- + +Then run the function locally. +This is provided by the Google Cloud Functions `function-maven-plugin` described in the project dependencies section. + +---- +mvn function:run +---- + +Invoke the HTTP function: + +---- +curl localhost:8080 -H "Content-Type: application/json" -d '{"data":"hello"}' +---- + +Verify that the function was invoked by viewing the logs. + +===== Deploy to GCP + +In order to deploy your background function to GCP, first package your application. + +---- +mvn package +---- + +If you added the custom `spring-boot-maven-plugin` plugin defined above, you should see the resulting JAR in `target/deploy` directory. +This JAR is correctly formatted for deployment to Google Cloud Functions. + +Next, make sure that you have the https://cloud.google.com/sdk/install[Cloud SDK CLI] installed. + +From the project base directory run the following command to deploy. + +---- +gcloud alpha functions deploy function-sample-gcp-background \ +--entry-point org.springframework.cloud.function.adapter.gcp.GcfJarLauncher \ +--runtime java11 \ +--trigger-topic my-functions-topic \ +--source target/deploy \ +--memory 512MB +---- + +Google Cloud Function will now invoke the function every time a message is published to the topic specified by `--trigger-topic`. + +For a walkthrough on testing and verifying your background function, see the instructions for running the https://github.com/spring-cloud/spring-cloud-function/tree/master/spring-cloud-function-samples/function-sample-gcp-background/[GCF Background Function sample]. + +==== Sample Functions + +The project provides the following sample functions as reference: + +* The https://github.com/spring-cloud/spring-cloud-function/tree/master/spring-cloud-function-samples/function-sample-gcp-http/[function-sample-gcp-http] is an HTTP Function which you can test locally and try deploying. +* The https://github.com/spring-cloud/spring-cloud-function/tree/master/spring-cloud-function-samples/function-sample-gcp-background/[function-sample-gcp-background] shows an example of a background function that is triggered by a message being published to a specified Pub/Sub topic. diff --git a/spring-cloud-function-adapters/spring-cloud-function-adapter-gcp/src/main/java/org/springframework/cloud/function/adapter/gcp/GcfJarLauncher.java b/spring-cloud-function-adapters/spring-cloud-function-adapter-gcp/src/main/java/org/springframework/cloud/function/adapter/gcp/GcfJarLauncher.java index 9e7a1d393..6588fd890 100644 --- a/spring-cloud-function-adapters/spring-cloud-function-adapter-gcp/src/main/java/org/springframework/cloud/function/adapter/gcp/GcfJarLauncher.java +++ b/spring-cloud-function-adapters/spring-cloud-function-adapter-gcp/src/main/java/org/springframework/cloud/function/adapter/gcp/GcfJarLauncher.java @@ -16,9 +16,11 @@ package org.springframework.cloud.function.adapter.gcp; +import com.google.cloud.functions.Context; import com.google.cloud.functions.HttpFunction; import com.google.cloud.functions.HttpRequest; import com.google.cloud.functions.HttpResponse; +import com.google.cloud.functions.RawBackgroundFunction; import org.springframework.boot.loader.JarLauncher; import org.springframework.boot.loader.jar.JarFile; @@ -30,11 +32,11 @@ import org.springframework.boot.loader.jar.JarFile; * @author Ray Tsang * @author Daniel Zou */ -public class GcfJarLauncher extends JarLauncher implements HttpFunction { +public class GcfJarLauncher extends JarLauncher implements HttpFunction, RawBackgroundFunction { private final ClassLoader loader; - private final HttpFunction delegate; + private final Object delegate; public GcfJarLauncher() throws Exception { JarFile.registerUrlProtocolHandler(); @@ -43,12 +45,18 @@ public class GcfJarLauncher extends JarLauncher implements HttpFunction { Class clazz = this.loader .loadClass("org.springframework.cloud.function.adapter.gcp.FunctionInvoker"); - this.delegate = (HttpFunction) clazz.getConstructor().newInstance(); + this.delegate = clazz.getConstructor().newInstance(); } @Override public void service(HttpRequest httpRequest, HttpResponse httpResponse) throws Exception { Thread.currentThread().setContextClassLoader(this.loader); - delegate.service(httpRequest, httpResponse); + ((HttpFunction) delegate).service(httpRequest, httpResponse); + } + + @Override + public void accept(String json, Context context) { + Thread.currentThread().setContextClassLoader(this.loader); + ((RawBackgroundFunction) delegate).accept(json, context); } } diff --git a/spring-cloud-function-samples/function-sample-gcp-background/README.adoc b/spring-cloud-function-samples/function-sample-gcp-background/README.adoc new file mode 100644 index 000000000..a2f4fc587 --- /dev/null +++ b/spring-cloud-function-samples/function-sample-gcp-background/README.adoc @@ -0,0 +1,107 @@ +:branch: master + +=== Google Cloud Background Functions Sample Application + +Google Cloud Functions supports deploying https://cloud.google.com/functions/docs/writing/background[Background Functions] which are invoked indirectly in response to an event, such as a message on a https://cloud.google.com/pubsub[Pub/Sub] topic, a change in a https://cloud.google.com/storage[Cloud Storage] bucket, or a https://firebase.google.com/[Firebase] event. + +This sample demonstrates a simple background function which triggers from a Pub/Sub event. + +===== Test locally + +In a terminal window, run: + +---- +mvn function:run +---- + +In a separate window, invoke the background function by issuing the `curl` command below. + +NOTE: In this sample, we demonstrate a background function which is triggered when a message is https://cloud.google.com/functions/docs/calling/pubsub[published to a specified Cloud Pub/Sub topic]. +The `curl` request simulates sending the message that would be received by your function when a Pub/Sub event occurs. + +In order to simulate how Pub/Sub will send you this message, we use a base64-encoded string as the input because Pub/Sub encodes the message data in base64 in the Pub/Sub event. +See the notes section below for a more information on the Pub/Sub event structure. + +---- +curl localhost:8080 -H "Content-Type: application/json" -d '{"data":"aGVsbG8="}' +---- + +In the original terminal window where the `mvn function:run` was invoked, you should see a message printed. + +---- +Received Pub/Sub message with data: hello +---- + +===== Deploy to GCP + +As of March 2020, Google Cloud Functions for Java is in Alpha. +You can get on the https://docs.google.com/forms/d/e/1FAIpQLScC98jGi7CfG0n3UYlj7Xad8XScvZC8-BBOg7Pk3uSZx_2cdQ/viewform[whitelist] to try it out. + +The next steps will demonstrate how to deploy your background function to GCP and have it be invoked by a Pub/Sub event. + +To complete the next steps, make sure that you have the https://cloud.google.com/sdk/install[Cloud SDK CLI] installed. + +1. Create a new Pub/Sub topic. This topic will be used as the trigger for the background function. ++ +---- +gcloud pubsub topics create my-functions-topic +---- + +2. In this sample directory, package the application by running: ++ +---- +mvn package +---- ++ +You should see the JAR to deploy in the `target/deploy` directory. + +3. Deploy the JAR that you created by running ++ +---- +gcloud alpha functions deploy function-sample-gcp-background \ +--entry-point org.springframework.cloud.function.adapter.gcp.GcfJarLauncher \ +--runtime java11 \ +--trigger-topic my-functions-topic \ +--source target/deploy \ +--memory 512MB +---- ++ +Notice the parameter `--trigger-topic` which topic will trigger the function invocation when new messages are published to it. + +4. Invoke the background function by publishing a message to your topic. ++ +---- +gcloud pubsub topics publish my-functions-topic --message="Hello world" +---- + +5. To verify that this sample background function was invoked, check the logs of the background function on GCF by running: ++ +---- +gcloud functions logs read function-sample-gcp-background --filter=Received +---- ++ +You should see a log message that looks like below. +It might take a minute or two for the log to appear. ++ +---- +function-sample-gcp-background-2 h8q1jt46069r 2020-05-19 19:48:27.960 Received Pub/Sub message with data: Hello world +---- ++ +This log output is produced by the sample background function, so this entry in the logs means that the background function was successfully invoked. + +===== Notes + +One important note about the Pub/Sub background function provided in `BackgroundFunctionMain` is that the `data` field must be decoded from base64 encoding because Pub/Sub will encode the published message in base64 according to the https://cloud.google.com/functions/docs/calling/pubsub#event_structure[PubSubMessage event structure]. + +[source, java] +---- +@Bean +public Consumer pubSubFunction() { + return message -> { + // The PubSubMessage data field arrives as a base-64 encoded string and must be decoded. + // See: https://cloud.google.com/functions/docs/calling/pubsub#event_structure + String decodedMessage = new String(Base64.decode(message.getData()), StandardCharsets.UTF_8); + System.out.println("Received Pub/Sub message with data: " + decodedMessage); + }; +} +---- \ No newline at end of file diff --git a/spring-cloud-function-samples/function-sample-gcp-background/pom.xml b/spring-cloud-function-samples/function-sample-gcp-background/pom.xml new file mode 100644 index 000000000..c9ba1efb5 --- /dev/null +++ b/spring-cloud-function-samples/function-sample-gcp-background/pom.xml @@ -0,0 +1,58 @@ + + + + 4.0.0 + + io.spring.sample + function-sample-gcp-background + 2.0.0.RELEASE + jar + + function-sample-gcp-background + + + org.springframework.boot + spring-boot-starter-parent + 2.3.0.BUILD-SNAPSHOT + + + + + + org.springframework.cloud + spring-cloud-function-adapter-gcp + 3.1.0.BUILD-SNAPSHOT + + + + + + + org.springframework.boot + spring-boot-maven-plugin + + target/deploy + + + + org.springframework.cloud + spring-cloud-function-adapter-gcp + 3.1.0.BUILD-SNAPSHOT + + + + + + com.google.cloud.functions + function-maven-plugin + 0.9.1 + + org.springframework.cloud.function.adapter.gcp.GcfJarLauncher + 8080 + + + + + diff --git a/spring-cloud-function-samples/function-sample-gcp-background/src/main/java/com/example/BackgroundFunctionMain.java b/spring-cloud-function-samples/function-sample-gcp-background/src/main/java/com/example/BackgroundFunctionMain.java new file mode 100644 index 000000000..753fe6322 --- /dev/null +++ b/spring-cloud-function-samples/function-sample-gcp-background/src/main/java/com/example/BackgroundFunctionMain.java @@ -0,0 +1,31 @@ +package com.example; + +import java.nio.charset.StandardCharsets; +import java.util.Base64; +import java.util.function.Consumer; + +import org.springframework.boot.SpringApplication; +import org.springframework.boot.autoconfigure.SpringBootApplication; +import org.springframework.context.annotation.Bean; + +@SpringBootApplication +public class BackgroundFunctionMain { + + public static void main(String[] args) { + SpringApplication.run(BackgroundFunctionMain.class, args); + } + + /** + * The background function which triggers on an event from Pub/Sub and consumes the Pub/Sub + * event message. + */ + @Bean + public Consumer pubSubFunction() { + return message -> { + // The PubSubMessage data field arrives as a base-64 encoded string and must be decoded. + // See: https://cloud.google.com/functions/docs/calling/pubsub#event_structure + String decodedMessage = new String(Base64.getDecoder().decode(message.getData()), StandardCharsets.UTF_8); + System.out.println("Received Pub/Sub message with data: " + decodedMessage); + }; + } +} diff --git a/spring-cloud-function-samples/function-sample-gcp-background/src/main/java/com/example/PubSubMessage.java b/spring-cloud-function-samples/function-sample-gcp-background/src/main/java/com/example/PubSubMessage.java new file mode 100644 index 000000000..01b5084fa --- /dev/null +++ b/spring-cloud-function-samples/function-sample-gcp-background/src/main/java/com/example/PubSubMessage.java @@ -0,0 +1,56 @@ +package com.example; + +import java.util.Map; + +/** + * A class that can be mapped to the GCF Pub/Sub Message event type. This is for use in + * the background functions. + * + *

See the PubSubMessage definition for reference: + * https://cloud.google.com/pubsub/docs/reference/rest/v1/PubsubMessage + * + * @author Mike Eltsufin + */ +public class PubSubMessage { + + private String data; + + private Map attributes; + + private String messageId; + + private String publishTime; + + public String getData() { + return data; + } + + public void setData(String data) { + this.data = data; + } + + public Map getAttributes() { + return attributes; + } + + public void setAttributes(Map attributes) { + this.attributes = attributes; + } + + public String getMessageId() { + return messageId; + } + + public void setMessageId(String messageId) { + this.messageId = messageId; + } + + public String getPublishTime() { + return publishTime; + } + + public void setPublishTime(String publishTime) { + this.publishTime = publishTime; + } + +} diff --git a/spring-cloud-function-samples/function-sample-gcp-background/src/main/resources/META-INF/MANIFEST.MF b/spring-cloud-function-samples/function-sample-gcp-background/src/main/resources/META-INF/MANIFEST.MF new file mode 100644 index 000000000..edd9a3bf2 --- /dev/null +++ b/spring-cloud-function-samples/function-sample-gcp-background/src/main/resources/META-INF/MANIFEST.MF @@ -0,0 +1 @@ +Main-Class: com.example.BackgroundFunctionMain diff --git a/spring-cloud-function-samples/function-sample-gcp/README.adoc b/spring-cloud-function-samples/function-sample-gcp-http/README.adoc similarity index 92% rename from spring-cloud-function-samples/function-sample-gcp/README.adoc rename to spring-cloud-function-samples/function-sample-gcp-http/README.adoc index 000f0aa3c..385e9123f 100644 --- a/spring-cloud-function-samples/function-sample-gcp/README.adoc +++ b/spring-cloud-function-samples/function-sample-gcp-http/README.adoc @@ -34,7 +34,7 @@ Make sure that you have the https://cloud.google.com/sdk/install[Cloud SDK CLI] Run the following command from the project root to deploy. ---- -gcloud alpha functions deploy function-sample-gcp \ +gcloud alpha functions deploy function-sample-gcp-http \ --entry-point org.springframework.cloud.function.adapter.gcp.GcfJarLauncher \ --runtime java11 \ --trigger-http \ @@ -45,5 +45,5 @@ gcloud alpha functions deploy function-sample-gcp \ Invoke the HTTP function: ---- -curl https://REGION-PROJECT_ID.cloudfunctions.net/function-sample-gcp -d "hello" +curl https://REGION-PROJECT_ID.cloudfunctions.net/function-sample-gcp-http -d "hello" ---- diff --git a/spring-cloud-function-samples/function-sample-gcp/pom.xml b/spring-cloud-function-samples/function-sample-gcp-http/pom.xml similarity index 97% rename from spring-cloud-function-samples/function-sample-gcp/pom.xml rename to spring-cloud-function-samples/function-sample-gcp-http/pom.xml index 02463d804..7229cff48 100644 --- a/spring-cloud-function-samples/function-sample-gcp/pom.xml +++ b/spring-cloud-function-samples/function-sample-gcp-http/pom.xml @@ -6,11 +6,11 @@ 4.0.0 io.spring.sample - function-sample-gcp + function-sample-gcp-http 2.0.0.RELEASE jar - function-sample-gcp + function-sample-gcp-http org.springframework.boot @@ -77,7 +77,6 @@ - diff --git a/spring-cloud-function-samples/function-sample-gcp/src/main/java/com/example/CloudFunctionMain.java b/spring-cloud-function-samples/function-sample-gcp-http/src/main/java/com/example/CloudFunctionMain.java similarity index 100% rename from spring-cloud-function-samples/function-sample-gcp/src/main/java/com/example/CloudFunctionMain.java rename to spring-cloud-function-samples/function-sample-gcp-http/src/main/java/com/example/CloudFunctionMain.java diff --git a/spring-cloud-function-samples/function-sample-gcp/src/main/resources/META-INF/MANIFEST.MF b/spring-cloud-function-samples/function-sample-gcp-http/src/main/resources/META-INF/MANIFEST.MF similarity index 100% rename from spring-cloud-function-samples/function-sample-gcp/src/main/resources/META-INF/MANIFEST.MF rename to spring-cloud-function-samples/function-sample-gcp-http/src/main/resources/META-INF/MANIFEST.MF diff --git a/spring-cloud-function-samples/function-sample-gcp/src/test/java/com/example/FunctionSampleGcpIntegrationTest.java b/spring-cloud-function-samples/function-sample-gcp-http/src/test/java/com/example/FunctionSampleGcpIntegrationTest.java similarity index 100% rename from spring-cloud-function-samples/function-sample-gcp/src/test/java/com/example/FunctionSampleGcpIntegrationTest.java rename to spring-cloud-function-samples/function-sample-gcp-http/src/test/java/com/example/FunctionSampleGcpIntegrationTest.java diff --git a/spring-cloud-function-samples/pom.xml b/spring-cloud-function-samples/pom.xml index f98633316..4b793b8f7 100644 --- a/spring-cloud-function-samples/pom.xml +++ b/spring-cloud-function-samples/pom.xml @@ -24,8 +24,9 @@ function-sample-aws-custom function-sample-supplier-exporter function-sample-azure - function-sample-spring-integration - function-sample-gcp + function-sample-spring-integration--> + function-sample-gcp-http + function-sample-gcp-background