From 5d54cb280db4bc2bec481576263e9eef4a893c55 Mon Sep 17 00:00:00 2001 From: Sabby Anandan Date: Tue, 17 Oct 2017 13:59:46 -0700 Subject: [PATCH] Add scfn + scdf integration sample - add a new folder for scfn + scdf integration - include quick-link from the root README - replace spring-cloud-stream-modules with spring-cloud-stream-app-starters - adjust text content for better continuity - replace older bit.ly's with bacon release-train Add function-runner repo link for clarity --- README.adoc | 8 +- analytics/twitter-analytics/README.adoc | 24 ++-- datascience/species-prediction/README.adoc | 18 +-- functions/README.adoc | 155 +++++++++++++++++++++ streaming/http-to-cassandra/README.adoc | 20 +-- streaming/http-to-mysql/README.adoc | 18 +-- 6 files changed, 202 insertions(+), 41 deletions(-) create mode 100644 functions/README.adoc diff --git a/README.adoc b/README.adoc index 04dabe6..ddab21a 100644 --- a/README.adoc +++ b/README.adoc @@ -18,6 +18,12 @@ A data pipeline demonstration that consumes data from an `http` endpoint and wri A simple Spring Batch job running as short-lived task in Cloud Foundry. +## Functions + +### link:functions/README.adoc[functions-in-scdf] + +A simple demonstration of Spring Cloud Function and Spring Cloud Data Flow integration. + ## Analytics ### link:analytics/twitter-analytics/README.adoc[twitter-analytics] @@ -28,4 +34,4 @@ A data pipeline demonstration that consumes data from twitter-firehose using `tw ### link:datascience/species-prediction/README.adoc[species-prediction] -A simple demonstration to walkthrough the steps to compute real-time predictions using https://en.wikipedia.org/wiki/Predictive_Model_Markup_Language[PMML] data microservice application. +A simple demonstration to walkthrough the steps to compute real-time predictions using https://en.wikipedia.org/wiki/Predictive_Model_Markup_Language[PMML] application. diff --git a/analytics/twitter-analytics/README.adoc b/analytics/twitter-analytics/README.adoc index 3a13d39..54d1720 100644 --- a/analytics/twitter-analytics/README.adoc +++ b/analytics/twitter-analytics/README.adoc @@ -1,15 +1,15 @@ :sectnums: = Twitter Analytics -In this demonstration, you will learn how to orchestrate a data pipeline using http://cloud.spring.io/spring-cloud-dataflow/[Spring Cloud Data Flow] to consume data from _TwitterStream_ and compute simple analytics over data-in-transit using _Field-Value-Counter_. +In this demonstration, you will learn how to orchestrate a data pipeline using http://cloud.spring.io/spring-cloud-dataflow/[Spring Cloud Data Flow] to consume data from _TwitterStream_ and compute simple analytics over data-in-transit using _Field-Value-Counter_. -We will begin by discussing the steps to prep, configure and operationalize Spring Cloud Data Flow's `Local` server, a Spring Boot application. +We will begin by discussing the steps to prep, configure and operationalize Spring Cloud Data Flow's `Local` server. == Using Local Server === Prerequisites -In order to get started, make sure that you have the following components: +Make sure that you have the following components: * Local build of link:https://github.com/spring-cloud/spring-cloud-dataflow[Spring Cloud Data Flow] * Running instance of link:http://redis.io/[Redis] @@ -56,7 +56,7 @@ dataflow:>version + ``` -dataflow:>app import --uri http://bit.ly/Bacon-RELEASE-stream-applications-kafka-10-maven +dataflow:>app import --uri http://bit.ly/Bacon-RELEASE-stream-applications-kafka-10-maven ``` + @@ -75,7 +75,7 @@ Created and deployed new stream 'tagcount' (4) dataflow:>stream deploy tweets Deployed stream 'tweets' ``` -NOTE: To get a consumerKey and consumerSecret you need to register a twitter application. If you don’t already have one set up, you can create an app at the link:https://apps.twitter.com/[Twitter Developers] site to get these credentials. The tokens ``, ``, ``, and `` are required to be replaced with your account credentials. +NOTE: To get a consumerKey and consumerSecret you need to register a twitter application. If you don’t already have one set up, you can create an app at the link:https://apps.twitter.com/[Twitter Developers] site to get these credentials. The tokens ``, ``, ``, and `` are required to be replaced with your account credentials. + . Verify the streams are successfully deployed. Where: (1) is the primary pipeline; (2) and (3) are tapping the primary pipeline with the DSL syntax `.