SpringCloud/spring-cloud-dataflow-samples
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Update samples to adapt to M3 changes

Browse Source 
Correct link references
This commit is contained in:
Sabby Anandan
2016-05-02 09:57:00 -07:00
parent d7e484aad2
commit 65f8905b3b
4 changed files with 68 additions and 37 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
analytics/twitter-analytics/README.adoc
View File

@@ -3,9 +3,9 @@
In this demonstration, you will learn how to orchestrate a data pipeline using http://cloud.spring.io/spring-cloud-dataflow/[Spring Cloud Data Flow] to consume data from _TwitterStream_ and compute simple analytics over data-in-transit using _Field-Value-Counter_.
We will begin by discussing the steps to prep, configure and operationalize Spring Cloud Data Flow's `local-server`, a Spring Boot application.
We will begin by discussing the steps to prep, configure and operationalize Spring Cloud Data Flow's `Local` server, a Spring Boot application.
== Using Local SPI
== Using Local Server
=== Prerequisites
@@ -13,6 +13,7 @@ In order to get started, make sure that you have the following components:
* Local build of link:https://github.com/spring-cloud/spring-cloud-dataflow[Spring Cloud Data Flow]
* Running instance of link:http://redis.io/[Redis]
* Running instance of link:http://kafka.apache.org/downloads.html[Kafka]
* Twitter credentials from link:https://apps.twitter.com/[Twitter Developers] site
=== Running the Sample Locally
@@ -155,6 +156,6 @@ image::images/twitter_analytics.png[Twitter Analytics Visualization]
In this sample, you have learned:
* How to use Spring Cloud Data Flow in `local-server` mode
* How to use Spring Cloud Data Flow's `Local` server
* How to use Spring Cloud Data Flow's `shell`
* How to create streaming data pipeline to compute simple analytics using `Twitter Stream` and `Field Value Counter` data microservices

7
datascience/species-prediction/README.adoc
View File

@@ -3,9 +3,9 @@
In this demonstration, you will learn how to use https://en.wikipedia.org/wiki/Predictive_Model_Markup_Language[PMML] model in the context of streaming data pipeline orchestrated by http://cloud.spring.io/spring-cloud-dataflow/[Spring Cloud Data Flow].
We will begin by discussing the steps to prep, configure and operationalize Spring Cloud Data Flow's `local-server`, a Spring Boot application.
We will begin by discussing the steps to prep, configure and operationalize Spring Cloud Data Flow's `Local` server, a Spring Boot application.
== Using Local SPI
== Using Local Server
=== Prerequisites
@@ -13,6 +13,7 @@ In order to get started, make sure that you have the following components:
* Local build of link:https://github.com/spring-cloud/spring-cloud-dataflow[Spring Cloud Data Flow]
* Running instance of link:http://redis.io/[Redis]
* Running instance of link:http://kafka.apache.org/downloads.html[Kafka]
=== Running the Sample Locally
@@ -153,6 +154,6 @@ NOTE: `petalWidth` value changed from `1.5` to `1.8`
In this sample, you have learned:
* How to use Spring Cloud Data Flow in `local-server` mode
* How to use Spring Cloud Data Flow's `Local` server
* How to use Spring Cloud Data Flow's `shell`
* How to use `pmml` processor to compute real-time predictions

45
streaming/http-to-cassandra/README.adoc
View File

@@ -3,9 +3,9 @@
In this demonstration, you will learn how to orchestrate a data pipeline using http://cloud.spring.io/spring-cloud-dataflow/[Spring Cloud Data Flow] to consume data from an _HTTP_ endpoint and write the payload to a _Cassandra_ database.
We will begin by discussing the steps to prep, configure and operationalize Spring Cloud Data Flow's `server` Spring Boot application. We will deploy the `server` using https://github.com/spring-cloud/spring-cloud-dataflow/tree/master/spring-cloud-dataflow-server-local[Local] as well as https://github.com/spring-cloud/spring-cloud-dataflow-admin-cloudfoundry[Cloud Foundry] SPIs (Service Provider Interface) to demonstrate how Spring Cloud Data Flow takes advantage of _dev-sandbox_ and _cloud-native_ platform capabilities respectively.
We will begin by discussing the steps to prep, configure and operationalize Spring Cloud Data Flow's `server` Spring Boot application. We will deploy the `server` using https://github.com/spring-cloud/spring-cloud-dataflow/tree/master/spring-cloud-dataflow-server-local[Local] as well as https://github.com/spring-cloud/spring-cloud-dataflow-server-cloudfoundry[Cloud Foundry] SPIs (Service Provider Interface) to demonstrate how Spring Cloud Data Flow takes advantage of _dev-sandbox_ and _cloud-native_ platform capabilities respectively.
== Using Local SPI
== Using Local Server
=== Prerequisites
@@ -13,6 +13,7 @@ In order to get started, make sure that you have the following components:
* Local build of link:https://github.com/spring-cloud/spring-cloud-dataflow[Spring Cloud Data Flow]
* Running instance of link:http://redis.io/[Redis]
* Running instance of link:http://kafka.apache.org/downloads.html[Kafka]
* Running instance of link:http://cassandra.apache.org/[Apache Cassandra]
* A database utility tool such as link:http://dbeaver.jkiss.org/[DBeaver] to connect to the Cassandra instance. You might have to provide `host`, `port`, `username` and `password` depending on the Cassandra configuration you are using.
* Create a keyspace and a `book` table in Cassandra using:
@@ -108,16 +109,17 @@ select * from clouddata.book;
. That's it; you're done!
== Using Pivotal Cloud Foundry SPI
== Using Cloud Foundry Server
=== Prerequisites for Pivotal Cloud Foundry SPI
=== Prerequisites
In order to get started, make sure that you have the following components:
* Cloud Foundry instance
* Local build of https://github.com/spring-cloud/spring-cloud-dataflow[Spring Cloud Data Flow]
* Local build of Spring Cloud Data Flow's https://github.com/spring-cloud/spring-cloud-dataflow-admin-cloudfoundry[Cloud-Foundry-Server]
* Local build of Spring Cloud Data Flow's https://github.com/spring-cloud/spring-cloud-dataflow-server-cloudfoundry[Cloud Foundry Server]
* Running instance of `redis` in Cloud Foundry
* Running instance of `rabbit` in Cloud Foundry
* Running instance of `cassandra` in Cloud Foundry or from another Cloud provider
* A database utility tool such as link:http://dbeaver.jkiss.org/[DBeaver] to connect to the Cassandra instance. You might have to provide `host`, `port`, `username` and `password` depending on the Cassandra configuration you are using.
* Create a `book` table in your Cassandra keyspace using:
@@ -148,10 +150,10 @@ OK
No apps found
```
+
. Follow the instructions to deploy Spring Cloud Data Flow's `server` from https://github.com/spring-cloud/spring-cloud-dataflow-admin-cloudfoundry/blob/master/README.adoc[CF SPI] repo
. Follow the instructions to deploy Spring Cloud Data Flow's `server` from https://github.com/spring-cloud/spring-cloud-dataflow-server-cloudfoundry/blob/master/README.adoc[Cloud Foundry Server] repo
+
. Once you complete step#3 from https://github.com/spring-cloud/spring-cloud-dataflow-admin-cloudfoundry/blob/master/README.adoc[CF SPI] instructions, you'll be able to list the newly deployed `s-c-dataflow-server` application in Cloud Foundry
. Once you complete step#3 from https://github.com/spring-cloud/spring-cloud-dataflow-server-cloudfoundry/blob/master/README.adoc[Cloud Foundry Server] instructions, you'll be able to list the newly deployed `dataflow-server` application in Cloud Foundry
+
```
@@ -160,11 +162,11 @@ Getting apps in org sabby-dataflow / space development as sabby...
OK
name requested state instances memory disk urls
s-c-dataflow-server started 1/1 1G 1G s-c-dataflow-server.app.navy.springapps.io
dataflow-server started 1/1 1G 1G dataflow-server.app.navy.springapps.io
```
+
. Notice that `s-c-dataflow-server` application is started and ready for interaction via `http://s-c-dataflow-server.app.navy.springapps.io` endpoint
. Notice that `dataflow-server` application is started and ready for interaction via `http://dataflow-server.app.navy.springapps.io` endpoint
. Connect to Spring Cloud Data Flow's `shell`.
+
@@ -190,15 +192,28 @@ Welcome to the Spring Cloud Data Flow shell. For assistance hit TAB or type "hel
server-unknown:>
```
+
. Connect the `shell` with `server` running at `http://s-c-dataflow-server.app.navy.springapps.io`
. Connect the `shell` with `server` running at `http://dataflow-server.app.navy.springapps.io`
+
```
server-unknown:>dataflow config server http://s-c-dataflow-server.app.navy.springapps.io
Successfully targeted http://s-c-dataflow-server.app.navy.springapps.io
server-unknown:>dataflow config server http://dataflow-server.app.navy.springapps.io
Successfully targeted http://dataflow-server.app.navy.springapps.io
dataflow:>version
1.0.0.BUILD-SNAPSHOT
```
+
. https://github.com/spring-cloud/spring-cloud-dataflow/blob/master/spring-cloud-dataflow-docs/src/main/asciidoc/streams.adoc#register-a-stream-app[Register] RabbitMQ binder variant of the `http` and `cassandra` applications
+
```
dataflow:>module register --type source --name http --uri maven://org.springframework.cloud.stream.app:http-source-rabbit:1.0.0.BUILD-SNAPSHOT --force
Successfully registered module 'source:http'
dataflow:>module register --type sink --name cassandra --uri maven://org.springframework.cloud.stream.app:cassandra-sink-rabbit:1.0.0.BUILD-SNAPSHOT --force
Successfully registered module 'sink:cassandra'
```
+
. Create the stream
+
@@ -226,7 +241,7 @@ OK
name requested state instances memory disk urls
cassandrastream-cassandra started 1/1 1G 1G cassandrastream-cassandra.app.navy.springapps.io
cassandrastream-http started 1/1 1G 1G cassandrastream-http.app.navy.springapps.io
s-c-dataflow-server started 1/1 1G 1G s-c-dataflow-server.app.navy.springapps.io
dataflow-server started 1/1 1G 1G dataflow-server.app.navy.springapps.io
```
+
. Lookup the `url` for `cassandrastream-http` application from the list above. Post sample data pointing to the `http` endpoint: `<YOUR-cassandrastream-http-APP-URL>`
@@ -262,7 +277,7 @@ OK
name requested state instances memory disk urls
cassandrastream-cassandra started 1/1 1G 1G cassandrastream-cassandra.app.navy.springapps.io
cassandrastream-http started 3/3 1G 1G cassandrastream-http.app.navy.springapps.io
s-c-dataflow-server started 1/1 1G 1G s-c-dataflow-server.app.navy.springapps.io
dataflow-server started 1/1 1G 1G dataflow-server.app.navy.springapps.io
```
+
. That's it; you're done!
@@ -272,7 +287,7 @@ s-c-dataflow-server started 1/1 1G 1G s-c-da
In this sample, you have learned:
* How to use Spring Cloud Data Flow in `Local` and `Pivotal Cloud Foundry`
* How to use Spring Cloud Data Flow's `Local` and `Cloud Foundry` servers
* How to use Spring Cloud Data Flow's `shell`
* How to create streaming data pipeline to connect and write to `Cassandra`
* How to scale data microservice applications on `Pivotal Cloud Foundry`

46
streaming/http-to-mysql/README.adoc
View File

@@ -1,11 +1,11 @@
:sectnums:
= HTTP to MySQL Demo
In this demonstration, you will learn how to orchestrate a data pipeline using http://cloud.spring.io/spring-cloud-dataflow/[Spring Cloud Data Flow] to consume data from an `http` endpoint and write to MySQL database through `jdbc` sink.
In this demonstration, you will learn how to orchestrate a data pipeline using http://cloud.spring.io/spring-cloud-dataflow/[Spring Cloud Data Flow] to consume data from an `http` endpoint and write to MySQL database using `jdbc` sink.
We will begin by discussing the steps to prep, configure and operationalize Spring Cloud Data Flow's `server` Spring Boot application. We will deploy the `server` using https://github.com/spring-cloud/spring-cloud-dataflow/tree/master/spring-cloud-dataflow-server-local[Local] as well as https://github.com/spring-cloud/spring-cloud-dataflow-admin-cloudfoundry[Cloud Foundry] SPIs (Service Provider Interface) to demonstrate how Spring Cloud Data Flow takes advantage of _dev-sandbox_ and _cloud-native_ platform capabilities respectively.
We will begin by discussing the steps to prep, configure and operationalize Spring Cloud Data Flow's `server` Spring Boot application. We will deploy the `server` using https://github.com/spring-cloud/spring-cloud-dataflow/tree/master/spring-cloud-dataflow-server-local[Local] as well as https://github.com/spring-cloud/spring-cloud-dataflow-server-cloudfoundry[Cloud Foundry] SPIs (Service Provider Interface) to demonstrate how Spring Cloud Data Flow takes advantage of _dev-sandbox_ and _cloud-native_ platform capabilities, respectively.
== Using Local SPI
== Using Local Server
=== Prerequisites
@@ -13,6 +13,7 @@ In order to get started, make sure that you have the following components:
* Local build of https://github.com/spring-cloud/spring-cloud-dataflow[Spring Cloud Data Flow]
* Running instance of link:http://redis.io/[Redis]
* Running instance of link:http://kafka.apache.org/downloads.html[Kafka]
* Running instance of link:http://www.mysql.com/[MySQL]
* A database utility tool such as link:http://dbeaver.jkiss.org/[DBeaver] or link:https://www.dbvis.com/[DbVisualizer]
* Create the `test` database with a `names` table (in MySQL) using:
@@ -28,7 +29,7 @@ CREATE TABLE names
=== Running the Sample Locally
. Launch the locally built `server` application
. Launch the locally built `server`
+
```
@@ -107,7 +108,7 @@ select * from test.names;
+
. That's it; you're done!
== Using Pivotal Cloud Foundry SPI
== Using Cloud Foundry Server
=== Prerequisites
@@ -115,8 +116,9 @@ In order to get started, make sure that you have the following components:
* Cloud Foundry instance
* Local build of https://github.com/spring-cloud/spring-cloud-dataflow[Spring Cloud Data Flow]
* Local build of Spring Cloud Data Flow's https://github.com/spring-cloud/spring-cloud-dataflow-admin-cloudfoundry[Cloud-Foundry-Server]
* Local build of Spring Cloud Data Flow's https://github.com/spring-cloud/spring-cloud-dataflow-server-cloudfoundry[Cloud Foundry Server]
* Running instance of `redis` in Cloud Foundry
* Running instance of `rabbit` in Cloud Foundry
* Running instance of `mysql`
* A database utility tool such as link:http://dbeaver.jkiss.org/[DBeaver] or link:https://www.dbvis.com/[DbVisualizer]
* Create the `names` table (in MySQL) using:
@@ -144,10 +146,10 @@ OK
No apps found
```
+
. Follow the instructions to deploy Spring Cloud Data Flow's `server` from https://github.com/spring-cloud/spring-cloud-dataflow-admin-cloudfoundry/blob/master/README.adoc[CF SPI] repo
. Follow the instructions to deploy Spring Cloud Data Flow's `server` from https://github.com/spring-cloud/spring-cloud-dataflow-server-cloudfoundry/blob/master/README.adoc[Cloud Foundry Server] repo
+
. Once you complete step#3 from https://github.com/spring-cloud/spring-cloud-dataflow-admin-cloudfoundry/blob/master/README.adoc[CF SPI] instructions, you'll be able to list the newly deployed `s-c-dataflow-server` application in Cloud Foundry
. Once you complete step#3 from https://github.com/spring-cloud/spring-cloud-dataflow-server-cloudfoundry/blob/master/README.adoc[Cloud Foundry Server] instructions, you'll be able to list the newly deployed `dataflow-server` application in Cloud Foundry
+
```
@@ -156,11 +158,11 @@ Getting apps in org sabby-dataflow / space development as sabby...
OK
name requested state instances memory disk urls
s-c-dataflow-server started 1/1 1G 1G s-c-dataflow-server.app.navy.springapps.io
dataflow-server started 1/1 1G 1G dataflow-server.app.navy.springapps.io
```
+
. Notice that `s-c-dataflow-server` application is started and ready for interaction via `http://s-c-dataflow-server.app.navy.springapps.io` endpoint
. Notice that `dataflow-server` application is started and ready for interaction via `http://dataflow-server.app.navy.springapps.io` endpoint
. Connect to Spring Cloud Data Flow's `shell`
+
@@ -186,15 +188,27 @@ Welcome to the Spring Cloud Data Flow shell. For assistance hit TAB or type "hel
server-unknown:>
```
+
. Connect the `shell` with `server` running at `http://s-c-dataflow-server.app.navy.springapps.io`
. Connect the `shell` with `server` running at `http://dataflow-server.app.navy.springapps.io`
+
```
server-unknown:>dataflow config server http://s-c-dataflow-server.app.navy.springapps.io
Successfully targeted http://s-c-dataflow-server.app.navy.springapps.io
server-unknown:>dataflow config server http://dataflow-server.app.navy.springapps.io
Successfully targeted http://dataflow-server.app.navy.springapps.io
dataflow:>version
1.0.0.BUILD-SNAPSHOT
```
+
. https://github.com/spring-cloud/spring-cloud-dataflow/blob/master/spring-cloud-dataflow-docs/src/main/asciidoc/streams.adoc#register-a-stream-app[Register] RabbitMQ binder variant of the `http` and `jdbc` applications
+
```
dataflow:>module register --type source --name http --uri maven://org.springframework.cloud.stream.app:http-source-rabbit:1.0.0.BUILD-SNAPSHOT --force
Successfully registered module 'source:http'
dataflow:>module register --type sink --name jdbc --uri maven://org.springframework.cloud.stream.app:jdbc-sink-rabbit:1.0.0.BUILD-SNAPSHOT --force
Successfully registered module 'sink:jdbc'
```
+
. Create the stream
+
@@ -222,7 +236,7 @@ OK
name requested state instances memory disk urls
mysqlstream-http started 1/1 1G 1G mysqlstream-http.app.navy.springapps.io
mysqlstream-jdbc started 1/1 1G 1G mysqlstream-jdbc.app.navy.springapps.io
s-c-dataflow-server started 1/1 1G 1G s-c-dataflow-server.app.navy.springapps.io
dataflow-server started 1/1 1G 1G dataflow-server.app.navy.springapps.io
```
+
. Lookup the `url` for `mysqlstream-http` application from the list above. Post sample data pointing to the `http` endpoint: `<YOUR-mysqlstream-http-APP-URL>`
@@ -258,7 +272,7 @@ OK
name requested state instances memory disk urls
mysqlstream-http started 3/3 1G 1G mysqlstream-http.app.navy.springapps.io
mysqlstream-jdbc started 1/1 1G 1G mysqlstream-jdbc.app.navy.springapps.io
s-c-dataflow-server started 1/1 1G 1G s-c-dataflow-server.app.navy.springapps.io
dataflow-server started 1/1 1G 1G dataflow-server.app.navy.springapps.io
```
+
. That's it; you're done!
@@ -268,7 +282,7 @@ s-c-dataflow-server started 1/1 1G 1G s-c-da
In this sample, you have learned:
* How to use Spring Cloud Data Flow in `Local` and `Pivotal Cloud Foundry`
* How to use Spring Cloud Data Flow's `Local` and `Cloud Foundry` servers
* How to use Spring Cloud Data Flow's `shell`
* How to create streaming data pipeline to connect and write to `MySQL`
* How to scale data microservice applications on `Pivotal Cloud Foundry`