Update instructions with latest CF and SCDF support

Fixes https://github.com/spring-cloud/spring-cloud-dataflow-samples/issues/22

polish
This commit is contained in:
Eric Bottard
2017-05-04 17:06:43 +02:00
committed by Glenn Renfro
parent cb052f9b29
commit 337b41ecaa
3 changed files with 59 additions and 26 deletions

1
.gitignore vendored Normal file
View File

@@ -0,0 +1 @@
.DS_Store

View File

@@ -10,7 +10,7 @@ In this demonstration, you will learn how to orchestrate short-lived data proces
In order to get started, make sure that you have the following components:
* Local https://pivotal.io/pcf-dev[PCFDev] instance
* Local install of https://github.com/cloudfoundry/v3-cli-plugin[V3-CF-CLI] plugin
* Local install of https://github.com/cloudfoundry/cli[cf CLI] command line tool
* Local build of https://github.com/spring-cloud/spring-cloud-dataflow[Spring Cloud Data Flow]
* Local build of Spring Cloud Data Flow's https://github.com/spring-cloud/spring-cloud-dataflow-server-cloudfoundry[Cloud Foundry Server]
* Running instance of mysql in PCFDev
@@ -23,8 +23,9 @@ NOTE: PCF 1.7.12 or greater is required to run Tasks on Spring Cloud Data Flow.
+
```
→ cf api
API endpoint: https://api.local.pcfdev.io (API version: 2.54.0)
cf api
api endpoint: https://api.local.pcfdev.io
api version: 2.75.0
$ cf apps
Getting apps in org user-dataflow / space development as user...
@@ -33,13 +34,24 @@ OK
No apps found
```
+
. Task support needs to be enabled on pcf-dev. Being logged as `admin`, issue the following command:
+
```
cf enable-feature-flag task_creation
Setting status of task_creation as admin...
OK
Feature task_creation Enabled.
```
+
. Follow the http://docs.spring.io/spring-cloud-dataflow-server-cloudfoundry/docs/current-SNAPSHOT/reference/htmlsingle/#getting-started[getting-started] instructions to deploy Spring Cloud Data Flow's Cloud Foundry Server
+
NOTE: For this sample, all you need is the `mysql` service and in PCFDev, the `mysql` service comes with a different plan. From CF CLI, create the service by: `cf create-service p-mysql 512mb mysql` and bind this service to `dataflow-server` by: `cf bind-service dataflow-server mysql`.
+
NOTE: All the apps deployed to PCFDev start with low memory by default. It is recommended to change it to at least 512MB for `dataflow-server`. Change the memory by: `cf set-env dataflow-server SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_MEMORY 512`. Likewise, we would have to skip SSL validation by: `cf set-env dataflow-server SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_SKIP_SSL_VALIDATION true`.
NOTE: All the apps deployed to PCFDev start with low memory by default. It is recommended to change it to at least 768MB for `dataflow-server`. Ditto for every app spawned *by* Spring Cloud Data Flow. Change the memory by: `cf set-env dataflow-server SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_MEMORY 512`. Likewise, we would have to skip SSL validation by: `cf set-env dataflow-server SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_SKIP_SSL_VALIDATION true`.
. Once you complete the instructions, you'll be able to list the newly deployed `dataflow-server`
@@ -57,7 +69,7 @@ dataflow-server started 1/1 512M 512M dataflow-server.l
NOTE: The `ORG`, `SPACE` and `USER` could vary depending on your Cloud Foundry environment in use.
. Tasks in Spring Cloud Data Flow requires an RDBMS to host "task repository" (see http://docs.spring.io/spring-cloud-dataflow/docs/1.0.0.RELEASE/reference/htmlsingle/#spring-cloud-dataflow-task-repository[here] for more details), so let's bind `mysql` service and restage the `dataflow-server`
. Tasks in Spring Cloud Data Flow require an RDBMS to host "task repository" (see http://docs.spring.io/spring-cloud-dataflow/docs/current-SNAPSHOT/reference/htmlsingle/#spring-cloud-dataflow-task-repository[here] for more details), so let's instruct the Spring Cloud Data Flow server to bind the `mysql` service to each deployed task:
+
@@ -69,6 +81,29 @@ $ cf restage dataflow-server
NOTE: We only need `mysql` service for this sample.
. As a recap, here is what you should see as configuration for the Spring Cloud Data Flow server:
+
```
cf env dataflow-server
....
User-Provided:
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_DOMAIN: local.pcfdev.io
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_MEMORY: 512
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_ORG: pcfdev-org
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_PASSWORD: pass
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_SKIP_SSL_VALIDATION: false
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_SPACE: pcfdev-space
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_TASK_SERVICES: mysql
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_URL: https://api.local.pcfdev.io
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_USERNAME: user
No running env variables have been set
No staging env variables have been set
```
+
. Notice that `dataflow-server` application is started and ready for interaction via `http://dataflow-server.local.pcfdev.io` endpoint
. Connect to Spring Cloud Data Flow's `shell`.
@@ -101,17 +136,6 @@ server-unknown:>
```
server-unknown:>dataflow config server http://dataflow-server.local.pcfdev.io
Successfully targeted http://dataflow-server.local.pcfdev.io
dataflow:>version
<VERSION>
```
+
. Verify there's no Task applications running on PCFDev using `v3-cf-cli`
+
```
→ cf v3-apps
No v3 apps found.
```
+
@@ -120,7 +144,7 @@ No v3 apps found.
+
```
dataflow:>app register --type task --name simple_batch_job --uri https://github.com/spring-cloud/spring-cloud-dataflow-samples/raw/master/tasks/simple-batch-job/batch-job-1.0.0.BUILD-SNAPSHOT.jar
dataflow:>app register --type task --name simple_batch_job --uri https://github.com/spring-cloud/spring-cloud-dataflow-samples/raw/master/tasks/simple-batch-job/batch-job-1.3.0.BUILD-SNAPSHOT.jar
```
+
@@ -138,8 +162,12 @@ NOTE: Unlike Streams, the Task definitions don't require explicit deployment. Th
+
```
cf v3-apps
No v3 apps found.
cf apps
Getting apps in org pcfdev-org / space pcfdev-space as user...
OK
name requested state instances memory disk urls
dataflow-server started 1/1 768M 512M dataflow-server.local.pcfdev.io
```
+
@@ -157,8 +185,8 @@ dataflow:>task launch foo
+
```
→ cf v3-logs foo
Tailing logs for app foo...
→ cf logs foo
Retrieving logs for app foo in org pcfdev-org / space pcfdev-space as user...
2016-08-14T18:48:54.22-0700 [APP/TASK/foo/0]OUT Creating container
2016-08-14T18:48:55.47-0700 [APP/TASK/foo/0]OUT
@@ -200,9 +228,13 @@ NOTE: Unlike LRPs in Cloud Foundry, tasks are short-lived, so the logs aren't al
+
```
→ cf v3-apps
name total_desired_instances
foo 0
→ cf apps
Getting apps in org pcfdev-org / space pcfdev-space as user...
OK
name requested state instances memory disk urls
dataflow-server started 1/1 768M 512M dataflow-server.local.pcfdev.io
foo stopped 0/1 1G 1G
```
+
@@ -215,7 +247,7 @@ dataflow:>task execution list
╔══════════════════════════╤══╤════════════════════════════╤════════════════════════════╤═════════╗
║ Task Name │ID│ Start Time │ End Time │Exit Code║
╠══════════════════════════╪══╪════════════════════════════╪════════════════════════════╪═════════╣
Demo Batch Job Task:cloud:│1 │Sun Aug 14 18:49:05 PDT 2016│Sun Aug 14 18:49:07 PDT 2016│0 ║
foo │1 │Sun Aug 14 18:49:05 PDT 2016│Sun Aug 14 18:49:07 PDT 2016│0 ║
╚══════════════════════════╧══╧════════════════════════════╧════════════════════════════╧═════════╝
```
@@ -240,5 +272,5 @@ dataflow:>job execution list
In this sample, you have learned:
* How to register and orchestrate Spring Batch jobs in Spring Cloud Data Flow
* How to use `v3-cf-cli` in the context of Task applications orchestrated by Spring Cloud Data Flow
* How to use the `cf` CLI in the context of Task applications orchestrated by Spring Cloud Data Flow
* How to verify task executions and task repository