Update instructions with latest CF and SCDF support

Fixes https://github.com/spring-cloud/spring-cloud-dataflow-samples/issues/22

polish
This commit is contained in:
Eric Bottard
2017-05-04 17:06:43 +02:00
committed by Glenn Renfro
parent cb052f9b29
commit 337b41ecaa
3 changed files with 59 additions and 26 deletions

1
.gitignore vendored Normal file
View File

@@ -0,0 +1 @@
.DS_Store

View File

@@ -10,7 +10,7 @@ In this demonstration, you will learn how to orchestrate short-lived data proces
In order to get started, make sure that you have the following components: In order to get started, make sure that you have the following components:
* Local https://pivotal.io/pcf-dev[PCFDev] instance * Local https://pivotal.io/pcf-dev[PCFDev] instance
* Local install of https://github.com/cloudfoundry/v3-cli-plugin[V3-CF-CLI] plugin * Local install of https://github.com/cloudfoundry/cli[cf CLI] command line tool
* Local build of https://github.com/spring-cloud/spring-cloud-dataflow[Spring Cloud Data Flow] * Local build of https://github.com/spring-cloud/spring-cloud-dataflow[Spring Cloud Data Flow]
* Local build of Spring Cloud Data Flow's https://github.com/spring-cloud/spring-cloud-dataflow-server-cloudfoundry[Cloud Foundry Server] * Local build of Spring Cloud Data Flow's https://github.com/spring-cloud/spring-cloud-dataflow-server-cloudfoundry[Cloud Foundry Server]
* Running instance of mysql in PCFDev * Running instance of mysql in PCFDev
@@ -23,8 +23,9 @@ NOTE: PCF 1.7.12 or greater is required to run Tasks on Spring Cloud Data Flow.
+ +
``` ```
→ cf api cf api
API endpoint: https://api.local.pcfdev.io (API version: 2.54.0) api endpoint: https://api.local.pcfdev.io
api version: 2.75.0
$ cf apps $ cf apps
Getting apps in org user-dataflow / space development as user... Getting apps in org user-dataflow / space development as user...
@@ -33,13 +34,24 @@ OK
No apps found No apps found
``` ```
+ +
. Task support needs to be enabled on pcf-dev. Being logged as `admin`, issue the following command:
+
```
cf enable-feature-flag task_creation
Setting status of task_creation as admin...
OK
Feature task_creation Enabled.
```
+
. Follow the http://docs.spring.io/spring-cloud-dataflow-server-cloudfoundry/docs/current-SNAPSHOT/reference/htmlsingle/#getting-started[getting-started] instructions to deploy Spring Cloud Data Flow's Cloud Foundry Server . Follow the http://docs.spring.io/spring-cloud-dataflow-server-cloudfoundry/docs/current-SNAPSHOT/reference/htmlsingle/#getting-started[getting-started] instructions to deploy Spring Cloud Data Flow's Cloud Foundry Server
+ +
NOTE: For this sample, all you need is the `mysql` service and in PCFDev, the `mysql` service comes with a different plan. From CF CLI, create the service by: `cf create-service p-mysql 512mb mysql` and bind this service to `dataflow-server` by: `cf bind-service dataflow-server mysql`. NOTE: For this sample, all you need is the `mysql` service and in PCFDev, the `mysql` service comes with a different plan. From CF CLI, create the service by: `cf create-service p-mysql 512mb mysql` and bind this service to `dataflow-server` by: `cf bind-service dataflow-server mysql`.
+ +
NOTE: All the apps deployed to PCFDev start with low memory by default. It is recommended to change it to at least 512MB for `dataflow-server`. Change the memory by: `cf set-env dataflow-server SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_MEMORY 512`. Likewise, we would have to skip SSL validation by: `cf set-env dataflow-server SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_SKIP_SSL_VALIDATION true`. NOTE: All the apps deployed to PCFDev start with low memory by default. It is recommended to change it to at least 768MB for `dataflow-server`. Ditto for every app spawned *by* Spring Cloud Data Flow. Change the memory by: `cf set-env dataflow-server SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_MEMORY 512`. Likewise, we would have to skip SSL validation by: `cf set-env dataflow-server SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_SKIP_SSL_VALIDATION true`.
. Once you complete the instructions, you'll be able to list the newly deployed `dataflow-server` . Once you complete the instructions, you'll be able to list the newly deployed `dataflow-server`
@@ -57,7 +69,7 @@ dataflow-server started 1/1 512M 512M dataflow-server.l
NOTE: The `ORG`, `SPACE` and `USER` could vary depending on your Cloud Foundry environment in use. NOTE: The `ORG`, `SPACE` and `USER` could vary depending on your Cloud Foundry environment in use.
. Tasks in Spring Cloud Data Flow requires an RDBMS to host "task repository" (see http://docs.spring.io/spring-cloud-dataflow/docs/1.0.0.RELEASE/reference/htmlsingle/#spring-cloud-dataflow-task-repository[here] for more details), so let's bind `mysql` service and restage the `dataflow-server` . Tasks in Spring Cloud Data Flow require an RDBMS to host "task repository" (see http://docs.spring.io/spring-cloud-dataflow/docs/current-SNAPSHOT/reference/htmlsingle/#spring-cloud-dataflow-task-repository[here] for more details), so let's instruct the Spring Cloud Data Flow server to bind the `mysql` service to each deployed task:
+ +
@@ -69,6 +81,29 @@ $ cf restage dataflow-server
NOTE: We only need `mysql` service for this sample. NOTE: We only need `mysql` service for this sample.
. As a recap, here is what you should see as configuration for the Spring Cloud Data Flow server:
+
```
cf env dataflow-server
....
User-Provided:
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_DOMAIN: local.pcfdev.io
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_MEMORY: 512
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_ORG: pcfdev-org
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_PASSWORD: pass
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_SKIP_SSL_VALIDATION: false
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_SPACE: pcfdev-space
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_TASK_SERVICES: mysql
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_URL: https://api.local.pcfdev.io
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_USERNAME: user
No running env variables have been set
No staging env variables have been set
```
+
. Notice that `dataflow-server` application is started and ready for interaction via `http://dataflow-server.local.pcfdev.io` endpoint . Notice that `dataflow-server` application is started and ready for interaction via `http://dataflow-server.local.pcfdev.io` endpoint
. Connect to Spring Cloud Data Flow's `shell`. . Connect to Spring Cloud Data Flow's `shell`.
@@ -101,17 +136,6 @@ server-unknown:>
``` ```
server-unknown:>dataflow config server http://dataflow-server.local.pcfdev.io server-unknown:>dataflow config server http://dataflow-server.local.pcfdev.io
Successfully targeted http://dataflow-server.local.pcfdev.io Successfully targeted http://dataflow-server.local.pcfdev.io
dataflow:>version
<VERSION>
```
+
. Verify there's no Task applications running on PCFDev using `v3-cf-cli`
+
```
→ cf v3-apps
No v3 apps found.
``` ```
+ +
@@ -120,7 +144,7 @@ No v3 apps found.
+ +
``` ```
dataflow:>app register --type task --name simple_batch_job --uri https://github.com/spring-cloud/spring-cloud-dataflow-samples/raw/master/tasks/simple-batch-job/batch-job-1.0.0.BUILD-SNAPSHOT.jar dataflow:>app register --type task --name simple_batch_job --uri https://github.com/spring-cloud/spring-cloud-dataflow-samples/raw/master/tasks/simple-batch-job/batch-job-1.3.0.BUILD-SNAPSHOT.jar
``` ```
+ +
@@ -138,8 +162,12 @@ NOTE: Unlike Streams, the Task definitions don't require explicit deployment. Th
+ +
``` ```
cf v3-apps cf apps
No v3 apps found. Getting apps in org pcfdev-org / space pcfdev-space as user...
OK
name requested state instances memory disk urls
dataflow-server started 1/1 768M 512M dataflow-server.local.pcfdev.io
``` ```
+ +
@@ -157,8 +185,8 @@ dataflow:>task launch foo
+ +
``` ```
→ cf v3-logs foo → cf logs foo
Tailing logs for app foo... Retrieving logs for app foo in org pcfdev-org / space pcfdev-space as user...
2016-08-14T18:48:54.22-0700 [APP/TASK/foo/0]OUT Creating container 2016-08-14T18:48:54.22-0700 [APP/TASK/foo/0]OUT Creating container
2016-08-14T18:48:55.47-0700 [APP/TASK/foo/0]OUT 2016-08-14T18:48:55.47-0700 [APP/TASK/foo/0]OUT
@@ -200,9 +228,13 @@ NOTE: Unlike LRPs in Cloud Foundry, tasks are short-lived, so the logs aren't al
+ +
``` ```
→ cf v3-apps → cf apps
name total_desired_instances Getting apps in org pcfdev-org / space pcfdev-space as user...
foo 0 OK
name requested state instances memory disk urls
dataflow-server started 1/1 768M 512M dataflow-server.local.pcfdev.io
foo stopped 0/1 1G 1G
``` ```
+ +
@@ -215,7 +247,7 @@ dataflow:>task execution list
╔══════════════════════════╤══╤════════════════════════════╤════════════════════════════╤═════════╗ ╔══════════════════════════╤══╤════════════════════════════╤════════════════════════════╤═════════╗
║ Task Name │ID│ Start Time │ End Time │Exit Code║ ║ Task Name │ID│ Start Time │ End Time │Exit Code║
╠══════════════════════════╪══╪════════════════════════════╪════════════════════════════╪═════════╣ ╠══════════════════════════╪══╪════════════════════════════╪════════════════════════════╪═════════╣
Demo Batch Job Task:cloud:│1 │Sun Aug 14 18:49:05 PDT 2016│Sun Aug 14 18:49:07 PDT 2016│0 ║ foo │1 │Sun Aug 14 18:49:05 PDT 2016│Sun Aug 14 18:49:07 PDT 2016│0 ║
╚══════════════════════════╧══╧════════════════════════════╧════════════════════════════╧═════════╝ ╚══════════════════════════╧══╧════════════════════════════╧════════════════════════════╧═════════╝
``` ```
@@ -240,5 +272,5 @@ dataflow:>job execution list
In this sample, you have learned: In this sample, you have learned:
* How to register and orchestrate Spring Batch jobs in Spring Cloud Data Flow * How to register and orchestrate Spring Batch jobs in Spring Cloud Data Flow
* How to use `v3-cf-cli` in the context of Task applications orchestrated by Spring Cloud Data Flow * How to use the `cf` CLI in the context of Task applications orchestrated by Spring Cloud Data Flow
* How to verify task executions and task repository * How to verify task executions and task repository