Update instructions with latest CF and SCDF support
Fixes https://github.com/spring-cloud/spring-cloud-dataflow-samples/issues/22 polish
This commit is contained in:
committed by
Glenn Renfro
parent
cb052f9b29
commit
337b41ecaa
1
.gitignore
vendored
Normal file
1
.gitignore
vendored
Normal file
@@ -0,0 +1 @@
|
||||
.DS_Store
|
||||
@@ -10,7 +10,7 @@ In this demonstration, you will learn how to orchestrate short-lived data proces
|
||||
In order to get started, make sure that you have the following components:
|
||||
|
||||
* Local https://pivotal.io/pcf-dev[PCFDev] instance
|
||||
* Local install of https://github.com/cloudfoundry/v3-cli-plugin[V3-CF-CLI] plugin
|
||||
* Local install of https://github.com/cloudfoundry/cli[cf CLI] command line tool
|
||||
* Local build of https://github.com/spring-cloud/spring-cloud-dataflow[Spring Cloud Data Flow]
|
||||
* Local build of Spring Cloud Data Flow's https://github.com/spring-cloud/spring-cloud-dataflow-server-cloudfoundry[Cloud Foundry Server]
|
||||
* Running instance of mysql in PCFDev
|
||||
@@ -23,8 +23,9 @@ NOTE: PCF 1.7.12 or greater is required to run Tasks on Spring Cloud Data Flow.
|
||||
+
|
||||
|
||||
```
|
||||
→ cf api
|
||||
API endpoint: https://api.local.pcfdev.io (API version: 2.54.0)
|
||||
→ cf api
|
||||
api endpoint: https://api.local.pcfdev.io
|
||||
api version: 2.75.0
|
||||
|
||||
$ cf apps
|
||||
Getting apps in org user-dataflow / space development as user...
|
||||
@@ -33,13 +34,24 @@ OK
|
||||
No apps found
|
||||
```
|
||||
+
|
||||
. Task support needs to be enabled on pcf-dev. Being logged as `admin`, issue the following command:
|
||||
+
|
||||
```
|
||||
cf enable-feature-flag task_creation
|
||||
Setting status of task_creation as admin...
|
||||
|
||||
OK
|
||||
|
||||
Feature task_creation Enabled.
|
||||
```
|
||||
+
|
||||
. Follow the http://docs.spring.io/spring-cloud-dataflow-server-cloudfoundry/docs/current-SNAPSHOT/reference/htmlsingle/#getting-started[getting-started] instructions to deploy Spring Cloud Data Flow's Cloud Foundry Server
|
||||
|
||||
+
|
||||
NOTE: For this sample, all you need is the `mysql` service and in PCFDev, the `mysql` service comes with a different plan. From CF CLI, create the service by: `cf create-service p-mysql 512mb mysql` and bind this service to `dataflow-server` by: `cf bind-service dataflow-server mysql`.
|
||||
+
|
||||
|
||||
NOTE: All the apps deployed to PCFDev start with low memory by default. It is recommended to change it to at least 512MB for `dataflow-server`. Change the memory by: `cf set-env dataflow-server SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_MEMORY 512`. Likewise, we would have to skip SSL validation by: `cf set-env dataflow-server SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_SKIP_SSL_VALIDATION true`.
|
||||
NOTE: All the apps deployed to PCFDev start with low memory by default. It is recommended to change it to at least 768MB for `dataflow-server`. Ditto for every app spawned *by* Spring Cloud Data Flow. Change the memory by: `cf set-env dataflow-server SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_MEMORY 512`. Likewise, we would have to skip SSL validation by: `cf set-env dataflow-server SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_SKIP_SSL_VALIDATION true`.
|
||||
|
||||
. Once you complete the instructions, you'll be able to list the newly deployed `dataflow-server`
|
||||
|
||||
@@ -57,7 +69,7 @@ dataflow-server started 1/1 512M 512M dataflow-server.l
|
||||
|
||||
NOTE: The `ORG`, `SPACE` and `USER` could vary depending on your Cloud Foundry environment in use.
|
||||
|
||||
. Tasks in Spring Cloud Data Flow requires an RDBMS to host "task repository" (see http://docs.spring.io/spring-cloud-dataflow/docs/1.0.0.RELEASE/reference/htmlsingle/#spring-cloud-dataflow-task-repository[here] for more details), so let's bind `mysql` service and restage the `dataflow-server`
|
||||
. Tasks in Spring Cloud Data Flow require an RDBMS to host "task repository" (see http://docs.spring.io/spring-cloud-dataflow/docs/current-SNAPSHOT/reference/htmlsingle/#spring-cloud-dataflow-task-repository[here] for more details), so let's instruct the Spring Cloud Data Flow server to bind the `mysql` service to each deployed task:
|
||||
|
||||
+
|
||||
|
||||
@@ -69,6 +81,29 @@ $ cf restage dataflow-server
|
||||
|
||||
NOTE: We only need `mysql` service for this sample.
|
||||
|
||||
. As a recap, here is what you should see as configuration for the Spring Cloud Data Flow server:
|
||||
+
|
||||
```
|
||||
cf env dataflow-server
|
||||
|
||||
....
|
||||
User-Provided:
|
||||
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_DOMAIN: local.pcfdev.io
|
||||
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_MEMORY: 512
|
||||
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_ORG: pcfdev-org
|
||||
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_PASSWORD: pass
|
||||
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_SKIP_SSL_VALIDATION: false
|
||||
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_SPACE: pcfdev-space
|
||||
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_TASK_SERVICES: mysql
|
||||
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_URL: https://api.local.pcfdev.io
|
||||
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_USERNAME: user
|
||||
|
||||
No running env variables have been set
|
||||
|
||||
No staging env variables have been set
|
||||
```
|
||||
+
|
||||
|
||||
. Notice that `dataflow-server` application is started and ready for interaction via `http://dataflow-server.local.pcfdev.io` endpoint
|
||||
|
||||
. Connect to Spring Cloud Data Flow's `shell`.
|
||||
@@ -101,17 +136,6 @@ server-unknown:>
|
||||
```
|
||||
server-unknown:>dataflow config server http://dataflow-server.local.pcfdev.io
|
||||
Successfully targeted http://dataflow-server.local.pcfdev.io
|
||||
dataflow:>version
|
||||
<VERSION>
|
||||
```
|
||||
+
|
||||
|
||||
. Verify there's no Task applications running on PCFDev using `v3-cf-cli`
|
||||
|
||||
+
|
||||
```
|
||||
→ cf v3-apps
|
||||
No v3 apps found.
|
||||
```
|
||||
+
|
||||
|
||||
@@ -120,7 +144,7 @@ No v3 apps found.
|
||||
+
|
||||
|
||||
```
|
||||
dataflow:>app register --type task --name simple_batch_job --uri https://github.com/spring-cloud/spring-cloud-dataflow-samples/raw/master/tasks/simple-batch-job/batch-job-1.0.0.BUILD-SNAPSHOT.jar
|
||||
dataflow:>app register --type task --name simple_batch_job --uri https://github.com/spring-cloud/spring-cloud-dataflow-samples/raw/master/tasks/simple-batch-job/batch-job-1.3.0.BUILD-SNAPSHOT.jar
|
||||
```
|
||||
+
|
||||
|
||||
@@ -138,8 +162,12 @@ NOTE: Unlike Streams, the Task definitions don't require explicit deployment. Th
|
||||
|
||||
+
|
||||
```
|
||||
→ cf v3-apps
|
||||
No v3 apps found.
|
||||
cf apps
|
||||
Getting apps in org pcfdev-org / space pcfdev-space as user...
|
||||
OK
|
||||
|
||||
name requested state instances memory disk urls
|
||||
dataflow-server started 1/1 768M 512M dataflow-server.local.pcfdev.io
|
||||
```
|
||||
+
|
||||
|
||||
@@ -157,8 +185,8 @@ dataflow:>task launch foo
|
||||
+
|
||||
|
||||
```
|
||||
→ cf v3-logs foo
|
||||
Tailing logs for app foo...
|
||||
→ cf logs foo
|
||||
Retrieving logs for app foo in org pcfdev-org / space pcfdev-space as user...
|
||||
|
||||
2016-08-14T18:48:54.22-0700 [APP/TASK/foo/0]OUT Creating container
|
||||
2016-08-14T18:48:55.47-0700 [APP/TASK/foo/0]OUT
|
||||
@@ -200,9 +228,13 @@ NOTE: Unlike LRPs in Cloud Foundry, tasks are short-lived, so the logs aren't al
|
||||
+
|
||||
|
||||
```
|
||||
→ cf v3-apps
|
||||
name total_desired_instances
|
||||
foo 0
|
||||
→ cf apps
|
||||
Getting apps in org pcfdev-org / space pcfdev-space as user...
|
||||
OK
|
||||
|
||||
name requested state instances memory disk urls
|
||||
dataflow-server started 1/1 768M 512M dataflow-server.local.pcfdev.io
|
||||
foo stopped 0/1 1G 1G
|
||||
```
|
||||
+
|
||||
|
||||
@@ -215,7 +247,7 @@ dataflow:>task execution list
|
||||
╔══════════════════════════╤══╤════════════════════════════╤════════════════════════════╤═════════╗
|
||||
║ Task Name │ID│ Start Time │ End Time │Exit Code║
|
||||
╠══════════════════════════╪══╪════════════════════════════╪════════════════════════════╪═════════╣
|
||||
║Demo Batch Job Task:cloud:│1 │Sun Aug 14 18:49:05 PDT 2016│Sun Aug 14 18:49:07 PDT 2016│0 ║
|
||||
║foo │1 │Sun Aug 14 18:49:05 PDT 2016│Sun Aug 14 18:49:07 PDT 2016│0 ║
|
||||
╚══════════════════════════╧══╧════════════════════════════╧════════════════════════════╧═════════╝
|
||||
```
|
||||
|
||||
@@ -240,5 +272,5 @@ dataflow:>job execution list
|
||||
In this sample, you have learned:
|
||||
|
||||
* How to register and orchestrate Spring Batch jobs in Spring Cloud Data Flow
|
||||
* How to use `v3-cf-cli` in the context of Task applications orchestrated by Spring Cloud Data Flow
|
||||
* How to use the `cf` CLI in the context of Task applications orchestrated by Spring Cloud Data Flow
|
||||
* How to verify task executions and task repository
|
||||
|
||||
Reference in New Issue
Block a user