Update instructions with latest CF and SCDF support
Fixes https://github.com/spring-cloud/spring-cloud-dataflow-samples/issues/22 polish
This commit is contained in:
committed by
Glenn Renfro
parent
cb052f9b29
commit
337b41ecaa
1
.gitignore
vendored
Normal file
1
.gitignore
vendored
Normal file
@@ -0,0 +1 @@
|
|||||||
|
.DS_Store
|
||||||
@@ -10,7 +10,7 @@ In this demonstration, you will learn how to orchestrate short-lived data proces
|
|||||||
In order to get started, make sure that you have the following components:
|
In order to get started, make sure that you have the following components:
|
||||||
|
|
||||||
* Local https://pivotal.io/pcf-dev[PCFDev] instance
|
* Local https://pivotal.io/pcf-dev[PCFDev] instance
|
||||||
* Local install of https://github.com/cloudfoundry/v3-cli-plugin[V3-CF-CLI] plugin
|
* Local install of https://github.com/cloudfoundry/cli[cf CLI] command line tool
|
||||||
* Local build of https://github.com/spring-cloud/spring-cloud-dataflow[Spring Cloud Data Flow]
|
* Local build of https://github.com/spring-cloud/spring-cloud-dataflow[Spring Cloud Data Flow]
|
||||||
* Local build of Spring Cloud Data Flow's https://github.com/spring-cloud/spring-cloud-dataflow-server-cloudfoundry[Cloud Foundry Server]
|
* Local build of Spring Cloud Data Flow's https://github.com/spring-cloud/spring-cloud-dataflow-server-cloudfoundry[Cloud Foundry Server]
|
||||||
* Running instance of mysql in PCFDev
|
* Running instance of mysql in PCFDev
|
||||||
@@ -23,8 +23,9 @@ NOTE: PCF 1.7.12 or greater is required to run Tasks on Spring Cloud Data Flow.
|
|||||||
+
|
+
|
||||||
|
|
||||||
```
|
```
|
||||||
→ cf api
|
→ cf api
|
||||||
API endpoint: https://api.local.pcfdev.io (API version: 2.54.0)
|
api endpoint: https://api.local.pcfdev.io
|
||||||
|
api version: 2.75.0
|
||||||
|
|
||||||
$ cf apps
|
$ cf apps
|
||||||
Getting apps in org user-dataflow / space development as user...
|
Getting apps in org user-dataflow / space development as user...
|
||||||
@@ -33,13 +34,24 @@ OK
|
|||||||
No apps found
|
No apps found
|
||||||
```
|
```
|
||||||
+
|
+
|
||||||
|
. Task support needs to be enabled on pcf-dev. Being logged as `admin`, issue the following command:
|
||||||
|
+
|
||||||
|
```
|
||||||
|
cf enable-feature-flag task_creation
|
||||||
|
Setting status of task_creation as admin...
|
||||||
|
|
||||||
|
OK
|
||||||
|
|
||||||
|
Feature task_creation Enabled.
|
||||||
|
```
|
||||||
|
+
|
||||||
. Follow the http://docs.spring.io/spring-cloud-dataflow-server-cloudfoundry/docs/current-SNAPSHOT/reference/htmlsingle/#getting-started[getting-started] instructions to deploy Spring Cloud Data Flow's Cloud Foundry Server
|
. Follow the http://docs.spring.io/spring-cloud-dataflow-server-cloudfoundry/docs/current-SNAPSHOT/reference/htmlsingle/#getting-started[getting-started] instructions to deploy Spring Cloud Data Flow's Cloud Foundry Server
|
||||||
|
|
||||||
+
|
+
|
||||||
NOTE: For this sample, all you need is the `mysql` service and in PCFDev, the `mysql` service comes with a different plan. From CF CLI, create the service by: `cf create-service p-mysql 512mb mysql` and bind this service to `dataflow-server` by: `cf bind-service dataflow-server mysql`.
|
NOTE: For this sample, all you need is the `mysql` service and in PCFDev, the `mysql` service comes with a different plan. From CF CLI, create the service by: `cf create-service p-mysql 512mb mysql` and bind this service to `dataflow-server` by: `cf bind-service dataflow-server mysql`.
|
||||||
+
|
+
|
||||||
|
|
||||||
NOTE: All the apps deployed to PCFDev start with low memory by default. It is recommended to change it to at least 512MB for `dataflow-server`. Change the memory by: `cf set-env dataflow-server SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_MEMORY 512`. Likewise, we would have to skip SSL validation by: `cf set-env dataflow-server SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_SKIP_SSL_VALIDATION true`.
|
NOTE: All the apps deployed to PCFDev start with low memory by default. It is recommended to change it to at least 768MB for `dataflow-server`. Ditto for every app spawned *by* Spring Cloud Data Flow. Change the memory by: `cf set-env dataflow-server SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_MEMORY 512`. Likewise, we would have to skip SSL validation by: `cf set-env dataflow-server SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_SKIP_SSL_VALIDATION true`.
|
||||||
|
|
||||||
. Once you complete the instructions, you'll be able to list the newly deployed `dataflow-server`
|
. Once you complete the instructions, you'll be able to list the newly deployed `dataflow-server`
|
||||||
|
|
||||||
@@ -57,7 +69,7 @@ dataflow-server started 1/1 512M 512M dataflow-server.l
|
|||||||
|
|
||||||
NOTE: The `ORG`, `SPACE` and `USER` could vary depending on your Cloud Foundry environment in use.
|
NOTE: The `ORG`, `SPACE` and `USER` could vary depending on your Cloud Foundry environment in use.
|
||||||
|
|
||||||
. Tasks in Spring Cloud Data Flow requires an RDBMS to host "task repository" (see http://docs.spring.io/spring-cloud-dataflow/docs/1.0.0.RELEASE/reference/htmlsingle/#spring-cloud-dataflow-task-repository[here] for more details), so let's bind `mysql` service and restage the `dataflow-server`
|
. Tasks in Spring Cloud Data Flow require an RDBMS to host "task repository" (see http://docs.spring.io/spring-cloud-dataflow/docs/current-SNAPSHOT/reference/htmlsingle/#spring-cloud-dataflow-task-repository[here] for more details), so let's instruct the Spring Cloud Data Flow server to bind the `mysql` service to each deployed task:
|
||||||
|
|
||||||
+
|
+
|
||||||
|
|
||||||
@@ -69,6 +81,29 @@ $ cf restage dataflow-server
|
|||||||
|
|
||||||
NOTE: We only need `mysql` service for this sample.
|
NOTE: We only need `mysql` service for this sample.
|
||||||
|
|
||||||
|
. As a recap, here is what you should see as configuration for the Spring Cloud Data Flow server:
|
||||||
|
+
|
||||||
|
```
|
||||||
|
cf env dataflow-server
|
||||||
|
|
||||||
|
....
|
||||||
|
User-Provided:
|
||||||
|
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_DOMAIN: local.pcfdev.io
|
||||||
|
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_MEMORY: 512
|
||||||
|
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_ORG: pcfdev-org
|
||||||
|
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_PASSWORD: pass
|
||||||
|
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_SKIP_SSL_VALIDATION: false
|
||||||
|
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_SPACE: pcfdev-space
|
||||||
|
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_TASK_SERVICES: mysql
|
||||||
|
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_URL: https://api.local.pcfdev.io
|
||||||
|
SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_USERNAME: user
|
||||||
|
|
||||||
|
No running env variables have been set
|
||||||
|
|
||||||
|
No staging env variables have been set
|
||||||
|
```
|
||||||
|
+
|
||||||
|
|
||||||
. Notice that `dataflow-server` application is started and ready for interaction via `http://dataflow-server.local.pcfdev.io` endpoint
|
. Notice that `dataflow-server` application is started and ready for interaction via `http://dataflow-server.local.pcfdev.io` endpoint
|
||||||
|
|
||||||
. Connect to Spring Cloud Data Flow's `shell`.
|
. Connect to Spring Cloud Data Flow's `shell`.
|
||||||
@@ -101,17 +136,6 @@ server-unknown:>
|
|||||||
```
|
```
|
||||||
server-unknown:>dataflow config server http://dataflow-server.local.pcfdev.io
|
server-unknown:>dataflow config server http://dataflow-server.local.pcfdev.io
|
||||||
Successfully targeted http://dataflow-server.local.pcfdev.io
|
Successfully targeted http://dataflow-server.local.pcfdev.io
|
||||||
dataflow:>version
|
|
||||||
<VERSION>
|
|
||||||
```
|
|
||||||
+
|
|
||||||
|
|
||||||
. Verify there's no Task applications running on PCFDev using `v3-cf-cli`
|
|
||||||
|
|
||||||
+
|
|
||||||
```
|
|
||||||
→ cf v3-apps
|
|
||||||
No v3 apps found.
|
|
||||||
```
|
```
|
||||||
+
|
+
|
||||||
|
|
||||||
@@ -120,7 +144,7 @@ No v3 apps found.
|
|||||||
+
|
+
|
||||||
|
|
||||||
```
|
```
|
||||||
dataflow:>app register --type task --name simple_batch_job --uri https://github.com/spring-cloud/spring-cloud-dataflow-samples/raw/master/tasks/simple-batch-job/batch-job-1.0.0.BUILD-SNAPSHOT.jar
|
dataflow:>app register --type task --name simple_batch_job --uri https://github.com/spring-cloud/spring-cloud-dataflow-samples/raw/master/tasks/simple-batch-job/batch-job-1.3.0.BUILD-SNAPSHOT.jar
|
||||||
```
|
```
|
||||||
+
|
+
|
||||||
|
|
||||||
@@ -138,8 +162,12 @@ NOTE: Unlike Streams, the Task definitions don't require explicit deployment. Th
|
|||||||
|
|
||||||
+
|
+
|
||||||
```
|
```
|
||||||
→ cf v3-apps
|
cf apps
|
||||||
No v3 apps found.
|
Getting apps in org pcfdev-org / space pcfdev-space as user...
|
||||||
|
OK
|
||||||
|
|
||||||
|
name requested state instances memory disk urls
|
||||||
|
dataflow-server started 1/1 768M 512M dataflow-server.local.pcfdev.io
|
||||||
```
|
```
|
||||||
+
|
+
|
||||||
|
|
||||||
@@ -157,8 +185,8 @@ dataflow:>task launch foo
|
|||||||
+
|
+
|
||||||
|
|
||||||
```
|
```
|
||||||
→ cf v3-logs foo
|
→ cf logs foo
|
||||||
Tailing logs for app foo...
|
Retrieving logs for app foo in org pcfdev-org / space pcfdev-space as user...
|
||||||
|
|
||||||
2016-08-14T18:48:54.22-0700 [APP/TASK/foo/0]OUT Creating container
|
2016-08-14T18:48:54.22-0700 [APP/TASK/foo/0]OUT Creating container
|
||||||
2016-08-14T18:48:55.47-0700 [APP/TASK/foo/0]OUT
|
2016-08-14T18:48:55.47-0700 [APP/TASK/foo/0]OUT
|
||||||
@@ -200,9 +228,13 @@ NOTE: Unlike LRPs in Cloud Foundry, tasks are short-lived, so the logs aren't al
|
|||||||
+
|
+
|
||||||
|
|
||||||
```
|
```
|
||||||
→ cf v3-apps
|
→ cf apps
|
||||||
name total_desired_instances
|
Getting apps in org pcfdev-org / space pcfdev-space as user...
|
||||||
foo 0
|
OK
|
||||||
|
|
||||||
|
name requested state instances memory disk urls
|
||||||
|
dataflow-server started 1/1 768M 512M dataflow-server.local.pcfdev.io
|
||||||
|
foo stopped 0/1 1G 1G
|
||||||
```
|
```
|
||||||
+
|
+
|
||||||
|
|
||||||
@@ -215,7 +247,7 @@ dataflow:>task execution list
|
|||||||
╔══════════════════════════╤══╤════════════════════════════╤════════════════════════════╤═════════╗
|
╔══════════════════════════╤══╤════════════════════════════╤════════════════════════════╤═════════╗
|
||||||
║ Task Name │ID│ Start Time │ End Time │Exit Code║
|
║ Task Name │ID│ Start Time │ End Time │Exit Code║
|
||||||
╠══════════════════════════╪══╪════════════════════════════╪════════════════════════════╪═════════╣
|
╠══════════════════════════╪══╪════════════════════════════╪════════════════════════════╪═════════╣
|
||||||
║Demo Batch Job Task:cloud:│1 │Sun Aug 14 18:49:05 PDT 2016│Sun Aug 14 18:49:07 PDT 2016│0 ║
|
║foo │1 │Sun Aug 14 18:49:05 PDT 2016│Sun Aug 14 18:49:07 PDT 2016│0 ║
|
||||||
╚══════════════════════════╧══╧════════════════════════════╧════════════════════════════╧═════════╝
|
╚══════════════════════════╧══╧════════════════════════════╧════════════════════════════╧═════════╝
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -240,5 +272,5 @@ dataflow:>job execution list
|
|||||||
In this sample, you have learned:
|
In this sample, you have learned:
|
||||||
|
|
||||||
* How to register and orchestrate Spring Batch jobs in Spring Cloud Data Flow
|
* How to register and orchestrate Spring Batch jobs in Spring Cloud Data Flow
|
||||||
* How to use `v3-cf-cli` in the context of Task applications orchestrated by Spring Cloud Data Flow
|
* How to use the `cf` CLI in the context of Task applications orchestrated by Spring Cloud Data Flow
|
||||||
* How to verify task executions and task repository
|
* How to verify task executions and task repository
|
||||||
|
|||||||
Reference in New Issue
Block a user