diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..e43b0f9 --- /dev/null +++ b/.gitignore @@ -0,0 +1 @@ +.DS_Store diff --git a/tasks/simple-batch-job/README.adoc b/tasks/simple-batch-job/README.adoc index f22873a..662243b 100644 --- a/tasks/simple-batch-job/README.adoc +++ b/tasks/simple-batch-job/README.adoc @@ -10,7 +10,7 @@ In this demonstration, you will learn how to orchestrate short-lived data proces In order to get started, make sure that you have the following components: * Local https://pivotal.io/pcf-dev[PCFDev] instance -* Local install of https://github.com/cloudfoundry/v3-cli-plugin[V3-CF-CLI] plugin +* Local install of https://github.com/cloudfoundry/cli[cf CLI] command line tool * Local build of https://github.com/spring-cloud/spring-cloud-dataflow[Spring Cloud Data Flow] * Local build of Spring Cloud Data Flow's https://github.com/spring-cloud/spring-cloud-dataflow-server-cloudfoundry[Cloud Foundry Server] * Running instance of mysql in PCFDev @@ -23,8 +23,9 @@ NOTE: PCF 1.7.12 or greater is required to run Tasks on Spring Cloud Data Flow. + ``` -→ cf api -API endpoint: https://api.local.pcfdev.io (API version: 2.54.0) +→ cf api +api endpoint: https://api.local.pcfdev.io +api version: 2.75.0 $ cf apps Getting apps in org user-dataflow / space development as user... @@ -33,13 +34,24 @@ OK No apps found ``` + +. Task support needs to be enabled on pcf-dev. Being logged as `admin`, issue the following command: ++ +``` +cf enable-feature-flag task_creation +Setting status of task_creation as admin... + +OK + +Feature task_creation Enabled. +``` ++ . Follow the http://docs.spring.io/spring-cloud-dataflow-server-cloudfoundry/docs/current-SNAPSHOT/reference/htmlsingle/#getting-started[getting-started] instructions to deploy Spring Cloud Data Flow's Cloud Foundry Server + NOTE: For this sample, all you need is the `mysql` service and in PCFDev, the `mysql` service comes with a different plan. From CF CLI, create the service by: `cf create-service p-mysql 512mb mysql` and bind this service to `dataflow-server` by: `cf bind-service dataflow-server mysql`. + -NOTE: All the apps deployed to PCFDev start with low memory by default. It is recommended to change it to at least 512MB for `dataflow-server`. Change the memory by: `cf set-env dataflow-server SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_MEMORY 512`. Likewise, we would have to skip SSL validation by: `cf set-env dataflow-server SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_SKIP_SSL_VALIDATION true`. +NOTE: All the apps deployed to PCFDev start with low memory by default. It is recommended to change it to at least 768MB for `dataflow-server`. Ditto for every app spawned *by* Spring Cloud Data Flow. Change the memory by: `cf set-env dataflow-server SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_MEMORY 512`. Likewise, we would have to skip SSL validation by: `cf set-env dataflow-server SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_SKIP_SSL_VALIDATION true`. . Once you complete the instructions, you'll be able to list the newly deployed `dataflow-server` @@ -57,7 +69,7 @@ dataflow-server started 1/1 512M 512M dataflow-server.l NOTE: The `ORG`, `SPACE` and `USER` could vary depending on your Cloud Foundry environment in use. -. Tasks in Spring Cloud Data Flow requires an RDBMS to host "task repository" (see http://docs.spring.io/spring-cloud-dataflow/docs/1.0.0.RELEASE/reference/htmlsingle/#spring-cloud-dataflow-task-repository[here] for more details), so let's bind `mysql` service and restage the `dataflow-server` +. Tasks in Spring Cloud Data Flow require an RDBMS to host "task repository" (see http://docs.spring.io/spring-cloud-dataflow/docs/current-SNAPSHOT/reference/htmlsingle/#spring-cloud-dataflow-task-repository[here] for more details), so let's instruct the Spring Cloud Data Flow server to bind the `mysql` service to each deployed task: + @@ -69,6 +81,29 @@ $ cf restage dataflow-server NOTE: We only need `mysql` service for this sample. +. As a recap, here is what you should see as configuration for the Spring Cloud Data Flow server: ++ +``` +cf env dataflow-server + +.... +User-Provided: +SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_DOMAIN: local.pcfdev.io +SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_MEMORY: 512 +SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_ORG: pcfdev-org +SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_PASSWORD: pass +SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_SKIP_SSL_VALIDATION: false +SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_SPACE: pcfdev-space +SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_TASK_SERVICES: mysql +SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_URL: https://api.local.pcfdev.io +SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_USERNAME: user + +No running env variables have been set + +No staging env variables have been set +``` ++ + . Notice that `dataflow-server` application is started and ready for interaction via `http://dataflow-server.local.pcfdev.io` endpoint . Connect to Spring Cloud Data Flow's `shell`. @@ -101,17 +136,6 @@ server-unknown:> ``` server-unknown:>dataflow config server http://dataflow-server.local.pcfdev.io Successfully targeted http://dataflow-server.local.pcfdev.io -dataflow:>version - -``` -+ - -. Verify there's no Task applications running on PCFDev using `v3-cf-cli` - -+ -``` -→ cf v3-apps -No v3 apps found. ``` + @@ -120,7 +144,7 @@ No v3 apps found. + ``` -dataflow:>app register --type task --name simple_batch_job --uri https://github.com/spring-cloud/spring-cloud-dataflow-samples/raw/master/tasks/simple-batch-job/batch-job-1.0.0.BUILD-SNAPSHOT.jar +dataflow:>app register --type task --name simple_batch_job --uri https://github.com/spring-cloud/spring-cloud-dataflow-samples/raw/master/tasks/simple-batch-job/batch-job-1.3.0.BUILD-SNAPSHOT.jar ``` + @@ -138,8 +162,12 @@ NOTE: Unlike Streams, the Task definitions don't require explicit deployment. Th + ``` -→ cf v3-apps -No v3 apps found. +cf apps +Getting apps in org pcfdev-org / space pcfdev-space as user... +OK + +name requested state instances memory disk urls +dataflow-server started 1/1 768M 512M dataflow-server.local.pcfdev.io ``` + @@ -157,8 +185,8 @@ dataflow:>task launch foo + ``` -→ cf v3-logs foo -Tailing logs for app foo... +→ cf logs foo +Retrieving logs for app foo in org pcfdev-org / space pcfdev-space as user... 2016-08-14T18:48:54.22-0700 [APP/TASK/foo/0]OUT Creating container 2016-08-14T18:48:55.47-0700 [APP/TASK/foo/0]OUT @@ -200,9 +228,13 @@ NOTE: Unlike LRPs in Cloud Foundry, tasks are short-lived, so the logs aren't al + ``` -→ cf v3-apps -name total_desired_instances -foo 0 +→ cf apps +Getting apps in org pcfdev-org / space pcfdev-space as user... +OK + +name requested state instances memory disk urls +dataflow-server started 1/1 768M 512M dataflow-server.local.pcfdev.io +foo stopped 0/1 1G 1G ``` + @@ -215,7 +247,7 @@ dataflow:>task execution list ╔══════════════════════════╤══╤════════════════════════════╤════════════════════════════╤═════════╗ ║ Task Name │ID│ Start Time │ End Time │Exit Code║ ╠══════════════════════════╪══╪════════════════════════════╪════════════════════════════╪═════════╣ -║Demo Batch Job Task:cloud:│1 │Sun Aug 14 18:49:05 PDT 2016│Sun Aug 14 18:49:07 PDT 2016│0 ║ +║foo │1 │Sun Aug 14 18:49:05 PDT 2016│Sun Aug 14 18:49:07 PDT 2016│0 ║ ╚══════════════════════════╧══╧════════════════════════════╧════════════════════════════╧═════════╝ ``` @@ -240,5 +272,5 @@ dataflow:>job execution list In this sample, you have learned: * How to register and orchestrate Spring Batch jobs in Spring Cloud Data Flow -* How to use `v3-cf-cli` in the context of Task applications orchestrated by Spring Cloud Data Flow +* How to use the `cf` CLI in the context of Task applications orchestrated by Spring Cloud Data Flow * How to verify task executions and task repository diff --git a/tasks/simple-batch-job/batch-job-1.0.0.BUILD-SNAPSHOT.jar b/tasks/simple-batch-job/batch-job-1.3.0.BUILD-SNAPSHOT.jar similarity index 67% rename from tasks/simple-batch-job/batch-job-1.0.0.BUILD-SNAPSHOT.jar rename to tasks/simple-batch-job/batch-job-1.3.0.BUILD-SNAPSHOT.jar index eac5b49..0559780 100644 Binary files a/tasks/simple-batch-job/batch-job-1.0.0.BUILD-SNAPSHOT.jar and b/tasks/simple-batch-job/batch-job-1.3.0.BUILD-SNAPSHOT.jar differ