SpringCloud/spring-cloud-dataflow-samples
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
147 Commits 1 Branch 0 Tags
158f455ffcaefb0ebe164c38ac41aef98b2c283c
Commit Graph

147 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Glenn Renfro
158f455ffc Supports CTR section of the dataflow.spring.io 
resolves another part of #180

It is associated with PR https://github.com/spring-io/dataflow.spring.io/pull/188
 2020-01-29 15:57:42 -05:00
Mark Pollack
f205626b4f Update to junit5/assert4j in batch sample 
Fixes #127
 2020-01-29 15:37:01 -05:00
Janne Valkealahti
42f83b10c8 Apply suggestions from code review 
Co-Authored-By: Sabby Anandan <sanandan@pivotal.io>
 2020-01-27 15:51:30 +00:00
Janne Valkealahti
f6062684ea Update javadsl sample 
- Bump up to boot and dataflow versions.
- Document authentication settings.
- Fixes #126
 2020-01-27 15:51:30 +00:00
Christian Tzolov
a128253930 Add SCDF stream autoscaling example 
- Add K8s on-disk installation configuration yaml files to extend the SCDF k8s on-disk installation.
 - Add K8s on-disk installation configuration yaml files to extend the SCDF k8s Helm installation.
 - Add AlertWebhookReciver Spring Boot app.
 - Publish the webhook app to springcloud DockerHub
 2019-12-31 17:44:13 +01:00
Glenn Renfro
c3f5c9e65c Migrates existing schedules to SCDF 2.3.0 
resolves #121

This app pulls in data from the PCFScheduler, AppSummary, AppManifest, the SCDF Task Definition Schema
to construct the new schedule for the migration.

You will note that for the App Summary, the code was pulled directly from the CF-Deployer, the reason
this was done was to obtain additional schedule information required for the migration, without having to make
another call to retrieve the app summary (i.e. obtain this detail while its still there).

As discussed in the README, you will see that not all properties could be obtained from the
available resources and thus have to be populated by a property that user can establish at runtime.
However this also means this affects all schedules to be migrated.

Updated to allow user to setup maven repository

Updated instructions to include how to execute migration

Updated docs

Updated to support deployer format for tasks

Also updated deployers to current release
 2019-12-17 11:37:29 -06:00
Christian Tzolov
f6a75bac23 fix broken grafana-twitter-scdf-analytics.json link 2019-11-13 20:12:30 +01:00
Christian Tzolov
e42c2888a3 udate task-apps pom dependencies and docs 2019-11-07 16:28:04 +01:00
Christian Tzolov
2b295dafb1 udate task-apps pom versions 2019-11-07 16:02:28 +01:00
Christian Tzolov
3a417a8a23 fix formating 2019-11-07 10:20:09 +01:00
Christian Tzolov
36607f7d6c Update the Monitoring samples sections 
- Add links to the microsite.
 - Remove obsoleted content
 2019-11-06 18:27:19 +01:00
Christian Tzolov
f8078ed08f RSocket Proxy for Prometheus monitoring 
- Rename spring-cloud-dataflow-samples/tree/master/micrometer to spring-cloud-dataflow-samples/tree/master/micrometer-samples
 - Rename spring-cloud-dataflow-samples/tree/master/micrometer/tasks to spring-cloud-dataflow-samples/tree/master/micrometer-samples/task-apps
 - Add the apps under spring-cloud-dataflow-samples/tree/master/micrometer/stream-apps
 - Describe or cross-link the description of the new architecture in the root README
 - Add microsite SCDF monitoring links

 Resolves #122
 2019-11-06 16:29:17 +01:00
Christian Tzolov
369ecce9ba remove noise messages form the snippets 2019-10-04 12:44:09 +02:00
Christian Tzolov
94ec1b534e Update tiwtter analytics diagram 2019-10-04 11:20:19 +02:00
Christian Tzolov
c1b5463d5c Fix Twitter Analytics Guide 
- Update the pre-build dashboard to set the prometheus Datasource name on import.
  - Fix the Data Flow and Shell download links.
  - Simplify docker compose installation instructions.
  - Add animated workflow diagram diagram.
 2019-10-04 10:22:40 +02:00
Christian Tzolov
56b745bef3 address review comments 2019-09-30 13:42:47 +02:00
Christian Tzolov
ab8f6970b1 Create Task Monitoring Sample 
- Add Influx Task Monitoring sample
 - Add Prometheus Task Monitoring sample

 Resolves #119
 2019-09-25 14:23:40 +02:00
Christian Tzolov
72498e0bdc move monitoring demo tasks under dataflow-website structure 2019-08-28 14:54:59 +02:00
Christian Tzolov
ad50f3883d add pre-build influx and prometheus demo task apps 2019-08-28 14:02:06 +02:00
Sabby Anandan
119bbafc37 Remove any and all references to bit.ly 2019-08-07 12:29:11 +05:30
Spring Operator
2d40a251fe URL Cleanup 
This commit updates URLs to prefer the https protocol. Redirects are not followed to avoid accidentally expanding intentionally shortened URLs (i.e. if using a URL shortener).

# Fixed URLs

## Fixed Success
These URLs were switched to an https URL with a 2xx status. While the status was successful, your review is still recommended.

* [ ] http://www.apache.org/licenses/ with 4 occurrences migrated to:
  https://www.apache.org/licenses/ ([https](https://www.apache.org/licenses/) result 200).
* [ ] http://www.apache.org/licenses/LICENSE-2.0 with 27 occurrences migrated to:
  https://www.apache.org/licenses/LICENSE-2.0 ([https](https://www.apache.org/licenses/LICENSE-2.0) result 200).
 2019-08-07 12:09:13 +05:30
Spring Operator
4876279a01 URL Cleanup 
This commit updates URLs to prefer the https protocol. Redirects are not followed to avoid accidentally expanding intentionally shortened URLs (i.e. if using a URL shortener).

These URLs were unable to be fixed. Please review them to see if they can be manually resolved.

* [ ] http://my-token.com (200) with 1 occurrences could not be migrated:
   ([https](https://my-token.com) result ConnectTimeoutException).
* [ ] http://xslthl.sf.net (301) with 4 occurrences could not be migrated:
   ([https](https://xslthl.sf.net) result AnnotatedConnectException).
* [ ] http://exslt.org/common (404) with 1 occurrences could not be migrated:
   ([https](https://exslt.org/common) result SSLHandshakeException).
* [ ] http://data-flow-server-dpvuo77-stocks-http.apps.scdf-gcp.springapps.io/ (502) with 1 occurrences could not be migrated:
   ([https](https://data-flow-server-dpvuo77-stocks-http.apps.scdf-gcp.springapps.io/) result SSLHandshakeException).

These URLs were fixed, but the https status was not OK. However, the https status was the same as the http request or http redirected to an https URL, so they were migrated. Your review is recommended.

* [ ] http://192.168.64.1:23057 (ConnectTimeoutException) with 1 occurrences migrated to:
  https://192.168.64.1:23057 ([https](https://192.168.64.1:23057) result ConnectTimeoutException).
* [ ] http://192.168.64.1:60081 (ConnectTimeoutException) with 1 occurrences migrated to:
  https://192.168.64.1:60081 ([https](https://192.168.64.1:60081) result ConnectTimeoutException).
* [ ] http://dataflow-server.local.pcfdev.io (ConnectTimeoutException) with 1 occurrences migrated to:
  https://dataflow-server.local.pcfdev.io ([https](https://dataflow-server.local.pcfdev.io) result ConnectTimeoutException).
* [ ] http://www.w3.org/2000/09/xmldsig (ReadTimeoutException) with 2 occurrences migrated to:
  https://www.w3.org/2000/09/xmldsig ([https](https://www.w3.org/2000/09/xmldsig) result SSLException).
* [ ] http://.../pulse (UnknownHostException) with 3 occurrences migrated to:
  https://.../pulse ([https](https://.../pulse) result UnknownHostException).
* [ ] http://cassandrastream-http.app.io (UnknownHostException) with 1 occurrences migrated to:
  https://cassandrastream-http.app.io ([https](https://cassandrastream-http.app.io) result UnknownHostException).
* [ ] http://dataflow-server.app.io (UnknownHostException) with 3 occurrences migrated to:
  https://dataflow-server.app.io ([https](https://dataflow-server.app.io) result UnknownHostException).
* [ ] http://dataflow.local:8080/uaa/check_token (UnknownHostException) with 2 occurrences migrated to:
  https://dataflow.local:8080/uaa/check_token ([https](https://dataflow.local:8080/uaa/check_token) result UnknownHostException).
* [ ] http://dataflow.local:8080/uaa/oauth/authorize (UnknownHostException) with 2 occurrences migrated to:
  https://dataflow.local:8080/uaa/oauth/authorize ([https](https://dataflow.local:8080/uaa/oauth/authorize) result UnknownHostException).
* [ ] http://dataflow.local:8080/uaa/oauth/token (UnknownHostException) with 2 occurrences migrated to:
  https://dataflow.local:8080/uaa/oauth/token ([https](https://dataflow.local:8080/uaa/oauth/token) result UnknownHostException).
* [ ] http://dataflow.local:8080/uaa/userinfo (UnknownHostException) with 2 occurrences migrated to:
  https://dataflow.local:8080/uaa/userinfo ([https](https://dataflow.local:8080/uaa/userinfo) result UnknownHostException).
* [ ] http://link.to/icon.jpg (UnknownHostException) with 1 occurrences migrated to:
  https://link.to/icon.jpg ([https](https://link.to/icon.jpg) result UnknownHostException).
* [ ] http://my-auth.com (UnknownHostException) with 2 occurrences migrated to:
  https://my-auth.com ([https](https://my-auth.com) result UnknownHostException).
* [ ] http://mysqlstream-http.app.io (UnknownHostException) with 2 occurrences migrated to:
  https://mysqlstream-http.app.io ([https](https://mysqlstream-http.app.io) result UnknownHostException).

These URLs were switched to an https URL with a 2xx status. While the status was successful, your review is still recommended.

* [ ] http://cassandra.apache.org/ with 1 occurrences migrated to:
  https://cassandra.apache.org/ ([https](https://cassandra.apache.org/) result 200).
* [ ] http://cloud.spring.io/spring-cloud-dataflow/ with 14 occurrences migrated to:
  https://cloud.spring.io/spring-cloud-dataflow/ ([https](https://cloud.spring.io/spring-cloud-dataflow/) result 200).
* [ ] http://cloud.spring.io/spring-cloud-function/ with 1 occurrences migrated to:
  https://cloud.spring.io/spring-cloud-function/ ([https](https://cloud.spring.io/spring-cloud-function/) result 200).
* [ ] http://cloud.spring.io/spring-cloud-task/ with 1 occurrences migrated to:
  https://cloud.spring.io/spring-cloud-task/ ([https](https://cloud.spring.io/spring-cloud-task/) result 200).
* [ ] http://docs.spring.io/spring-cloud-dataflow/docs/2.0.0.BUILD-SNAPSHOT/reference/htmlsingle/ with 8 occurrences migrated to:
  https://docs.spring.io/spring-cloud-dataflow/docs/2.0.0.BUILD-SNAPSHOT/reference/htmlsingle/ ([https](https://docs.spring.io/spring-cloud-dataflow/docs/2.0.0.BUILD-SNAPSHOT/reference/htmlsingle/) result 200).
* [ ] http://docs.spring.io/spring-cloud-dataflow/docs/current/reference/htmlsingle/ with 2 occurrences migrated to:
  https://docs.spring.io/spring-cloud-dataflow/docs/current/reference/htmlsingle/ ([https](https://docs.spring.io/spring-cloud-dataflow/docs/current/reference/htmlsingle/) result 200).
* [ ] http://docs.spring.io/spring-cloud-stream/docs/ with 1 occurrences migrated to:
  https://docs.spring.io/spring-cloud-stream/docs/ ([https](https://docs.spring.io/spring-cloud-stream/docs/) result 200).
* [ ] http://example.com/ with 2 occurrences migrated to:
  https://example.com/ ([https](https://example.com/) result 200).
* [ ] http://geode.apache.org with 1 occurrences migrated to:
  https://geode.apache.org ([https](https://geode.apache.org) result 200).
* [ ] http://geode.apache.org/ with 1 occurrences migrated to:
  https://geode.apache.org/ ([https](https://geode.apache.org/) result 200).
* [ ] http://kafka.apache.org/downloads.html with 5 occurrences migrated to:
  https://kafka.apache.org/downloads.html ([https](https://kafka.apache.org/downloads.html) result 200).
* [ ] http://projects.spring.io/spring-batch/ with 1 occurrences migrated to:
  https://projects.spring.io/spring-batch/ ([https](https://projects.spring.io/spring-batch/) result 200).
* [ ] http://prometheus.io with 3 occurrences migrated to:
  https://prometheus.io ([https](https://prometheus.io) result 200).
* [ ] http://raw.githubusercontent.com/spring-cloud/spring-cloud-dataflow/master/spring-cloud-dataflow-docs/src/main/asciidoc/images/micrometer-arch.png with 1 occurrences migrated to:
  https://raw.githubusercontent.com/spring-cloud/spring-cloud-dataflow/master/spring-cloud-dataflow-docs/src/main/asciidoc/images/micrometer-arch.png ([https](https://raw.githubusercontent.com/spring-cloud/spring-cloud-dataflow/master/spring-cloud-dataflow-docs/src/main/asciidoc/images/micrometer-arch.png) result 200).
* [ ] http://repo.spring.io/snapshot/org/springframework/cloud/spring-cloud-skipper-server/2.0.0.BUILD-SNAPSHOT/spring-cloud-skipper-server-2.0.0.BUILD-SNAPSHOT.jar with 1 occurrences migrated to:
  https://repo.spring.io/snapshot/org/springframework/cloud/spring-cloud-skipper-server/2.0.0.BUILD-SNAPSHOT/spring-cloud-skipper-server-2.0.0.BUILD-SNAPSHOT.jar ([https](https://repo.spring.io/snapshot/org/springframework/cloud/spring-cloud-skipper-server/2.0.0.BUILD-SNAPSHOT/spring-cloud-skipper-server-2.0.0.BUILD-SNAPSHOT.jar) result 200).
* [ ] http://start.spring.io/ with 1 occurrences migrated to:
  https://start.spring.io/ ([https](https://start.spring.io/) result 200).
* [ ] http://www.mysql.com/ with 1 occurrences migrated to:
  https://www.mysql.com/ ([https](https://www.mysql.com/) result 200).
* [ ] http://bit.ly/Dearborn-GA-task-applications-maven with 1 occurrences migrated to:
  https://bit.ly/Dearborn-GA-task-applications-maven ([https](https://bit.ly/Dearborn-GA-task-applications-maven) result 301).
* [ ] http://bit.ly/Einstein-RC1-stream-applications-rabbit-maven with 1 occurrences migrated to:
  https://bit.ly/Einstein-RC1-stream-applications-rabbit-maven ([https](https://bit.ly/Einstein-RC1-stream-applications-rabbit-maven) result 301).
* [ ] http://dbeaver.jkiss.org/ with 4 occurrences migrated to:
  https://dbeaver.jkiss.org/ ([https](https://dbeaver.jkiss.org/) result 301).
* [ ] http://www.okta.com/k2lw4l5bPODCMIIDBRYZ with 1 occurrences migrated to:
  https://www.okta.com/k2lw4l5bPODCMIIDBRYZ ([https](https://www.okta.com/k2lw4l5bPODCMIIDBRYZ) result 301).
* [ ] http://www.okta.com/k36wkjw6EAEJVZXFFDAU with 1 occurrences migrated to:
  https://www.okta.com/k36wkjw6EAEJVZXFFDAU ([https](https://www.okta.com/k36wkjw6EAEJVZXFFDAU) result 301).
* [ ] http://bit.ly/ with 2 occurrences migrated to:
  https://bit.ly/ ([https](https://bit.ly/) result 302).
* [ ] http://repo.spring.io/libs-milestone with 1 occurrences migrated to:
  https://repo.spring.io/libs-milestone ([https](https://repo.spring.io/libs-milestone) result 302).
* [ ] http://repo.spring.io/libs-release with 1 occurrences migrated to:
  https://repo.spring.io/libs-release ([https](https://repo.spring.io/libs-release) result 302).

These URLs were intentionally ignored.

* http://docbook.org/ns/docbook with 4 occurrences
* http://docbook.sourceforge.net/xmlns/l10n/1.0 with 2 occurrences
* http://localhost:3000 with 2 occurrences
* http://localhost:3001 with 1 occurrences
* http://localhost:35357/v2.0/tokens with 1 occurrences
* http://localhost:5000/v3/auth/tokens with 1 occurrences
* http://localhost:7577/login with 1 occurrences
* http://localhost:8080/dashboard with 1 occurrences
* http://localhost:8080/uaa with 4 occurrences
* http://localhost:8080/uaa/approvals with 1 occurrences
* http://localhost:8080/uaa/authenticate with 1 occurrences
* http://localhost:8080/uaa/oauth/authorize with 1 occurrences
* http://localhost:8080/uaa/oauth/token with 3 occurrences
* http://localhost:8081/openam/saml2/jsp/exportmetadata.jsp?entityid=http://localhost:8081/openam with 1 occurrences
* http://localhost:8787 with 3 occurrences
* http://localhost:8888 with 3 occurrences
* http://localhost:9000 with 14 occurrences
* http://localhost:9001 with 23 occurrences
* http://localhost:9090 with 6 occurrences
* http://localhost:9100 with 3 occurrences
* http://localhost:9393 with 4 occurrences
* http://localhost:9393/about with 1 occurrences
* http://localhost:9393/apps with 1 occurrences
* http://localhost:9393/audit-records with 1 occurrences
* http://localhost:9393/completions with 1 occurrences
* http://localhost:9393/dashboard with 2 occurrences
* http://localhost:9393/jobs with 1 occurrences
* http://localhost:9393/login with 1 occurrences
* http://localhost:9393/runtime with 1 occurrences
* http://localhost:9393/runtime/apps with 1 occurrences
* http://localhost:9393/security with 1 occurrences
* http://localhost:9393/streams with 1 occurrences
* http://localhost:9393/tasks with 1 occurrences
* http://localhost:9393/tasks/executions/current with 1 occurrences
* http://localhost:9393/tools with 1 occurrences
* http://localhost:9900 with 1 occurrences
* http://www.w3.org/1999/XSL/Format with 2 occurrences
* http://www.w3.org/1999/XSL/Transform with 6 occurrences
* http://www.w3.org/1999/xlink with 1 occurrences
 2019-08-07 12:05:16 +05:30
Spring Operator
c3829ad82b URL Cleanup 
This commit updates URLs to prefer the https protocol. Redirects are not followed to avoid accidentally expanding intentionally shortened URLs (i.e. if using a URL shortener).

# HTTP URLs that Could Not Be Fixed
These URLs were unable to be fixed. Please review them to see if they can be manually resolved.

* http://xslthl.sf.net (301) with 1 occurrences could not be migrated:
   ([https](https://xslthl.sf.net) result AnnotatedConnectException).

# Fixed URLs

## Fixed But Review Recommended
These URLs were fixed, but the https status was not OK. However, the https status was the same as the http request or http redirected to an https URL, so they were migrated. Your review is recommended.

* http://www.puppycrawl.com/dtds/configuration_1_2.dtd (404) with 1 occurrences migrated to:
  https://www.puppycrawl.com/dtds/configuration_1_2.dtd ([https](https://www.puppycrawl.com/dtds/configuration_1_2.dtd) result 404).

## Fixed Success
These URLs were switched to an https URL with a 2xx status. While the status was successful, your review is still recommended.

* http://asciidoctor.org with 1 occurrences migrated to:
  https://asciidoctor.org ([https](https://asciidoctor.org) result 200).
* http://maven.apache.org/xsd/assembly-1.1.2.xsd with 1 occurrences migrated to:
  https://maven.apache.org/xsd/assembly-1.1.2.xsd ([https](https://maven.apache.org/xsd/assembly-1.1.2.xsd) result 200).
* http://maven.apache.org/xsd/maven-4.0.0.xsd with 11 occurrences migrated to:
  https://maven.apache.org/xsd/maven-4.0.0.xsd ([https](https://maven.apache.org/xsd/maven-4.0.0.xsd) result 200).
* http://sourceforge.net/projects/xslthl/ with 14 occurrences migrated to:
  https://sourceforge.net/projects/xslthl/ ([https](https://sourceforge.net/projects/xslthl/) result 200).
* http://www.w3.org/TR/CSS21/propidx.html with 1 occurrences migrated to:
  https://www.w3.org/TR/CSS21/propidx.html ([https](https://www.w3.org/TR/CSS21/propidx.html) result 200).

# Ignored
These URLs were intentionally ignored.

* http://maven.apache.org/POM/4.0.0 with 22 occurrences
* http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2 with 2 occurrences
* http://www.w3.org/2001/XMLSchema-instance with 12 occurrences
 2019-08-07 11:19:05 +05:30
Janne Valkealahti
b3baeaa4bc Add scdf custom build example for 2.2.x 2019-07-30 21:01:41 +01:00
Janne Valkealahti
2d0c3cd5d7 Maven and Gradle samples for dataflow server 
- Minimal and fully working sample of building custom
  dataflow server adding mysql driver.
 2019-07-26 08:18:13 +01:00
David Turanski
dbbf78588b Merge pull request #109 from dturanski/issue-108 
Fix MAXCONCURRENTTASKS
 2019-07-11 11:29:18 -04:00
David Turanski
fdca5776c6 Fix MAXCONCURRENTTASKS 2019-07-10 11:36:28 -04:00
Sabby Anandan
f5da6f8812 Add Kinesis sample 
* Adjust consumer group property from producer vs. consumer
 2019-06-28 14:53:26 -04:00
David Turanski
71a217b5d7 Add zip dist 2019-06-13 09:20:42 -04:00
David Turanski
6590b02e93 Update generic SCDF install instructions,app releases, and fix stream commands format 2019-06-13 07:49:30 -04:00
David Turanski
50bf5a10b0 Moved file-to-jdbc code 2019-06-12 10:30:19 -04:00
Gunnar Hillert
6d6cced468 Add Data Flow Load Generator Example 
The `dataflow-template-load-generator` will be able to create a parameterized amount of task applications and generate a definable amount of task/batch executions.
 2019-06-05 22:40:00 -10:00
Sabby Anandan
15b91f35cf Fix typo in python recipe 
Resolves #104
 2019-06-06 08:54:46 +02:00
Christian Tzolov
4686ceb8d8 replace some orders/bar occurences 2019-05-17 20:28:20 +02:00
Christian Tzolov
90ba371006 Replace the Barista by Python-Router app use case 2019-05-17 19:19:47 +02:00
Christian Tzolov
3be812b758 add polyglot-python-processor.zip archive 2019-05-16 14:29:54 +02:00
Christian Tzolov
1959f91c83 Resolve a gihub python-processor issue 2 2019-05-16 12:37:34 +02:00
Christian Tzolov
3b6cdc1417 Resolve a gihub python-processor issue 2019-05-16 12:36:57 +02:00
Christian Tzolov
5669221fb0 Add polyglot-python-processor sample 2019-05-16 11:50:55 +02:00
Christian Tzolov
c92d406dcc add 2019-05-16 11:50:55 +02:00
Ilayaperumal Gopinathan
52f6dd4964 Update samples to use built-in configuration properties 2019-05-14 23:29:17 +05:30
Christian Tzolov
7d3e7efbda rename scdf-python-app to polyglot-python-app 2019-05-14 16:22:04 +02:00
Christian Tzolov
3e1628bd54 Add scdf-python-app. fix polyglot-python-task issue 2019-05-13 23:41:25 +02:00
Ilayaperumal Gopinathan
0d84299e61 Refactor function composition samples 2019-05-09 23:24:01 +05:30
Ilayaperumal Gopinathan
45337f11dc Add source code zip of function composition samples 2019-05-09 22:14:57 +05:30
Ilayaperumal Gopinathan
d68c26d2e4 Add function composition samples 2019-05-09 22:00:06 +05:30
Ilayaperumal Gopinathan
a09872c0bf Add zip of stream dev guide samples' source code 2019-05-08 15:37:35 +05:30
Ilayaperumal Gopinathan
e6f5ae2adc Add tests for stream samples in dev guide 2019-05-08 07:38:54 +05:30
Christian Tzolov
4294d5f97a Add zipped archive of the polyglot-python-task sample 2019-05-07 10:57:24 +02:00
Christian Tzolov
108299b07e Add the polyglot python-task sample 2019-05-07 09:47:19 +02:00