diff --git a/applications/processor/aggregator-processor/README.adoc b/applications/processor/aggregator-processor/README.adoc index 9feec308..7c4f6339 100644 --- a/applications/processor/aggregator-processor/README.adoc +++ b/applications/processor/aggregator-processor/README.adoc @@ -7,7 +7,7 @@ Aggregator processor enables an application to aggregates incoming messages into Change kafka to rabbit if you want to run it against RabbitMQ. -=== Payload +== Payload If an input payload is a `byte[]` and content-type header is a JSON, then `JsonBytesToMap` function tries to deserialize this payload to a `Map` for better data representation on the output of the aggregator function. Also, such a `Map` data representation makes it easy to access to the payload content from SpEL expressions mentioned below. diff --git a/applications/processor/bridge-processor/README.adoc b/applications/processor/bridge-processor/README.adoc index 82ce758b..1b94ad1c 100644 --- a/applications/processor/bridge-processor/README.adoc +++ b/applications/processor/bridge-processor/README.adoc @@ -3,7 +3,7 @@ A processor that bridges the input and ouput by simply passing the incoming payload to the outbound. -=== Payload +== Payload Any diff --git a/applications/processor/filter-processor/README.adoc b/applications/processor/filter-processor/README.adoc index de47f738..4d6030e0 100644 --- a/applications/processor/filter-processor/README.adoc +++ b/applications/processor/filter-processor/README.adoc @@ -8,7 +8,7 @@ For example, if the incoming payload is of type `String` and you want to filter Change kafka to rabbit if you want to run it against RabbitMQ. -=== Payload +== Payload You can pass any type as payload and then apply SpEL expressions against it to filter. If the incoming type is `byte[]` and the content type is set to `text/plain` or `application/json`, then the application converts the `byte[]` into `String`. diff --git a/applications/processor/image-recognition-processor/README.adoc b/applications/processor/image-recognition-processor/README.adoc index 7fb79c0b..ca4ab9b0 100644 --- a/applications/processor/image-recognition-processor/README.adoc +++ b/applications/processor/image-recognition-processor/README.adoc @@ -13,18 +13,21 @@ The input of the model is an image as binary array. The output is a JSON message in this format: -```json +[source,json] +.... { "labels" : [ {"giant panda":0.98649305} ] } -``` +.... + Result contains the name of the recognized category (e.g. label) along with the confidence (e.g. confidence) that the image represents this category. If the `response-seize` is set to value higher then 1, then the result will include the top `response-seize` probable labels. For example `response-size=3` would return: -```json +[source,json] +.... { "labels": [ {"giant panda":0.98649305}, @@ -32,9 +35,9 @@ If the `response-seize` is set to value higher then 1, then the result will incl {"ice bear":0.001130851} ] } -``` +.... -=== Payload +== Payload If the incoming type is `byte[]` and the content type is set to `application/octet-stream` , then the application process the input `byte[]` image into and outputs augmented `byte[]` image payload and json header. diff --git a/applications/processor/object-detection-processor/README.adoc b/applications/processor/object-detection-processor/README.adoc index 345fa79c..99f742b0 100644 --- a/applications/processor/object-detection-processor/README.adoc +++ b/applications/processor/object-detection-processor/README.adoc @@ -19,7 +19,8 @@ image::{image-root}/scdf-tensorflow-object-detection-arch.png[] Processor's input is an image byte array, and the output is an augmented image, and a header, called `detected_objects`, that provides textual description of the detected objects: -```json +[source,json] +.... { "labels" : [ {"name":"person", "confidence":0.9996774,"x1":0.0,"y1":0.3940161,"x2":0.9465165,"y2":0.5592592,"cid":1}, @@ -28,7 +29,7 @@ Processor's input is an image byte array, and the output is an augmented image, {"name":"backpack", "confidence":0.963343,"x1":0.1273736,"y1":0.57658505,"x2":0.47765,"y2":0.6986431,"cid":23} ] } -``` +.... The `detected_objects` header format is: @@ -36,7 +37,7 @@ The `detected_objects` header format is: * *x1*, *y1*, *x2*, *y2* - Response also provides the bounding box of the detected objects represented as `(x1, y1, x2, y2)`. The coordinates are relative to the size of the image size. * *cid* - Classification identifier as defined in the provided https://github.com/tensorflow/models/tree/865c14c/research/object_detection/data[labels] configuration file. -=== Payload +== Payload The incoming type is `byte[]`, and the content type is `application/octet-stream`. The processor processes the input `byte[]` image and outputs an augmented `byte[]` image payload and a JSON header (`detected_objects`). diff --git a/applications/processor/semantic-segmentation-processor/README.adoc b/applications/processor/semantic-segmentation-processor/README.adoc index 2a8ba95a..56a04c24 100644 --- a/applications/processor/semantic-segmentation-processor/README.adoc +++ b/applications/processor/semantic-segmentation-processor/README.adoc @@ -10,20 +10,21 @@ For implementing `Instance Segmentation` consult the https://github.com/spring-c The `Semantic Segmentation Processor` uses the https://github.com/spring-cloud/stream-applications/tree/master/functions/function/semantic-segmentation-function[Semantic Segmentation Function] library and the https://github.com/spring-cloud/stream-applications/tree/master/functions/common/tensorflow-common[TensorFlow Service]. -=== Payload +== Payload The incoming type is `byte[]`, and the content type is `application/octet-stream`. The processor processes the input `byte[]` image and outputs augmented `byte[]` image payload and json header. Processor's input is an image byte array, and the output is an augmented image byte array, and a JSON header `semantic_segmentation` in this format: -```json +[source,json] +.... [ [ 0, 0, 0 ], [ 127, 127, 127 ], - [ 255, 255, 255 ] + [ 255, 255, 255 ], ... ] -``` +.... The output header json format represents the color pixel map computed from the input image. diff --git a/applications/processor/splitter-processor/README.adoc b/applications/processor/splitter-processor/README.adoc index 08834790..482fa9da 100644 --- a/applications/processor/splitter-processor/README.adoc +++ b/applications/processor/splitter-processor/README.adoc @@ -5,7 +5,7 @@ The splitter app builds upon the concept of the same name in Spring Integration The processor uses a function that takes a `Message` as input and then produces a `List` as output based on various properties (see below). You can use a SpEL expression or a delimiter to specify how you want to split the incoming message. -=== Payload +== Payload * Incoming payload - `Message diff --git a/applications/processor/transform-processor/README.adoc b/applications/processor/transform-processor/README.adoc index 42f58824..4a4eeb4f 100644 --- a/applications/processor/transform-processor/README.adoc +++ b/applications/processor/transform-processor/README.adoc @@ -5,12 +5,16 @@ Transformer processor allows you to convert the message payload structure based Here is an example of how you can run this application. -`java -jar transform-processor-kafka-.jar --spel.function.expression=payload.toUpperCase()` +[source,shell] +.... +java -jar transform-processor-kafka-.jar \ + --spel.function.expression=payload.toUpperCase() +.... Change kafka to rabbit if you want to run it against RabbitMQ. -=== Payload +== Payload The incoming message can contain any type of payload. diff --git a/applications/sink/analytics-sink/README.adoc b/applications/sink/analytics-sink/README.adoc index 3d1b92b9..c73cf07a 100644 --- a/applications/sink/analytics-sink/README.adoc +++ b/applications/sink/analytics-sink/README.adoc @@ -28,26 +28,31 @@ You can add support for additional monitoring systems by just adding their micro Please visit the https://dataflow.spring.io/docs/2.6.0.SNAPSHOT/feature-guides/streams/monitoring/[Spring Cloud Data Flow Stream Monitoring] for detailed instructions for configuring the Monitoring Systems. The following quick snippets can help you start. * To enable the Prometheus meter registry, set the following properties. -``` + +[source,properties] +.... management.metrics.export.prometheus.enabled=true management.metrics.export.prometheus.rsocket.enabled=true management.metrics.export.prometheus.rsocket.host=$$`)* //end::configuration-properties[] //end::ref-doc[] diff --git a/applications/source/debezium-source/README.adoc b/applications/source/debezium-source/README.adoc index 94dac3bf..b3e7025c 100644 --- a/applications/source/debezium-source/README.adoc +++ b/applications/source/debezium-source/README.adoc @@ -140,7 +140,7 @@ The debezium integration tests use databases fixtures, running on the local mach To run and debug the tests from your IDE you need to deploy the required database images from the command line. Instructions below explains how to run pre-configured test databases form Docker images. -==== MySQL +=== MySQL Start the `debezium/example-mysql` in a docker: [source, bash] @@ -149,14 +149,13 @@ docker run -it --rm --name mysql -p 3306:3306 -e MYSQL_ROOT_PASSWORD=debezium -e ---- [TIP] -==== (optional) Use `mysql` client to connected to the database and to create a `debezium` user with required credentials: [source, bash] ---- docker run -it --rm --name mysqlterm --link mysql --rm mysql:5.7 sh -c 'exec mysql -h"$MYSQL_PORT_3306_TCP_ADDR" -P"$MYSQL_PORT_3306_TCP_PORT" -uroot -p"$MYSQL_ENV_MYSQL_ROOT_PASSWORD"' mysql> GRANT SELECT, RELOAD, SHOW DATABASES, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'debezium' IDENTIFIED BY 'dbz'; ---- -==== + Use following properties to connect the Debezium Source to MySQL DB: @@ -201,7 +200,7 @@ You can run also the `DebeziumDatabasesIntegrationTest#mysql()` using this mysql NOTE: Disable the mysql GenericContainer test initialization code. -==== PostgreSQL +=== PostgreSQL Start a pre-configured postgres server from the `debezium/example-postgres:1.0` Docker image: [source, bash] @@ -256,7 +255,7 @@ You can run also the `DebeziumDatabasesIntegrationTest#postgres()` using this po NOTE: Disable the postgres GenericContainer test initialization code. -==== MongoDB +=== MongoDB Start a pre-configured mongodb from the `debezium/example-mongodb:2.2.0.Final` container image: [source, bash] @@ -319,7 +318,7 @@ debezium.properties.transforms.unwrap.drop.tombstones=true # <6> You can run also the `DebeziumDatabasesIntegrationTest#mongodb()` using this mongodb configuration. -==== SQL Server +=== SQL Server Start a `sqlserver` from the `debezium/example-postgres:1.0` Docker image: [source, bash] @@ -361,7 +360,7 @@ debezium.properties.database.port=1433 # <4> You can run also the `DebeziumDatabasesIntegrationTest#sqlServer()` using this SqlServer configuration. -==== Oracle +=== Oracle Start Oracle reachable from localhost and set up with the configuration, users and grants described in the https://github.com/debezium/oracle-vagrant-box[Debezium Vagrant set-up] diff --git a/applications/source/http-source/README.adoc b/applications/source/http-source/README.adoc index 2bb03088..06ee1262 100644 --- a/applications/source/http-source/README.adoc +++ b/applications/source/http-source/README.adoc @@ -5,7 +5,7 @@ A source application that listens for HTTP requests and emits the body as a mess If the Content-Type matches `text/*` or `application/json`, the payload will be a String, otherwise the payload will be a byte array. -==== Payload: +== Payload: If content type matches `text/*` or `application/json` diff --git a/applications/source/jdbc-source/README.adoc b/applications/source/jdbc-source/README.adoc index 0a59e162..662d466f 100644 --- a/applications/source/jdbc-source/README.adoc +++ b/applications/source/jdbc-source/README.adoc @@ -4,7 +4,7 @@ This source polls data from an RDBMS. This source is fully based on the `DataSourceAutoConfiguration`, so refer to the https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-sql.html[Spring Boot JDBC Support] for more information. -=== Payload +== Payload * `Map` when `jdbc.split == true` (default) and `List>` otherwise diff --git a/applications/source/mqtt-source/README.adoc b/applications/source/mqtt-source/README.adoc index ed098c9b..14fe33c8 100644 --- a/applications/source/mqtt-source/README.adoc +++ b/applications/source/mqtt-source/README.adoc @@ -3,7 +3,7 @@ Source that enables receiving messages from MQTT. -==== Payload: +== Payload: * `String` if binary setting is `false` (default) * `byte[]` if binary setting is `true` diff --git a/applications/source/s3-source/README.adoc b/applications/source/s3-source/README.adoc index 51ce8f74..ec604edf 100644 --- a/applications/source/s3-source/README.adoc +++ b/applications/source/s3-source/README.adoc @@ -18,9 +18,9 @@ The payload of these 2 additional marker messages is of type `FileSplitter.FileM See also link:../../../functions/common/metadata-store-common/README.adoc[MetadataStore] options for possible shared persistent store configuration used to prevent duplicate messages on restart. -=== mode = lines +== mode = lines -==== Headers: +=== Headers: * `Content-Type: text/plain` * `file_orginalFile: ` @@ -29,7 +29,7 @@ See also link:../../../functions/common/metadata-store-common/README.adoc[Metada * `sequenceNumber: ` * `sequenceSize: 0` (number of lines is not know until the file is read) -==== Payload: +=== Payload: A `String` for each line. @@ -38,13 +38,13 @@ The last line is optionally followed by a message with an `END` marker payload. Marker presence and format are determined by the `with-markers` and `markers-json` properties. -=== mode = ref +== mode = ref -==== Headers: +=== Headers: None. -==== Payload: +=== Payload: A `java.io.File` object. diff --git a/applications/stream-applications-core/stream-applications-test-support/src/main/java/org/springframework/cloud/stream/app/test/integration/rabbitmq/RabbitMQConfig.java b/applications/stream-applications-core/stream-applications-test-support/src/main/java/org/springframework/cloud/stream/app/test/integration/rabbitmq/RabbitMQConfig.java index e6087aeb..cbbdee6a 100644 --- a/applications/stream-applications-core/stream-applications-test-support/src/main/java/org/springframework/cloud/stream/app/test/integration/rabbitmq/RabbitMQConfig.java +++ b/applications/stream-applications-core/stream-applications-test-support/src/main/java/org/springframework/cloud/stream/app/test/integration/rabbitmq/RabbitMQConfig.java @@ -37,6 +37,7 @@ public abstract class RabbitMQConfig { static { rabbitmq = new RabbitMQContainer(DockerImageName.parse("rabbitmq:3.8-management")) .withNetwork(network) + .withNetworkAliases("rabbitmq") .withExposedPorts(5672, 15672) .withStartupTimeout(Duration.ofSeconds(120)) .withStartupAttempts(3); diff --git a/applications/stream-applications-core/stream-applications-test-support/src/main/java/org/springframework/cloud/stream/app/test/integration/rabbitmq/RabbitMQStreamAppContainerTestConfiguration.java b/applications/stream-applications-core/stream-applications-test-support/src/main/java/org/springframework/cloud/stream/app/test/integration/rabbitmq/RabbitMQStreamAppContainerTestConfiguration.java index 4c4e6d26..e13d5ce2 100644 --- a/applications/stream-applications-core/stream-applications-test-support/src/main/java/org/springframework/cloud/stream/app/test/integration/rabbitmq/RabbitMQStreamAppContainerTestConfiguration.java +++ b/applications/stream-applications-core/stream-applications-test-support/src/main/java/org/springframework/cloud/stream/app/test/integration/rabbitmq/RabbitMQStreamAppContainerTestConfiguration.java @@ -133,7 +133,7 @@ public abstract class RabbitMQStreamAppContainerTestConfiguration { public Client rabbitClient() { Client client; try { - client = new Client("http://guest:guest@localhost:" + rabbitmq.getMappedPort(15672) + "/api"); + client = new Client("http://guest:guest@" + rabbitmq.getHost() + ":" + rabbitmq.getMappedPort(15672) + "/api"); } catch (Exception e) { throw new BeanCreationException(e.getMessage(), e); diff --git a/applications/stream-applications-integration-tests/README.adoc b/applications/stream-applications-integration-tests/README.adoc index 361577ff..38a3083a 100644 --- a/applications/stream-applications-integration-tests/README.adoc +++ b/applications/stream-applications-integration-tests/README.adoc @@ -32,7 +32,7 @@ options. These tests use Spring but not boot currently. The most important setting is the image versions to test. To override it, set the System property, e.g., -./mvnw clean test -Dspring.cloud.stream.applications.version=3.0.0-M3 +./mvnw clean test -Dspring.cloud.stream.applications.version=4.0.0-RC1 diff --git a/applications/stream-applications-integration-tests/src/test/java/org/springframework/cloud/stream/app/integration/test/common/Configuration.java b/applications/stream-applications-integration-tests/src/test/java/org/springframework/cloud/stream/app/integration/test/common/Configuration.java index d5d1a14d..fa7c1489 100644 --- a/applications/stream-applications-integration-tests/src/test/java/org/springframework/cloud/stream/app/integration/test/common/Configuration.java +++ b/applications/stream-applications-integration-tests/src/test/java/org/springframework/cloud/stream/app/integration/test/common/Configuration.java @@ -35,6 +35,7 @@ public abstract class Configuration { static { VERSION = System.getProperty(SPRING_CLOUD_STREAM_APPLICATIONS_VERSION, "latest"); + System.out.println("VERSION=" + VERSION); } public static class VersionSupplier implements Supplier { diff --git a/applications/stream-applications-integration-tests/src/test/java/org/springframework/cloud/stream/app/integration/test/source/s3/S3SourceTests.java b/applications/stream-applications-integration-tests/src/test/java/org/springframework/cloud/stream/app/integration/test/source/s3/S3SourceTests.java index 8a09e0f3..91d3033e 100644 --- a/applications/stream-applications-integration-tests/src/test/java/org/springframework/cloud/stream/app/integration/test/source/s3/S3SourceTests.java +++ b/applications/stream-applications-integration-tests/src/test/java/org/springframework/cloud/stream/app/integration/test/source/s3/S3SourceTests.java @@ -36,6 +36,8 @@ import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Tag; import org.junit.jupiter.api.Test; import org.junit.jupiter.api.extension.ExtendWith; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; import org.testcontainers.containers.GenericContainer; import org.testcontainers.containers.wait.strategy.Wait; import org.testcontainers.junit.jupiter.Container; @@ -55,7 +57,7 @@ import static org.springframework.cloud.stream.app.test.integration.StreamAppCon @Tag("integration") @ExtendWith(BaseContainerExtension.class) abstract class S3SourceTests { - + private static final Logger logger = LoggerFactory.getLogger(S3SourceTests.class); private static AmazonS3 s3Client; private StreamAppContainer source; @@ -65,8 +67,9 @@ abstract class S3SourceTests { @Container private static final GenericContainer minio = new GenericContainer( - DockerImageName.parse("minio/minio:RELEASE.2020-10-18T21-54-12Z")) + DockerImageName.parse("minio/minio:latest")) .withExposedPorts(9000) + .withNetworkAliases("minio-host") .withEnv("MINIO_ACCESS_KEY", "minio") .withEnv("MINIO_SECRET_KEY", "minio123") .waitingFor(Wait.forHttp("/minio/health/live")) @@ -82,11 +85,11 @@ abstract class S3SourceTests { static void initS3() { AWSCredentials credentials = new BasicAWSCredentials("minio", "minio123"); ClientConfiguration clientConfiguration = new ClientConfiguration(); + String minioAddress = "http://" + minio.getHost() + ":" + minio.getMappedPort(9000); + logger.info("minio:address={}", minioAddress); s3Client = AmazonS3ClientBuilder .standard() - .withEndpointConfiguration( - new AwsClientBuilder.EndpointConfiguration("http://localhost:" + minio.getMappedPort(9000), - Regions.US_EAST_1.name())) + .withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration(minioAddress, Regions.US_EAST_1.name())) .withPathStyleAccessEnabled(true) .withClientConfiguration(clientConfiguration) .withCredentials(new AWSStaticCredentialsProvider(credentials)) @@ -97,7 +100,7 @@ abstract class S3SourceTests { @BeforeEach void configureSource() { source = BaseContainerExtension.containerInstance() - .withEnv("S3_COMMON_ENDPOINT_URL", "http://" + localHostAddress() + ":" + minio.getMappedPort(9000)) + .withEnv("S3_COMMON_ENDPOINT_URL", "http://minio-host:" + minio.getMappedPort(9000)) .withEnv("S3_COMMON_PATH_STYLE_ACCESS", "true") .withEnv("CLOUD_AWS_STACK_AUTO", "false") .withEnv("CLOUD_AWS_CREDENTIALS_ACCESS_KEY", "minio") diff --git a/applications/stream-applications-integration-tests/src/test/java/org/springframework/cloud/stream/app/integration/test/source/sftp/SftpSourceTests.java b/applications/stream-applications-integration-tests/src/test/java/org/springframework/cloud/stream/app/integration/test/source/sftp/SftpSourceTests.java index 7e7790b3..74ff17fe 100644 --- a/applications/stream-applications-integration-tests/src/test/java/org/springframework/cloud/stream/app/integration/test/source/sftp/SftpSourceTests.java +++ b/applications/stream-applications-integration-tests/src/test/java/org/springframework/cloud/stream/app/integration/test/source/sftp/SftpSourceTests.java @@ -37,6 +37,7 @@ import org.springframework.cloud.stream.app.test.integration.junit.jupiter.BaseC import static org.awaitility.Awaitility.await; import static org.springframework.cloud.stream.app.integration.test.common.Configuration.DEFAULT_DURATION; + @Tag("integration") @ExtendWith(BaseContainerExtension.class) abstract class SftpSourceTests { @@ -45,21 +46,22 @@ abstract class SftpSourceTests { @Container private static final GenericContainer sftp = new GenericContainer(DockerImageName.parse("atmoz/sftp")) - .withExposedPorts(22) - .withCommand("user:pass:::remote") - .withClasspathResourceMapping("sftp", "/home/user/remote", BindMode.READ_ONLY) - .withStartupTimeout(DEFAULT_DURATION); + .withExposedPorts(22) + .withNetworkAliases("sftp-host") + .withCommand("user:pass:::remote") + .withClasspathResourceMapping("sftp", "/home/user/remote", BindMode.READ_ONLY) + .withStartupTimeout(DEFAULT_DURATION); @BeforeEach void configureSource() { - await().atMost(DEFAULT_DURATION).until(() -> sftp.isRunning()); + await().atMost(DEFAULT_DURATION).until(sftp::isRunning); source = BaseContainerExtension.containerInstance() - .withEnv("SFTP_SUPPLIER_FACTORY_ALLOW_UNKNOWN_KEYS", "true") - .withEnv("SFTP_SUPPLIER_REMOTE_DIR", "/remote") - .withEnv("SFTP_SUPPLIER_FACTORY_USERNAME", "user") - .withEnv("SFTP_SUPPLIER_FACTORY_PASSWORD", "pass") - .withEnv("SFTP_SUPPLIER_FACTORY_PORT", String.valueOf(sftp.getMappedPort(22))) - .withEnv("SFTP_SUPPLIER_FACTORY_HOST", StreamAppContainerTestUtils.localHostAddress()); + .withEnv("SFTP_SUPPLIER_FACTORY_ALLOW_UNKNOWN_KEYS", "true") + .withEnv("SFTP_SUPPLIER_REMOTE_DIR", "/remote") + .withEnv("SFTP_SUPPLIER_FACTORY_USERNAME", "user") + .withEnv("SFTP_SUPPLIER_FACTORY_PASSWORD", "pass") + .withEnv("SFTP_SUPPLIER_FACTORY_PORT", String.valueOf(sftp.getMappedPort(22))) + .withEnv("SFTP_SUPPLIER_FACTORY_HOST", "sftp-host"); } @Autowired @@ -71,7 +73,7 @@ abstract class SftpSourceTests { startContainer(Collections.singletonMap("FILE_CONSUMER_MODE", "ref")); await().atMost(DEFAULT_DURATION) - .until(outputMatcher.payloadMatches((String s) -> s.equals("\"/tmp/sftp-supplier/data.txt\""))); + .until(outputMatcher.payloadMatches((String s) -> s.equals("\"/tmp/sftp-supplier/data.txt\""))); } private void startContainer(Map environment) { diff --git a/applications/stream-applications-integration-tests/src/test/java/org/springframework/cloud/stream/app/integration/test/source/time/TimeSourceTests.java b/applications/stream-applications-integration-tests/src/test/java/org/springframework/cloud/stream/app/integration/test/source/time/TimeSourceTests.java index 4a1638d4..ae4ec26e 100644 --- a/applications/stream-applications-integration-tests/src/test/java/org/springframework/cloud/stream/app/integration/test/source/time/TimeSourceTests.java +++ b/applications/stream-applications-integration-tests/src/test/java/org/springframework/cloud/stream/app/integration/test/source/time/TimeSourceTests.java @@ -74,26 +74,26 @@ abstract class TimeSourceTests { void testActuator() { WebClient webClient = WebClient.create(); webClient.get() - .uri("http://localhost:" + source.getMappedPort(8080) + "/actuator/health") + .uri("http://" + source.getHost() + ":" + source.getMappedPort(8080) + "/actuator/health") .headers(h -> h.setBasicAuth("user", "password")) .exchangeToMono(response -> { assertThat(response.statusCode()).isEqualTo(HttpStatus.OK); return response.toBodilessEntity(); }).block(); webClient.get() - .uri("http://localhost:" + source.getMappedPort(8080) + "/actuator/info") + .uri("http://" + source.getHost() + ":" + source.getMappedPort(8080) + "/actuator/info") .exchangeToMono(response -> { assertThat(response.statusCode()).isEqualTo(HttpStatus.OK); return response.toBodilessEntity(); }).block(); webClient.get() - .uri("http://localhost:" + source.getMappedPort(8080) + "/actuator/bindings") + .uri("http://" + source.getHost() + ":" + source.getMappedPort(8080) + "/actuator/bindings") .exchangeToMono(response -> { assertThat(response.statusCode()).isEqualTo(HttpStatus.OK); return response.toBodilessEntity(); }).block(); webClient.get() - .uri("http://localhost:" + source.getMappedPort(8080) + "/actuator/env") + .uri("http://" + source.getHost() + ":" + source.getMappedPort(8080) + "/actuator/env") .exchangeToMono(response -> { assertThat(response.statusCode()).isEqualTo(HttpStatus.NOT_FOUND); return response.toBodilessEntity(); diff --git a/build-docs.sh b/build-docs.sh new file mode 100755 index 00000000..d440e842 --- /dev/null +++ b/build-docs.sh @@ -0,0 +1,8 @@ +#!/usr/bin/env bash +SCDIR=$(dirname "$(readlink -f "${BASH_SOURCE[0]}")") +SCDIR=$(realpath $SCDIR) +MAVEN_GOALS="install deploy" +if [ "$1" != "" ]; then + MAVEN_GOALS="$*" +fi +$SCDIR/build-folder.sh stream-applications-release-train/stream-applications-docs "$MAVEN_GOALS" diff --git a/build-folder.sh b/build-folder.sh index fa9df4f6..48c444da 100755 --- a/build-folder.sh +++ b/build-folder.sh @@ -30,7 +30,7 @@ if [ "$MAVEN_OPTS" == "" ]; then else MAVEN_OPTS="$MAVEN_OPTS -s $SCDIR/.settings.xml -B" fi - +MAVEN_OPTS="$MAVEN_OPTS -Pfull" if [ "$1" == "" ]; then echo -e "Options: ${bold} []*${end}" fi diff --git a/stream-applications-build/pom.xml b/stream-applications-build/pom.xml index 7d96bf33..37bc2bd3 100644 --- a/stream-applications-build/pom.xml +++ b/stream-applications-build/pom.xml @@ -12,7 +12,7 @@ 17 - 3.4.1 + 3.5.0 3.2.1 3.0.0 UTF-8 @@ -53,7 +53,10 @@ 4.0.5 3.1.0 - + 1.12.322 + 2.5.2 + 3.0.0 + 5.3.0 1 diff --git a/stream-applications-release-train/stream-applications-docs/pom.xml b/stream-applications-release-train/stream-applications-docs/pom.xml index 4873e65c..27459251 100644 --- a/stream-applications-release-train/stream-applications-docs/pom.xml +++ b/stream-applications-release-train/stream-applications-docs/pom.xml @@ -13,6 +13,10 @@ 0.2.5 + ${basedir}/../.. + + 6.1.0 + 5.2.3 @@ -372,14 +376,52 @@ prepare-package + + + com.amazonaws + aws-java-sdk-dynamodb + ${aws-java-sdk.version} + + + org.apache.curator + curator-framework + ${curator.version} + + + org.apache.curator + curator-client + ${curator.version} + + + org.springframework.integration + spring-integration-hazelcast + ${spring-integration-hazelcast.version} + + + org.springframework.integration + spring-integration-zookeeper + ${spring-integration.version} + + + jakarta.jms + jakarta.jms-api + ${jakarta-jms.version} + + + com.hazelcast + hazelcast + ${hazelcast.version} + + true org.springframework.cloud.stream.app:* org.springframework.cloud.fn:* + false - true - none + false + true ${basedir}/src/main/javadoc/spring-javadoc.css @@ -391,36 +433,6 @@ - - - org.apache.maven.plugins - maven-dependency-plugin - 3.3.0 - - - unpack-doc-resources - - unpack-dependencies - - generate-resources - - io.spring.docresources - spring-doc-resources - zip - true - ${project.build.directory}/refdocs/ - - - - - - io.spring.docresources - spring-doc-resources - ${docs.resources.version} - zip - - - org.apache.maven.plugins maven-resources-plugin @@ -447,16 +459,20 @@ org.asciidoctor asciidoctor-maven-plugin - 2.2.2 + 2.2.4 org.asciidoctor asciidoctorj-pdf - 2.3.4 + 2.3.7 + + + io.spring.asciidoctor.backends + spring-asciidoctor-backends + 0.0.5 - html generate-resources @@ -464,7 +480,7 @@ process-asciidoc - html5 + spring-html highlight.js ${project.build.directory}/contents/reference/html @@ -479,6 +495,7 @@ spring.css js/highlight github + ${rootDir} @@ -494,6 +511,9 @@ ${project.build.directory}/contents/reference/pdf coderay + + ${rootDir} + @@ -511,6 +531,9 @@ 4 true ${project.basedir} + ${rootDir} + ${spring-cloud-stream-dependencies.version} + ${spring.version} @@ -597,7 +620,7 @@ org.codehaus.mojo build-helper-maven-plugin - 1.9.1 + 3.4.0 attach-zip @@ -610,7 +633,7 @@ ${project.build.directory}/${project.artifactId}-${project.version}.zip - zip;zip.type=docs;zip.deployed=false + zip diff --git a/stream-applications-release-train/stream-applications-docs/src/main/asciidoc/index.adoc b/stream-applications-release-train/stream-applications-docs/src/main/asciidoc/index.adoc index 3b93e54b..f4692d4e 100644 --- a/stream-applications-release-train/stream-applications-docs/src/main/asciidoc/index.adoc +++ b/stream-applications-release-train/stream-applications-docs/src/main/asciidoc/index.adoc @@ -13,11 +13,13 @@ Sabby Anandan; Artem Bilan; Marius Bogoevici; Eric Bottard; Mark Fisher; Ilayape :branch: main +:stream-apps-folder: {stream-apps-root}/${branch} + :stream-apps-asciidoc: https://raw.githubusercontent.com/spring-cloud/stream-applications/main/stream-applications-release-train/stream-applications-docs/src/main/asciidoc -:spring-core-version: 5.3.20 +:spring-core-version: 6.0.9 -:scst-core-version: 3.2.0 +:scst-core-version: 4.0.3 ifdef::backend-html5[] diff --git a/stream-applications-release-train/stream-applications-docs/src/main/asciidoc/processors.adoc b/stream-applications-release-train/stream-applications-docs/src/main/asciidoc/processors.adoc index 16379f45..a7733d39 100644 --- a/stream-applications-release-train/stream-applications-docs/src/main/asciidoc/processors.adoc +++ b/stream-applications-release-train/stream-applications-docs/src/main/asciidoc/processors.adoc @@ -1,46 +1,47 @@ +:stream-apps-folder: ../../../../.. [[processors]] == Processors :leveloffset: +2 [[spring-cloud-stream-modules-aggregator-processor]] -include::{stream-apps-root}/{branch}/applications/processor/aggregator-processor/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/processor/aggregator-processor/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-bridge-processor]] -include::{stream-apps-root}/{branch}/applications/processor/bridge-processor/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/processor/bridge-processor/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-filter-processor]] -include::{stream-apps-root}/{branch}/applications/processor/filter-processor/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/processor/filter-processor/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-groovy-processor]] -include::{stream-apps-root}/{branch}/applications/processor/groovy-processor/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/processor/groovy-processor/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-header-enricher-processor]] -include::{stream-apps-root}/{branch}/applications/processor/header-enricher-processor/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/processor/header-enricher-processor/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-http-request-processor]] -include::{stream-apps-root}/{branch}/applications/processor/http-request-processor/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/processor/http-request-processor/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-image-recognition-processor]] -include::{stream-apps-root}/{branch}/applications/processor/image-recognition-processor/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/processor/image-recognition-processor/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-object-detection-processor]] -include::{stream-apps-root}/{branch}/applications/processor/object-detection-processor/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/processor/object-detection-processor/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-semantic-segmentation-processor]] -include::{stream-apps-root}/{branch}/applications/processor/semantic-segmentation-processor/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/processor/semantic-segmentation-processor/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-script-processor]] -include::{stream-apps-root}/{branch}/applications/processor/script-processor/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/processor/script-processor/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-splitter-processor]] -include::{stream-apps-root}/{branch}/applications/processor/splitter-processor/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/processor/splitter-processor/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-transform-processor]] -include::{stream-apps-root}/{branch}/applications/processor/transform-processor/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/processor/transform-processor/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-twitter-trend-processor]] -include::{stream-apps-root}/{branch}/applications/processor/twitter-trend-processor/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/processor/twitter-trend-processor/README.adoc[tags=ref-doc] :leveloffset: -2 diff --git a/stream-applications-release-train/stream-applications-docs/src/main/asciidoc/sinks.adoc b/stream-applications-release-train/stream-applications-docs/src/main/asciidoc/sinks.adoc index db77f9e2..fc674c99 100644 --- a/stream-applications-release-train/stream-applications-docs/src/main/asciidoc/sinks.adoc +++ b/stream-applications-release-train/stream-applications-docs/src/main/asciidoc/sinks.adoc @@ -1,75 +1,76 @@ +:stream-apps-folder: ../../../../.. [[sinks]] == Sinks :leveloffset: +2 [[spring-cloud-stream-modules-cassandra-sink]] -include::{stream-apps-root}/{branch}/applications/sink/cassandra-sink/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/sink/cassandra-sink/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-analytics-sink]] -include::{stream-apps-root}/{branch}/applications/sink/analytics-sink/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/sink/analytics-sink/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-elasticsearch-sink]] -include::{stream-apps-root}/{branch}/applications/sink/elasticsearch-sink/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/sink/elasticsearch-sink/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-file-sink]] -include::{stream-apps-root}/{branch}/applications/sink/file-sink/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/sink/file-sink/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-ftp-sink]] -include::{stream-apps-root}/{branch}/applications/sink/ftp-sink/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/sink/ftp-sink/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-jdbc-sink]] -include::{stream-apps-root}/{branch}/applications/sink/jdbc-sink/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/sink/jdbc-sink/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-log-sink]] -include::{stream-apps-root}/{branch}/applications/sink/log-sink/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/sink/log-sink/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-mongodb-sink]] -include::{stream-apps-root}/{branch}/applications/sink/mongodb-sink/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/sink/mongodb-sink/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-mqtt-sink]] -include::{stream-apps-root}/{branch}/applications/sink/mqtt-sink/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/sink/mqtt-sink/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-pgcopy-sink]] -include::{stream-apps-root}/{branch}/applications/sink/pgcopy-sink/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/sink/pgcopy-sink/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-rabbit-sink]] -include::{stream-apps-root}/{branch}/applications/sink/rabbit-sink/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/sink/rabbit-sink/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-redis-sink]] -include::{stream-apps-root}/{branch}/applications/sink/redis-sink/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/sink/redis-sink/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-router-sink]] -include::{stream-apps-root}/{branch}/applications/sink/router-sink/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/sink/router-sink/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-rsocket-sink]] -include::{stream-apps-root}/{branch}/applications/sink/rsocket-sink/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/sink/rsocket-sink/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-s3-sink]] -include::{stream-apps-root}/{branch}/applications/sink/s3-sink/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/sink/s3-sink/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-sftp-sink]] -include::{stream-apps-root}/{branch}/applications/sink/sftp-sink/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/sink/sftp-sink/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-tcp-sink]] -include::{stream-apps-root}/{branch}/applications/sink/tcp-sink/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/sink/tcp-sink/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-throughput-sink]] -include::{stream-apps-root}/{branch}/applications/sink/throughput-sink/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/sink/throughput-sink/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-twitter-message-sink]] -include::{stream-apps-root}/{branch}/applications/sink/twitter-message-sink/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/sink/twitter-message-sink/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-twitter-update-sink]] -include::{stream-apps-root}/{branch}/applications/sink/twitter-update-sink/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/sink/twitter-update-sink/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-wavefront-sink]] -include::{stream-apps-root}/{branch}/applications/sink/wavefront-sink/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/sink/wavefront-sink/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-websocket-sink]] -include::{stream-apps-root}/{branch}/applications/sink/websocket-sink/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/sink/websocket-sink/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-zeromq-sink]] -include::{stream-apps-root}/{branch}/applications/sink/zeromq-sink/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/sink/zeromq-sink/README.adoc[tags=ref-doc] :leveloffset: -2 diff --git a/stream-applications-release-train/stream-applications-docs/src/main/asciidoc/sources.adoc b/stream-applications-release-train/stream-applications-docs/src/main/asciidoc/sources.adoc index 80e7dc64..b7640b88 100644 --- a/stream-applications-release-train/stream-applications-docs/src/main/asciidoc/sources.adoc +++ b/stream-applications-release-train/stream-applications-docs/src/main/asciidoc/sources.adoc @@ -1,69 +1,70 @@ +:stream-apps-folder: ../../../../.. [[sources]] == Sources :leveloffset: +2 [[spring-cloud-stream-modules-debezium-source]] -include::{stream-apps-root}/{branch}/applications/source/debezium-source/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/source/debezium-source/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-file-source]] -include::{stream-apps-root}/{branch}/applications/source/file-source/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/source/file-source/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-ftp-source]] -include::{stream-apps-root}/{branch}/applications/source/ftp-source/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/source/ftp-source/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-http-source]] -include::{stream-apps-root}/{branch}/applications/source/http-source/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/source/http-source/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-jdbc-source]] -include::{stream-apps-root}/{branch}/applications/source/jdbc-source/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/source/jdbc-source/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-jms-source]] -include::{stream-apps-root}/{branch}/applications/source/jms-source/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/source/jms-source/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-load-generator-source]] -include::{stream-apps-root}/{branch}/applications/source/load-generator-source/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/source/load-generator-source/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-mail-source]] -include::{stream-apps-root}/{branch}/applications/source/mail-source/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/source/mail-source/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-mongodb-source]] -include::{stream-apps-root}/{branch}/applications/source/mongodb-source/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/source/mongodb-source/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-mqtt-source]] -include::{stream-apps-root}/{branch}/applications/source/mqtt-source/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/source/mqtt-source/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-rabbit-source]] -include::{stream-apps-root}/{branch}/applications/source/rabbit-source/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/source/rabbit-source/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-s3-source]] -include::{stream-apps-root}/{branch}/applications/source/s3-source/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/source/s3-source/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-sftp-source]] -include::{stream-apps-root}/{branch}/applications/source/sftp-source/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/source/sftp-source/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-syslog-source]] -include::{stream-apps-root}/{branch}/applications/source/syslog-source/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/source/syslog-source/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-tcp-source]] -include::{stream-apps-root}/{branch}/applications/source/tcp-source/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/source/tcp-source/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-time-source]] -include::{stream-apps-root}/{branch}/applications/source/time-source/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/source/time-source/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-twitter-message-source]] -include::{stream-apps-root}/{branch}/applications/source/twitter-message-source/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/source/twitter-message-source/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-twitter-search-source]] -include::{stream-apps-root}/{branch}/applications/source/twitter-search-source/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/source/twitter-search-source/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-twitter-stream-source]] -include::{stream-apps-root}/{branch}/applications/source/twitter-stream-source/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/source/twitter-stream-source/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-websocket-source]] -include::{stream-apps-root}/{branch}/applications/source/websocket-source/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/source/websocket-source/README.adoc[tags=ref-doc] [[spring-cloud-stream-modules-zeromq-source]] -include::{stream-apps-root}/{branch}/applications/source/zeromq-source/README.adoc[tags=ref-doc] +include::{stream-apps-folder}/applications/source/zeromq-source/README.adoc[tags=ref-doc] :leveloffset: -2