Commit Graph

97 Commits

Author SHA1 Message Date
Chris Bono
4f393ef9dc Polish "Move xmpp testcontainer to function-test-support"
- Removed custom container
- Removed extra whitespace
- Moved static props to SBT.properties
2022-10-28 13:38:32 -05:00
Daniel Frey
d35657a5b7 Move xmpp testcontainer to function-test-support 2022-10-28 13:22:03 -05:00
Chris Bono
378ac48e6a Update CDC Debezium to 1.9.6
- The update fixes integration tests
2022-10-25 23:15:47 -05:00
Chris Bono
afd5f82ec4 Update checkstyle plugin to 3.2.0
- Spring Javaformat update to 0.0.35
- Puppycrawl update to 10.3.4
2022-10-24 23:55:49 -05:00
Artem Bilan
09c3c742b9 Fix AmazonS3ConfigurationTests for proper configs
The `CompatibleStorageAmazonS3Configuration` and `AmazonS3Configuration`
 are really auto-configurations, so they have to take an effect already
 after end-user configurations processing

* Fix `AmazonS3ConfigurationTests` to configure an `ApplicationContextRunner`
in a proper way putting auto-configurations to their `AutoConfigurations.of()`
part and leaving only a `TestConfiguration.class` as a `UserConfiguration`
2022-10-21 09:31:38 -04:00
Chris Bono
d1e13e683a [SB3 Update] Fix checkstyle issues from previous AWS S3 commit 2022-10-20 23:23:15 -05:00
Daniel Frey
31bc02d65c Add XMPP consumer function 2022-10-20 19:51:36 -05:00
Artem Bilan
9f4ccd5e7a GH-368: Update AWS code to the latest deps (#378)
Fixes https://github.com/spring-cloud/stream-applications/issues/368

* Upgrade to `spring-cloud-aws-2.4.2`; fix respective package changes
* Remove `AutoConfigurationExclusionEnvironmentPostProcessor` since classes in the SC-AWS
are not considered as auto-configuration anymore.
Instead, add an explicit `@Import` into `AmazonS3Configuration`.
Not sure, though, why it was excluded before at all
* Fix `MetadataStoreAutoConfigurationTests` with the current state of code.
Looks like before it was designed for auto-configuration based on classpath content
* Re-enable other AWS S3 tests

This fix has helped to spot a bug in Spring Integration Testing framework:
e2d8eeb04d

Co-authored-by: Chris Bono <cbono@vmware.com>
2022-10-19 20:09:10 -05:00
Artem Bilan
c4df061599 GH-132: Upgrade to SI-HZ 6.0
Fixes https://github.com/spring-cloud/stream-applications/issues/132

* Also upgrade to `curator-5.3.0`
2022-10-19 17:54:50 -04:00
Corneil du Plessis
028df89c29 Ensure module projects skip deploy 2022-10-18 15:18:54 +02:00
Chris Bono
a07c04e856 [SB3 Update] remove unused embedded mongodb and activemq dependencies 2022-10-17 22:04:59 -05:00
Chris Bono
7098e64d12 [SB3 Update] AWS S3 updates
- aws-s3-common fix package name change
- s3-consumer fix package name change
- s3-supplier disable tests
2022-10-16 23:52:42 -05:00
Chris Bono
8800a9c002 [SB3 Update] Move MongoDbTestContainerSupport to functions/common
- use in mongodb-consumer
- use in mongodb-supplier
2022-10-16 23:52:42 -05:00
Chris Bono
e15c98183a [SB3 Update] SftpConsumer + AWS S3 (#347)
* Adjust sftp consumer knownHosts API

* AWS S3 disable tests
2022-10-09 21:09:13 -05:00
Corneil du Plessis
c64b259d91 [SB3 Update] Remove Apache Geode (#345)
- Also update SFTP usage
2022-10-07 08:22:57 -05:00
Corneil du Plessis
b5ed0bdf1e Adding redis-common and disable elasticsearch. 2022-10-06 22:43:17 +02:00
Corneil du Plessis
68f2900aad Add intermediate module poms (#337) 2022-10-03 13:51:55 -05:00
Chris Bono
790daf0d91 Convert to Autoconfiguration.imports 2022-10-02 10:58:48 -05:00
Chris Bono
208a26930c Fix spel converter auto-config 2022-10-02 10:58:48 -05:00
Chris Bono
87df01dc00 Remove ${revision} from pom.xml files 2022-10-02 10:58:48 -05:00
Chris Bono
85106dd122 Initial support for SB3/SF6
- Main branch builds successfully (with tests skipped)

See #326
2022-09-22 17:46:11 -05:00
Artem Bilan
910f116332 GH-276 Apply ComponentCustomizer in other modules (#284)
* GH-276 Apply ComponentCustomizer in other modules

Fixes https://github.com/spring-cloud/stream-applications/issues/276

Some changes are straightforward as just a `ComponentCustomizer<?>`
injection and its optional usage for the target component customization.
Other changes are more drastic since it is better to use `proxyBeanMethods = false`,
so some direct bean method calls had to be reworked to the bean method argument
injections

* Fix some deprecations from Project Reactor
* Remove `UpdatingMongoDbMessageSource` since `MongoDbMessageSource`
supports now an `update` option in Spring Integration

NOTE: The `ZeroMqSupplierConfiguration` has its own customizer already
like `Consumer<ZMQ.Socket>`, so it can be revised to a more broader solution,
but in the future version.

Some modules just does not make sense to modify for more advance customization
since all their configuration is covered with plain configuration properties.
See `splitter-function` or `wavefront-consumer`

* * Use `ObjectProvider<WebClientCustomizer>` for `HttpRequestFunctionConfiguration`
instead of our own `ComponentCustomizer`.
Although this has to be revised in favor of fully auto-configured `WebClient.Builder`.
Even that `maximumBufferSize` is included into codecs configuration properties.
* The `MongoDbMessageSource` has now an `update` option in Spring Integration
therefore remove redundant already `UpdatingMongoDbMessageSource`
and populate `this.properties.getUpdateExpression()` directly ot the
`MongoDbMessageSource` bean
* Modify supplier configurations to avoid manual starts for endpoints on
reactive `Publisher` subscriptions.
Spring Integration provides now a `toReactivePublisher(true)` operator
to have all the endpoints stopped on start, but started automatically
when subscription happens to the provided `Publisher`
* Update Copyright to the current year for all the affected classes
* Use `@Nullable` instead of `ObjectProvider` in the `AggregatorFunctionConfiguration`
dependency injection for consistency
* Fix `LogConsumerConfiguration` to not have a `log()` in the end - it is not terminal
in Spring Integration anymore.
Use recommended `nullChannel()` terminating operator for use-cases like this.

* * Remove `beanName` from `ComponentCustomizer` contract

Co-authored-by: Chris Bono <cbono@vmware.com>
2022-07-15 19:36:21 -05:00
Chris Bono
50d75ef1e7 GH-271: Remove duplication from pom.xml files
This proposal fixes many build warnings and is a general cleanup of the pom.xml files.

* Remove duplicate `dependency` entries  from pom.xml files
* Remove duplicate `repository` entries from pom.xml files
* Replace deprecated use of `receive(long)` in `HttpSourceTests`
* Reduce logging for Twitter integration tests

Fixes #271
2022-05-26 13:57:44 -04:00
Artem Bilan
78933b5474 Remove auto-config for ComponentCustomizer (#281)
Since we still need to modify every single function, consumer & supplier
for previously introduced `@CustomizationAware`, it does not do too much
difference for us if we add a `ComponentCustomizer` dependency to the
bean for customization and call it manually.

One benefit of this change is to avoid a `static` confusion and performance
overhead with extra `BeanPostProcessor`.

* Remove `ComponentCustomizerBeanPostProcessor` and
 respective `ComponentCustomizationAutoConfiguration`
* Use `@Nullable ComponentCustomizer<FileInboundChannelAdapterSpec>`
as dependency in the `FileSupplierConfiguration.fileMessageSource()` bean
2022-05-25 14:46:33 -05:00
Chris Bono
71daaaef99 Fix circular dependency and checkstyle on ComponentCustomizationAutoConfiguration (#280) 2022-05-25 00:05:08 -05:00
Artem Bilan
89cd8812c3 GH-195: Introduce ComponentCustomizer (#273)
Fixes #195 

* Introduce a `ComponentCustomizer` contract to be exposed in
the end-user configuration for particular bean type
* introduce a `@CustomizationAware` annotation to mark a component 
as customizable
* Introduce a `ComponentCustomizationBeanPostProcessor` as an auto-configured
bean to apply `ComponentCustomizer` to a matching `@CustomizationAware` (if any)
* Use `@CustomizationAware` in the `FileSupplierConfiguration`
and demonstrate `ComponentCustomizer` feature in its tests
* Move `config-common` to functions parent to make the mentioned auto-configuration
available for all the functions
* Remove some other duplicated dependencies from the affected poms
2022-05-24 20:55:11 -05:00
Chris Bono
7aa166e54c GH-268: Update Boot 2.6.7 / Spring Cloud 2021.0.2 (#269)
Also updates other dependencies accordingly. 

Fixes #268
2022-05-20 13:22:42 -05:00
Corneil du Plessis
373a2669b4 Convert snippets to Asciidoc format and added types.
Fixes #241
2022-05-04 15:10:06 +02:00
Artem Bilan
06b09013f2 Remove some redundant dependencies 2022-01-28 15:31:03 -05:00
dependabot[bot]
bb8618af86 Bump spring-core in /functions/common/tensorflow-common
Bumps [spring-core](https://github.com/spring-projects/spring-framework) from 5.2.7.RELEASE to 5.3.14.
- [Release notes](https://github.com/spring-projects/spring-framework/releases)
- [Commits](https://github.com/spring-projects/spring-framework/compare/v5.2.7.RELEASE...v5.3.14)

---
updated-dependencies:
- dependency-name: org.springframework:spring-core
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-01-28 14:24:23 -05:00
dependabot[bot]
4d7a1774cf Bump commons-compress in /functions/common/tensorflow-common
Bumps commons-compress from 1.20 to 1.21.

---
updated-dependencies:
- dependency-name: org.apache.commons:commons-compress
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-01-28 14:22:06 -05:00
Artem Bilan
fb4926f66e GH-182: Upgrade Debezium to 1.7.1
Fixes https://github.com/spring-cloud/stream-applications/issues/182

* Upgrade Debezium dependency to 1.7.1
* Remove explicit dependencies for DBs and remove excludes for them from Debezium deps;
rely fully on whatever Debezium connectors bring for us
* Fix `EmbeddedEngine` for compatibility with the Debezium 1.7.1
* Fix CDC tests for the current state of Debezium results
* Fix Checkstyle violations in the MQTT modules
2021-11-19 20:25:07 -05:00
Artem Bilan
9e9fca06d4 GH-183: Expose MQTT SSL properties
Fixes https://github.com/spring-cloud/stream-applications/issues/183

* Add `MqttProperties.sslProperties` common MQTT configuration property as a `Map` structure
* Delegate a content fo the `MqttProperties.sslProperties` into `mqttConnectOptions.setSSLProperties()`
in the common `MqttConfiguration`
* Verify that SSL properties are applied in both `MqttSupplierTests` & `MqttConsumerTests`
* Document a new `MqttProperties.sslProperties` configuration property for both consumer and supplier
* Use `eclipse-mosquitto` Docker image for MQTT tests instead of RabbitMQ plugin: the Mosquitto starts much faster
2021-11-16 14:56:43 -05:00
Christian Tzolov
54d9f29f41 jdk8_301 TLS12 jdbc handling
Related to eeeedda5cee470beb8ca8c4a76574d7c58914e9b
2021-09-28 08:33:43 -04:00
Artem Bilan
e3775e4c1a GH-171 Fix SFTP Supplier
Related to: https://github.com/spring-cloud/stream-applications/issues/171

It might not be necessary in real world, but with `SshServer` we fail
when we disconnect the session and cannot connect it back because
that server decides to die for a strange reason like no connections.

* Upgrade to SSHD 2.7.0
* Wrap default SFTP SF into a `CachingSessionFactory` to keep sessions opened
and don't let the server to die because on connections from clients
* Reenable SFTP Supplier tests
2021-06-08 16:29:17 -04:00
Soby Chacko
6184fa5a27 Disabling CDC Boot starter tests 2021-06-07 16:44:26 -04:00
Soby Chacko
2751122bcd Preparing for 2021.0.0/3.1.0 apps
Spring Boot 2.5.0
Spring Cloud Stream 3.1.x
Spring Cloud Function 3.1.x
Spring Cloud 2020.0.x

Update code/deprecations
Fix tests
Update copyrights
2021-06-04 11:37:32 -04:00
David Turanski
5d622ec07a Fix restore Geode test support used by integration tests 2021-05-20 09:39:08 -04:00
David Turanski
50ef3d647f Restore Geode test support used by integration tests 2021-05-19 10:55:32 -04:00
Andrea Montemaggio
7f37815663 GH-167: rename remote files in sftp-supplier
Fixes https://github.com/spring-cloud/stream-applications/issues/167

* Support for renaming remote files after processing via 'sftp.supplier.rename-remote-files-to' property.
2021-05-04 15:00:22 -04:00
Artem Bilan
db03925c8c GH-156: Fix NPE in the RemoteFileDeletingAdvice
Fixes https://github.com/spring-cloud/stream-applications/issues/156

The `MessageSource.receive()` may produce `null`:
The `MessageSourceMutator` impl must honor such an input

* Add `MonoProcessor<Boolean> subscriptionBarrier` to delay
subscription to the source `Flux` until subscription happens
to the supplier's flux.
This way we don't have unexpected interaction with the source when
there are regular endpoints in the flow in between
2021-04-13 09:26:30 -04:00
Dan Smith
943a1c9faa Use Spring Test Data Geode instead of containerized geode server
Use spring test data geode in the geode integration tests
test to fork a separate geode process, instead of running
a geode server in a docker container.

STDG is faster than launching Geode through gfsh inside of a container.
- It doesn't run in docker
- It doesn't start management components necessary for gfsh
- It doesn't start extra gfsh shell processes

Removing the GeodeContainer test utility
2021-04-13 08:39:16 -04:00
David Turanski
d6d2e37b48 Tag integration tests and enable integration profile 2021-02-22 14:09:22 -05:00
Artem Bilan
456d8d9333 Use compatible image version with Debezium
* Fix MongoDb driver problem.
Probably we need to think about upgrading Debezium support to the latest
version, so its MongoDb connector is based on the latest Mongo Driver
which already is pulled by Spring Boot
2021-02-19 12:31:34 -05:00
Artem Bilan
b384f1ab97 Upgrade Testcontainers tests to the latest images
* Un`@Disable` updated tests
2021-02-11 16:21:06 -05:00
Artem Bilan
85d7494e6d Upgrade to the latest Testcontainers
* Use `debezium/example-mysql:latest` image in the `CdcBootStarterIntegrationTest`
to avoid failing when some transitive dependency does not exist any more
2021-02-11 11:08:12 -05:00
Christian Tzolov
55855fc74a Fix cdc logging classpath issue
- Exclude a slf4j-log4j12 dependency inherited by slf4j-log4j12 that clashes with the spring boot logging.
2021-02-01 10:41:39 +01:00
Christian Tzolov
64523b0bae Fix CDC issues
- Fix cdc_key header string wrapping issue that causes problems with the spring.cloud.stream.kafka.default.producer.messageKeyExpression.
 - Rename Flattering property to the correct Flattening.
 - Remove wrong dependencies.
2020-12-04 19:50:45 +01:00
David Turanski
5b3de3ac18 Upgrade Testcontainers, Debezium, and fix exceptions in MailSupplierTests 2020-11-06 12:32:33 -05:00
David Turanski
e0f01a4481 Upgrade AWS S3 dependencies 2020-10-30 09:37:53 -04:00