187 Commits

Author SHA1 Message Date
Artem Bilan
6a380c4d19 Change doOnSubscribe to doOnRequest
The `aggregatorFunction` has a flaw subscribing and emitting
data from a source `Flux` too early: from a `doOnSubscribe()`
callback which is really called *before* the real subscription is
registered in the target `Publisher`

Change the logic to the `doOnRequest()` which happens already
after subscription is registered in the `Publisher`

**Cherry-pick to `2020.0.x`**
2022-01-20 18:35:30 -05:00
Christian Tzolov
993bc7f42b jdk8_301 TLS12 jdbc handling
Related to eeeedda5ce
2021-09-28 09:47:54 -04:00
Soby Chacko
3a3e9ff831 Next version updates (non-apps) 2021-05-28 12:50:33 -04:00
Soby Chacko
d3980438df Functions 1.0.3 Release 2021-05-27 18:25:37 -04:00
Soby Chacko
a70912b50f Update dependencies 2021-05-27 11:10:54 -04:00
Soby Chacko
255fc69b0c Fix checkstyle issues 2021-05-27 10:25:12 -04:00
Soby Chacko
2712dab49b Refactor the usage of output channel in suppliers
Redesigning suppliers that use reactive output channel name as
the literal "output" explicitly, as this  cuases some conflicts
with the output channel name usage in Spring Cloud Stream.
The fix is done either by simply renaming the channel name
or by avoid using it directly, but delegate its use through the SI Java DSL.

Resolves https://github.com/spring-cloud/stream-applications/issues/160
2021-05-26 15:53:01 -04:00
Soby Chacko
4ebd18ee4e Update versions
boot 2.3.11
Spring Kafka 2.5.13
SCSt Horsham.SR12/3.0.12.RELEASE
SCFn 3.0.14.RELEASE
Spring Cloud Hoxton.SR11
2021-05-20 18:27:05 -04:00
David Turanski
fbeb89b5b7 Fix restore Geode test support used by integration tests 2021-05-20 09:39:08 -04:00
David Turanski
5cbd171f1b Restore Geode test support used by integration tests 2021-05-19 10:55:32 -04:00
Andrea Montemaggio
f2e3ce271e GH-167: rename remote files in sftp-supplier
Fixes https://github.com/spring-cloud/stream-applications/issues/167

* Support for renaming remote files after processing via 'sftp.supplier.rename-remote-files-to' property.
2021-05-04 15:00:22 -04:00
Andrea Montemaggio
63dd5bc043 Added support for sync and async bulk-indexing.
New properties for controlling the batch size and the timeout for a message group have been added, consistently with the jdbc-consumer implementation.
Added support for per message index override via INDEX_NAME header.
Overriding behavior of message headers mentioned in properties javadoc.
Improved debug log messages.

Copyright and authors update.

README
2021-05-03 17:49:31 -04:00
Artem Bilan
04beb9ecaf GH-156: Fix NPE in the RemoteFileDeletingAdvice
Fixes https://github.com/spring-cloud/stream-applications/issues/156

The `MessageSource.receive()` may produce `null`:
The `MessageSourceMutator` impl must honor such an input

* Add `MonoProcessor<Boolean> subscriptionBarrier` to delay
subscription to the source `Flux` until subscription happens
to the supplier's flux.
This way we don't have unexpected interaction with the source when
there are regular endpoints in the flow in between
2021-04-13 09:26:30 -04:00
Dan Smith
9faa2a6a38 Use Spring Test Data Geode instead of containerized geode server
Use spring test data geode in the geode integration tests
test to fork a separate geode process, instead of running
a geode server in a docker container.

STDG is faster than launching Geode through gfsh inside of a container.
- It doesn't run in docker
- It doesn't start management components necessary for gfsh
- It doesn't start extra gfsh shell processes

Removing the GeodeContainer test utility
2021-04-13 08:39:16 -04:00
Soby Chacko
2fed22a13f Update spring-cloud-function to 3.0.14.BUILD-SNAPSHOT.
Enable filter processor test.

Resolves https://github.com/spring-cloud/stream-applications/issues/162
2021-04-07 17:05:50 -04:00
Artem Bilan
bbf04eff2e Limit aggregator tests for 10 seconds to wait 2021-04-07 12:02:02 -04:00
Soby Chacko
2a0edea722 Fixing TCP Source issues
Exclude `IpHeaders.LOCAL_ADDRESS` from TCP source output as it
causes issues with message conversion downstream.

Update SCSt/Spring-Cloud to the latest snapshots.

* Refactor `TcpSupplierConfiguration` for better readability and
current state of requirements for Spring libraries configuration
2021-03-30 09:00:21 -04:00
Thomas Recloux
126c53e8ad GH-151: Remove output bean from the Rabbit Supplier
Fixes https://github.com/spring-cloud/stream-applications/issues/151

The `RabbitSupplierConfiguration` comes with the `output` bean definition which
clashes with the `output` binding for Spring Cloud Stream.
The last one doesn't register the binding bean and leave the rest of the logic
to deal with existing bean when we resolve it by the `output` name.

* Rework the logic of the `RabbitSupplierConfiguration` to avoid an extra bean
in between and the binding in Spring Cloud Stream do it job
2021-03-10 13:09:12 -05:00
Artem Bilan
4eb34fb245 GH-154: Change FORM CT to JSON for HTTP supplier
Fixes https://github.com/spring-cloud/stream-applications/issues/154

The Binder conversion mechanism is not aware about a `MediaType.APPLICATION_FORM_URLENCODED`
and does not know how to convert the `MultiValueMap` with such a content-type

* Change the Http-Supplier to replace a `MediaType.APPLICATION_FORM_URLENCODED`
content-type header to the `MediaType.APPLICATION_JSON` for the proper
JSON serialization for the `MultiValueMap` on the binder level before sending to
the output destination
2021-03-09 11:49:08 -05:00
Nicolas Labrot
c899e71a64 GH-152: RabbitMQ consumer: Add headersMappedLast
Fixes https://github.com/spring-cloud/stream-applications/issues/152

* Add the possibilities to configure the `headersMappedLast` for the RabbitMQ consumer

* Changes following the code review
2021-02-26 17:00:36 -05:00
Soby Chacko
fa58243b31 Next version updates (non-apps) 2021-02-26 15:25:26 -05:00
Soby Chacko
eceda16e0a Functions 1.0.2 Release 2021-02-25 15:34:54 -05:00
David Turanski
f1d9f36dee Tag integration tests and enable integration profile 2021-02-22 14:09:22 -05:00
Artem Bilan
386dad6562 Use compatible image version with Debezium
* Fix MongoDb driver problem.
Probably we need to think about upgrading Debezium support to the latest
version, so its MongoDb connector is based on the latest Mongo Driver
which already is pulled by Spring Boot
2021-02-19 12:31:34 -05:00
Soby Chacko
721c506891 Update dependencies
Spring Boot 2.3.9
Spring Kafka 2.5.11
Spring Cloud Stream - Horsham.SR11
Spring Cloud - Hoxon.SR10
Spring Cloud Function - 3.0.13
2021-02-18 16:09:25 -05:00
Soby Chacko
dc0e31e132 Update next versions 2021-02-18 15:49:23 -05:00
Soby Chacko
b986495332 Functions 1.0.1 Release 2021-02-18 14:31:44 -05:00
Artem Bilan
e2c11b4773 Upgrade Testcontainers tests to the latest images
* Un`@Disable` updated tests
2021-02-11 16:21:06 -05:00
Artem Bilan
d3c38406e8 Upgrade to the latest Testcontainers
* Use `debezium/example-mysql:latest` image in the `CdcBootStarterIntegrationTest`
to avoid failing when some transitive dependency does not exist any more
2021-02-11 11:08:12 -05:00
Soby Chacko
63c22a85d7 tests debugging continues 2021-02-10 14:26:24 -05:00
Artem Bilan
1e6a2e1f44 GH-143: Add an update option to MongoDB Supplier (#147)
* GH-143: Add an update option to MongoDB Supplier

Fixes https://github.com/spring-cloud/stream-applications/issues/143

* * Fix Checkstyle violations
* Update `README.adoc` for `mongodb-source`
2021-02-05 15:36:03 -05:00
Soby Chacko
fec91314b9 More checkstyle fixes 2021-02-05 11:36:33 -05:00
Soby Chacko
172e345c2d Fixing checkstyle issues 2021-02-05 11:16:45 -05:00
Soby Chacko
b15a798a78 Fix checkstyle issues 2021-02-05 10:45:23 -05:00
Artem Bilan
17162dd775 GH-146: SFTP supplier: Use proper FileListFilter
Fixes https://github.com/spring-cloud/stream-applications/issues/146

The `SftpRegexPatternFileListFilter` is used for the `filenamePattern` option, too.
It seems like a copy/paste artefact

* Change `filenamePattern`-based configuration for the `SftpSimplePatternFileListFilter`
* Fix `SftpSupplierApplicationTests` for multi-OS compatibility
2021-02-03 10:20:10 -05:00
Artem Bilan
70d89c7a3b GH-145: Some SFTP consumer props were not applied
Fixes https://github.com/spring-cloud/stream-applications/issues/145
2021-02-02 20:41:19 -05:00
Christian Tzolov
dd9a2f709d Fix cdc logging classpath issue
- Exclude a slf4j-log4j12 dependency inherited by slf4j-log4j12 that clashes with the spring boot logging.
2021-02-01 10:41:39 +01:00
Andrea Montemaggio
65f3b82d0b Introduce basic support for user-configurable sorting of the remote file list. 2021-01-14 10:57:43 -05:00
Soby Chacko
1aebb2abff Non application components next version update
stream-applications-build: 1.0.1-SNAPSHOT
functions: 1.0.1-SNAPSHOT
stream-applications-core: 3.0.2-SNAPSHOT
2020-12-16 13:06:52 -05:00
Soby Chacko
6c0e3b3167 Functions release: 1.0.0 2020-12-15 15:44:50 -05:00
Soby Chacko
c07aeeae13 remove unnecessary mvnw scripts from rabbit-supplier 2020-12-15 14:09:26 -05:00
Soby Chacko
26eeae18f0 Back to snapshot for non-apps components 2020-12-11 17:17:01 -05:00
Soby Chacko
af63c3fd09 Functions release: 1.0.0-RC2 2020-12-11 15:40:32 -05:00
Christian Tzolov
0d6c21b405 Analytics: prevent metrics duplication if multiple meter registries and the compositemeterregistry are present 2020-12-09 12:52:16 +01:00
Christian Tzolov
3f0bacfe7a Analytics: fix failing tests 2020-12-08 10:07:21 +01:00
Christian Tzolov
48b0f17489 Analytics: handle NPE condition 2020-12-08 09:53:46 +01:00
Christian Tzolov
ace860fb5e analticis: imprive handling for valueless tags 2020-12-07 22:22:16 +01:00
Soby Chacko
9a2ca148c8 Cleaning up repositories/pluginRepositories declarations 2020-12-07 16:04:07 -05:00
Christian Tzolov
18a29fd0fc Improve analytcs meter tas consistency 2020-12-07 13:25:58 +01:00
Christian Tzolov
a9692efc75 Fix CDC issues
- Fix cdc_key header string wrapping issue that causes problems with the spring.cloud.stream.kafka.default.producer.messageKeyExpression.
 - Rename Flattering property to the correct Flattening.
 - Remove wrong dependencies.
2020-12-04 19:50:45 +01:00