7 Commits

Author SHA1 Message Date
David Turanski
6645b390e8 Avoid timeout when attemting to connect to AWS 2020-10-11 11:30:04 -04:00
Artem Bilan
696ebca135 Defer subscription to the S3 until downstream one
It turns out that endpoint for S3 Source is starting to
consume the `Flux` earlier than an actual subscription
happens for the whole flow downstream

* Introduce a `MonoProcessor<Subscription> downstreamSubscription`
to fulfill from the downstream subscription and handle from the
`delaySubscription()` on the `Flux` for source data
2020-09-25 13:28:18 -04:00
David Turanski
ddcc10363e List only enhancements
* Add metadata.store.type Property and idempotent SftpSupplier for list-only

* Implement list-only for S3 source and Optimize metadastore access

* Fixed build and READMEs

* Change to ConditionalOnProperty

* Change to ReactiveMessageProducer

* Update cdc-debezium-source/README.adoc

* Make all MetadataStoreProperties visible
2020-09-17 15:36:36 -04:00
David Turanski
a68c7c96f6 Add new visible properties metadata file 2020-08-10 18:15:07 -04:00
Timo Salm
c21533e42c Support for s3 compatible storage
* Add support for s3 compatible storage to s3 consumer and supplier.

* Add enhancements to s3 compatible storage implementation based on code review

* Move s3 compatible storage endpoint configuration bean to AmazonS3Configuration to remove code duplication

* Disable auto configuration of AmazonS3 Bean to enable custom configurations

* Add documentation for endpoint url configuration property

* Use localhost instead of a real URL to avoid an unexpected connection to the internet
2020-07-07 14:27:31 -04:00
Soby Chacko
6e06aa3258 Checkstyle cleanup in s3 source tests 2020-06-19 18:05:27 -04:00
Soby Chacko
cffbe2fa56 FTP supplier/source
AWS S3 apps migration

s3-supplier/s3-source
s3-consumer/s3-sink

Fix merge conflicts

Addressing PR review comments

Rebasing

Add ConditionalOnMissingClass to Spel converter config

Test cleanup

* Fix `AwsS3SourceTests` to properly mock before starting endpoint
2020-06-18 17:06:58 -04:00