* Disable DB init for JDBC Source and Sink
The `IntegrationAutoConfiguration` provides mechanism
to initialize DB with some out-of-the-box scripts.
The default behavior is `EMBEDDED` even if we don't use
any components for those DB objects, e.g. no `JdbcMessageStore` bean.
* Add `spring.integration.jdbc.initialize-schema=NEVER` to the
`jdbc-consumer` and `jdbc-supplier` to not attempt to initialize DB
* Fix Checkstyle violations in other modules
* GH-276 Apply ComponentCustomizer in other modules
Fixes https://github.com/spring-cloud/stream-applications/issues/276
Some changes are straightforward as just a `ComponentCustomizer<?>`
injection and its optional usage for the target component customization.
Other changes are more drastic since it is better to use `proxyBeanMethods = false`,
so some direct bean method calls had to be reworked to the bean method argument
injections
* Fix some deprecations from Project Reactor
* Remove `UpdatingMongoDbMessageSource` since `MongoDbMessageSource`
supports now an `update` option in Spring Integration
NOTE: The `ZeroMqSupplierConfiguration` has its own customizer already
like `Consumer<ZMQ.Socket>`, so it can be revised to a more broader solution,
but in the future version.
Some modules just does not make sense to modify for more advance customization
since all their configuration is covered with plain configuration properties.
See `splitter-function` or `wavefront-consumer`
* * Use `ObjectProvider<WebClientCustomizer>` for `HttpRequestFunctionConfiguration`
instead of our own `ComponentCustomizer`.
Although this has to be revised in favor of fully auto-configured `WebClient.Builder`.
Even that `maximumBufferSize` is included into codecs configuration properties.
* The `MongoDbMessageSource` has now an `update` option in Spring Integration
therefore remove redundant already `UpdatingMongoDbMessageSource`
and populate `this.properties.getUpdateExpression()` directly ot the
`MongoDbMessageSource` bean
* Modify supplier configurations to avoid manual starts for endpoints on
reactive `Publisher` subscriptions.
Spring Integration provides now a `toReactivePublisher(true)` operator
to have all the endpoints stopped on start, but started automatically
when subscription happens to the provided `Publisher`
* Update Copyright to the current year for all the affected classes
* Use `@Nullable` instead of `ObjectProvider` in the `AggregatorFunctionConfiguration`
dependency injection for consistency
* Fix `LogConsumerConfiguration` to not have a `log()` in the end - it is not terminal
in Spring Integration anymore.
Use recommended `nullChannel()` terminating operator for use-cases like this.
* * Remove `beanName` from `ComponentCustomizer` contract
Co-authored-by: Chris Bono <cbono@vmware.com>
This proposal fixes many build warnings and is a general cleanup of the pom.xml files.
* Remove duplicate `dependency` entries from pom.xml files
* Remove duplicate `repository` entries from pom.xml files
* Replace deprecated use of `receive(long)` in `HttpSourceTests`
* Reduce logging for Twitter integration tests
Fixes#271
Since we still need to modify every single function, consumer & supplier
for previously introduced `@CustomizationAware`, it does not do too much
difference for us if we add a `ComponentCustomizer` dependency to the
bean for customization and call it manually.
One benefit of this change is to avoid a `static` confusion and performance
overhead with extra `BeanPostProcessor`.
* Remove `ComponentCustomizerBeanPostProcessor` and
respective `ComponentCustomizationAutoConfiguration`
* Use `@Nullable ComponentCustomizer<FileInboundChannelAdapterSpec>`
as dependency in the `FileSupplierConfiguration.fileMessageSource()` bean
Fixes#195
* Introduce a `ComponentCustomizer` contract to be exposed in
the end-user configuration for particular bean type
* introduce a `@CustomizationAware` annotation to mark a component
as customizable
* Introduce a `ComponentCustomizationBeanPostProcessor` as an auto-configured
bean to apply `ComponentCustomizer` to a matching `@CustomizationAware` (if any)
* Use `@CustomizationAware` in the `FileSupplierConfiguration`
and demonstrate `ComponentCustomizer` feature in its tests
* Move `config-common` to functions parent to make the mentioned auto-configuration
available for all the functions
* Remove some other duplicated dependencies from the affected poms
Fixes https://github.com/spring-cloud/stream-applications/issues/248
The Geode SSL auto-config is done via `SslAutoConfiguration.SslEnvironmentPostProcessor`
which cannot be excluded as an auto-config.
* Set `SslAutoConfiguration.SECURITY_SSL_ENVIRONMENT_POST_PROCESSOR_ENABLED_PROPERTY`
to `false` from the `ExcludeStoresAutoConfigurationEnvironmentPostProcessor`
when the store type is not Geode
Fixes https://github.com/spring-cloud/stream-applications/issues/249
Currently, the metadata store is supported by remote files source (S3, ftp and sftp), but not by file source.
There are cases where it is convenient to be able to use the metadata store with file source as well.
* Add externally configured `ConcurrentMetadataStore` into `FileSupplierConfiguration`
which is based on the auto-configuration provided by the `metadata-store-common` artifact
* Test with an embedded JDBC store
* Remove redundant dependencies in the file-supplier pom - supplied by the parent
* Add `MetadataStoreProperties` to allow-list for dataflow
* Fix typo in the `s3-supplier` README
The `Flux.doOnSubscribe()` happens really before the actual subscription is set to the `Publisher`.
So, there is possible a race condition when we start producing messages, but `FluxMessageChannel`
didn't get the actual subscriber to process.
**Cherry-pick to `2021.1.x` & `2020.0.x`**
* Make constructor public
* Appease checkstyle/format gods
* Bump to testcontainers 1.16.3
* Bump to redis testcontainers 2.0.11
* Fix circular bean dependencies in the configuration classes
The `aggregatorFunction` has a flaw subscribing and emitting
data from a source `Flux` too early: from a `doOnSubscribe()`
callback which is really called *before* the real subscription is
registered in the target `Publisher`
Change the logic to the `doOnRequest()` which happens already
after subscription is registered in the `Publisher`
**Cherry-pick to `2020.0.x`**
Fixes https://github.com/spring-cloud/stream-applications/issues/207
When no HTTP `Content-Type` header, the NPE is thrown from the `enrichHeaders()`
endpoint
* Fix NPE extracting header value via `get(MessageHeaders.CONTENT_TYPE, MediaType.class)`
* Make `httpHeaderMapper` bean as a `@ConditionalOnMissingBean` to let end-users
to provide their own custom `HeaderMapper<HttpHeaders>`
* Move `httpSupplierProperties.getMappedRequestHeaders()` setting to the `httpHeaderMapper` bean
* Use `httpHeaderMapper` bean injection into the `WebFlux.inboundChannelAdapter()` instead of
just `httpSupplierProperties.getMappedRequestHeaders()`
* Mention custom `HeaderMapper<HttpHeaders>` in the README
SO: https://stackoverflow.com/questions/70242824/spring-cloud-data-flow-http-source-kafka-application-request-body-memory-size-is
The `WebFlux.inboundChannelAdapter()` creates internally `ServerCodecConfigurer.create()` if not provided.
And with that any customizations for codes are not possible from end-user perspective
* Fix `HttpSupplierConfiguration` to autowire `ServerCodecConfigurer`
which is auto-configured by Spring Boot.
This way end-user can use external configuration properties to customize his/her requirements,
e.g. `spring.codec.max-in-memory-size`
* Cover the feature with unit test to make sure that `spring.codec.max-in-memory-size=10MB`
is propagated properly to the `WebFluxInboundEndpoint.codecConfigurer`
Fixes https://github.com/spring-cloud/stream-applications/issues/182
* Upgrade Debezium dependency to 1.7.1
* Remove explicit dependencies for DBs and remove excludes for them from Debezium deps;
rely fully on whatever Debezium connectors bring for us
* Fix `EmbeddedEngine` for compatibility with the Debezium 1.7.1
* Fix CDC tests for the current state of Debezium results
* Fix Checkstyle violations in the MQTT modules
Fixes https://github.com/spring-cloud/stream-applications/issues/183
* Add `MqttProperties.sslProperties` common MQTT configuration property as a `Map` structure
* Delegate a content fo the `MqttProperties.sslProperties` into `mqttConnectOptions.setSSLProperties()`
in the common `MqttConfiguration`
* Verify that SSL properties are applied in both `MqttSupplierTests` & `MqttConsumerTests`
* Document a new `MqttProperties.sslProperties` configuration property for both consumer and supplier
* Use `eclipse-mosquitto` Docker image for MQTT tests instead of RabbitMQ plugin: the Mosquitto starts much faster
There is a partition issue that was addressed in SCFn and SCSt and needs to be
picked by the apps.
Update both SCFn and SCSt to 3.1.5-SNAPSHOT.
See relevant commits below in SCFn and SCSt for more details.
0cb5483d0ac2326740ed
