Commit Graph

73 Commits

Author SHA1 Message Date
Corneil du Plessis
392b5bfbbe [Issues] Change http-request to non-reactive (#402)
Fixes #206
2022-11-04 17:40:01 +02:00
Chris Bono
d3b339ae01 Update checkstyle plugin to 3.2.0
- Spring Javaformat update to 0.0.35
- Puppycrawl update to 10.3.4
2022-10-24 23:55:49 -05:00
Artem Bilan
735d42048c GH-373: Re-enable aggregator tests (#377)
Fixes https://github.com/spring-cloud/stream-applications/issues/373

The fix in Spring Integration has made it into the latest SNAPSHOT.
See more info in the linked issue

* Update to the latest Testcontainers: it does not require credential for pull command any more
* Remove obsolete `MessageStoreType.GEMFIRE`
* Add `<doclint>syntax</doclint>` to ignore missed javadocs on public methods
2022-10-19 14:09:32 -05:00
Artem Bilan
f17a0b3ec0 GH-369: Fix HeaderEnricher for no proxyBeanMethod (#376)
Spring Boot `@AutoConfiguration` comes now with a `proxyBeanMethods = false`,
so we cannot call bean methods within the same configuration class.

* Rework `HeaderEnricherFunctionConfiguration` for bean methods autowiring
* Create an `ExpressionEvaluatingHeaderValueMessageProcessor` instances directly
in the `headerEnricher` bean definition and propagate an injected `BeanFactory`
* Re-enable disabled tests

Fixes #369
2022-10-19 12:18:50 -05:00
Corneil du Plessis
8a7dca70ed Ensure module projects skip deploy 2022-10-18 15:18:54 +02:00
Chris Bono
f4d396ab40 [SB3 Update] Adjust jsonpath on TwitterUsersFunctionTests 2022-10-16 23:52:42 -05:00
Chris Bono
1a941bf649 [SB3 Update] Add HttpMethod converter to http-request function
- HttpMethod moved from enum to class in Spring Framework. Needs converter now.
2022-10-16 23:52:42 -05:00
Chris Bono
d351163cd2 [SB3 Update] Disable HeaderEnricherFunctionApplicationTests 2022-10-16 23:52:42 -05:00
Chris Bono
e6a2824f61 [SB3 Update] Agg function use MongoDB testcontainers
- Disable tests
2022-10-16 23:52:42 -05:00
Corneil du Plessis
ec7b11e707 [SB3 Update] Remove Apache Geode (#345)
- Also update SFTP usage
2022-10-07 08:22:57 -05:00
Corneil du Plessis
f6a3ab653d Add intermediate module poms (#337) 2022-10-03 13:51:55 -05:00
Chris Bono
6f9776f94e Convert to Autoconfiguration.imports 2022-10-02 10:58:48 -05:00
Chris Bono
002204270e Remove ${revision} from pom.xml files 2022-10-02 10:58:48 -05:00
Chris Bono
a8814f876f Initial support for SB3/SF6
- Main branch builds successfully (with tests skipped)

See #326
2022-09-22 17:46:11 -05:00
Artem Bilan
5e78da73ef Disable DB init for JDBC Source and Sink (#325)
* Disable DB init for JDBC Source and Sink

The `IntegrationAutoConfiguration` provides mechanism
to initialize DB with some out-of-the-box scripts.
The default behavior is `EMBEDDED` even if we don't use
any components for those DB objects, e.g. no `JdbcMessageStore` bean.

* Add `spring.integration.jdbc.initialize-schema=NEVER` to the
`jdbc-consumer` and `jdbc-supplier` to not attempt to initialize DB
* Fix Checkstyle violations in other modules
2022-09-21 10:31:18 -05:00
Artem Bilan
61f1412365 GH-276 Apply ComponentCustomizer in other modules (#284)
* GH-276 Apply ComponentCustomizer in other modules

Fixes https://github.com/spring-cloud/stream-applications/issues/276

Some changes are straightforward as just a `ComponentCustomizer<?>`
injection and its optional usage for the target component customization.
Other changes are more drastic since it is better to use `proxyBeanMethods = false`,
so some direct bean method calls had to be reworked to the bean method argument
injections

* Fix some deprecations from Project Reactor
* Remove `UpdatingMongoDbMessageSource` since `MongoDbMessageSource`
supports now an `update` option in Spring Integration

NOTE: The `ZeroMqSupplierConfiguration` has its own customizer already
like `Consumer<ZMQ.Socket>`, so it can be revised to a more broader solution,
but in the future version.

Some modules just does not make sense to modify for more advance customization
since all their configuration is covered with plain configuration properties.
See `splitter-function` or `wavefront-consumer`

* * Use `ObjectProvider<WebClientCustomizer>` for `HttpRequestFunctionConfiguration`
instead of our own `ComponentCustomizer`.
Although this has to be revised in favor of fully auto-configured `WebClient.Builder`.
Even that `maximumBufferSize` is included into codecs configuration properties.
* The `MongoDbMessageSource` has now an `update` option in Spring Integration
therefore remove redundant already `UpdatingMongoDbMessageSource`
and populate `this.properties.getUpdateExpression()` directly ot the
`MongoDbMessageSource` bean
* Modify supplier configurations to avoid manual starts for endpoints on
reactive `Publisher` subscriptions.
Spring Integration provides now a `toReactivePublisher(true)` operator
to have all the endpoints stopped on start, but started automatically
when subscription happens to the provided `Publisher`
* Update Copyright to the current year for all the affected classes
* Use `@Nullable` instead of `ObjectProvider` in the `AggregatorFunctionConfiguration`
dependency injection for consistency
* Fix `LogConsumerConfiguration` to not have a `log()` in the end - it is not terminal
in Spring Integration anymore.
Use recommended `nullChannel()` terminating operator for use-cases like this.

* * Remove `beanName` from `ComponentCustomizer` contract

Co-authored-by: Chris Bono <cbono@vmware.com>
2022-07-15 19:36:21 -05:00
sunny151091
7b3309707f Revise HttpRequestFunctionConfiguration configuration (#295)
Fixes #286
2022-07-15 19:01:30 -05:00
Chris Bono
1c50531c13 GH-271: Remove duplication from pom.xml files
This proposal fixes many build warnings and is a general cleanup of the pom.xml files.

* Remove duplicate `dependency` entries  from pom.xml files
* Remove duplicate `repository` entries from pom.xml files
* Replace deprecated use of `receive(long)` in `HttpSourceTests`
* Reduce logging for Twitter integration tests

Fixes #271
2022-05-26 13:57:44 -04:00
Chris Bono
32e9b27ca0 GH-272: Update to Spring Boot 2.6.8 (#278)
Fixes #272
2022-05-24 21:47:52 -05:00
Artem Bilan
7ed748687e GH-195: Introduce ComponentCustomizer (#273)
Fixes #195 

* Introduce a `ComponentCustomizer` contract to be exposed in
the end-user configuration for particular bean type
* introduce a `@CustomizationAware` annotation to mark a component 
as customizable
* Introduce a `ComponentCustomizationBeanPostProcessor` as an auto-configured
bean to apply `ComponentCustomizer` to a matching `@CustomizationAware` (if any)
* Use `@CustomizationAware` in the `FileSupplierConfiguration`
and demonstrate `ComponentCustomizer` feature in its tests
* Move `config-common` to functions parent to make the mentioned auto-configuration
available for all the functions
* Remove some other duplicated dependencies from the affected poms
2022-05-24 20:55:11 -05:00
Chris Bono
b6895e626c Polish "GH-248: Exclude Geode SSL config"
* Add comments in tests about geode ssl flag verification
2022-04-06 00:06:58 -05:00
Artem Bilan
5d83512da4 GH-248: Exclude Geode SSL config
Fixes https://github.com/spring-cloud/stream-applications/issues/248

The Geode SSL auto-config is done via `SslAutoConfiguration.SslEnvironmentPostProcessor`
which cannot be excluded as an auto-config.

* Set `SslAutoConfiguration.SECURITY_SSL_ENVIRONMENT_POST_PROCESSOR_ENABLED_PROPERTY`
to `false` from the `ExcludeStoresAutoConfigurationEnvironmentPostProcessor`
when the store type is not Geode
2022-04-06 00:01:35 -05:00
Chris Bono
041ffc5e12 Upgrade Geode to support Spring Boot 2.6.3
* Fix Geode integration tests
* Upgrade spring-geode-starter -> 1.6.3
* Upgrade apachegeode/geode -> 1.14.3
* Upgrade spring-data-geode-test -> 0.0.28
2022-01-28 12:13:01 -05:00
Artem Bilan
fc6e93e820 * Use spring.mongodb.embedded.version=3.5.5 - aligned with Spring Boot 2022-01-25 09:37:14 -05:00
Artem Bilan
e5222501c8 Restore embedded MongoDb tests 2022-01-24 11:30:33 -05:00
Soby Chacko
f8d8f1d6df Temporarily disabling a few tests 2022-01-21 17:52:10 -05:00
Artem Bilan
e815e3084c Change doOnSubscribe to doOnRequest
The `aggregatorFunction` has a flaw subscribing and emitting
data from a source `Flux` too early: from a `doOnSubscribe()`
callback which is really called *before* the real subscription is
registered in the target `Publisher`

Change the logic to the `doOnRequest()` which happens already
after subscription is registered in the `Publisher`

**Cherry-pick to `2020.0.x`**
2022-01-20 18:34:21 -05:00
David Turanski
1330de11c6 Remove tasklauncher-function and tasklauncher-sink 2021-08-17 10:20:53 -04:00
Artem Bilan
bbf04eff2e Limit aggregator tests for 10 seconds to wait 2021-04-07 12:02:02 -04:00
Soby Chacko
760683cc2a Restructuing with maven revision feature
Preserving the maven revision feature from the previous commit,
but perform a bit more restructuring the hierarchy.
2020-10-13 15:17:52 -04:00
Daniel Frey
5e9382c23a Refactor hardcoded versions in functions submodule branches
This is so they can be updated in a minimal amount of places and leverage the maven hierarchy to propagate them where appropriate.
While working with SNAPSHOT releases of Spring Boot, Spring Integration, and Spring Cloud, it required touching every pom.xml file
in order to reflect the use of the SNAPSHOT. After this change, it will only be required to change the property in the top
functions pom.xml and the versions will reflect to all submodules.

Maven introduced the ability use a key property, revision to propagate this information to the <parent> declarations in each pom.

This was enabled in Maven 3.5+
https://maven.apache.org/maven-ci-friendly.html
2020-10-13 15:15:59 -04:00
David Turanski
f286ced1a0 Upgrade apps to 3.0.0-SNAPSHOT 2020-10-11 10:43:16 -04:00
Soby Chacko
f257f48f78 Next version updates
Functions: 1.0.0-SNAPSHOT
  Core Apps: 3.0.0-SNAPSHOT
  Apps: 3.0.0-SNAPSHOT
  Aggregate Next:2020.0.0-SNAPSHOT
2020-09-29 18:28:20 -04:00
Soby Chacko
73a3e55299 Functions: Release - 1.0.0-M3 2020-09-29 14:59:33 -04:00
David Turanski
f7efc39164 Remove SplitterFunction autoconfiguration for sources 2020-09-14 14:00:52 -04:00
Christian Tzolov
b9f6d9e9a9 Fix another filter function checkstyle issue 2020-08-31 08:47:15 +02:00
Christian Tzolov
d899d0856b Fix a [ImportOrder] checkstyle issue 2020-08-31 08:05:36 +02:00
Artem Bilan
a91aa2cd3f GH-110: Fix filter function for emitting null (#111)
* GH-110: Fix filter function for emitting `null`

Fixes https://github.com/spring-cloud/stream-applications/issues/110

Turns out when functions are composed in the Spring Cloud Stream environment,
they are called via reactive wrappers which don't allow to emit `null` from the
`map()` operator.

* Make `filterFunction` fully reactive to reply on the `Flux.filter()` operator
* Add `config-common` for auto-conversion string configuration options into `Expression` instances
* Add `spring-boot-starter-json` dependency since it is required by the `SpelExpressionConverterConfiguration`
* Add `reactor-test` dependency to test the final solution
* Remove redundant dependencies from the `filter-processor`

* * Add `proxyBeanMethods = false` into `FilterFunctionConfiguration`
* Fix default expression to `true` instead of `payload`, which does not fit to filter logic
* Fix JavaDoc for `expression` property
* Remove redundant `application.properties` from the `filter-function`
2020-08-28 15:55:08 -04:00
Soby Chacko
d962e6e233 Fixing missing filter processor properties in README 2020-08-25 14:07:06 -04:00
Artem Bilan
b27897227d Add Aggregator function & processor
* Have all the required dependencies in class path which give us a `MessageGroupStore` implementation
* Disable all their auto-configurations in the `ExcludeStoresAutoConfigurationEnvironmentPostProcessor`
* Enable only those which are configured by the `aggregator.messageStoreType` property
* Implement a fully reactive function around an aggregator making an interaction via a pair of `FluxMessageChannel`
* Fix checkstyle issues
* Change the app generator maven plugin to new coordinates
2020-08-20 15:54:51 -04:00
David Turanski
d13a1fceaf Add max-buffer-size and fix app deps 2020-08-13 13:27:15 -04:00
David Turanski
eee707da00 Refactor to use SpEL in function properties
* Clean up per review
2020-08-12 15:39:56 -04:00
Soby Chacko
3d01487565 Remove duplicate dependency in twitter-function
Add relativePath to core parent
2020-08-07 16:50:15 -04:00
Christian Tzolov
2ffbe371b6 Remove double quotes in the properties comments 2020-08-05 15:18:25 +02:00
David Turanski
f8c342077c Support filterFunction and spelFunction composition 2020-07-28 12:26:26 -04:00
David Turanski
562aa5c990 Fix function definition property (#91)
* Remove deprecated spring.cloud.stream.function.definition property

* Restore tasklauncher-function

Co-authored-by: David Turanski <dturanski@pivotal.io>
2020-07-20 18:23:30 -04:00
Soby Chacko
d59e4f5d86 Next version updates
Functions: 1.0.0-SNAPSHOT
  Core Apps: 3.0.0-SNAPSHOT
  Apps: 3.0.0-SNAPSHOT
  Aggregate Next:2020.0.0-SNAPSHOT
2020-07-13 16:02:05 -04:00
Soby Chacko
7066283aa2 Functions: Release - 1.0.0-M2 2020-07-13 14:43:36 -04:00
Soby Chacko
49468e6148 Renaming internal test boot applications in functions 2020-07-02 18:31:58 -04:00
Soby Chacko
5c59ff6823 Next version updates
Functions: 1.0.0-SNAPSHOT
  Core Apps: 3.0.0-SNAPSHOT
  Apps: 3.0.0-SNAPSHOT
  Aggregate Build: 2020.0.0-SNAPSHOT
2020-06-26 13:49:35 -04:00