Commit Graph

75 Commits

Author SHA1 Message Date
Corneil du Plessis
cab24caef8 Merge remote-tracking branch 'origin/main'
# Conflicts:
#	.github/workflows/ci-pr.yml
2022-11-07 15:56:35 +02:00
Corneil du Plessis
7469320c91 [GitHub Actions] Improve dependencies for main. Provide for building impacted dependencies for a PR. 2022-11-07 15:56:07 +02:00
Corneil du Plessis
97fdd0ccc5 [Issues] Change http-request to non-reactive (#402)
Fixes #206
2022-11-04 17:40:01 +02:00
Chris Bono
afd5f82ec4 Update checkstyle plugin to 3.2.0
- Spring Javaformat update to 0.0.35
- Puppycrawl update to 10.3.4
2022-10-24 23:55:49 -05:00
Artem Bilan
6cdc80c204 GH-373: Re-enable aggregator tests (#377)
Fixes https://github.com/spring-cloud/stream-applications/issues/373

The fix in Spring Integration has made it into the latest SNAPSHOT.
See more info in the linked issue

* Update to the latest Testcontainers: it does not require credential for pull command any more
* Remove obsolete `MessageStoreType.GEMFIRE`
* Add `<doclint>syntax</doclint>` to ignore missed javadocs on public methods
2022-10-19 14:09:32 -05:00
Artem Bilan
0f7488f84c GH-369: Fix HeaderEnricher for no proxyBeanMethod (#376)
Spring Boot `@AutoConfiguration` comes now with a `proxyBeanMethods = false`,
so we cannot call bean methods within the same configuration class.

* Rework `HeaderEnricherFunctionConfiguration` for bean methods autowiring
* Create an `ExpressionEvaluatingHeaderValueMessageProcessor` instances directly
in the `headerEnricher` bean definition and propagate an injected `BeanFactory`
* Re-enable disabled tests

Fixes #369
2022-10-19 12:18:50 -05:00
Corneil du Plessis
028df89c29 Ensure module projects skip deploy 2022-10-18 15:18:54 +02:00
Chris Bono
ab83dcf743 [SB3 Update] Adjust jsonpath on TwitterUsersFunctionTests 2022-10-16 23:52:42 -05:00
Chris Bono
e5843418e8 [SB3 Update] Add HttpMethod converter to http-request function
- HttpMethod moved from enum to class in Spring Framework. Needs converter now.
2022-10-16 23:52:42 -05:00
Chris Bono
d002bbf7c4 [SB3 Update] Disable HeaderEnricherFunctionApplicationTests 2022-10-16 23:52:42 -05:00
Chris Bono
ea800682b2 [SB3 Update] Agg function use MongoDB testcontainers
- Disable tests
2022-10-16 23:52:42 -05:00
Corneil du Plessis
c64b259d91 [SB3 Update] Remove Apache Geode (#345)
- Also update SFTP usage
2022-10-07 08:22:57 -05:00
Corneil du Plessis
68f2900aad Add intermediate module poms (#337) 2022-10-03 13:51:55 -05:00
Chris Bono
790daf0d91 Convert to Autoconfiguration.imports 2022-10-02 10:58:48 -05:00
Chris Bono
87df01dc00 Remove ${revision} from pom.xml files 2022-10-02 10:58:48 -05:00
Chris Bono
85106dd122 Initial support for SB3/SF6
- Main branch builds successfully (with tests skipped)

See #326
2022-09-22 17:46:11 -05:00
Artem Bilan
2c5d83f926 Disable DB init for JDBC Source and Sink (#325)
* Disable DB init for JDBC Source and Sink

The `IntegrationAutoConfiguration` provides mechanism
to initialize DB with some out-of-the-box scripts.
The default behavior is `EMBEDDED` even if we don't use
any components for those DB objects, e.g. no `JdbcMessageStore` bean.

* Add `spring.integration.jdbc.initialize-schema=NEVER` to the
`jdbc-consumer` and `jdbc-supplier` to not attempt to initialize DB
* Fix Checkstyle violations in other modules
2022-09-21 10:31:18 -05:00
Artem Bilan
910f116332 GH-276 Apply ComponentCustomizer in other modules (#284)
* GH-276 Apply ComponentCustomizer in other modules

Fixes https://github.com/spring-cloud/stream-applications/issues/276

Some changes are straightforward as just a `ComponentCustomizer<?>`
injection and its optional usage for the target component customization.
Other changes are more drastic since it is better to use `proxyBeanMethods = false`,
so some direct bean method calls had to be reworked to the bean method argument
injections

* Fix some deprecations from Project Reactor
* Remove `UpdatingMongoDbMessageSource` since `MongoDbMessageSource`
supports now an `update` option in Spring Integration

NOTE: The `ZeroMqSupplierConfiguration` has its own customizer already
like `Consumer<ZMQ.Socket>`, so it can be revised to a more broader solution,
but in the future version.

Some modules just does not make sense to modify for more advance customization
since all their configuration is covered with plain configuration properties.
See `splitter-function` or `wavefront-consumer`

* * Use `ObjectProvider<WebClientCustomizer>` for `HttpRequestFunctionConfiguration`
instead of our own `ComponentCustomizer`.
Although this has to be revised in favor of fully auto-configured `WebClient.Builder`.
Even that `maximumBufferSize` is included into codecs configuration properties.
* The `MongoDbMessageSource` has now an `update` option in Spring Integration
therefore remove redundant already `UpdatingMongoDbMessageSource`
and populate `this.properties.getUpdateExpression()` directly ot the
`MongoDbMessageSource` bean
* Modify supplier configurations to avoid manual starts for endpoints on
reactive `Publisher` subscriptions.
Spring Integration provides now a `toReactivePublisher(true)` operator
to have all the endpoints stopped on start, but started automatically
when subscription happens to the provided `Publisher`
* Update Copyright to the current year for all the affected classes
* Use `@Nullable` instead of `ObjectProvider` in the `AggregatorFunctionConfiguration`
dependency injection for consistency
* Fix `LogConsumerConfiguration` to not have a `log()` in the end - it is not terminal
in Spring Integration anymore.
Use recommended `nullChannel()` terminating operator for use-cases like this.

* * Remove `beanName` from `ComponentCustomizer` contract

Co-authored-by: Chris Bono <cbono@vmware.com>
2022-07-15 19:36:21 -05:00
sunny151091
5b5fb8fd0f Revise HttpRequestFunctionConfiguration configuration (#295)
Fixes #286
2022-07-15 19:01:30 -05:00
Chris Bono
50d75ef1e7 GH-271: Remove duplication from pom.xml files
This proposal fixes many build warnings and is a general cleanup of the pom.xml files.

* Remove duplicate `dependency` entries  from pom.xml files
* Remove duplicate `repository` entries from pom.xml files
* Replace deprecated use of `receive(long)` in `HttpSourceTests`
* Reduce logging for Twitter integration tests

Fixes #271
2022-05-26 13:57:44 -04:00
Chris Bono
1d153ae728 GH-272: Update to Spring Boot 2.6.8 (#278)
Fixes #272
2022-05-24 21:47:52 -05:00
Artem Bilan
89cd8812c3 GH-195: Introduce ComponentCustomizer (#273)
Fixes #195 

* Introduce a `ComponentCustomizer` contract to be exposed in
the end-user configuration for particular bean type
* introduce a `@CustomizationAware` annotation to mark a component 
as customizable
* Introduce a `ComponentCustomizationBeanPostProcessor` as an auto-configured
bean to apply `ComponentCustomizer` to a matching `@CustomizationAware` (if any)
* Use `@CustomizationAware` in the `FileSupplierConfiguration`
and demonstrate `ComponentCustomizer` feature in its tests
* Move `config-common` to functions parent to make the mentioned auto-configuration
available for all the functions
* Remove some other duplicated dependencies from the affected poms
2022-05-24 20:55:11 -05:00
Chris Bono
77a63f39fd Polish "GH-248: Exclude Geode SSL config"
* Add comments in tests about geode ssl flag verification
2022-04-06 00:06:58 -05:00
Artem Bilan
10ac6a45cb GH-248: Exclude Geode SSL config
Fixes https://github.com/spring-cloud/stream-applications/issues/248

The Geode SSL auto-config is done via `SslAutoConfiguration.SslEnvironmentPostProcessor`
which cannot be excluded as an auto-config.

* Set `SslAutoConfiguration.SECURITY_SSL_ENVIRONMENT_POST_PROCESSOR_ENABLED_PROPERTY`
to `false` from the `ExcludeStoresAutoConfigurationEnvironmentPostProcessor`
when the store type is not Geode
2022-04-06 00:01:35 -05:00
Chris Bono
3af4e8cfe0 Upgrade Geode to support Spring Boot 2.6.3
* Fix Geode integration tests
* Upgrade spring-geode-starter -> 1.6.3
* Upgrade apachegeode/geode -> 1.14.3
* Upgrade spring-data-geode-test -> 0.0.28
2022-01-28 12:13:01 -05:00
Artem Bilan
361c0866b9 * Use spring.mongodb.embedded.version=3.5.5 - aligned with Spring Boot 2022-01-25 09:37:14 -05:00
Artem Bilan
de92924f66 Restore embedded MongoDb tests 2022-01-24 11:30:33 -05:00
Soby Chacko
db61080499 Temporarily disabling a few tests 2022-01-21 17:52:10 -05:00
Artem Bilan
e1987527cf Change doOnSubscribe to doOnRequest
The `aggregatorFunction` has a flaw subscribing and emitting
data from a source `Flux` too early: from a `doOnSubscribe()`
callback which is really called *before* the real subscription is
registered in the target `Publisher`

Change the logic to the `doOnRequest()` which happens already
after subscription is registered in the `Publisher`

**Cherry-pick to `2020.0.x`**
2022-01-20 18:34:21 -05:00
David Turanski
caee3ffa48 Remove tasklauncher-function and tasklauncher-sink 2021-08-17 10:20:53 -04:00
Artem Bilan
daebf0f786 Limit aggregator tests for 10 seconds to wait 2021-04-07 12:02:02 -04:00
Soby Chacko
d0a0df5a44 Restructuing with maven revision feature
Preserving the maven revision feature from the previous commit,
but perform a bit more restructuring the hierarchy.
2020-10-13 15:17:52 -04:00
Daniel Frey
724c4ace5f Refactor hardcoded versions in functions submodule branches
This is so they can be updated in a minimal amount of places and leverage the maven hierarchy to propagate them where appropriate.
While working with SNAPSHOT releases of Spring Boot, Spring Integration, and Spring Cloud, it required touching every pom.xml file
in order to reflect the use of the SNAPSHOT. After this change, it will only be required to change the property in the top
functions pom.xml and the versions will reflect to all submodules.

Maven introduced the ability use a key property, revision to propagate this information to the <parent> declarations in each pom.

This was enabled in Maven 3.5+
https://maven.apache.org/maven-ci-friendly.html
2020-10-13 15:15:59 -04:00
David Turanski
80f9545ae9 Upgrade apps to 3.0.0-SNAPSHOT 2020-10-11 10:43:16 -04:00
Soby Chacko
86f0ca0c07 Next version updates
Functions: 1.0.0-SNAPSHOT
  Core Apps: 3.0.0-SNAPSHOT
  Apps: 3.0.0-SNAPSHOT
  Aggregate Next:2020.0.0-SNAPSHOT
2020-09-29 18:28:20 -04:00
Soby Chacko
e1d29f8d24 Functions: Release - 1.0.0-M3 2020-09-29 14:59:33 -04:00
David Turanski
13fe23ccc0 Remove SplitterFunction autoconfiguration for sources 2020-09-14 14:00:52 -04:00
Christian Tzolov
fcbf941457 Fix another filter function checkstyle issue 2020-08-31 08:47:15 +02:00
Christian Tzolov
35608d28cb Fix a [ImportOrder] checkstyle issue 2020-08-31 08:05:36 +02:00
Artem Bilan
0144476a25 GH-110: Fix filter function for emitting null (#111)
* GH-110: Fix filter function for emitting `null`

Fixes https://github.com/spring-cloud/stream-applications/issues/110

Turns out when functions are composed in the Spring Cloud Stream environment,
they are called via reactive wrappers which don't allow to emit `null` from the
`map()` operator.

* Make `filterFunction` fully reactive to reply on the `Flux.filter()` operator
* Add `config-common` for auto-conversion string configuration options into `Expression` instances
* Add `spring-boot-starter-json` dependency since it is required by the `SpelExpressionConverterConfiguration`
* Add `reactor-test` dependency to test the final solution
* Remove redundant dependencies from the `filter-processor`

* * Add `proxyBeanMethods = false` into `FilterFunctionConfiguration`
* Fix default expression to `true` instead of `payload`, which does not fit to filter logic
* Fix JavaDoc for `expression` property
* Remove redundant `application.properties` from the `filter-function`
2020-08-28 15:55:08 -04:00
Soby Chacko
800b177852 Fixing missing filter processor properties in README 2020-08-25 14:07:06 -04:00
Artem Bilan
4463c1afb6 Add Aggregator function & processor
* Have all the required dependencies in class path which give us a `MessageGroupStore` implementation
* Disable all their auto-configurations in the `ExcludeStoresAutoConfigurationEnvironmentPostProcessor`
* Enable only those which are configured by the `aggregator.messageStoreType` property
* Implement a fully reactive function around an aggregator making an interaction via a pair of `FluxMessageChannel`
* Fix checkstyle issues
* Change the app generator maven plugin to new coordinates
2020-08-20 15:54:51 -04:00
David Turanski
6b0f08712b Add max-buffer-size and fix app deps 2020-08-13 13:27:15 -04:00
David Turanski
93d87a4155 Refactor to use SpEL in function properties
* Clean up per review
2020-08-12 15:39:56 -04:00
Soby Chacko
0c07b4af05 Remove duplicate dependency in twitter-function
Add relativePath to core parent
2020-08-07 16:50:15 -04:00
Christian Tzolov
f8ed9f9c38 Remove double quotes in the properties comments 2020-08-05 15:18:25 +02:00
David Turanski
be818e02de Support filterFunction and spelFunction composition 2020-07-28 12:26:26 -04:00
David Turanski
30caf06198 Fix function definition property (#91)
* Remove deprecated spring.cloud.stream.function.definition property

* Restore tasklauncher-function

Co-authored-by: David Turanski <dturanski@pivotal.io>
2020-07-20 18:23:30 -04:00
Soby Chacko
d29a238569 Next version updates
Functions: 1.0.0-SNAPSHOT
  Core Apps: 3.0.0-SNAPSHOT
  Apps: 3.0.0-SNAPSHOT
  Aggregate Next:2020.0.0-SNAPSHOT
2020-07-13 16:02:05 -04:00
Soby Chacko
0e685334b2 Functions: Release - 1.0.0-M2 2020-07-13 14:43:36 -04:00