Commit Graph

66 Commits

Author SHA1 Message Date
Chris Bono
d002bbf7c4 [SB3 Update] Disable HeaderEnricherFunctionApplicationTests 2022-10-16 23:52:42 -05:00
Chris Bono
ea800682b2 [SB3 Update] Agg function use MongoDB testcontainers
- Disable tests
2022-10-16 23:52:42 -05:00
Corneil du Plessis
c64b259d91 [SB3 Update] Remove Apache Geode (#345)
- Also update SFTP usage
2022-10-07 08:22:57 -05:00
Corneil du Plessis
68f2900aad Add intermediate module poms (#337) 2022-10-03 13:51:55 -05:00
Chris Bono
790daf0d91 Convert to Autoconfiguration.imports 2022-10-02 10:58:48 -05:00
Chris Bono
87df01dc00 Remove ${revision} from pom.xml files 2022-10-02 10:58:48 -05:00
Chris Bono
85106dd122 Initial support for SB3/SF6
- Main branch builds successfully (with tests skipped)

See #326
2022-09-22 17:46:11 -05:00
Artem Bilan
2c5d83f926 Disable DB init for JDBC Source and Sink (#325)
* Disable DB init for JDBC Source and Sink

The `IntegrationAutoConfiguration` provides mechanism
to initialize DB with some out-of-the-box scripts.
The default behavior is `EMBEDDED` even if we don't use
any components for those DB objects, e.g. no `JdbcMessageStore` bean.

* Add `spring.integration.jdbc.initialize-schema=NEVER` to the
`jdbc-consumer` and `jdbc-supplier` to not attempt to initialize DB
* Fix Checkstyle violations in other modules
2022-09-21 10:31:18 -05:00
Artem Bilan
910f116332 GH-276 Apply ComponentCustomizer in other modules (#284)
* GH-276 Apply ComponentCustomizer in other modules

Fixes https://github.com/spring-cloud/stream-applications/issues/276

Some changes are straightforward as just a `ComponentCustomizer<?>`
injection and its optional usage for the target component customization.
Other changes are more drastic since it is better to use `proxyBeanMethods = false`,
so some direct bean method calls had to be reworked to the bean method argument
injections

* Fix some deprecations from Project Reactor
* Remove `UpdatingMongoDbMessageSource` since `MongoDbMessageSource`
supports now an `update` option in Spring Integration

NOTE: The `ZeroMqSupplierConfiguration` has its own customizer already
like `Consumer<ZMQ.Socket>`, so it can be revised to a more broader solution,
but in the future version.

Some modules just does not make sense to modify for more advance customization
since all their configuration is covered with plain configuration properties.
See `splitter-function` or `wavefront-consumer`

* * Use `ObjectProvider<WebClientCustomizer>` for `HttpRequestFunctionConfiguration`
instead of our own `ComponentCustomizer`.
Although this has to be revised in favor of fully auto-configured `WebClient.Builder`.
Even that `maximumBufferSize` is included into codecs configuration properties.
* The `MongoDbMessageSource` has now an `update` option in Spring Integration
therefore remove redundant already `UpdatingMongoDbMessageSource`
and populate `this.properties.getUpdateExpression()` directly ot the
`MongoDbMessageSource` bean
* Modify supplier configurations to avoid manual starts for endpoints on
reactive `Publisher` subscriptions.
Spring Integration provides now a `toReactivePublisher(true)` operator
to have all the endpoints stopped on start, but started automatically
when subscription happens to the provided `Publisher`
* Update Copyright to the current year for all the affected classes
* Use `@Nullable` instead of `ObjectProvider` in the `AggregatorFunctionConfiguration`
dependency injection for consistency
* Fix `LogConsumerConfiguration` to not have a `log()` in the end - it is not terminal
in Spring Integration anymore.
Use recommended `nullChannel()` terminating operator for use-cases like this.

* * Remove `beanName` from `ComponentCustomizer` contract

Co-authored-by: Chris Bono <cbono@vmware.com>
2022-07-15 19:36:21 -05:00
sunny151091
5b5fb8fd0f Revise HttpRequestFunctionConfiguration configuration (#295)
Fixes #286
2022-07-15 19:01:30 -05:00
Chris Bono
50d75ef1e7 GH-271: Remove duplication from pom.xml files
This proposal fixes many build warnings and is a general cleanup of the pom.xml files.

* Remove duplicate `dependency` entries  from pom.xml files
* Remove duplicate `repository` entries from pom.xml files
* Replace deprecated use of `receive(long)` in `HttpSourceTests`
* Reduce logging for Twitter integration tests

Fixes #271
2022-05-26 13:57:44 -04:00
Chris Bono
1d153ae728 GH-272: Update to Spring Boot 2.6.8 (#278)
Fixes #272
2022-05-24 21:47:52 -05:00
Artem Bilan
89cd8812c3 GH-195: Introduce ComponentCustomizer (#273)
Fixes #195 

* Introduce a `ComponentCustomizer` contract to be exposed in
the end-user configuration for particular bean type
* introduce a `@CustomizationAware` annotation to mark a component 
as customizable
* Introduce a `ComponentCustomizationBeanPostProcessor` as an auto-configured
bean to apply `ComponentCustomizer` to a matching `@CustomizationAware` (if any)
* Use `@CustomizationAware` in the `FileSupplierConfiguration`
and demonstrate `ComponentCustomizer` feature in its tests
* Move `config-common` to functions parent to make the mentioned auto-configuration
available for all the functions
* Remove some other duplicated dependencies from the affected poms
2022-05-24 20:55:11 -05:00
Chris Bono
77a63f39fd Polish "GH-248: Exclude Geode SSL config"
* Add comments in tests about geode ssl flag verification
2022-04-06 00:06:58 -05:00
Artem Bilan
10ac6a45cb GH-248: Exclude Geode SSL config
Fixes https://github.com/spring-cloud/stream-applications/issues/248

The Geode SSL auto-config is done via `SslAutoConfiguration.SslEnvironmentPostProcessor`
which cannot be excluded as an auto-config.

* Set `SslAutoConfiguration.SECURITY_SSL_ENVIRONMENT_POST_PROCESSOR_ENABLED_PROPERTY`
to `false` from the `ExcludeStoresAutoConfigurationEnvironmentPostProcessor`
when the store type is not Geode
2022-04-06 00:01:35 -05:00
Chris Bono
3af4e8cfe0 Upgrade Geode to support Spring Boot 2.6.3
* Fix Geode integration tests
* Upgrade spring-geode-starter -> 1.6.3
* Upgrade apachegeode/geode -> 1.14.3
* Upgrade spring-data-geode-test -> 0.0.28
2022-01-28 12:13:01 -05:00
Artem Bilan
361c0866b9 * Use spring.mongodb.embedded.version=3.5.5 - aligned with Spring Boot 2022-01-25 09:37:14 -05:00
Artem Bilan
de92924f66 Restore embedded MongoDb tests 2022-01-24 11:30:33 -05:00
Soby Chacko
db61080499 Temporarily disabling a few tests 2022-01-21 17:52:10 -05:00
Artem Bilan
e1987527cf Change doOnSubscribe to doOnRequest
The `aggregatorFunction` has a flaw subscribing and emitting
data from a source `Flux` too early: from a `doOnSubscribe()`
callback which is really called *before* the real subscription is
registered in the target `Publisher`

Change the logic to the `doOnRequest()` which happens already
after subscription is registered in the `Publisher`

**Cherry-pick to `2020.0.x`**
2022-01-20 18:34:21 -05:00
David Turanski
caee3ffa48 Remove tasklauncher-function and tasklauncher-sink 2021-08-17 10:20:53 -04:00
Artem Bilan
daebf0f786 Limit aggregator tests for 10 seconds to wait 2021-04-07 12:02:02 -04:00
Soby Chacko
d0a0df5a44 Restructuing with maven revision feature
Preserving the maven revision feature from the previous commit,
but perform a bit more restructuring the hierarchy.
2020-10-13 15:17:52 -04:00
Daniel Frey
724c4ace5f Refactor hardcoded versions in functions submodule branches
This is so they can be updated in a minimal amount of places and leverage the maven hierarchy to propagate them where appropriate.
While working with SNAPSHOT releases of Spring Boot, Spring Integration, and Spring Cloud, it required touching every pom.xml file
in order to reflect the use of the SNAPSHOT. After this change, it will only be required to change the property in the top
functions pom.xml and the versions will reflect to all submodules.

Maven introduced the ability use a key property, revision to propagate this information to the <parent> declarations in each pom.

This was enabled in Maven 3.5+
https://maven.apache.org/maven-ci-friendly.html
2020-10-13 15:15:59 -04:00
David Turanski
80f9545ae9 Upgrade apps to 3.0.0-SNAPSHOT 2020-10-11 10:43:16 -04:00
Soby Chacko
86f0ca0c07 Next version updates
Functions: 1.0.0-SNAPSHOT
  Core Apps: 3.0.0-SNAPSHOT
  Apps: 3.0.0-SNAPSHOT
  Aggregate Next:2020.0.0-SNAPSHOT
2020-09-29 18:28:20 -04:00
Soby Chacko
e1d29f8d24 Functions: Release - 1.0.0-M3 2020-09-29 14:59:33 -04:00
David Turanski
13fe23ccc0 Remove SplitterFunction autoconfiguration for sources 2020-09-14 14:00:52 -04:00
Christian Tzolov
fcbf941457 Fix another filter function checkstyle issue 2020-08-31 08:47:15 +02:00
Christian Tzolov
35608d28cb Fix a [ImportOrder] checkstyle issue 2020-08-31 08:05:36 +02:00
Artem Bilan
0144476a25 GH-110: Fix filter function for emitting null (#111)
* GH-110: Fix filter function for emitting `null`

Fixes https://github.com/spring-cloud/stream-applications/issues/110

Turns out when functions are composed in the Spring Cloud Stream environment,
they are called via reactive wrappers which don't allow to emit `null` from the
`map()` operator.

* Make `filterFunction` fully reactive to reply on the `Flux.filter()` operator
* Add `config-common` for auto-conversion string configuration options into `Expression` instances
* Add `spring-boot-starter-json` dependency since it is required by the `SpelExpressionConverterConfiguration`
* Add `reactor-test` dependency to test the final solution
* Remove redundant dependencies from the `filter-processor`

* * Add `proxyBeanMethods = false` into `FilterFunctionConfiguration`
* Fix default expression to `true` instead of `payload`, which does not fit to filter logic
* Fix JavaDoc for `expression` property
* Remove redundant `application.properties` from the `filter-function`
2020-08-28 15:55:08 -04:00
Soby Chacko
800b177852 Fixing missing filter processor properties in README 2020-08-25 14:07:06 -04:00
Artem Bilan
4463c1afb6 Add Aggregator function & processor
* Have all the required dependencies in class path which give us a `MessageGroupStore` implementation
* Disable all their auto-configurations in the `ExcludeStoresAutoConfigurationEnvironmentPostProcessor`
* Enable only those which are configured by the `aggregator.messageStoreType` property
* Implement a fully reactive function around an aggregator making an interaction via a pair of `FluxMessageChannel`
* Fix checkstyle issues
* Change the app generator maven plugin to new coordinates
2020-08-20 15:54:51 -04:00
David Turanski
6b0f08712b Add max-buffer-size and fix app deps 2020-08-13 13:27:15 -04:00
David Turanski
93d87a4155 Refactor to use SpEL in function properties
* Clean up per review
2020-08-12 15:39:56 -04:00
Soby Chacko
0c07b4af05 Remove duplicate dependency in twitter-function
Add relativePath to core parent
2020-08-07 16:50:15 -04:00
Christian Tzolov
f8ed9f9c38 Remove double quotes in the properties comments 2020-08-05 15:18:25 +02:00
David Turanski
be818e02de Support filterFunction and spelFunction composition 2020-07-28 12:26:26 -04:00
David Turanski
30caf06198 Fix function definition property (#91)
* Remove deprecated spring.cloud.stream.function.definition property

* Restore tasklauncher-function

Co-authored-by: David Turanski <dturanski@pivotal.io>
2020-07-20 18:23:30 -04:00
Soby Chacko
d29a238569 Next version updates
Functions: 1.0.0-SNAPSHOT
  Core Apps: 3.0.0-SNAPSHOT
  Apps: 3.0.0-SNAPSHOT
  Aggregate Next:2020.0.0-SNAPSHOT
2020-07-13 16:02:05 -04:00
Soby Chacko
0e685334b2 Functions: Release - 1.0.0-M2 2020-07-13 14:43:36 -04:00
Soby Chacko
4b1dd074ba Renaming internal test boot applications in functions 2020-07-02 18:31:58 -04:00
Soby Chacko
51ed4d2f6d Next version updates
Functions: 1.0.0-SNAPSHOT
  Core Apps: 3.0.0-SNAPSHOT
  Apps: 3.0.0-SNAPSHOT
  Aggregate Build: 2020.0.0-SNAPSHOT
2020-06-26 13:49:35 -04:00
Soby Chacko
87e4a2d740 Functions: Release - 1.0.0-M1 2020-06-25 17:42:02 -04:00
Christian Tzolov
a1e26baba8 Replace http https 2020-06-25 18:29:17 +02:00
Christian Tzolov
2a965d1fdc fix readme links 2020-06-25 14:33:55 +02:00
Christian Tzolov
e60184839a fix readme links 2020-06-25 14:30:17 +02:00
Christian Tzolov
db740ebdfd semantic segmentation processor 2020-06-25 12:41:39 +02:00
Christian Tzolov
dffb467da4 Tensorflow functions and applications
* initial step
 * Tensorflow models functional model redesign

      -- Based on https://tzolov.github.io/mind-model-services
      -- Resolves #5

 * Add object detection processor README
 * Add image recognition processor README
 * Initial Tensorflow commonn README
 * Initial Tensorflow commonn README
 * Tensorflow common diagram
 * Tensorflow docs code
 * Tensorflow docs code snippets improve
 * Tensorflow docs code snippets improve
 * Tensorflow docs code snippets improve
 * Tensorflow docs code snippets improve
 * Add semantic segmentation function. add object detecteion function readme
 * oo images
 * Furether oo readme improvments
 * Final obj detection readme fixes
 * Add image recognition readme
 * Add image recognition readme 2
 * Semantic segmentation readme
 * Segmentation readme
 * Semantic segmentation readme 3
 * Fix image recognition and object detcion app starter dependecies

 * Add metadata for Tensorflow apps
2020-06-24 18:37:03 -04:00
Christian Tzolov
3bb9e066b9 Port in the Geo and Users twitter function. Fix issue with handling SpEL expression porperties in the twitter functions. 2020-06-24 12:58:47 +02:00