Commit Graph

106 Commits

Author SHA1 Message Date
Artem Bilan
46dc4d0dd5 GH-500: Workaround for non-serializable header (#503)
Fixes https://github.com/spring-cloud/stream-applications/issues/500

When `listeners` are provided for `DefaultKafkaConsumerFactory`,
the target `KafkaConsumer` instance is proxied.
The `java.lang.reflect.Proxy` is `Serializable`,
but the value it is wrapping is not.
When the `MessageHeaders` is serialized (e.g. into persistent `MessageStore`),
it checks for `Serializable` type only on top-level object of the header.
Therefore, the `Proxy` is passing condition, but eventually we fail
with `NotSerializableException`, since the proxied object is not like that

* Remove `kafka_consumer` from a message before it reaches an aggregator
with its logic to serialize message into the store

This is a workaround until Spring for Apache Kafka is released
with the fix: https://github.com/spring-projects/spring-kafka/pull/2822
2023-09-22 15:02:02 -05:00
GitHub Actions Bot
33f92ab047 Next development version 4.0.0-SNAPSHOT / 2022.0.0-SNAPSHOT 2023-07-21 12:40:45 +00:00
GitHub Actions Bot
99bd5982c9 Release 4.0.0-RC2 / 2022.0.0-RC2 2023-07-21 12:40:08 +00:00
GitHub Actions Bot
5f15796c8a Next development version 4.0.0-SNAPSHOT / 2022.0.0-SNAPSHOT 2023-07-20 14:56:01 +00:00
GitHub Actions Bot
f109933fda Release 4.0.0-RC2 / 2022.0.0-RC2 2023-07-20 14:55:34 +00:00
GitHub Actions Bot
f65cb72e0c Next development version 4.0.0-SNAPSHOT 2023-06-14 04:47:46 +00:00
GitHub Actions Bot
354c82a114 Release 4.0.0-RC1 2023-06-14 04:47:29 +00:00
Artem Bilan
a841beba15 GH-475: Add some missed nested props for apps (#476)
Fixes https://github.com/spring-cloud/stream-applications/issues/475

* Fix JDBC props for limited listed number of them
2023-06-12 23:35:10 -05:00
Corneil du Plessis
ee8738e838 Restore change for Mongo Tests. 2023-06-08 23:56:57 +02:00
Corneil du Plessis
f99016592f [Update] Spring Boot to 3.1.0 2023-06-08 18:47:25 +02:00
abilan
35a37fac5a GH-275: Mention ComponentCustomizer in READMEs
Fixes https://github.com/spring-cloud/stream-applications/issues/275

Many functions make use of `ComponentCustomizer` ot let end-user
to provide any custom configuration for target component used by the function.
First of all for use-case which are not covered by standard configuration
properties and secondly for those options which cannot be represented
as a configuration option.

* Add a sentence to READMEs of those functions which make use of `ComponentCustomizer`
with an expected generic argument type
* Add README into `rabbit-supplier` where it is fully missed at the moment
2023-06-05 22:21:01 -04:00
Artem Bilan
99d2525b3b GH-441: Add JsonBytesToMap function (#461)
Add `JsonBytesToMap` function to allow conversion between byte[] and JSON Map.

Fixes https://github.com/spring-cloud/stream-applications/issues/441

* Introduce a `JsonBytesToMap` as a part of a `payload-converter-function` module
which is auto-discovered by Spring Cloud Function scanning algorithm - the `functions` package.
* Add a `payload-converter-function` as dependency into an `aggregator-function`
* Compose `jsonBytesToMap|aggregatorFunction` for the `aggregator-processor`
* Verify a `JsonBytesToMap` function in action with an `AggregatorProcessorTests`
* Mentioned such a payload conversion in the `aggregator-processor` README
2023-05-30 18:25:27 -05:00
GitHub Actions Bot
bdc100a6c1 Next development version 4.0.0-SNAPSHOT 2023-05-25 11:30:37 +00:00
GitHub Actions Bot
251e1fb998 Release 4.0.0-M2 2023-05-25 11:30:19 +00:00
abilan
75fcc55b25 Disable DefaultAggregatorTests 2023-05-02 17:12:36 -04:00
Corneil du Plessis
85f1629fee Revert deploy-plugin to 2.8.2 for stream-applications-descriptor 2023-04-21 11:07:43 +02:00
Corneil du Plessis
b0b3f14992 Fix header filter test. Improve aggregator-filter test configuration. 2023-04-06 12:21:08 +02:00
Corneil du Plessis
2b66ea3bff Added header-filter function and processor. 2023-04-05 18:02:33 +02:00
Corneil du Plessis
8e4c20dc77 GH-430: Change Filter function to non-reactive
Fixes https://github.com/spring-cloud/stream-applications/issues/430

Fix greenmail exclusion.
Added -X for github.debug
2023-02-14 11:23:31 -05:00
Corneil du Plessis
d29546c9bd Added logging to aggregator. 2023-02-14 16:59:41 +02:00
Corneil du Plessis
49736da55e Remove assertion for list 2023-02-09 12:29:59 +02:00
Corneil du Plessis
3bf8656c8f Remove assertion for list 2023-02-09 11:39:33 +02:00
Corneil du Plessis
3a0479a3ad Increase timeout for DefaultAggregatorTests 2023-02-07 21:40:28 +02:00
Corneil du Plessis
e9d8fb0c31 Added xmpp to the documentation.
Fixing remaining versions.

Updates #429
2023-02-02 18:51:01 +02:00
Corneil du Plessis
722fb9182a Added <protocArtifact>com.google.protobuf:protoc:${protobuf.version}</protocArtifact> to protoc-jar-maven-plugin in object-detection-function. 2023-01-17 11:25:55 +02:00
Corneil du Plessis
9f9b6a0c19 Fix checkstyle errors. 2023-01-12 11:05:43 +02:00
Corneil du Plessis
1abad6d76c Fix object-detection dependencies and cleanup samples. 2023-01-12 10:50:28 +02:00
Chris Bono
14390fe040 GH-327: Use TestSocketUtils from spring-test (#420)
Fixes https://github.com/spring-cloud/stream-applications/issues/327

- Delete local TSU temporary implementations
- Replace local TSU w/ spring-test TSU
2023-01-11 11:33:31 -05:00
Corneil du Plessis
570a8ec777 Upodated aggregator RedisMessageStoreAggregatorTests to use RedisTestContainerSupport. 2023-01-11 16:20:14 +02:00
Corneil du Plessis
189341b8f4 Added TODOs update versions. 2023-01-06 16:49:14 +02:00
Corneil du Plessis
3917ce241b [SB 3] Fix checkstyle in http-request-function. (#406)
* [SB 3] Fix checkstyle in http-request-function.
2022-11-07 16:57:11 +02:00
Corneil du Plessis
dd2a70dda3 Merge remote-tracking branch 'origin/main'
# Conflicts:
#	.github/workflows/ci-pr.yml
2022-11-07 15:56:35 +02:00
Corneil du Plessis
00249f911e [GitHub Actions] Improve dependencies for main. Provide for building impacted dependencies for a PR. 2022-11-07 15:56:07 +02:00
Corneil du Plessis
392b5bfbbe [Issues] Change http-request to non-reactive (#402)
Fixes #206
2022-11-04 17:40:01 +02:00
Chris Bono
d3b339ae01 Update checkstyle plugin to 3.2.0
- Spring Javaformat update to 0.0.35
- Puppycrawl update to 10.3.4
2022-10-24 23:55:49 -05:00
Artem Bilan
735d42048c GH-373: Re-enable aggregator tests (#377)
Fixes https://github.com/spring-cloud/stream-applications/issues/373

The fix in Spring Integration has made it into the latest SNAPSHOT.
See more info in the linked issue

* Update to the latest Testcontainers: it does not require credential for pull command any more
* Remove obsolete `MessageStoreType.GEMFIRE`
* Add `<doclint>syntax</doclint>` to ignore missed javadocs on public methods
2022-10-19 14:09:32 -05:00
Artem Bilan
f17a0b3ec0 GH-369: Fix HeaderEnricher for no proxyBeanMethod (#376)
Spring Boot `@AutoConfiguration` comes now with a `proxyBeanMethods = false`,
so we cannot call bean methods within the same configuration class.

* Rework `HeaderEnricherFunctionConfiguration` for bean methods autowiring
* Create an `ExpressionEvaluatingHeaderValueMessageProcessor` instances directly
in the `headerEnricher` bean definition and propagate an injected `BeanFactory`
* Re-enable disabled tests

Fixes #369
2022-10-19 12:18:50 -05:00
Corneil du Plessis
8a7dca70ed Ensure module projects skip deploy 2022-10-18 15:18:54 +02:00
Chris Bono
f4d396ab40 [SB3 Update] Adjust jsonpath on TwitterUsersFunctionTests 2022-10-16 23:52:42 -05:00
Chris Bono
1a941bf649 [SB3 Update] Add HttpMethod converter to http-request function
- HttpMethod moved from enum to class in Spring Framework. Needs converter now.
2022-10-16 23:52:42 -05:00
Chris Bono
d351163cd2 [SB3 Update] Disable HeaderEnricherFunctionApplicationTests 2022-10-16 23:52:42 -05:00
Chris Bono
e6a2824f61 [SB3 Update] Agg function use MongoDB testcontainers
- Disable tests
2022-10-16 23:52:42 -05:00
Corneil du Plessis
ec7b11e707 [SB3 Update] Remove Apache Geode (#345)
- Also update SFTP usage
2022-10-07 08:22:57 -05:00
Corneil du Plessis
f6a3ab653d Add intermediate module poms (#337) 2022-10-03 13:51:55 -05:00
Chris Bono
6f9776f94e Convert to Autoconfiguration.imports 2022-10-02 10:58:48 -05:00
Chris Bono
002204270e Remove ${revision} from pom.xml files 2022-10-02 10:58:48 -05:00
Chris Bono
a8814f876f Initial support for SB3/SF6
- Main branch builds successfully (with tests skipped)

See #326
2022-09-22 17:46:11 -05:00
Artem Bilan
5e78da73ef Disable DB init for JDBC Source and Sink (#325)
* Disable DB init for JDBC Source and Sink

The `IntegrationAutoConfiguration` provides mechanism
to initialize DB with some out-of-the-box scripts.
The default behavior is `EMBEDDED` even if we don't use
any components for those DB objects, e.g. no `JdbcMessageStore` bean.

* Add `spring.integration.jdbc.initialize-schema=NEVER` to the
`jdbc-consumer` and `jdbc-supplier` to not attempt to initialize DB
* Fix Checkstyle violations in other modules
2022-09-21 10:31:18 -05:00
Artem Bilan
61f1412365 GH-276 Apply ComponentCustomizer in other modules (#284)
* GH-276 Apply ComponentCustomizer in other modules

Fixes https://github.com/spring-cloud/stream-applications/issues/276

Some changes are straightforward as just a `ComponentCustomizer<?>`
injection and its optional usage for the target component customization.
Other changes are more drastic since it is better to use `proxyBeanMethods = false`,
so some direct bean method calls had to be reworked to the bean method argument
injections

* Fix some deprecations from Project Reactor
* Remove `UpdatingMongoDbMessageSource` since `MongoDbMessageSource`
supports now an `update` option in Spring Integration

NOTE: The `ZeroMqSupplierConfiguration` has its own customizer already
like `Consumer<ZMQ.Socket>`, so it can be revised to a more broader solution,
but in the future version.

Some modules just does not make sense to modify for more advance customization
since all their configuration is covered with plain configuration properties.
See `splitter-function` or `wavefront-consumer`

* * Use `ObjectProvider<WebClientCustomizer>` for `HttpRequestFunctionConfiguration`
instead of our own `ComponentCustomizer`.
Although this has to be revised in favor of fully auto-configured `WebClient.Builder`.
Even that `maximumBufferSize` is included into codecs configuration properties.
* The `MongoDbMessageSource` has now an `update` option in Spring Integration
therefore remove redundant already `UpdatingMongoDbMessageSource`
and populate `this.properties.getUpdateExpression()` directly ot the
`MongoDbMessageSource` bean
* Modify supplier configurations to avoid manual starts for endpoints on
reactive `Publisher` subscriptions.
Spring Integration provides now a `toReactivePublisher(true)` operator
to have all the endpoints stopped on start, but started automatically
when subscription happens to the provided `Publisher`
* Update Copyright to the current year for all the affected classes
* Use `@Nullable` instead of `ObjectProvider` in the `AggregatorFunctionConfiguration`
dependency injection for consistency
* Fix `LogConsumerConfiguration` to not have a `log()` in the end - it is not terminal
in Spring Integration anymore.
Use recommended `nullChannel()` terminating operator for use-cases like this.

* * Remove `beanName` from `ComponentCustomizer` contract

Co-authored-by: Chris Bono <cbono@vmware.com>
2022-07-15 19:36:21 -05:00
sunny151091
7b3309707f Revise HttpRequestFunctionConfiguration configuration (#295)
Fixes #286
2022-07-15 19:01:30 -05:00