Commit Graph

277 Commits

Author SHA1 Message Date
Artem Bilan
02036520b9 Move Spring dependencies to SNAPSHOT
* Fix `AbstractAmqpChannel` to add messaging packages
as allowed for deserialization
* Some code style clean up in the `AbstractSubscribableAmqpChannel`
* Disable Kraft (default) for Kafka tests since they
are not reliable(perhaps only on Windows)
2023-10-10 16:28:15 -04:00
Artem Bilan
7b4d4561f5 Update dependencies; prepare for release
* Fix Checkstyle violations for `private` inner classes where they have to be also `final`
* Fix deprecations after upgrades
* Remove manual coroutines handling in the `GatewayProxyFactoryBean`
in favor of the same functionality in the `JdkDynamicProxy`
2023-09-19 20:49:14 -04:00
Artem Bilan
b8378fc7dc Refactor KafkaMDChA for code duplication (#8724)
Simplify a `KafkaMessageDrivenChannelAdapter.enhanceHeadersAndSaveAttributes()`
method for future maintenance avoiding code duplication
2023-09-06 11:03:20 -04:00
Artem Bilan
d85c5e3a0a GH-8704: Add global property for defaultTimeout (#8706)
* GH-8704: Add global property for `defaultTimeout`

Fixes https://github.com/spring-projects/spring-integration/issues/8704

The default timeout for requests and replies in the integration endpoints
is 30 seconds to avoid indefinite blocking in threads.
Sometime those 30 seconds is not enough.

* Introduce a `spring.integration.endpoints.defaultTimeout` global property
to allow overriding all the timeouts to desired value.
The negative number indicates an indefinite waiting time: similar to what
was there before introducing 30 seconds by default

* Fix language in docs

Co-authored-by: Gary Russell <grussell@vmware.com>

---------

Co-authored-by: Gary Russell <grussell@vmware.com>
2023-08-22 18:04:31 -04:00
Artem Bilan
c157b9b094 Upgrade some deps; fix deprecations 2023-08-12 10:27:48 -04:00
Artem Bilan
b6d39e429b GH-3696: DeserializationEx support for KafkaMS (#8689)
* GH-3696: DeserializationEx support for KafkaMS

Fixes https://github.com/spring-projects/spring-integration/issues/3696

* Add an internal logic into `KafkaMessageSource` to react properly for the
`ErrorHandlingDeserializer` configuration and re-throw `DeserializationException`

* * Use `ErrorHandlingUtils` from `spring-kafka` to determine a `deserializer` from consumer properties

* Revise the test in favor of just single `await().untilAsserted()` -
no reason to wait for assignment or check for returned record.
We just need to be sure that `DeserializationException` is thrown eventually
2023-07-26 16:19:44 -04:00
Artem Bilan
34f901f7db Improve split() DSL (#8666)
* Introduce a `SplitterSpec` which can accept possible
splitter variants: `expression`, `function`, `ref` etc.
This way we are going to have a complex endpoint configuration
only with a single method argument.
* Use `SplitterSpec` in a newly introduced `splitWith()`
* Deprecate those `split()` methods which use `SplitterEndpointSpec`.
This method are complex enough because of their several arguments
* Refactor some common `MessageHandler` options initialization
into the `ConsumerEndpointSpec.doGet()`
* Disable failing now `MethodInvokingMessageProcessorTests.testCollectionArgument()`:
or Jackson problem, or fresh SF
* Groovy DSL will be fixed in the separate PR:
https://stackoverflow.com/questions/76595843/groovy-selects-a-defaultgroovymethods-split-instead-of-mine-one
2023-07-03 10:35:21 -04:00
abilan
8abd72d7c0 Upgrade dependencies; fix problems
* Mockito doesn't allow to `spy()` mocks any more or "overspy"
* Deprecate `KotlinScriptExecutor` if favor of fully supported JSR223
service loaded via `DefaultScriptExecutor`
* Use only one `kotlin-scripting-jsr223` dependency
2023-06-21 17:39:55 -04:00
Christian Tzolov
c38ed96ee9 GH-8643: Replace synchronized with Lock
Fixes https://github.com/spring-projects/spring-integration/issues/8643

* First pass - trivial synchronized blocks
  - Convert the "trivial" `synchronized` block into `ReentrantLock`.

* fix checkstyle

* use blocking lock

* Secon pass - handle multi-lock cases

* javadoc + year

* addres first batch of review suggestions

* fix checkstyle issues

* fix the mqtt parent/child lock monitor sharing

* fix the mqtt parent/child lock monitor sharing, v2

* patch the stomp test
2023-06-21 13:25:45 -04:00
Gary Russell
c4ee5512f6 GH-8638: Kafka: Send All Fails to Failure Channel
Resolves https://github.com/spring-projects/spring-integration/issues/8638

Previously, immediate failures (e.g. timeout getting metadata) were
only thrown as exceptions, and not sent to the failure channel, if present.

**cherry-pick to all supported branches**
2023-06-08 13:44:29 -04:00
Artem Bilan
8b8a7a4c16 Fix DSL for inner bean names generation (#8639)
The `IntegrationFlowBeanPostProcessor.processIntegrationComponentSpec()`
uses a wrong `generateBeanName()` for component to register making the
provided `id` as a prefix

* Use `generateBeanName(Object instance, String prefix, @Nullable String fallbackId, boolean useFlowIdAsPrefix)`
instead to properly "fallback" to the provided name

**Cherry-pick to `6.1.x`**
2023-06-08 09:50:32 -04:00
Artem Bilan
ba417de680 Remove deprecations from previous versions (#8628) 2023-05-22 11:19:13 -04:00
abilan
edbaf6d590 Start version 6.2.0
* Add Christian Tzolov to author name in the doc
* Move `whats-new.adoc` content to new `changes-6.0-6.1.adoc`
* Upgrade to Gradle `8.1.1`
* Upgrade to Spring Framework `6.1`
* Fix `StompSessionManagerTests` for deprecated `ConcurrentTaskScheduler` ctor
* Fix XML configs for Kafka to use types for ctor args instead of their names.
Apparently SF doesn't do a discovery by names anymore.
2023-05-16 20:40:11 -04:00
abilan
d5181bf0d7 Add Nullability support into Java DSL 2023-04-14 14:16:36 -04:00
Artem Bilan
b99729544d GH-8586: Deprecate IntegrationComponentSpec.get() (#8594)
* GH-8586: Deprecate IntegrationComponentSpec.get()

Fixes https://github.com/spring-projects/spring-integration/issues/8586

The `IntegrationComponentSpec` is not a plain wrapper around single component.
Sometimes it comes with several components where all of them must be registered
as beans.
If `IntegrationComponentSpec.get()` is called from end-user code, we may lose
other related components, for example filters in the `FileInboundChannelAdapterSpec`.

* Deprecate `IntegrationComponentSpec.get()` with no-op for end-user,
rather encourage to leave it as is and let the framework take care about its lifecycle
and related components registration
* Fix `IntegrationComponentSpec` logic to deal as a simple `FactoryBean` instead of
extra overhead via `AbstractFactoryBean`
* Use `IntegrationComponentSpec.getObject()` in the framework code where `get()` was called
* Fix tests to expose `IntegrationComponentSpec` as beans instead of previously called `get()`
* Some other clean up and typos fixes in the affected classes
* Document the change

* * Revert `ObjectStringMapBuilder` in the `KafkaInboundGatewaySpec.getComponentsToRegister()`

* Fix language in docs

Co-authored-by: Gary Russell <grussell@vmware.com>

* * Remove trailing whitespace in the `ScriptMessageSourceSpec`

---------

Co-authored-by: Gary Russell <grussell@vmware.com>
2023-04-13 09:16:42 -04:00
abilan
79870dc78c Upgrade dependencies; prepare for release
* Add `spring-web` test dep for `spring-integration-graphql` module
* Fix `MultipartAsRawByteArrayTests` according to the changes in the
`ByteArrayHttpMessageConverter` where it uses `ReadNBytes()` even for
`0` in `Content-Length` header
* Fix Kafka tests to use `RetryListener` interface with `default`
methods instead of already deprecated `RetryListenerSupport`
2023-03-22 16:41:03 -04:00
Artem Bilan
1bec420fd1 Do not block by default (#8580)
Currently, many timeouts in the project are like `-1` or other negative value
with a meaning to wait indefinitely.

According to distributed systems design and bad demo developing experience
it is not OK to block forever.

* Rework most of the timeouts in the framework to be `30` seconds.
Only one remained as `1` seconds is a `PollingConsumer` where it is
better to not block even for those 30 seconds when no messages in the queue,
but let the polling task be rescheduled.
* Remove the `MessagingGatewaySupport.replyTimeout` propagation down to the
`PollingConsumer` correlator where it was a `-1` before and blocked
the polling thread on the `Queue.poll()`.
This fixed the problem with a single thread in a pool for auto-configured `TaskScheduler`.
Now with 1 seconds wait time we are able to switch to other scheduled tasks
even with only 1 thread in the pool
2023-03-21 17:43:00 -04:00
abilan
19d6b7617f More Mockito fixes for test XML configs
Looks like sometime Spring Framework XML parser can properly determine the target factory method to chose,
but sometimes it does that in a wrong order.

* Add `type="java.lang.Class"` to the ctor args everywhere for `class="org.mockito.Mockito" factory-method="mock"`
`<bean>` definitions
2022-12-21 11:16:56 -05:00
abilan
52d43ea8ed Upgrade dependencies; prepare for release
Fix XML configs for new `Mockito.mock()` factory method
2022-12-20 16:25:06 -05:00
abilan
ab1254dcf8 GH-3966: Kafka XML config: Expose more attributes
Fixes https://github.com/spring-projects/spring-integration/issues/3966

Exposed setters in the `KafkaInboundGateway` and `KafkaMessageDrivenChannelAdapter`
as an XML attributes for `kafka` namespace

**Cherry-pick to `5.5.x`**
2022-12-12 16:00:40 -05:00
abilan
66ca0743ed Fix deprecation for RetryListenerSupport
Related to https://github.com/spring-projects/spring-retry/pull/326
2022-12-01 12:24:11 -05:00
Artem Bilan
3c40b01915 Expose more reflection hints (#3951)
* Expose more reflection hints

* Bring back `@Reflective` on `Pausable` - for possible end-user usage
* Add `ReactiveMessageHandler` hint since its method is used reflectively
in the `IntegrationRSocketMessageHandler`
* Add `@Reflective` on the `ServerRSocketMessageHandler.handleConnectionSetup()`
since it is used reflectively for a `registerHandlerMethod()`
* Add `KafkaRuntimeHints` to expose `Pausable` contract on Kafka inbound endpoints
for SpEL invocation via Control Bus
* Document native images support

* Fix language in docs

Co-authored-by: Gary Russell <grussell@vmware.com>

Co-authored-by: Gary Russell <grussell@vmware.com>
2022-11-18 11:26:46 -05:00
Gary Russell
77a752934d Improve Kafka Exception Message
- when we can't determine reply topic
2022-11-15 14:41:39 -05:00
Gary Russell
e19e6d4726 GH-3942: Fix Race in Kafka OB Gateway
Resolves https://github.com/spring-projects/spring-integration/issues/3942

When determining the default reply-to topic/partition, we need to wait for assignment.

Already covered by `KafkaDslTests` (a recent build failure exposed this problem).

**No back-port - 5.5.x uses 2.7.x by default, which does not support this.**

5.5.x users can call `waitForAssignment` on the `ReplyingKafkaTemplate` that is
supplied to the gateways before sending messages.
2022-11-15 08:39:54 -05:00
abilan
7864658d01 GH-3686: Apply SF editor config
Fixes https://github.com/spring-projects/spring-integration/issues/3686

* Add `src/idea` with respective editor config for IntelliJ IDEA.
Must be imported into an IDE
* Add `src/eclipse` with respective editor config for Eclipse/STS
* Reformat imports in source code according a new editor config
2022-11-14 10:55:21 -05:00
Artem Bilan
beef2e68c7 Use unique consumer groups in KafkaDslKotlinTests 2022-10-18 13:20:38 -04:00
Artem Bilan
daaa30e67f Make replyContainer as bean in Kafka tests
It looks like `replyContainer()` is not registered as a bean,
so its lifecycle is somehow out of application context control

* Mark `replyContainer()` method as a `@Bean` in the `KafkaDslTests` and `KafkaDslKotlinTests`
to see if this fixes flaky state of the test suite
* Upgrade to Kotlin `1.7.20` and fix `KafkaDslKotlinTests` according to its requirements
2022-10-06 16:01:08 -04:00
Artem Bilan
a667171c4f Use unique Kafka consumer groups in tests
so, it doesn't cause unexpected rebalances in the global shared embedded Kafka broker
2022-10-06 10:50:01 -04:00
Gary Russell
cbfa150dfa GH-3661: Fix Javadocs for RetryingMLA
Use of the `RetryingMessageListenerAdapter` was removed in
0550380704
but the javadocs were not corrected.
2022-09-21 16:04:20 -04:00
Artem Bilan
91a8b14b00 Move dependencies to Milestones for release
* Adapt code to the changes in upstream libs
* Change consumer groups in Kafka tests to unique values to avoid clashes
2022-09-20 11:43:36 -04:00
Artem Bilan
47cfae9d14 Close volatile resource in some Kafka tests 2022-09-07 17:27:39 -04:00
Artem Bilan
96d3b005fb Add @DirtiesContext to a couple Kafka tests 2022-09-07 16:05:48 -04:00
Artem Bilan
690743ee0c Use Global Embedded Kafka whenever possible
For better test suite lifecycle (higher performance)
reuse one global embedded Kafka broker introduced in Spring for Apache Kafka `3.0`

Some tests have left with their own `EmbeddedKafkaBroker` definitions since they
rely on different partitions
2022-08-01 12:27:58 -04:00
Artem Bilan
5572c2161d Fix deprecations around ListenableFuture (#3865)
* Fix deprecations around ListenableFuture

SF has deprecated a `ListenableFuture` and API around it

* Migrate to `CompletableFuture` everywhere a `ListenableFuture` has been used
* Suppress a deprecation for `ListenableFuture` keeping the functionality until the next version
* Resolve deprecations nad removals from the latest Spring for Apache Kafka
* Fix documentation for the `ListenableFuture` in favor of `CompletableFuture`

NOTE: the AMQP module is left as is until `ListenableFuture` deprecation is resolved in Spring AMQP

* * Restore some `ListenableFuture` test for messaging gateway
2022-07-28 09:32:01 -04:00
Artem Bilan
573c29793a Fix SIK module for latest SK compatibility
Spring for Apache Kafka does not produce `ListenableFuture` anymore.

* Rework `KafkaProducerMessageHandler` to deal with the `CompletableFuture` from now on
* Add support for `CompletableFuture` replies handling into `AbstractMessageProducingHandler`
* Remove redundant cast in the `DispatcherHasNoSubscribersTests`
2022-07-25 17:19:19 -04:00
Artem Bilan
d9a42c9e14 Fix KafkaOutAdapterParserTests race condition
The `Future` for `Producer.send()` can be fulfilled before
we return it (`isDone()`), so `KafkaTemplate` short-circuits
with a `KafkaException`.

* Modify `KafkaOutboundAdapterParserTests.testSyncMode()`
to assert against `KafkaException` which may be thrown earlier
then previously expected `KafkaProducerException`

**Cherry-pick to `5.5.x`**
2022-07-18 13:05:42 -04:00
Artem Vozhdayenko
53dd050c5b GH-3623: Deprecarte an IntegrationFlows
Fixes https://github.com/spring-projects/spring-integration/issues/3623

* `IntegrationFlow` refactoring
* Apply several code style improvements and good practices
* Code style: no empty lines for methods javadocs
* make deprecated implementation reuse actual one instead of the copy-paste approach
* add whats-new comments
* Fix whats-new page according to standards
2022-07-05 15:47:30 -04:00
Artem Bilan
2022c40d55 Fix compatibility with the latest SF
* Upgrade Spring dependencies to the latest SNAPSHOTs
* Fix tests to verify against stack traces: the message
of the `NestedRuntimeException`  does not include the nested exception information.
Related to https://github.com/spring-projects/spring-framework/issues/25162
* Fix `JdbcMessageStore` and `DefaultLockRepository` to rely on the `DataIntegrityViolationException`
instead of only its `DuplicateKeyException` extension.
This is the current behavior of the SQL errors translation
* Disable `WebFluxDslTests.testValidation()` - doesn't subscribe to the reply somehow...
* Refine `SimplePool.PoolSemaphore.reducePermits()`
2022-06-27 20:30:14 -04:00
Artem Bilan
cdcc986d11 Fix KafkaMDChannelAdapter error handling
Current `IntegrationRecordMessageListener.onMessage()`
send a conversion error to the `errorChannel`, but
it does not return if it was successful leading to
the NPE `enhanceHeadersAndSaveAttributes()` because the `message`
is null

* Change the `IntegrationRecordMessageListener.onMessage()` logic to check
the result of the `sendErrorMessageIfNecessary()` and return immediately
if success.
Rethrow an exception otherwise.
2022-05-31 14:08:07 -04:00
Artem Bilan
22b74c7a33 GH-3790: Use new header constants for Kafka headers
Fixes https://github.com/spring-projects/spring-integration/issues/3790

Some `KafkaHeaders` constants have been removed and replaced with new
more meaningful

* Fix removed constants everywhere in the code and docs in favor of
newly introduced, which replaces old
2022-05-05 21:51:16 -04:00
Artem Bilan
d5501d8273 GH-3734: Support MessageHistory on Kafka deser
Fixes https://github.com/spring-projects/spring-integration/issues/3734

The `MessageHistory` is not in the trusted packages of the `DefaultKafkaHeaderMapper`
therefore it fails when `MessageHistory.read()` is performed.

* Configure default `DefaultKafkaHeaderMapper` in the `SubscribableKafkaChannel`,
`KafkaInboundGateway`, `KafkaMessageDrivenChannelAdapter` and `KafkaMessageSource`
to also trust packages exposed via `JacksonJsonUtils.DEFAULT_TRUSTED_PACKAGES`
which include a `MessageHistory`, too.

* Verify in some integration Kafka tests to be sure that `MessageHistory` is
deserialized properly in the transferred headers
* Rework some tests to use `@EmbeddedKafka` instead of `@BeforeAll/@AfterAll`

**Cherry-pick to `5.5.x`**
2022-03-22 15:39:00 -04:00
Artem Bilan
50cf364cde Fix kafka tests according latest spring-kafka 2022-03-16 17:21:40 -04:00
Gary Russell
0550380704 GH-3661: Resolve Spring Apache Kafka Deprecations
Resolves https://github.com/spring-projects/spring-integration/issues/3661

* Don't allow existing listener; remove unnecessary try/catch; add what's new.
* Fix link in whats-new.adoc
2022-03-16 14:46:10 -04:00
Artem Bilan
11c3acdf2c Remove usage of isOnlyLogRecordMetadata
The `ConsumerProperties.isOnlyLogRecordMetadata` has been removed
in Spring for Apache Kafka in favor of `KafkaUtils.setConsumerRecordFormatter()`
2022-03-15 13:48:07 -04:00
Artem Bilan
5c912b1315 Fix KafkaMessageSource for the latest SK
* Add `mockito-inline` dep to be able to mock `final` classes, e.g. `record`
2022-03-08 16:24:30 -05:00
Gary Russell
12f2084ae8 Switch to spring-kafka 3.0.0-SNAPSHOT
- deprecations are temporarily restored there
 - fix deprecation warnings
 - re-enable tests for Windows
2022-03-08 10:10:27 -05:00
Artem Bilan
b3ae24eac6 Fixes according latest upgrades
* Add `--add-opens` to `asciidoctor` Gradle task to avoid compilation warning
* Upgrade to Kotlin `1.6.10` and modify its Gradle task respective
* Add empty impl for new `AnnotationMetadataAdapter.getDeclaredMethods()`
* Parse HTTP methods to new `String methodNames` property of the `RequestMapping`
since `HttpMethod` is not an `enum` anymore.
* Ignore JDBC tests which rely on a map property resolution: the `[]` placeholder
is not handled in the latest SF anymore.
* Fix `WebFluxInboundEndpoint` for deprecations
2022-01-18 14:38:53 -05:00
Artem Bilan
a80b22638d Start 6.0 version
* Upgrade to Java 17, SF-6.0, Gradle 7.2
* Upgrade to Jakarta dependencies and respective namespaces
* Fix some tests for Java 17 compatibility
* Fix wrong Javadocs
* Add some missed Javadocs
* Fix more `jakarta` namespace
* Fix WS & XML modules to use Jakarta EE
* `--add-opens` in some modules for their reflection-based tests
* Disable Kafka tests which does not work on Windows; see Apache Kafka `3.0.1`
* Upgrade to JUnit `5.8.1`
* Migrate JMS tests to Artemis
* Remove RMI module as it was deprecated before
* Fix `pr-build-workflow.yml` for Java 17
* Fix JavaDocs warnings using `Xdoclint:syntax` per module, not in the top-level `api` task
* Move docs for version `6.0`
2022-01-18 14:38:50 -05:00
Artem Bilan
6f3fdc76c1 Some docs clean up 2021-11-15 10:00:14 -05:00
Artem Bilan
34d2b0904f Fix new Sonar smells 2021-10-07 10:59:43 -04:00