Fixes: #8950
The `IntegrationFlow.toReactivePublisher(true)` makes `PublisherIntegrationFlow`
to be stopped from the beginning and waiting for the `Publisher.subscribe()`.
However only `EndpointSpec` components are marked as `autoStartup(false)`.
The `MessageProducerSupport` is not included, therefore unexpected messages
are produced to the channel in the end of flow without subscribers.
* Check for `AbstractEndpoint` component type in the `PublisherIntegrationFlow`
and mark it as `setAutoStartup(false)`
* Ensure in a new `ReactiveStreamsTests.messageProducerIsNotStartedAutomatically()` test
that `MessageProducerSupport` is not started from the beginning,
but rather when we subscribe to the `Publisher` via `StepVerifier`
(cherry picked from commit 2b576c57d6)
There is no need to post-process the framework internal BPPs.
Therefore, mark `GatewayProxyInstantiationPostProcessor`, `GlobalChannelInterceptorProcessor`, `IntegrationFlowBeanPostProcessor`
and `BaseIntegrationFlowDefinition.ReplyProducerCleaner` as an `AopInfrastructureBean`.
This way they are not post-processed via AOP `BeanPostProcessor`
* Migrate `org.springframework.integration.aop` tests to JUnit 5
* Fix race condition in the `FluxMessageChannelTests.noMemoryLeakInFluxMessageChannelForVolatilePublishers()`
where `FluxMessageChannel.upstreamSubscriptions` is updated from a different thread
(cherry picked from commit a8ed1dd9aa)
Related to: #8856
Many tests create their own `ThreadPoolTaskScheduler` beans.
Therefore, its default phase might affect the memory and performance.
* Use `phase = SmartLifecycle.DEFAULT_PHASE / 2` for manual
`ThreadPoolTaskScheduler` beans
* Migrate affected tests classes to JUnit 5
* Make some other configuration adjustments for better performance
**Cherry-pick to `6.2.x`**
(cherry picked from commit 39c99c0719)
Fixes: #8856
The `ThreadPoolTaskScheduler` in Spring Framework manages now a lifecycle
with a `Integer.MAX_VALUE` phase by default.
This causes a wait block for scheduled tasks in the `AbstractPollingEndpoint` instances.
These tasks are cancelled by in the later `Integer.MAX_VALUE / 2` phase.
So, we have a lifecycle deadlock on context close
* Fix `DefaultConfiguringBeanFactoryPostProcessor.registerTaskScheduler()`
with a `.addPropertyValue("phase", SmartLifecycle.DEFAULT_PHASE / 2)`
to let the `AbstractPollingEndpoint` to cancel its currently scheduled task
to avoid possible pollution with unexpected (and lost) messages
**Cherry-pick to `6.2.x`**
(cherry picked from commit 11e0ea1305)
Fixes https://github.com/spring-projects/spring-integration/issues/8757
* Add ctor `ObjectStringMessageConverter(Charset)`
* create tests for `ObjectStringMessageConverter`
* fix codestyle issues
* revise JavaDoc
* `@NonNullApi` for `org.springframework.integration.support.converter`
* revise `SimpleMessageConverter`
* `inboundMessageMapper` and `outboundMessageMapper` are initialized to their default implementations
Bboth are set in non-default ctors with final setters
* remove `@Nullable` from `private BeanFactory beanFactory`
* Handle empty `String... headerNames`
* Some code clean up
* Change the default polling trigger to 1 second
The current default trigger for the poller is 10 milliseconds fixed delay.
This is very tight policy for Microservices where we might not have too many
scheduled threads to distribute polling endpoint jobs evenly.
* Change the default trigger to 1 second to align with what Spring Boot already
claims.
Same 1 second policy is used in Spring Cloud Stream as well
* Fix language in Docs
Co-authored-by: Gary Russell <grussell@vmware.com>
---------
Co-authored-by: Gary Russell <grussell@vmware.com>
* Fix maxMessagesPerPoll for SourcePollingChAdapter
The `AbstractMethodAnnotationPostProcessor` does not check
for `PollerMetadata.MAX_MESSAGES_UNBOUNDED` before setting
`maxMessagesPerPoll` into a `SourcePollingChannelAdapter`
which in this case must be `1`
Also fix `SourcePollingChannelAdapterFactoryBean` to not mutate
the provided `PollerMetadata` (which might be global default)
with a new `maxMessagesPerPoll`
**Cherry-pick to `6.1.x` & `6.0.x`**
* * Fix `this.` prefix in `SourcePollingChannelAdapterFactoryBean`
* Fix ThreadSPropagationChInterceptor for stacking
Related SO thread: https://stackoverflow.com/questions/77058188/multiple-threadstatepropagationchannelinterceptors-not-possible
The current `ThreadStatePropagationChannelInterceptor` logic is to wrap one
message to another (`MessageWithThreadState`), essentially stacking contexts.
The `postReceive()` logic is to unwrap a `MessageWithThreadState`,
therefore we deal with the latest pushed context which leads to the `ClassCastException`
* Rework `ThreadStatePropagationChannelInterceptor` logic to reuse existing `MessageWithThreadState`
and add the current context to its `stateQueue`.
Therefore, the `postReceive()` will `poll()` the oldest context which is, essentially,
the one populated by this interceptor before, according to the interceptors order
* Fix `AbstractMessageChannel.setInterceptors()` to not modify provided list of interceptors
* The new `ThreadStatePropagationChannelInterceptorTests` demonstrates the problem
described in that mentioned SO question and verifies that context are propagated
in the order they have been populated
**Cherry-pick to `6.1.x` & `6.0.x`**
* * Fix `ThreadStatePropagationChannelInterceptor` for publish-subscribe scenario.
Essentially, copy the state queue to a new decorated message
* Fix `BroadcastingDispatcher` to always decorate message, even if not `applySequence`
* * Fix unused import in the `BroadcastingDispatcher`
* * Fix unused import in the `ThreadStatePropagationChannelInterceptor`
* GH-2971: Add `LockRegistry.executeLocked()` API
Fixes https://github.com/spring-projects/spring-integration/issues/2971
* Following best practice and well-known patterns with `Jdbc`, `Rest` or `Jms` templates,
introduce `default` methods into `LockRegistry` interface to make it easier to perform
tasks when within a lock.
* Since all the required logic is now covered by those `LockRegistry.executeLocked()` methods,
there is no need in the dedicated abstract `WhileLockedProcessor` class.
Deprecated it for removal in the next version
* Use a new `LockRegistry.executeLocked()` API in the `FileWritingMessageHandler`
instead of just deprecated `WhileLockedProcessor`
* To satisfy Java limitations for checked lambdas, introduce `CheckedCallable` and `CheckedRunnable` utilities
similar to interfaces in the `io.micrometer.observation.Observation`
* Change existing `CheckedFunction` to expose extra generic argument for `Throwable`
* Add dedicated chapter for distributed lock into docs
* Fix some links and typos in the docs
* * Fix Javadoc for `CheckedFunction`
* Fix language in docs
Co-authored-by: Gary Russell <grussell@vmware.com>
---------
Co-authored-by: Gary Russell <grussell@vmware.com>
Fixes https://github.com/spring-projects/spring-integration/issues/8705
an internal `MethodInvocationGateway` is a `MessagingGatewaySupport`
extension with all the logic available there.
One of the option introduced in `5.2.2` to be able to throw a `MessageTimeoutException`
instead of returning `null` when no reply received in time from downstream flow
* Expose an `errorOnTimeout` on the `@MessagingGateway` and `GatewayEndpointSpec`
* Propagate this option from a `GatewayProxyFactoryBean` down to its internal
`MethodInvocationGateway` implementation
* Modify couple tests to react for `errorOnTimeout` set to `true`
* Document the feature
Fix language in Docs
Co-authored-by: Gary Russell <grussell@vmware.com>
* GH-8704: Add global property for `defaultTimeout`
Fixes https://github.com/spring-projects/spring-integration/issues/8704
The default timeout for requests and replies in the integration endpoints
is 30 seconds to avoid indefinite blocking in threads.
Sometime those 30 seconds is not enough.
* Introduce a `spring.integration.endpoints.defaultTimeout` global property
to allow overriding all the timeouts to desired value.
The negative number indicates an indefinite waiting time: similar to what
was there before introducing 30 seconds by default
* Fix language in docs
Co-authored-by: Gary Russell <grussell@vmware.com>
---------
Co-authored-by: Gary Russell <grussell@vmware.com>
Fixes https://github.com/spring-projects/spring-integration/issues/8703
* Instantiate a `MessagingAnnotationBeanPostProcessor` via factory method from `MessagingAnnotationPostProcessor`
avoiding extra code generation on an explicitly provided complex `Map` for bean definition property
* Fix test to react properly to a new logic of `MessagingAnnotationBeanPostProcessor` bean registration
Fixes https://github.com/spring-projects/spring-integration/issues/8664
When an `Observation` is turned to `NoopObservation`,
the `observeWithContext()` fails with `ClassCastException`
since `NoopObservation` serves just plain `Context` not the one we supplied
* Fix `MessagingGatewaySupport.sendAndReceiveWithObservation()` same way
as it is in version `6.0.x`
* Modify `IntegrationObservabilityZipkinTests` to reject some `Observation`
via `observationPredicate()` configuration
**Cherry-pick to `6.1.x`**
* Introduce a `SplitterSpec` which can accept possible
splitter variants: `expression`, `function`, `ref` etc.
This way we are going to have a complex endpoint configuration
only with a single method argument.
* Use `SplitterSpec` in a newly introduced `splitWith()`
* Deprecate those `split()` methods which use `SplitterEndpointSpec`.
This method are complex enough because of their several arguments
* Refactor some common `MessageHandler` options initialization
into the `ConsumerEndpointSpec.doGet()`
* Disable failing now `MethodInvokingMessageProcessorTests.testCollectionArgument()`:
or Jackson problem, or fresh SF
* Groovy DSL will be fixed in the separate PR:
https://stackoverflow.com/questions/76595843/groovy-selects-a-defaultgroovymethods-split-instead-of-mine-one
Fixes https://github.com/spring-projects/spring-integration/issues/8611
According to the latest Spring Framework requirements related to AOT the
`BeanDefinitionRegistryPostProcessor` must not do anything but only bean registrations
* Extract a `MessagingAnnotationBeanPostProcessor` component with a `BeanPostProcessor` logic
from the `MessagingAnnotationPostProcessor` and leave the last one as just
a `BeanDefinitionRegistryPostProcessor` impl
* Fix tests according to a new logic
The `ConsumerEndpointSpec` extensions for Kotlin
don't delegate to the provided `endpointFactoryBean`
* Introduce `KotlinConsumerEndpointSpec` extension for `ConsumerEndpointSpec`
with the proper delegation to the provided spec
* Use `KotlinConsumerEndpointSpec` in the Kotlin-specific `Spec` classes
**Cherry-pick to `6.1.x`, `6.0.x` & `5.5.x`**
* GH-8626: Provide cleaner `transform()` DSL
Fixes https://github.com/spring-projects/spring-integration/issues/8626
* Add missed `transform(String beanName, @Nullable String methodName)` API
* Introduce a `TransformerSpec` to expose a strict API to configure
transformer variants.
* Introduce `transformWith(Consumer<TransformerSpec>)` as a single point
of all possible transformer and its endpoint options
* Deprecate those `IntegrationFlowDefinition.transform()` variants
which are harder to configure as several lambda arguments
This change will make Kotlin & Groovy DSL more readable and straightforward
* * Use new `transformWith()` in tests where a deprecated API still used
* Add JavaDocs to `TransformerSpec`
* Fix generic types for `BaseIntegrationFlowDefinition.transformWith()` -
they make sense exactly on the `TransformerSpec.transformer()` only
* Apply `transformWith()` for Groovy DSL
* Introduce a new `ClassUtils.isLambda(Object candidate)`
and add a check for Groovy `Closure`
* Fix `GroovyIntegrationFlowDefinition.createConfigurerIfAny()` to propagate a `Consumer` argument
down to the `Closure`
* * Rename `TransformerSpec -> TransformerEndpointSpec` for better context
meaning of the class
* Introduce `KotlinTransformerEndpointSpec` as an extension of the `TransformerEndpointSpec`
to have an `inline fun <reified P> transformer(crossinline function: (P) -> Any)`
for Kotlin style
* Add `KotlinIntegrationFlowDefinition.transformWith(KotlinTransformerEndpointSpec)`
* Deprecate Kotlin methods which are covered by the mentioned `transformWith()`
* Fix tests to use new API
* Mentioned the change in the doc
* GH-8642: Revise executors in the project
Fixes https://github.com/spring-projects/spring-integration/issues/8642
* Rework some `Executors.newSingleThreadExecutor()` to `ExecutorServiceAdapter(new SimpleAsyncTaskExecutor())`
* Expose `TaskExecutor` setters; deprecate `ExecutorService`-based
* Some other code clean up in the effected classes: `LogAccessor`, no `synchronized` in critical blocks
* Give a meaningful prefix for default threads in the context of components, e.g. `SubscribableRedisChannel` - `getBeanName() + "-"`
* * Fix `PostgresChannelMessageTableSubscriberTests` for
`PostgresSubscribableChannel` initialization to let it create
its internal `Executor`
* Use an `AsyncTaskExecutor` injection instead of `ExecutorServiceAdapter` wrapping
* Fix `LockRegistryLeaderInitiatorTests` for `taskExecutor` injection
* Bring back `LockRegistryLeaderInitiator.setExecutorService()`
as an accident after property auto-renaming
* Improve Delayer DSL
Move `groupId` option from a `delay()` method arg to the `DelayerEndpointSpec`
to make it cleaner from code reading perspective
* Expose new DSL method based on just a `DelayerEndpointSpec` for Kotlin &v Groovy
* Deprecate multi-arg `delay()` methods in favor of `Consumer<DelayerEndpointSpec>`-based
* * Fix language and code style
* Introduce HeaderFilterSpec to streamline DSL API
The concern has been driven by the discussion from: https://github.com/spring-projects/spring-integration/issues/8625
The point is that Java method arguments are not so descriptive when we read the code.
Therefore, it is better to design DSL the way it would be cleaner from reading perspective.
Plus less choice of methods to chain would give a better end-user experience from coding.
* Add a `HeaderFilterSpec` which can accept `headersToRemove` and `patternMatch` as individual
options instead of top-level deprecated `headerFilter(headersToRemove, patternMatch)` `IntegrationFlow` method.
This way Kotlin and Groovy DSLs get a gain from their "inner section" style.
* Such a `Consumer<HeaderFilterSpec>` way to configure an endpoint is similar to already
existing `aggregate(Consumer<AggregatorSpec>)`, `resequence(Consumer<ResequencerSpec>)` etc.
In other words those components which has a dedicated `ConsumerEndpointSpec` extension are OK
from an idiomatic DSL style perspective
* Expose a `HeaderFilter.setHeadersToRemove()` to make it working smoothly with this new
DSL requirements
* Apply a new `headerFilter()` style into Kotlin and Groovy DSLs
This is just an initial work to surface an idea.
If it is OK, I'll slow continue with others to realign and simplify the paradox of choice.
* * Fix asterisk imports in the `KotlinIntegrationFlowDefinition`
* GH-8625: Add Duration support for `<poller>`
Fixes https://github.com/spring-projects/spring-integration/issues/8625
The duration can be represented in a ISO 8601 format, e.g. `PT10S`, `P1D` etc.
The `<poller>` and `@Poller` don't support such a format.
* Introduce a `PeriodicTriggerFactoryBean` to accept string values for
trigger options and parse them manually before creating the target `PeriodicTrigger`
* Use this `PeriodicTriggerFactoryBean` in the `PollerParser` and `AbstractMethodAnnotationPostProcessor`
where we parse options for the `PeriodicTrigger`
* Modify tests to ensure that feature works
* Document the duration option
* Add more cross-links into polling docs
* Fix typos in the affected doc files
* Add `-parameters` for compiler options since SF 6.1 does not support `-debug` anymore
for method parameter names discovery
* Fix typos
Co-authored-by: Gary Russell <grussell@vmware.com>
---------
Co-authored-by: Gary Russell <grussell@vmware.com>
The `FluxMessageChannel` can subscribe to any volatile `Publisher`.
For example, we can call Reactor Kafka `Sender.send()` for
input data and pass its result to the `FluxMessageChannel`
for on demand subscription.
These publishers are subscribed in the `FluxMessageChannel`
and their `Disposable` is stored in the internal `Disposable.Composite`
which currently only cleared on `destroy()`
* Extract `Disposable` from those internal `subscribe()` calls
into an `AtomicReference`.
* Use this `AtomicReference` in the `doOnTerminate()`
to remove from the `Disposable.Composite` and `dispose()`
when such a volatile `Publisher` is completed
**Cherry-pick to `6.0.x` & `5.5.x`**
* Introduce `PartitionedChannel`
* Implement a `PartitionedChannel` as an extension of the `AbstractExecutorChannel`
* Supply this channel with a `PartitionedDispatcher` which is an extension of the `AbstractDispatcher`
* The target partition is essentially a `UnicastingDispatcher` with a single thead executor
* * Fix language in Javadocs
* Add docs for `PartitionedChannel` into `channel.adoc`
* Pre-populate partitions in the `PartitionedDispatcher`
to ensure a thread number reflection of the partition it is used for (for default `ThreadFactory`)
* Add Javadocs to `public` methods
* Add Java DSL `PartitionedChannelSpec` and respective factory methods into `Channels`
* Use `IntegrationMessageHeaderAccessor.CORRELATION_ID` header as a default partition key
* * Fix language in Javadocs
* Fix Checkstyle violations
* Mark `PartitionedDispatcher.prePopulatedPartitions()` with `synchronized`
* * Populate partitions from `PartitionedDispatcher` ctor
and when a custom `ThreadFactory` is set
* * Clear `executors` in `populatedPartitions()`
* * Bring back `synchronized populatedPartitions()` and use it in the `dispatch()`
Turns out `done` in the `SinkManyEmitterProcessor`
is set to `true` when we already processed all the data.
Therefore, it is better to `await().until()` for `done`
condition in the end of test
* GH-3763: Add `handleReactive()` for Java DSL
Fixes https://github.com/spring-projects/spring-integration/issues/3763
Add a convenient terminal operator to `BaseIntegrationFlowDefinition`
based on a `ReactiveMessageHandler`.
Also add an overload like `handleReactive(ReactiveMessageHandlerSpec)`
to let end-user to choose a protocol-specific channel adapter
* * Fix `Namespace Factory` wording in the `BaseIntegrationFlowDefinition` Javadocs
* Fix language in Docs
Co-authored-by: Gary Russell <grussell@vmware.com>
---------
Co-authored-by: Gary Russell <grussell@vmware.com>
* Fix `IntegrationGraphServerTests` to not use `net.minidev.json.JSONArray` API
since it is not available as transitive from `json-path` in the `testCompileClasspath`
* GH-8586: Deprecate IntegrationComponentSpec.get()
Fixes https://github.com/spring-projects/spring-integration/issues/8586
The `IntegrationComponentSpec` is not a plain wrapper around single component.
Sometimes it comes with several components where all of them must be registered
as beans.
If `IntegrationComponentSpec.get()` is called from end-user code, we may lose
other related components, for example filters in the `FileInboundChannelAdapterSpec`.
* Deprecate `IntegrationComponentSpec.get()` with no-op for end-user,
rather encourage to leave it as is and let the framework take care about its lifecycle
and related components registration
* Fix `IntegrationComponentSpec` logic to deal as a simple `FactoryBean` instead of
extra overhead via `AbstractFactoryBean`
* Use `IntegrationComponentSpec.getObject()` in the framework code where `get()` was called
* Fix tests to expose `IntegrationComponentSpec` as beans instead of previously called `get()`
* Some other clean up and typos fixes in the affected classes
* Document the change
* * Revert `ObjectStringMapBuilder` in the `KafkaInboundGatewaySpec.getComponentsToRegister()`
* Fix language in docs
Co-authored-by: Gary Russell <grussell@vmware.com>
* * Remove trailing whitespace in the `ScriptMessageSourceSpec`
---------
Co-authored-by: Gary Russell <grussell@vmware.com>
Fixes https://github.com/spring-projects/spring-integration/issues/3869
* Move `ContextHolderRequestHandlerAdvice` to the `core` module for more general purposes
* Add `ContextHolderRequestHandlerAdviceTests`
* Rework `DelegatingSessionFactoryTests` to rely on the `ContextHolderRequestHandlerAdvice`.
This allows us to remove unnecessary XML configuration for this test class
* Document the feature
Some tests deliberately don't expect a reply, but they still block
on a gateway's `sendAndReceive()`
* Improve `Jsr223ScriptExecutingMessageProcessorTests`
to verify that script variables work
Currently, many timeouts in the project are like `-1` or other negative value
with a meaning to wait indefinitely.
According to distributed systems design and bad demo developing experience
it is not OK to block forever.
* Rework most of the timeouts in the framework to be `30` seconds.
Only one remained as `1` seconds is a `PollingConsumer` where it is
better to not block even for those 30 seconds when no messages in the queue,
but let the polling task be rescheduled.
* Remove the `MessagingGatewaySupport.replyTimeout` propagation down to the
`PollingConsumer` correlator where it was a `-1` before and blocked
the polling thread on the `Queue.poll()`.
This fixed the problem with a single thread in a pool for auto-configured `TaskScheduler`.
Now with 1 seconds wait time we are able to switch to other scheduled tasks
even with only 1 thread in the pool
Fixes https://github.com/spring-projects/spring-integration/issues/3772
* Add basic To/From Protocol Buffer's com.google.protobuf.Message transformers.
* Allow the proto_type header to specify the type.
* Add tests.
* add Protobuf docs
* Leverage the Spring ProtobufMessageConverter
* move protobuf-java-util to test dependecies as optional
* protobuf docs improvements
* improve the expected type handling
* expected type expression
* fix indentation
* fix indentation for generated test classes
* suppress style check for proto generated classes
* address the transformer doc format
* fix whats new merge conflict
* fix doc sample code
* Some code clean up; fixing typos
Fixes https://github.com/spring-projects/spring-integration/issues/3994
Essentially, migrate those hints from Spring Boot Actuator:
an `IntegrationGraphServer` can be used without Spring Boot
* Introduce `IntegrationGraphRuntimeHints` - implementation of `RuntimeHintsRegistrar`
to register reflection hints for `Graph` and top-level `IntegrationNode` types.
* Use `@ImportRuntimeHints` on the `IntegrationGraphServer` to make those hints
conditional.
* Do not build exception instances in advance
When no `cause` and no clean context of the exception thrown,
the code place where an exception has been created is confusing.
* Fix `SimpleMessageStore` to create an `out of capacity` exception in the
`addMessagesToGroup()` exactly at the point where it is thrown
* Fix `FunctionExpression` and `SupplierExpression` do not pre-create `readOnlyException`:
it is unlikely these kind of expressions are going to be used in the SpEL `write` context,
so we save some time and memory not creating extra object and the plain call `throw new`
gives us a clean context what method call has ended up with such an exception
* * Extract `outOfCapacityException` to the method in the `SimpleMessageStore`
to avoid duplication.
Even if we got in the end extra line in th stack trace, it is still clear from
where it is thrown:
```
org.springframework.messaging.MessagingException: SimpleMessageStore was out of capacity (1) for group 'foo', try constructing it with a larger number.
at org.springframework.integration.store.SimpleMessageStore.outOfCapacityException(SimpleMessageStore.java:324)
at org.springframework.integration.store.SimpleMessageStore.addMessagesToGroup(SimpleMessageStore.java:302)
at org.springframework.integration.store.AbstractMessageGroupStore.addMessageToGroup(AbstractMessageGroupStore.java:189)
at org.springframework.integration.store.SimpleMessageStoreTests.shouldNotHoldMoreThanGroupCapacity(SimpleMessageStoreTests.java:128)
```
