* GH-3959: MqttConFailedEvent for normal disconnect
Fixes https://github.com/spring-projects/spring-integration/issues/3959
The `MqttCallback.disconnected(MqttDisconnectResponse)` in Paho v5 client is also
called when server initiates a disconnection.
In this case that `MqttDisconnectResponse` does not have a `cause` value
* Modify `MqttConnectionFailedEvent` to make a `cause` property optional
* Fix `Mqttv5PahoMessageDrivenChannelAdapter` & `Mqttv5PahoMessageHandler`
to not check for `cause`, but emit an `MqttConnectionFailedEvent` for any `disconnected()` calls
Unfortunately current Paho v3 client does not call `connectionLost()` for normal disconnections
and we cannot react for this callback with an `MqttConnectionFailedEvent`
**Cherry-pick to `5.5.x`**
* Fix language in doc
Co-authored-by: Gary Russell <grussell@vmware.com>
Co-authored-by: Gary Russell <grussell@vmware.com>
Fixes: https://github.com/spring-projects/spring-integration/issues/3822
* Apply spring-framework code style on modified class
* Remove unwanted formatting
* Take pull request comments into account
* Code and JavaDocs clean up
* Improve `Mqttv5BackToBackAutomaticReconnectTests` removing non-related code
* Improve `mqtt.adoc` for this new manual reconnection feature
**Cherry-pick to `5.5.x`**
Fixes https://github.com/spring-projects/spring-integration/issues/3805
The Redis Pub-Sub doesn't work in all the environment, therefore there has to
be a choice to use old busy-spin algorithm
* Change to select between spinLock method and pub-sub method
* Make spinLock as a default one to let the `RedisLockRegistry` work everywhere
* Fix javadoc, convention, lazy init
* Fix javadoc, convention
* Code clean up and docs for `RedisLockType` feature
**Cherry-pick to 5.5.x**
* Remove usage of non-stable `org.springframework.util.SocketUtils`
* Replace it with `0` for those tests where it is possible to select OS port
* Remove the mentioning of the `SocketUtils` from the `testing.adoc`
* Use `TransportConstants.DEFAULT_STOMP_PORT` for `StompServerIntegrationTests`.
We may disable this test in the future for CI if it is not going to be stable
* Introduce `Supplier<String> connectUrl` variants for `ZeroMqMessageHandler`
to let it defer connection evaluation until subscription to the socket `Mono`
in the `ZeroMqMessageHandler`.
* Move connection logic in the `ZeroMqMessageHandler` to `Lifecycle.start()`
Related to https://github.com/spring-projects/spring-framework/issues/28054
**Cherry-pick to `5.5.x`**
# Conflicts:
# spring-integration-stomp/src/test/java/org/springframework/integration/stomp/client/StompServerIntegrationTests.java
* GH-3677: Doc for URL conn customization in FeedCA
Fixes https://github.com/spring-projects/spring-integration/issues/3677
If there is need to have a `URLConnection` customized, the `UrlResource`
has to be used instead of plain `URL` injection into the `FeedEntryMessageSource`
* * Add a sample to docs for connection customization
* Add `toReactivePublisher(autoStartOnSubscribe)`
The `IntegrationFlowBuilder.toReactivePublisher()` returns a `Publisher<Message<?>>`
which may be subscribed somewhere late in the application logic, e.g.
when WebSocket (or RSocket) subscription is initiated by the external client.
In between application context startup and that subscription moment, the `IntegrationFlow`
must not try to produce messages since there is nothing to consumer them from the
`Publisher<Message<?>>` side.
One of the way is to have a source endpoint not started automatically and control its
lifecycle from the point fo reactive subscription
* Introduce an `IntegrationFlowBuilder.toReactivePublisher(boolean autoStartOnSubscribe)`
to let the framework do a job for an `IntegrationFlow` lifecycle control.
This way end-user doesn't need to know autowire a starting endpoint and use `doOnSubscribe()`
and similar callbacks
* Change `ConsumerEndpointFactoryBean` log message about a `FixedSubscriberChannel`
to `INFO` since an `autoStartup = false` really does not have any effect and there is nothing
for end-user to worry about.
The `IntegrationFlow` knows nothing about each endpoint internals and cannot control which
mark as `autoStartup = false` and which not
* Fix languge in JavaDocs
Co-authored-by: Gary Russell <grussell@vmware.com>
Co-authored-by: Gary Russell <grussell@vmware.com>
* GH-3647: Use remoteDirExpression in MV command
Fixes https://github.com/spring-projects/spring-integration/issues/3647
To simplify a source and renameTo remote file expressions, the `remoteDirectoryExpression`
is consulted now, when they are not full paths.
This is useful when we want just to rename a remote file in some dir
* * Add JavaDoc for `getDirectoryExpressionProcessor()`
* Fix language in docs
Resolves https://github.com/spring-projects/spring-integration/issues/3648
When configuring a gateway proxy with XML, but specifying the payload expression
on the method `@Gateway` annotation, the expression was ignored, even though it
had been parsed.
`@Payload` worked.
With this change, if both `@Payload` and `@Gateway` are defined on a gateway method,
`@Gateway.payloadExpression` wins.
* Fix doc links.
* Improve JavaDocs for `BaseIntegrationFlowDefinition` with renaming `payloadType`
param to `expectedType` since it can also be as a `Message.class`, not only type
for the payload.
* Add JavaDoc for `LambdaMessageProcessor` ctor
* GH-3432: Add MQTT v5 channel adapters
Fixes https://github.com/spring-projects/spring-integration/issues/3432
* Add `optional` dependency for `org.eclipse.paho:org.eclipse.paho.mqttv5.client`
* Add `MqttProtocolErrorEvent` and emit it from the `mqttErrorOccurred()` callback of the MQTT v5 client
* Add `MqttHeaderMapper` since MQTT v5 has introduced user properties pair to transfer over the protocol
* Add `Mqttv5PahoMessageHandler` as one more extension of the `AbstractMqttMessageHandler`
* Add more convenient `MqttHeaders` constants for easier headers mapping configuration
* Ensure via `Mqttv5BackToBackTests` that MQTT v5 is supported by the provided components
* Change `pr-build-workflow.yml` to use `eclipse-mosquitto` container for testing all the MQTT interactions
* Change `cyrilix/rabbitmq-mqtt` service to the `rabbitmq:management` since RabbitMQ does not support MQTT v5
* * Handle manual acks
* Add `Mqttv5PahoMessageDrivenChannelAdapter.persistence` property
* * Add documentation
* Add `MosquittoContainerTest` for TestContainers support with Mosquitto image
* Fix language in the docs after review
Co-authored-by: Gary Russell <grussell@vmware.com>
Co-authored-by: Gary Russell <grussell@vmware.com>
* Fix Kotlin deprecation warning in the test
* Replace wrong `IntervalTrigger` mentioning in the docs to the proper `PeriodicTrigger`
* Fix code snippet in the `channel-adapter.adoc`
* Introduce high-level API for flows composition
For better end-user experience and more smooth integration logic
decomposition and distribution introduce an `IntegrationFlows.from(IntegrationFlow)`
to let to start the current flow from existing one.
On the other hand introduce an `BaseIntegrationFlowDefinition.to(IntegrationFlow)`
to let to continue the flow logic in the other existing one.
This way we can extract some templating logic into separate `IntegrationFlow` definitions
allowing at the same time to decompose a complex flow definition into logical reusable parts
* * Add more tests
* * Fix Checkstyle violation
* Add `@SuppressWarnings("overloads")` to new `from(IntegrationFlow)` and existing `from(Publisher)`.
Technically it does not make sense since `PublisherIntegrationFlow` is not a `public` class
* * Add docs
* Fix language in JavaDocs according review
* Fix language in the docs after review
Co-authored-by: Gary Russell <grussell@vmware.com>
Co-authored-by: Gary Russell <grussell@vmware.com>
Related to https://stackoverflow.com/questions/67923303/dynamic-registration-of-websocket-output-adapter-is-not-working
By default the `AbstractHandlerMapping` comes with the `order = Ordered.LOWEST_PRECEDENCE` which sorts
added mappings to the end of chain.
At the same time Spring Boot registers a `SimpleUrlHandlerMapping` as a fallback for all not handled requests
leaving our own mapping behind consideration
* Add `order = 0` to the `IntegrationDynamicWebSocketHandlerMapping` bean registration to let it to be
consulted before `SimpleUrlHandlerMapping`
* Add a note into `web-sockets.adoc` that `@EnableWebsocket` would disable Spring Integration dynamic WebSocket
endpoints
* Add a link from the HTML to the PDF and vice-versa.
* Update doc link text
* I missed that we have three formats: single-page HTML, multiple HTML pages, and PDF.
This commit links the single-page HTML and PDF to the other two formats.
Fixes https://github.com/spring-projects/spring-integration/issues/3451
The GreenMail provides a comprehensive support for e-mail scenarios
testing.
* Use GreenMail instead our own `TestMailServer` for better and broader
support
* Convert SmtpTests and Pop3Tests to GreenMail
* Document that Pop3MailReceiver does not support autoCloseFolder=false
* Pop3MailReceiver uses `folder.getMessageCount()` to determine if there
are new messages.
This count only gets updated when opening the folder.
* Use GreenMail for remaining mail tests
* convert MailTests and ImapMailReceiverTests to GreenMail
* remove duplicate mail dependencies
* route com.sun.mail logging to log4j (via jul and slf4j)
* INT-1828: Docs about filter & release strategy
JIRA: https://jira.spring.io/browse/INT-1828
Document a corner case when filter endpoint is involved in
the flow logic before an aggregator with sequence size release strategy
* Fix typos in the doc changes
Co-authored-by: Gary Russell <grussell@vmware.com>
Co-authored-by: Gary Russell <grussell@vmware.com>
* GH-3504: Explain more an async error handling
Fixes https://github.com/spring-projects/spring-integration/issues/3504
The reaper for an aggregator (as we as a group timeout) are based on the
`TaskScheduler`.
If non-default `TaskScheduler` is used in the configuration, recommend to
supply it with a `MessagePublishingErrorHandler`
* Rewording changes for proper language
Co-authored-by: Gary Russell <grussell@vmware.com>
Co-authored-by: Gary Russell <grussell@vmware.com>
Currently the `MessageHistoryRegistrar` can parse several sources for message history -
`@EnableMessageHistory` and/or `<message-history>`.
Since its logic relies on the reflection it is not compatible with Spring Native.
Plus it causes confusion when several sources are declared so when time comes to
change something in that configuration, we may miss some place to re-align with
our new requirements.
Better to reject extra configurations and enforce end-users to use only one
`@EnableMessageHistory` or `<message-history>`.
This is actually a preferences for many other `@Enable...` in Spring portfolio.
Plus we got a benefit with a Spring Native compatibility
* Fix `MessageHistoryRegistrar` to parse only one `@EnableMessageHistory`.
Register `MessageHistoryConfigurer` function way for Spring Native compatibility
* Clean up `PublisherRegistrar` for better readability
* Fix message history tests which exposed several configurations
* Add JavaDoc into `EnableMessageHistory`
* Refactor `MessageHistoryConfigurer` to let to override patterns configuration
at runtime
* Clean up `message-history.adoc`
* GH-3533: Register WebSocket endpoints at runtime
Fixes https://github.com/spring-projects/spring-integration/issues/3533
* Rework `WebSocketIntegrationConfigurationInitializer` to register beans functional way
to avoid reflection for Spring Native support
* Move `IntegrationServletWebSocketHandlerRegistry` into a separate file for better readability
* Implement `DestructionAwareBeanPostProcessor` for `IntegrationServletWebSocketHandlerRegistry`
to track runtime bean registrations and removals
* Introduce an `IntegrationDynamicWebSocketHandlerMapping` to manage runtime mapping
registrations and removals
* Add `servlet-api` dependency into `websocket` to be able to compile an
`IntegrationDynamicWebSocketHandlerMapping`
* Fix typo in the exception message of the `StandardIntegrationFlowRegistration`
* Start dynamically added beans together with associated `IntegrationFlow` in the
`StandardIntegrationFlowContext`
* Document new feature
* * Fix language in docs
* Don't start those `SmartLifecycle`s together with a dynamic flow
which are not `isAutoStartup()`
* * Fix `TomcatWebSocketTestServer` to configure servlet for `loadOnStartup = 1`
* Fix `WebSocketDslTests` to make `clientWebSocketContainer.setAutoStartup(true)`
* GH-3529: Add HTTP & WebFlux extractResponseBody
Fixes https://github.com/spring-projects/spring-integration/issues/3529
* Expose a convenient `extractResponseBody` option on the HTTP client components
to let end-user to decide if the body of `ResponseEntity` must be extracted (default)
or the whole `ResponseEntity` should be produced as a reply message payload
* Remove a deprecated since `5.3` `encode-uri` option
* Document the new feature
* Rework `webflux.adoc` chapter for the code snippet switcher
* * Fix language in Docs
* Mention a default value in JavaDocs
* GH-3521: Delayer: schedule release task with TX
Fixes https://github.com/spring-projects/spring-integration/issues/3521
There is a race condition when transactional `MessageStore` is used
for `DelayHandler`, so the message is not visible for reads until after TX is
committed, but a scheduled release task may be already ready after delay
* Register a `TransactionSynchronization` with scheduling a releasing task
when TX is committed
**Cherry-pick to `5.4.x` & `5.3.x`**
* Fix language in delayer.adoc
Co-authored-by: Gary Russell <grussell@vmware.com>
Co-authored-by: Gary Russell <grussell@vmware.com>
* Introduce a `groupConditionSupplier` for MGS
* Add a `MessageGroup.condition` option
* Add a `MessageGroupStore.conditionSupplier` option
* Use it from the `SimpleMessageStore.addMessagesToGroup()` API
to populate `condition` (if any) into a `MessageGroup`
* Introduce a `GroupConditionProvider` contract to be implemented
on those `ReleaseStrategy` contracts which could be aware of group condition
* Populated a `GroupConditionProvider.getGroupConditionSupplier()`
into a `MessageGroupStore` from the `AbstractCorrelatingMessageHandler`
for end-user convenience
* Rework a `FileMarkerReleaseStrategy` to implement a `GroupConditionProvider`
to provide a function which produces a condition from `file_lineCount` header
of the `END` marker message
* Make the `FileMarkerReleaseStrategy` logic already based on the condition from a group
* Delegate `GroupConditionProvider` from the `FileAggregator`
* Add test for empty file aggregation
* * Implement `condition` in the `AbstractKeyValueMessageStore` and `MongoDbMessageStore`
* Test `condition` for `mongo-aggregator-config.xml` and `FileAggregatorTests` against GemFire
* * Implement `condition` in the `ConfigurableMongoDbMessageStore`
* * Implement `condition` in the `JdbcMessageStore`
* `FileAggregatorTests` against `JdbcMessageStore`
* Refactor `JdbcMessageStore` for better handling of message group metadata
* Remove unused `MARKED` column in the DDL in favor of newly introduced `CONDITION`
* * Add docs for message group condition
* * Move `conditionSupplier` option from MGS to AbstractCorrelatingMH
* Make it as a `BiFunction` to propagate existing condition alongside with the
message to consult
* Expose `groupConditionSupplier` in Java & XML DSLs
* * Fix language in docs
* INT-4116: Introduce FileAggregator
JIRA: https://jira.spring.io/browse/INT-4116
* Implement a `FileSplitter.FileMarker`-based aggregation strategies
and utilize them in a general `FileAggregator` component
* Make `HeaderAttributeCorrelationStrategy.attributeName` as `final`; add `Assert.notEmpty()`
* Fix `AggregatorFactoryBean` and `AggregatorSpec` to parse the provided processor
for possible `CorrelationStrategy` and/or `ReleaseStrategy`
* Introduce short-cut methods into Java & Kotlin DSL for an `aggregate()` configuration
* Introduce a `FileHeaders.LINE_COUNT` for header to be populated in the `FileSplitter`.
We need this info in the `FileAggregator` to avoid possible overhead with JSON deserialization
of the `FileSplitter.FileMarker` messages
* Test and document the feature
* Improve `FileSplitter` doc for code block switch (tabs)
* * Rework FileAggregator do not use Java Streams
* INT-4377: aggregator groupTimeout as Date
JIRA: https://jira.spring.io/browse/INT-4377
Change the `groupTimeoutExpression` logic to let it to be evaluated to `Date`
instance for some fine-grained scheduling use-case, e.g. to determine a
scheduling moment from the group creation time (`timestamp`) instead of a
current message arrival
* Fix language in docs accoridng PR review
Co-authored-by: Gary Russell <grussell@vmware.com>
Co-authored-by: Gary Russell <grussell@vmware.com>
