Fixes: #9297
For a normal `jakarta.mail.StoreClosedException: * BYE Jakarta Mail Exception: java.net.SocketException: Connection reset`
the `ImapIdleChannelAdapter.callIdle()` results in a resubmission (and reconnection)
with logging 'Failed to execute IDLE task. Will attempt to resubmit in 10000 milliseconds.'`
However, when `selectorExpression` is in used and that one is based on mail `Message` object,
we may fail with `FolderClosedException` which is wrapped to the `SpelEvaluationException` and some
other stack traces.
So, `jakarta.mail.MessagingException` might be deep in the cause chain.
* Fix `ImapIdleChannelAdapter` via introducing `getJakartaMailMessagingExceptionFromCause()` utility
method which searches for the `jakarta.mail.MessagingException` in cause chain.
* Rework `ImapIdleChannelAdapter.callIdle()` logic to rely on this new method
**Auto-cherry-pick to `6.3.x` & `6.2.x`**
* Fix ImapIdleChA for scheduling race condition
The IMAP IDLE is long-lived process and can be blocked
waiting for any reply from the server.
This way it is not suited to be used in a `TaskScheduler`
especially when it has only one thread in its pool in Spring Boot by default.
Another concurrent scheduled task is exactly an `ImapMailReceiver.IdleCanceller`.
With a single thread in a `TaskScheduler` pool it cannot be reached
therefore we never cancel and IDLE task and cannot react to the connection loss properly
* Rework the `ImapIdleChannelAdapter` logic to use a regular `Executor` and `while()` loop
with a `Thread.sleep()` when we lose connection
* Clean up the `ImapMailReceiverTests` from `TaskScheduler` not used anymore.
* Expose new `taskExecutor` option in the `ImapIdleChannelAdapterSpec` for Java DSL
* Enable `ImapMailReceiverTests.testIdleWithMessageMapping()` with an attempt to see
if this fix covers an unclear problem exposed before
* * The `testIdleWithMessageMapping()` still fails on GH actions
* GH-8577: Revise `ImapIdleChannelAdapter` logic
Fixes https://github.com/spring-projects/spring-integration/issues/8577
When we process mail messages in async manner, it is possible that we end up
in a race condition situation where the next idle cycle closes the folder.
It is possible to reopen the folder, but feels better to block the current idle
cycle until we are done with the message and therefore keep folder opened.
* Deprecate `ImapIdleChannelAdapter.sendingTaskExecutor` in favor of an `ExecutorChannel`
as an output for this channel adapter or similar async hand-off downstream.
* Make use of `shouldReconnectAutomatically` as it is advertised for this channel adapter
* Optimize the proxy creation for message sending task
* * Remove `ImapIdleChannelAdapter.sendingTaskExecutor`
* Fix language in docs
Co-authored-by: Gary Russell <grussell@vmware.com>
---------
Co-authored-by: Gary Russell <grussell@vmware.com>
Fixes https://github.com/spring-projects/spring-integration/issues/3686
* Add `src/idea` with respective editor config for IntelliJ IDEA.
Must be imported into an IDE
* Add `src/eclipse` with respective editor config for Eclipse/STS
* Reformat imports in source code according a new editor config
Fixes https://github.com/spring-projects/spring-integration/issues/3421
Remove `throws Exception;` from production code to honor
the rule `Generic exceptions should never be thrown` which is enabled on SonarQube
* Rework affected usages to `try..catch` with throwing respective runtime exception
or just logging
* Some other refactoring for the affected classes
* Mostly changes are related to the `TaskScheduler` and `Trigger` APIs
* Migrate to `micrometer-tracing` dependency
* Rework `SocketTestUtils` to use a `InetAddress.getLocalHost()`
for more stability and performance on Windows
* Fix docs for new `PeriodicTrigger` API
* Upgrade to Java 17, SF-6.0, Gradle 7.2
* Upgrade to Jakarta dependencies and respective namespaces
* Fix some tests for Java 17 compatibility
* Fix wrong Javadocs
* Add some missed Javadocs
* Fix more `jakarta` namespace
* Fix WS & XML modules to use Jakarta EE
* `--add-opens` in some modules for their reflection-based tests
* Disable Kafka tests which does not work on Windows; see Apache Kafka `3.0.1`
* Upgrade to JUnit `5.8.1`
* Migrate JMS tests to Artemis
* Remove RMI module as it was deprecated before
* Fix `pr-build-workflow.yml` for Java 17
* Fix JavaDocs warnings using `Xdoclint:syntax` per module, not in the top-level `api` task
* Move docs for version `6.0`
If application is connected to a Domino mail server via IMAP, it can happen from time to time that a message is expunged.
This leads to a `MessageRemovedException` when calling `IMAPMessage#getSubject`.
And although debug is set to false this again leads to a `MessageException` and the whole integration flow stops.
* Check for `isExpunged()` before logging a filtered message
* Add `@author`
* Fix debug message for expunged to reflect reality
* Clean up new unit tests a bit
**Cherry-pick to `5.4.x`**
Fixes https://github.com/spring-projects/spring-integration/issues/3560
According RFC 5322 `FROM` and `REPLY-TO` received mail message can be
as an array of addresses.
* Fix `MailUtils` to present those arrays as comma-delimited strings for
Spring message headers
* Fix tests to deal already with several addresses for `FROM` mime header.
In the future we may change the logic to map those mime headers to arrays
as it states according the mentioned RFC
**Cherry-pick to `5.4.x` & `5.3.x`**
`IMAPFolder.idle()` by default keeps idle-ing after each response.
We don't need that, because we want to fetch the new mails immediately (over the same connection).
To make `idle()` not keep on going, we called
`folder.isOpen()` which in most cases makes `idle()` stop by calling `noop()`
internally (to keep the connection alive) which leads to the current
`idle()`-call being canceled.
The problem this commits addresses is that the call to `noop()` only
happens if there were no calls in the last second.
There is a check for that in `com.sun.mail.imap.IMAPFolder.keepConnectionAlive`.
So if a new message appears less then a second after idle started, we will miss it.
This new way interrupts `idle()` more often than before, e.g. when there
is an expunge-message (i.e. a message was deleted), which we don't care about.
But the subsequent `receive()` in IdleTask will simply get no
messages and then turn around and start idle-ing again.
**Cherry-pick to `5.3.x` & `master`**
Fixes https://github.com/spring-projects/spring-integration/issues/3492
The `simpleContent` option is missed on the `ImapIdleChannelAdapterSpec`
* Add `ImapIdleChannelAdapterSpec.simpleContent(boolean)` option with
delegation to its internal `ImapMailReceiver`
* Convert all the `mail` tests to JUnit 5
* GH-3356: Add the way to not load mail message
Fixes https://github.com/spring-projects/spring-integration/issues/3356
In some cases it is not necessary to have a whole mail message to be load
when it could be filtered out downstream
* Treat "no eager load" for `MimeMessage` in the `AbstractMailReceiver`
when no `headerMapper`, not `simpleContent` and no `autoCloseFolder`
* Fix language in Docs
Co-authored-by: Gary Russell <grussell@vmware.com>
Co-authored-by: Gary Russell <grussell@vmware.com>
* Upgrade to Spring Data Moore-SR6
* Fix `TransactionHandleMessageAdvice` and `TransactionInterceptorBuilder`
for deprecations in the `TransactionInterceptor`
* Fix `TransactionHandleMessageAdvice` and `TransactionInterceptorBuilder`
consumers to expose new `TransactionManager`-based options and
deprecate `PlatformTransactionManager`-based
* Fix failing JMS tests to reuse an ActiveMQ Connection Factory with a
`trustedPackaged(*)`
**Cherry-pick to master**
# Conflicts:
# build.gradle
# spring-integration-core/src/main/java/org/springframework/integration/dsl/ConsumerEndpointSpec.java
# spring-integration-core/src/main/java/org/springframework/integration/dsl/PollerSpec.java
# spring-integration-core/src/main/java/org/springframework/integration/transaction/TransactionHandleMessageAdvice.java
# spring-integration-core/src/main/java/org/springframework/integration/transaction/TransactionInterceptorBuilder.java
* GH-3155: Add support for Java DSL extensions
Fixes https://github.com/spring-projects/spring-integration/issues/3155
Provide an `IntegrationFlowExtension` for possible custom EI-operators
in the target project use-cases.
* * Move `IntegrationFlowExtension` tests ot its own test class
* Make all the `IntegrationComponentSpec` ctors as `protected` for possible custom extensions
* Make some `BaseIntegrationFlowDefinition` methods and properties as `protected` to get them
access from the `IntegrationFlowExtension` implementations
* Document the feature
* * Fix language and typos in docs
* * Add `protected` to one more `GatewayEndpointSpec` ctor
* Add JavaDocs to `GatewayEndpointSpec` methods
* * Add `protected` to one more `JmsPollableMessageChannelSpec` ctor
* Remove Boon dependency and its usage
* Remove overloaded methods from the `IntegrationFlowDefinition`
- we can simply rely now on the super class
* Remove (or rework) deprecated entities in the docs
* Fix tests for removed deprecated APIs
* Rework affected tests to JUnit 5
Fixes https://github.com/spring-projects/spring-integration/issues/3132
It turns out that Checkstyle EmptyBlock doesn't complain about
empty default ctor.
Plus a new check for `super();` call treats it as a violation
* Remove `super();` from all the no-arg ctors
* Code style clean up in the affected classes according
IDEA suggestions
* Fix new Sonar smells
* Move `What's New` into `changes-5.1-5.2.adoc`
* Remove version from the `XSD` files:
1. Align with SF
2. The version for XSD doesn't matter for the current jar version,
since only the version is available in classpath is from the current
jar version
3. Modify `spring.schemas` to map all the possible versions for XSD
into the current one in a jar.
This way target applications can upgrade without changing the version
for XSD location and we don't need to require `versionless` variant
any more
4. The jar in classpath can handle only its own XSD, so independently
of version or no-version variants we still may fail because the current
jar may not support end-user code any more - need some fix over there
anyway.
* Remove `checkTestConfigs` Gradle task since we don't worry about XSD
version any more
* Modify `schemaZip` Gradle task to populate XSDs into a distribution
still with a version to avoid overriding on-line XSDs for version `1.0.x`.
We may consider not do that since SF doesn't and just bite a bullet
for always overriding those on-line XSDs to the actual one from the latest
release.
* Improve ImapIdleChannelAdapter
* We should not destroy a `TaskExecutor` in the `stop()`, especially
when we are going to restart eventually.
Move that logic into `destroy()`
* we should not destroy `MailReceiver` in the `stop()`; we don't
reinstate it in the `start()`.
Move the logic into `destroy()`
* Wrap `ReceivingTask` and `IdleTask` into `isRunning()` condition to
avoid task executions when we are in stopped state
* Remove `ImapMailReceiverTests.testExecShutdown()` since it is not
relevant any more and doesn't reflect `mail` module requirements
* * Add `ImapMailReceiver.cancelPing()` hook
* Also close folder for each `stop()`, as well as in the `destroy()`
Fixes https://github.com/spring-projects/spring-integration/issues/2748
* Refactor more `MessageHandlingException`s to include `this` into an
exception message
* Revert using `MessagingException` in some places which really are not
about messaging.
This helps to wrap them into `MessageHandlingException` later in the
`MessageHandler` for the `BeanDefinition` reference
* Remove `volatile` from configuration properties in the affected
classes
* Remove already deprecated `JmsOutboundGateway.setPriority()`
* Add `resource` and `source` for `BeanDefinition` in the
`AbstractChannelAdapterParser` & `AbstractInboundGatewayParser`
* Document the feature
Fixes https://github.com/spring-projects/spring-integration/issues/2987
To resolve XSD files properly from the classpath, their HTTPS reference
must be present in the `spring.schemas` to avoid the Internet interaction
for resolving an XSD file
**Cherry-pick to 5.1.x, 5.0.x & 4.3.x**
* INT-4299: Add AbstractMailReceiver.autoCloseFolder
JIRA: https://jira.spring.io/browse/INT-4299
* Optimize `AbstractMessageSource` do not evaluate headers when polling
result is `null`
* Optimize `MailReceivingMessageSource` do not wrap polling result to
the message: the `AbstractMessageSource` will do that later
* * Fix JavaDocs
* Document `autoCloseFolder` option
* Do not fix urls for the `http://apache.org/xml/features` when we
configure XML components
* Do not fix urls for localhost and fake links in tests
* Fix all the dead links in docs
* Fix mail lock race condition & Sonar smells
https://build.spring.io/browse/INT-MASTERSPRING40-599/
* The `folderReadLock` might not be re-locked when `openFolder()`
throws an exception
* Fix all the Sonar smells for mail module
* Optimize all the dynamic `Assert` messages to `Supplier`
* Refactor `MailReceiverFactoryBean` to be based on the `AbstractFactoryBean`
* * Fix new reported Sonar smells
* * Log error during cancel idle state in the `ImapMailReceiver`
* Remove commented code in the `MailTransportUtils`
According Sonar we can't just leave lock unlocked in th method, so
apply refactoring for the `AbstractMailReceiver.receive()` where we don't
have uncontrolled `lock()` in the separate method
* * Fix new Sonar issue in the `AbstractMailReceiver`
* * Move `unlock()` to its own `finally`
* * Make `ImapMailReceiverTests.imapIdleServer` non-static to allow
recreate the mail server for each test method
https://build.spring.io/browse/INT-MASTERSPRING40-591/
* * Remove MongoDb download from Travis config
* * Fix Checkstyle violation
JIRA: https://jira.spring.io/browse/INT-2543
We may have a mail server host configured in the properties and the
target session will resolve it properly from provided `javaMailProperties`.
The same applies for `username`
* Do not require `host` and `username`, when `javaMailProperties` is
provided
* Add additional factory methods into the `Mail` factory for Java DSL
* Add Java DSL sample into the `mail.adoc`
* Enable ModifierOrderCheck Checkstyle rule
* Fix violations for `static` and `abstract` modifier
* Remove redundant code in the `TcpNioConnection`
* Mark `connectionFactoryName` as `@Nullable` in the `TcpConnectionSupport`
ctor and its inheritors
* Fix some smells according IDEA suggestions in the affected classes
* This should fix some Sonar smells as well
* * Fix `HeaderMapperTests`
* * Polishing `TcpConnection` code style and fix Javdocs
https://build.spring.io/browse/INT-MASTERSPRING40-553
The `ImapMailReceiverTests` fails sporadically according some race
condition or wrong logic.
* Fix `ImapIdleChannelAdapter` to check for folder not null before
performing logic in the `IdleTask`
* Polishing for logs which are based on constant strings
* Remove `volatile` from configuration properties in the
`ImapIdleChannelAdapter` and `AbstractMailReceiver`
* Refactor some smells into the `protected` getters instead of direct
access to the property
* Stop channel adapters in the `ImapMailReceiverTests`
* Also destroy task schedulers in the `ImapMailReceiverTests`
