Fixes: #9633
Issue link: https://github.com/spring-projects/spring-integration/issues/9633
The message flags of the received messages are outdated and don't contain the changes of `AbstractMailReceiver#setMessageFlags`,
because the flags are only applied to the original message and not the copy (`AbstractMailReceiver.IntegrationMimeMessage`).
* Call `setMessageFlagsAndMaybeDeleteMessages()` for `IntegrationMimeMessage` copies as well
(cherry picked from commit 1d85d6b43c)
Fixes: #9546
Issue link: https://github.com/spring-projects/spring-integration/pull/9546
Sometimes the IMAP connection breaks just after the message flags have been set, but the message has not been copied yet.
This then leads to the message never being received by (as it has been flagged and the next search will not return it).
* Flag and maybe delete messages after messages have been copied
**Auto-cherry-pick to `6.2.x`**
# Conflicts:
# spring-integration-mail/src/main/java/org/springframework/integration/mail/AbstractMailReceiver.java
* Fix ImapIdleChA for scheduling race condition
The IMAP IDLE is long-lived process and can be blocked
waiting for any reply from the server.
This way it is not suited to be used in a `TaskScheduler`
especially when it has only one thread in its pool in Spring Boot by default.
Another concurrent scheduled task is exactly an `ImapMailReceiver.IdleCanceller`.
With a single thread in a `TaskScheduler` pool it cannot be reached
therefore we never cancel and IDLE task and cannot react to the connection loss properly
* Rework the `ImapIdleChannelAdapter` logic to use a regular `Executor` and `while()` loop
with a `Thread.sleep()` when we lose connection
* Clean up the `ImapMailReceiverTests` from `TaskScheduler` not used anymore.
* Expose new `taskExecutor` option in the `ImapIdleChannelAdapterSpec` for Java DSL
* Enable `ImapMailReceiverTests.testIdleWithMessageMapping()` with an attempt to see
if this fix covers an unclear problem exposed before
* * The `testIdleWithMessageMapping()` still fails on GH actions
* GH-8577: Revise `ImapIdleChannelAdapter` logic
Fixes https://github.com/spring-projects/spring-integration/issues/8577
When we process mail messages in async manner, it is possible that we end up
in a race condition situation where the next idle cycle closes the folder.
It is possible to reopen the folder, but feels better to block the current idle
cycle until we are done with the message and therefore keep folder opened.
* Deprecate `ImapIdleChannelAdapter.sendingTaskExecutor` in favor of an `ExecutorChannel`
as an output for this channel adapter or similar async hand-off downstream.
* Make use of `shouldReconnectAutomatically` as it is advertised for this channel adapter
* Optimize the proxy creation for message sending task
* * Remove `ImapIdleChannelAdapter.sendingTaskExecutor`
* Fix language in docs
Co-authored-by: Gary Russell <grussell@vmware.com>
---------
Co-authored-by: Gary Russell <grussell@vmware.com>
Looks like sometime Spring Framework XML parser can properly determine the target factory method to chose,
but sometimes it does that in a wrong order.
* Add `type="java.lang.Class"` to the ctor args everywhere for `class="org.mockito.Mockito" factory-method="mock"`
`<bean>` definitions
Fixes https://github.com/spring-projects/spring-integration/issues/3686
* Add `src/idea` with respective editor config for IntelliJ IDEA.
Must be imported into an IDE
* Add `src/eclipse` with respective editor config for Eclipse/STS
* Reformat imports in source code according a new editor config
* Fix mail module to fit to the latest mail API requirements
* Fix the last Sonar smell in the `SmbShare`
* Remove redundant dependency for `hamcrest-core` in various modules
Fixes https://github.com/spring-projects/spring-integration/issues/3623
* `IntegrationFlow` refactoring
* Apply several code style improvements and good practices
* Code style: no empty lines for methods javadocs
* make deprecated implementation reuse actual one instead of the copy-paste approach
* add whats-new comments
* Fix whats-new page according to standards
* Upgrade Spring dependencies to the latest SNAPSHOTs
* Fix tests to verify against stack traces: the message
of the `NestedRuntimeException` does not include the nested exception information.
Related to https://github.com/spring-projects/spring-framework/issues/25162
* Fix `JdbcMessageStore` and `DefaultLockRepository` to rely on the `DataIntegrityViolationException`
instead of only its `DuplicateKeyException` extension.
This is the current behavior of the SQL errors translation
* Disable `WebFluxDslTests.testValidation()` - doesn't subscribe to the reply somehow...
* Refine `SimplePool.PoolSemaphore.reducePermits()`
* Upgrade to Java 17, SF-6.0, Gradle 7.2
* Upgrade to Jakarta dependencies and respective namespaces
* Fix some tests for Java 17 compatibility
* Fix wrong Javadocs
* Add some missed Javadocs
* Fix more `jakarta` namespace
* Fix WS & XML modules to use Jakarta EE
* `--add-opens` in some modules for their reflection-based tests
* Disable Kafka tests which does not work on Windows; see Apache Kafka `3.0.1`
* Upgrade to JUnit `5.8.1`
* Migrate JMS tests to Artemis
* Remove RMI module as it was deprecated before
* Fix `pr-build-workflow.yml` for Java 17
* Fix JavaDocs warnings using `Xdoclint:syntax` per module, not in the top-level `api` task
* Move docs for version `6.0`
If application is connected to a Domino mail server via IMAP, it can happen from time to time that a message is expunged.
This leads to a `MessageRemovedException` when calling `IMAPMessage#getSubject`.
And although debug is set to false this again leads to a `MessageException` and the whole integration flow stops.
* Check for `isExpunged()` before logging a filtered message
* Add `@author`
* Fix debug message for expunged to reflect reality
* Clean up new unit tests a bit
**Cherry-pick to `5.4.x`**
Fixes https://github.com/spring-projects/spring-integration/issues/3451
The GreenMail provides a comprehensive support for e-mail scenarios
testing.
* Use GreenMail instead our own `TestMailServer` for better and broader
support
* Convert SmtpTests and Pop3Tests to GreenMail
* Document that Pop3MailReceiver does not support autoCloseFolder=false
* Pop3MailReceiver uses `folder.getMessageCount()` to determine if there
are new messages.
This count only gets updated when opening the folder.
* Use GreenMail for remaining mail tests
* convert MailTests and ImapMailReceiverTests to GreenMail
* remove duplicate mail dependencies
* route com.sun.mail logging to log4j (via jul and slf4j)
Fixes https://github.com/spring-projects/spring-integration/issues/3560
According RFC 5322 `FROM` and `REPLY-TO` received mail message can be
as an array of addresses.
* Fix `MailUtils` to present those arrays as comma-delimited strings for
Spring message headers
* Fix tests to deal already with several addresses for `FROM` mime header.
In the future we may change the logic to map those mime headers to arrays
as it states according the mentioned RFC
**Cherry-pick to `5.4.x` & `5.3.x`**
This forces the race condition that was fixed in #3535 consistently.
Before the `IdleCanceler` was used to cancel the `idle()` call in addition to the normal flow.
That should not be necessary any more.
`IMAPFolder.idle()` by default keeps idle-ing after each response.
We don't need that, because we want to fetch the new mails immediately (over the same connection).
To make `idle()` not keep on going, we called
`folder.isOpen()` which in most cases makes `idle()` stop by calling `noop()`
internally (to keep the connection alive) which leads to the current
`idle()`-call being canceled.
The problem this commits addresses is that the call to `noop()` only
happens if there were no calls in the last second.
There is a check for that in `com.sun.mail.imap.IMAPFolder.keepConnectionAlive`.
So if a new message appears less then a second after idle started, we will miss it.
This new way interrupts `idle()` more often than before, e.g. when there
is an expunge-message (i.e. a message was deleted), which we don't care about.
But the subsequent `receive()` in IdleTask will simply get no
messages and then turn around and start idle-ing again.
**Cherry-pick to `5.3.x` & `master`**
Fixes https://github.com/spring-projects/spring-integration/issues/3492
The `simpleContent` option is missed on the `ImapIdleChannelAdapterSpec`
* Add `ImapIdleChannelAdapterSpec.simpleContent(boolean)` option with
delegation to its internal `ImapMailReceiver`
* Convert all the `mail` tests to JUnit 5
* GH-3356: Add the way to not load mail message
Fixes https://github.com/spring-projects/spring-integration/issues/3356
In some cases it is not necessary to have a whole mail message to be load
when it could be filtered out downstream
* Treat "no eager load" for `MimeMessage` in the `AbstractMailReceiver`
when no `headerMapper`, not `simpleContent` and no `autoCloseFolder`
* Fix language in Docs
Co-authored-by: Gary Russell <grussell@vmware.com>
Co-authored-by: Gary Russell <grussell@vmware.com>
* Improve ImapIdleChannelAdapter
* We should not destroy a `TaskExecutor` in the `stop()`, especially
when we are going to restart eventually.
Move that logic into `destroy()`
* we should not destroy `MailReceiver` in the `stop()`; we don't
reinstate it in the `start()`.
Move the logic into `destroy()`
* Wrap `ReceivingTask` and `IdleTask` into `isRunning()` condition to
avoid task executions when we are in stopped state
* Remove `ImapMailReceiverTests.testExecShutdown()` since it is not
relevant any more and doesn't reflect `mail` module requirements
* * Add `ImapMailReceiver.cancelPing()` hook
* Also close folder for each `stop()`, as well as in the `destroy()`
* Add `io.spring.nohttp` Gradle plugin into the build script and allow
it only on CI server, since it is going to parse every file
* Fix tests do not use `http` any more
* Fix `test.html` in the `mail` modules tests to exclude a Namespace for
OExchange XRD protocol
* Fix some files for new line in the end of file according Checkstyle
violations
https://build.spring.io/browse/INT-MASTERSPRING40-681/
* Create an `ImapServer` only in the `@Before` method.
Looks like the `final` property is initialized only once for the whole
test class
* Change `LongRunningIntegrationTest` to the `@ClassRule`
* Fix `testMessageHistory()` and re-enable it
* Remove redundant `this.imapIdleServer.resetServer()` since we now have
a fresh `ImapServer` instance for every test method.
* INT-4299: Add AbstractMailReceiver.autoCloseFolder
JIRA: https://jira.spring.io/browse/INT-4299
* Optimize `AbstractMessageSource` do not evaluate headers when polling
result is `null`
* Optimize `MailReceivingMessageSource` do not wrap polling result to
the message: the `AbstractMessageSource` will do that later
* * Fix JavaDocs
* Document `autoCloseFolder` option
* Do not fix urls for the `http://apache.org/xml/features` when we
configure XML components
* Do not fix urls for localhost and fake links in tests
* Fix all the dead links in docs
Mostly thanks to IDEA's plugin: https://plugins.jetbrains.com/plugin/10345-assertions2assertj
There is still a lot of work to do when complex and composite matchers are used.
* Add `awaitility` dependency and deprecate `EventuallyMatcher` in favor
of `awaitility`
* Remove Hamcrest from dependencies and disable JUnit & Hamcrest
static imports to encourage to use only AssertJ
* Migrate JUnit assumptions in rules to AssertJ's assumptions
* Deprecate some custom matchers in favor of existing in Hamcrest
after upgrading the last to version `2.1`
* Replace `ExpectedException` rules with `assertThatThrownBy()`
* Mention `MessagePredicate` in the `testing.adoc`
