JIRA: https://jira.spring.io/browse/INT-4067
When `FileSplitter` is configured with `markers = true` and file is empty, an `iterator` for file throws `IOException: Stream closed`,
because we close the `buffer` just after the first `readLine()` attempt, but still return `true` from the first `hasNext()` call
where the `this.sof` and `this.eof` are `true` for markers.
Add logic to mark internal splitter `iterator` as `done` where we don't have content and still in `sof` state.
JIRA: https://jira.spring.io/browse/INT-4046
Since not all FTP servers provide proper `STAT` command implementation,
plus the `NLIST` doesn't work properly for directories cases, introduce the `FtpRemoteFileTemplate.ExistsMode`
to let:
* to perform `STAT` by default (previous) behavior;
* to switch to `NLIST` for `FtpRemoteFileTemplate` internal use;
* perform the full `NLIST` and `FTPClient.changeWorkingDirectory()` algorithm if needed.
* Improve (S)Ftp components to use proper `RemoteFileTemplate` for internal instantiation
* Introduce `FtpMessageHandler` to wrap `FtpRemoteFileTemplate` with the proper `NLIST` `ExistsMode`
* Cover `NLIST` switching from the `FtpOutboundChannelAdapterParser` and `FtpOutboundGatewayParser`
* Document the `FtpRemoteFileTemplate.ExistsMode`
* Fix typo in the recently introduced `RemoteFileOperations.getSession()` method name
* Add JavaDoc to `Session.exists()`
* Add `NLIST` support for the `FtpSession.exists()` to meet the API requirements
**Cherry-pick to 4.2.x and 4.1.x**
Addressing PR comments
Doc Polishing
Conflicts:
spring-integration-file/src/main/java/org/springframework/integration/file/remote/AbstractRemoteFileStreamingMessageSource.java
spring-integration-file/src/main/java/org/springframework/integration/file/remote/RemoteFileOperations.java
spring-integration-file/src/main/java/org/springframework/integration/file/remote/RemoteFileTemplate.java
spring-integration-ftp/src/main/java/org/springframework/integration/ftp/session/FtpSession.java
spring-integration-ftp/src/test/java/org/springframework/integration/ftp/config/FtpOutboundChannelAdapterParserTests.java
spring-integration-sftp/src/main/java/org/springframework/integration/sftp/gateway/SftpOutboundGateway.java
spring-integration-sftp/src/main/java/org/springframework/integration/sftp/outbound/SftpMessageHandler.java
spring-integration-sftp/src/main/java/org/springframework/integration/sftp/outbound/package-info.java
Conflicts:
spring-integration-ftp/src/main/java/org/springframework/integration/ftp/gateway/FtpOutboundGateway.java
spring-integration-sftp/src/main/java/org/springframework/integration/sftp/gateway/SftpOutboundGateway.java
The test isn't compatible with `4.2.x` and `4.1.x` and its purpose is demonstration.
Therefore only the `master` version is enough.
(cherry picked from commit f6d6fc9)
JIRA: https://jira.spring.io/browse/INT-4045
Handle the situation when the first element of a container type (or map)
is null; set the generic type of the container to `Object`.
Also, clarify how the compatibility between the transformer and Spring AMQP
message converter is achieved.
(cherry picked from commit 2650aef)
(cherry picked from commit d5e753c)
JIRA: https://jira.spring.io/browse/INT-4043
The `ExecutorChannel` overrides `onInit()` but fails to call the super
which is where the message converter for datatype conversion is set up.
Also, when Jackson is not on the class path and there are no converters in the
context, the default integration conversion service is not registered.
The `DefaultDatatypeChannelMessageConverter` overwites its default conversion
service with this bean, unconditionally - setting it to null in this case.
Check for a null conversion service before replacing the default.
* Polishing according PR comments
(cherry picked from commit dae1a01)
JIRA: https://jira.spring.io/browse/INT-3999
Since the scheduled tasks may live for a long time it can finish
with the `OutOfMemory` if we use the direct reference to big objects, like `Message<?>`.
* Fix `AbstractCorrelatingMessageHandler` to deal only with the `groupId`
from the `scheduleGroupToForceComplete()` when we `schedule` `Runnable` for the `forceRelease` logic.
* Fix `DelayHandler` to deal only with `messageId` in the `releaseMessageAfterDelay()`, when we
`schedule` `Runnable` for the `releaseMessageAfterDelay`.
* Since the logic hasn't been changed for those components, there is no any new test.
There is just enough to be sure that all existing tests are fine.
**Cherry-pick to 4.0.x, 4.1.x, 4.2.x**
Optimise the release task for the `SimpleMessageStore` case
Conflicts:
spring-integration-core/src/main/java/org/springframework/integration/aggregator/AbstractCorrelatingMessageHandler.java
JIRA: https://jira.spring.io/browse/INT-3950
Previously there was a mention of the `MessageGroupStore.expireMessageGroup(groupId)` which just doesn't existing
in the Framework and never has been there.
* Fix the documentation for the existing `MessageGroupStore.expireMessageGroups(timeout)`.
Although the mention there of `Control Bus` requires to have `@ManagedOperation` on the method.
* Add `@ManagedOperation` for the `MessageGroupStore.expireMessageGroups(timeout)` and confirm with the test-case: `AggregatorWithMessageStoreParserTests`
* Fix the same docs in the XSD for `<aggregator>`
* Fix other typos in the `aggregator.adoc` and `resequencer.adoc`
Conflicts:
spring-integration-core/src/main/java/org/springframework/integration/store/AbstractMessageGroupStore.java
Conflicts:
spring-integration-core/src/main/java/org/springframework/integration/store/AbstractMessageGroupStore.java
src/reference/asciidoc/aggregator.adoc
src/reference/asciidoc/resequencer.adoc
JIRA: https://jira.spring.io/browse/INT-3907
The IO-1.1.x is based on the Kryo-2.22 and can't be upgraded to 3.0.
Therefore we should downgrade.
* Change `PojoCodec` to use `StdInstantiatorStrategy` directly, because the `DefaultInstantiatorStrategy` logic is as a core code of `Kryo`.
* Introduce `org.springframework.integration.codec.kryo.pool` package and copy/paste `com.esotericsoftware.kryo.pool` classes,
since they have been introduced since Kryo-3.0.
* That copy/paste seemed to me the simplest fix, since the `KryoPool` logic is encapsulated in the `AbstractKryoCodec`
JIRA: https://jira.spring.io/browse/INT-3904
When we use the same expression several times, the SpEL engine cache an `accessor` after the first use.
The next evaluation just bypass `canRead()` and in case of JSON that mean that we don't check that the income has the field or not.
For this case the `read()` must return `TypedValue.NULL` instead of just `null`.
JIRA: https://jira.spring.io/browse/INT-3885
Possible dropped reply, causing timeout.
WARN org.springframework.integration.jms.JmsOutboundGateway#1.replyListener-1 jms.JmsOutboundGateway:1202
- Failed to consume reply with correlationId 164a49bf-c41d-4c0b-b012-55deecf001d1_2
java.lang.RuntimeException: No sender waiting for reply
- Reproduced by running the test in a loop
- Cleaned up test to aid debugging - capture a unique message at each stage
- Added additional debug logging to th gateway
Polishing
(cherry picked from commit f1bd6e3)
Message producers are started in phase `Integer.MAX_INT / 2`.
The test case has a stomp inbound adapter and an event producer.
Since they are both in the same phase, the event producer can miss the `StompReceiptEvent`.
Change the phase of the event producer to ensure it is started before other message producers.
(cherry picked from commit 1107bfa)
Examining the code revealed a possible (but improbable) NPE.
Checking the connection was disjoint from the publish; encapsulate the check within publish.
Synchronize the connectionLost method so it can't null the client while it is being checked.
Add `@SuppressWarnings("deprecation")` on the deprecated `connectIfNeeded()` usage.
(cherry picked from commit f5fa979)
JIRA: https://jira.spring.io/browse/INT-3859
The Java Mail `MessageCache.getMessageBySeqnum()` has the code:
````java
if (msgnum < 0) { // XXX - < 1 ?
if (logger.isLoggable(Level.FINE))
logger.fine("no message seqnum " + seqnum);
return null;
}
````
about which the `IMAPFolder.search` doesn't care:
````java
matchMsgs[i] = getMessageBySeqNumber(matches[i]);
````
therefore pops `null`s to the top for the `ImapMailReceiver`
* Fix `NPE` filtering the `Message[]` from `null` items
Note: its enough difficult to reproduce it because it isn't clear how we can end up with:
````java
if (seqnums[msgnum-1] > seqnum)
break; // message doesn't exist
````
in the `MessageCache`.
That's why there is no test-cases on the matter.
**Cherry-pick to 4.1.x**
Polishing
- Only create a new array if needed.
- Add a test case
JIRA: https://jira.spring.io/browse/INT-3853
Previously the placeholder definitions for the Messaging Annotation weren't be resolved
if we use `<context:property-placeholder>` instead of `@PropertySource`.
Fix `MessagingAnnotationPostProcessor` and its "kindergarten" to use
`beanFactory.resolveEmbeddedValue()` instead of `environment.resolvePlaceholders()`.
**Cherry-pick to 4.0.x**
JIRA: https://jira.spring.io/browse/INT-3848
When the `ConsumerEndpointFactoryBean` is created programmatically
the `beanName` property may be missed and the `catch` for the `NPE`
just hides an issue with the `DEBUG` log message.
Add check for the `null` on the `bean` and log the issue on ERROR level.
**Cherry-pick to the 4.1.x, 4.0.x and 3.0.x**
JIRA: https://jira.spring.io/browse/INT-3841
Previously the `isPubSub` was as `Boolean` object and `null` by default.
Convert it to the primitive to achieve the `false` logic by default as expected.
INT-3841: Fix New Test
New test channel remains as a listener on the connection factory.
JIRA: https://jira.spring.io/browse/INT-3840
* In addition: fix some typos in the `SftpInboundRemoteFileSystemSynchronizerTests`
**Cherry-pick to 4.1.x**
JIRA: https://jira.spring.io/browse/INT-3827
Provide a hook to enable removing a file from an `AcceptOnceFileListFilter`,
for example after a message processing failure.
Make the `CompositeFileListFilter` a `ReversibleFileListFilter` so it can
delegate to any of its composed filters that are reversible.
INT-3827: Polishing - PR Comments
(cherry picked from commit 0d721739e9)
Conflicts:
src/reference/asciidoc/ftp.adoc
src/reference/asciidoc/sftp.adoc
JIRA: https://jira.spring.io/browse/INT-3837
INT-3103 introduced exception propagation to waiting gateway threads.
However, `SocketTimeoutException`s were not propagated (in all cases
since 4.2 and for single-use sockets since 3.0).
Conflicts:
spring-integration-ip/src/test/java/org/springframework/integration/ip/tcp/TcpOutboundGatewayTests.java
Resolved.
JIRA: https://jira.spring.io/browse/INT-3801
NPE if the server is stopped before it fully started.
Also fix SOLinger tests.
Fix `ConnectionFactoryTests` for Java < 8 compatibility
Fix cherry-picking conflicts
