JIRA: https://jira.spring.io/browse/INT-3894
Allow UDP server port (and sender Ack port) to be selected by the operating system.
Also fix multicast tests to run on all platforms.
Change tests to use OS selected port.
* `ClientWebSocketContainer`: add some synchronization to avoid race conditions: https://build.spring.io/browse/INT-B41-492
* `TomcatWebSocketTestServer`: convert to `0` port to rely on the OS resolution for `localPort`
* Add `LogAdjustingTestSupport` for STOMP test
* `SftpServerTests`: use `0` port to rely on the OS resolution for `localPort`
* `ImapMailReceiver`, `OutboundGatewayFunctionTests` (JMS), `CachingClientConnectionFactoryTests`,
`AsyncGatewayTests`, `AsyncMessagingTemplateTests`, `GatewayParserTests`, `PriorityChannelTests`, `AggregatorIntegrationTests`: increase timeout
* `EnableIntegrationTests`: use `LogAdjustingTestSupport`
* `FileOutboundChannelAdapterParserTests`: rework `Thread.sleep()` with `CountDownLatch`
* `ConnectionToConnectionTests`: increase timeout and count iteration. Previously with `1sec` we may lose some events. And we can't just rely on the `10sec`,
because the last iteration will be so long
* `TcpOutboundGatewayTests`: `500ms` is so big timeout to wait for the `Exception` that in the high load environment we can yield to other Thread so long.
Like in our case to `server` Thread to send the reply for us. Therefore decrease the Exception timeout to the `50ms` and increase server delay to `2sec`
* `StompInboundChannelAdapterWebSocketIntegrationTests`: remove `@Qualifier("taskScheduler")` as a potential candidate to test against latest SF changes.
We're fine with `SF-4.2.2` and it is just a test-case. So, I don't see reason to wait for their fix here.
STOMP: `session = null` in adapters for any transportError
Polishing
JIRA: https://jira.spring.io/browse/INT-3886
An SO user reported a short message delivery when an NIO connection was timed out.
See JIRA for link.
We could not produce the problem with his deserializer but it did identify a problem
with the standard `ByteArrayRawSerializer`.
With NIO, socket timeouts were reported to the deserializer as a normal EOF (-1). This
caused the raw serializer to emit a short message - it's signal for end of message is the
socket closure.
We can't treat a timeout as a normal EOF.
If the socket is forcibly timed out due to no recent data, throw a `SocketTimeoutException` to
the deserializer.
Just in case the old behavior is being relied upon, a boolean has been added to restore that
behavior. This is not recommended, as using timeout to delimit messages is not reliable.
* Fix typos
* Increase `MongoDbInboundChannelAdapterIntegrationTests` timeouts
* Some code polishing
https://build.spring.io/browse/INT-MJATS41-434
BindException.
Avoid using `findAvailableServerSocket()` where easily possible.
Some still remain, but they would require major rework of the tests.
Also fix a race in `TcpOutboundGatewayTests.testGoodNetGWTimeoutGuts()`. Although an
`AtomicBoolean` was used, both threads might still see the lower (500ms) timeout.
Polishing
* `JmsOutboundGatewayTests`: just increase timeouts. Looks like embedded ActiveMQ Broker takes more time to interact on high-loaded builds
* `TcpOutboundGatewayTests`: Rework the race condition fix to the atomic `remoteTimeout` change using mocks
* Modify `ControlBusTests.testControlHeaderChannelReaper()` do not rely on the `timeout`.
Make artificial expiration for the entry in the `registry` to allow `reaper` to remove it according the test logic.
* Fix race condition around `gateway.setRemoteTimeout(5000);` in the `TcpOutboundGatewayTests`.
The problem was that the gateway is called concurrently and `setRemoteTimeout` might be changed before the `handleMessage()` even for the first attempt.
JIRA: https://jira.spring.io/browse/INT-3860
Similar to INT-3794, if some other process is bound to all adapters
$ lsof -i tcp -nP | grep 9000
java 6604 grussell 380u IPv6 0x407e749ec47fa337 0t0 TCP *:9000 (LISTEN)
The socket utils reports that the localhost is available for binding; however, the test then
binds to all adapters.
1. Where (easily) possible, change tests to use port=0 and let the OS choose.
2. For other tests, use high-numbered ports.
3. Change tests to explicitly bind to localhost (like the utility did).
4. Fix `tcpOutboundGatewayInsideChain` test to use the correct port.
Polishing - Fix Travis
Bind SOLingerTests to localhost.
Since we're using port 0, we have to wait until listening before calling getPort().
JIRA: https://jira.spring.io/browse/INT-3837
INT-3103 introduced exception propagation to waiting gateway threads.
However, `SocketTimeoutException`s were not propagated (in all cases
since 4.2 and for single-use sockets since 3.0).
JIRA: https://jira.spring.io/browse/INT-3801
NPE if the server is stopped before it fully started.
Also fix SOLinger tests.
Fix `ConnectionFactoryTests` for Java < 8 compatibility
JIRA: https://jira.spring.io/browse/INT-3800
If the server port is 0, return the actual port after the factory is started.
Also add more TCP Test Diagnostics.
Part 1 - eliminate references from support to other packages
Part 2 - handler to aggregator/router
- move BarrierMH to aggregator
- Fix ScatterGather references
Part 3 - move TrackableComponent
- to support.management
Part 4 - class tangles
- TcpConnectionServerExceptionEvent
- RecipientListRouterManagement
INT-3799: Polishing
Race condition in `ServerSocket`; the `accept` thread is exiting due to the
close, but it still accepted a new message.
All tests need to wait until the server has stopped listening (as well as
waiting for the close to be propagated to the client).
JIRA: https://jira.spring.io/browse/INT-3795
Previously, a `CachingClientConnectionFactory` could wrap `FailoverClientConnectionFactory` instances.
It was not possible to wrap `CachingClientConnectionFactory` instances in a `FailoverClientConnectionFactory`.
The failover logic only failed over when an `IOException` occurred, whereas the CCCF wrapped the IOException
in a `MessagingException`.
- Catch all exceptions in the failover logic
- Add real TCP test cases for both scenarios
- test failover when a connection is closed
- test failover when the first factory cannot connect at all
INT-3795: Polishing - Fix Test Race
Revert `.travis.yml` changes
JIRA: https://jira.spring.io/browse/INT-3425
Deprecate `TcpConnectionEventListeningMessageProducer` in favor
of using a generic event inbound channel adapter.
Polishing
JIRA: https://jira.spring.io/browse/INT-3654
Previously, single-use `TcpConnections` self-closed when their use was complete.
This is unnatural and caused issues such as INT-3722.
Remove the self-closing behavior; connection close is now (properly) the responsibility
of the client using the connection:
- Client Side:
-- `TcpOutboundGateway` after the reply is received
-- `TcpSendingMessageHandler` after the send, when there is no collaborating inbound adapter
-- `TcpReceivingChannelAdapter` when it is collaborating (after receiving the reply)
- Server Side:
-- `TcpInboundGateway` after the reply is sent
-- `TcpReceivingChannelAdapter` after the receive, when there is no collaborating outbound adapter
-- `TcpSendingMessageHandler` when it is colllaborating (after sending the reply)
As before, the `CachingClientConnectionFactory` always sets single use on the target factory
to force it to create new connections on demand. It is always a single-use factory itself
so the clients return the connections to the pool (via `close()`).
__Needs a migration guide entry__
INT-3654: Fix Late Listener Registration
Timing failures in `TcpOutboundGatewayTests.testFailoverCached()` (null listener).
The `FailoverClientConnectionFactory` registers its listener with the actual
connections it retrieves from a delegate.
When nesting failover and cached connection factories, we need to propagate the
`enableManualListenerRegistration` to the delegate factories so the connection
will wait for its listener to be registered.
JIRA: https://jira.spring.io/browse/INT-3745
Emit events whenever a message is received that can't be correlated to
a connection (or request in the case of an outbound gateway).
Also, when sending messages, wrap the exception in a `MessagingException` so that
the `TcpConnectionExceptionEvent` provides access to the failed message.
INT-3745: Polishing - PR Comments
Fix JavaDocs errors
JIRA: https://jira.spring.io/browse/INT-3730
Expose the `SSLSession` on `TcpConnection` to support custom header
mapping of properties from the session.
JIRA: https://jira.spring.io/browse/INT-3728
The `CachingClientConnectionFactory` uses a temporary (rejecting) listener until
the connection is established and then replaces the listener in the connection.
There is a race condition in that if the server starts sending messages before the
listener is replaced, the message is rejected.
Add a mechanism to delay `onMessage` calls until the real listener has been
registered.
Also fix `onMessage` in the cached connection so an `ErrorMessage` is propagated correctly.
To reproduce: revert src/main; add `Thread.sleep(1000)` before `registerListener(tcpListener);`
in `CachedConnection` ctor and run the new test.
To introduce a similar timing hole with the new code, add the sleep before
`this.theConnection.registerListener(this);` in `TcpConnectionInterceptorSupport`.
Summary of changes:
`CachingClientConnectionFactory`
- register the underlying connection's listener in the ctor, utilizing the
`TcpConnectionInterceptorSupport.registerListener()` method.
- fix `ErrorMessage` propagation.
`AbstractClientConnectionFactory`
- propagate the `enableManualListenerRegistration` to connections.
`TcpConnectionSuport`
- implement delay when manual listener registration is enabled.
Add test case.
Fix Failover Tests
`FailoverClientConnectionFactory`
- propagate enable manual listener registration to underlying factories
Polishing
JIRA: https://jira.spring.io/browse/INT-3722
While the initial fix solved the reported problem, it caused a memory leak when
the gateway completed its work before the reader thread "closed" the connection,
releasing it to the pool. This left the connection in the deferred close state.
Handle the condition where the gateway signals it has completed its work
before the connection is released to the pool.
JIRA: https://jira.spring.io/browse/INT-3722
The TCP outbound gateway correlates replies based on the connection id.
When a `CachingClientConnectionFactory` is being used, the connection is
returned to the pool too early, and can be reused. It is possible that
the current thread then removes the "next" pending reply from the correlation map.
Add code to the `CachingClientConnectionFactory` so that the "self" close from the
connection (called after `onMessage`) is deferred and the actual close (return to cache)
is controlled by the gateway itself.
This mechanism will no longer be needed when INT-3654 is resolved (removal of the "self"
closing by connections). At that time, connection users (such as the gateway) will be
in complete control.
JIRA: https://jira.spring.io/browse/INT-3661
Previously there were a lot of noise from the `PostProcessorRegistrationDelegate$BeanPostProcessorChecker` for early access for beans.
That may produce some side-effects when some of `BeanFactoryPostProcessor`s won't adjust those beans.
The issue is based on two facts:
1. Loading beans from `BPP`, e.g. `IntegrationEvaluationContextAwareBeanPostProcessor` (or `ChannelSecurityInterceptorBeanPostProcessor` - https://jira.spring.io/browse/INT-3663)
2. Loading beans from `setBeanFactory()/setApplicationContext()` container methods
* Move all code from `setBeanFactory()` with access to the `BeanFactory` (e.g. `this.messageBuilderFactory = IntegrationUtils.getMessageBuilderFactory(this.beanFactory);`)
to some other lazy-load methods like `getMessageBuilderFactory()`
* Fix parser tests to remove `messageBuilderFactory` tests since there is no activity for target components to lazy-load them
* Polish some test according the new lazy-load logic
* Rework `IntegrationEvaluationContextAwareBeanPostProcessor` to the `SmartInitializingSingleton` and make it `Ordered`
* Populate `beanFactory` for the internal instance of `connectionFactory` in the `TcpSyslogReceivingChannelAdapter`
* Populate `beanFactory` for the internal `UnicastReceivingChannelAdapter` in the `UdpSyslogReceivingChannelAdapter`
* Add `log.info` that `UdpSyslogReceivingChannelAdapter` overrides `outputChannel` for the provided `UnicastReceivingChannelAdapter`
* Change the internal `MessageChannel` in the `UdpSyslogReceivingChannelAdapter` to the `FixedSubscriberChannel` for better performance
* Fix `AbstractExpressionEvaluator`
* Add JavaDocs for the `IntegrationEvaluationContextAware`
Fix `MongoDbMessageStoreClaimCheckIntegrationTests`
Addressing PR comments
JIRA: https://jira.spring.io/browse/INT-3635
Add an expression as an alternative to `remote-timeout`.
INT-3635: Tcp GW Docs
Also removed the relationship between reply-timeout and
remote-timeout (introduced in 2.2 to provide backward compatibility
after the incorrect use of the reply-timeout attribute).
* Add log for Thread interruption state to the `EnableIntegrationTests`: https://build.spring.io/browse/INT-B41-JOB1-233
JIRA: https://jira.spring.io/browse/INT-3589, https://jira.spring.io/browse/INT-3624
* SF - 4.2. Fix Breaking changes for the `ApplicationEventPublisher`
* AMQP 1.5. Without issues
* Reactor - 2.0. Fix for the new `Stream` foundation
* jsonPath - 1.2.0. Fix for new `Predicate` abstraction. Add 'fail-fast error' to the `IntegrationRegistrar`
* Sshd - 0.13.0. Fix for new `VirtualFileSystemFactory` usage
* Spring Data - Fowler
* And others without issues
* Get rid of `reactor.util.StringUtils` usage
INT-3589: Address PR comments
Polishing
JIRA: https://jira.spring.io/browse/INT-3580
* Uncomment `org.springframework.integration.ip` category for the IP tests `log4j.properties`
* Since all tests in the `StompIntegrationTests` uses the same application context and therefore the same `SubscriptionRegistry`
the `waitForSubscribe()` should wait exactly for that `subscription` in which it is interested in.
The previous fix wasn't enough
The HelloWorldInterceptor (server side) set the
negotiated flag after sending the handshake back.
The next message could be received before the flag
was set and caused the test to fail.
Synchronize the server-side handshaking code so that
we don't start processing the next message before
the negotiated flag is set.
Leave the diagnostic log settings in place for now.
JIRA: https://jira.spring.io/browse/INT-3578
Adding `readDelay` to XML namespace for the `TcpConnectionFactoryFactoryBean`
caused an `IllegalArgumentException` when configuring with Java instea of
XML.
Polishing
JIRA: https://jira.spring.io/browse/INT-275
In addition fix the `Lifecycle` issue in the `ServiceActivatorAnnotationPostProcessor`
Add Namespace support, Addition tests
and fix some typos in the XSD
INT-275: Fix failed tests
INT-275: Fix for `replyChannel` Header
Add `sync` reply test-case
INT-275: Make `ScatterGatherHandler` sync
Fix some `MessageHandler`s from `SmartLifecycle`
INT-275: Fix `ScatterGatherHandler.handleRequestMessage` logic
INT-275: Fix `ScatterGatherHandler` JMX proxying issues
* Make `AbstractCorrelatingMessageHandler.getMessageStore()` as `public`
* Move `ScatterGatherHandlerIntegrationTests` to the JMX module to be sure that `ScatterGatherHandler`
works well with `@EnableIntegrationMBeanExport`
* Add xml config sample how to use an internal gatherer's `MessageStore` in the `MessageGroupStoreReaper`
* Add `What's New` notice
JIRA: https://jira.spring.io/browse/INT-3507
TcpNoDelay (false) helps to buffer IOs but only after the
first write.
Use `BufferedOutputStreams` for TCP writes.
Use the socket `sendBufferSize` for the buffer size.
JIRA: https://jira.spring.io/browse/INT-3370
* Add `BoonJsonObjectMapper`
* Provide some tests
* Remove deprecations
* Polishing test according removed deprecations
* Change `JsonObjectMapper#populateJavaTypes` to get deal with `payload`, not its `class`,
since we can't (and Jackson's `TypeFactory`, too) determine generic type from `Collection`.
Use `iterators` instead to retrieve the type from the first item.
Conflicts:
build.gradle
spring-integration-amqp/src/test/java/org/springframework/integration/amqp/support/DefaultAmqpHeaderMapperTests.java
INT-3370: Polishing according PR comments
* Apply Gary's polishing to the Docs
* Fix `RecipientListRouter` JavaDoc warn
* Fix typo in test name for `UdpUnicastEndToEndTests`
* Fix `RedisQueueOutboundChannelAdapterTests` do not use Jackson 1.x
Final Polish
Fixed hung assembler described in INT-3453.
Used CountDownLatch instead of sleep()
INT-3453 Add Test Case
Currently failing for me - assembler thread stuck on CDL.
Remove redundant `if (TcpNioConnection.this.writingLatch != null)` from `ChannelInputStream#write`
JIRA: https://jira.spring.io/browse/INT-3433`
If a read fails due to insufficient threads, delay the read
for (default 100ms) - do not re-enable OP_READ until that
time has elapsed. Avoids spinning the CPU.
INT-3433 More Polishing
If the assembler couldn't execute a new assembler after assembling
the current message (when it detected there is more data), in the
finally block it would "continue" only if the socket was still
open.
The test case closes the socket after sending 4 messages so when
this condition occurred, the assembler failed to continue and
data was left in the buffer.
Remove the `isOpen()` check in the finally block and always continue
if there's not another assembler running and there's data available.
INT-3433 More Polishing
We can still get starvation if the selector is in a long
wait (in select()) when a read is delayed.
Whenever a read is delayed, wake the selector so its next
select will use the readDelay timeout.
INT-3433 Reference Docs
Also remove Thread.yield().
Revert redundant boolean return from `TcpNioConnection#checkForAssembler()`
