JIRA: https://jira.spring.io/browse/INT-3893
Polishing
Send on the calling thread if the executor is shut down.
Conflicts:
spring-integration-ip/src/test/java/org/springframework/integration/ip/udp/UdpChannelAdapterTests.java
Resolved.
JIRA: https://jira.spring.io/browse/INT-3843
**Cherry-pick to the 3.0.x**
Conflicts:
spring-integration-ip/src/main/java/org/springframework/integration/ip/tcp/connection/TcpNioServerConnectionFactory.java
Resolved.
JIRA: https://jira.spring.io/browse/INT-3837
INT-3103 introduced exception propagation to waiting gateway threads.
However, `SocketTimeoutException`s were not propagated (in all cases
since 4.2 and for single-use sockets since 3.0).
Conflicts:
spring-integration-ip/src/test/java/org/springframework/integration/ip/tcp/TcpOutboundGatewayTests.java
Resolved.
Remove beanfactory from backported test.
Fixed hung assembler described in INT-3453.
Used CountDownLatch instead of sleep()
INT-3453 Add Test Case
Currently failing for me - assembler thread stuck on CDL.
Remove redundant `if (TcpNioConnection.this.writingLatch != null)` from `ChannelInputStream#write`
Conflicts:
spring-integration-ip/src/test/java/org/springframework/integration/ip/tcp/connection/TcpNioConnectionTests.java
JIRA: https://jira.spring.io/browse/INT-3433`
If a read fails due to insufficient threads, delay the read
for (default 100ms) - do not re-enable OP_READ until that
time has elapsed. Avoids spinning the CPU.
INT-3433 More Polishing
If the assembler couldn't execute a new assembler after assembling
the current message (when it detected there is more data), in the
finally block it would "continue" only if the socket was still
open.
The test case closes the socket after sending 4 messages so when
this condition occurred, the assembler failed to continue and
data was left in the buffer.
Remove the `isOpen()` check in the finally block and always continue
if there's not another assembler running and there's data available.
INT-3433 More Polishing
We can still get starvation if the selector is in a long
wait (in select()) when a read is delayed.
Whenever a read is delayed, wake the selector so its next
select will use the readDelay timeout.
INT-3433 Reference Docs
Also remove Thread.yield().
Revert redundant boolean return from `TcpNioConnection#checkForAssembler()`
Conflicts:
spring-integration-ip/src/main/java/org/springframework/integration/ip/tcp/connection/AbstractConnectionFactory.java
spring-integration-ip/src/main/java/org/springframework/integration/ip/tcp/connection/TcpNioConnection.java
JIRA: https://jira.spring.io/browse/INT-3410
When running a fixed thread pool with a bound queue,
and CALLER_RUNS execution rejection policy, it
was possible to deadlock the IO selector thread.
Add a `CompositeExecutor` to use different threads for
IO to those used for message assembly.
Add a `CallerBlocksPolicy` to block the invoking thread
(for a specified time) if the pool is exhausted.
Add documentation.
Polishing
ByteArraySingleTerminatorSerializer does not support multiple
subsequent terminators (zero length messages).
JIRA: https://jira.springsource.org/browse/INT-3290
* Add unit test to verify bug.
* Fix bug.
Polishing:
* Remove test for `n >= 0`; n cannot be < 0 here
* Move test case to `DeserializationTests`
JIRA: https://jira.springsource.org/browse/INT-3233
With Windows 7 and Java 7/8 closing a server channel does
not close the underlying socket. Closing the Selector does
close the socket.
It is not clear why this is needed, given that a selector can
be used for multiple sockets.
However, we only use the server side selector for a single
server socket so there is no detriment in closing it.
Also close the selector on the client side (even though it is
used for multiple sockets, because we are stopping the factory
anyway and all sockets will be closed).
However, closing the selector opens us up to 'ClosedSelectorException's
in several places. Add catch blocks to deal with this exception, and only
log an error if the factory is active.
While debugging this issue, I found that a number of (older) tests
left threads running, sockets open etc.
INT-3233 Polishing - PR Comments
Use `Arrays.asList()` instead of `Collections.arrayToList()` in
places where a generic type needs to be supplied by the caller.
This maintains compile-time compatibility with both Spring 3 and 4,
where Spring 4 added the generic type to the utility class.
The four occurrences where this was needed are not impacted
by the fact that `Arrays.asList()` returns an unmodifiable list.
JIRA: https://jira.springsource.org/browse/INT-3212
INT-3212 Fix Failing TCP Test
TcpNioConnections can deadlock when a fixed thread pool is being used.
The deadlock is detected after 60 seconds, but the test case failed
after 20 seconds - before the deadlock was detected.
Change the test to use a cached thread pool instead.
Also, increase concurrency by using threading in the test server
so each socket is handled on a separate thread.
Enchance connectionId to include both local and remote ports to
aid debugging.
Some minor formatting corrections.
Intercepted TCP connections did not publish events.
TcpReceivingChannelAdapterTests did not close server factories after
tests.
Suppress 'no publisher' log messages.
Previously, the TcpConnectionOpenEvent was published as soon
as the socket was connected, but the TcpConnection was not yet
ready for use.
Defer publishing the event until the connection is fully
initialized and registered with the factory and thus available
for use.
Update the test cases to reflect this behavior.
Since the event is now published by the appropriate concrete
factory implementations, enhance the ConnectionToConnectionTests
to verify proper eventing with both Net and NIO implementations.
INT-3178: Add `theConnection` `ReadWriteLock`
INT-3178 Polishing
- Refactor obtainConnection() into obtainSharedConnection() and obtainNewConnection() for Net and NIO Client factories.
- obtainConnection is now common for these two factories, and overridden by the caching and failover factories.
- ReadWriteLock - double check for shared connection after lock obtained.
INT-3178: Polishing
Previously, when an exception occurred on a send, the
connection was not forcibly closed. When using a
CachingClientConnectionFactory, this prevented the connection
(albeit stale) from being returned to the cache.
Perform a forced (physical) close whenever a send fails.
Add test cases for both Net and NIO implementations, using
a CCCF, to verify the connection is returned to the pool so
the closed state can be detected on the next retrieval, causing
a refresh.
Also, change the synchronization in the NIO send to
synchronize on the socketChannel, not the mapper (which is
shared).
JIRA: https://jira.springsource.org/browse/INT-3163
Add a `@Rule` to skip long-runing tests during normal builds.
Add the rule to long-running tests in gemfire, ip, jms, jmx.
Add an environment variable `RUN_LONG_INTEGRATION_TESTS`; when set
to true, all tests are run.
Set the environment variable to true on all nightly builds.
Build now runs in 13 minutes on my 3 year old laptop.
* Add <spel-property-accessors> to configure a list of beans
that implement `org.springframework.expression.PropertyAccessor`
* Add `SpelPropertyAccessorRegistrar` to manage a list of PropertyAccessors
for Integration infrastructure
* Refactoring for `SpelFunctionRegistrar` and `IntegrationEvaluationContextFactoryBean`
to fix the issue when there is no `SpelFunctionRegistrar`(`SpelPropertyAccessorRegistrar`)
in the child AC, but there is one in the parent.
Previously the inheritance did't work for that reason.
* Now parent/child logic moved to `IntegrationEvaluationContextFactoryBean`
* Add documentation
JIRA: https://jira.springsource.org/browse/INT-3133
INT-3133: Rebasing, polishing and documentation
INT-3133: PropertyAccessors override support
* `@Ignore` `SOLingerTests#finReceivedNioLinger()` test
INT-3133: Change JavaDoc link to 3.0.0.RC1
Polishing
- Remove duplicate check - last bean def wins
- Doc Polishing
https://jira.springsource.org/browse/INT-3146
Incompatibility of using socket timeouts with a caching client
connection factory.
When a socket option timeout (soTimeout) is set on a TCP connection
and the timeout occurs, the socket is closed.
When a connection is intercepted, the close is performed through
the interceptor (for example to allow a closing handshake before the
physical close).
However, the caching client connection factory is implemented
using an interceptor, which returns the underlying connection to
the cache pool (for reuse). In the case of a TcpNetConnection,
the reader thread has terminated meaning that, the next time the connection
is used, no reply will ever be received.
The work-around (when using a gateway) is to use the 'remote-timeout'
attribute instead of relying on the soTimeout. There is no
work around when using collaborating channel adapters with a Net
connection. Using NIO works because there is no reader thread
in that case, but the socket is never closed on a timeout.
Always physically close the connection whenever an exception occurs
even if the close was delegated to an interceptor.
Add test cases for Net and NIO implementations.
* Add XSD attributeGroup `smartLifeCycleAttributeGroup`
* Use it for all adapters tags
* Refactoring `AbstractChannelAdapterParser` hierarchy
* Refactoring some `MessageHandler`'s that should not have `autoStartup` & `phase`.
It is the responsibility of `AbstrctEndpoint`
* Polishing some tests according new logic
JIRA: https://jira.springsource.org/browse/INT-2998
INT-2998: Polishing
Polishing
- Reformat schemas where indentation changed
- Add SLC to the UDP inbound adapter
Polishing tests to follow with new changes in the SPR 4.0
Polishing build.gradle to be compatible with SPR 4.0
Build passes with SPR 3.2.4 too
JIRA: https://jira.springsource.org/browse/INT-3131
The second chance NIO logic that detects a close being received
in a race condition with a reply needs to be enabled always because
a Failover or Caching connection factory might be using NIO
underneath.
Also, add an epoch to the FailoverTcpConnection's connectionId which
is incremented on each use to prevent a close after a previous
use being received by the current user.
Finally, add code in the FailoverTcpConnection to not propagate
messages from an old (defunct) delegate connection - if the
delegate has changed, the connection must be being used by
another client.
Previously, when a calling thread was waiting for a reply, and an exception
occurred on the socket, the exception was not propagated to the thread and
it would eventually get a timeout, but with no indication of the problem.
Propagate the exception to the calling thread by invoking onMessage()
with an ErrorMessage.
Ignore the ErrorMessage in other TcpListeners (inbound adapter, gateway).
Ensure NIO closes are not missed by sending an ErrorMessage when the selector
thread detects a closed channel.
Add tests for Net, NIO, cached and failover connection factories.
Polishing
Remove left-over debug logs; add comment to setReply().
Add comments for ErrorMessages in onMessage().
Support nesting the fail over and caching client connection
factories.
The TcpListener chain was not set up properly; this prevented
caching Failover connections, or Failover cached connections.
Each nested interceptor needs the next outer interceptor set
up as its listener, so the underlying connection calls
onMessage() all the way up to the ultimate listener (adapter
or gateway).
Fix the Listener hierarchy; add tests for caching failover
connections and failing over cached connections.
Don't overrwrite the actual connection id if it has already
been set up by a lower level wrapper.
The SimplePool maintains an 'allocated' set, for the sole reason
of preventing a "foreign" (non-managed) object being returned.
When a pool item is detected as stale, it is removed from the pool
but remains in the 'allocated' set.
Add a test to verify the allocated size is reduced when a stale
item is popped from the pool.
Add a FileTransferringMessageHandler test (where the problem was
discovered).
Fix a test in TCP to expect a close().
Polishing
Do not allow returning null items to just release a permit - it
cannot remove the item from allocated. Clients must return the
stale item to the pool so it can be refreshed on the next get.
Only used by the TCP caching CF when returning a connection when
the factory is not running; but should not be allowed.
Also, protect against double release - not currently an issue with
existing users of SimplePool, but should be protected against. Could
cause the permit count to exceed the pool size.
Add inUse Set to the pool so we can detect attempts to release an
item that has already been released.
A TCP Connection is closed when an outbound gateway times out.
However, with the CachingClientConnectionFactory, the connection
was returned to the pool instead of being physically closed.
Override forceClose() and physically close the connection so the
next time it is retrieved from the pool it is detected as stale
and re-established.
Refactor test case and invoke it twice; once with a regular connection
factory and once with a cached factory.
Remove an unused logger.
TCP streams have no standard message structure. Therefore, the
TCP implementation previously only transferred the message
payload.
If someone wanted to convey header information, they would have
to write their own wrapper and/or use Java serialization for
the entire message.
This change provides a strategy to allow users to determine
which headers are transferred, and how.
A MessageConvertingMessageMapper is now provided that invokes
any MessageConverter. A MapMessageConverter is provided that
converts the payload, and selected heades to a Map with two
entries ("payload") and ("headers").
A MapJsonSerializer is provided that converts a Map to/from
JSON. Jackson can't delimit multiple objects in a stream
so another serializer is required to encode/decode structure.
A ByteArrayLfSerializer is used by default, inserting a
linefeed between JSON objects.
The combination of these elements now allows header
information to be transferred over TCP. Of course, users
can implment their own (de)serializer to format the
bits on the wire exactly as needed by their application.
INT-1807 Polishing
Add a test that uses a Map MessageConverter with a
Java (de)serializer.
INT-1807: Polishing
INT-1807: Rebased and polished
Change `MapJsonSerializer` to use `JsonObjectMapper` abstraction
Doc Polishing
Allow an application to be informed of problems on the IMAP idle
thread by emitting an event containing the exception.
Introduce IntegrationApplicationEvent hierarchy for all
events emitted by SI components.
INT-3099 Polishing (PR Comments)
Separate TCP events into discrete subclasses.
* Introduce XSD 'id' attribute for all `<chain>` sub-components
* Register `<chain>` sub-components as Beans with names based on `<chain>` 'id'
* Generate aliases for sub-components based on element 'id'
* Add `<chain>` parser test to check sub-components as Beans within `BeanFactory`
* Ignore `NestedChainParserTests` as abnormal and confused for SI-configs
* Now `<chain>` sub-components are trackable by `MessageHistory`
* Now `<chain>` sub-components are eligible for JMX export
* Polishing some failed tests
https://jira.springsource.org/browse/INT-2755https://jira.springsource.org/browse/INT-2321
INT-2755: Register as Beans only if 'id' provided
INT-2755: Set 'componentName' for handlers with 'id'
When components within `<chain>` are configured with an 'id' attribute
the 'componentName' of the handler is constructed as:
chainHandler.componentName + '.' + handler.componentName.
Otherwise it is as was before.
INT-2755: Polishing
INT-2755: changes according PR's comments
* Add INFO about preference of `id` attribute for `<chain>` in the `ChainParser`
* Change `<chain>`'s sub-components `id` generation algorithm
* Configure `<chain>`'s sub-component `componentName` in the `ChainParser` instead of in the `MessageHandlerChain`
* Add `componentName` property to the `JpaOutboundGatewayFactoryBean`
* Add a note to the Reference Manual
INT-2755: Localize duplicate 'id' within chain
INT-2755: remove child beans aliasing
* Polishing tests
* Polishing docs
INT-2755 Polishing: Handle Nested Chain Bean Names
Previously named beans within a nested chain did not get unique
bean names.
INT-2755: Polishing nested chains
Doc Polishing
The TcpNioConnection uses an internal InputStream.
The standard deserializers only use the read() method.
Custom Deserializers might use read(byte[]); the
internal InputStream did not override these methods, possibly
causing the deserializer to hang awaiting data, when none
was coming.
Override these methods, with the appropriate semantics
from the super class...
- Block until at least one byte arrives.
- Return early if no data is available after at least one byte arrives.
- Return -1 if closed when no data arrived.
- Return the number of bytes actually read.
Polishing - PR Comments remove read(byte[]) because it doesn't do
anything different to the superclass method.
Use latches and the new CLOSE event instead of arbitrary waits for
sockets to close.
Factor out common test code into setup methods.
removed system.out
Sporadic test failures (fairly consistently on Mac).
Problem was some tests sent a message to a client socket before the
close notification had been received.
Wait for client connection to close rather than waiting for the
server to stop.
Also, with NIO, close the channel on accept exception.
Also improve debugging by including a log of new connections
including the id.
INT-2964 - JavaDoc: <emphasis> should be <em>
INT-2964 - JavaDoc: <p/> should be <p>
INT-2964 - JavaDoc polishing
* <li></li> should be wrapped in <ul></ul>
* wrap code snippets in {@code myCode()}
* change <code>false</false> to <code>false</code>
INT-2964 - Polish - Fix more JavaDoc errors
* Mockito-all dependency causes JavaDoc error. Change dependency to Mockito-Core.
INT-2932
- Reference Documentation
INT-2941
- Add getter for Throwable
- Add auto-startup to Parser, Endpoint (already in schema)
- Add phase to schema, Parser
(Schema changes are mostly indentation because the element now
extends SmartLifecycleType).
Inadvertent sign extension on binary data with bit 7 set
causes early termination of binary deserializers.
The ChannelInputStream (which replaced the piped input
and output streams) failed to mask off the top 24 bits
of "normal" bytes received, causing Deserializers to
believe the stream was closed.
Add a mask of 0xff to bytes read.
Add a test case.
Assertion that server side connection was established could
occur after the connection was established, but before it was
added to the collection.
Add a latch.
Reference: https://jira.springsource.org/browse/INT-2126
* Add TcpConnectionEvent for OPEN, CLOSE, EXCEPTION.
* Add Event Type Enum - Provides type safety for event types.
* Add <int-ip:tcp-connection-event-inbound-channel-adapter />
* Add the ability to filter events (restrict to one or more subclasses)
* Add publishEvent() to TcpConnectionSupport to permit, for example, connection interceptors to publish events via the connection, but the event source can only be the connection used to publish the event.
* Add documentation to Reference Documentation section *What's New*
INT-2871 Provide Mechanism to Find TCP Connections
Reference: https://jira.springsource.org/browse/INT-2871
* Add getOpenConnectionIds() to Abstract Connection Factory
* Add closeConnection(String connectionID)
* Introduce removeClosedConnectionsAndReturnOpenConnectionIds
INT-2877 TCP Extension Improvements
Reference: https://jira.springsource.org/browse/INT-2877
* Make improvements to make extension easier.
* Add setCustomHeaders() to mapper.
* Add mapper to connection factory namespace parser.
* Add plumbing for stateful Deserializers.
* Refactor setting standard headers
* rename setCustomHeaders to supplyCustomHeaders - returning a Map and not exposing the MessageBuilder to subclasses
INT-2872 Remove Deprecated Items in ip Module
Reference: https://jira.springsource.org/browse/INT-2872
* pool-size Attribute
* setScheduler
* TcpSendingMessageHandler.getConnection()
* Deprecated attribute on TCP Connection Factory.
TcpNetConnection IO errors were only logged under DEBUG.
* Code was added to suppress the error log when the exception was due to a normal close. This inadvertently suppressed all ERROR logs.
* Capture a local copy of this.noReadErrorOnClose before closing the socket after an exception.
* Add test case to ensure ERROR log is emitted.
* MessageGroupStore <-> MessageGroupCallback
- Change callback to be an inner interface of the MGS.
* INT-2829 Fix Class Tangle in IP Module
- 3 way tangle between TcpSender, TcpConnection, TcpMessageMapper
- Clean up interfaces (remove setters); add top-level support classes for TcpConnection, TcpConnectionInterceptor.
* INT-2829 Remove TCP Package Tangle
- tcp.connection <-> tcp.connection.support
- Remove ...connection.support package - move classes to ...connection.
