Previously, the TcpConnectionOpenEvent was published as soon
as the socket was connected, but the TcpConnection was not yet
ready for use.
Defer publishing the event until the connection is fully
initialized and registered with the factory and thus available
for use.
Update the test cases to reflect this behavior.
Since the event is now published by the appropriate concrete
factory implementations, enhance the ConnectionToConnectionTests
to verify proper eventing with both Net and NIO implementations.
INT-3178: Add `theConnection` `ReadWriteLock`
INT-3178 Polishing
- Refactor obtainConnection() into obtainSharedConnection() and obtainNewConnection() for Net and NIO Client factories.
- obtainConnection is now common for these two factories, and overridden by the caching and failover factories.
- ReadWriteLock - double check for shared connection after lock obtained.
INT-3178: Polishing
Previously, when an exception occurred on a send, the
connection was not forcibly closed. When using a
CachingClientConnectionFactory, this prevented the connection
(albeit stale) from being returned to the cache.
Perform a forced (physical) close whenever a send fails.
Add test cases for both Net and NIO implementations, using
a CCCF, to verify the connection is returned to the pool so
the closed state can be detected on the next retrieval, causing
a refresh.
Also, change the synchronization in the NIO send to
synchronize on the socketChannel, not the mapper (which is
shared).
JIRA: https://jira.springsource.org/browse/INT-3163
Add a `@Rule` to skip long-runing tests during normal builds.
Add the rule to long-running tests in gemfire, ip, jms, jmx.
Add an environment variable `RUN_LONG_INTEGRATION_TESTS`; when set
to true, all tests are run.
Set the environment variable to true on all nightly builds.
Build now runs in 13 minutes on my 3 year old laptop.
* Add <spel-property-accessors> to configure a list of beans
that implement `org.springframework.expression.PropertyAccessor`
* Add `SpelPropertyAccessorRegistrar` to manage a list of PropertyAccessors
for Integration infrastructure
* Refactoring for `SpelFunctionRegistrar` and `IntegrationEvaluationContextFactoryBean`
to fix the issue when there is no `SpelFunctionRegistrar`(`SpelPropertyAccessorRegistrar`)
in the child AC, but there is one in the parent.
Previously the inheritance did't work for that reason.
* Now parent/child logic moved to `IntegrationEvaluationContextFactoryBean`
* Add documentation
JIRA: https://jira.springsource.org/browse/INT-3133
INT-3133: Rebasing, polishing and documentation
INT-3133: PropertyAccessors override support
* `@Ignore` `SOLingerTests#finReceivedNioLinger()` test
INT-3133: Change JavaDoc link to 3.0.0.RC1
Polishing
- Remove duplicate check - last bean def wins
- Doc Polishing
https://jira.springsource.org/browse/INT-3146
Incompatibility of using socket timeouts with a caching client
connection factory.
When a socket option timeout (soTimeout) is set on a TCP connection
and the timeout occurs, the socket is closed.
When a connection is intercepted, the close is performed through
the interceptor (for example to allow a closing handshake before the
physical close).
However, the caching client connection factory is implemented
using an interceptor, which returns the underlying connection to
the cache pool (for reuse). In the case of a TcpNetConnection,
the reader thread has terminated meaning that, the next time the connection
is used, no reply will ever be received.
The work-around (when using a gateway) is to use the 'remote-timeout'
attribute instead of relying on the soTimeout. There is no
work around when using collaborating channel adapters with a Net
connection. Using NIO works because there is no reader thread
in that case, but the socket is never closed on a timeout.
Always physically close the connection whenever an exception occurs
even if the close was delegated to an interceptor.
Add test cases for Net and NIO implementations.
* Add XSD attributeGroup `smartLifeCycleAttributeGroup`
* Use it for all adapters tags
* Refactoring `AbstractChannelAdapterParser` hierarchy
* Refactoring some `MessageHandler`'s that should not have `autoStartup` & `phase`.
It is the responsibility of `AbstrctEndpoint`
* Polishing some tests according new logic
JIRA: https://jira.springsource.org/browse/INT-2998
INT-2998: Polishing
Polishing
- Reformat schemas where indentation changed
- Add SLC to the UDP inbound adapter
Polishing tests to follow with new changes in the SPR 4.0
Polishing build.gradle to be compatible with SPR 4.0
Build passes with SPR 3.2.4 too
JIRA: https://jira.springsource.org/browse/INT-3131
The second chance NIO logic that detects a close being received
in a race condition with a reply needs to be enabled always because
a Failover or Caching connection factory might be using NIO
underneath.
Also, add an epoch to the FailoverTcpConnection's connectionId which
is incremented on each use to prevent a close after a previous
use being received by the current user.
Finally, add code in the FailoverTcpConnection to not propagate
messages from an old (defunct) delegate connection - if the
delegate has changed, the connection must be being used by
another client.
Previously, when a calling thread was waiting for a reply, and an exception
occurred on the socket, the exception was not propagated to the thread and
it would eventually get a timeout, but with no indication of the problem.
Propagate the exception to the calling thread by invoking onMessage()
with an ErrorMessage.
Ignore the ErrorMessage in other TcpListeners (inbound adapter, gateway).
Ensure NIO closes are not missed by sending an ErrorMessage when the selector
thread detects a closed channel.
Add tests for Net, NIO, cached and failover connection factories.
Polishing
Remove left-over debug logs; add comment to setReply().
Add comments for ErrorMessages in onMessage().
Support nesting the fail over and caching client connection
factories.
The TcpListener chain was not set up properly; this prevented
caching Failover connections, or Failover cached connections.
Each nested interceptor needs the next outer interceptor set
up as its listener, so the underlying connection calls
onMessage() all the way up to the ultimate listener (adapter
or gateway).
Fix the Listener hierarchy; add tests for caching failover
connections and failing over cached connections.
Don't overrwrite the actual connection id if it has already
been set up by a lower level wrapper.
The SimplePool maintains an 'allocated' set, for the sole reason
of preventing a "foreign" (non-managed) object being returned.
When a pool item is detected as stale, it is removed from the pool
but remains in the 'allocated' set.
Add a test to verify the allocated size is reduced when a stale
item is popped from the pool.
Add a FileTransferringMessageHandler test (where the problem was
discovered).
Fix a test in TCP to expect a close().
Polishing
Do not allow returning null items to just release a permit - it
cannot remove the item from allocated. Clients must return the
stale item to the pool so it can be refreshed on the next get.
Only used by the TCP caching CF when returning a connection when
the factory is not running; but should not be allowed.
Also, protect against double release - not currently an issue with
existing users of SimplePool, but should be protected against. Could
cause the permit count to exceed the pool size.
Add inUse Set to the pool so we can detect attempts to release an
item that has already been released.
A TCP Connection is closed when an outbound gateway times out.
However, with the CachingClientConnectionFactory, the connection
was returned to the pool instead of being physically closed.
Override forceClose() and physically close the connection so the
next time it is retrieved from the pool it is detected as stale
and re-established.
Refactor test case and invoke it twice; once with a regular connection
factory and once with a cached factory.
Remove an unused logger.
TCP streams have no standard message structure. Therefore, the
TCP implementation previously only transferred the message
payload.
If someone wanted to convey header information, they would have
to write their own wrapper and/or use Java serialization for
the entire message.
This change provides a strategy to allow users to determine
which headers are transferred, and how.
A MessageConvertingMessageMapper is now provided that invokes
any MessageConverter. A MapMessageConverter is provided that
converts the payload, and selected heades to a Map with two
entries ("payload") and ("headers").
A MapJsonSerializer is provided that converts a Map to/from
JSON. Jackson can't delimit multiple objects in a stream
so another serializer is required to encode/decode structure.
A ByteArrayLfSerializer is used by default, inserting a
linefeed between JSON objects.
The combination of these elements now allows header
information to be transferred over TCP. Of course, users
can implment their own (de)serializer to format the
bits on the wire exactly as needed by their application.
INT-1807 Polishing
Add a test that uses a Map MessageConverter with a
Java (de)serializer.
INT-1807: Polishing
INT-1807: Rebased and polished
Change `MapJsonSerializer` to use `JsonObjectMapper` abstraction
Doc Polishing
Allow an application to be informed of problems on the IMAP idle
thread by emitting an event containing the exception.
Introduce IntegrationApplicationEvent hierarchy for all
events emitted by SI components.
INT-3099 Polishing (PR Comments)
Separate TCP events into discrete subclasses.
* Introduce XSD 'id' attribute for all `<chain>` sub-components
* Register `<chain>` sub-components as Beans with names based on `<chain>` 'id'
* Generate aliases for sub-components based on element 'id'
* Add `<chain>` parser test to check sub-components as Beans within `BeanFactory`
* Ignore `NestedChainParserTests` as abnormal and confused for SI-configs
* Now `<chain>` sub-components are trackable by `MessageHistory`
* Now `<chain>` sub-components are eligible for JMX export
* Polishing some failed tests
https://jira.springsource.org/browse/INT-2755https://jira.springsource.org/browse/INT-2321
INT-2755: Register as Beans only if 'id' provided
INT-2755: Set 'componentName' for handlers with 'id'
When components within `<chain>` are configured with an 'id' attribute
the 'componentName' of the handler is constructed as:
chainHandler.componentName + '.' + handler.componentName.
Otherwise it is as was before.
INT-2755: Polishing
INT-2755: changes according PR's comments
* Add INFO about preference of `id` attribute for `<chain>` in the `ChainParser`
* Change `<chain>`'s sub-components `id` generation algorithm
* Configure `<chain>`'s sub-component `componentName` in the `ChainParser` instead of in the `MessageHandlerChain`
* Add `componentName` property to the `JpaOutboundGatewayFactoryBean`
* Add a note to the Reference Manual
INT-2755: Localize duplicate 'id' within chain
INT-2755: remove child beans aliasing
* Polishing tests
* Polishing docs
INT-2755 Polishing: Handle Nested Chain Bean Names
Previously named beans within a nested chain did not get unique
bean names.
INT-2755: Polishing nested chains
Doc Polishing
The TcpNioConnection uses an internal InputStream.
The standard deserializers only use the read() method.
Custom Deserializers might use read(byte[]); the
internal InputStream did not override these methods, possibly
causing the deserializer to hang awaiting data, when none
was coming.
Override these methods, with the appropriate semantics
from the super class...
- Block until at least one byte arrives.
- Return early if no data is available after at least one byte arrives.
- Return -1 if closed when no data arrived.
- Return the number of bytes actually read.
Polishing - PR Comments remove read(byte[]) because it doesn't do
anything different to the superclass method.
Use latches and the new CLOSE event instead of arbitrary waits for
sockets to close.
Factor out common test code into setup methods.
removed system.out
Sporadic test failures (fairly consistently on Mac).
Problem was some tests sent a message to a client socket before the
close notification had been received.
Wait for client connection to close rather than waiting for the
server to stop.
Also, with NIO, close the channel on accept exception.
Also improve debugging by including a log of new connections
including the id.
INT-2964 - JavaDoc: <emphasis> should be <em>
INT-2964 - JavaDoc: <p/> should be <p>
INT-2964 - JavaDoc polishing
* <li></li> should be wrapped in <ul></ul>
* wrap code snippets in {@code myCode()}
* change <code>false</false> to <code>false</code>
INT-2964 - Polish - Fix more JavaDoc errors
* Mockito-all dependency causes JavaDoc error. Change dependency to Mockito-Core.
INT-2932
- Reference Documentation
INT-2941
- Add getter for Throwable
- Add auto-startup to Parser, Endpoint (already in schema)
- Add phase to schema, Parser
(Schema changes are mostly indentation because the element now
extends SmartLifecycleType).
Inadvertent sign extension on binary data with bit 7 set
causes early termination of binary deserializers.
The ChannelInputStream (which replaced the piped input
and output streams) failed to mask off the top 24 bits
of "normal" bytes received, causing Deserializers to
believe the stream was closed.
Add a mask of 0xff to bytes read.
Add a test case.
Assertion that server side connection was established could
occur after the connection was established, but before it was
added to the collection.
Add a latch.
Reference: https://jira.springsource.org/browse/INT-2126
* Add TcpConnectionEvent for OPEN, CLOSE, EXCEPTION.
* Add Event Type Enum - Provides type safety for event types.
* Add <int-ip:tcp-connection-event-inbound-channel-adapter />
* Add the ability to filter events (restrict to one or more subclasses)
* Add publishEvent() to TcpConnectionSupport to permit, for example, connection interceptors to publish events via the connection, but the event source can only be the connection used to publish the event.
* Add documentation to Reference Documentation section *What's New*
INT-2871 Provide Mechanism to Find TCP Connections
Reference: https://jira.springsource.org/browse/INT-2871
* Add getOpenConnectionIds() to Abstract Connection Factory
* Add closeConnection(String connectionID)
* Introduce removeClosedConnectionsAndReturnOpenConnectionIds
INT-2877 TCP Extension Improvements
Reference: https://jira.springsource.org/browse/INT-2877
* Make improvements to make extension easier.
* Add setCustomHeaders() to mapper.
* Add mapper to connection factory namespace parser.
* Add plumbing for stateful Deserializers.
* Refactor setting standard headers
* rename setCustomHeaders to supplyCustomHeaders - returning a Map and not exposing the MessageBuilder to subclasses
INT-2872 Remove Deprecated Items in ip Module
Reference: https://jira.springsource.org/browse/INT-2872
* pool-size Attribute
* setScheduler
* TcpSendingMessageHandler.getConnection()
* Deprecated attribute on TCP Connection Factory.
TcpNetConnection IO errors were only logged under DEBUG.
* Code was added to suppress the error log when the exception was due to a normal close. This inadvertently suppressed all ERROR logs.
* Capture a local copy of this.noReadErrorOnClose before closing the socket after an exception.
* Add test case to ensure ERROR log is emitted.
* MessageGroupStore <-> MessageGroupCallback
- Change callback to be an inner interface of the MGS.
* INT-2829 Fix Class Tangle in IP Module
- 3 way tangle between TcpSender, TcpConnection, TcpMessageMapper
- Clean up interfaces (remove setters); add top-level support classes for TcpConnection, TcpConnectionInterceptor.
* INT-2829 Remove TCP Package Tangle
- tcp.connection <-> tcp.connection.support
- Remove ...connection.support package - move classes to ...connection.
Copy schemas, cleanup whitespace, update spring.schemas files,
update version in AbstractIntegrationNamespaceHandler.
For each schema compared the 2.2 version with the 3.0 version
(diff -w), ensuring the only difference is the import of
3.0 schemas (where appropriate) instead of 2.2.
Previously TcpNioConnection used a pair of Piped(In|Out)putStreams
to pass data from the read event to the deserializer.
However, it is possible to get a 'Broken Pipe' exception if the
last thread that wrote to the pipe terminates.
Replace the use of this pair of streams with an inner class
ChannelInputStream, which provides the same functionality, but
with no dependency on threading.
Uses a BlockingQueue to store read buffers which are consumed by
the message assembler thread (read from the InputStream in the
Deserializer). To avoid OOM conditions, the queue is limited to
5 "unread" buffers; and a timeout will occur if the reader
doesn't consume some data to free up space in the queue.
When an exception occurred while obtaining a connection,
the root cause was lost.
In addition, a MessageMappingException was thrown instead
of a MessageHandlingException.
- Deprecate getConnection() in favor of obtainConnection()
- Capture the underlying exception as the cause of a MessageHandlingException
- Add test case
fixed @link (when merging)
removed @return with no args (when merging)
A connection used by a gateway may timeout prematurely.
Previously, the socket read timeout simply controlled when
a socket would be closed after the timeout occurred.
Consider a connection with so-timeout set to 10 seconds; the
application initializes at T+0 and the first message is sent,
with the server responding immediately; the timeout clock starts.
Next, a message is sent at T+5 to a service that takes 6 seconds
to respond. The connection will timeout at T+10 before the
response is received; the socket is closed and client does not
receive the response.
The solution is to wait 2 timeout cycles *IF* a message has
been sent within the current timeout.
Maintain a timer for the last send() operation. When a socket
timeout occurs, examine the last sent time; if within the
timeout, defer the close until the next timeout.
We cannot simply rely on the last send time because, when
using collaborating adapters, continuous sends (with no replies)
would defer the close indefinitely. Hence, the second test looking
to see if we have not had a successful read for the last 2 timeouts.
NIO does not directly support socket timeouts (because there is no
thread hanging on the read); instead, the timeout logic is
performed on the selector thread.
Rename DefaultTimeoutTests to ConnectionTimeoutTests.
Add tests (for both Socket and NIO connections) to assert the correct
operation when a send is performed within a timeout, as well as
when the server takes > 2x the timeout to respond.
INT-2860 Fix Typo in Exception Message
Error sending meeeage.
removed invalid comment from test (while merging)
INT-2861
Close connection after 'remoteTimeout' because the
socket is dirty (may contain an in-flight reply).
Add test that demonstrates the problem and that it
is resolved.
INT-2862
Remove entries from pendingReplies (map of async responses
for which we are waiting).
Add an assertion to the above test to ensure cleanup.
Documentation indicates that the keystore and truststore
arguments can be resource patterns (file:..., classpath:..., etc)
but the code always used a ClasspathResource.
Use a PathMatchingResourcePatternResolver to interpret the
pattern correctly.
Add tests.
A secondary part of the new gateway test was to ensure we
reused the cached connection. However, there was a race condition
in that the connection may not yet have been returned to the pool
by the reader thread that received the response.
Using the CachingClientConnectionFactory with a gateway
doesn't work. The Listener (gateway) was not being set up
properly by the wrapper. Instead, the actualListener in the
wrapped connection was set to null. This caused the connection
to "close" itself (return to the pool) immediately after the send.
In addition, even with the actualListener set up to properly
reference the gateway, it still wouldn't work because the
gateway can't correlate the reply (the cached connectionId is
prefixed with "cached:" so the gateway can't find the reply.
Further, when onMessage() returned, it caused the reader loop
to end.
Fixes:
1. Properly set up the cached connection with the real listener
so that ultimately the underlying connection sees there is an
actualListener and so doesn't close itself after the send.
2. Override getListener() on the cached connection to return the
real listener.
3. Override onMessage() in order to fix up the connection id in
the message, and return true (intercepted) so the connection doesn't
close itself (terminate the reader thread).
4. close() (return to the pool) after invoking the gateway's
onMessage().
5. Add test cases to verify proper operation including an assertion
that the pooled connection is reused.
This is made a little more complex by the tangle between
connections and connection interceptors; mainly because single-use
connections close themselves after use. This will be addressed in
3.0 (INT-2829) making connection interceptors simpler.
* Remove Class Tangle in JPA
* tcp.connection
* tcp.connection.support
* Add Comment to MessageGroupCallback
- Comment that it is moving into MGS in 3.0.
- Convert DOS newlines to Unix.
For reference see: https://jira.springsource.org/browse/INT-2828
Test data was sent before the error-channel was added to
the inbound adapter; the expected error could occur before
the error-channel was in place and the test failed
because the expected error message was absent.
Move the error-channel setup to before the test
data is sent.
1. Schema docs indicated client-mode is true by default, when
it is false.
2. Use IntegrationObjectSupport taskScheduler instead of
a field in the subclass.
INT-2599 polishing
