INT-2932
- Reference Documentation
INT-2941
- Add getter for Throwable
- Add auto-startup to Parser, Endpoint (already in schema)
- Add phase to schema, Parser
(Schema changes are mostly indentation because the element now
extends SmartLifecycleType).
Inadvertent sign extension on binary data with bit 7 set
causes early termination of binary deserializers.
The ChannelInputStream (which replaced the piped input
and output streams) failed to mask off the top 24 bits
of "normal" bytes received, causing Deserializers to
believe the stream was closed.
Add a mask of 0xff to bytes read.
Add a test case.
Assertion that server side connection was established could
occur after the connection was established, but before it was
added to the collection.
Add a latch.
Reference: https://jira.springsource.org/browse/INT-2126
* Add TcpConnectionEvent for OPEN, CLOSE, EXCEPTION.
* Add Event Type Enum - Provides type safety for event types.
* Add <int-ip:tcp-connection-event-inbound-channel-adapter />
* Add the ability to filter events (restrict to one or more subclasses)
* Add publishEvent() to TcpConnectionSupport to permit, for example, connection interceptors to publish events via the connection, but the event source can only be the connection used to publish the event.
* Add documentation to Reference Documentation section *What's New*
INT-2871 Provide Mechanism to Find TCP Connections
Reference: https://jira.springsource.org/browse/INT-2871
* Add getOpenConnectionIds() to Abstract Connection Factory
* Add closeConnection(String connectionID)
* Introduce removeClosedConnectionsAndReturnOpenConnectionIds
INT-2877 TCP Extension Improvements
Reference: https://jira.springsource.org/browse/INT-2877
* Make improvements to make extension easier.
* Add setCustomHeaders() to mapper.
* Add mapper to connection factory namespace parser.
* Add plumbing for stateful Deserializers.
* Refactor setting standard headers
* rename setCustomHeaders to supplyCustomHeaders - returning a Map and not exposing the MessageBuilder to subclasses
INT-2872 Remove Deprecated Items in ip Module
Reference: https://jira.springsource.org/browse/INT-2872
* pool-size Attribute
* setScheduler
* TcpSendingMessageHandler.getConnection()
* Deprecated attribute on TCP Connection Factory.
TcpNetConnection IO errors were only logged under DEBUG.
* Code was added to suppress the error log when the exception was due to a normal close. This inadvertently suppressed all ERROR logs.
* Capture a local copy of this.noReadErrorOnClose before closing the socket after an exception.
* Add test case to ensure ERROR log is emitted.
* MessageGroupStore <-> MessageGroupCallback
- Change callback to be an inner interface of the MGS.
* INT-2829 Fix Class Tangle in IP Module
- 3 way tangle between TcpSender, TcpConnection, TcpMessageMapper
- Clean up interfaces (remove setters); add top-level support classes for TcpConnection, TcpConnectionInterceptor.
* INT-2829 Remove TCP Package Tangle
- tcp.connection <-> tcp.connection.support
- Remove ...connection.support package - move classes to ...connection.
Copy schemas, cleanup whitespace, update spring.schemas files,
update version in AbstractIntegrationNamespaceHandler.
For each schema compared the 2.2 version with the 3.0 version
(diff -w), ensuring the only difference is the import of
3.0 schemas (where appropriate) instead of 2.2.
Previously TcpNioConnection used a pair of Piped(In|Out)putStreams
to pass data from the read event to the deserializer.
However, it is possible to get a 'Broken Pipe' exception if the
last thread that wrote to the pipe terminates.
Replace the use of this pair of streams with an inner class
ChannelInputStream, which provides the same functionality, but
with no dependency on threading.
Uses a BlockingQueue to store read buffers which are consumed by
the message assembler thread (read from the InputStream in the
Deserializer). To avoid OOM conditions, the queue is limited to
5 "unread" buffers; and a timeout will occur if the reader
doesn't consume some data to free up space in the queue.
When an exception occurred while obtaining a connection,
the root cause was lost.
In addition, a MessageMappingException was thrown instead
of a MessageHandlingException.
- Deprecate getConnection() in favor of obtainConnection()
- Capture the underlying exception as the cause of a MessageHandlingException
- Add test case
fixed @link (when merging)
removed @return with no args (when merging)
A connection used by a gateway may timeout prematurely.
Previously, the socket read timeout simply controlled when
a socket would be closed after the timeout occurred.
Consider a connection with so-timeout set to 10 seconds; the
application initializes at T+0 and the first message is sent,
with the server responding immediately; the timeout clock starts.
Next, a message is sent at T+5 to a service that takes 6 seconds
to respond. The connection will timeout at T+10 before the
response is received; the socket is closed and client does not
receive the response.
The solution is to wait 2 timeout cycles *IF* a message has
been sent within the current timeout.
Maintain a timer for the last send() operation. When a socket
timeout occurs, examine the last sent time; if within the
timeout, defer the close until the next timeout.
We cannot simply rely on the last send time because, when
using collaborating adapters, continuous sends (with no replies)
would defer the close indefinitely. Hence, the second test looking
to see if we have not had a successful read for the last 2 timeouts.
NIO does not directly support socket timeouts (because there is no
thread hanging on the read); instead, the timeout logic is
performed on the selector thread.
Rename DefaultTimeoutTests to ConnectionTimeoutTests.
Add tests (for both Socket and NIO connections) to assert the correct
operation when a send is performed within a timeout, as well as
when the server takes > 2x the timeout to respond.
INT-2860 Fix Typo in Exception Message
Error sending meeeage.
removed invalid comment from test (while merging)
INT-2861
Close connection after 'remoteTimeout' because the
socket is dirty (may contain an in-flight reply).
Add test that demonstrates the problem and that it
is resolved.
INT-2862
Remove entries from pendingReplies (map of async responses
for which we are waiting).
Add an assertion to the above test to ensure cleanup.
Documentation indicates that the keystore and truststore
arguments can be resource patterns (file:..., classpath:..., etc)
but the code always used a ClasspathResource.
Use a PathMatchingResourcePatternResolver to interpret the
pattern correctly.
Add tests.
A secondary part of the new gateway test was to ensure we
reused the cached connection. However, there was a race condition
in that the connection may not yet have been returned to the pool
by the reader thread that received the response.
Using the CachingClientConnectionFactory with a gateway
doesn't work. The Listener (gateway) was not being set up
properly by the wrapper. Instead, the actualListener in the
wrapped connection was set to null. This caused the connection
to "close" itself (return to the pool) immediately after the send.
In addition, even with the actualListener set up to properly
reference the gateway, it still wouldn't work because the
gateway can't correlate the reply (the cached connectionId is
prefixed with "cached:" so the gateway can't find the reply.
Further, when onMessage() returned, it caused the reader loop
to end.
Fixes:
1. Properly set up the cached connection with the real listener
so that ultimately the underlying connection sees there is an
actualListener and so doesn't close itself after the send.
2. Override getListener() on the cached connection to return the
real listener.
3. Override onMessage() in order to fix up the connection id in
the message, and return true (intercepted) so the connection doesn't
close itself (terminate the reader thread).
4. close() (return to the pool) after invoking the gateway's
onMessage().
5. Add test cases to verify proper operation including an assertion
that the pooled connection is reused.
This is made a little more complex by the tangle between
connections and connection interceptors; mainly because single-use
connections close themselves after use. This will be addressed in
3.0 (INT-2829) making connection interceptors simpler.
* Remove Class Tangle in JPA
* tcp.connection
* tcp.connection.support
* Add Comment to MessageGroupCallback
- Comment that it is moving into MGS in 3.0.
- Convert DOS newlines to Unix.
For reference see: https://jira.springsource.org/browse/INT-2828
Test data was sent before the error-channel was added to
the inbound adapter; the expected error could occur before
the error-channel was in place and the test failed
because the expected error message was absent.
Move the error-channel setup to before the test
data is sent.
1. Schema docs indicated client-mode is true by default, when
it is false.
2. Use IntegrationObjectSupport taskScheduler instead of
a field in the subclass.
INT-2599 polishing
A taskExecutor thread could deadlock in getTaskExector() if
stop() is attempting to shut down the task executor.
This could delay the stop by up to 20 seconds.
Move the if(isActive) test outside of the synchronized
block.
Initial Implementation; UDP test.
syslog - Add TCP Test
Add single byte terminating deserializer and a
subclass that terminates on LF (syslog uses
LF terminator).
Move to Core
Allow for transport of undecoded packet over, say, AMQP.
INT-2711 Syslog Transformer
PR Comments; Polishing.
Remove List option; support to-map version only.
When using an outbound adapter with a server connection factory
(an inbound adapter 'owns' the connections), the outbound adapter
simply logged exceptions.
There are use cases where flows need to know the exception occurs.
Change the adapter to throw a MessagingException.
Update the 2.1 to 2.2. Migration guide explaining that the
ExpressionEvaluatingRequestHandlerAdvice can be used to restore
2.1 behavior, and trap the exception.
Add general capability to advise just the handleRequestMessage
part of an AbstractReplyProducingMessageHandler.
This is to advise just the immediate operation, and not the
entire downstream flow.
Uses include:
* outbound gateway post processing
* adding retry behavior using spring-retry
* adding circuit breaker functionality
Initial commit for review.
Also need to advise simple message handlers (such as file
etc) to allow them to post-process file operations
with payload.delete(), payload.renameTo(...) etc.
INT-2250 Add Circuit Breaker Advice
INT-343 Add Retry Advice
Stateless and Stateful retry using spring-retry. Stateless
means the RetryTemplate performs the retries internally.
Stateful means the exception is thrown (e.g. to JMS container)
and the retry state is maintained by spring-retry.
INT-2215, INT-343, INT-2250 Refactoring
Factor out common abstract Advice class.
INT-2214 Catch Evaluation Expression Exceptions
If an onSuccess expression evaluation fails, add an
option so the user can decide whether such an exception is
caught, or propagated to the caller.
INT-2214 etc PR Review Polishing
INT-2214 etc Namespace Core, File, FTP
Add <request-handler-advice-chain/> to outbound endpoints.
INT-2214 etc. More Namespace Support
amqp, event, gemfire, groovy, http, ip, jdbc, jms, jmx, jpa, mail, rmi, sftp, twitter, ws, xmpp
INT-2214 etc Polishing
PR Review
INT-2214 etc Polishing
Don't catch Throwable.
Move Advice classes to handler.advice package.
* Ensure that no deprecation warnings occur
* Spring Integration builds with all tests successfully for Spring 3.1.2.RELEASE and 3.0.7.RELEASE (Except Spring Integration HTTP)
For reference see: https://jira.springsource.org/browse/INT-2689
INT-2694 - Fix Http Test Failures with Spring 3.0
As part of INT-2689, fix Http Test Failures in the Spring Integration Http Module when using Spring 3.0.7.RELEASE
For reference see: https://jira.springsource.org/browse/INT-2694
DefaultTcpNioSSLConnectionSupport and DefaultTcpNetSSLSocketFactorySupport need
an SSLContext which is set up in afterPropertiesSet(). However, when configuring
the connection factories with default strategies, afterPropertiesSet() is not
called. Result: NullPointerException.
Add call to afterPropertiesSet() where appropriate and tests to verify.
Previously, the UDP inbound adapter was incorrectly using
the default task scheduler (taskScheduler) to run its main
receive activity. Once a packet is read, it is handed off
to another thread for message processing, using a configured
(or default) task executor.
We should not hold on to a scheduler thread permanently; the
default scheduler has only 10 threads and one user's app
stopped working when using 10 inbound adapters.
The work around was to define an explicit 'taskScheduler'
bean with more threads.
Improve performance.
For NIO sockets, a pair of PipedInput/OutputStreams are
used to transfer data from the reading thread to the
assembling thread.
The stream used the default buffer size (1024) which was
inefficient for large messages.
This change uses the underlying socket's receiveBufferSize
attribute to set the size of the piped stream, allowing
for more efficient data transfer.
Also adjusts the mocks in the deadlock detection test
to ensure the received data is larger than the pipe
buffer size.
* Add beginShutdown() and endShutdown() to OrderlyShutdownCapable
* JMS/AMQP stop listener containers
* TCP (server side)
** after beginShutdown() disallow new connections, drop (log) new messages
** after endShutdown() close server socket
* HTTP (server side)
** after beginShutdown() disallow any new requests (503 Service Unavailable)
* Docbook updates
** What's new section
** Orderly Shutdown section.
INT-1141 polished code
INT-1141 added tests
INT-1141 improved how iterator is obtaind from OrderAwareLikedHashSet
INT-1141 polishing
INT-1141 stashed commit with List-based collection and performance tests
INT-1141 improved OrderedAwareLinkedHashSet
INT-1141 polishing
INT-1141, INT-2627 improved LoadBalancingStrategy and
RoundRobinLoadBalancingStrategy to obtain handler's iterator faster
INT-1141 improved getHandlerIterator method to ensure that it only executes reordering logic if there are more then one handler
INT-1141 simplified OrderedAwareLinkedHashSet to not to extend from LinkedHashSet
INT-1141
changed OrderedAwareLinkedHashSet to OrderedAwareCopyOnWriteArraySetTests, imporoved array creation in RoundRobinLoadBalancingStrategy
INT-1141 polished failing tests and RoundRobinLoadBalancingStrategy. Removed 'transient' from OrderedAwareCopyOnWriteArraySet
Move SocketTestUtils.findAvailableServerSocket to Test Module
For reference see: https://jira.springsource.org/browse/INT-2564
* Move SocketTestUtils.findAvailableServerSocket to Spring Integration Test Module
* Fix tests
* polishing TCP test to use SmartLifecycle from 'chain'
* polishing XSD to exclude using 'poller' element inside 'nested-chain'
INT-2605: eliminate breaking change in the XSD
Added testing utility from samples to provide a tool for
users to wait for servers to start during tests.
Also refactored all tests that had similar code to use
this class.
Also corrected two failing tests that needed this wait.
XSD refactoring: remove use="required" from 'request-channel' attribute of all 'outbound-gateways'
Tests for all 'outbound-gateways' inside the <chain>
The IPHeaders.CONNECTION_SEQ was replaced by the
standard sequenceNumber header (when applySequence is true)
in 2.1, in order to facilitate resequencing using the
standard resequencer.
This header was deprecated at that time. It is now
removed.
Users that were relying on this header should set
applySequence to true on the connection factory.
Implemented a ConnectionFactory that wraps a list
of connection factories, used to fail over if
a connection fails.
INT-2459 Polishing
Rebase; fix test; move boolean resets to finally block
in server factories.
INT-2459 Polishing
PR Review
Fix a test that could block indefinitely when error
in code under test.
reply-timeout had the wrong function, when compared to
other gateways.
Add remote-timeout to reflect the time we will wait
for a reply from the remote system
Make reply-timeout set the sendTimeout on the messaging template
Set remore-timeout to reply-timeout (if set) unless remote-timeout
is explicitly set.
Update reference doc, and migration guide on Wiki.
Previously, it was possible to run out of threads in
a fixed thread pool, and this could cause an indefinite
deadlock.
With this change, the deadlock is detected causing the
current message to fail, but freeing up the stuck
threads.
This was only seen with very small thread pools but
the potential was there for the condition to occur
under extreme conditions.
INT-2419 Deprecate poolSize Property
INT-2419 Polishing
Add note about OOM possibility now that an
unbounded task executor is used by default.
@ExpectedException is deprecated.
Also fixes a sporadic timing issue in the client mode test;
the default retry interval is 60 seconds and we only wait
10 seconds for the connection. If the initial connection
fails, we don't retry before the timeout.
INT-2527 Remove Event Deprecation
@Test @ExpectedException... -> @Test(expected...
When sending a message to the server, we checked that the
user hook for the socket creation was called. However,
it was possible (likely) that we tested the counter before
the socket actually opened.
Add strategy interfaces
- obtaining ServerSocketFactory and SocketFactory
- post processing ServerSockets and Sockets
- obtaining initialized SSLContext
Provide SSL and non-SSL implementations of the strategies, to
serve up the appropriate socket factories, and do nothing in the
post processing methods.
The postprocessors allow the user to modify sockets after
configured attributes have been applied but before the sockets
are used.
This is particularly useful with SSL in case additional SSL
options need to be applied.
Docs
Polishing JavaDocs
Polishing
Javadocs, polishing
Polishing
Polishing
INT-2153 Polishing
PR Review
White Space
INT-2513 Polishing
Fixed method names.
Handlers for consumer endpoints get a generated name
derived from the endpoint class name.
It is useful (for example for autowiring in tests)
to give the handlers a well-known name.
If the endpoint has an ID attribute, the handler
now gets a bean name of "<ID>.handler".
Add re-init logic for nested chains
Add logic about nested element for AbstractChannelAdapterParser
Refactor of DefaultOutboundChannelAdapterParser
Test for non-last nested chain with some outbound-channel-adapter
Improve XSD for chain-type
Manual outbound-channel-adapter ability for chain
Integration tests for all outbound-channel-adapter within <chain>
Remove redundant 'return-value-required' attribute from <stored-proc-outbound-channel-adapter>
Add support 'expectReply' for FileWritingMessageHandler
INT-2275 polishing & refactor FileOutbound*Parser
HttpRequestExecutingMessageHandlerTests polishing
INT-2275: polishing JavaDoc
Initial commit - @ManagedOperation on IMBE
- can be invoked via JMX, <control-bus/>, or getting a reference to
the IMBE from the application context.
INT-2485 Updates After Review Comments (JIRA)
* Shutdown Schedulers first, and wait for them
* Add a force parameter, which overrides thread pool shutdown options
* Shutdown Sources/Channels after all thread pools have stopped
* Mark other components as OrderlyShutdownCapable (e.g. JMS/AMQP Listener containers) and shut them down first
* Wait for remaining time to allow for quiescence
Also
* remove TimeUnit parameter (not JMX-friendly); time limit is now always milliseconds
* If thread pools don't stop in time limit, force them down.
INT-2485 Handle Self-Destruction
Add shutdown-executor to IMBE.
When the shutdown was called on a Spring-Managed thread, the shutdown
was not clean because we timed out waiting for the current thread to
terminate. After that, we force terminated other components.
Now, by providing a dedicated Executor for the shutdown process, it
is used for the shutdown instead of the current thread. This Executor
is *not* shutdown.
It is not necessary to provide an Executor if the stopActiveComponents()
method is called on some other thread that is not involved in the
shutdown.
Also adds executor name to logs, when available.
INT-2485 Polishing
Fix MBean object name collision when running all tests.
INT-2485 Enable TCP Shutdown
Make TCP connection factories 'OrderlyShutdownCapable' so
they are stopped before schedulers/executors in order for
them to release any executor threads they are holding.
INT-2485 Polishing
Didn't need DirectFieldAccessor - scheduler and executor have
an accessor for the native ExecutorService.
Copyrights
INT-2485 Polishing
schemaLocation version.
INT-2485 PR Review Polishing
Depending on timing, it was possible the event the test
case was waiting for would never happen. With no timeout
the selector will block until the next IO event occurs,
and there is none in this test.
Adding a call to factory.close() forces a wakeup of
the selector, which will force the harvest to occur.
This is a test-only problem.
When using NIO, a Map of connections is maintained (keyed
by the SocketChannel) and used to manage registrations
for NIO events, timeouts etc.
When a connection is closed, the corresponding entry
should be removed from the map. The code to do this
is in AbstractConnectionFactory.processNioSelections().
However, if no socket timeout (soTimeout) has been set,
or it was explicitly set to 0, connections are not
removed from the map. This was because the timeout
logic and map cleanup is done in the same iteration
loop.
Now, if soTimeout is not set, the clean up loop
operates every nioHarvestInterval milliseconds
(default 2000), we don't want to run it on
every selector event.
In addition, it runs if the selectionCount is
zero - this might occur when a socket
is closed, when the selector.wakeup() is called.
INT-2519 Polishing
Rename connections field in NIO connection factories. Name
collision caused TestUtils problem on some platforms.
