Commit Graph

87 Commits

Author SHA1 Message Date
Artem Bilan
29b4a296be INT-4563: Add JMX test for MQTT
JIRA: https://jira.spring.io/browse/INT-4563

* Add a `MqttDslTests` with the JMX configured to be sure that SI
managed components are registered in JMX properly.
* Also this test covers a Java DSL configuration for MQTT channel adapters
* Some polishing for `AbstractMqttMessageHandler`
* Document Java DSL configuration for the MQTT channel adapters
2018-12-04 15:31:22 -05:00
Gary Russell
c594e75616 Fix MQTT synchronization for recoveryInterval 2018-10-26 16:42:48 -04:00
Artem Bilan
fc18476fc1 More MQTT polishing according Sonar report 2018-10-26 14:58:08 -04:00
Artem Bilan
fef0a4db03 RMI & MQTT Sonar fixes 2018-10-26 13:27:42 -04:00
Artem Bilan
41264110ce Some various polishing
* Add `RotatingServerAdvice.StandardRotationPolicy.getCurrent()` method
for better end-user experience when this class is extended
* Add NPE protection into the MQTT Channel Adapters.
Some code style polishing for them

**Cherry-pick to 5.0.x**
2018-10-10 15:21:58 -04:00
Gary Russell
11286907e8 GH-2471: Close client on outbound adapter
Resolves https://github.com/spring-projects/spring-integration/issues/2471
2018-10-10 11:49:50 -04:00
Gary Russell
4b9eaebd42 MQTT: completionTimeout should be long
Incorrectly `int` - inconvenient cast required when using `Duration.toMillis()`.
2018-08-27 16:48:44 -04:00
Gary Russell
5ab7d24934 Make Headers PREFIX Constants Public
**cherry-pick to 5.0.x**

* Make `IpHeaders` prefix as `public` as well
2018-08-27 16:44:49 -04:00
Artem Bilan
a1f016a94c INT-4498: StreamUtils -> FileCopyUtils
JIRA: https://jira.spring.io/browse/INT-4498

* Use `FileCopyUtils` instead of `StreamUtils` in test-cases when we deal
with `InputStream` s and files resources.
* Fix reflection-based tests with AOP to use an `ArrayList` access
instead of `LinkedList` since an optimization in SF:
6d6aa72e8f

**Cherry-pick to 5.0.x and 4.3.x**
2018-07-09 13:39:56 -04:00
Gary Russell
f9cea64a79 GH-2471: MQTT: Fix Thread Leak
Fixes https://github.com/spring-projects/spring-integration/issues/2471

Call `close()` on the client whenever the connection is lost or can't be
established, to release resources in the client.

**cherry-pick to 5.0.x, 4.3.x**
2018-06-06 12:32:17 -04:00
Gary Russell
07141302bb INT-4463: Full access to MqttConnectOptions
JIRA: https://jira.spring.io/browse/INT-4463

Certain options, such as `maxInFlight` were not exposed.

Deprecate the setters on the factory and allow the user to inject a pre-configured
`MqttConnectOptions`, thus making all (and any new) properties available to be
configured.
2018-05-07 12:39:15 -04:00
Artem Bilan
2f1b55f48c Upgrade to Gradle 4.7 and others, polishing build (#2423)
* Upgrade to Gradle 4.7 and others, polishing build

* Fix `MqttAdapterTests` according requirement of the latest Mockito
* Move `int-derby.properties` to the `/resources` from `/java`

* Fix Checkstyle violations
2018-04-20 11:11:05 -04:00
Artem Bilan
027021d6e4 Move XSDs to version 5.1
* Upgrade to Spring AMQP-2.1 and fix compatibility
2018-03-17 09:58:40 -04:00
Gary Russell
82f252dd00 INT-4390: Fix tangles
JIRA: https://jira.spring.io/browse/INT-4390

Phase I: flow context/registration cycle

- extract interfaces
- rename implementations to `Standard...`

Phase II - dsl<->dsl.channel tangles

Move the channel specs to dsl.

Fix missing refactorings

(Not related to DSL) - rename core `event` package to `events`

- avoid collision with event module package
- fix tangle caused by `MessageGroupExpiredEvent`.

Phase III - dsl<->config.dsl

- move classes from config.dsl to dsl

(Not related to DSL) - fix many tangles caused by graph being a sub-package of support.management
 - make `graph` a top-level package

(Not related to DSL) - move `IntegrationManagementConfigurer` from
`management` to `config` - tangle between core and management
* Polishing Copyrights, diamonds, some JavaDocs and What's New
2018-03-16 14:59:47 -04:00
Artem Bilan
30450c48be INT-4376: Upgage to Log4J 2 (#2321)
* INT-4376: Upgage to Log4J 2

JIRA: https://jira.spring.io/browse/INT-4376

* Deprecate Log4J 1.x components in favor of newly added
a `Log4j2LevelAdjuster` JUnit `@Rule`
* Update all the logging configuration to Log4J 2

* Polishing after testing
2018-01-11 10:59:55 -05:00
Artem Bilan
23137fd0cb GH-2275: Fix MqttMDCA for callbacks after stop
Resolves: spring-projects/spring-integration#2275

* Remove the `MqttPahoMessageDrivenChannelAdapter` as a callback from
the `IMqttClient` during `stop()`
* Check the `isRunning()` from the `connectionLost` callback to avoid
unexpected `scheduleReconnect()` when we are not running

**Cherry-pick to 4.3.x**
2017-11-10 10:50:50 -05:00
Artem Bilan
bc4337ffa8 GH-2071: Upgrade to S-WS-3.0 and Smack-4.2.1
Fixes: spring-projects/spring-integration#2071

* Clean up `build.gradle` for redundant excludes
* Add `javax.mail` dependency to WS module to avoid WARN about missed providers
* Refactoring for the XMPP module according changes in the latest Smack
* Polishing for the `BackToBackAdapterTests` to avoid extra wait for `null`
on the channel and some race conditions when client is closed during by the
`stop()` during publishing

Polishing `UriVariableTests` according the latest S-WS B-S

Revert excludes removal
2017-10-16 16:25:01 -04:00
Gary Russell
73d9c6b6c5 INT-4255: JSON Embedded Headers Message Mapper
JIRA: https://jira.spring.io/browse/INT-4255

- Support embedding headers for transports that don't support headers (TCP, Kafka, etc)
- Use the new message-aware Jackson ObjectMapper
- Provide a mechanism to more efficiently support byte[] payloads (avoid Base64 encoding)
- Support decoding "legacy" SCSt embedded headers

Polishing; PR Comments - Add Support to MQTT and TCP

Switch to simple patterns instead of regex

* Fix JavaDoc typo
* Upgrade to Jackson 2.9.1
2017-09-08 16:45:50 -04:00
shawn-yang-1979
3fb902855e Update MqttMessageDeliveredEvent.java 2017-08-03 10:55:46 -04:00
Gary Russell
07891ed9a3 Fix Sonar False Positive 2017-05-08 12:52:30 -04:00
Gary Russell
d8dc9d427e AMQP-4238: Detect Subscription Failures and QOS
JIRA: https://jira.spring.io/browse/INT-4238

Revert to using the sync client in the message-driven adapter so we can detect
subscription failures (the sync client throws an exception).

The only reason to use the async client was to timeout disconnects; this can
be achieved with the sync client and `disconnectForcibly`.

Also, the subscribe method updates the qos argument with the granted QOS values.

Detect and log if any QOS does not match the request.

Polishing

Polishing - PR Comments
2017-03-06 17:50:35 -05:00
Gary Russell
d8cbcd1310 INT-3923: Fix MQTT Reconnect Logic
JIRA: https://jira.spring.io/browse/INT-3923

Fixes: #2046

Previously, when connection is lost, the inbound adapter attempted to reconnect on
a schedule with a fixed delay.

If a connection was again lost, while the schedule is still running, we can end up
with another scheduled task running.

This is benign aside from the DEBUG log noise because the scheduled task tests the
connection before reconnecting.

However, if the `recoveryInterval` is short, it could consume CPU.

Change the reconnect to be a one-time scheduled task and reschedule if it fails to
reconnect.

Synchronize all access to the `connected` field.

Add a test case with a short recovery interval, before this fix, we see many logs
`Attempting reconnect`.
2017-02-02 13:45:44 -05:00
Artem Bilan
54654546b9 INT-4206: Upgrade to Mockito 2.5
JIRA: https://jira.spring.io/browse/INT-4206

* Fix unnecessary dependency resolution in BOM module
* Fix `MessagingMethodInvokerHelper` to handle `$MockitoMock$` generated classed which isn't CGLib `Proxies` any more
* Provide fixes for test classes according upgrade to Mockito `2.5`
* Fix Ceckstyle do not allow static imports for deprecated Mockito classes
2017-01-09 18:39:25 -05:00
Gary Russell
395f08b50d INT-4199: Fix Asserts with no Message
JIRA: https://jira.spring.io/browse/INT-4199

MQTT and JDBC.

Also remove unused message builder from the `JdbcMessageStore`.
2017-01-03 13:22:26 -05:00
Gary Russell
3035bc716d INT-4138: MQTT: Outbound Adapter Improvements
JIRA:https://jira.spring.io/browse/INT-4138

Expressions for topic, qos, retained.

Also change inbound mapping to `RECEIVED_...` headers.

Fix some minor asciidoc problems in (s)ftp.

Rework Qos/Retained Expressions/Defaults

Encapsulate the logic entirely in the converter.

Polishing - PR Comments
2016-11-16 11:22:10 -05:00
Gary Russell
67d6cd0c89 Lambdas for Remaining Modules JPA -> ZK
Polishing - PR Comments and Closeable Warnings

Eclipse emits bogus warnings with exceptions in lambdas.
Even though the lambda might run on another thread, elipse thinks it could
cause the context to not be closed.

SPR-14854: MessageChannel is now a @FunctionalInterface

* Additional Lambda polishing and some code style fixes
2016-10-28 16:06:11 -04:00
Gary Russell
c5fbd93787 Mockito Polishing
`$ find spring-integration-* | grep '\.java$' | xargs sed -E -i '' -e 's/(\(?)\(([^<()]*)\) *invocation.getArguments\(\)\[([0-9]*)\]/\1invocation.getArgumentAt(\3, \2.class)/'`
2016-10-28 11:06:05 -04:00
Artem Bilan
12fb590b95 Mainly Checkstyle Violation Fixes
* Upgrade to Checkstyle `7.1`
* Relax `RequireThis` rule a bit. Right now it does the effort only in case of overlapping. See https://github.com/checkstyle/checkstyle/issues/2362 for more info
* Enable some annotation rules and provide fixes for violations
* Enable `tabs indents` rule. This was the biggest fix in this PR
* Resolve `TODO` in the `MessagingMethodInvokerHelper` and fix tests to meet `IllegalStateException` now

Remain Checkstyle version `6.16.1` and reinstate `RequireThis` rule

The latest Checkstyle has a bug with local scope variables if they have the same names as property.

Revert some literals splitting

Fix some line length exceeding and code style
2016-09-22 13:23:51 -04:00
Artem Bilan
6c6e1d319d INT-4108: Fix idempotency for some Lifecycles
JIRA: https://jira.spring.io/browse/INT-4108

Some `Lifecycle.start()/stop()` usage doesn't ensure robustness for components causing unexpected and difficulty tracing issues

* Fix `Lifecycle.start()/stop()` for `FileReadingMessageSource`, `FileWritingMessageHandler`, `AbstractMqttMessageHandler`
* In the `DefaultHeaderChannelRegistry`, `LockRegistryLeaderInitiator`, `MqttPahoMessageHandler` rework logic for shared variables to avoid `NPE`
* Increase receive timeouts in the `PayloadDeserializingTransformerParserTests` and `UdpChannelAdapterTests`
* Prove with the `WatchServiceDirectoryScannerTests` changes that several invocation for `FileReadingMessageSource.start()` are idempotent

**Cherry-pick to 4.3.x**
2016-09-13 12:09:56 -04:00
Artem Bilan
eaed954458 INT-3913 Remove/resolve deprecation from the past
JIRA: https://jira.spring.io/browse/INT-3913

* Remove deprecated classes and methods/constructors, deprecated XML attributes
* Remove `TcpConnectionEventListeningMessageProducer` and rework tests logic to the `ApplicationEventListeningMessageProducer`
* Fix several typos
* Remove/rework deprecated entities mentioning
2016-08-26 13:42:47 -04:00
Gary Russell
04a42f54be Rename Schemas to 5.0
Fix Schema Imports

spring.schemas
2016-08-18 14:40:38 -04:00
Gary Russell
05cc7be644 checkstyle WhiteAround
WhiteAroundCheck script
2016-04-05 13:50:11 -04:00
Gary Russell
842aded9a4 checkstyle MutableException
checkstyle EmptyBlock

checkstyle fixRightCurly Script

checkstyle EmptyStatement

checkstyle RightCurly

checkstyle TailingWhite

checkstyle NeedBraces

Fix the line separator in the `fixRightCurly.gradle`
2016-04-05 13:10:33 -04:00
Gary Russell
4ac3a79df7 checkstyle FinalClassCheck
fixes

fixModifiers after fixFinal

Revert CachingSessionFactory

Class is spied in tests.

checkstyle - Import Rules

checkstyle InterfaceIsType

checkstyle InnerTypeLast

checkstyle OneStatementPerLine

CovariantEquals
OneTopLevelClass

* Revert `'\n'` -> `System.lineSeparator()` in the Gradle scripts to meet Git `autocrlf = true` on Windows
* Fix timing issue with the `LastModifiedFileListFilterTests`, when the `age = 1` might not be enough for the file object when we have some delay before checking
2016-04-04 13:49:56 -04:00
Artem Bilan
2b0598291c RequireThis rule and fixThis Gradle task
* `gradlew clean check -x test --parallel --continue` - to collect reports
* `gradlew fixThis --parallel` - to fix all possible vulnerabilities. With `-Dfile.encoding=UTF-8` on Windows

Since the `RequireThisCheck` doesn't see parents for anonymous classes (e.g. `Runnable` callback), its report doesn't contains the outer class name with `this.`,
therefore we still have to fix those cases manually.
Thanks to the wrong `replacer` just with `this.` we have uncompilable code enough easy to find problems.
Not so easy to fix for good readability though...

* Upgrade to Grade 2.12
* Upgrade to SonarQube native plugin

The fix contains at about 300 files. So, will be done on merge.

Fix `fixThis.gradle` according PR comments

Apply `fixThis` and also `fixModifiers` for test classes.
 Fix some `this.` inner issues manually.
 Make code polishing for long lines after `fixThis`

Fix conflicts and vulnerabilities after the rebase
2016-03-22 20:18:27 -04:00
Artem Bilan
66cd92a6e7 INT-3964: More @EnableIntegration Documentation
JIRA: https://jira.spring.io/browse/INT-3964

Doc Polish

XSD: document classes for element implementations

Fix Redis XSD typo for CDATA
2016-03-22 10:41:07 -04:00
Artem Bilan
ef40a939cf Add check header rule and fixHeaders task
Add "new line" in the header end

* Optimization for the `fixHeaders.gradle`
* Apply the `fixHeaders` for the affected classes
2016-03-07 15:08:51 -05:00
Artem Bilan
b1a09f5b0d INT-3881: MQTT: add XML recovery-intervalattr
JIRA: https://jira.spring.io/browse/INT-3881

Also fix a couple typos in the WS xsd
2016-01-15 13:12:36 -05:00
Gary Russell
5edc910a7f MQTT: Increase Test Timeouts 2016-01-12 09:42:53 -05:00
Artem Bilan
74cbc58698 INT-3926: Add <poller> for all outbound tags
JIRA: https://jira.spring.io/browse/INT-3926

* Add `<poller>` for all `<outbound-adapter(gateway)>`
* Add/modify tests for `<queue>` as an input channel and particular `<poller>`
* Move XSD from the `xml` package to the `config` level to achieve the consistency throughout the project.
2016-01-07 12:16:40 -05:00
Gary Russell
255247ca9a Code Polishing (Sonar)
- remove redundant modifiers
- overridable methods called from ctors

Polishing - PR Comments
2015-12-07 15:39:13 -05:00
Gary Russell
9fff767f71 INT-3900: MQTT: Don't Unsubscribe if !cleanSession
JIRA: https://jira.spring.io/browse/INT-3900

The message-driven adapter should not unsubscribe if `cleanSession` is
`false`.

INT-3900: MQTT: Changes for 4.3 Only

Doc Polishing
2015-12-07 14:10:24 -05:00
Gary Russell
051b393ed0 Prepare Master for 4.3
Reference to 4.3

Refactor `what's new` for 4.3
2015-11-17 12:12:49 -05:00
Artem Bilan
9d8e2b61f6 Fixes for Sporadic Test Failures
* `ClientWebSocketContainer`: add some synchronization to avoid race conditions: https://build.spring.io/browse/INT-B41-492
* `TomcatWebSocketTestServer`: convert to `0` port to rely on the OS resolution for `localPort`
* Add `LogAdjustingTestSupport` for STOMP test
* `SftpServerTests`: use `0` port to rely on the OS resolution for `localPort`
* `ImapMailReceiver`, `OutboundGatewayFunctionTests` (JMS), `CachingClientConnectionFactoryTests`,
`AsyncGatewayTests`, `AsyncMessagingTemplateTests`, `GatewayParserTests`, `PriorityChannelTests`, `AggregatorIntegrationTests`: increase timeout
* `EnableIntegrationTests`: use `LogAdjustingTestSupport`
* `FileOutboundChannelAdapterParserTests`: rework `Thread.sleep()` with `CountDownLatch`
* `ConnectionToConnectionTests`: increase timeout and count iteration. Previously with `1sec` we may lose some events. And we can't just rely on the `10sec`,
because the last iteration will be so long
* `TcpOutboundGatewayTests`: `500ms` is so big timeout to wait for the `Exception` that in the high load environment we can yield to other Thread so long.
Like in our case to `server` Thread to send the reply for us. Therefore decrease the Exception timeout to the `50ms` and increase server delay to `2sec`
* `StompInboundChannelAdapterWebSocketIntegrationTests`: remove `@Qualifier("taskScheduler")` as a potential candidate to test against latest SF changes.
We're fine with `SF-4.2.2` and it is just a test-case. So, I don't see reason to wait for their fix here.

STOMP: `session = null` in adapters for any transportError

Polishing
2015-11-14 16:12:23 -05:00
Artem Bilan
b9730cd183 INT-3882: StompSession Reconnection Logic
JIRA: https://jira.spring.io/browse/INT-3882

Add reconnect support for the StompSession:
* Introduce `recoveryInterval` for the `AbstractStompSessionManager`
* Add reconnect scheduled task
* Add handling of the `ConnectionLostException` into the `AbstractStompSessionManager`,
as well as for the `StompInboundChannelAdapter` and `StompMessageHandler`
* Cover adapters reconnection feature with tests
* Documentation polishing

Rework logic according PR comments

* Make `StompMessageHandler` as a "lazy-load" for the connection
* Add "direct" connect interaction for the `AbstractStompSessionManager`
* Polishing tests

Polishing

Fix `StompAdaptersParserTests#testStompSessionManagerReconnect()` to use "fake" server port

Address PR comments

The further polishing

Some further polishing

* `AbstractStompSessionManager`: `reconnectFuture.cancel(true)` on each `scheduleReconnect()` to avoid something like "DDoS attack"
* Add reconnect feature test for the `StompInboundChannelAdapterWebSocketIntegrationTests`, closing and then refreshing again the `serverContext`
2015-11-13 14:19:18 -05:00
Artem Bilan
febea4fe40 MQTT: Document Java Config
http://stackoverflow.com/questions/33655970/eclipse-paho-mqtt-spring-java-configuration
2015-11-12 12:12:12 -05:00
Gary Russell
f5fa979681 MQTT: Possible NPE
Examining the code revealed a possible (but improbable) NPE.

Checking the connection was disjoint from the publish; encapsulate the check within publish.
Synchronize the connectionLost method so it can't null the client while it is being checked.

Add `@SuppressWarnings("deprecation")` on the deprecated `connectIfNeeded()` usage.
2015-11-09 18:29:14 -05:00
Gary Russell
b486247105 INT-3878: MQTT Application Events (inbound)
JIRA: https://jira.spring.io/browse/INT-3878

Publish `ApplicationEvent`s to report inbound channel adapter connection
status.

Fixing typos and simple polishing.
2015-11-05 17:39:08 -05:00
Gary Russell
814698fbb9 INT-3617: ZK Leader Event Processing
JIRA: https://jira.spring.io/browse/INT-3617

Start/stop `SmartLifecyle` beans with leader election/revocation.

ZK Namespace Support

Add Annotation Support

INT-3617: Polishing

Add `SmartLifecycle` to listener parser.

INT-3617: Polishing; PR Comments

Revert SmartLifecycleRoleController

Remove reflection.

Remove Dependence on spring-cloud-cluster

Temporarily move the relevant classes here.

Polishing; PR Comments and Fix Test

Test was incorrectly stopping the LeaderInitiator before it was elected.
Set auto-startup="false" in the i-c-a so we wait for its start before
stopping the LeaderInitiator.

Fix JavaDocs
2015-07-06 15:54:25 -04:00
Artem Bilan
2bde14b742 INT-3661: Fix the eager BF access from BPPs (P I)
JIRA: https://jira.spring.io/browse/INT-3661

Previously there were a lot of noise from the `PostProcessorRegistrationDelegate$BeanPostProcessorChecker` for early access for beans.
That may produce some side-effects when some of `BeanFactoryPostProcessor`s won't adjust those beans.

The issue is based on two facts:
1. Loading beans from `BPP`, e.g. `IntegrationEvaluationContextAwareBeanPostProcessor` (or `ChannelSecurityInterceptorBeanPostProcessor` - https://jira.spring.io/browse/INT-3663)
2. Loading beans from `setBeanFactory()/setApplicationContext()` container methods

* Move all code from `setBeanFactory()` with access to the `BeanFactory` (e.g. `this.messageBuilderFactory = IntegrationUtils.getMessageBuilderFactory(this.beanFactory);`)
to some other lazy-load methods like `getMessageBuilderFactory()`
* Fix parser tests to remove `messageBuilderFactory` tests since there is no activity for target components to lazy-load them
* Polish some test according the new lazy-load logic
* Rework `IntegrationEvaluationContextAwareBeanPostProcessor` to the `SmartInitializingSingleton` and make it `Ordered`
* Populate `beanFactory` for the internal instance of `connectionFactory` in the `TcpSyslogReceivingChannelAdapter`
* Populate `beanFactory` for the internal `UnicastReceivingChannelAdapter` in the `UdpSyslogReceivingChannelAdapter`
* Add `log.info` that `UdpSyslogReceivingChannelAdapter` overrides `outputChannel` for the provided `UnicastReceivingChannelAdapter`
* Change the internal `MessageChannel` in the `UdpSyslogReceivingChannelAdapter` to the `FixedSubscriberChannel` for better performance

* Fix `AbstractExpressionEvaluator`
* Add JavaDocs for the `IntegrationEvaluationContextAware`

Fix `MongoDbMessageStoreClaimCheckIntegrationTests`

Addressing PR comments
2015-03-02 18:23:44 -05:00