Commit Graph

443 Commits

Author SHA1 Message Date
Gary Russell
4ac3a79df7 checkstyle FinalClassCheck
fixes

fixModifiers after fixFinal

Revert CachingSessionFactory

Class is spied in tests.

checkstyle - Import Rules

checkstyle InterfaceIsType

checkstyle InnerTypeLast

checkstyle OneStatementPerLine

CovariantEquals
OneTopLevelClass

* Revert `'\n'` -> `System.lineSeparator()` in the Gradle scripts to meet Git `autocrlf = true` on Windows
* Fix timing issue with the `LastModifiedFileListFilterTests`, when the `age = 1` might not be enough for the file object when we have some delay before checking
2016-04-04 13:49:56 -04:00
Gary Russell
43af472c3a checkstyle HideUtilityClassConstructor
https://sonar.spring.io/issues/search#componentRoots=org.springframework.integration%3Aspring-integration|createdAt=2016-04-01T10%3A00%3A36%2B0000|sort=UPDATE_DATE|asc=false
2016-04-01 16:18:28 -04:00
Gary Russell
4bfcdb9dfa INT-3944: Async JMS Outbound Gateway
JIRA: https://jira.spring.io/browse/INT-3944

Polishing (PR comments) and Doc Polish

Polishing - PR Comments

Remove custom `async` boolean in favor of the now public setter.

Add exception to `JmsException` hierarchy.

INT-3944: Polishing

* Rename `asyncReplySupported` just to `async`. As well as its getter and setter. Plus fix docs on the matter
* Polishing for the `JmsOutboundGateway` according PR comments
* Rework `JmsOutboundGateway` to return `AbstractIntegrationMessageBuilder` instead of `Message`
since `AbstractMessageProducingHandler` rebuilds `Message` for a new one to copy headers from request
* Add `getPayload()` and `getHeaders()` to the `AbstractIntegrationMessageBuilder` to make the `Routing Slip`
users happy, when the `AbstractIntegrationMessageBuilder` is pushed to the `Routing Slip path` function
* Fix race condition in the `ChatMessageListeningEndpointTests`: the `stanza` parsing is done in the different Thread.

Doc Polishing (Routing Slip)

Fix timing issue in the `LastModifiedFileListFilterTests`:
`Thread.sleep()` not always reflects the reality.
Switch to the "past simulation" via explicit `File.setLastModified()`
2016-03-25 16:44:35 -04:00
Gary Russell
516846b750 INT-3960: File Outbound - Preserve Timestamp
JIRA: https://jira.spring.io/browse/INT-3960

Polishing - PR Comments

Fix errors according Checkstyle report.
Wrap `warn()` to `isWarnEnable()`.
Polishing for `setPreserveTimestamp()` JavaDocs.
2016-03-23 15:00:40 -04:00
Gary Russell
ad0839da8b INT-3973: SFTP - Support chmod
JIRA: https://jira.spring.io/browse/INT-3973

Add `chmod` to outbound adapter and gateway (put methods).

Polishing - PR Comments

Fix Checkstyle vulnerabilities
2016-03-23 12:28:17 -04:00
Artem Bilan
2b0598291c RequireThis rule and fixThis Gradle task
* `gradlew clean check -x test --parallel --continue` - to collect reports
* `gradlew fixThis --parallel` - to fix all possible vulnerabilities. With `-Dfile.encoding=UTF-8` on Windows

Since the `RequireThisCheck` doesn't see parents for anonymous classes (e.g. `Runnable` callback), its report doesn't contains the outer class name with `this.`,
therefore we still have to fix those cases manually.
Thanks to the wrong `replacer` just with `this.` we have uncompilable code enough easy to find problems.
Not so easy to fix for good readability though...

* Upgrade to Grade 2.12
* Upgrade to SonarQube native plugin

The fix contains at about 300 files. So, will be done on merge.

Fix `fixThis.gradle` according PR comments

Apply `fixThis` and also `fixModifiers` for test classes.
 Fix some `this.` inner issues manually.
 Make code polishing for long lines after `fixThis`

Fix conflicts and vulnerabilities after the rebase
2016-03-22 20:18:27 -04:00
Artem Bilan
66cd92a6e7 INT-3964: More @EnableIntegration Documentation
JIRA: https://jira.spring.io/browse/INT-3964

Doc Polish

XSD: document classes for element implementations

Fix Redis XSD typo for CDATA
2016-03-22 10:41:07 -04:00
Artem Bilan
c2954881ad Enable RedundantModifier checkstyle, add fixer
Add `fixModifiers.gradle` to run after `check` task.

The `RedundantModifierChecker` doesn't catch all errors at once.
We need run `check -> fixModifiers` pair several times to reach `SUCCESSFUL`.

The `fixThis` has been ported from the SA, but without applying.

* Polishing for `fixModifiers.gradle`
* Fix all redundant modifier with the `fixModifiers.gradle`

NOTE: Since all these task modify files on Windows we have to run them with the `-Dfile.encoding=UTF-8`.
Otherwise they are read and write with the Windows default `cp1251` making them incompatible with Unix system.
2016-03-17 13:10:05 -04:00
Artem Bilan
c9ef27ae87 Revert UTF-8 Symbols in source code
https://build.spring.io/browse/INT-MJATS41-569/

The `fixHeaders` Gradle task (and similar) must be run from Windows with the `-Dfile.encoding=UTF-8`.
Otherwise read/write operations for files are done in the native Windows `cp1251`
2016-03-10 10:24:11 -05:00
Artem Bilan
ef40a939cf Add check header rule and fixHeaders task
Add "new line" in the header end

* Optimization for the `fixHeaders.gradle`
* Apply the `fixHeaders` for the affected classes
2016-03-07 15:08:51 -05:00
Gary Russell
5fe827a3dd Checkstyle - Phase I - Unused Imports
I copied the checkstyle config from spring-boot and commented
out most of the rules.

Over time, we should uncomment each rule and fix violations.
2016-03-04 15:50:08 -05:00
Artem Bilan
82c2da8dc4 Move BigMGetTests out of the main
Even it isn't harmful, the one useless class in the `.file.test` package looks confusing.

Move it out to the `test` scope and change the dependant projects respectively.
2016-02-24 16:59:40 -05:00
Artem Bilan
7595e4f142 INT-3919: FTP: Allow null for Remote Directory
JIRA: https://jira.spring.io/browse/INT-3919

Since `FtpClient` supports `null` for the `LS` command, treating it as a current `working directory`,
there is no reason to forbid `null` from the FTP adapters end-user perspective.

* Allow `null` for the `FtpSession` `list()` and `listNames()` methods
* Allow `null` in the `remote-directory` for the `<int-ftp:inbound-channel-adapter>`
* Allow `null` in the `expression` for the `FtpOutboundGateway`

Polishing - send error if error on async output

Cover `onFailure()` from `onSuccess()` with the `errorChannel`

Address PR Comments

* Get rid of `null` population for the `remoteDirectoryExpression` in the `AbstractPollingInboundChannelAdapterParser`
* Populate `new LiteralExpression(null)` from the `FtpInboundFileSynchronizer` ctor
* Introduce `buildRemotePath(parent, child)` function in the `AbstractRemoteFileOutboundGateway` with the `null` logic for `parent`
* Rework `mGetWithoutRecursion()` to use `LS` command and allow `null` for the dir.
* Fix tests according the new `mGetWithoutRecursion()` logic

Polishing
2016-02-16 14:45:22 -05:00
Artem Bilan
51f319bd4f INT-3949: FileWritingMH: use Files.move()
JIRA: https://jira.spring.io/browse/INT-3949
2016-02-11 15:48:19 -05:00
Gary Russell
baa8fdc4f5 INT-3574: Polishing
Synchronize state clearance when message received while stopped.

Make `DefaultFlushPredicate` `static`.
2016-01-15 11:04:37 -05:00
Gary Russell
c7724c340c INT-3574: File Outbound - Don't Flush File
JIRA: https://jira.spring.io/browse/INT-3574

Initial commit.

Polishing

Prevent the flusher task from closing while a write is in process.

Add MessageTriggerAction

Flush file(s) on demand.

Polishing - PR Comments

Add FlushPredicate

Polishing - PR Comments

Polishing - More PR Comments and Schema

Polish and Namespace

Docs

flushIfNeeded Must be Synchronized

Fix Race on Stop

Flush immediately after the write if the handler has been stopped.

Remove states from the internal store if handler is stopped.
2016-01-14 17:48:14 -05:00
Ryan Barker
201f0fc2b1 INT-3846: SimpleMessageStore Improvements
JIRAs:
https://jira.spring.io/browse/INT-3830
https://jira.spring.io/browse/INT-3523
https://jira.spring.io/browse/INT-3846

* Fix the OOM condition, when we `release()` `UpperBound` independently of the previous `remove` result (https://jira.spring.io/browse/INT-3846)
* Fix "confuse" around `groupCapacity`, when we really didn't care about individual groups (https://jira.spring.io/browse/INT-3523)
* Add `upperBoundTimeout` to have a hook to wait some time for the empty slot in the store (https://jira.spring.io/browse/INT-3830)
* Fix some JavaDocs warnings
* Fix some typos

* Fix inconsistency in the `DelayHandler` around `removeMessageFromGroup` when `MS` is `SimpleMessageStore`
 * Remove `UpperBound.release()` operation from `SimpleMessageStore.removeGroup()`.
 The waiting process should worry about the new `UpperBound` instance.

Some other polishing

`tryAcquire` outside of the `lock`

Move `tryAcquire` within the `addMessageToGroup` outside of `lock`.
But do that only for groups which already exist.
For the new groups we have a fresh `UpperBound`, so no need to worry about dead lock and
 we can obtain  a permit immediately.

SimpleMessageGroup: BlockingQueue -> LinkedHashSet

`SimpleMessageStore`: use "unsynchonized" `SimpleMessageGroup`

Make some synchronization fixes according to the migration to the `LinkedHashSet`

Avoid extra `Collection`

`ResequencingMessageHandler`: compare `size()` of collections instead of `containsAll()`

Fix `ConcurrentModificationException` in the `AbstractKeyValueMessageStore`

Add `SimpleMessageStore.clearMessageGroup()`

Accept polishing and fix `RedisChannelMessageStoreTests`

`@Deprecated` `MessageGroupStore.removeMessageFromGroup()`

Fix some typos

Introduce `SimpleMessageGroupFactory`

Extract `MessageGroupFactory` and address PR comments

Polishing after rebase

JavaDocs and Reference Manual

Fix JavaDocs
2016-01-12 11:51:05 -05:00
Artem Bilan
2be12160e9 INT-3726: Support Path for FileHeaders.FILENAME
JIRA: https://jira.spring.io/browse/INT-3726

Since the `FileHeaders.FILENAME` header is relaxed String, we can use any value there.
For example some use-cases (e.g. unzipping) would like to use the relative path together with the file name
to restore the original directory structure in the target directory using `FileWritingMessageHandler`.

Use `resultFile.getParentFile().mkdirs()` in the `FileWritingMessageHandler` to recreate the directory hierarchy before the target file manipulating.

Add Documentation

Address PR comments
2016-01-05 17:08:03 -05:00
Gary Russell
b5ec73b638 INT-3915: Possible Memory Leak in FileChannelCache
JIRA: https://jira.spring.io/browse/INT-3915

Close the redundant `FileChannel` when Map collision occurs.
2015-12-15 19:53:15 -05:00
Artem Bilan
cf528c0b5d INT-3665: Remove Deprecations and Resolve Issues
https://jira.spring.io/browse/INT-3665

Fixes according Travis report

Introduce `...ExpressionString(String)` setter

Some further fixes and polishing

Address PR comments
2015-12-08 16:48:29 -05:00
Gary Russell
d16cd9748c INT-799: Remove Core Deps from s-i-test
JIRA: https://jira.spring.io/browse/INT-799

Some test classes (e.g. `TestUtils`) were duplicated in core to avoid cyclic
dependency.

Now that core messaging has been moved to spring-messaging, it is possible
to remove the dependencies on `spring-integration-core` from `spring-integration-test`.

A few minor test cases have been moved to `spring-integration-core`.

The simple polishing to the `build.gradle` and `ServiceActivatorOnMockitoMockTests`
2015-12-07 20:39:00 -05:00
Gary Russell
255247ca9a Code Polishing (Sonar)
- remove redundant modifiers
- overridable methods called from ctors

Polishing - PR Comments
2015-12-07 15:39:13 -05:00
Gary Russell
9fff767f71 INT-3900: MQTT: Don't Unsubscribe if !cleanSession
JIRA: https://jira.spring.io/browse/INT-3900

The message-driven adapter should not unsubscribe if `cleanSession` is
`false`.

INT-3900: MQTT: Changes for 4.3 Only

Doc Polishing
2015-12-07 14:10:24 -05:00
Gary Russell
051b393ed0 Prepare Master for 4.3
Reference to 4.3

Refactor `what's new` for 4.3
2015-11-17 12:12:49 -05:00
Artem Bilan
9d8e2b61f6 Fixes for Sporadic Test Failures
* `ClientWebSocketContainer`: add some synchronization to avoid race conditions: https://build.spring.io/browse/INT-B41-492
* `TomcatWebSocketTestServer`: convert to `0` port to rely on the OS resolution for `localPort`
* Add `LogAdjustingTestSupport` for STOMP test
* `SftpServerTests`: use `0` port to rely on the OS resolution for `localPort`
* `ImapMailReceiver`, `OutboundGatewayFunctionTests` (JMS), `CachingClientConnectionFactoryTests`,
`AsyncGatewayTests`, `AsyncMessagingTemplateTests`, `GatewayParserTests`, `PriorityChannelTests`, `AggregatorIntegrationTests`: increase timeout
* `EnableIntegrationTests`: use `LogAdjustingTestSupport`
* `FileOutboundChannelAdapterParserTests`: rework `Thread.sleep()` with `CountDownLatch`
* `ConnectionToConnectionTests`: increase timeout and count iteration. Previously with `1sec` we may lose some events. And we can't just rely on the `10sec`,
because the last iteration will be so long
* `TcpOutboundGatewayTests`: `500ms` is so big timeout to wait for the `Exception` that in the high load environment we can yield to other Thread so long.
Like in our case to `server` Thread to send the reply for us. Therefore decrease the Exception timeout to the `50ms` and increase server delay to `2sec`
* `StompInboundChannelAdapterWebSocketIntegrationTests`: remove `@Qualifier("taskScheduler")` as a potential candidate to test against latest SF changes.
We're fine with `SF-4.2.2` and it is just a test-case. So, I don't see reason to wait for their fix here.

STOMP: `session = null` in adapters for any transportError

Polishing
2015-11-14 16:12:23 -05:00
Gary Russell
81cd1e1ad9 Increase File Test Timeouts 2015-11-10 12:57:49 -05:00
Artem Bilan
7ef053da22 Increase timeouts for FileToChannelIntTests
https://build.spring.io/browse/INT-B41-JOB1-460
2015-10-26 16:37:39 -04:00
Gary Russell
b6467e1329 INT-3849: Fix File Inbound Adapter Regression
JIRA: https://jira.spring.io/browse/INT-3849

A file inbound adapter with just a `filter` attribute (no implied filter attributes)
did not get its filter injected.

The parser took an early exit from the filter configuration if none of the implied
filter attributes were specified.

Work around is to explicitly set `ignore-hidden="true"`.
2015-10-09 11:03:02 -04:00
Gary Russell
0d721739e9 INT-3827: ResettableFileListFilter
JIRA: https://jira.spring.io/browse/INT-3827

Provide a hook to enable removing a file from an `AcceptOnceFileListFilter`,
for example after a message processing failure.

Make the `CompositeFileListFilter` a `ReversibleFileListFilter` so it can
delegate to any of its composed filters that are reversible.

INT-3827: Polishing - PR Comments
2015-09-29 13:59:51 -04:00
Yi EungJun
a9f50d32e4 Add InputStream to FileWritingMH JavaDoc
InputStream has been supported since 247232bdde.
2015-09-22 11:55:19 -04:00
Artem Bilan
d8fc94602c INT-3820: Fix MethodInvokerHelper for the Proxy
JIRA: https://jira.spring.io/browse/INT-3820

Some Proxy may be based on the specif non-user classes and advised with
the provided end-user interfaces.
The best sample is `@Repository` from Spring Data projects.
In this case the `MessagingMethodInvokerHelper` just missed the user interfaces
to consider for the candidate methods or thrown an `Exception` like `NoSuchMethodError`.

* Add `((Advised) targetObject).getProxiedInterfaces()` for the scanning algorithm
* Move the `UniqueMethodFilter` to the internal `Set<Method>` to allow iteration
and filtering for methods from the `targetClass` as well as from all those user interfaces
* Add Spring JPA repository test-case
* Polishing form some time-weak tests

Revert MessagingMethodInvokerHelper

INT-3820: Simple Proxy Interface Method Matching

No-risk solution for INT-3820.

Instead of considering all interfaces on the proxy, only look at proxy interfaces
for a single matching method name if we find no candidate or fallback methods.

Consider the complete solution for 4.3.

Add `Message<?>` method distinguishing
2015-09-09 14:33:38 -04:00
Artem Bilan
f607f83bf2 INT-3804: WatchServiceDirScan: Add OVERFLOW Event
JIRA: https://jira.spring.io/browse/INT-3804

INT-3804: Polishing

Fix race conditions.
2015-09-04 17:07:46 -04:00
Artem Bilan
057c004e6c INT-3619: Prevent scanner Options Overrides
JIRA: https://jira.spring.io/browse/INT-3619

Previously the `FileReadingMessageSource` allowed overriding `filter` and `locker` properties
on the `scanner` through its own setters.
It caused confusing and unexpected behavior.

Provide refactoring to check that provided `filter` and `locker` are applied for an internal `scanner`
instance, not for the injected one.

PR Comments
2015-08-12 16:59:35 -04:00
Artem Bilan
e408331e67 INT-3622: (S)FTP: Add MessageSessionCallback
JIRA: https://jira.spring.io/browse/INT-3622

INT-3622: Polishing
2015-08-11 09:33:34 -04:00
Artem Bilan
0f689558ab INT-3792: Fix Some SONAR Vulnerabilities
JIRA: https://jira.spring.io/browse/INT-3792

* Remove `Serializable` requirement for `ThreadStatePropagationChannelInterceptor` generic type
* Increase `PriorityChannelTests` timeout
2015-08-05 14:20:08 -04:00
Artem Bilan
ec12b289c2 INT-3786: Fix Several Sporadic Test Failures
JIRA: https://jira.spring.io/browse/INT-3786

* Increase `receiveTimeout` for several `QueueChannel` and `CountDownLatch` based tests
* Fix `FileSplitterTests` for Windows compatibility - `UTF-8` for bytes conversion
* Make some STOMP tests as `LongRunning`

The next fixing phase
2015-07-30 15:28:02 -04:00
Gary Russell
13b4facc7a INT-3491: (S)FTP Delegation Session Factory
JIRA: https://jira.spring.io/browse/INT-3491

Select the session factory at runtime.

Polishing; PR Comments

Also add/remove factories.
2015-07-30 12:39:55 -04:00
Gary Russell
9e0b2fb319 INT-3593: (S)FTP OG Partial Updates (mget/mput)
JIRA: https://jira.spring.io/browse/INT-3593

Throw a `PartialSuccessException` if an exception occurs after partial success (some files transferred).
2015-07-20 12:10:33 -04:00
Gary Russell
00540cb5df INT-3229: Fix WatchService Test
JIRA: https://jira.spring.io/browse/INT-3229
2015-07-16 15:09:59 -04:00
Gary Russell
1dba450e8c INT-3773: (S)FTP RemoteFileTemplate Fail/Ignore
JIRA: https://jira.spring.io/browse/INT-3773

When the `FileExistsMode` was `FAIL` or `IGNORE` and the remote file
did NOT exist, the file was not written.
2015-07-15 13:44:00 -04:00
Gary Russell
c986206059 INT-3229: Add WatchServiceDirectoryScanner
JIRA: https://jira.spring.io/browse/INT-3229

Polishing
2015-07-15 10:54:59 -04:00
Gary Russell
583d432f22 INT-3764: LastModifiedFileListFilter
JIRA: https://jira.spring.io/browse/INT-3764

* Add JavaDocs to the `LastModifiedFileListFilter`
* Add `Namespace Support` to the `file.adoc`
2015-07-15 09:54:17 -04:00
Gary Russell
c68c7b3412 INT-3763: (S)FTP - Inbound Remote Dir. Expression
JIRA: https://jira.spring.io/browse/INT-3763

Fix `AbstractRemoteFileSynchronizerTests` according to this change.
2015-07-09 18:26:46 -04:00
Gary Russell
cef99f3584 INT-2507: File: Fix HeadDirectoryScanner
JIRA: https://jira.spring.io/browse/INT-2507

When using a `HeadDirectoryScanner`, the use of any filter removed the
size limitation passed into the contstructor (or `queue-size` attribute
of a file inbound channel adapter.

The `HeadFilter` was overwritten.

Combine the head filter into a `CompositeFileListFilter` (creating one
if necessary).
2015-06-30 12:13:55 -04:00
Artem Bilan
9c96a14cd1 INT-3749: Revert setters for EC
JIRA: https://jira.spring.io/browse/INT-3749

Since some target application could inject there is `EvaluationContext` before, removal of
 `setIntegrationEvaluationContext()` is breaking change.

 This fix reverts all those setters.
2015-06-27 12:36:33 -04:00
Artem Bilan
3392d4e1ab INT-3749: Get rid of IECA logic
JIRA: https://jira.spring.io/browse/INT-3749

Since `IntegrationEvaluationContextAware` isn't so "context-free" resource like `BeanFactory`
 and `ApplicationContext`, but just a specific bean in the context, we can't follow with `BeanPostProcessor` logic - bad architecture by level of responsibility.
 Therefore we should follow with standard Dependency Injection mechanism to retrieve `evaluationContext` for the particular component.

 Since we can't rely on the `@Autowired` because SI can be used from the raw XML configuration,
 we use utility method instead. The pattern to get the proper `integrationEvaluationContext` is:
 ```
 @Override
 protected void onInit() throws Exception {
 		super.onInit();
 		this.evaluationContext = ExpressionUtils.createStandardEvaluationContext(getBeanFactory());
 }
 ```
2015-06-24 14:57:00 -04:00
Gary Russell
f4104c199a INT-3745: More TCP Events (Failed Correlation)
JIRA: https://jira.spring.io/browse/INT-3745

Emit events whenever a message is received that can't be correlated to
a connection (or request in the case of an outbound gateway).

Also, when sending messages, wrap the exception in a `MessagingException` so that
the `TcpConnectionExceptionEvent` provides access to the failed message.

INT-3745: Polishing - PR Comments

Fix JavaDocs errors
2015-06-19 13:49:10 -04:00
Gary Russell
65d2024937 INT-3737: (S)FTP Outbound Gateway Streaming GET
JIRA: https://jira.spring.io/browse/INT-3737

Support returning an `InputStream` from a GET operation.

This can be used in conjuction with a `<file:splitter/>` to stream a text file.

The user is responsible for releasing the session when the download is complete.

The session object is stored in the message header and `RemoteFileUtils.closeSession()` should be called
to clean up and close the session.

INT-3737: Polishing and Docs

Fix `CachedSession.close()` bug

Preventing `Caused by: java.io.IOException: Previous raw read was not finalized`
when `Session` is returned to the pool, but `readingRaw` hasn't been finalized
because the real `close()` isn't invoked in case of normal return to pool.
2015-06-19 13:39:36 -04:00
Gary Russell
3d5f7db4b2 INT-4740: FileSplitter - Add Headers
JIRA: https://jira.spring.io/browse/INT-3740

For `File` and `String` payloads add `FileHeaders.ORIGINAL_FILE` and `FileHeaders.FILENAME` headers.

Reworked to create headers once only.

Fix typos and Java > 6  API usage
2015-06-18 13:55:52 -04:00
Gary Russell
6436aa6d32 INT-3600: FileSplitter Namespace and Docs
JIRA: https://jira.spring.io/browse/INT-3600

Add `<int-file:splitter/>` namespace component.
Add `FileSplitter` documentation.

Polishing
2015-06-18 13:45:16 -04:00