Commit Graph

598 Commits

Author SHA1 Message Date
Artem Bilan
1d54f9663a GH-2760: use assertThatExceptionOfType in tests
Fixes https://github.com/spring-projects/spring-integration/issues/2760

* The `assertThatExceptionOfType()` is more convenient,
than `assertThatThrownBy`, so, replace all the usages accordingly
* Fix JavaDoc typo in the `AbstractScriptExecutingMessageProcessor`
* Fix Sonar smells for `throws Exception` in `AbstractScriptExecutingMessageProcessor`
hierarchy and some code polishing for them
2019-02-22 13:54:36 -05:00
Artem Bilan
221393e02f MessageHandlingException consistency
* Use `IntegrationUtils.wrapInHandlingExceptionIfNecessary()` whenever
it is possible to avoid double wrapping into the `MessageHandlingException`
* Some code polishing for affected classes `@Nullable`, streams, diamonds etc.
2019-02-22 09:43:59 -05:00
Artem Bilan
7e3b56939e Fix XSDs for actual core schema version 2019-02-21 12:43:21 -05:00
Artem Bilan
622d42c71a Migrate tests to AssertJ
Mostly thanks to IDEA's plugin: https://plugins.jetbrains.com/plugin/10345-assertions2assertj
There is still a lot of work to do when complex and composite matchers are used.

* Add `awaitility` dependency and deprecate `EventuallyMatcher` in favor
of `awaitility`
* Remove Hamcrest from dependencies and disable JUnit & Hamcrest
static imports to encourage to use only AssertJ
* Migrate JUnit assumptions in rules to AssertJ's assumptions
* Deprecate some custom matchers in favor of existing in Hamcrest
after upgrading the last to version `2.1`
* Replace `ExpectedException` rules with `assertThatThrownBy()`
* Mention `MessagePredicate` in the `testing.adoc`
2019-02-21 10:20:45 -05:00
Gary Russell
520ebbbd2c Move master to 5.2 2019-02-15 12:01:24 -05:00
Artem Bilan
020ea76d59 Make IOS.setTaskScheduler() as public (#2725)
* Make IOS.setTaskScheduler() as public

* Make `IntegrationObjectSupport.setTaskScheduler()` as `public` and
remove all the overrides for visibility
* Fix Sonar smells for all the affected classes

* * Fix `throws Exception` for affected classes to avoid compilation errors

* * Fix "merely rethrow" smell
* Revert `throws Exception` for `AbstractEndpoint.destroy()`

* * Catch a couple exceptions from `destroy()`
2019-01-30 13:59:37 -05:00
Artem Bilan
a609bd9398 Some file module Sonar fixes (#2718)
* Some file module Sonar fixes

* * Fixes after testing

* * Some further Sonar fixes for file module

* * Further Sonar fixes for file module

* * More Sonar fixes for file module

* * Fix `RemoteFileTemplate` issues

* * Fix new sonar smells after previous one

* * Review feedback
2019-01-25 14:37:57 -05:00
Gary Russell
4ea3b1c23c GH-2705: public CTORs on AMFPFileListFilter Impls
Fixes https://github.com/spring-projects/spring-integration/issues/2705
2019-01-18 11:12:38 -05:00
Artem Bilan
1d5e66b0ee RemoteFileTemplate: Fix Checkstyle violation
* Add `if (session != null)` in the `RemoteFileTemplate` around `dirty()`
2019-01-10 11:05:18 -05:00
Alen Turkovic
6ddcfb3a8d Custom Session dirty logic
Fixes spring-projects/spring-integration#2687

Allows any session to implement its dirty logic.

* License year was not updated

**Cherry-pick to 5.0.x**
2019-01-10 10:52:45 -05:00
Gary Russell
e40cfe101e Sonar fixes
Hidden fields, `o.s.i.a*` through `o.s.i.j*`.
2019-01-09 10:02:19 -05:00
Gary Russell
1bafe89d49 Sonar Fixes
- avoid parameter assignments
2019-01-07 11:35:58 -05:00
Gary Russell
e8df546b04 Sonar Fixes - Javadoc-only imports 2019-01-02 12:26:22 -05:00
Gary Russell
113a371f2c Remove Exception from IOS.onInit() 2018-12-22 17:25:56 -05:00
Artem Bilan
81b4ea1bef Sonar fixes according latest report (#2676)
* Sonar fixes according latest report

* Fix initialization order in the `GatewayMethodInboundMessageMapper`

* Fix mock in the `DelegatingSessionFactoryTests`

* Fix `AbstractRemoteFileOutboundGateway.listFilesInRemoteDir`

* * PR comments

* * Fix `AbstractRemoteFileOutboundGateway.listFilesInRemoteDir` complexity
2018-12-22 11:57:35 -05:00
Artem Bilan
93d7c58b64 Enable ModifierOrderCheck Checkstyle rule (#2673)
* Enable ModifierOrderCheck Checkstyle rule

* Fix violations for `static` and `abstract` modifier
* Remove redundant code in the `TcpNioConnection`
* Mark `connectionFactoryName` as `@Nullable` in the `TcpConnectionSupport`
ctor and its inheritors
* Fix some smells according IDEA suggestions in the affected classes
* This should fix some Sonar smells as well

* * Fix `HeaderMapperTests`

* * Polishing `TcpConnection` code style and fix Javdocs
2018-12-20 19:19:47 -05:00
Gary Russell
62fc7df693 Sonar Fixes
Critical smells, packages `o.s.i.a*` to `f*`.

Plus new smells caused by these changes.
2018-12-03 10:44:26 -05:00
Gary Russell
39fef2c70a INT-4560: Fix Race in FileSystemPersistentAOFLF
JIRA: https://jira.spring.io/browse/INT-4560

Reproduced and tested with

```java
@SpringBootApplication
public class So53521593Application {

	private static final Logger logger = LoggerFactory.getLogger(So53521593Application.class);

	public static void main(String[] args) {
		SpringApplication.run(So53521593Application.class, args);
	}

	@Bean
	public IntegrationFlow flow() {
		ExecutorService exec = Executors.newFixedThreadPool(10);
		return IntegrationFlows.from(Files.inboundAdapter(new File("/tmp/foo")).filter(
					new MyFilter(new SimpleMetadataStore(), "foo")),
						e -> e.poller(Pollers.fixedDelay(5, TimeUnit.SECONDS)
								.maxMessagesPerPoll(10)))
				.channel(MessageChannels.executor(exec))
				.<File>handle((p, h) -> {
					try {
						p.delete();
						logger.info(p.toString());
						Thread.sleep(10_000);
					}
					catch (InterruptedException e1) {
						Thread.currentThread().interrupt();
					}
					return null;
				})
				.get();
	}
}

class MyFilter extends FileSystemPersistentAcceptOnceFileListFilter {

	public MyFilter(ConcurrentMetadataStore store, String prefix) {
		super(store, prefix);
	}

	@Override
	protected long modified(File file) {
		long modified = super.modified(file);
		System.out.println(modified);
		return modified;
	}

}
```

**cherry-pick to 5.0.x, 4.3.x**
2018-11-28 14:18:09 -05:00
Artem Bilan
648aeb8e96 Unable ImportOrder Checkstyle rule for tests
* Fix wrong import order in tests
* Polishing for not closed application contexts
* Use diamonds
* Use proper assertion operators
* Remove redundant classes
* Improve performance in JPA tests
* Fix typos
2018-11-07 15:32:31 -05:00
Gary Russell
12bc24dd02 File, FTP, IP Sonar Fixes 2018-10-26 13:53:54 -04:00
Artem Bilan
674d5adfc0 INT-4114: Add missed <poller> to XSDs
JIRA: https://jira.spring.io/browse/INT-4114
2018-10-26 13:51:18 -04:00
Gary Russell
a4f779d09e INT-4547: (S)FTP RFOG MPUT with collection payload
JIRA: https://jira.spring.io/browse/INT-4547

* PR Comments; use `MutableMessage` internally.
2018-10-24 15:33:29 -04:00
Gary Russell
bd7a3bc4ed GH-2605: (S)FTP test cached sessions
Resolves https://github.com/spring-projects/spring-integration/issues/2605

* Suppress unused field warning.

* Fix typos.

* Use lstat() - don't follow symbolic lincs
2018-10-23 13:59:00 -04:00
Gary Russell
0a7c77d853 INT-4534: Add remote directory to exception
JIRA: https://jira.spring.io/browse/INT-4534

Include the remote directory in exception and debug log messages when synchronizing.
2018-10-11 08:58:53 -04:00
Artem Bilan
41264110ce Some various polishing
* Add `RotatingServerAdvice.StandardRotationPolicy.getCurrent()` method
for better end-user experience when this class is extended
* Add NPE protection into the MQTT Channel Adapters.
Some code style polishing for them

**Cherry-pick to 5.0.x**
2018-10-10 15:21:58 -04:00
Artem Bilan
9e9fa2cf58 RemoteFTempl: InputStream.close() in the finally
If exception happens in the `callback.doWithInputStream(inputStream)`,
we don't close the `inputStream = session.readRaw(remotePath)`.

* Move the `InputStream.close()` to the `finally` block of the
`SessionCallback` action in the `RemoteFileTemplate.get()`

**Cherry-pick to 5.0.x and 4.3.x**
2018-10-05 12:17:50 -04:00
Steven
4f2372b02a FileReadingMS: Make an Assert message more clear
* Changed an Assert error message in the `FileReadingMessageSource` to make the requirements 
when using an external scanner clearer, and added some explanatory comments.

* Removed use of 'local' to eliminate confusion.

* Added javadoc addendum
2018-09-21 11:36:59 -04:00
Cengis K
b295fb8486 GH-2569: Add ChainFileListFilter constructors
* Adding ChainFileListFilter constructors to behave like CompositeFileListFilter

Fixes https://github.com/spring-projects/spring-integration/issues/2569

Adding constructors in `ChainFileListFilter` matching super, so this class behave 
like `CompositeFileListFilter` when using XML flow configuration.

* Fix import error

* Fix checkstyle

* Fix checkstyle

* Fix checkstyle

* Fix author name case
Remove unnecessary javadoc

* Fix code smell squid:S1155

* Add a simple test case for initializing filters with constructor
Fix code smell squid:S1192

* Add a simple test case for initializing filters with constructor
Fix code smell squid:S1192

* Fix missing import
2018-09-21 11:32:16 -04:00
Michael Forstner
5bfd6971db INT-4531: Use Object type KeyDirectory.key
JIRA: https://jira.spring.io/browse/INT-4531

The `DelegatingSessionFactory` uses plain `Object` as key type
whereas the default `RotatingServerAdvice` does not.

* Change the type from `String` to `Object` in the `RotatingServerAdvice.KeyDirectory`
* Fix typos in the (s)ftp.adoc

**Cherry-pick to 5.0.x**
2018-09-14 12:49:05 -04:00
Artem Bilan
e4ca8571e0 File Source: do not deal with dir until start()
StackOverflow: https://stackoverflow.com/questions/52273537/spring-boot-failed-to-load-applicationcontext
2018-09-12 10:13:45 -04:00
Gary Russell
5ab7d24934 Make Headers PREFIX Constants Public
**cherry-pick to 5.0.x**

* Make `IpHeaders` prefix as `public` as well
2018-08-27 16:44:49 -04:00
Artem Bilan
e053e2ab4a INT-4524: Fix MailReceivingMessageSource BF usage
JIRA: https://jira.spring.io/browse/INT-4524
2018-08-20 12:17:48 -04:00
Gary Russell
23a3462d8b Add @NonNullApi where appropriate
Packages with `@Nullable` annotations need to have `@NonNulApi` at the package level.

There may be classes in these packages that have nullable arguments/returns that are
not currently marked as `@Nullable`. These will be fixed over time.

Also suppress JSR305 warnings when compiling tests.

* Polishing - PR Comments
2018-08-01 16:53:42 -04:00
Artem Bilan
0d0605be78 INT-4381: MessageSources refactoring (#2517)
* INT-4381: MessageSources refactoring

JIRA: https://jira.spring.io/browse/INT-4381

* Make all the out-of-the-box `MessageSource` implementations based
on the `AbstractMessageSource`
* Fix `JdbcPollingChannelAdapterIntegrationTests` for sporadic failure
because of `fixed-rate` for the poller

* Fix HeaderEnricherTests race condition

The `errorChannel()` expect an error in the `testErrorChannel` after
`1000` ms, but at the same time the `poller` in configured for the
`3000` ms.

* Increase all the timeouts for replies
* Decrease a `fixed-delay` on the `poller`
* Some other code style polishing for the `HeaderEnricherTests`
2018-07-25 13:01:22 -04:00
Gary Russell
d6c8baf50f INT-4505: Fix package tangle
JIRA: https://jira.spring.io/browse/INT-4505
2018-07-24 16:30:10 -04:00
Artem Bilan
44b7df94c1 Polishing for the FileWritingMessageHandler
* Fix typos in docs
* Add `om.google.code.findbugs:jsr305` to avoid warnings like
`Cannot find annotation`
2018-07-24 15:50:56 -04:00
Alen Turkovic
9ac0746f19 INT-4502: Add new file callback in FileWritingMH
JIRA: https://jira.spring.io/browse/INT-4502

Documentation
Using a BiConsumer instead of a custom interface

Doc fixes

Calling callback while locked

More requested fixes

Missing `FileWritingMessageHandler.this` on `fileExistsMode`
2018-07-24 15:50:05 -04:00
Artem Bilan
76877b9440 INT-4499: Add JdbcMetadataStore.setLockHint()
JIRA https://jira.spring.io/browse/INT-4499
Fixes https://github.com/spring-projects/spring-integration/issues/2483

To allow to reconfigure a `SELECT ... FOR UPDATE` for particular RDBMS
vendor, we need an option to specify possible hint for the target data base.

**Cherry-pick to 5.0.x**
2018-07-11 17:52:44 -04:00
Artem Bilan
a1f016a94c INT-4498: StreamUtils -> FileCopyUtils
JIRA: https://jira.spring.io/browse/INT-4498

* Use `FileCopyUtils` instead of `StreamUtils` in test-cases when we deal
with `InputStream` s and files resources.
* Fix reflection-based tests with AOP to use an `ArrayList` access
instead of `LinkedList` since an optimization in SF:
6d6aa72e8f

**Cherry-pick to 5.0.x and 4.3.x**
2018-07-09 13:39:56 -04:00
Gary Russell
11d39b7da1 INT-4496: (S)FTP: Sort file list earlier
JIRA: https://jira.spring.io/browse/INT-4496

Sort the file list before applying `maxFetchSize` and filters.

* Polishing - PR Comments
2018-07-02 16:49:23 -04:00
Gary Russell
3f218a97a0 INT-4495: (S)FTP: fix max-fetch with directories
JIRA: https://jira.spring.io/browse/INT-4495

Previously, `maxFetch` was applied before directories were removed from
the fetch list.

Remove the directories before filtering and applying `maxFetch`.

* Polishing - PR Comments

* More polishing

* More polishing.

* Check for empty array.

* Remove test main method.
2018-06-29 14:21:06 -04:00
Gary Russell
26cfab579c INT-4491: (S)FTP inbound rotate dirs/servers
JIRA: https://jira.spring.io/browse/INT-4491

Add Rotating Server/Directory Polling Advice.

**cherry-pick to 5.0.x**

* Polishing - PR Comments.

* Polishing

* Polishing; revert `KeyDirectory`; WARN about `TaskExecutor` and `MessageSoureMutator`(s).

* More polishing - PR comments

* Apply stashed changes.

* Fix WARN log - the `SyncTaskExecutor` is wrapped.
2018-06-27 13:43:41 -04:00
Artem Bilan
ad4dc11758 GH-2478: RecursiveDScanner: Files.walk().close()
Fixes https://github.com/spring-projects/spring-integration/issues/2478

The `Files.walk()` stream must be closed in the end after usage

* Wrap `Files.walk()` to the `try-with-resources`

**Cherry-pick to 5.0.x**
2018-06-13 12:33:16 -04:00
Artem Bilan
85bc183849 INT-4486: Properly implement stop(Runnable)
JIRA: https://jira.spring.io/browse/INT-4486

The `SmartLifecycle.stop(Runnable callback)` must always call the
`callback` in the end independently of the internal state

* Revise all the `SmartLifecycle` implementations for the proper
`callback` handling

**Cherry-pick to 5.0.x and 4.3.x**
2018-06-11 17:51:20 -04:00
Artem Bilan
1f05d3ec4c Polishing some Docs and JavaDocs
* Mentioned `byte[]` support for the `UnmarshallingTransformer`
* Fix `AbstractInboundFileSynchronizingMessageSource` JavaDocs to
mention the current `FileSystemPersistentAcceptOnceFileListFilter`
usage by default, not an `AcceptOnceFileListFilter`
* Polishing `AbstractInboundFileSynchronizingMessageSource` code style
a bit
2018-05-16 16:10:50 -04:00
Artem Bilan
0ef659b8a7 INT-4351: Add DiscardAwareFileListFilter
JIRA: https://jira.spring.io/browse/INT-4351

The `WatchService` reacts to the events in the file system and keep
track ove the events until we poll them, e.g. via `listEligibleFiles()`
in the `WatchServiceDirectoryScanner`.
On the other hand the `SourcePollingChannelAdapter` calls the mentioned
`listEligibleFiles()` according its polling period.
At this moment the `FileListFilter` is applied to the polled files.
It may happen that `LastModifiedFileListFilter` can't accept too young
files yet and they are lost for the future consideration.

* To allow, for example, to retain young files by the
`LastModifiedFileListFilter` judgment for the future cycles add
`DiscardAwareFileListFilter` with the `DiscardCallback` support.
The `WatchServiceDirectoryScanner` now registers such a callback into
the filter and stores discarded files into the `filesToPoll` queue
for the future poll cycle.

**Cherry-pick to 5.0**

Fix compilation warnings

* Replace `DiscardCallback` with the plain `Consumer`
* Ensure uniqueness in the `WatchServiceDirectoryScanner` internal
queue via a `Set` implementation, since discard callback may be called
several times for the same file from the `CompositeFileListFilter`
according to its nature
* Add JavaDocs and Docs

Change `@since` to `5.0.5`
2018-04-24 14:10:15 -04:00
Artem Bilan
027021d6e4 Move XSDs to version 5.1
* Upgrade to Spring AMQP-2.1 and fix compatibility
2018-03-17 09:58:40 -04:00
Gary Russell
82f252dd00 INT-4390: Fix tangles
JIRA: https://jira.spring.io/browse/INT-4390

Phase I: flow context/registration cycle

- extract interfaces
- rename implementations to `Standard...`

Phase II - dsl<->dsl.channel tangles

Move the channel specs to dsl.

Fix missing refactorings

(Not related to DSL) - rename core `event` package to `events`

- avoid collision with event module package
- fix tangle caused by `MessageGroupExpiredEvent`.

Phase III - dsl<->config.dsl

- move classes from config.dsl to dsl

(Not related to DSL) - fix many tangles caused by graph being a sub-package of support.management
 - make `graph` a top-level package

(Not related to DSL) - move `IntegrationManagementConfigurer` from
`management` to `config` - tangle between core and management
* Polishing Copyrights, diamonds, some JavaDocs and What's New
2018-03-16 14:59:47 -04:00
Ruslan Stelmachenko
7f25cba262 INT-4430: FileSplitter close reader on exception
JIRA: https://jira.spring.io/browse/INT-4430

When Iterator-based `FileSplitter` splits the file, and an exception 
throws in downstream flow (in the same thread), the exception propagates 
to the caller leaving underlying file reader opened.

This commit changes `AbstractMessageSplitter` the way, that, 
when any exception happens, if Iterator implements `java.io.Closeable`, 
its `close()` method will be called before propagating exception.

Also `FileSplitter`'s underlying iterator implements `Closeable` now.

* Make `CloseableIterator` to follow `Closeable` contract.

Now `CloseableIterator.close()` declares `IOException` and can be used 
as base interface for `FunctionIterator`.

* Adjust tests.

Adjust tests to reflect the fact that we call `close()` on the reader 
one more time in the end of iterator.

**Cherry-pick to 5.0.x and 4.3.x**
2018-03-16 10:26:41 -04:00
Gary Russell
4552a80095 INT-4392: Fix tangles
JIRA: https://jira.spring.io/browse/INT-4392
JIRA: https://jira.spring.io/browse/INT-4391
JIRA: https://jira.spring.io/browse/INT-4393

Resolve class and leaf package tangles in:

- core (except DSL)
- file
- http

- move `StaticMessageHeaderAccessor` to root `integration` package
- move acknowledgment related classes to a new `acks` package
- move `MMIH` to `handler.support`, alongside argument resolvers, also `HandlerMethodArgumentResolversHolder`
- move `ErrorMessagePublisher` to `core`
- move `PatternMatchUtils` to `support.utils`
- move `MessageSourceManagement` to `support.management`
- remove direct reference to `MicrometerMetricsCapter` from `IntegrationManagementSupport`
- add new class `MetricsCaptorLoader`

- move `OperationsCallback` to inner interface in `RemoteFileOperations`

- move `HttpContextUtils` to `config`

* Simplify MetricsCaptor loading.

* Polishing
2018-03-13 11:22:56 -04:00