Commit Graph

918 Commits

Author SHA1 Message Date
Gary Russell
230d12d425 TCP, SSL, Enable Host Verification by default
Make key/trust store types configurable; add a test with host violation.

* Fix some typos and code style in the related classed and docs
* Add asserts for the store type properties
2018-08-29 16:31:45 -04:00
Gary Russell
075d237c04 INT-4527: Delayer Retries and Error Handling
JIRA: https://jira.spring.io/browse/INT-4527
Fixes https://github.com/spring-projects/spring-integration/issues/2543

- Add error handling within the scope of a transaction
- Add `retryDelay` and `maxAttempts`
- Include the delivery attempt header in the `ErrorMessage`
- Add `transactionalRelease()` methods to the DSL

Check context in `ContextRefreshedEvent`.

Complete test case; don't schedule for re-release after a successful send after a failure.

Use `identityHashCode` for deliveries map key - error flow might change the message `hashCode`.

Debug logging

Polishing; remove parameter from the `DelayerEndpointSpec.transactionalRelease()`

* Polishing some code style
2018-08-28 15:45:04 -04:00
Gary Russell
e6489abb52 INT-4526: Fix Channel Interceptor NonNullApi Call
JIRA: https://jira.spring.io/browse/INT-4526

- `Message<?>` cannot be null in `postReceive()`.

Polishing - PR Comments

* Polishing code style
* Fix `PollableJmsChannel`
* Increase receive timeout in the `PollableJmsChannel`
2018-08-27 17:07:14 -04:00
Artem Bilan
0545a7b0ff INT-4519: Improve JMS inbound-c-a receiveTimeout (#2540)
* INT-4519: Improve JMS inbound-c-a receiveTimeout

JIRA: https://jira.spring.io/browse/INT-4519

* Modify `DynamicJmsTemplate` to default for `-1` if we get
`CachingConnectionFactory` and it is with `cacheConsumers`, otherwise
1 second
* Polishing for some JMS XML parsers, in particular remove
`receiveTimeout` population

**Cherry-pick to 5.0.x**

* * Polishing according PR comments

* * Properly use a `CachingConnectionFactory` in `JmsTests`
2018-08-20 13:44:19 -04:00
Artem Bilan
26229c9862 INT-4480: Use component type for ids in the flow (#2536)
* INT-4480: Use component type for ids in the flow

JIRA: https://jira.spring.io/browse/INT-4480

For better readability of bean names, e.g. from the metrics collectors,
generate them from the component type instead of their fully qualified
class names

* * Fix bean name for injection in the WebFluxDslTests
2018-08-09 15:01:41 -04:00
Artem Bilan
055e9a40db INT-4457: Add logAndReply() terminal operator
JIRA: https://jira.spring.io/browse/INT-4457

When an `IntegrationFlow` is reply-based (we expect a reply in the
beginning) and `log()` (or `wireTap()`) is used in the end, we are
forced to add an empty `bridge()` in the end to ensure a `replyChannel`
header routing

* Fix `IntegrationFlowDefinition` to add `enrichHeaders()` in the end
to populate a `nullChannel` as a `replyChannel` header if that is missed
in the request message headers.
This way we cover both use-cases when we expect reply from the flow and
when it is used as a one-way scenario
* Improve a `HeaderEnricher` do not create a new `Message` if there are
no new headers to add/remove
* Remove a note from the `dsl.adoc` about now redundant `bridge()` after
`log()`
* Resolve TODO in the `.handle()` paragraph

* Restore anonymous class in the MessageHistoryIntegrationTests:
Mockito can't mock/spy lambdas because they are `final` classes

* Introduce `IntegrationFlowDefinition.logAndReply()` operator

* Fix logging message

Doc Polishing
2018-08-07 11:35:31 -04:00
Artem Bilan
4a85849bcc INT-3364: Add batchUpdate into JdbcMessageHandler (#2534)
* INT-3364: Add batchUpdate into JdbcMessageHandler

JIRA: https://jira.spring.io/browse/INT-3364

* * Optimize items mapping with an internal `Message` implementation
* Polishing Docs and Javadocs
2018-08-02 17:41:25 -04:00
Gary Russell
03a3bbb65f More JDBC doc fixes 2018-07-30 15:29:26 -04:00
Gary Russell
1a4de86b30 Fix appendix warnings and other minor fixes 2018-07-24 15:56:14 -04:00
Artem Bilan
44b7df94c1 Polishing for the FileWritingMessageHandler
* Fix typos in docs
* Add `om.google.code.findbugs:jsr305` to avoid warnings like
`Cannot find annotation`
2018-07-24 15:50:56 -04:00
Alen Turkovic
9ac0746f19 INT-4502: Add new file callback in FileWritingMH
JIRA: https://jira.spring.io/browse/INT-4502

Documentation
Using a BiConsumer instead of a custom interface

Doc fixes

Calling callback while locked

More requested fixes

Missing `FileWritingMessageHandler.this` on `fileExistsMode`
2018-07-24 15:50:05 -04:00
Artem Bilan
a7ffbde07f Fix WebSocketInChA according latest SF
* The `SimpleBrokerMessageHandler` has been fixed the way it consults
a `session` store for sending reply.
The session can be added to that store only when we send a `CONNECT`
command.
Fix `WebSocketInboundChannelAdapter` to send additional `CONNECT` message
in the `afterSessionStarted()` and when selected `SubProtocolHandler` is
a `StompSubProtocolHandler`
* Add extract check into the `StompIntegrationTests` for events: now
we also get a `SessionConnectedEvent` as well
* Fix typo in the `index-docinfo.xml`
2018-07-24 14:59:39 -04:00
Jay Bryant
997183bee2 Removed the troublesome backticks
Removed the backticks that were making links not render. Fixed a few typoes, too.

Integrating feedback from Artem

Artem caught some errors that I had made (and that my editor, Atom, had made - grrr). I fixed them.

Thanks, Artem. :)

Removing duplicate content

I caught that duplicate content somehow got added to preface.adoc. I removed one set of it to make that content not be duplicated.

* Fix `jdbc.adoc` typos
2018-07-24 12:24:34 -04:00
Artem Bilan
5c1c82efd7 Fix zookeeper.adoc typos 2018-07-24 12:12:40 -04:00
Gary Russell
b23147b9ef Fix PDF overflows; missing block fences
- Fix most PDF overflows (2 remain)
- Add missing block fences to fix ToC/Headings
2018-07-24 12:10:56 -04:00
Artem Bilan
c6f459c130 INT-4453: Move Twitter module to Extensions
JIRA: https://jira.spring.io/browse/INT-4453

Fix typos and language in docs

Fix more typos and language in docs
2018-07-23 14:14:41 -04:00
Gary Russell
09d6b76748 More doc fixes
Fix Java syntax rendering leak.
2018-07-19 17:00:07 -04:00
Gary Russell
70db967d4f Doc ToC Fixes 2018-07-19 16:48:58 -04:00
Gary Russell
55f69206e3 Gateway void return type docs
* Polishing - PR Comments
2018-07-19 16:03:53 -04:00
Artem Bilan
aac4051d68 Remove max-rows from Stored Proc-in-ch-ad Docs
https://stackoverflow.com/questions/51361379/spring-integration-jdbc-what-is-the-replacement-for-the-missing-attribute-max

**Cherry-pick to 5.0.x and 4.3.x**
2018-07-16 11:29:22 -04:00
Artem Bilan
76877b9440 INT-4499: Add JdbcMetadataStore.setLockHint()
JIRA https://jira.spring.io/browse/INT-4499
Fixes https://github.com/spring-projects/spring-integration/issues/2483

To allow to reconfigure a `SELECT ... FOR UPDATE` for particular RDBMS
vendor, we need an option to specify possible hint for the target data base.

**Cherry-pick to 5.0.x**
2018-07-11 17:52:44 -04:00
Artem Bilan
9a7ae3dfad Polishing some concerns and warnings 2018-07-03 14:46:49 -04:00
Jay Bryant
f3bf2a68d6 Full editing pass for the Reference Guide
I went through all the Asciidoc files to correct spelling, grammar,
punctuation, and usage. Wherever possible, I also made it more clear and more concise.
I also added links wherever they were reasonable.

Accommodating changes that came while I was editing

Various folks made changes (one of which I recommended) to the documents
while I spent a month editing.
I had to edit that new and changed content as well, so I need this second commit.

Accommodating Artem's requested changes for `aggregator.adoc`

Artem reviewed `aggregator.adoc` as a starting point, so that
he could get some questions answered.
I have made the changes he requested (and answered his questions on Github).
2018-07-03 14:11:02 -04:00
Gary Russell
11d39b7da1 INT-4496: (S)FTP: Sort file list earlier
JIRA: https://jira.spring.io/browse/INT-4496

Sort the file list before applying `maxFetchSize` and filters.

* Polishing - PR Comments
2018-07-02 16:49:23 -04:00
Gary Russell
26cfab579c INT-4491: (S)FTP inbound rotate dirs/servers
JIRA: https://jira.spring.io/browse/INT-4491

Add Rotating Server/Directory Polling Advice.

**cherry-pick to 5.0.x**

* Polishing - PR Comments.

* Polishing

* Polishing; revert `KeyDirectory`; WARN about `TaskExecutor` and `MessageSoureMutator`(s).

* More polishing - PR comments

* Apply stashed changes.

* Fix WARN log - the `SyncTaskExecutor` is wrapped.
2018-06-27 13:43:41 -04:00
Gary Russell
586afc9901 INT-3916: ZK: Fix missing doc update
JIRA: https://jira.spring.io/browse/INT-3916

Changed CTOR injection to setters.
2018-06-27 12:26:55 -04:00
Artem Bilan
69af967c78 INT-4488: Require @EnablePublisher
JIRA: https://jira.spring.io/browse/INT-4488

Since `PublisherAnnotationBeanPostProcessor` is quite expensive
(1.5x the CPU used by `MessagingAnnotationPostProcessor` in a simple
application)

* So not register a `PublisherAnnotationBeanPostProcessor` by default.
The `@EnablePublisher` has to be presented on the `@Configuration`
to allow the `@Publisher` AOP
* Change the `<int:annotation-config>` to require a new
`<int:enable-publisher>` sub-element for similar purpose - do not
register `PublisherAnnotationBeanPostProcessor` by default
2018-06-26 16:00:45 -04:00
Artem Bilan
13510df575 INT-4452: Expire immediately when group timeout<0
JIRA: https://jira.spring.io/browse/INT-4452

This is pretty typical in practice to get a `groupTimeout` to be
evaluated to a negative value: some business decisions, sensitive
calculations and so on.

* Treat any non-positive `groupTimeout` as a signal to expire group
immediately without scheduling.
Only `null` is considered as a signal do nothing for the current message
* Polishing for some tests for better performance - saves some execution
time
2018-06-26 12:41:48 -04:00
Artem Bilan
07c32ae93b INT-4489: MessageHistory: Allow runtime beans
JIRA: https://jira.spring.io/browse/INT-4489

Make `MessageHistoryConfigurer` as a `BeanPostProcessor` and apply
tracking logic to processed beans if it is already started.

**Cherry-pick to 5.0.x**
2018-06-22 13:55:20 -04:00
Artem Bilan
e6a35c4a60 INT-4487: Allow reconfiguring ARPMH.adviceChain
JIRA: https://jira.spring.io/browse/INT-4487

Currently the `AbstractReplyProducingMessageHandler` applies advices
only in the `afterPropertiesSet()` phase.
When we use bean references for `MessageHandler`s in Java DSL, the
`.advice()` provided in the flow definition is not applied to the
`AbstractReplyProducingMessageHandler`.

* For consistency with the endpoint configured by the DSL definition,
override and reconfigure advices in the `AbstractReplyProducingMessageHandler`
bean reference during DSL parsing and processing.

Fix `TwitterSearchOutboundGatewayParserTests` for the proper assert
2018-06-22 13:06:18 -04:00
Gary Russell
bf10821d32 Add hack for Atom asciidoc syntax highlighting bug
https://github.com/atom/language-java/issues/88
2018-06-19 14:52:44 -04:00
Artem Bilan
32030c3233 INT-2260: JdbcPollingChA: rename prop to maxRows
JIRA: https://jira.spring.io/browse/INT-2260

Having a feedback about confusing with the `max-rows-per-poll` property
name and its responsibility it would be better do not mention `per-poll`
at all

* Deprecate `max-rows-per-poll` in favor of new `max-rows`
* Some code style polishing, tests improvements
* Docs polishing on the matter

* Add `What's New` bullet

* Optimize `maxRows` logic
* Document vendor-specific native SELECT limiting options
* Raise warning in the parsers about deprecated `max-rows-per-poll`

Doc polishing
2018-06-19 12:13:39 -04:00
Gary Russell
a954290561 Fix docs isClientModeConnected() 2018-06-12 11:43:46 -04:00
Artem Bilan
0fc8bec645 GH-2464: WebFlux: Get rid of Mono.block() (#2465)
* GH-2464: WebFlux: Get rid of Mono.block()

Fixes https://github.com/spring-projects/spring-integration/issues/2464

The `WebFluxInboundEndpoint` resolves a `Principal` via `Mono.block()`
operation.
This is prohibited situation in the non-blocking thread, like Reactor
Netty

* Defer `Mono<Principal>` resolution to the message header via
deferring the whole `Message` creating via `flatMap()` operation on the
main `doHandle()` `Mono`

**Cherry-pick to 5.0.x**

* * Fix Checkstyle for the Reactive Spring Security testing utils static imports
* Add `defaultIfEmpty()`, when `exchange.getPrincipal()` is an empty `Mono`
2018-06-11 16:34:04 -04:00
Gary Russell
bb5528c761 INT-4481: Fix notice.txt line delimiters
CRLF->LF
2018-06-05 13:07:47 -04:00
Artem Bilan
89f902885a INT-4481: Add licence and notice to each artifact
JIRA: https://jira.spring.io/browse/INT-4481

* Now each generated jar has `licence.txt` and `notice.txt` entries in
its `META-INF`
* Copy `notice.txt` from the Spring Framework with an appropriate
polishing
2018-06-05 13:04:12 -04:00
Gary Russell
35c7ef1131 INT-4387: Fix incorrect contentType mapping
JIRA: https://jira.spring.io/browse/INT-4387

Don't put a copy in the message properties headers map.
2018-06-01 12:45:01 -04:00
Gary Russell
997b07afed INT-4476: Fall back for ID and Timestamp Headers
JIRA: https://jira.spring.io/browse/INT-4476

When mapping outbound headers, if the `AmqpHeaders` `ID` and `Timestamp` headers
are not present, fall back to mapping the `MessageHeaders` variants (if present).

Also fix some PDF overflows.
2018-05-31 17:18:39 -04:00
Artem Bilan
17e794d2c8 INT-4471: PubSubChannel: Add errorHandler warn (#2459)
* INT-4471: PubSubChannel: Add errorHandler warn

JIRA: https://jira.spring.io/browse/INT-4471

* When an `Executor` is not provided, log warn that the provided
`ErrorHandler` is ignored.

**Cherry-pick to 5.0.x and 4.3.x**

* * Polish warn message
2018-05-31 15:25:38 -04:00
Gary Russell
73dbdb80be INT-4477: Add getRole() to LeaderInitiator Context
JIRA: https://jira.spring.io/browse/INT-4477

When running multiple roles, it is useful to be able to determine the
role for a particular context, e.g. in an event.

* Docs

**cherry-pick to 5.0.x**
2018-05-31 10:54:35 -04:00
Gary Russell
7ab024df88 Fix broken links in reference appendices 2018-05-28 14:40:34 -04:00
Gary Russell
97d08d9720 INT-4473: Support prefix bean names with flow id
JIRA: https://jira.spring.io/browse/INT-4473

Previously, dynamic registration of integration flows with components configured
with the same `id` would fail with duplicate bean names.

Add `useFlowIdAsPrefix()` to the registration builder to enable the option.
Then, in the BPP, check the flag before naming the beans.

**cherry-pick to 5.0.x**

* Polishing according PR comments
* Widen `flowNamePrefix` responsibility in the
`IntegrationFlowBeanPostProcessor` since we may have many other
components in the dynamic flow with the same id, not only consumer
endpoints
2018-05-22 13:51:13 -04:00
Artem Bilan
b21dc0dff3 Fix IntegrationFlowContext concurrency issue
When we register `IntegrationFlow` s concurrently at runtime, we may
end up with the problem when we register the same object with the same
bean name, but in different places.
Or when we turn off bean overriding, we end up with the exception that
bean with the name already registered

* Wrap `IntegrationFlow` bean registration in the
`StandardIntegrationFlowContext` into the `Lock` when its bean name
is generating
* Make `StandardIntegrationFlowContext.registry` as `ConcurrentHashMap`
to avoid `ConcurrentModificationException` during `put()` and `remove()`
* Fix concurrency for beans registration with the generation names in
the `IntegrationFlowBeanPostProcessor` using an `IntegrationFlow` id
as a prefix for uniqueness.

**Cherry-pick to 5.0.x**

Fix generated bean name in the WebFluxDslTests

Use only single `Lock` in the `StandardIntegrationFlowContext`:
we don't need a fully blown `LockRegistry` there anymore since we have
only one synchronization block there and it is always around the same
type
* Add `What's New` note, and mention changes in the `dsl.adoc`

Minor doc polishing.
2018-05-21 13:05:45 -04:00
Artem Bilan
1f05d3ec4c Polishing some Docs and JavaDocs
* Mentioned `byte[]` support for the `UnmarshallingTransformer`
* Fix `AbstractInboundFileSynchronizingMessageSource` JavaDocs to
mention the current `FileSystemPersistentAcceptOnceFileListFilter`
usage by default, not an `AcceptOnceFileListFilter`
* Polishing `AbstractInboundFileSynchronizingMessageSource` code style
a bit
2018-05-16 16:10:50 -04:00
Artem Bilan
1b46224e41 INT-4398: Replace Gemfire dependency with Geode (#2426)
* INT-4398: Replace Gemfire dependency with Geode

JIRA: https://jira.spring.io/browse/INT-4398

* Polishing `gemfire.adoc` and mention the procedure for an
exclusion/inclusion appropriate dependency

* More Docs polishing

* * Fix tests to be based on SD Geode XML namespace
* Fix Docs according PR comments

* Polishing Docs according PR comments
2018-05-15 11:18:14 -04:00
Artem Bilan
61b272dd5b Wrap non-StandardIntegrationFlow into Proxy (#2440)
* Wrap non-`StandardIntegrationFlow` into Proxy

In previous version all the `IntegrationFlow` beans have been replaced
by the `StandardIntegrationFlow` i the `IntegrationFlowBeanPostProcessor`
if they are lambda.
That works for Java, but doesn't with Kotlin, since lambdas i last one
are not synthetic classes.
Therefore some Java DSL definitions (especially `.subFlowMapping()`)
don't work consistently in two languages.

* Introduce `IntegrationFlowLifecycleAdvice` to wrap all the
non-`StandardIntegrationFlow`s (excluding `IntegrationFlowAdapter`)
into the `Proxy` to expose `SmartLifecycle` and `getInputChannel()`
operations and delegate them to the internal `StandardIntegrationFlow`
created by the `IntegrationFlowBeanPostProcessor`.
* This way any custom `IntegrationFlow` implementations can be used
for manual flow registration via `IntegrationFlowContext`
* Polish `RouterDslTests.kt` for the `@Bean`s for sub-flows.
* Document in the `dsl.adoc` a request-reply approach for the case
when `.subFlowMapping()` refers to an `IntegrationFlow` `@Bean`.
* Polishing for the `FlowServiceTests` since all the
non-`StandardIntegrationFlow`s and not-`IntegrationFlowAdapter`s are
wrapped now to the Proxy.
* Add missing `from()` delegations into the `IntegrationFlowAdapter`
* Polishing for the `ManualFlowTests` since all the `IntegrationFlow`
now are `Lifecycle` after wrapping to the Proxy.

* Add JavaDocs to the IntegrationFlowLifecycleAdvice and polishing for the dsl.adoc

* * Add JavaDocs for the `IntegrationFlowBeanPostProcessor.processIntegrationFlowImpl()`
* Improve JavaDoc for the `RouterSpec.subFlowMapping()`
* Assert in the `FlowServiceTests` that proxied custom flow implements
all the expected interfaces
2018-05-11 12:53:33 -04:00
Artem Bilan
3c1b547b26 Fix MQTT clientFactory bean definition in Docs
**Cherry-pick to 5.0.x and 4.3.x**
2018-05-07 14:13:55 -04:00
Artem Bilan
a14514ada0 INT-4462: WebFluxInbound: cope with empty body
JIRA: https://jira.spring.io/browse/INT-4462

When the HTTP request body is empty, the `HttpMessageReader` ends up
with the empty `Mono` which can't be evaluated to any reasonable value.

* Add fallback to `requestParams` when `Mono` for body is empty and
also when `payloadExpression` returns null

**Cherry-pick to 5.0.x**
2018-05-07 12:46:08 -04:00
Gary Russell
07141302bb INT-4463: Full access to MqttConnectOptions
JIRA: https://jira.spring.io/browse/INT-4463

Certain options, such as `maxInFlight` were not exposed.

Deprecate the setters on the factory and allow the user to inject a pre-configured
`MqttConnectOptions`, thus making all (and any new) properties available to be
configured.
2018-05-07 12:39:15 -04:00
Artem Bilan
0ef659b8a7 INT-4351: Add DiscardAwareFileListFilter
JIRA: https://jira.spring.io/browse/INT-4351

The `WatchService` reacts to the events in the file system and keep
track ove the events until we poll them, e.g. via `listEligibleFiles()`
in the `WatchServiceDirectoryScanner`.
On the other hand the `SourcePollingChannelAdapter` calls the mentioned
`listEligibleFiles()` according its polling period.
At this moment the `FileListFilter` is applied to the polled files.
It may happen that `LastModifiedFileListFilter` can't accept too young
files yet and they are lost for the future consideration.

* To allow, for example, to retain young files by the
`LastModifiedFileListFilter` judgment for the future cycles add
`DiscardAwareFileListFilter` with the `DiscardCallback` support.
The `WatchServiceDirectoryScanner` now registers such a callback into
the filter and stores discarded files into the `filesToPoll` queue
for the future poll cycle.

**Cherry-pick to 5.0**

Fix compilation warnings

* Replace `DiscardCallback` with the plain `Consumer`
* Ensure uniqueness in the `WatchServiceDirectoryScanner` internal
queue via a `Set` implementation, since discard callback may be called
several times for the same file from the `CompositeFileListFilter`
according to its nature
* Add JavaDocs and Docs

Change `@since` to `5.0.5`
2018-04-24 14:10:15 -04:00