Commit Graph

2144 Commits

Author SHA1 Message Date
Artem Bilan
29b60c2932 INT-4438: No lifecycle twice in the same role
JIRA: https://jira.spring.io/browse/INT-4438

The `SmartLifecycleRoleController` is based on the `MultiValueMap`
which used internally a `List` for the values.
With such an architecture we can add the same value several times.
On the other hand we are iterating over `Lifecycle`s in the role and
build a `Map` for their running status.
In this case when `NamesComponent`s return the same name the Java
`Collectors.toMap()` fails with a duplicate key error.

In any cases it would be better do not allow to add the same lifecylce
several time to the role or different with the same name.

* Add search logic to the `addLifecycleToRole()` to fail fast with the
`IllegalArgumentException` because a lifecycle with the same name is
already present in the role

**Cherry-pick to 5.0.x**

* Remove redundant `this.initialized = false` from the
`AbstractPollingEndpoint.doStop()`

Add `allEndpointsRunning()` verification to the `EndpointRoleParserTests`

Polishing
2018-04-02 17:33:59 -04:00
Artem Bilan
9ce329f1c6 Increase timeout for PollingConsumerEndpointTests
https://build.spring.io/browse/INT-SI50X-20

Looks like 5 seconds is not enough to wait for the test completion,
especially on CI server

* Refactor `PollingConsumerEndpointTests` to use `OnlyOnceTrigger`
from the test-support module instead of local `Trigger` implementation
* Increase timeout for latch in the `OnlyOnceTrigger` to 10 seconds

**Cherry-pick to 5.0.x**

(cherry picked from commit a65d05c)
2018-04-02 14:48:29 -04:00
Artem Bilan
04a61f1dd5 Fix Transformers usage in Docs
**Cherry-pick to 5.0.x**

(cherry picked from commit 8afdcb4)
2018-03-27 12:23:54 -04:00
Gary Russell
2277675c40 INT-4443: Use SimpleEC for uriVariablesExpression
JIRA: https://jira.spring.io/browse/INT-4443

**cherry-pick to 5.0.x, 4.3.x**

Polishing; use data binding accessor in test evaluation contexts.

Add `.withInstanceMethods()`

See https://jira.spring.io/browse/SPR-16588?focusedCommentId=158041&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-158041

* Polishing according PR comments

# Conflicts:
#	build.gradle
#	src/reference/asciidoc/changes-4.3-5.0.adoc
2018-03-26 16:56:46 -04:00
Nathan Kurtyka
2b6cafdba4 INT-4440: Support serialized UUID in headers
JIRA: https://jira.spring.io/browse/INT-4440

The previous fix eliminated an extra `generateId()` call, but at the
same introduced regression do not populate `id` and `timestamp` from
the serialized state, e.g. after JSON transferring over the network

* Introduce a couple utility methods in the `MutableMessageHeaders`
to extract and parse `id` and `timestamp` from the provided headers

**Cherry-pick to 5.0.x**

(cherry picked from commit ae2aa8b)
2018-03-23 10:02:11 -04:00
Gary Russell
cb3c7a0036 INT-4441: Fix ACMH Concurrency Problem
JIRA: https://jira.spring.io/browse/INT-4441

Possible concurrent updates to `AbstractCorrelatingMessageHandler.groupIds` and
`expireGroupScheduledFutures`.

(cherry picked from commit f9d5198)
2018-03-22 15:27:10 -04:00
Artem Bilan
f9da4382c2 INT-4434: Allow to use sub-flows from beans
JIRA: https://jira.spring.io/browse/INT-4434

There were a restriction introduced since Java DSL `1.2` do not use
`IntegrationFlow` beans for sub-flow definitions, e.g. in routers.
It is considered as regression by community because it worked before
in version `1.1`

* Introduce `IntegrationFlow.getInputChannel()` to be able to bridge
from the main flow to the flow which is treated as sub-flow.
In most cases we talk about an independent bean for the `IntegrationFlow`
which can be used as a stand along one and as a sub-flow in other flow

**Cherry-pick to 5.0.x**

Add JavaDocs to the `EndpointSpec.obtainInputChannelFromFlow()`
2018-03-22 14:38:36 -04:00
Artem Bilan
2d3924a7e6 INT-4433: Optimize @Publisher metadata
JIRA: https://jira.spring.io/browse/INT-4433

The current `MessagePublishingInterceptor` behavior is to parse expressions
on each method invocation what is not so efficient at runtime

* Introduce `default` `Expression`-based method to the `PublisherMetadataSource`
contract and call existing String-based methods for backward compatibility.
* Deprecate String-based `PublisherMetadataSource` methods in favor of newly
introduced `Expression`-based
* Implement new `getExpressionForPayload()` and `getExpressionsForHeaders()`
in all the `PublisherMetadataSource` implementations
* Cache parsed `Expression` s during initialization in the `PublisherMetadataSource`
implementations or do that on demand in the `MethodAnnotationPublisherMetadataSource`
by provided method basis
* Introduce `MethodAnnotationPublisherMetadataSource#metadataCacheLimit` and populate
its value from the `@EnablePublisher` or `<int:annotation-configuration>`
* Implement `entrySet()` and `values()` in the `ExpressionEvalMap`

**Cherry-pick to 5.0.x**

* Add `@SuppressWarnings("varargs")` to avoid compilation warning

* Remove LRU cache logic - it's fine to cache all the info about methods
in the classpath
2018-03-21 16:35:46 -04:00
Artem Bilan
f19f23f37b LockRegistryLeaderInitiator: Add DEBUG for errors
For better traceability for errors during lock acquiring add DEBUG
logging message in the `catch` block before returning back to the main
loop for the next acquiring attempt

**Cherry-pick to 5.0.x and 4.3.x**
2018-03-21 12:12:02 -04:00
Artem Bilan
64a37381e0 INT-4437: Scatter-Gather: reinstate replyChannel
JIRA: https://jira.spring.io/browse/INT-4437

The `ScatterGatherHandler` overrides a `replyChannel` header for the
scatter message to its internal queue and doesn't reinstate the original
`replyChannel` header when producer a gather result message

* Rebuild gather result message with population a proper `replyChannel`
header from the request message and removing a `gatherResultChannel`
header

**Cherry-pick to 5.0.3 and 4.3.x**

Polishing
2018-03-20 16:50:34 -04:00
Ruslan Stelmachenko
ed563b9f3f INT-4430: FileSplitter close reader on exception
JIRA: https://jira.spring.io/browse/INT-4430

When Iterator-based `FileSplitter` splits the file, and an exception
throws in downstream flow (in the same thread), the exception propagates
to the caller leaving underlying file reader opened.

This commit changes `AbstractMessageSplitter` the way, that,
when any exception happens, if Iterator implements `java.io.Closeable`,
its `close()` method will be called before propagating exception.

Also `FileSplitter`'s underlying iterator implements `Closeable` now.

* Make `CloseableIterator` to follow `Closeable` contract.

Now `CloseableIterator.close()` declares `IOException` and can be used
as base interface for `FunctionIterator`.

* Adjust tests.

Adjust tests to reflect the fact that we call `close()` on the reader
one more time in the end of iterator.

**Cherry-pick to 5.0.x and 4.3.x**

(cherry picked from commit 7f25cba)
2018-03-16 10:28:39 -04:00
Artem Bilan
fa47b0fa20 INT-4428: MethodParam usage: fix race condition
JIRA: https://jira.spring.io/browse/INT-4428

Using `increaseNestingLevel()` and `decreaseNestingLevel()` is not
thread-safe and may cause a race conditions

* Use `MethodParameter.nested()` instead which creates and cache a new
`MethodParameter` for the nested generic type

**Cherry-pick to 5.0.x**
2018-03-14 09:22:54 -04:00
Artem Bilan
194480d76f GH-2388: Fix HeaderEnricherSpec for adviceChain
Fixes spring-projects/spring-integration#2388

During `HeaderEnricherSpec` refactoring the `adviceChain` population
has been missed, alongside with many other `AbstractReplyProducingMessageHandler`
options from the `ConsumerEndpointSpec`

* Refactor `HeaderEnricherSpec` to build `HeaderEnricher` and an
appropriate `MessageTransformingHandler` from the ctor to be able
to pick up an `adviceChain` automatically in the `ConsumerEndpointSpec.get()`

**Cherry-pick to 5.0.x**
2018-03-13 12:26:09 -04:00
Gary Russell
6718d8d433 INT-4423: Make Micrometer dependency optional
JIRA: https://jira.spring.io/browse/INT-4423

Add a facade on top of micrometer so the dependency can
be optional.t

Next iteration - PR comments

Checkstyle

More polishing; static classes etc.

checkstyle

Further polishing; docs; auto-register `MicrometerMetricsCaptor`.

Polishing - move captor load to a static method on the captor.

* Fix Checkstyle violation
* Some code style polishing
2018-03-09 15:24:29 -05:00
jmaxwell
a92933e800 INT-4427 add value attribute to the @Publisher
JIRA: https://jira.spring.io/browse/INT-4427
2018-03-07 20:12:46 -05:00
Gary Russell
f479270dcd Support JUnit5 in tests
* wrapper
2018-03-05 16:50:33 -05:00
Gary Russell
212f478151 INT-4417: Add @EndpointId
JIRA: https://jira.spring.io/browse/INT-4417

Provide more flexibility with bean naming when using EIP Annotations with
Java configuration.

* Polishing - PR Comments

* More polishing - remove `@Inherited`; disallow with more than one EIP annotation.
2018-03-05 14:34:04 -05:00
Gary Russell
bc25eff34c Polishing 2018-03-05 11:21:40 -05:00
Gary Russell
6065745913 INT-4421: Fix failedMessage in some exceptions
JIRA: https://jira.spring.io/browse/INT-4421

If a component invoked by an SPCA threw a `MessagingException` with no
`failedMessage`, the resulting ErrorMessage payload had no `failedMessage`.

The `UnicastingDispatcher` had a check for this so it wasn't an issue
as long as at least one `DirectChannel` was between the poller and the
component.

Promote the wrapping code to `IntgrationUtils` and invoke it from
places that blindly rethrew `MessagingException`s.
2018-03-05 11:21:40 -05:00
Gary Russell
1c4db65b1c Channel Incorrectly started a second timer
* Some code style polishing for affected classes
2018-03-02 14:08:53 -05:00
Gary Russell
82a8982857 INT-4418: Micrometer docs and polishing
JIRA: https://jira.spring.io/browse/INT-4418

Add docs for the rework and optimize `Meter` creation.

Polishing

* Polishing according PR comments

* Fix typo in `dsl.adoc`
2018-02-28 14:09:04 -05:00
Artem Bilan
cb0d43db6b Fix JDBC tests and some upgrades
https://build.spring.io/browse/INT-FATS5IC-430

Looks like there is a race condition when our polling rate around
data base is too fast and we have access to the DB files even during
application context close.

* Stop polling channel adapter in the tests explicitly after test methods
* Increase some tests performance decreasing timeouts to wait
* Upgrade to Reactor-3.1.5, AssertJ-3.9.1, Derby-10.14.1.0 and some
Gradle plugins

* Upgrade to `reactor-netty-0.7.5`
* Remove workaround from the `StompServerIntegrationTests`

* Upgrade to Spring Data Kay SR5

* Upgrade to Spring Security 5.0.3
* Increase timeouts and performance in the `StreamTransformerParserTests`
2018-02-28 11:57:21 -05:00
Gary Russell
792b8fe6d1 INT-4416: Rework Micrometer Metrics
JIRA: https://jira.spring.io/browse/INT-4416

Metrics should be under a common name, discriminated with tags.
2018-02-27 16:52:28 -05:00
Artem Bilan
2c3ee7d68c MMIH: Convert from JSON only if SpELInvoker
If there is `json`-aware `contentType` header in the message to process
and payload type is `String` or `byte[]` and it isn't equal to the
expected exclusive method argument type (even generic from the `Message<>`),
the conversion should happen only if SpelInvoker is no method.

The regular `InvocableHandlerMethod` takes care about payload conversion
via configured `MessageConverter` in the `PayloadArgumentResolver`

* Fix `MessagingMethodInvokerHelper` to call JSON conversion only when
`this.handlerMethod.spelOnly`

NOTE: We can't do generics conversion right now because it is going
to be a breaking change around
`org.springframework.integration.support.json.JsonObjectMapper` property
in the `MessagingMethodInvokerHelper`.
We can do that only in `5.1` and rely there only on the functionality
from the Jackson Object mapper

* Convert from JSON only in the `invokeExpression()`
* Optimize message recreation only if target param type is `Message`.
Otherwise override just `payload` property of the `ParametersWrapper`
2018-02-24 13:40:16 -05:00
Artem Bilan
d5303ca4f3 INT-4413: Don't register flow under the same id
JIRA: https://jira.spring.io/browse/INT-4413

When we override entity in the `IntegrationFlowContext.registry`,
we may get dangling beans in the application context,
when the structure of a new `IntegrationFlow` is different.

* Fix `IntegrationFlowContext` to disallow to override the flow
registration under the same name
* Add JavaDocs to the `IntegrationFlowRegistrationBuilder` methods
* Add `toString()` to the `IntegrationFlowRegistration` and
`StandardIntegrationFlow` to make the logging of these components
friendlier
2018-02-23 15:07:31 -05:00
Artem Bilan
7d1680534f INT-4411: DSL: Fix sub-flows for dynamic routers
JIRA: https://jira.spring.io/browse/INT-4411

The `RouterSpec.RouterMappingProvider` relies on the
`ContextRefreshedEvent` which happens only during application start up.
When we register `IntegrationFlow` at runtime, this event doesn't
happen and therefore sub-flow mappings don't populated.

* Fix `RouterSpec.RouterMappingProvider` to parse sub-flow mappings in
the `onInit()`
* Reorder components registration for the router in the
`IntegrationFlowDefinition` to let lately sub-flows to start earlier,
then lifecycles in the main flow
2018-02-21 17:09:25 -05:00
Gary Russell
f6084939cc Add Pausable
See https://github.com/spring-projects/spring-integration-kafka/issues/198
2018-02-21 11:50:23 -05:00
Kiel Boatman
4bb4a0187a GH-2364 fix intermittent failing test
Resolves https://github.com/spring-projects/spring-integration/issues/2364
2018-02-20 14:52:37 -05:00
Artem Bilan
00c983d97a INT-4410 CollArgResolver only for group processor
JIRA: https://jira.spring.io/browse/INT-4410

The `CollectionArgumentResolver` has been introduced especially for the
cases to work with `MessageGroupProcessor` (an aggregator)
when the payload is a `Collection<Message<?>>`.

This use-case doesn't apply for the general collection parameter use-case.

* Register `CollectionArgumentResolver` only when `listCapable` option.
For all other collection-based use-cases fallback to the standard
`PayloadArgumentResolver` with an appropriate configured `MessageConverter`

Note: this is for backward compatibility.
In the `5.1` we may reconsider to use `MessageConverter` in the
`CollectionArgumentResolver` as well, or just remove it altogether with
an appropriate logic in the `PayloadArgumentResolver`, since this
`CollectionArgumentResolver` solution isn't robust
2018-02-20 13:59:14 -05:00
Artem Bilan
800184270a INT-4402: Apply global interceptors at runtime
JIRA: https://jira.spring.io/browse/INT-4402

Make `GlobalChannelInterceptorProcessor` as a `BeanPostProcessor`,
so it can apply global `ChannelInterceptor`s to any initialized channel
beans, even those created at runtime, like in case of dynamic flows with
Java DSL

**Cherry-pick to 4.3.x**

* Add `postProcessDynamicBeans` global property with `false` by default
* Rely on the `postProcessDynamicBeans` property in the
`GlobalChannelInterceptorProcessor.postProcessAfterInitialization()`
*  Add `postProcessDynamicBeans=true` to the
`GlobalChannelInterceptorTests-context.xml` to be sure in the test-case
that option works
* Document the option and effect from the global channel interceptors
2018-02-16 11:33:19 -05:00
Gary Russell
a77def1cdf INT-4403: Micrometer and Dynamic Components
JIRA: https://jira.spring.io/browse/INT-4402

Previously, Micrometer instrumentation was only applied to components during
ApplicationContext initialization.

`IntegrationManagementConfigurer` is now a `BeanPostProcessor` as well as a
`SmartInitializingSingleton`, but only acts as a BPP after the context is
initialized. This enables Micrometer instrumentation to beans added later,
either via a new `BeanDefinition` or `bf.initializeBean`.

NOTE: destroying and re-creating a bean will use the same `Meters`.
2018-02-15 10:30:21 -05:00
Kiel Boatman
78b80ef79f GH-2354 Injection Executor to LockLeaderInitiator
Fixes https://github.com/spring-projects/spring-integration/issues/2354

INT-2354 missed file in previous commit

INT-2354 fix failing test

INT-2354 changes following feedback

INT-2354 feedback changes

INT-2354 feedback changes

* Simple code style polishing
2018-02-13 14:14:56 -05:00
Artem Bilan
d361eefa0c Add missed options and JavaDocs to AMQP Java DSL 2018-02-12 11:43:36 -05:00
Artem Bilan
09aeaac4da INT-4396: Add retrying lock in case of exception
JIRA: https://jira.spring.io/browse/INT-4396

When target distributed `Lock` implementation throws an exception, e.g.
in case of no connection to the service, the `LockRegistryLeaderInitiator`
exists the loop and can come back to the elections only after restart

* Catch all the exception on `this.lock.tryLock()` and resubmit
`LeaderSelector` for a new locking cycle if
`LockRegistryLeaderInitiator.isRunning()` and `InterruptedException`

* Remove diagnostics from the `JdbcLockRegistryLeaderInitiatorTests`
since this fix confirms that we just didn't have a reconnect logic before
when this test failed sporadically

**Cherry-pick to 4.3.x**
2018-02-09 16:09:22 -05:00
Gary Russell
e53e8941a1 Add ObjectStringMapBuilder
Convenient builder for DSL `getComponentsToRegister()`.
2018-02-07 10:55:46 -05:00
Gary Russell
f4fbfa116a INT-4395: More micrometer meters
JIRA: https://jira.spring.io/browse/INT-4395

- pollable channel counts
- component counts (gauges)

Polishing - PR comments

Fix switch in test

* Simple code style polishing and fix JavaDocs
2018-02-05 16:36:18 -05:00
Artem Bilan
5f509ac02e INT-4394: Fix compatibility with Jackson JSON
JIRA: https://jira.spring.io/browse/INT-4394

Looks like Jackson has became much smarter and now it stores
the target type for the object, not returned interface.

We can't use Jackson annotations on the Framework classes and
we can't restrict the `messagingAwareMapper()` `ObjectMapper` just
to use fields for all the object passed through it.

As a compromise solution we shouldn't use
`Collections.unmodifiableList()` in the getter.
The copy of the internal collection is pretty enough to protect
our object from mutation.

**Cherry-pick to 4.3.x (excluding `build.gradle` change)**
2018-02-05 11:51:07 -05:00
Gary Russell
51397ec141 Make updateNotPropagatedHeaders protected final 2018-02-03 13:08:54 -05:00
Gary Russell
54754743a5 INT-4384: Micrometer Support
JIRA: https://jira.spring.io/browse/INT-4384

Add support for micrometer metrics collection.

Initial commit.

* Rebase; upgrade to Micrometer rc.8

* Polishing - PR Comments

* Final Polishing
2018-02-02 15:30:48 -05:00
Gary Russell
3bd4930813 Fix UDP Tangles
- Class tangle in new event
- Package tangle udp <-> util

* Fix JacksonJsonUtils class tangle

* Fix HTTP class tangle
2018-02-01 15:12:53 -05:00
Artem Bilan
9f6457fc31 INT-4383: Add test for EnricherSpec.errorChannel
JIRA: https://jira.spring.io/browse/INT-4383
2018-01-23 12:42:38 -05:00
Alexis Hafner
061dcb1b87 INT-4383 Expose errorChannel in EnricherSpec
JIRA: https://jira.spring.io/browse/INT-4383
2018-01-23 12:31:27 -05:00
Gary Russell
8aa91d1db0 Shutdown test executors in -core
- don't use `ExecutorService` as `@Bean` - spring can't stop them
- also add log adjuster for ftp test that sometimes fails
2018-01-19 10:21:55 -05:00
Artem Bilan
7e263aa618 GH-2301: Add SimpMessageGrFactory.GroupType.LIST
Fixes https://github.com/spring-projects/spring-integration/issues/2301

* For non-ordered, without deduplication logic use-case the `ArrayList`
option for internal `SimpleMessageGroup` collection is added
* Make `SimpleMessageGroup` `protected` ctor as `public` for better
reuse in case of custom `MessageGroupFactory`
2018-01-18 11:56:25 -05:00
Artem Bilan
86c76999f6 Optimize AbstractMessageSources
To avoid `Message` re-creation during `AbstractMessageSource.receive()`
logic, refactor `AbstractMessageSource` implementations
to return `AbstractIntegrationMessageBuilder`

* Add `AbstractIntegrationMessageBuilder<File> doReceive()` to the
`FileReadingMessageSource` to be called from the `AbstractInboundFileSynchronizingMessageSource`
to avoid message recreation in its `doReceive()`
* Some code style refactoring in the `AbstractMessageSource`
2018-01-18 11:17:03 -05:00
Yilin Wei
e6ec86c505 INT-3945: Add async to the @ServiceActivator
JIRA: https://jira.spring.io/browse/INT-3945

Adding test for async annotated ServiceActivator

fix whitespace

Address comments

Change copyright and author

* Some code style polishing
* Rename `Log4j2LevelAdjuster.level()` to more friendly `forLevel()`
factory method name.
* Add `@param level` to the `Log4j2LevelAdjuster.forLevel()` to fix
JavaDoc warning
2018-01-18 09:48:22 -05:00
Gary Russell
1bb4f86151 Minor Fixes
- use passive mode for FTP tests
(active fails on one of my machines for some reason)

* Fix generic args instead of `@SuppressWarnings("rawtypes")`
2018-01-16 16:37:50 -05:00
Artem Bilan
c4c4e51627 Some JsonPropertyAccessor polishing
* Add `WrappedJsonNode.getTarget()` API to let target users to get
access to the target `JsonNode` for their logic
* Some code style and JavaDocs improvements in the `JsonPropertyAccessor`
2018-01-12 11:54:02 -05:00
Artem Bilan
a4eb55bf7a Fix SpelPropertyAccessorRegistrar
The `SpelPropertyAccessorRegistrar` ctor and
`add(PropertyAccessor... propertyAccessors)` method use wrong variable
to build a key for the provided `PropertyAccessor`.
Therefore during iteration the next `PropertyAccessor` overrides the
previous and we end up just only with one instance in the registry

* Extract `private static obtainAccessorKey()` method to build
decapitalized key for the `PropertyAccessor`
* Change the map store to the `LinkedHashMap` to keep order of the
provided `PropertyAccessor` s
* Modify `EnableIntegrationTests` to ensure that we support several
`PropertyAccessor` s and in the proper order

**Cherry-pick to 4.3.x**
2018-01-12 11:50:33 -05:00
Artem Bilan
30450c48be INT-4376: Upgage to Log4J 2 (#2321)
* INT-4376: Upgage to Log4J 2

JIRA: https://jira.spring.io/browse/INT-4376

* Deprecate Log4J 1.x components in favor of newly added
a `Log4j2LevelAdjuster` JUnit `@Rule`
* Update all the logging configuration to Log4J 2

* Polishing after testing
2018-01-11 10:59:55 -05:00