Commit Graph

8590 Commits

Author SHA1 Message Date
Artem Bilan
87e8beb9e6 Remove Jackson Imports from JacksonJsonUtils
Since the classes load their imports explicitly we can't use them for
classes which provides utilities to check classpath

* Move imports to fully qualified names in the target utility method

**Cherry-pick to 4.3.x**

Move `JacksonJsonUtils.isPresent` methods to separate `JacksonPresent` class

Polishing

Conflicts:
	spring-integration-core/src/main/java/org/springframework/integration/support/converter/ConfigurableCompositeMessageConverter.java
Resolved.
2017-05-11 15:34:43 -04:00
Artem Bilan
0fb4b8070d Fix [UnusedImport] Checkstyle violation
https://build.spring.io/browse/INT-SI43X-152
2017-05-10 11:52:47 -04:00
Artem Bilan
376405c39b INT-4267: Add JSON (De)Serializers for Messaging
JIRA: https://jira.spring.io/browse/INT-4267
Fixes: spring-projects/spring-integration#2110

The documentation clearly point that we can simply use JSON (de)serialization
with the `RedisMessageStore`, but actually it isn't so easy

* Fix `MessageGroupMetadata`, `MessageHolder`, `MessageMetadata` for Jackson
deserialization compatibility
* Add `MessageHeaders`-based ctor to the `AdviceMessage`
* Add `MessageHeadersJacksonSerializer` to serialize `MessageHeaders`
to the `HashMap` for easier deserialization afterwards
* Add deserializer implementations for all `Message` types
* Add convenient `JsonObjectMapperProvider#jacksonMessageAwareMapper()`
factory method to build `ObjectMapper` supplied with mentioned above
(de)serializers

**Cherry-pick to 4.3.10 without `MessageHolder` and `MessageMetadata`**

Address PR comments and document the feature

Doc Polishing

Conflicts:
	spring-integration-core/src/main/java/org/springframework/integration/store/MessageHolder.java
	spring-integration-core/src/main/java/org/springframework/integration/store/MessageMetadata.java
Resolved.
2017-05-10 11:27:09 -04:00
Artem Bilan
194d710c3b Fix Race Condition in TxSyncQueueChannelTests
https://build.spring.io/browse/INT-MASTER-653

The test-case uses only 1 second to wait for the message
in the `QueueChannel`.
That isn't enough on slow environment like CI causing failure for
the current test and unexpected value in the queue for the subsequent
tests

* Fix timeouts
* Get rid of `CountDownLatch` - the same is done by the wait on queues
* Purge queue in between tests.
This way we may not have failed subsequent test, thus failure analyze
will be much easier
* Rework `testRollback()` to wait for good result after one retry over
rollback
* Remove duplicate bean definition from XML config

**Cherry-pick to 4.3.x and 4.2.x**

Conflicts:
	spring-integration-core/src/test/java/org/springframework/integration/channel/TransactionSynchronizationQueueChannelTests.java
Resolved.
2017-05-08 16:19:06 -04:00
Artem Bilan
2c96f6a958 Fix race condition in the AMQP ChannelTests
https://build.spring.io/browse/INT-AT42SIO-502

There is a small time window when we remove the current consumer
from the local store, but there is no a new one yet.
So, we have to check the `Set`(`Map`) size before calling its `iterator`

**Cherry-pick to 4.3.x**
2017-05-08 13:11:25 -04:00
Gary Russell
18ef827bfc INT-4270: CGLib Compatibility
JIRA: https://jira.spring.io/browse/INT-4270

Initial Commit

Remove `final` modifier from certain critical framework methods that prevent CGLib proxies working.

Since Spring Boot 2.0 now uses proxyTargetClass=true by default (and has done that for
transactional proxies since 1.4), we must relax these restrictions so that CGLib can
proxy channels (which is often done to make subflows run in a transaction).

When CGLib can't override a `final` method, the fields used within those methods are uninitialized.
In the case of `AbstractMessageChannel` this causes an NPE on the unitiallized `dataTypes` field.
With `MessageHandler`, fields like `shouldTrack` are always false.

Further, methods on `IntegrationObjectSupport` - such as `getComponentName()` return null.

I have left setters and `afterPropertiesSet` (on IOS) `final` since these will typically be called
before the object is proxied.

I suspect there will be other methods we need to open up, but perhaps we should only do them on-demand.

__cherry-pick to 4.3.x__
2017-05-05 09:39:50 -04:00
Gary Russell
fc7febfe70 Make DefaultHttpHeaderMapper Easier to Subclass
It's difficult to subclass this mapper (e.g. to extend `toHeaders` to tweak the map)
because the factory methods for inbound and outbound mappers use private static fields
to set up the standard header mapping.

Extract that logic so that subclasses can create similar static methods and invoke the
setup code to set the standard header mapping.

Polishing: make more `DefaultHttpHeaderMapper` members available for inheritors

Conflicts:
	spring-integration-http/src/main/java/org/springframework/integration/http/support/DefaultHttpHeaderMapper.java
2017-05-04 17:34:52 -04:00
Artem Bilan
e8396c5dbf INT-4259: ACorrelationMH: WARN group reschedule
JIRA: https://jira.spring.io/browse/INT-4259

Since DEBUG can switched off for the application, it becomes difficult
to determine the wrong configuration for the discard flow from
the scheduler thread.

* Change logging level to the `WARN` and log the full stack-trace as well
to give more information about the reason of rescheduling group for
`forceComplete`

**Cherry-pick to 4.3.x**

Conflicts:
	spring-integration-core/src/main/java/org/springframework/integration/aggregator/AbstractCorrelatingMessageHandler.java
Resolved.
2017-05-03 10:10:51 -04:00
Artem Bilan
89bd2b7f64 Groovy test: Fix format for MM instead of mm
https://build.spring.io/browse/INT-AT42SIO-498/

**Cherry-pick to 4.3.x and 4.2.x**

(cherry picked from commit 9de2856)
2017-05-02 17:32:21 -04:00
Artem Bilan
883281f58f INT-4266: Disable multicast for Gemfire tests
JIRA: https://jira.spring.io/browse/INT-4266
2017-05-01 13:40:16 -04:00
Artem Bilan
12d2d8c677 INT-4248-4.3: Deprecate RedisLockRegistry methods
JIRA: https://jira.spring.io/browse/INT-4248

Some methods in `RedisLockRegistry` is planed for removal in `5.0` because of
new architecture.
Therefore mark them `@Deprecated` in `4.3.x`
2017-04-28 15:40:45 -04:00
Artem Bilan
ecc6824a8c INT-4262-4.3: Fix SQL scripts for IF EXISTS
JIRA: https://jira.spring.io/browse/INT-4262

Since `IF EXISTS` doesn't exist for `DROP INDEX` command and even more
this command isn't necessary when we fully drop the table, we don't need it
as a separate command.
`DROP TABLE IF EXISTS` is fully sufficient for schema recreation requirements

* Add `NODROPINDEX` variable for VPP engine to let it do not add `DROP INDEX` command
* Fix `channel/schema-mysql.sql` for proper INDEX names
2017-04-28 15:39:54 -04:00
Gary Russell
b5c6158117 INT-4257: Polishing
Make `EnhancedErrorMessage` ctors consistent with the new 5.0 ErrorMessage

Fix ctor usage
2017-04-24 17:29:37 -04:00
Gary Russell
089298e6be INT-4257 Polishing - Subclasses: buildErrorMessage
JIRA: https://jira.spring.io/browse/INT-4257

So that subclasses can also generate error message for conditions such
as message conversion errors.

Polishing - PR Comments

Fix JMS buildErrorMessage Override

Update Copyrights

(cherry picked from commit c665405)
2017-04-24 13:36:47 -04:00
Gary Russell
c91b20a20b Update to Spring Data Hopper SR10 2017-04-20 14:09:09 -04:00
Gary Russell
c54c76aad9 EnhancedErrorMessage - Add toString() from Core EM 2017-04-20 12:07:37 -04:00
Artem Bilan
f7adfc0628 INT-4257: Introduce ErrorMessagePublisher
JIRA: https://jira.spring.io/browse/INT-4257

To make a target `ErrorMessage` customizable introduce
`ErrorMessagePublishingRecoveryCallback` and
`ErrorMessageStrategy` to inject

**Cherry-pick to 4.3.x**

Fix `ErrorMessagePublishingRecoveryCallback` generic type for
`RequestHandlerRetryAdvice` compatibility

Polishing

- Support publishing when no retry context is available
- Use a constant for the message context key

Add EnhancedErrorMessage

- contains input message at time of error message generation
- make the default RecovererErrorMessageStrategy public and extensible

Fix `RetryAdviceParserTests` for new `RecoveryCallback` architecture

Rename to `ErrorMessagePublisher`
Decouple ErrorMessagePublisher from Retry

Refactoring for better purpose reflection and JavaDocs

Use `ErrorMessageStrategy` in message producers.

Add JavaDocs to the `DefaultErrorMessageStrategy`

Deprecate EnhancedErrorMessage

Minor Polish for Subclass Use

- make setters final
- allow subclasses to supply the AttributeAccessor

Fix Tests
2017-04-19 17:55:45 -04:00
Gary Russell
052aab0e47 Update to Spring Data Hopper SR9 2017-04-19 10:24:21 -04:00
Spring Buildmaster
af25c69102 [artifactory-release] Next development version 2017-04-18 22:06:12 +00:00
Spring Buildmaster
072be15de1 [artifactory-release] Release version 4.3.9.RELEASE 2017-04-18 22:06:05 +00:00
Gary Russell
d44a311371 Upgrade Spring Framework to 4.3.8.RELEASE 2017-04-18 17:39:06 -04:00
Artem Bilan
0fe4c2e8ae INT-4248-4.3.x: Fix LockRegistryLeaderInitiator
JIRA: https://jira.spring.io/browse/INT-4248

The `RedisLockRegistry` doesn't update store for `expire` property on the
subsequent lock interaction.
Only `RedisLockRegistry.obtain()` lets us to know if we expired already or not

To minimize the time window between `lock` and `expire` facts, move `obtain()`
function in the `LockRegistryLeaderInitiator` close to `Lock` usage.
2017-04-18 16:38:11 -04:00
Artem Bilan
2dc21e5d40 INT-4254: LockRegLInit: Fix unconditional unlock
JIRA: https://jira.spring.io/browse/INT-4254

The `LockRegistryLeaderInitiator#LeaderSelector` unconditional calls
`unlock()` in the `finally` block when the `Lock` might not be locked.

Another problem that `running = true` is set after submitting `LeaderSelector` task.
That might bring the problem that `LeaderSelector` won't be selected because of
`this.running` race condition

* Move `this.running = true` before submitting `LeaderSelector` task
* Move `unlock()` into the `if (this.locked)` condition

**Cherry-pick to 4.3.x**

Conflicts:
	spring-integration-core/src/test/java/org/springframework/integration/support/leader/LockRegistryLeaderInitiatorTests.java
Resolved.
2017-04-18 15:28:02 -04:00
Artem Bilan
71b118ed61 INT-2625(4.3.x): Refactoring for JdbcMessageStore
JIRA: https://jira.spring.io/browse/INT-2625

Since `jdbcOperations` is mandatory property of the `JdbcMessageStore`,
make it as constructor arg to encourage users always specify `DataSource`
or `JdbcOperations`.
With that we don't need `afterPropertiesSet()` any more
2017-04-18 15:17:21 -04:00
Gary Russell
a422631ef5 Increase Timeout for Promise Test 2017-04-16 13:16:21 -04:00
Artem Bilan
d41f77bc66 Fix LockRegLeaderInitiatorTests race condition
https://build.spring.io/browse/INT-FATS5IC-129/

Since leader selection is done on background thread in the
`LockRegistryLeaderInitiatorTests`, there is no guarantee that
after `Thread.sleep(100)` the lock will be obtained and leader state is
changed.

* Waiting for the `granted` `CountDownLatch` is much robust condition

(cherry picked from commit d5c633d)
2017-04-07 09:02:05 -04:00
Gary Russell
a48b5393cb INT-4250: Fix Statistics Mean Decay with Time
JIRA: https://jira.spring.io/browse/INT-4250

When retrieving the mean from `ExponentialMovingAverageRate` or
`ExponentialMovingAverageRatio` via `getStatistics()` the mean did not
decay over time.

The mean did decay when using `getMean()`.

This was caused by the statistics performance refactoring.

(cherry picked from commit fac04ae)
2017-03-29 13:54:33 -04:00
Gary Russell
03793e3dc4 INT-4247: Jdbc Lock Registry Test Diagnostics
https://jira.spring.io/browse/INT-4247
2017-03-24 12:52:28 -04:00
Artem Bilan
b86f90a69b Stop more initiators in LeaderInitiatorTests
JIRA: https://jira.spring.io/browse/INT-4246

**Cherry-pick to master**
2017-03-23 11:50:01 -04:00
Artem Bilan
5ae2edb548 Stop initiators in the end of test
JIRA: https://jira.spring.io/browse/INT-4246
2017-03-23 11:12:06 -04:00
Vedran Pavic
d9340030bb INT-4246: Revoke leader when lock isn't acquired
JIRA: https://jira.spring.io/browse/INT-4246

The `LockRegistryLeaderInitiator` currently does not revoke leadership
when leading instance is unable to acquire lock from the underlying lock.
This can result in multiple `LockRegistryLeaderInitiator` instances becoming
 leaders in the exceptional situations such as lock timeouts.

* Handle leadership revoking when leading `LockRegistryLeaderInitiator`
is unable to acquire lock

Polish javadoc

**Cherry-pick to master**
2017-03-23 10:04:34 -04:00
Artem Bilan
decfb9d86c Fix Checkstyle violations
https://build.spring.io/browse/INT-SI43X-123
2017-03-11 13:50:23 -05:00
Artem Bilan
dade5d9f00 Add Comparator to (S)FTP order-sensitive tests
https://build.spring.io/browse/INT-MASTER-583/

**Cherry-pick to 4.3.x**

Conflicts:
	spring-integration-ftp/src/test/java/org/springframework/integration/ftp/inbound/FtpStreamingMessageSourceTests.java
	spring-integration-sftp/src/test/java/org/springframework/integration/sftp/dsl/SftpTests.java
	spring-integration-sftp/src/test/java/org/springframework/integration/sftp/inbound/SftpStreamingMessageSourceTests.java
2017-03-11 13:32:39 -05:00
Gary Russell
96331d5415 INT-4243: Upgrade to sshd 1.4
JIRA: https://jira.spring.io/browse/INT-4243

Remove newline char from key exchange.

Fix CR and Add Windows Delete Diagnostics

Fix For Windows

More diagnostics.

It appears the new server doesn't close the file when the session is closed.

In the streaming test, consume the stream before closing the session.

Polishing - PR Comments

* More polishing according PR comments

Conflicts:
	spring-integration-sftp/src/test/java/org/springframework/integration/sftp/SftpTestSupport.java
	spring-integration-sftp/src/test/java/org/springframework/integration/sftp/dsl/SftpTests.java
	spring-integration-sftp/src/test/java/org/springframework/integration/sftp/session/SftpRemoteFileTemplateTests.java
	spring-integration-sftp/src/test/java/org/springframework/integration/sftp/session/SftpServerTests.java
	spring-integration-sftp/src/test/java/org/springframework/integration/sftp/session/SftpSessionFactoryTests.java
2017-03-11 13:18:25 -05:00
Gary Russell
f47b4bf222 INT-4240: Redis Delayer Test Diagnostics
https://jira.spring.io/browse/INT-4240

(cherry picked from commit 6a88328)
2017-03-08 15:37:33 -05:00
Gary Russell
789ed9d1b8 AMQP-4238: Detect Subscription Failures and QOS
JIRA: https://jira.spring.io/browse/INT-4238

Revert to using the sync client in the message-driven adapter so we can detect
subscription failures (the sync client throws an exception).

The only reason to use the async client was to timeout disconnects; this can
be achieved with the sync client and `disconnectForcibly`.

Also, the subscribe method updates the qos argument with the granted QOS values.

Detect and log if any QOS does not match the request.

Polishing

Polishing - PR Comments

Conflicts:
	spring-integration-mqtt/src/test/java/org/springframework/integration/mqtt/MqttAdapterTests.java

* Remove all new tests since Paho lib has class signature check, so we can't mock its classes
2017-03-06 20:48:31 -05:00
Gary Russell
0070d38d0c Update to S-D-Gemfire 1.8.8 (Hopper SR8) 2017-03-06 15:17:21 -05:00
Gary Russell
099ab8a6ae Fix Sporadic Test Failure
https://build.spring.io/browse/INT-MJATS41-925/
2017-03-03 09:49:58 -05:00
Gary Russell
4ada89d0c6 Javadoc Polishing 2017-03-02 09:11:38 -05:00
Spring Buildmaster
7ed6151913 [artifactory-release] Next development version 2017-03-01 20:14:57 +00:00
Spring Buildmaster
32282e1da8 [artifactory-release] Release version 4.3.8.RELEASE 2017-03-01 20:14:52 +00:00
Artem Bilan
9a19607f51 Upgrade to SF 4.3.7 and Spring AMQP 1.6.8 2017-03-01 14:50:02 -05:00
Artem Bilan
3bf2ca12ba INT-4229: SpEL Customization Via Java Config
JIRA: https://jira.spring.io/browse/INT-4229

* Make `SpelPropertyAccessorRegistrar` as `public` class and provide more API to customize it.
This class allows to register `PropertyAccessor` s for shared `EvaluationContext`
* Document how to configure SpEL functions and `PropertyAccessor` s with Java Config

**cherry-pick to 4.3.x**
2017-03-01 11:20:32 -05:00
Gary Russell
e872791a69 INT-4237: FWMH: Acquire Lock Before Flushing
JIRA: https://jira.spring.io/browse/INT-4237

It was possible to flush (close) the file while a write was in process; more likely when
`flushWhenIdle` is false.

This probably would not occur in the real world, just tests with short flush intervals,
but certainly possible.

There is already a lock used to prevent concurrent writes while appending; use the same
lock when flushing.

Conflicts:
	spring-integration-file/src/main/java/org/springframework/integration/file/FileWritingMessageHandler.java
	spring-integration-file/src/test/java/org/springframework/integration/file/FileWritingMessageHandlerTests.java
2017-03-01 11:18:57 -05:00
Artem Bilan
909060abd5 INT-4232: (S)FTP inbound: Fix lastModified Logic
JIRA: https://jira.spring.io/browse/INT-4232

Currently to implement the synchronization logic when remote file is modified there is only one way: remove local file and provide a `localFilter` which can be clear from the removed file as well.

The Reference Manual claims that `FileSystemPersistentAcceptOnceFileListFilter` can let us pick up a fresh version of the remote file if we configure `preserveTimestamp = true`, but actually `AbstractInboundFileSynchronizer` just bypass that logic because it check a local file for existence.

* Modify `AbstractInboundFileSynchronizer` to compare `lastModified` of the remote file with local version if `preserveTimestamp == true`
* Also replace `AcceptOnceFileListFilter` to the `FileSystemPersistentAcceptOnceFileListFilter` since the first one doesn't care about `lastModified`.
If `preserveTimestamp` isn't in use the `FileSystemPersistentAcceptOnceFileListFilter` works fully similar to the `AcceptOnceFileListFilter`

**Cherry-pick to 4.3.x except test-case**

Make `SftpStreamingMessageSourceTests` compatible with Windows

Looks like SFTP embedded server provides different permissions for files on different OSs

Doc Polishing

Conflicts:
	spring-integration-file/src/test/java/org/springframework/integration/file/remote/synchronizer/AbstractRemoteFileSynchronizerTests.java
	spring-integration-ftp/src/test/java/org/springframework/integration/ftp/dsl/FtpTests.java
	spring-integration-sftp/src/test/java/org/springframework/integration/sftp/inbound/SftpStreamingMessageSourceTests.java
	src/reference/asciidoc/whats-new.adoc
Resolved.
2017-02-24 15:20:11 -05:00
Artem Bilan
2d4385a871 Fix tests for wrong messages in assertions
https://build.spring.io/browse/INT-MASTER-560/

**Cherry-pick to 4.3.x**

(cherry picked from commit af711a0)
2017-02-24 13:14:24 -05:00
Gary Russell
250405ddb8 INT-4224: Improve Direct Handler Binding Docs
JIRA: https://jira.spring.io/browse/INT-4224

Explain when `ref` can can be used on certain endpoints.

(cherry picked from commit 56f72cb)
2017-02-24 12:32:01 -05:00
Artem Bilan
73e4bdc6b6 INT-4227: (4.3) Allow Custom Messaging Annotations
JIRA: https://jira.spring.io/browse/INT-4227

Expose `MessagingAnnotationPostProcessor` for inheritors.
The `CustomMessagingAnnotationTests` demonstrates how custom Messaging Annotation can be registered and used

**Cherry-pick to 4.3.x**

Add `setupCustomPostProcessors()` and `addMessagingAnnotationPostProcessor` to the `MessagingAnnotationPostProcessor`

Expose some API for inheritors

Conflicts:
	spring-integration-core/src/main/java/org/springframework/integration/config/annotation/MessagingAnnotationPostProcessor.java
2017-02-24 12:22:55 -05:00
Gary Russell
006f7cfd66 INT-4236: Status for SmartLifecycleRoleControlller
JIRA: https://jira.spring.io/browse/INT-4236

Add methods to obtain status from the `SmartLifecycleRoleController`.

* Polishing according PR comments.
* Fix log messages in the `SmartLifecycleRoleController` from the Zookeeper mentioning

Conflicts:
	src/reference/asciidoc/endpoint.adoc
Resolved.

Also Java 6 Compatibility.
2017-02-23 16:42:41 -05:00
Gary Russell
9f7e10f927 INT-4230: Fix (S)FTP Preserve Timestamp on MGET
JIRA: https://jira.spring.io/browse/INT-4230

The `get()` method only preserves the timestamp when an LS operation is done within it.
For MGET ops, we don't perform another LS, so the timestamp was not updated.

Add code to the recursive and non-recursive MGET methods to preserve the timestamp if so configured.

Conflicts:
	spring-integration-file/src/main/java/org/springframework/integration/file/remote/gateway/AbstractRemoteFileOutboundGateway.java
	spring-integration-ftp/src/test/java/org/springframework/integration/ftp/outbound/FtpServerOutboundTests-context.xml
	spring-integration-sftp/src/test/java/org/springframework/integration/sftp/outbound/SftpServerOutboundTests-context.xml
2017-02-21 12:56:07 -05:00