Commit Graph

10402 Commits

Author SHA1 Message Date
Artem Bilan
8b16aede89 Fix new Sonar smells 2020-07-16 11:36:04 -04:00
Artem Bilan
b3111414fb Fix Checkstyle violation about import order 2020-07-15 13:29:46 -04:00
Alexandre Strubel
e4611d97a1 JDBC Lock acquire optimization
Thanks to the pair of `CLIENT_ID`/`CREATED_DATE`,
it is possible to know if the lock is expired, already held or can be reacquire.
I think that the pre delete row is unnecessary and redundant with these two columns.
The rows doesn't need to be purged because the number of row in the table is finite and equal to the number of locks.
At worst, the table will be able to be purged by `deleteExpired()`.

I propose to overwrite these two columns at acquire lock time instead of pre delete the row.
So `delete` then `update` or `insert` become `update` or `insert`.

One client held a lock.
- Another client will be able to hold the same lock only if:
1. the row doesn't exist. Done when the first client will call `unlock()`.
2. the lock expire in case of hardware shutdown. Done by "`OR CREATED_DATE<?`" in `updateQuery`.
- The same client will be able to reacquire it only if:
1. the row doesn't exist.
2. the lock is already held by him. Done by "`CLIENT_ID=? OR`" in `updateQuery`.

**Cost with PostgreSQL:**
```
EXPLAIN DELETE FROM INT_LOCK WHERE REGION='DEFAULT' AND LOCK_KEY='FOO' AND CREATED_DATE<'2020-07-14';
Delete on int_lock  (cost=0.14..8.17 rows=1 width=6)
  ->  Index Scan using int_lock_pk on int_lock  (cost=0.14..8.17 rows=1 width=6)
        Index Cond: (((region)::text = 'DEFAULT'::text) AND (lock_key = 'FOO'::bpchar))
        Filter: (created_date < '2020-07-14 00:00:00'::timestamp without time zone)

EXPLAIN UPDATE INT_LOCK SET CREATED_DATE='2020-07-15' WHERE REGION='DEFAULT' AND LOCK_KEY='FOO' AND CLIENT_ID=NULL;
Update on int_lock  (cost=0.00..11.40 rows=1 width=520)
  ->  Result  (cost=0.00..11.40 rows=1 width=520)
        One-Time Filter: false
        ->  Seq Scan on int_lock  (cost=0.00..11.40 rows=1 width=520)
```
```
EXPLAIN UPDATE INT_LOCK SET CLIENT_ID=NULL, CREATED_DATE='2020-07-15' WHERE REGION='DEFAULT' AND LOCK_KEY='FOO' AND (CLIENT_ID=NULL OR CREATED_DATE<'2020-07-14')
Update on int_lock  (cost=0.14..8.17 rows=1 width=372)
  ->  Index Scan using int_lock_pk on int_lock  (cost=0.14..8.17 rows=1 width=372)
        Index Cond: (((region)::text = 'DEFAULT'::text) AND (lock_key = 'FOO'::bpchar))
        Filter: (created_date < '2020-07-14 00:00:00'::timestamp without time zone)
```
2020-07-15 13:21:08 -04:00
Gary Russell
9c252be028 Fix Kafka Send Timeout
Resolves https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/928

Kafka has a longer default send timeout; this means a send could be successful long
after Spring has timed out the send.

**I will backport to the 3.3.x extension after merge**

* Use setSendTimeout; make it final.
2020-07-15 10:35:33 -04:00
Artem Bilan
a19e37237b Fix new and some old Sonar smells 2020-07-14 10:49:28 -04:00
Jay Bryant
df1f260a59 Wording changes
Replacing some terms
2020-07-13 16:31:03 -04:00
Artem Bilan
6e11d4cdf9 Fix new and some old Sonar smells 2020-07-13 14:07:07 -04:00
Gary Russell
9223613ee1 ConsumerRecord Logging Metadata Option 2020-07-10 15:14:49 -04:00
rohanmukesh12
eb3c6a017f INT-4566: Introduce R2DBC Inbound Channel Adapter
JIRA: https://jira.spring.io/browse/INT-4566

* Some code clean up
2020-07-08 15:26:38 -04:00
Artem Bilan
7a09358582 SO-62761903: Inject BF into Gateway's correlator
Related to https://stackoverflow.com/questions/62761903/spring-integration-reactive-streams-support-exception-in-creating-a-reactive

The `MessagingGatewaySupport` creates an internal endpoint
for consuming messages from the provided `replyChannel`.
The endpoint type depends on the channel type.
The `ReactiveStreamsConsumer` was missing a `BeanFactory` injection
causing an error when `ReactiveStreamsConsumer` tries to extract a default `ErrorHandler`
from `BeanFactory`

* Refactor `MessagingGatewaySupport` to inject a `BeanFactory` to all
the possible correlator endpoints.
Also always call `afterPropertiesSet()` for all of them

**Cherry-pick to 5.3.x & 5.2.x**
2020-07-08 14:58:38 -04:00
Gary Russell
b45d690a45 Fix CachedSessionFactory Race
Close the pool so that any sessions returned after the factory is
`destroy()`ed are closed.

* Call `removeAllIdleItems()` in `close()`.

* Close sessions in `SftpStreamingMessageSourceTests`.

**cherry-pick to all supported branches**
2020-07-08 14:19:57 -04:00
Artem Bilan
0f7f0845f4 Remove overrides from IntegrationFlowDefinition
Since we don't need a backward bytecode compatibility since this version,
we don't need methods override in the `IntegrationFlowDefinition` any more
2020-07-08 12:51:00 -04:00
Artem Bilan
16803beaab GH-3321: Add version section into bug_report.md
Fixes https://github.com/spring-projects/spring-integration/issues/3321
2020-07-07 13:09:34 -04:00
Gary Russell
e919428680 GH-3326: TCP: Support Unsolicited Server Messages
Resolves https://github.com/spring-projects/spring-integration/issues/3326

- OB Gateway - send unsolicited messages and late replies to a channel
- Support multiple `TcpSender` s

* Add channel variant to spec; code polishing
2020-07-06 15:05:57 -04:00
Alexandre Strubel
e2c6e77f2f GH-3272: Add lease renewal for distributed locks
Fixes https://github.com/spring-projects/spring-integration/issues/3272

* Introduce a `RenewableLockRegistry` since not all `LockRegistry` implementations
provide a way to renew the lease for the lock
* Implement `RenewableLockRegistry` in the `JdbcLockRegistry`
* Test and document the feature
2020-07-06 12:46:47 -04:00
Artem Bilan
381a071287 Clean up RedisLockRegistryLeaderInitiatorTests
Related to https://build.spring.io/browse/INT-MASTERSPRING40-1129

When there is no other candidates in the cluster, the current `yield()`
will give a leadership to the current active initiator back
Therefore a `revoke()` might not be called at all

* Fir the `RedisLockRegistryLeaderInitiatorTests` to check revoking
after `stop()` instead of `yield()`
2020-07-06 11:30:43 -04:00
Alexander Shaklein
7424cb215c GH-3324. fix NPE in StdIntFlowContext
Fixes https://github.com/spring-projects/spring-integration/issues/3324

The `StandardIntegrationFlowContext.remove()` has a possible NPE 

**Cherry-pick to 5.3.x & 5.2.x**
2020-07-04 15:58:49 -04:00
Artem Bilan
146b0af1f4 GH-3320: Refine lifecycle control in StdIntFlow (#3322)
* GH-3320: Refine lifecycle control in StdIntFlow

Fixes https://github.com/spring-projects/spring-integration/issues/3320

Turns out that lifecycle control for the whole bunch of components
in one `IntegrationFlow` is useful in fields.

* Change the logic in the `StandardIntegrationFlow` to let to call
`start()` and `stop()` independently how the flow was registered in
the application context.
This way it can be autowired as a `Lifecycle` to let end-user to
avoid the search for proper component in the flow to stop or start
manually - all the components registered with the flow are going
to be stopped or started respectively

* * Add `this.` to class property usage to satisfy Checkstyle

Co-authored-by: Artem Bilan <abilan@vmware.com>
2020-07-02 15:29:43 -04:00
Artem Bilan
412d9f5400 Fix new Sonar smells 2020-07-02 10:30:03 -04:00
Artem Bilan
f62b0916b6 Fix MessagingGatewaySupport for reactive error (#3319)
* Fix MessagingGatewaySupport for reactive error

The `onErrorResume()` was in a wrong place for the
`doSendAndReceiveMessageReactive()`: we have to catch all the exceptions
from the top level `Mono`, not only a reply one as it was before.

Ensure in HTTP and WebFlux test that behavior is fixed

**Cherry-pick to `5.3.x` & `5.2.x`**

* * Remove unused imports

Co-authored-by: Artem Bilan <abilan@vmware.com>
2020-07-01 15:53:35 -04:00
Gary Russell
f3d61ec44a GH-3315: Fix (S)FTP Stream with Fair Rotation
Resolves https://github.com/spring-projects/spring-integration/issues/3315

`maxFetchSize` ignored with filters supporting single file filtering (default).
This breaks "fair" rotation with the `RotatingServerAdvice`.

Honor `maxFetchSize`, even with filters that support single file filtering.

**cherry-pick to 5.3.x, 5.2.x**
2020-07-01 15:47:35 -04:00
Artem Bilan
0ddbfa6e6d Remove Redis service from Travis config
Since Travis uses an old Redis version we can't rely on its service any more
2020-07-01 14:20:37 -04:00
Spring Buildmaster
0dc03ad731 [artifactory-release] Next development version 2020-06-26 17:20:54 +00:00
Spring Buildmaster
696271bf6a [artifactory-release] Release version 5.4.0-M1 2020-06-26 17:20:49 +00:00
Artem Bilan
b5225b9c33 Upgrade dependencies; prepare for release 2020-06-26 12:54:49 -04:00
Attoumane AHAMADI
afa79d868b GH-3226: Redis Stream Outbound Channel Adapter
Fixes https://github.com/spring-projects/spring-integration/issues/3226

* Redis stream message handler support.
* This is the outbound part publishing message to the actual stream using ReactiveStreamOperations

* Addition of more test cases with one using `MessageChannel`.

* Improvements after PR review.

* Removed failed test reading List from a Stream
* Code style clean up
* Remove `rawtypes` usage
* Remove redundant inner classes for test model
* Add `What's New` note
2020-06-26 11:28:41 -04:00
Florian Schmaus
4761800528 Fix XMPP documentation
s/Enhancement Proposal/Extension Protocol/

XMPP XEPs are extension protocols of XMPP "core" (i.e., RFC 6120 and
RFC 6121).

See also https://xmpp.org/extensions/xep-0001.html:
"The focal point of the process is a series of protocol specifications
called XMPP Extension Protocols or XEPs."
2020-06-26 08:56:13 -04:00
fnu, rohan
dfd914254b INT-4566: R2DBC Outbound Channel Adapter
JIRA: https://jira.spring.io/browse/INT-4566

* Fixed review comments
* added `DELETE` and `Criteria` implementation for outbound channel adapter
* Clean up code style
* Add initial docs
2020-06-25 12:45:24 -04:00
Gary Russell
b860a2e818 Merge Spring Integration Kafka into Main Repo
- merge master branch from old repo (`git-filter-repo`) for src branch
- update build.gradle
- copy docs from Spring for Apache Kafka Chapter

Doc polishing.

Fix http: references in tests.

* Add `What's New` note and fix `polling-consumer.adoc`
not linking to external project any more
2020-06-25 11:46:38 -04:00
Gary Russell
3ec74f380f KMSource - fix RebalanceListener (incremental)
Support cooperative rebalancing
 - incremental assignment/revocation
 - re-pause if a rebalance occurs while paused

**cheerry-pick to 3.2.x**
2020-06-25 11:29:09 -04:00
Artem Bilan
a28647259c Upgrade dependencies; prepare for release 2020-06-25 11:29:09 -04:00
Gary Russell
8771f8e18c Fix tests for latest SK snapshot 2020-06-25 11:29:08 -04:00
Artem Bilan
04019dbb96 Fix checkstyle.xml for Windows new line 2020-06-25 11:29:08 -04:00
Artem Bilan
bfef5a49cf Fix Kotlin tests according upstream changes
* Upgrade dependencies
2020-06-25 11:29:08 -04:00
Gary Russell
213772dcaf Fix S-K Deprecations 2020-06-25 11:29:08 -04:00
Gary Russell
1e484d0fad Support flush after send
* Add `flushExpression`.

* Add DSL, XML configuration.
2020-06-25 11:29:08 -04:00
Artem Bilan
e0be544a40 Add more XSD version into spring.schemas
* Fix some deprecation in `build.gradle`
2020-06-25 11:29:07 -04:00
Gary Russell
a58169f4e1 Fix Gradle Deprecations
* Rename schema

* Upgrade to Gradle 6.3-rc-3 for transitive dependency fix.

* Remove version from schema; polish `build.gradle` to remove unneeded features.
2020-06-25 11:29:07 -04:00
Gary Russell
81b9ae4a7a Use core Kotlin DSL 2020-06-25 11:29:07 -04:00
Gary Russell
0b7ea6d3c2 Remove deprecated methods 2020-06-25 11:29:07 -04:00
Gary Russell
bdd236052b Upgrade to SK 2.5.0
- add deliveryAttempt header when retry is not configured
- temporary work around for gradle bug, jar with test classifier missing from CP
2020-06-25 11:29:07 -04:00
Gary Russell
aaaaf86c07 GH-296: DSL: Support inline pub/sub subscriptions
Resolves https://github.com/spring-projects/spring-integration-kafka/issues/296

* * Add generic type to specsto avoid cast to `BroadcastCapableChannel`.

* * Remove unneeded fields in the abstract spec

* Restore log4j config
2020-06-25 11:29:06 -04:00
Gary Russell
2331eee642 GH-296: Add Kafka-backed MessageChannels
Resolves https://github.com/spring-projects/spring-integration-kafka/issues/296

* * Add DSL support

* * Polishing and XML support
2020-06-25 11:29:06 -04:00
Gary Russell
905a5155bc Master to 3.3; support non-Tx if template allows
* SI to 5.3.0.B-S
2020-06-25 11:29:06 -04:00
Artem Bilan
f596e49242 Upgrade to latest releases
* Upgrade to Gradle 6.1.1
* Fix for latest Checkstyle
* Fix for latest SI-Kotlin-DSL
2020-06-25 11:29:06 -04:00
Artem Bilan
27f0775fb8 Apply SI-Kotlin-DSL in tests (#292)
* Apply SI-Kotlin-DSL in tests

* Upgrade dependencies

* Upgrade to latest releases
2020-06-25 11:29:05 -04:00
Gary Russell
386c64c617 Support ProducerRecord outbound payloads 2020-06-25 11:29:05 -04:00
Gary Russell
cf3aa29a23 GH-290: Add ProducerRecordCreator
Resolves https://github.com/spring-projects/spring-integration-kafka/issues/290
2020-06-25 11:29:05 -04:00
Gary Russell
6cddb2c169 Revert "Apply SI-Kotlin-DSL in tests"
This reverts commit b03ddcb953c18469564994921efb84a072a0e358.
2020-06-25 11:29:05 -04:00
Artem Bilan
9b0b87d37e Apply SI-Kotlin-DSL in tests 2020-06-25 11:29:05 -04:00