The default `TTL` in the `DefaultLockRepository` is 10 seconds which mislead us
when other client instance has just 100 millis.
So, not-adjusted client spins for expired lock wastefully leading to the
synchronization barriers to fail
**Cherry-pick to `5.4.x`**
* Increase a `default-delay="1000"` for `GatewayInterfaceTests`
to test the late reply scenario
* Use a `ConcurrentHashMap` for `JdbcMessageStore.queryCache`
to avoid a possible `ConcurrentModificationException`
* Upgrade to Gradle `7.3.3`
* Upgrade to the latest milestones for releasing
* Upgrade to Kryo-5.2.1 and update source code according breaking changes
* Upgrade to H2-2.1.210 and re-enable respective stored procedure tests
Turns out something is broken in H2 `Parser` for stored procedures.
So, parameters are now doubled which is definitely not expected.
* Disable those H2 tests where parametrized stored procedures are used
* Add `--add-opens` to `asciidoctor` Gradle task to avoid compilation warning
* Upgrade to Kotlin `1.6.10` and modify its Gradle task respective
* Add empty impl for new `AnnotationMetadataAdapter.getDeclaredMethods()`
* Parse HTTP methods to new `String methodNames` property of the `RequestMapping`
since `HttpMethod` is not an `enum` anymore.
* Ignore JDBC tests which rely on a map property resolution: the `[]` placeholder
is not handled in the latest SF anymore.
* Fix `WebFluxInboundEndpoint` for deprecations
* Upgrade to Java 17, SF-6.0, Gradle 7.2
* Upgrade to Jakarta dependencies and respective namespaces
* Fix some tests for Java 17 compatibility
* Fix wrong Javadocs
* Add some missed Javadocs
* Fix more `jakarta` namespace
* Fix WS & XML modules to use Jakarta EE
* `--add-opens` in some modules for their reflection-based tests
* Disable Kafka tests which does not work on Windows; see Apache Kafka `3.0.1`
* Upgrade to JUnit `5.8.1`
* Migrate JMS tests to Artemis
* Remove RMI module as it was deprecated before
* Fix `pr-build-workflow.yml` for Java 17
* Fix JavaDocs warnings using `Xdoclint:syntax` per module, not in the top-level `api` task
* Move docs for version `6.0`
Fixes https://github.com/spring-projects/spring-integration/issues/3683
If a transaction is already active while `JdbcLockRegistry` uses
`DefaultLockRepository` to acquire/release a lock, the repository
must execute SQL queries in a separate transaction to prevent
problems with blocking, deadlocking etc.
It also allows to properly follow transaction isolation level that
is set on some methods of `DefaultLockRepository`.
Previously all methods of DefaultLockRepository supported the current
transaction if there are any. Now all methods will always create new
transaction on each call.
* Change tests to be more unit-ish
* Change `@since` for the `DefaultLockRepositoryTests` to `5.3.10`
**Cherry-pick to `5.4.x` & `5.3.x`**
Fixes https://github.com/spring-projects/spring-integration/issues/3578
The `JdbcTemplate.queryForMap()` extract values for columns to the closer target driver types.
For example H2 and Derby return `Long` for `BIGINT`.
Oracle for its `NUMBER(19,0)` returns `BigInteger`.
This makes the code in the `JdbcMessageStore.getMessageGroup()`
not platform independent.
* Fix `JdbcMessageStore` to map `ResultSet` to the `MessageGroupMetadata`
directly.
Mostly reinstating the previous behavior
* For that reason expose a default ctor for `MessageGroupMetadata`
and extract some setters to make code in the `JdbcMessageStore` more cleaner.
* This opens for us a possibility to implement a `MessageGroupStore.getGroupMetadata(groupId)`
for `JdbcMessageStore`
* Fix deprecation for `Flux.limitRequest()`
Fixes https://github.com/spring-projects/spring-integration/issues/3567
* Retry for TransactionSystemException in JdbcLockRegistry
* Unit test added for `TransactionSystemException`
* Completing author and copyright year information
**Cherry-pick to `5.4.x` & `5.3.x`**
* Fix `MiscellaneousTests` to extend `ActiveMQMultiContextTests`
to let it to close ActiveMQ connection factory in the end of test
* Make `RequestReplyScenariosWithTempReplyQueuesTests` more robust
closing `DefaultMessageListenerContainer` and `ExecutorService`
in the end of tests
* Fix new Sonar smells
* Rework some SQL calls in the `JdbcMessageStore` to note expose a
`PreparedStatement` API
* Fix JavaDoc in the `CorrelationHandlerSpec`
* Introduce a `groupConditionSupplier` for MGS
* Add a `MessageGroup.condition` option
* Add a `MessageGroupStore.conditionSupplier` option
* Use it from the `SimpleMessageStore.addMessagesToGroup()` API
to populate `condition` (if any) into a `MessageGroup`
* Introduce a `GroupConditionProvider` contract to be implemented
on those `ReleaseStrategy` contracts which could be aware of group condition
* Populated a `GroupConditionProvider.getGroupConditionSupplier()`
into a `MessageGroupStore` from the `AbstractCorrelatingMessageHandler`
for end-user convenience
* Rework a `FileMarkerReleaseStrategy` to implement a `GroupConditionProvider`
to provide a function which produces a condition from `file_lineCount` header
of the `END` marker message
* Make the `FileMarkerReleaseStrategy` logic already based on the condition from a group
* Delegate `GroupConditionProvider` from the `FileAggregator`
* Add test for empty file aggregation
* * Implement `condition` in the `AbstractKeyValueMessageStore` and `MongoDbMessageStore`
* Test `condition` for `mongo-aggregator-config.xml` and `FileAggregatorTests` against GemFire
* * Implement `condition` in the `ConfigurableMongoDbMessageStore`
* * Implement `condition` in the `JdbcMessageStore`
* `FileAggregatorTests` against `JdbcMessageStore`
* Refactor `JdbcMessageStore` for better handling of message group metadata
* Remove unused `MARKED` column in the DDL in favor of newly introduced `CONDITION`
* * Add docs for message group condition
* * Move `conditionSupplier` option from MGS to AbstractCorrelatingMH
* Make it as a `BiFunction` to propagate existing condition alongside with the
message to consult
* Expose `groupConditionSupplier` in Java & XML DSLs
* * Fix language in docs
Fixes https://github.com/spring-projects/spring-integration/issues/3446
* For better resources utilization provide a `Stream<Message<?>>` API
on the `MessageGroupStore`, `MessageGroup` and `MessageGroupQueue`
* Use this API in the `DelayHandler` when it reschedules persisted messages
Fixes https://github.com/spring-projects/spring-integration/issues/3445
The `JdbcLockRegistry` improperly unlock the delegate (a `ReentrantLock`)
in the case of `TransientDataAccessException` or a `TransactionTimedOutException` is thrown.
As this occurs in a while loop, the next time the delegate is unlocked,
it not longer as a owner thread and throws an `IllegalMonitorStateException`.
* Move `delegate.unlock()` outside the while loop in the `JdbcLock.unlock()` method
* Add `JdbcLockRegistryDelegateTests` for mocked `LockRepository` to add more coverage
for `JdbcLockRegistry` functionality
**Cherry-pick to 5.4.x, 5.3.x & 5.2.x**
Fixes https://github.com/spring-projects/spring-integration/issues/3428
The Kotlin Gradle plugin starting with Kotlin `1.4` adds a standard
Kotlin lib into `compile` scope fully ignoring an `optional` variant.
See more info here: https://kotlinlang.org/docs/reference/using-gradle.html#dependency-on-the-standard-library
* Add `kotlin.stdlib.default.dependency=false` into `gradle.properties`
to rely only on the explicit dependency definitions in the project config
* Remove explicit Kotlin deps from test scope in the target modules since
it is declared globally for all the modules
* Fix author emails in the `publish-maven.gradle` to their VMware variants
* Remove usage of `org.jetbrains.annotations` in favor of `@Nullable`
from Spring Framework
* Implement `Emission` handling in the `FluxMessageChannel` and `IntegrationReactiveUtils`
* Upgrade to Spring Kafka `2.6.0`
* Fix R2DBC components for deprecation in the Spring Data R2DBC
* Implement `StatementMapper.SelectSpec` for query expression
* Clean up for some sporadic test failures
Thanks to the pair of `CLIENT_ID`/`CREATED_DATE`,
it is possible to know if the lock is expired, already held or can be reacquire.
I think that the pre delete row is unnecessary and redundant with these two columns.
The rows doesn't need to be purged because the number of row in the table is finite and equal to the number of locks.
At worst, the table will be able to be purged by `deleteExpired()`.
I propose to overwrite these two columns at acquire lock time instead of pre delete the row.
So `delete` then `update` or `insert` become `update` or `insert`.
One client held a lock.
- Another client will be able to hold the same lock only if:
1. the row doesn't exist. Done when the first client will call `unlock()`.
2. the lock expire in case of hardware shutdown. Done by "`OR CREATED_DATE<?`" in `updateQuery`.
- The same client will be able to reacquire it only if:
1. the row doesn't exist.
2. the lock is already held by him. Done by "`CLIENT_ID=? OR`" in `updateQuery`.
**Cost with PostgreSQL:**
```
EXPLAIN DELETE FROM INT_LOCK WHERE REGION='DEFAULT' AND LOCK_KEY='FOO' AND CREATED_DATE<'2020-07-14';
Delete on int_lock (cost=0.14..8.17 rows=1 width=6)
-> Index Scan using int_lock_pk on int_lock (cost=0.14..8.17 rows=1 width=6)
Index Cond: (((region)::text = 'DEFAULT'::text) AND (lock_key = 'FOO'::bpchar))
Filter: (created_date < '2020-07-14 00:00:00'::timestamp without time zone)
EXPLAIN UPDATE INT_LOCK SET CREATED_DATE='2020-07-15' WHERE REGION='DEFAULT' AND LOCK_KEY='FOO' AND CLIENT_ID=NULL;
Update on int_lock (cost=0.00..11.40 rows=1 width=520)
-> Result (cost=0.00..11.40 rows=1 width=520)
One-Time Filter: false
-> Seq Scan on int_lock (cost=0.00..11.40 rows=1 width=520)
```
```
EXPLAIN UPDATE INT_LOCK SET CLIENT_ID=NULL, CREATED_DATE='2020-07-15' WHERE REGION='DEFAULT' AND LOCK_KEY='FOO' AND (CLIENT_ID=NULL OR CREATED_DATE<'2020-07-14')
Update on int_lock (cost=0.14..8.17 rows=1 width=372)
-> Index Scan using int_lock_pk on int_lock (cost=0.14..8.17 rows=1 width=372)
Index Cond: (((region)::text = 'DEFAULT'::text) AND (lock_key = 'FOO'::bpchar))
Filter: (created_date < '2020-07-14 00:00:00'::timestamp without time zone)
```
Fixes https://github.com/spring-projects/spring-integration/issues/3272
* Introduce a `RenewableLockRegistry` since not all `LockRegistry` implementations
provide a way to renew the lease for the lock
* Implement `RenewableLockRegistry` in the `JdbcLockRegistry`
* Test and document the feature
Fixes https://github.com/spring-projects/spring-integration/issues/3307
The `DefaultLockRepository.acquire()` is transactional method and
can fail with the `TransactionTimedOutException`.
When we call `JdbcLock.lock()`, we expect an attempt until we really obtain a lock.
* Treat a `TransactionTimedOutException` as a `TransientDataAccessException` and
therefore retry a locking attempt logic
**Cherry-pick to 5.3.x, 5.2.x & 4.3.x**
* GH-3282: Fix JdbcMetadataStore for DuplicateKeyEx
Fixes: https://github.com/spring-projects/spring-integration/issues/3282
The `INSERT INTO ... SELECT ... FROM ... HAVING` may fail with
`DuplicateKeyException` in between transactions.
* Catch `DuplicateKeyException` from in the `tryToPutIfAbsent()`
and return `0` as a fact of not inserted to let other logic to work
as expected.
There is no test coverage for this since it is almost impossible to
reproduce such a race condition on DB
**Cherry-pick to 5.3.x & 5.2.x**
* * Remove misleading message in the comment sentence
Fixes https://github.com/spring-projects/spring-integration/issues/3243
It turns out that `ConcurrentModificationException` is thrown from the
`HashMap.computeIfAbsent(HashMap.java:1134)` since Java 9 when the map is
modified concurrently independently from the key we try to modify
* Check for the value presence before computing
* `synchronized(this.jdbcCallOperationsMap)` around `computeIfAbsent()`
when `get() == null`
**Cherry-pick to 5.2.x**
* Remove Boon dependency and its usage
* Remove overloaded methods from the `IntegrationFlowDefinition`
- we can simply rely now on the super class
* Remove (or rework) deprecated entities in the docs
* Fix tests for removed deprecated APIs
* Rework affected tests to JUnit 5
Fixes https://github.com/spring-projects/spring-integration/issues/3132
It turns out that Checkstyle EmptyBlock doesn't complain about
empty default ctor.
Plus a new check for `super();` call treats it as a violation
* Remove `super();` from all the no-arg ctors
* Code style clean up in the affected classes according
IDEA suggestions
* Fix new Sonar smells
* Move `What's New` into `changes-5.1-5.2.adoc`
* Remove version from the `XSD` files:
1. Align with SF
2. The version for XSD doesn't matter for the current jar version,
since only the version is available in classpath is from the current
jar version
3. Modify `spring.schemas` to map all the possible versions for XSD
into the current one in a jar.
This way target applications can upgrade without changing the version
for XSD location and we don't need to require `versionless` variant
any more
4. The jar in classpath can handle only its own XSD, so independently
of version or no-version variants we still may fail because the current
jar may not support end-user code any more - need some fix over there
anyway.
* Remove `checkTestConfigs` Gradle task since we don't worry about XSD
version any more
* Modify `schemaZip` Gradle task to populate XSDs into a distribution
still with a version to avoid overriding on-line XSDs for version `1.0.x`.
We may consider not do that since SF doesn't and just bite a bullet
for always overriding those on-line XSDs to the actual one from the latest
release.
Fixes https://github.com/spring-projects/spring-integration/issues/3129
The `CHAR` type in Oracle is for fixed length values.
If the value is not of expected length it is padded with whitespaces.
This causes wrong assumptions when we perform queries against tables.
* Change the `CHAR` into a `VARCHAR2` in those tables for Oracle
where we don't have values with a fixed length.
In fact only `MESSAGE_ID` comes with always fixed length as `36`
because it is fully based on the `UUID`
