Fixes https://github.com/spring-projects/spring-integration/issues/3213
* Expose a `LeaderInitiatorFactoryBean.setCandidate()` for better control over candidate options
* Expose a `candidate` attribute for the `<int-zk:leader-listener>` tag
* Fix `dokka` task to obtain `package-list` URL from the local dir
* Migrate ZK tests to JUnit 5
Fixes https://github.com/spring-projects/spring-integration/issues/3132
It turns out that Checkstyle EmptyBlock doesn't complain about
empty default ctor.
Plus a new check for `super();` call treats it as a violation
* Remove `super();` from all the no-arg ctors
* Code style clean up in the affected classes according
IDEA suggestions
* Fix new Sonar smells
* Move `What's New` into `changes-5.1-5.2.adoc`
* Remove version from the `XSD` files:
1. Align with SF
2. The version for XSD doesn't matter for the current jar version,
since only the version is available in classpath is from the current
jar version
3. Modify `spring.schemas` to map all the possible versions for XSD
into the current one in a jar.
This way target applications can upgrade without changing the version
for XSD location and we don't need to require `versionless` variant
any more
4. The jar in classpath can handle only its own XSD, so independently
of version or no-version variants we still may fail because the current
jar may not support end-user code any more - need some fix over there
anyway.
* Remove `checkTestConfigs` Gradle task since we don't worry about XSD
version any more
* Modify `schemaZip` Gradle task to populate XSDs into a distribution
still with a version to avoid overriding on-line XSDs for version `1.0.x`.
We may consider not do that since SF doesn't and just bite a bullet
for always overriding those on-line XSDs to the actual one from the latest
release.
Fixes https://github.com/spring-projects/spring-integration/issues/2987
To resolve XSD files properly from the classpath, their HTTPS reference
must be present in the `spring.schemas` to avoid the Internet interaction
for resolving an XSD file
**Cherry-pick to 5.1.x, 5.0.x & 4.3.x**
* Fix `SmartLifecycle.stop(Runnable)` usage
We always have to call `callback` in the `SmartLifecycle.stop(Runnable)`
implementation independently of component state
* Fix `StandardIntegrationFlow.stop(Runnable)` for a logic when we
don't have any `this.lifecycles`
* Remove those `stop(Runnable)` which are fully equivalent of the
`default` on in the `SmartLifecycle`
* Some other simple polishing for the affected classes, e.g.
`isSingleton()` is `default` with `true` in the `InitializingBean`
**Cherry-pick to 5.1.x**
* * Fix checkstyle violation
* Do not fix urls for the `http://apache.org/xml/features` when we
configure XML components
* Do not fix urls for localhost and fake links in tests
* Fix all the dead links in docs
* Fix new Sonar smells for MMIH
* Fix complexity smells in the `MessagingMethodInvokerHelper`
* Reuse `get()` in the `ZookeeperMetadataStore` in case of `putIfAbsent()` error
* Polishing for `ZookeeperMetadataStoreTests`
* Fix for `hasUnqualifiedMapParameter`
* Fix `findHandlerMethodsForTarget()` complexity
* * Fix typo and remove FQCN for `ClassUtils`
Mostly thanks to IDEA's plugin: https://plugins.jetbrains.com/plugin/10345-assertions2assertj
There is still a lot of work to do when complex and composite matchers are used.
* Add `awaitility` dependency and deprecate `EventuallyMatcher` in favor
of `awaitility`
* Remove Hamcrest from dependencies and disable JUnit & Hamcrest
static imports to encourage to use only AssertJ
* Migrate JUnit assumptions in rules to AssertJ's assumptions
* Deprecate some custom matchers in favor of existing in Hamcrest
after upgrading the last to version `2.1`
* Replace `ExpectedException` rules with `assertThatThrownBy()`
* Mention `MessagePredicate` in the `testing.adoc`
Creating a EPHEMERAL_SEQUENTIAL node was being used to ensure the
Zookeeper connection had been established. These were never explicitly
removed, which led to tryLock effectively leaking zNodes until the
client was terminated. This in turn lead to an extremely large number of
ephemeral nodes being deleted whenever a long running service
terminated!
This fix replaces the creation of a node with a stat of "/" instead.
This has the desired effect of re-establishing the Zookeeper connection,
but is a read-only operation.
* Remove unused import
There is a race condition when Zookeeper `LeaderInitiator` may start
earlier than `CuratorFramework` it depends on.
This is because both of them are configured for the same default `phase` (`0`)
* Make `LeaderInitiator` and `LockRegistryLeaderInitiator` to be started
in the `Integer.MAX_VALUE - 1000` phase by default - as late as possible
* Make `CuratorFrameworkFactoryBean` to start in the `Integer.MIN_VALUE + 1000`
phase - as early as possible
* Alight lifecycle properties from the `LeaderInitiatorFactoryBean` with
defaults in the `LeaderInitiator`
* Simplify a `LeaderListenerParser` to rely on the `LeaderInitiatorFactoryBean`
**Cherry-pick to 5.0.x**
JIRA: https://jira.spring.io/browse/INT-4477
When running multiple roles, it is useful to be able to determine the
role for a particular context, e.g. in an event.
* Docs
**cherry-pick to 5.0.x**
JIRA: https://jira.spring.io/browse/INT-4464
A `ZookeeperMetadataStore.get()` is based on the `this.cache` variable.
This one is initialized in the `start()`.
* Assert `isRunning()` in the `get()` before using.
**Cherry-pick to 5.0.x and 4.3.x**
* INT-4376: Upgage to Log4J 2
JIRA: https://jira.spring.io/browse/INT-4376
* Deprecate Log4J 1.x components in favor of newly added
a `Log4j2LevelAdjuster` JUnit `@Rule`
* Update all the logging configuration to Log4J 2
* Polishing after testing
* Check for `null` package in the `JacksonJsonUtils`.
Also don't call `this.delegate.typeFromId()` the second time
* Increase wait timeout in the `QueueChannelTests`
* Check `ZookeeperMetadataStore.addListener()` for `null`
**Cherry-pick to 4.3.x**
This commit harmonizes `LockRegistry#obtain` logic in Zookeeper and JDBC
implementations with Redis implementation
by using Java 8 `ConcurrentMap#computeIfAbsent`
* Remove `synchronized (this.locks)` from the `expireUnusedOlderThan`
implementations because `iterator()` is thread-safe from `ConcurrentHashMap`
* Fix deprecation in the `IntegrationGraphControllerRegistrar`
* Revert Spring Security version to `4.2.2`, since `5.0 B-S` is broken
JIRA: https://jira.spring.io/browse/INT-4220
Currently when an error is thrown from the event publishing the role granting is broken and we just go to the role revoking.
* Since it's just an event publishing it shouldn't effect the original leader election.
* `try...catch` event publishing in the `LeaderInitiator` and `logger.warn` an `Exception`
* Make `leader/Context` as `@FunctionalInterface` for simple Lambda use-case like `NULL_CONTEXT` - `() -> false`
* Remove all the internal `NullContext` implementations in favor of above mention Lambda
* In the `LockRegistryLeaderInitiator` use `CustomizableThreadFactory` instead of custom `ThreadFactory` for prefixing
* Add `zookeeper/leader/LeaderInitiator#getContext()` for external usage and consistency with other similar components
* Fix `CuratorContext.toString()` typo
JIRA: https://jira.spring.io/browse/INT-4167
Since the current `ZkLock.tryLock()` is based on the fact of waiting for **at most 1 second**, that still has some timing issue for test-case, when the network delay may affect our expectation.
* Add `Thread.sleep()` with `while()` to the `ZkLockRegistryTests.testTryLock()` test to spin until successful `tryLock()`
**Cherry-pick to 4.3.x & 4.2.x**
Note: to avoid cherry-pick conflict for `4.2.x` just fully copy/paste the modified test
Polishing - PR Comments and Closeable Warnings
Eclipse emits bogus warnings with exceptions in lambdas.
Even though the lambda might run on another thread, elipse thinks it could
cause the context to not be closed.
SPR-14854: MessageChannel is now a @FunctionalInterface
* Additional Lambda polishing and some code style fixes
JIRA: https://jira.spring.io/browse/INT-4127
Fixes GH-1913 (https://github.com/spring-projects/spring-integration/issues/1913)
The current `tryLock()` implementation with the `0` timeout to wait for underlying `InterProcessMutex` answer makes this function almost fully impossible
* Fix `ZkLock.tryLock()` to delegate with the `tryLock(1, TimeUnit.SECONDS)` to give some time window chance for network communication.
It isn't ideal for the `tryLock()` contract, but at least it is better then don't let the feature to work at all.
**Cherry-pick to 4.3.x & 4.2.x**
JIRA: https://jira.spring.io/browse/INT-3913
* Remove deprecated classes and methods/constructors, deprecated XML attributes
* Remove `TcpConnectionEventListeningMessageProducer` and rework tests logic to the `ApplicationEventListeningMessageProducer`
* Fix several typos
* Remove/rework deprecated entities mentioning
https://build.spring.io/browse/INT-AT42SIO-253/
```
java.lang.AssertionError: Should have been able to lock with zookeeper server restarted!
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.springframework.integration.zookeeper.lock.ZkLockRegistryTests.voidLockFailsWhenServerDown(ZkLockRegistryTests.java:339)
```
When we do `testingServer.restart();` we should give more time to let it started on background, therefore increase the next `lock2.tryLock()` to 10 seconds.
**Cherry-pick to 4.3.x and 4.2.x**
JIRA: https://jira.spring.io/browse/INT-4087
Fixes GH-1863 (https://github.com/spring-projects/spring-integration/issues/1863)
If ZK Client loses the connection to ensemble, it tries to reconnect on the next operation and does that exactly for the `connectionTimeoutMs` and provided `RetryPolicy`.
According to the `Lock.tryLock(long time, TimeUnit unit)` contract we can wait for the lock only during provided `timeout`.
* Introduce `mutexTaskExecutor` and perform `ZK.forPath()` command in the separate `Thread` and wait for the `Future` during provided timeout.
* Perform `mutex.acquire()` only after successful `Future` result and only for the remained timeout.
**Cherry-pick to 4.2.x**
Add `setMutexTaskExecutor()`
Fix Checkstyle typo
Javadoc
* Fix several typos in log messages. And some test on the matter as well
* Add comment to `AbstractPersistentAcceptOnceFileListFilter.rollback()` to clarify the reason of `rollingBack` variable
* Make `RemoteFileTemplate.StreamHolder` as `static` to avoid extra internal variable to outer class instance
* Replace `MessagingException` with `AbstractInboundFileSynchronizingMessageSource` in `init()` method of some components. It isn't Messaging yet in that phase
* Fix `SubscribableRedisChannel.MessageListenerDelegate` to handle `Object` not `String`, because with the `serializer` injection there is no guaranty that incoming is always `String`
* Move `JSch.setLogger(new JschLogger());` in the `DefaultSftpSessionFactory` to `static` block. It really should be done only once
* Remove `Assert.isTrue(this.port >= 0)` from the `DefaultSftpSessionFactory`. The subsequant `initJschSession()` convert it to default `22` port
* Change in the `JschProxyFactoryBean` `UnsupportedOperationException` to `IllegalArgumentException`. Wrong enum is wrong argument. That isn't a problem of operation
* Simplify `stop()` in the `CuratorFrameworkFactoryBean` and mark it as a `this.running = false`. Otherwise it wasn't able to be restarted
* Expose `leaderEventPublisher` in the `LeaderInitiatorFactoryBean` and fix `stop(Runnable callback)` with propagation `callback` to delegate.
Fix `SubscribableRedisChannelTests` for new `handleMessage(Object)` signature
Two initiators for the same path, same `SmartLifecycleRoleController` and, finally, same `adapter`.
So, one initiator after `yield()` stops the `adapter` and at the same time another starts it.
Since there is no barrier in between events and assertion, we end up with an early "re-granting".
* Add `CountDownLatch yieldBarrier` to `countDown()` after performing second `adapter.isRunning()` assert
* `LeaderEventPublisher` waits for the `yieldBarrier` after the first `OnRevokedEvent`
JIRA: https://jira.spring.io/browse/INT-4058
If you hold the lock, you are the leader. This simple idea gets you
a long way if there is no "native" leader initiator. E.g. you
can use this with a RDBMS with JdbcLockRegistry.
Add some docs on leader election under "endpoints"
Make thread name for leader initiator unique
In case there are multiple instances in the same context we would
like to be able to spot them in the logs.
INT-4058: Polishing
* Code refactoring in the `LockRegistryLeaderInitiator`: SI use 120 line length
* Add `Assert.notNull()` for required properties
* Add `this.lock.unlock()` and `Thread.sleep(LockRegistryLeaderInitiator.this.busyWaitMillis)` into the `catch` block to let distributed elections to work.
Otherwise there is a big chance that we will acquire the lock and become a leader again just after `yield()`
* Change `LockContext.toString()` to use simple `String` concatenation which is optimized by compiler to the `StringBuilder`.
That let to have some better micro-performance compared with with extra `Formatter` object in case of `String.format()`
* Add `JdbcLockRegistryLeaderInitiatorTests`
* Fix `yield()` logic based on the `Future.cancel(true)`.
Since one `FutureTask.cancel(true)` makes it as `INTERRUPTED` any subsequent `yield()` does not make any effect, therefore our infinite selector loop isn't interrupted one more time.
* Reschedule the `LeaderSelector` in the `yield()` after cancel(true).
* Rework `catch` in the selector loop just to the `InterruptedException` and `return null;` to stop looping and let reschedule the selector.
* Add one more `onRevoked` logic into the `final` of the selector loop to notify that we have lost leadership during `stop()`
* Improve `JdbcLockRegistryLeaderInitiatorTests` to ensure that several `yield()` on the same initiator work well.
* Add `What's New` note.
* Upgrade to SA and SF snapshots
* Address TODOs regarding those upgrades
* Resolve some other TODOs
* Replace `PropertyPlaceholderConfigurer` beans with the `<context:property-placeholder>`
* Upgrade to Spring Social Twitter 1.1.2 and resolve deprecations via mocks (https://github.com/spring-projects/spring-social-twitter/issues/91)
* Upgrade to Curator `3.1.0` and resolve deprecation in the `ZookeeperMetadataStore`
fixes
fixModifiers after fixFinal
Revert CachingSessionFactory
Class is spied in tests.
checkstyle - Import Rules
checkstyle InterfaceIsType
checkstyle InnerTypeLast
checkstyle OneStatementPerLine
CovariantEquals
OneTopLevelClass
* Revert `'\n'` -> `System.lineSeparator()` in the Gradle scripts to meet Git `autocrlf = true` on Windows
* Fix timing issue with the `LastModifiedFileListFilterTests`, when the `age = 1` might not be enough for the file object when we have some delay before checking
* `gradlew clean check -x test --parallel --continue` - to collect reports
* `gradlew fixThis --parallel` - to fix all possible vulnerabilities. With `-Dfile.encoding=UTF-8` on Windows
Since the `RequireThisCheck` doesn't see parents for anonymous classes (e.g. `Runnable` callback), its report doesn't contains the outer class name with `this.`,
therefore we still have to fix those cases manually.
Thanks to the wrong `replacer` just with `this.` we have uncompilable code enough easy to find problems.
Not so easy to fix for good readability though...
* Upgrade to Grade 2.12
* Upgrade to SonarQube native plugin
The fix contains at about 300 files. So, will be done on merge.
Fix `fixThis.gradle` according PR comments
Apply `fixThis` and also `fixModifiers` for test classes.
Fix some `this.` inner issues manually.
Make code polishing for long lines after `fixThis`
Fix conflicts and vulnerabilities after the rebase
JIRA: https://jira.spring.io/browse/INT-3926
* Add `<poller>` for all `<outbound-adapter(gateway)>`
* Add/modify tests for `<queue>` as an input channel and particular `<poller>`
* Move XSD from the `xml` package to the `config` level to achieve the consistency throughout the project.
JIRA: https://jira.spring.io/browse/INT-3916
The `JpaOutboundGatewayFactoryBean` used CTOR injection for the `JpaExecutor`.
That one, in turn, uses CTOR injection for the `EntityManagerFactory`.
Such a dependency may cause the `early bean instantiating` in case of `AbstractAutowireCapableBeanFactory.getSingletonFactoryBeanForTypeCheck()`.
And we end up with the `BeanCurrentlyInCreationException`.
Therefore no one `FactoryBean` should use CTOR injection if there is a potential hierarchical dependency.
NOTE: there is no tests on the matter, since we don't change the components behavior.
The `JPA` sample application will be changed to the Boot to track this fix.
**Cherry-pick to 4.2.x**
Address PR comments and fix other `FactoryBean`s for the same issue, when it is reasonable
Polishing
Address PR comments
Make setter `setSockJsTaskScheduler` as `public`
Leadership was yielded before testing whether the adapter was started when leadership was granted.
It was possible for the adapter to be stopped again before testing `isRunning()`.
JIRA: https://jira.spring.io/browse/INT-3781
- Add mechanisms to set status code on an inbound gateway timeout.
- Send a message to the error channel if configured.
Polishing - Add failedMessage to MTE
Fix Schema Docs; Timeout Detection on Error Flow
Add Zookeeper Leadership Logs
Fix typo in the `SmartLifecycleRoleController`
This should be treated as experimental but with these changes it seems to build ok.
BUILD SUCCESSFUL
Total time: 5 mins 15.533 secs
It turns out that our sporadic Redis problems were fixed in a later version of Jedis.
The problem was that the `connection.subscribe()` exited immediately. When I started adding debug
logic, the problem went away (because I was using a newer versio of Jedis).
spring-data-redis 1.5.2 is updated to work with the 2.7.3 version of Jedis.
Polishing
