Resolves: https://github.com/spring-projects/spring-integration-aws/issues/183
Exposes `BillingMode` option on the `DynamoDbMetadataStore` and
`DynamoDbLockRegistry`.
Moves the `LockRegistry` table creation to inside
this project as the DynamoDB Lock client doesn't support pay per request
model and that library seems abandoned.
Fixes https://github.com/spring-projects/spring-integration-aws/issues/184
During creating `KclMessageDrivenChannelAdapter`
if `config == null` a default `KinesisClientLibConfiguration` is created
with parameter `leaseCleanupIntervalMillis == 0` which causes an `IllegalArgumentExcpetoin`
when KPL is initialized
* Use default reasonable values for lease clean up intervals.
* Copy respective constants from the `KinesisClientLibConfiguration` into the `KclMessageDrivenChannelAdapter`
since they are `private` over there
Fixes https://github.com/spring-projects/spring-integration-aws/issues/177
* When the end of a shard is detected, checkpoint with the `endingSequenceNumber`
* Add a test
* code review
* Default to directly checkpointing closed shards except when in manual checkpoint mode
* update tests
* `endingSequenceNumber` should be higher than the last record's sequence number
* Checkpoint when in manual mode if shard was empty
Fixes https://github.com/spring-projects/spring-integration-aws/issues/158
It turns out that closed shards are still candidates for consumption
when we have concurrency
* Remove closed shards from the `shardConsumerManager.shardOffsetsToConsumer`
* Upgrade to Spring Integration 5.2.3, Jackson 2.10.2
Related to https://github.com/spring-cloud/spring-cloud-stream-binder-aws-kinesis/issues/117
It can be useful in the target application to have a full control
over KCL options to configure a `KclMessageDrivenChannelAdapter`
* Expose a `KinesisClientLibConfiguration`-based ctor in the
`KclMessageDrivenChannelAdapter`
* Add assert in setters when options must be configured on
the provided `KclMessageDrivenChannelAdapter`
* Extract `stream` and `group` options from the provided
`KclMessageDrivenChannelAdapter`
Fixes https://github.com/spring-projects/spring-integration-aws/issues/113
* Fix some properties in the `KclMessageDrivenChannelAdapter` for a
reasonable default value
* Introduce `LocalStackSslEnvironmentProvider` to populate a `USE_SSL`
ENV var into a Docker instance for Local Stack
* Add more factory methods into an `ExtendedDockerTestUtils`, especially
variants to obtain an HTTPS-based service clients
* Since Local Stack Kinesis support is based on Kinesalite, it turns out
that `System.setProperty(SDKGlobalConfiguration.AWS_CBOR_DISABLE_SYSTEM_PROPERTY, "true");`
does the trick to make Kinesis integration tests to pass with Local Stack
in Docker
* Suppress `serial` warning in the `AwsIntegrationEvent` classes
* Make `junit-jupiter-api` as `testCompile` dependency
* Fix README in regards Local Stack support for testing
Fixes https://github.com/spring-projects/spring-integration-aws/issues/112
* Introduce an `AwsIntegrationEvent` hierarchy
* Emit a `KinesisShardEndedEvent` from the `KinesisMessageDrivenChannelAdapter`
when current `shardIterator` is returned as `null` indicating the end
of the shard
* NOTE: `KclMessageDrivenChannelAdapter` cannot emit such an event
unit upgrade to KCL v2
* Introduce `EnvironmentHostNameResolver` to resolve Docker host name
from the environment variable
* Make Local Stack tests conditional based on the ENV variable mentioned
above
* Disable `KinesisIntegrationTests` because of `502 Bad Gateway` for
Kinesis service in Local Stack
* Disable
`DynamoDbLockRegistryLeaderInitiatorTests.testDistributedLeaderElection()` -
looks like two instances cannot interaction with table in Local Stack
concurrently properly
* Fix `DynamoDbLockRegistry` to catch and ignore `ResourceInUseException`
instead of re-throwing it without a reason
* Remove `junit-vintage-engine` - no JUnit 4 tests any more
Fixes https://github.com/spring-projects/spring-integration-aws/issues/110
Convert a `PutRecordsRequest` into a `Flux`, map its items into a
`this.kinesisProducer.addUserRecord()` and represent a result
`ListenableFuture` as a `Mono` for deferred execution.
Collect all the monos into a `List<PutRecordsResultEntry>` to populate
into a `putRecordsResult`.
Consume subscription into such a reactive stream via `SettableFuture`
for reply from the `KplMessageHandler`
Any failure in any `ListenableFuture` for `UserRecord`s results in the
failure for the whole `Future` for this `PutRecordsRequest`
The `KinesisMessageDrivenChannelAdapter` may exit its `active` state,
but `consumerInvokers` won't be cleared in the main consumer loop
**Cherry-pick to master**
Fixes https://github.com/spring-projects/spring-integration-aws/issues/109
The plain `Converter` is not able to deal with `MessageHeaders` as a
context for target payload conversion.
* Use `MessageConverter` abstraction to get a request message context
for target conversion
* Deprecate a `setConverter()` in favor of a new `setMessageConverter()`
* Use delegating `ConvertingFromMessageConverter` for default serialization
behavior and for backward compatibility
* Add `flushDuration` option into `KplMessageHandler` for calling a
`kinesisProducer.flush()` periodically
* Call `kinesisProducer.flush()` immediately after `addUserRecord()`
`if (this.flushDuration.toMillis() <= 0) {`
Fixes https://github.com/spring-projects/spring-integration-aws/issues/153
Right now we ignore an `embeddedHeadersMapper` result when we also have a `converter`.
* Move `converter` logic for `batch` mode into `else` of the
`if (KclMessageDrivenChannelAdapter.this.embeddedHeadersMapper != null) {`.
This way we cover all the possible use-case, and don't override results
of each other and also don't perform extra logic for nothing
Fixes https://github.com/spring-projects/spring-integration-aws/issues/152
`map()` into `build()`` the result of a stream for the
`embeddedHeadersMapper` in the `processMultipleRecords()`.
Otherwise our `payload` is going to have `AbstractIntegrationMessageBuilder` instances.
The idea is to allow to process a batch of Messages.
