GH-84: Kinesis Inbound: errors immunity

Fixes spring-projects/spring-integration-aws#84
Fixes spring-cloud/spring-cloud-stream-binder-aws-kinesis#38
Fixes spring-cloud/spring-cloud-stream-binder-aws-kinesis#36

Even if AWS Client has some reconnect and retry mechanism, it can be
exhausted and no connection error is rethrown to the `KinesisMessageDrivenChannelAdapter`
anyway.

On the other hand the error can be thrown from the record processor -
the flow on the `outputChannel`.

* log the exception around AWS Client calls and let background process
to restore/retry
* log the exception around message to send to let the processor to move
to the next record or perform the next task
* null the current `task` in the `ShardConsumer` in the `finally` block
to avoid hanging the thread without ability to moving to some other state
without end-user interaction
* When perform the `batch` checkpoint, check the result and if it is
negative, consider such a situation as processed and skip records from
sending downstream
* Upgrade dependencies
* Use Log4J2 for tests logging
This commit is contained in:
Artem Bilan
2018-03-01 15:22:02 -05:00
parent c57f0ea2e9
commit 54b7220459
7 changed files with 149 additions and 80 deletions

View File

@@ -42,6 +42,8 @@ import org.springframework.messaging.Message;
import org.springframework.messaging.MessageChannel;
import org.springframework.messaging.MessageHandler;
import org.springframework.messaging.PollableChannel;
import org.springframework.messaging.support.ChannelInterceptorAdapter;
import org.springframework.messaging.support.ErrorMessage;
import org.springframework.test.annotation.DirtiesContext;
import org.springframework.test.context.junit4.SpringRunner;
@@ -134,7 +136,19 @@ public class KinesisIntegrationTests {
@Bean
public PollableChannel errorChannel() {
return new QueueChannel();
QueueChannel queueChannel = new QueueChannel();
queueChannel.addInterceptor(new ChannelInterceptorAdapter() {
@Override
public void postSend(Message<?> message, MessageChannel channel, boolean sent) {
super.postSend(message, channel, sent);
if (message instanceof ErrorMessage) {
throw (RuntimeException) ((ErrorMessage) message).getPayload();
}
}
});
return queueChannel;
}
}