GH-23, GH-24: Support Queue Arguments

Resolves #23
Resolves #24

- support queue arguments - ttl,expires, max-length, max-length-bytes, max-priority
- also on DLQ (if auto-bound)
- also allow customization of dead letter exchange/routing key
- support DLX/DLQ configuration on the DLQ as well - combined with TTL can be used to route back to primary queue

Polishing and Document DLQ/TTL Retries Technique

Doc Polishing

spring-cloud-stream-binder-rabbit-docs/src/main/asciidoc/overview.adoc

@@ -104,6 +104,18 @@ bindQueue::
   Whether to bind the queue to the destination exchange; set to `false` if you have set up your own infrastructure and have previously created/bound the queue.
 +
 Default: `true`.
+deadLetterQueueName::
+  name of the DLQ
++
+Default: `prefix+destination.dlq`
+deadLetterExchange::
+  a DLX to assign to the queue; if autoBindDlq is true
++
+Default: 'prefix+DLX'
+deadLetterRoutingKey::
+  a dead letter routing key to assign to the queue; if autoBindDlq is true
++
+Default: `destination`
 declareExchange::
   Whether to declare the exchange for the destination.
 +
@@ -113,6 +125,34 @@ delayedExchange::
   The `x-delayed-type` argument is set to the `exchangeType`.
 +
 Default: `false`.
+dlqDeadLetterExchange::
+  if a DLQ is declared, a DLX to assign to that queue
++
+Default: `none`
+dlqDeadLetterRoutingKey::
+  if a DLQ is declared, a dead letter routing key to assign to that queue; default none
++
+Default: `none`
+dlqExpires::
+  how long before an unused dead letter queue is deleted (ms)
++
+Default: `no expiration`
+dlqMaxLength::
+  maximum number of messages in the dead letter queue
++
+Default: `no limit`
+dlqMaxLengthBytes::
+  maximum number of total bytes in the dead letter queue from all messages
++
+Default: `no limit`
+dlqMaxPriority::
+  maximum priority of messages in the dead letter queue (0-255)
++
+Default: `none`
+dlqTtl::
+  default time to live to apply to the dead letter queue when declared (ms)
++
+Default: `no limit`
 durableSubscription::
   Whether subscription should be durable.
 Only effective if `group` is also set.
@@ -127,8 +167,26 @@ exchangeType::
   The exchange type; `direct`, `fanout` or `topic` for non-partitioned destinations; `direct` or `topic` for partitioned destinations.
 +
 Default: `topic`.
+expires::
+  how long before an unused queue is deleted (ms)
++
+Default: `no expiration`
 maxConcurrency::
-  Default: `1`.
+  the maximum number of consumers
++
+Default: `1`.
+maxLength::
+  maximum number of messages in the queue
++
+Default: `no limit`
+maxLengthBytes::
+  maximum number of total bytes in the queue from all messages
++
+Default: `no limit`
+maxPriority::
+  maximum priority of messages in the queue (0-255)
++
+Default:: `none`
 prefetch::
   Prefetch count.
 +
@@ -163,6 +221,10 @@ transacted::
   Whether to use transacted channels.
 +
 Default: `false`.
+ttl::
+default time to live to apply to the queue when declared (ms)
++
+Default: `no limit`
 txSize::
   The number of deliveries between acks.
 +
@@ -198,6 +260,21 @@ compress::
   Whether data should be compressed when sent.
 +
 Default: `false`.
+deadLetterQueueName::
+  name of the DLQ
+  Only applies if `requiredGroups` are provided and then only to those groups.
++
+Default: `prefix+destination.dlq`
+deadLetterExchange::
+  a DLX to assign to the queue; if autoBindDlq is true
+  Only applies if `requiredGroups` are provided and then only to those groups.
++
+Default: 'prefix+DLX'
+deadLetterRoutingKey::
+  a dead letter routing key to assign to the queue; if autoBindDlq is true
+  Only applies if `requiredGroups` are provided and then only to those groups.
++
+Default: `destination`
 declareExchange::
   Whether to declare the exchange for the destination.
 +
@@ -215,16 +292,71 @@ deliveryMode::
   Delivery mode.
 +
 Default: `PERSISTENT`.
+dlqDeadLetterExchange::
+  if a DLQ is declared, a DLX to assign to that queue
+  Only applies if `requiredGroups` are provided and then only to those groups.
++
+Default: `none`
+dlqDeadLetterRoutingKey::
+  if a DLQ is declared, a dead letter routing key to assign to that queue; default none
+  Only applies if `requiredGroups` are provided and then only to those groups.
++
+Default: `none`
+dlqExpires::
+  how long before an unused dead letter queue is deleted (ms)
+  Only applies if `requiredGroups` are provided and then only to those groups.
++
+Default: `no expiration`
+dlqMaxLength::
+  maximum number of messages in the dead letter queue
+  Only applies if `requiredGroups` are provided and then only to those groups.
++
+Default: `no limit`
+dlqMaxLengthBytes::
+  maximum number of total bytes in the dead letter queue from all messages
+  Only applies if `requiredGroups` are provided and then only to those groups.
++
+Default: `no limit`
+dlqMaxPriority::
+  maximum priority of messages in the dead letter queue (0-255)
+  Only applies if `requiredGroups` are provided and then only to those groups.
++
+Default: `none`
+dlqTtl::
+  default time to live to apply to the dead letter queue when declared (ms)
+  Only applies if `requiredGroups` are provided and then only to those groups.
++
+Default: `no limit`
 exchangeRoutingKey::
   The routing key with which to bind the queue to the exchange (if `bindQueue` is `true`).
   Only applies to non-partitioned destinations.
-  Only applies if `requiredGroups` are provided.
+  Only applies if `requiredGroups` are provided and then only to those groups.
 +
 Default: `#`.
 exchangeType::
   The exchange type; `direct`, `fanout` or `topic` for non-partitioned destinations; `direct` or `topic` for partitioned destinations.
 +
 Default: `topic`.
+expires::
+  how long before an unused queue is deleted (ms)
+  Only applies if `requiredGroups` are provided and then only to those groups.
++
+Default: `no expiration`
+maxLength::
+  maximum number of messages in the queue
+  Only applies if `requiredGroups` are provided and then only to those groups.
++
+Default: `no limit`
+maxLengthBytes::
+  maximum number of total bytes in the queue from all messages
+  Only applies if `requiredGroups` are provided and then only to those groups.
++
+Default: `no limit`
+maxPriority::
+  maximum priority of messages in the queue (0-255)
+  Only applies if `requiredGroups` are provided and then only to those groups.
++
+Default:: `none`
 prefix::
   A prefix to be added to the name of the `destination` exchange.
 +
@@ -245,9 +377,79 @@ transacted::
   Whether to use transacted channels.
 +
 Default: `false`.
+ttl::
+  default time to live to apply to the queue when declared (ms)
+  Only applies if `requiredGroups` are provided and then only to those groups.
++
+Default: `no limit`
 
 [NOTE]
 ====
 In the case of RabbitMQ, content type headers can be set by external applications.
 Spring Cloud Stream supports them as part of an extended internal protocol used for any type of transport (including transports, such as Kafka, that do not normally support headers).
 ====
+
+== Retry With the RabbitMQ Binder
+
+=== Overview
+
+When retry is enabled within the binder, the listener container thread is suspended for any back off periods that are configured.
+This might be important when strict ordering is required with a single consumer but for other use cases it prevents other messages from being processed on that thread.
+An alternative to using binder retry is to set up dead lettering with time to live on the dead-letter queue (DLQ), as well as dead-letter configuration on the DLQ itself.
+See <<rabbit-binder-properties>> for more information about the properties discussed here.
+Example configuration to enable this feature:
+
+* Set `autoBindDlq` to `true` - the binder will create a DLQ; you can optionally specify a name in `deadLetterQueueName`
+* Set `dlqTtl` to the back off time you want to wait between redeliveries
+* Set the `dlqDeadLetterExchange` to the default exchange - expired messages from the DLQ will be routed to the original queue since the default `deadLetterRoutingKey` is the queue name (`destination.group`)
+
+To force a message to be dead-lettered, either throw an `AmqpRejectAndDontRequeueException`, or set `requeueRejected` to `true` and throw any exception.
+
+The loop will continue without end, which is fine for transient problems but you may want to give up after some number of attempts.
+Fortunately, RabbitMQ provides the `x-death` header which allows you to determine how many cycles have occurred.
+
+To acknowledge a message after giving up, throw an `ImmediateAcknowledgeAmqpException`.
+
+=== Putting it All Together
+
+[source]
+---
+spring.cloud.stream.bindings.input.destination=myDestination
+spring.cloud.stream.bindings.input.group=consumerGroup
+#disable binder retries
+spring.cloud.stream.bindings.input.consumer.max-attempts=1
+#dlx/dlq setup
+spring.cloud.stream.rabbit.bindings.input.consumer.auto-bind-dlq=true
+spring.cloud.stream.rabbit.bindings.input.consumer.dlq-ttl=5000
+spring.cloud.stream.rabbit.bindings.input.consumer.dlq-dead-letter-exchange=
+---
+
+This configuration creates an exchange `myDestination` with queue `myDestination.consumerGroup` bound to a topic exchange with a wildcard routing key `#`.
+It creates a DLQ bound to a direct exchange `DLX` with routing key `myDestination.consumerGroup`.
+When messages are rejected, they are routed to the DLQ.
+After 5 seconds, the message expires and is routed to the original queue using the queue name as the routing key.
+
+.Spring Boot application
+[source, java]
+----
+@SpringBootApplication
+@EnableBinding(Sink.class)
+public class XDeathApplication {
+
+    public static void main(String[] args) {
+        SpringApplication.run(XDeathApplication.class, args);
+    }
+
+    @StreamListener(Sink.INPUT)
+    public void listen(String in, @Header(name = "x-death", required = false) Map<?,?> death) {
+        if (death != null && death.get("count").equals(3L)) {
+            // giving up - don't send to DLX
+            throw new ImmediateAcknowledgeAmqpException("Failed after 4 attempts");
+        }
+        throw new AmqpRejectAndDontRequeueException("failed");
+    }
+
+}
+----
+
+Notice that the count property in the `x-death` header is a `Long`.