GH-3432: Add MQTT v5 channel adapters (#3639)

* GH-3432: Add MQTT v5 channel adapters

Fixes https://github.com/spring-projects/spring-integration/issues/3432

* Add `optional` dependency for `org.eclipse.paho:org.eclipse.paho.mqttv5.client`
* Add `MqttProtocolErrorEvent` and emit it from the `mqttErrorOccurred()` callback of the MQTT v5 client
* Add `MqttHeaderMapper` since MQTT v5 has introduced user properties pair to transfer over the protocol
* Add `Mqttv5PahoMessageHandler` as one more extension of the `AbstractMqttMessageHandler`
* Add more convenient `MqttHeaders` constants for easier headers mapping configuration
* Ensure via `Mqttv5BackToBackTests` that MQTT v5 is supported by the provided components
* Change `pr-build-workflow.yml` to use `eclipse-mosquitto` container for testing all the MQTT interactions
* Change `cyrilix/rabbitmq-mqtt` service to the `rabbitmq:management` since RabbitMQ does not support MQTT v5

* * Handle manual acks
* Add `Mqttv5PahoMessageDrivenChannelAdapter.persistence` property

* * Add documentation
* Add `MosquittoContainerTest` for TestContainers support with Mosquitto image

* Fix language in the docs after review

Co-authored-by: Gary Russell <grussell@vmware.com>

Co-authored-by: Gary Russell <grussell@vmware.com>

src/checkstyle/checkstyle-suppressions.xml

@@ -12,4 +12,5 @@
 	<suppress files="[\\/]test[\\/]" checks="Javadoc*"/>
 	<suppress files="[\\/]test[\\/]" checks="EqualsHashCode"/>
 	<suppress files="[\\/]test[\\/]" checks="Atclause*"/>
+	<suppress files="[\\/]test[\\/]" checks="InterfaceIsType"/>
 </suppressions>

src/reference/asciidoc/mqtt.adoc

@@ -25,6 +25,9 @@ compile "org.springframework.integration:spring-integration-mqtt:{project-versio
 
 The current implementation uses the https://www.eclipse.org/paho/[Eclipse Paho MQTT Client] library.
 
+IMPORTANT: The XML configuration and most of this chapter are about MQTT v3.1 protocol support and respective Paho Client.
+See <<mqtt-v5>> paragraph for respective protocol support.
+
 Configuration of both adapters is achieved using the `DefaultMqttPahoClientFactory`.
 Refer to the Paho documentation for more information about configuration options.
 
@@ -417,3 +420,78 @@ String beanName = source.getBeanName();
 MqttConnectOptions options = source.getConnectionInfo();
 ----
 ====
+
+[[mqtt-v5]]
+=== MQTT v5 Support
+
+Starting with version 5.5.5, the `spring-integration-mqtt` module provides channel adapter implementations for the MQTT v5 protocol.
+The `org.eclipse.paho:org.eclipse.paho.mqttv5.client` is an `optional` dependency, so has to be included explicitly in the target project.
+
+Since the MQTT v5 protocol supports extra arbitrary properties in an MQTT message, the `MqttHeaderMapper` implementation has been introduced to map to/from headers on publish and receive operations.
+By default (via the `*` pattern) it maps all the received `PUBLISH` frame properties (including user properties).
+On the outbound side it maps this subset of headers for `PUBLISH` frame: `contentType`, `mqtt_messageExpiryInterval`, `mqtt_responseTopic`, `mqtt_correlationData`.
+
+The outbound channel adapter for the MQTT v5 protocol is present as an `Mqttv5PahoMessageHandler`.
+It requires a `clientId` and MQTT broker URL or `MqttConnectionOptions` reference.
+It supports a `MqttClientPersistence` option, can be `async` and can emit `MqttIntegrationEvent` objects in that case (see `asyncEvents` option).
+If a request message payload is an `org.eclipse.paho.mqttv5.common.MqttMessage`, it is published as is via the internal `IMqttAsyncClient`.
+If the payload is `byte[]` it is used as is for the target `MqttMessage` payload to publish.
+If the payload is a `String` it is converted to `byte[]` to publish.
+The remaining use-cases are delegated to the provided `MessageConverter` which is a `IntegrationContextUtils.ARGUMENT_RESOLVER_MESSAGE_CONVERTER_BEAN_NAME` `ConfigurableCompositeMessageConverter` bean from the application context.
+Note: the provided `HeaderMapper<MqttProperties>` is not used when the requested message payload is already an `MqttMessage`.
+The following Java DSL configuration sample demonstrates how to use this channel adapter in the integration flow:
+
+====
+[source, java]
+----
+@Bean
+public IntegrationFlow mqttOutFlow() {
+    Mqttv5PahoMessageHandler messageHandler = new Mqttv5PahoMessageHandler(MQTT_URL, "mqttv5SIout");
+    MqttHeaderMapper mqttHeaderMapper = new MqttHeaderMapper();
+    mqttHeaderMapper.setOutboundHeaderNames("some_user_header", MessageHeaders.CONTENT_TYPE);
+    messageHandler.setHeaderMapper(mqttHeaderMapper);
+    messageHandler.setAsync(true);
+    messageHandler.setAsyncEvents(true);
+    messageHandler.setConverter(mqttStringToBytesConverter());
+
+    return f -> f.handle(messageHandler);
+}
+----
+====
+
+IMPORTANT: The `org.springframework.integration.mqtt.support.MqttMessageConverter` cannot be used with the `Mqttv5PahoMessageHandler` since its contract is aimed only for the MQTT v3 protocol.
+
+See more information in the `Mqttv5PahoMessageHandler` javadocs and its superclass.
+
+The inbound channel adapter for the MQTT v5 protocol is present as an `Mqttv5PahoMessageDrivenChannelAdapter`.
+It requires a `clientId` and MQTT broker URL or `MqttConnectionOptions` reference, plus topics to which to subscribe and consume from.
+It supports a `MqttClientPersistence` option, which is in-memory by default.
+The expected `payloadType` (`byte[]` by default) can be configured and it is propagated to the provided `SmartMessageConverter` for conversion from `byte[]` of the received `MqttMessage`.
+If the `manualAck` option is set, then an `IntegrationMessageHeaderAccessor.ACKNOWLEDGMENT_CALLBACK` header is added to the message to produce as an instance of `SimpleAcknowledgment`.
+The `HeaderMapper<MqttProperties>` is used to map `PUBLISH` frame properties (including user properties) into the target message headers.
+Standard `MqttMessage` properties, such as `qos`, `id`, `dup`, `retained`, plus received topic are always mapped to headers.
+See `MqttHeaders` for more information.
+
+The following Java DSL configuration sample demonstrates how to use this channel adapter in the integration flow:
+
+====
+[source, java]
+----
+@Bean
+public IntegrationFlow mqttInFlow() {
+    Mqttv5PahoMessageDrivenChannelAdapter messageProducer =
+        new Mqttv5PahoMessageDrivenChannelAdapter(MQTT_URL, "mqttv5SIin", "siTest");
+    messageProducer.setPayloadType(String.class);
+    messageProducer.setMessageConverter(mqttStringToBytesConverter());
+    messageProducer.setManualAcks(true);
+
+    return IntegrationFlows.from(messageProducer)
+            .channel(c -> c.queue("fromMqttChannel"))
+            .get();
+}
+----
+====
+
+IMPORTANT: The `org.springframework.integration.mqtt.support.MqttMessageConverter` cannot be used with the `Mqttv5PahoMessageDrivenChannelAdapter` since its contract is aimed only for the MQTT v3 protocol.
+
+See more information in the `Mqttv5PahoMessageDrivenChannelAdapter` javadocs and its superclass.

src/reference/asciidoc/whats-new.adoc

@@ -21,6 +21,12 @@ If you are interested in more details, see the Issue Tracker tickets that were r
 A `FileSplitter.FileMaker`-based implementation of `CorrelationStrategy`, `ReleaseStrategy` and `MessageGroupProcessor` as a `FileAggregator` component was introduced.
 See <<./file.adoc#file-aggregator, File Aggregator>> for more information.
 
+[[x5.5-mqtt-v5]]
+==== MQTT v5 Support
+
+The `Mqttv5PahoMessageDrivenChannelAdapter` and `Mqttv5PahoMessageHandler` (including respective `MqttHeaderMapper`) were introduced to support MQTT v5 protocol communication.
+See <<./mqtt.adoc#mqtt-v5, MQTT v5 Support>> for more information.
+
 [[x5.5-general]]
 === General Changes