Spring/spring-integration
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

INT-3045: Add in & out ZeroMq channel adapters (#3388)

Browse Source 
* INT-3045: Add in & out ZeroMq channel adapters

JIRA: https://jira.spring.io/browse/INT-3045

* Add `ZeroMqMessageHandler` to produce messages into one-way ZeroMq sockets
* Add `ZeroMqMessageProducer` to consumer messages from one-way ZeroMq sockets
* Add `ConvertingBytesMessageMapper` impl for the `BytesMessageMapper` to
delegate an actual conversion into the provided `MessageConverter`
* Add `ZeroMqHeaders` for message headers constants representing ZeroMq message attributes
* Fix `ZeroMqChannel` for the proper deferred `zeroMqProxy` evaluation
* Add more JavaDocs
* Fix `ZeroMqChannelTests.testPubSubBind()` to be sure that really all the
subscribed channels get the same message from the `PUB` socket

* * Fix typo in the `ConvertingBytesMessageMapper`
* Add `this` for `doOnError()` in the `ZeroMqChannel` & `ZeroMqMessageProducer`
* Change the bind logic in the `ZeroMqMessageProducer` to `port` and let it to
bind to random port.
The actual port is available later via `getBoundPort()`
* Introduce a `ZeroMqMessageProducer.receiveRaw()` to let received `ZMsg` to
be produce as a `payload`
* Add a logic into `ZeroMqMessageHandler` to treat `ZMsg` in the payload of
request message as is without any conversion
* Fix race condition in the `ZeroMqMessageProducer` to destroy `consumerScheduler`
when the main `Flux` is complete

* * Add Java DSL for ZeroMq components
* Extract `ReactiveMessageHandlerSpec` for `ReactiveMessageHandler` impls
* Add debug message into `EmbeddedJsonHeadersMessageMapper` when cannot `decodeNativeFormat()`
* Make `ReactiveMongoDbMessageHandlerSpec` extending `ReactiveMessageHandlerSpec`
* Make `ZeroMqProxy` `autoStartup` by default
* Add `ZeroMqDslTests` to cover all the Java DSL for ZeroMq
* Introduce a `MimeTypeSerializer`  to serialize a `MimeType` into JSON as a plain string;
use it as extra serializer in the `JacksonJsonUtils.messagingAwareMapper()`
* Fix typo for the `AllowListTypeResolverBuilder` inner class

* * Add some docs
* Fix Checkstyle violations

* * More docs

* Fix language in Docs

Co-authored-by: Gary Russell <grussell@vmware.com>

Co-authored-by: Gary Russell <grussell@vmware.com>
This commit is contained in:
Artem Bilan
2020-09-25 13:46:01 -04:00
committed by GitHub
parent 9d34cfd4dd
commit a62a7d1ddd
24 changed files with 1864 additions and 53 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
src/reference/asciidoc/whats-new.adoc
View File

@@ -44,7 +44,7 @@ See <<./jdbc.adoc#jdbc-lock-registry,JDBC implementation>> for more information.
[[x5.4-zeromq]]
==== ZeroMQ Support
A `ZeroMqChannel` has been introduced.
`ZeroMqChannel`, `ZeroMqMessageHandler` and `ZeroMqMessageProducer` have been introduced.
See <<./zeromq.adoc#zeromq,ZeroMQ Support>> for more information.
[[x5.4-general]]

111
src/reference/asciidoc/zeromq.adoc
View File

@@ -102,3 +102,114 @@ ZeroMqChannel zeroMqPubSubChannel(ZContext context) {
}
----
====
[[zeromq-inbound-channel-adapter]]
=== ZeroMq Inbound Channel Adapter
The `ZeroMqMessageProducer` is a `MessageProducerSupport` implementation with reactive semantics.
It constantly reads the data from a ZeroMq socket in a non-blocking manner and publishes the messages to an infinite `Flux` which is subscribed to by a `FluxMessageChannel` or explicitly in the `start()` method, if the output channel is not reactive.
When no data are received on the socket, a `consumeDelay` (defaults to 1 second) is applied before the next read attempt.
Only `SocketType.PAIR`, `SocketType.PULL` and `SocketType.SUB` are supported by the `ZeroMqMessageProducer`.
This component can connect to the remote socket or bind onto TCP protocol with the provided or random port.
The actual port can be obtained via `getBoundPort()` after this component is started and ZeroMq socket is bound.
The socket options (e.g. security or write timeout) can be configured via `setSocketConfigurer(Consumer<ZMQ.Socket> socketConfigurer)` callback.
If the `receiveRaw` option is set to `true`, a `ZMsg`, consumed from the socket, is sent as is in the payload of the produced `Message`: it's up to the downstream flow to parse and convert the `ZMsg`.
Otherwise an `InboundMessageMapper` is used to convert the consumed data into a `Message`.
If the received `ZMsg` is multi-frame, the first frame is treated as the `ZeroMqHeaders.TOPIC` header this ZeroMq message was published to.
With `SocketType.SUB`, the `ZeroMqMessageProducer` uses the provided `topics` option for subscriptions; defaults to subscribe to all.
Subscriptions can be adjusted at runtime using `subscribeToTopics()` and `unsubscribeFromTopics()` `@ManagedOperation` s.
Here is a sample of `ZeroMqMessageProducer` configuration:
====
[source,java]
----
@Bean
ZeroMqMessageProducer zeroMqMessageProducer(ZContext context, MessageChannel outputChannel) {
ZeroMqMessageProducer messageProducer = new ZeroMqMessageProducer(context, SocketType.SUB);
messageProducer.setOutputChannel(outputChannel);
messageProducer.setTopics("some");
messageProducer.setReceiveRaw(true);
messageProducer.setBindPort(7070);
messageProducer.setConsumeDelay(Duration.ofMillis(100));
return messageProducer;
}
----
====
[[zeromq-outbound-channel-adapter]]
=== ZeroMq Outbound Channel Adapter
The `ZeroMqMessageHandler` is a `ReactiveMessageHandler` implementation to produce publish messages into a ZeroMq socket.
Only `SocketType.PAIR`, `SocketType.PUSH` and `SocketType.PUB` are supported.
The `ZeroMqMessageHandler` only supports connecting the ZeroMq socket; binding is not supported.
When the `SocketType.PUB` is used, the `topicExpression` is evaluated against a request message to inject a topic frame into a ZeroMq message if it is not null.
The subscriber side (`SocketType.SUB`) must receive the topic frame first before parsing the actual data.
When the payload of the request message is a `ZMsg`, no conversion or topic extraction is performed: the `ZMsg` is sent into a socket as is and it is not destroyed for possible further reuse.
Otherwise an `OutboundMessageMapper<byte[]>` is used to convert a request message (or just its payload) into a ZeroMq frame to publish.
By default a `ConvertingBytesMessageMapper` is used supplied with a `ConfigurableCompositeMessageConverter`.
The socket options (e.g. security or write timeout) can be configured via `setSocketConfigurer(Consumer<ZMQ.Socket> socketConfigurer)` callback.
Here is a sample of `ZeroMqMessageHandler` configuration:
====
[source,java]
----
@Bean
@ServiceActivator(inputChannel = "zeroMqPublisherChannel")
ZeroMqMessageHandler zeroMqMessageHandler(ZContext context) {
ZeroMqMessageHandler messageHandler =
new ZeroMqMessageHandler(context, "tcp://localhost:6060", SocketType.PUB);
messageHandler.setTopicExpression(
new FunctionExpression<Message<?>>((message) -> message.getHeaders().get("topic")));
messageHandler.setMessageMapper(new EmbeddedJsonHeadersMessageMapper());
}
----
====
[[zeromq-dsl]]
=== ZeroMq Java DSL Support
The `spring-integration-zeromq` provide a convenient Java DSL fluent API via `ZeroMq` factory and `IntegrationComponentSpec` implementations for the components mentioned above.
This is a sample of Java DSL for `ZeroMqChannel`:
====
[source,java]
----
.channel(ZeroMq.zeroMqChannel(this.context)
.connectUrl("tcp://localhost:6001:6002")
.consumeDelay(Duration.ofMillis(100)))
}
----
====
The Inbound Channel Adapter for ZeroMq Java DSL is:
====
[source,java]
----
IntegrationFlows.from(
ZeroMq.inboundChannelAdapter(this.context, SocketType.SUB)
.connectUrl("tcp://localhost:9000")
.topics("someTopic")
.receiveRaw(true)
.consumeDelay(Duration.ofMillis(100)))
}
----
====
The Outbound Channel Adapter for ZeroMq Java DSL is:
====
[source,java]
----
.handle(ZeroMq.outboundChannelAdapter(this.context, "tcp://localhost:9001", SocketType.PUB)
.topicFunction(message -> message.getHeaders().get("myTopic")))
}
----
====