INT-3855: Docs: File adapters - Java & DSL Config

JIRA: https://jira.spring.io/browse/INT-3855

PR Comments
This commit is contained in:
Artem Bilan
2015-10-16 12:47:29 -04:00
committed by Gary Russell
parent 2fb039004d
commit e1e89915ce
2 changed files with 159 additions and 23 deletions

View File

@@ -687,6 +687,7 @@ public class AmqpJavaApplication {
void sendToRabbit(String data); void sendToRabbit(String data);
} }
}
---- ----
[[amqp-outbound-gateway]] [[amqp-outbound-gateway]]

View File

@@ -46,7 +46,7 @@ The `AcceptOnceFileListFilter` ensures files are picked up only once from the di
[NOTE] [NOTE]
===== =====
The `AcceptOnceFileListFilter` stores its state in memory. The `AcceptOnceFileListFilter` stores its state in memory.
If you wish the state to survive a system restart, consider using the`FileSystemPersistentAcceptOnceFileListFilter` instead. If you wish the state to survive a system restart, consider using the `FileSystemPersistentAcceptOnceFileListFilter` instead.
This filter stores the accepted file names in a `MetadataStore` implementation (<<metadata-store>>). This filter stores the accepted file names in a `MetadataStore` implementation (<<metadata-store>>).
This filter matches on the filename and modified time. This filter matches on the filename and modified time.
@@ -261,14 +261,80 @@ Generally, instead of using an `AcceptOnceFileListFilter` in this case, one woul
files so that the previously filtered files will be available on a future poll. files so that the previously filtered files will be available on a future poll.
===== =====
==== Configuring with Java Configuration
The following Spring Boot application provides an example of configuring the inbound adapter using Java configuration:
[source, java]
----
@SpringBootApplication
public class FileReadingJavaApplication {
public static void main(String[] args) {
new SpringApplicationBuilder(FileReadingJavaApplication.class)
.web(false)
.run(args);
}
@Bean
public MessageChannel fileInputChannel() {
return new DirectChannel();
}
@Bean
@InboundChannelAdapter(value = "fileInputChannel", poller = @Poller(fixed-delay = "1000"))
public MessageSource<File> fileReadingMessageSource() {
FileReadingMessageSource source = new FileReadingMessageSource();
source.setDirectory(new File(INBOUND_PATH));
source.setFilter(new SimplePatternFileListFilter("*.txt"));
return source;
}
@Bean
@Transformer(inputChannel = "fileInputChannel", outputChannel = "processFileChannel")
public FileToStringTransformer fileToStringTransformer() {
return new FileToStringTransformer();
}
}
----
==== Configuring with the Java DSL
The following Spring Boot application provides an example of configuring the inbound adapter using the Java DSL:
[source, java]
----
@SpringBootApplication
public class FileReadingJavaApplication {
public static void main(String[] args) {
new SpringApplicationBuilder(FileReadingJavaApplication.class)
.web(false)
.run(args);
}
@Bean
public IntegrationFlow fileReadingFlow() {
return IntegrationFlows
.from(s -> s.file(new File(INBOUND_PATH))
.patternFilter("*.txt"),
e -> e.poller(Pollers.fixedDelay(1000)))
.transform(Transformers.fileToString())
.channel("processFileChannel")
.get();
}
}
----
[[file-tailing]] [[file-tailing]]
==== 'Tail'ing Files ==== 'Tail'ing Files
Another popular use case is to get 'lines' from the end (or tail) of a file, capturing new lines when they are added. Another popular use case is to get 'lines' from the end (or tail) of a file, capturing new lines when they are added.
Two implementations are provided; the first, `OSDelegatingFileTailingMessageProducer`, uses the native `tail` command (on operating systems that have one). Two implementations are provided; the first, `OSDelegatingFileTailingMessageProducer`, uses the native `tail` command (on operating systems that have one).
This is likely the most efficient implementation on those platforms. This is likely the most efficient implementation on those platforms.
For operating systems that do not have a `tail` command, the second implementation `ApacheCommonsFileTailingMessageProducer` which uses the Apache `commons-io For operating systems that do not have a `tail` command, the second implementation `ApacheCommonsFileTailingMessageProducer`
Tailer` class. which uses the Apache `commons-io` `Tailer` class.
In both cases, file system events, such as files being unavailable etc, are published as `ApplicationEvent` s using the normal Spring event publishing mechanism. In both cases, file system events, such as files being unavailable etc, are published as `ApplicationEvent` s using the normal Spring event publishing mechanism.
Examples of such events are: Examples of such events are:
@@ -334,7 +400,7 @@ IMPORTANT: Specifying the `delay`, `end` or `reopen` attributes, forces the use
[[file-writing]] [[file-writing]]
=== Writing files === Writing files
To write messages to the file system you can use a http://static.springsource.org/spring-integration/api/org/springframework/integration/file/FileWritingMessageHandler.html[FileWritingMessageHandler]. To write messages to the file system you can use a http://docs.spring.io/spring-integration/api/org/springframework/integration/file/FileWritingMessageHandler.html[FileWritingMessageHandler].
This class can deal with the following payload types: This class can deal with the following payload types:
* _File_, * _File_,
@@ -347,13 +413,13 @@ You can configure the encoding and the charset that will be used in case of a St
To make things easier, you can configure the `FileWritingMessageHandler` as part of an _Outbound Channel Adapter_ or _Outbound Gateway_ using the provided XML namespace support. To make things easier, you can configure the `FileWritingMessageHandler` as part of an _Outbound Channel Adapter_ or _Outbound Gateway_ using the provided XML namespace support.
[[file-writing-file-names]] [[file-writing-file-names]]
==== Generating Filenames ==== Generating File Names
In its simplest form, the `FileWritingMessageHandler` only requires a destination directory for writing the files. In its simplest form, the `FileWritingMessageHandler` only requires a destination directory for writing the files.
The name of the file to be written is determined by the handler'shttp://static.springsource.org/spring-integration/api/org/springframework/integration/file/FileNameGenerator.html[FileNameGenerator]. The name of the file to be written is determined by the handler's http://docs.spring.io/spring-integration/api/org/springframework/integration/file/FileNameGenerator.html[FileNameGenerator].
The http://static.springsource.org/spring-integration/api/org/springframework/integration/file/DefaultFileNameGenerator.html[default implementation] looks for a Message header whose key matches the constant defined as http://static.springsource.org/spring-integration/api/constant-values.html#org.springframework.integration.file.FileHeaders.FILENAME[FileHeaders.FILENAME]. The http://docs.spring.io/spring-integration/api/org/springframework/integration/file/DefaultFileNameGenerator.html[default implementation] looks for a Message header whose key matches the constant defined as http://docs.spring.io/spring-integration/api/constant-values.html#org.springframework.integration.file.FileHeaders.FILENAME[FileHeaders.FILENAME].
Alternatively, you can specify an expression to be evaluated against the Message in order to generate a file name, e.g.:_headers['myCustomHeader'] + '.foo'_. Alternatively, you can specify an expression to be evaluated against the Message in order to generate a file name, e.g. _headers['myCustomHeader'] + '.foo'_.
The expression must evaluate to a `String`. The expression must evaluate to a `String`.
For convenience, the `DefaultFileNameGenerator` also provides the _setHeaderName_ method, allowing you to explicitly specify the Message header whose value shall be used as the filename. For convenience, the `DefaultFileNameGenerator` also provides the _setHeaderName_ method, allowing you to explicitly specify the Message header whose value shall be used as the filename.
@@ -363,7 +429,7 @@ Once setup, the `DefaultFileNameGenerator` will employ the following resolution
. Otherwise, if the payload is a `java.io.File`, use the file's filename. . Otherwise, if the payload is a `java.io.File`, use the file's filename.
. Otherwise, use the Message ID appended with .`msg` as the filename. . Otherwise, use the Message ID appended with .`msg` as the filename.
When using the XML namespace support, both, the _File Oubound Channel Adapter_ and the _File Outbound Gateway_ support the following two mutually exclusive configuration attributes: When using the XML namespace support, both, the _File Outbound Channel Adapter_ and the _File Outbound Gateway_ support the following two mutually exclusive configuration attributes:
* `filename-generator` (a reference to a `FileNameGenerator`) implementation) * `filename-generator` (a reference to a `FileNameGenerator`) implementation)
* `filename-generator-expression` (an expression evaluating to a `String`) * `filename-generator-expression` (an expression evaluating to a `String`)
@@ -372,14 +438,14 @@ When using the XML namespace support, both, the _File Oubound Channel Adapter_ a
While writing files, a temporary file suffix will be used (default: `.writing`). While writing files, a temporary file suffix will be used (default: `.writing`).
It is appended to the filename while the file is being written. It is appended to the filename while the file is being written.
To customize the suffix, you can set the _temporary-file-suffix_ attribute on both the _File Oubound Channel Adapter_ and the _File Outbound Gateway_. To customize the suffix, you can set the _temporary-file-suffix_ attribute on both the _File Outbound Channel Adapter_ and the _File Outbound Gateway_.
NOTE: When using the _APPEND_ file _mode_, the _temporary-file-suffix_ attribute is ignored, since the data is appended to the file directly. NOTE: When using the _APPEND_ file _mode_, the _temporary-file-suffix_ attribute is ignored, since the data is appended to the file directly.
[[file-writing-output-directory]] [[file-writing-output-directory]]
==== Specifying the Output Directory ==== Specifying the Output Directory
Both, the _File Oubound Channel Adapter_ and the _File Outbound Gateway_ provide two configuration attributes for specifying the output directory: Both, the _File Outbound Channel Adapter_ and the _File Outbound Gateway_ provide two configuration attributes for specifying the output directory:
* _directory_ * _directory_
* _directory-expression_ * _directory-expression_
@@ -390,8 +456,8 @@ NOTE: The _directory-expression_ attribute is available since Spring Integration
*Using the directory attribute* *Using the directory attribute*
When using the _directory_ attribute, the output directory will be set to a fixed value, that is set at intialization time of the `FileWritingMessageHandler`. When using the _directory_ attribute, the output directory will be set to a fixed value, that is set at initialization time of the `FileWritingMessageHandler`.
If you don't specify this attribute, then you must use the_directory-expression_ attribute. If you don't specify this attribute, then you must use the _directory-expression_ attribute.
*Using the directory-expression attribute* *Using the directory-expression attribute*
@@ -401,7 +467,7 @@ Thus, you have full access to a Message's payload and its headers to dynamically
The SpEL expression must resolve to either a `String` or to `java.io.File`. The SpEL expression must resolve to either a `String` or to `java.io.File`.
Furthermore the resulting `String` or `File` must point to a directory. Furthermore the resulting `String` or `File` must point to a directory.
If you don't specify the_directory-expression_ attribute, then you must set the _directory_ attribute. If you don't specify the _directory-expression_ attribute, then you must set the _directory_ attribute.
*Using the auto-create-directory attribute* *Using the auto-create-directory attribute*
@@ -502,30 +568,99 @@ However, after writing the file, it will also send it to the reply channel as th
---- ----
As mentioned earlier, you can also specify the _mode_ attribute, which defines the behavior of how to deal with situations where the destination file already exists. As mentioned earlier, you can also specify the _mode_ attribute, which defines the behavior of how to deal with situations where the destination file already exists.
Please see<<file-writing-destination-exists>> for further details. Please see <<file-writing-destination-exists>> for further details.
Generally, when using the_File Outbound Gateway_, the result file is returned as the Message payload on the reply channel. Generally, when using the _File Outbound Gateway_, the result file is returned as the Message payload on the reply channel.
This also applies when specifying the _IGNORE_ mode. This also applies when specifying the _IGNORE_ mode.
In that case the pre-existing destination file is returned. In that case the pre-existing destination file is returned.
If the payload of the request message was a file, you still have access to that original file through the Message Header http://static.springsource.org/spring-integration/api/org/springframework/integration/file/FileHeaders.html[FileHeaders.ORIGINAL_FILE]. If the payload of the request message was a file, you still have access to that original file through the Message Header http://docs.spring.io/spring-integration/api/org/springframework/integration/file/FileHeaders.html[FileHeaders.ORIGINAL_FILE].
NOTE: The 'outbound-gateway' works well in cases where you want to first move a file and then send it through a processing pipeline. NOTE: The 'outbound-gateway' works well in cases where you want to first move a file and then send it through a processing pipeline.
In such cases, you may connect the file namespace's 'inbound-channel-adapter' element to the 'outbound-gateway' and then connect that gateway's reply-channel to the beginning of the pipeline. In such cases, you may connect the file namespace's `inbound-channel-adapter` element to the `outbound-gateway` and then connect that gateway's `reply-channel` to the beginning of the pipeline.
If you have more elaborate requirements or need to support additional payload types as input to be converted to file content you could extend the `FileWritingMessageHandler`, but a much better option is to rely on a `Transformer`.
==== Configuring with Java Configuration
The following Spring Boot application provides an example of configuring the inbound adapter using Java configuration:
[source, java]
----
@SpringBootApplication
@IntegrationComponentScan
public class FileWritingJavaApplication {
public static void main(String[] args) {
ConfigurableApplicationContext context =
new SpringApplicationBuilder(FileWritingJavaApplication.class)
.web(false)
.run(args);
MyGateway gateway = context.getBean(MyGateway.class);
gateway.writeToFile("foo.txt", new File(tmpDir.getRoot(), "fileWritingFlow"), "foo");
}
@Bean
@ServiceActivator(inputChannel = "writeToFileChannel")
public MessageHandler fileWritingMessageHandler() {
Expression directoryExpression = new SpelExpressionParser().parseExpression("headers.directory");
FileWritingMessageHandler handler = new FileWritingMessageHandler(directoryExpression);
handler.setFileExistsMode(FileExistsMode.APPEND);
return handler;
}
@MessagingGateway(defaultRequestChannel = "writeToFileChannel")
public interface MyGateway {
void writeToFile(@Header(FileHeaders.FILENAME) String fileName,
@Header(FileHeaders.FILENAME) File directory, String data);
}
}
----
==== Configuring with the Java DSL
The following Spring Boot application provides an example of configuring the inbound adapter using the Java DSL:
[source, java]
----
@SpringBootApplication
public class FileWritingJavaApplication {
public static void main(String[] args) {
ConfigurableApplicationContext context =
new SpringApplicationBuilder(FileWritingJavaApplication.class)
.web(false)
.run(args);
MessageChannel fileWritingInput = context.getBean("fileWritingInput", MessageChannel.class);
fileWritingInput.send(new GenericMessage<>("foo"));
}
@Bean
public IntegrationFlow fileWritingFlow() {
return IntegrationFlows.from("fileWritingInput")
.enrichHeaders(h -> h.header(FileHeaders.FILENAME, "foo.txt")
.header("directory", new File(tmpDir.getRoot(), "fileWritingFlow")))
.handleWithAdapter(a -> a.fileGateway(m -> m.getHeaders().get("directory")))
.channel(MessageChannels.queue("fileWritingResultChannel"))
.get();
}
}
----
If you have more elaborate requirements or need to support additional payload types as input to be converted to file content you could extend the FileWritingMessageHandler, but a much better option is to rely on a `Transformer`.
[[file-transforming]] [[file-transforming]]
=== File Transformers === File Transformers
To transform data read from the file system to objects and the other way around you need to do some work. To transform data read from the file system to objects and the other way around you need to do some work.
Contrary to `FileReadingMessageSource` and to a lesser extent `FileWritingMessageHandler`, it is very likely that you will need your own mechanism to get the job done. Contrary to `FileReadingMessageSource` and to a lesser extent `FileWritingMessageHandler`, it is very likely that you will need your own mechanism to get the job done.
For this you can implement the`Transformer` interface. For this you can implement the `Transformer` interface.
Or extend the `AbstractFilePayloadTransformer` for inbound messages. Or extend the `AbstractFilePayloadTransformer` for inbound messages.
Some obvious implementations have been provided. Some obvious implementations have been provided.
`FileToByteArrayTransformer` transforms Files into byte[]s using Spring's `FileCopyUtils`. `FileToByteArrayTransformer` transforms Files into `byte[]` using Spring's `FileCopyUtils`.
It is often better to use a sequence of transformers than to put all transformations in a single class. It is often better to use a sequence of transformers than to put all transformations in a single class.
In that case the File to byte[] conversion might be a logical first step. In that case the `File` to `byte[]` conversion might be a logical first step.
`FileToStringTransformer` will convert Files to Strings as the name suggests. `FileToStringTransformer` will convert Files to Strings as the name suggests.
If nothing else, this can be useful for debugging (consider using with a Wire Tap). If nothing else, this can be useful for debugging (consider using with a Wire Tap).
@@ -542,7 +677,7 @@ To configure File specific transformers you can use the appropriate elements fro
---- ----
The _delete-files_ option signals to the transformer that it should delete the inbound File after the transformation is complete. The _delete-files_ option signals to the transformer that it should delete the inbound File after the transformation is complete.
This is in no way a replacement for using the`AcceptOnceFileListFilter` when the FileReadingMessageSource is being used in a multi-threaded environment (e.g. This is in no way a replacement for using the `AcceptOnceFileListFilter` when the `FileReadingMessageSource` is being used in a multi-threaded environment (e.g.
Spring Integration in general). Spring Integration in general).