From e1e89915ce41ee89b4e28f2df2a5b30940d8950d Mon Sep 17 00:00:00 2001 From: Artem Bilan Date: Fri, 16 Oct 2015 12:47:29 -0400 Subject: [PATCH] INT-3855: Docs: File adapters - Java & DSL Config JIRA: https://jira.spring.io/browse/INT-3855 PR Comments --- src/reference/asciidoc/amqp.adoc | 1 + src/reference/asciidoc/file.adoc | 181 +++++++++++++++++++++++++++---- 2 files changed, 159 insertions(+), 23 deletions(-) diff --git a/src/reference/asciidoc/amqp.adoc b/src/reference/asciidoc/amqp.adoc index 72cfe91f6c..165193000d 100644 --- a/src/reference/asciidoc/amqp.adoc +++ b/src/reference/asciidoc/amqp.adoc @@ -687,6 +687,7 @@ public class AmqpJavaApplication { void sendToRabbit(String data); } +} ---- [[amqp-outbound-gateway]] diff --git a/src/reference/asciidoc/file.adoc b/src/reference/asciidoc/file.adoc index 5f009bedbd..ee42976d6b 100644 --- a/src/reference/asciidoc/file.adoc +++ b/src/reference/asciidoc/file.adoc @@ -46,7 +46,7 @@ The `AcceptOnceFileListFilter` ensures files are picked up only once from the di [NOTE] ===== The `AcceptOnceFileListFilter` stores its state in memory. -If you wish the state to survive a system restart, consider using the`FileSystemPersistentAcceptOnceFileListFilter` instead. +If you wish the state to survive a system restart, consider using the `FileSystemPersistentAcceptOnceFileListFilter` instead. This filter stores the accepted file names in a `MetadataStore` implementation (<>). This filter matches on the filename and modified time. @@ -261,14 +261,80 @@ Generally, instead of using an `AcceptOnceFileListFilter` in this case, one woul files so that the previously filtered files will be available on a future poll. ===== +==== Configuring with Java Configuration + +The following Spring Boot application provides an example of configuring the inbound adapter using Java configuration: +[source, java] +---- +@SpringBootApplication +public class FileReadingJavaApplication { + + public static void main(String[] args) { + new SpringApplicationBuilder(FileReadingJavaApplication.class) + .web(false) + .run(args); + } + + @Bean + public MessageChannel fileInputChannel() { + return new DirectChannel(); + } + + @Bean + @InboundChannelAdapter(value = "fileInputChannel", poller = @Poller(fixed-delay = "1000")) + public MessageSource fileReadingMessageSource() { + FileReadingMessageSource source = new FileReadingMessageSource(); + source.setDirectory(new File(INBOUND_PATH)); + source.setFilter(new SimplePatternFileListFilter("*.txt")); + return source; + } + + @Bean + @Transformer(inputChannel = "fileInputChannel", outputChannel = "processFileChannel") + public FileToStringTransformer fileToStringTransformer() { + return new FileToStringTransformer(); + } + +} +---- + +==== Configuring with the Java DSL + +The following Spring Boot application provides an example of configuring the inbound adapter using the Java DSL: + +[source, java] +---- +@SpringBootApplication +public class FileReadingJavaApplication { + + public static void main(String[] args) { + new SpringApplicationBuilder(FileReadingJavaApplication.class) + .web(false) + .run(args); + } + + @Bean + public IntegrationFlow fileReadingFlow() { + return IntegrationFlows + .from(s -> s.file(new File(INBOUND_PATH)) + .patternFilter("*.txt"), + e -> e.poller(Pollers.fixedDelay(1000))) + .transform(Transformers.fileToString()) + .channel("processFileChannel") + .get(); + } + +} +---- + [[file-tailing]] ==== 'Tail'ing Files Another popular use case is to get 'lines' from the end (or tail) of a file, capturing new lines when they are added. Two implementations are provided; the first, `OSDelegatingFileTailingMessageProducer`, uses the native `tail` command (on operating systems that have one). This is likely the most efficient implementation on those platforms. -For operating systems that do not have a `tail` command, the second implementation `ApacheCommonsFileTailingMessageProducer` which uses the Apache `commons-io - Tailer` class. +For operating systems that do not have a `tail` command, the second implementation `ApacheCommonsFileTailingMessageProducer` +which uses the Apache `commons-io` `Tailer` class. In both cases, file system events, such as files being unavailable etc, are published as `ApplicationEvent` s using the normal Spring event publishing mechanism. Examples of such events are: @@ -334,7 +400,7 @@ IMPORTANT: Specifying the `delay`, `end` or `reopen` attributes, forces the use [[file-writing]] === Writing files -To write messages to the file system you can use a http://static.springsource.org/spring-integration/api/org/springframework/integration/file/FileWritingMessageHandler.html[FileWritingMessageHandler]. +To write messages to the file system you can use a http://docs.spring.io/spring-integration/api/org/springframework/integration/file/FileWritingMessageHandler.html[FileWritingMessageHandler]. This class can deal with the following payload types: * _File_, @@ -347,13 +413,13 @@ You can configure the encoding and the charset that will be used in case of a St To make things easier, you can configure the `FileWritingMessageHandler` as part of an _Outbound Channel Adapter_ or _Outbound Gateway_ using the provided XML namespace support. [[file-writing-file-names]] -==== Generating Filenames +==== Generating File Names In its simplest form, the `FileWritingMessageHandler` only requires a destination directory for writing the files. -The name of the file to be written is determined by the handler'shttp://static.springsource.org/spring-integration/api/org/springframework/integration/file/FileNameGenerator.html[FileNameGenerator]. -The http://static.springsource.org/spring-integration/api/org/springframework/integration/file/DefaultFileNameGenerator.html[default implementation] looks for a Message header whose key matches the constant defined as http://static.springsource.org/spring-integration/api/constant-values.html#org.springframework.integration.file.FileHeaders.FILENAME[FileHeaders.FILENAME]. +The name of the file to be written is determined by the handler's http://docs.spring.io/spring-integration/api/org/springframework/integration/file/FileNameGenerator.html[FileNameGenerator]. +The http://docs.spring.io/spring-integration/api/org/springframework/integration/file/DefaultFileNameGenerator.html[default implementation] looks for a Message header whose key matches the constant defined as http://docs.spring.io/spring-integration/api/constant-values.html#org.springframework.integration.file.FileHeaders.FILENAME[FileHeaders.FILENAME]. -Alternatively, you can specify an expression to be evaluated against the Message in order to generate a file name, e.g.:_headers['myCustomHeader'] + '.foo'_. +Alternatively, you can specify an expression to be evaluated against the Message in order to generate a file name, e.g. _headers['myCustomHeader'] + '.foo'_. The expression must evaluate to a `String`. For convenience, the `DefaultFileNameGenerator` also provides the _setHeaderName_ method, allowing you to explicitly specify the Message header whose value shall be used as the filename. @@ -363,7 +429,7 @@ Once setup, the `DefaultFileNameGenerator` will employ the following resolution . Otherwise, if the payload is a `java.io.File`, use the file's filename. . Otherwise, use the Message ID appended with .`msg` as the filename. -When using the XML namespace support, both, the _File Oubound Channel Adapter_ and the _File Outbound Gateway_ support the following two mutually exclusive configuration attributes: +When using the XML namespace support, both, the _File Outbound Channel Adapter_ and the _File Outbound Gateway_ support the following two mutually exclusive configuration attributes: * `filename-generator` (a reference to a `FileNameGenerator`) implementation) * `filename-generator-expression` (an expression evaluating to a `String`) @@ -372,14 +438,14 @@ When using the XML namespace support, both, the _File Oubound Channel Adapter_ a While writing files, a temporary file suffix will be used (default: `.writing`). It is appended to the filename while the file is being written. -To customize the suffix, you can set the _temporary-file-suffix_ attribute on both the _File Oubound Channel Adapter_ and the _File Outbound Gateway_. +To customize the suffix, you can set the _temporary-file-suffix_ attribute on both the _File Outbound Channel Adapter_ and the _File Outbound Gateway_. NOTE: When using the _APPEND_ file _mode_, the _temporary-file-suffix_ attribute is ignored, since the data is appended to the file directly. [[file-writing-output-directory]] ==== Specifying the Output Directory -Both, the _File Oubound Channel Adapter_ and the _File Outbound Gateway_ provide two configuration attributes for specifying the output directory: +Both, the _File Outbound Channel Adapter_ and the _File Outbound Gateway_ provide two configuration attributes for specifying the output directory: * _directory_ * _directory-expression_ @@ -390,8 +456,8 @@ NOTE: The _directory-expression_ attribute is available since Spring Integration *Using the directory attribute* -When using the _directory_ attribute, the output directory will be set to a fixed value, that is set at intialization time of the `FileWritingMessageHandler`. -If you don't specify this attribute, then you must use the_directory-expression_ attribute. +When using the _directory_ attribute, the output directory will be set to a fixed value, that is set at initialization time of the `FileWritingMessageHandler`. +If you don't specify this attribute, then you must use the _directory-expression_ attribute. *Using the directory-expression attribute* @@ -401,7 +467,7 @@ Thus, you have full access to a Message's payload and its headers to dynamically The SpEL expression must resolve to either a `String` or to `java.io.File`. Furthermore the resulting `String` or `File` must point to a directory. -If you don't specify the_directory-expression_ attribute, then you must set the _directory_ attribute. +If you don't specify the _directory-expression_ attribute, then you must set the _directory_ attribute. *Using the auto-create-directory attribute* @@ -502,30 +568,99 @@ However, after writing the file, it will also send it to the reply channel as th ---- As mentioned earlier, you can also specify the _mode_ attribute, which defines the behavior of how to deal with situations where the destination file already exists. -Please see<> for further details. -Generally, when using the_File Outbound Gateway_, the result file is returned as the Message payload on the reply channel. +Please see <> for further details. +Generally, when using the _File Outbound Gateway_, the result file is returned as the Message payload on the reply channel. This also applies when specifying the _IGNORE_ mode. In that case the pre-existing destination file is returned. -If the payload of the request message was a file, you still have access to that original file through the Message Header http://static.springsource.org/spring-integration/api/org/springframework/integration/file/FileHeaders.html[FileHeaders.ORIGINAL_FILE]. +If the payload of the request message was a file, you still have access to that original file through the Message Header http://docs.spring.io/spring-integration/api/org/springframework/integration/file/FileHeaders.html[FileHeaders.ORIGINAL_FILE]. NOTE: The 'outbound-gateway' works well in cases where you want to first move a file and then send it through a processing pipeline. -In such cases, you may connect the file namespace's 'inbound-channel-adapter' element to the 'outbound-gateway' and then connect that gateway's reply-channel to the beginning of the pipeline. +In such cases, you may connect the file namespace's `inbound-channel-adapter` element to the `outbound-gateway` and then connect that gateway's `reply-channel` to the beginning of the pipeline. + +If you have more elaborate requirements or need to support additional payload types as input to be converted to file content you could extend the `FileWritingMessageHandler`, but a much better option is to rely on a `Transformer`. + +==== Configuring with Java Configuration + +The following Spring Boot application provides an example of configuring the inbound adapter using Java configuration: +[source, java] +---- +@SpringBootApplication +@IntegrationComponentScan +public class FileWritingJavaApplication { + + public static void main(String[] args) { + ConfigurableApplicationContext context = + new SpringApplicationBuilder(FileWritingJavaApplication.class) + .web(false) + .run(args); + MyGateway gateway = context.getBean(MyGateway.class); + gateway.writeToFile("foo.txt", new File(tmpDir.getRoot(), "fileWritingFlow"), "foo"); + } + + @Bean + @ServiceActivator(inputChannel = "writeToFileChannel") + public MessageHandler fileWritingMessageHandler() { + Expression directoryExpression = new SpelExpressionParser().parseExpression("headers.directory"); + FileWritingMessageHandler handler = new FileWritingMessageHandler(directoryExpression); + handler.setFileExistsMode(FileExistsMode.APPEND); + return handler; + } + + @MessagingGateway(defaultRequestChannel = "writeToFileChannel") + public interface MyGateway { + + void writeToFile(@Header(FileHeaders.FILENAME) String fileName, + @Header(FileHeaders.FILENAME) File directory, String data); + + } +} +---- + +==== Configuring with the Java DSL + +The following Spring Boot application provides an example of configuring the inbound adapter using the Java DSL: + +[source, java] +---- +@SpringBootApplication +public class FileWritingJavaApplication { + + public static void main(String[] args) { + ConfigurableApplicationContext context = + new SpringApplicationBuilder(FileWritingJavaApplication.class) + .web(false) + .run(args); + MessageChannel fileWritingInput = context.getBean("fileWritingInput", MessageChannel.class); + fileWritingInput.send(new GenericMessage<>("foo")); + } + + @Bean + public IntegrationFlow fileWritingFlow() { + return IntegrationFlows.from("fileWritingInput") + .enrichHeaders(h -> h.header(FileHeaders.FILENAME, "foo.txt") + .header("directory", new File(tmpDir.getRoot(), "fileWritingFlow"))) + .handleWithAdapter(a -> a.fileGateway(m -> m.getHeaders().get("directory"))) + .channel(MessageChannels.queue("fileWritingResultChannel")) + .get(); + } + +} +---- -If you have more elaborate requirements or need to support additional payload types as input to be converted to file content you could extend the FileWritingMessageHandler, but a much better option is to rely on a `Transformer`. [[file-transforming]] === File Transformers To transform data read from the file system to objects and the other way around you need to do some work. Contrary to `FileReadingMessageSource` and to a lesser extent `FileWritingMessageHandler`, it is very likely that you will need your own mechanism to get the job done. -For this you can implement the`Transformer` interface. +For this you can implement the `Transformer` interface. Or extend the `AbstractFilePayloadTransformer` for inbound messages. Some obvious implementations have been provided. -`FileToByteArrayTransformer` transforms Files into byte[]s using Spring's `FileCopyUtils`. +`FileToByteArrayTransformer` transforms Files into `byte[]` using Spring's `FileCopyUtils`. It is often better to use a sequence of transformers than to put all transformations in a single class. -In that case the File to byte[] conversion might be a logical first step. +In that case the `File` to `byte[]` conversion might be a logical first step. `FileToStringTransformer` will convert Files to Strings as the name suggests. If nothing else, this can be useful for debugging (consider using with a Wire Tap). @@ -542,7 +677,7 @@ To configure File specific transformers you can use the appropriate elements fro ---- The _delete-files_ option signals to the transformer that it should delete the inbound File after the transformation is complete. -This is in no way a replacement for using the`AcceptOnceFileListFilter` when the FileReadingMessageSource is being used in a multi-threaded environment (e.g. +This is in no way a replacement for using the `AcceptOnceFileListFilter` when the `FileReadingMessageSource` is being used in a multi-threaded environment (e.g. Spring Integration in general).