Spring/spring-integration
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

INT-4067: Cover empty file case in FileSplitter

Browse Source 
JIRA: https://jira.spring.io/browse/INT-4067

When `FileSplitter` is configured with `markers = true` and file is empty, an `iterator` for file throws `IOException: Stream closed`,
because we close the `buffer` just after the first `readLine()` attempt, but still return `true` from the first `hasNext()` call
where the `this.sof` and `this.eof` are `true` for markers.

Add logic to mark internal splitter `iterator` as `done` where we don't have content and still in `sof` state.
This commit is contained in:
Artem Bilan
2016-07-07 13:05:23 -04:00
committed by Gary Russell
parent 544bb01027
commit 37a566efff
3 changed files with 33 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
spring-integration-file/src/main/java/org/springframework/integration/file/splitter/FileSplitter.java
View File

@@ -216,6 +216,9 @@ public class FileSplitter extends AbstractMessageSplitter {
if (!ready) {
if (this.markers) {
this.eof = true;
if (this.sof) {
this.done = true;
}
}
bufferedReader.close();
}

28
spring-integration-file/src/test/java/org/springframework/integration/file/splitter/FileSplitterTests.java
View File

@@ -207,6 +207,34 @@ public class FileSplitterTests {
assertEquals(2, fileMarker.getLineCount());
}
@Test
public void testMarkersEmptyFile() throws IOException {
QueueChannel outputChannel = new QueueChannel();
FileSplitter splitter = new FileSplitter(true, true);
splitter.setOutputChannel(outputChannel);
File file = File.createTempFile("empty", ".txt");
splitter.handleMessage(new GenericMessage<File>(file));
Message<?> received = outputChannel.receive(0);
assertNotNull(received);
assertNull(received.getHeaders().get(IntegrationMessageHeaderAccessor.SEQUENCE_SIZE));
assertEquals("START", received.getHeaders().get(FileHeaders.MARKER));
assertThat(received.getPayload(), instanceOf(FileSplitter.FileMarker.class));
FileMarker fileMarker = (FileSplitter.FileMarker) received.getPayload();
assertEquals(FileMarker.Mark.START, fileMarker.getMark());
assertEquals(file.getAbsolutePath(), fileMarker.getFilePath());
assertEquals(0, fileMarker.getLineCount());
received = outputChannel.receive(0);
assertNotNull(received);
assertEquals("END", received.getHeaders().get(FileHeaders.MARKER));
assertThat(received.getPayload(), instanceOf(FileSplitter.FileMarker.class));
fileMarker = (FileSplitter.FileMarker) received.getPayload();
assertEquals(FileMarker.Mark.END, fileMarker.getMark());
assertEquals(file.getAbsolutePath(), fileMarker.getFilePath());
assertEquals(0, fileMarker.getLineCount());
}
@Test
public void testMarkersJson() throws Exception {
JsonObjectMapper<?, ?> objectMapper = JsonObjectMapperProvider.newInstance();

3
src/reference/asciidoc/file.adoc
View File

@@ -751,7 +751,8 @@ Markers are messages with `FileSplitter.FileMarker` payloads (with `START` and `
Markers might be used when sequentially processing files in a downstream flow where some lines are filtered.
They enable the downstream processing to know when a file has been completely processed.
In addition, a header `file_marker` containing `START` or `END` are added to these messages.
The 'END' marker includes a line count.
The `END` marker includes a line count.
If the file is empty, only `START` and `END` markers are emitted with `0` as the `lineCount`.
Default: `false`.
When `true`, `apply-sequence` is `false` by default.