Spring/spring-integration
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

GH-9988: Add FileExistsMode expression support

Browse Source 
Fixes: #9988
Issue link: https://github.com/spring-projects/spring-integration/issues/9988

This change allows dynamic determination of `FileExistsMode` using SpEL expressions,
making the component more flexible when handling file existence conflicts.

* Add `fileExistsModeExpression` field and setter methods
* Use `resolveFileExistsMode()` in put and get operations
* Add changes to the docs
* Ignore `temporaryFileName` when `FileExistsMode.APPEND`

Improve runtime behavior by ignoring temporary filename settings when file exists mode is `APPEND`. 
Now, in `FileExistsMode.APPEND` mode, content is always appended directly to the original file regardless of `useTemporaryFileName` setting.

In `RemoteFileTemplate`:
- Remove exception validation when `APPEND` mode is used with temporary filenames
- Modify logic to skip applying `temporaryFileSuffix` in `APPEND` mode

In `AbstractRemoteFileOutboundGateway`:
- Remove logic that disabled temporary filenames when setting `APPEND` mode

* Apply review feedback on `FileExistsMode` expression

- Optimize `EvaluationContext` usage by creating it once in `doInit()`
- Enhance expression evaluation to support String representation of `FileExistsMode`
- Optimize temporary filename handling logic in `RemoteFileTemplate`
- Add warning message for incompatible `APPEND` mode with temporary filenames
- Rename method to `setFileExistsModeExpressionString` for consistency
- Update Java DSL support in `RemoteFileOutboundGatewaySpec`
- Update reference documentation and release notes

* Apply additional review feedback on `FileExistsMode` expression

- Add `Function` variant to `RemoteFileOutboundGatewaySpec`
- Update documentations to use one-sentence-per-line style
- Improve code flow in `resolveFileExistsMode()` method

* Fix additional review feedback on `FileExistsMode` expression

- Use `Object` instead of `String` in `fileExistsModeFunction`
- Fix return method call in `fileExistsModeFunction` (`remoteDirectoryExpression` -> `fileExistsModeExpression`)
- Fix `standardEvaluationContext` initialization in `doInit()`
- Ensure proper `EvaluationContext` usage in other methods

Signed-off-by: Jooyoung Pyoung <pyoungjy@gmail.com>
This commit is contained in:
Jooyoung Pyoung
2025-05-15 23:47:07 +09:00
committed by GitHub
parent 9bdbb6ed1c
commit debd4d45c2
6 changed files with 267 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

45
spring-integration-file/src/main/java/org/springframework/integration/file/dsl/RemoteFileOutboundGatewaySpec.java
View File

@@ -1,5 +1,5 @@
/*
* Copyright 2016-2023 the original author or authors.
* Copyright 2016-2025 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@@ -44,6 +44,7 @@ import org.springframework.messaging.Message;
*
* @author Artem Bilan
* @author Gary Russell
* @author Jooyoung Pyoung
*
* @since 5.0
*/
@@ -358,6 +359,48 @@ public abstract class RemoteFileOutboundGatewaySpec<F, S extends RemoteFileOutbo
return _this();
}
/**
* Specify a SpEL expression to determine the action to take when files already exist.
* Expression evaluation should return a {@link FileExistsMode} or a String representation.
* Used for GET and MGET operations when the file already exists locally,
* or PUT and MPUT when the file exists on the remote system.
* @param fileExistsModeExpression a SpEL expression to evaluate the file exists mode
* @return the Spec.
* @since 6.5
*/
public S fileExistsModeExpression(Expression fileExistsModeExpression) {
this.target.setFileExistsModeExpression(fileExistsModeExpression);
return _this();
}
/**
* Specify a SpEL expression to determine the action to take when files already exist.
* Expression evaluation should return a {@link FileExistsMode} or a String representation.
* Used for GET and MGET operations when the file already exists locally,
* or PUT and MPUT when the file exists on the remote system.
* @param fileExistsModeExpression the String in SpEL syntax.
* @return the Spec.
* @since 6.5
*/
public S fileExistsModeExpression(String fileExistsModeExpression) {
this.target.setFileExistsModeExpressionString(fileExistsModeExpression);
return _this();
}
/**
* Specify a {@link Function} to determine the action to take when files already exist.
* Expression evaluation should return a {@link FileExistsMode} or a String representation.
* Used for GET and MGET operations when the file already exists locally,
* or PUT and MPUT when the file exists on the remote system.
* @param fileExistsModeFunction the {@link Function} to use.
* @param <P> the expected payload type.
* @return the Spec.
* @since 6.5
*/
public <P> S fileExistsModeFunction(Function<Message<P>, Object> fileExistsModeFunction) {
return fileExistsModeExpression(new FunctionExpression<>(fileExistsModeFunction));
}
/**
* Determine whether the remote directory should automatically be created when
* sending files to the remote system.

10
spring-integration-file/src/main/java/org/springframework/integration/file/remote/RemoteFileTemplate.java
View File

@@ -1,5 +1,5 @@
/*
* Copyright 2013-2024 the original author or authors.
* Copyright 2013-2025 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@@ -63,6 +63,7 @@ import org.springframework.util.StringUtils;
* @author Gary Russell
* @author Artem Bilan
* @author Alen Turkovic
* @author Jooyoung Pyoung
*
* @since 3.0
*
@@ -303,8 +304,6 @@ public class RemoteFileTemplate<F> implements RemoteFileOperations<F>, Initializ
private String send(Message<?> message, String subDirectory, FileExistsMode mode) {
Assert.notNull(this.directoryExpressionProcessor, "'remoteDirectoryExpression' is required");
Assert.isTrue(!FileExistsMode.APPEND.equals(mode) || !this.useTemporaryFileName,
"Cannot append when using a temporary file name");
Assert.isTrue(!FileExistsMode.REPLACE_IF_MODIFIED.equals(mode),
"FilExistsMode.REPLACE_IF_MODIFIED can only be used for local files");
final StreamHolder inputStreamHolder = payloadToInputStream(message);
@@ -565,7 +564,10 @@ public class RemoteFileTemplate<F> implements RemoteFileOperations<F>, Initializ
String tempRemoteFilePath = temporaryRemoteDirectory + fileName;
// write remote file first with temporary file extension if enabled
String tempFilePath = tempRemoteFilePath + (this.useTemporaryFileName ? this.temporaryFileSuffix : "");
String tempFilePath = tempRemoteFilePath;
if (!FileExistsMode.APPEND.equals(mode) && this.useTemporaryFileName) {
tempFilePath += this.temporaryFileSuffix;
}
if (this.autoCreateDirectory) {
try {

74
spring-integration-file/src/main/java/org/springframework/integration/file/remote/gateway/AbstractRemoteFileOutboundGateway.java
View File

@@ -1,5 +1,5 @@
/*
* Copyright 2002-2024 the original author or authors.
* Copyright 2002-2025 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@@ -75,6 +75,7 @@ import org.springframework.util.StringUtils;
* @author Gary Russell
* @author Artem Bilan
* @author Mauro Molinari
* @author Jooyoung Pyoung
*
* @since 2.1
*/
@@ -114,8 +115,12 @@ public abstract class AbstractRemoteFileOutboundGateway<F> extends AbstractReply
private Expression localFilenameGeneratorExpression;
private Expression fileExistsModeExpression;
private FileExistsMode fileExistsMode;
private EvaluationContext standardEvaluationContext;
private Integer chmod;
private boolean remoteFileTemplateExplicitlySet;
@@ -486,6 +491,32 @@ public abstract class AbstractRemoteFileOutboundGateway<F> extends AbstractReply
this.localFilenameGeneratorExpression = EXPRESSION_PARSER.parseExpression(localFilenameGeneratorExpression);
}
/**
* Specify a SpEL expression to determine the action to take when files already exist.
* Expression evaluation should return a {@link FileExistsMode} object.
* Used for GET and MGET operations when the file already exists locally,
* or PUT and MPUT when the file exists on the remote system.
* @param fileExistsModeExpression the expression to use.
* @since 6.5
*/
public void setFileExistsModeExpression(Expression fileExistsModeExpression) {
Assert.notNull(fileExistsModeExpression, "'fileExistsModeExpression' must not be null");
this.fileExistsModeExpression = fileExistsModeExpression;
}
/**
* Specify a SpEL expression to determine the action to take when files already exist.
* Expression evaluation should return a {@link FileExistsMode} object.
* Used for GET and MGET operations when the file already exists locally,
* or PUT and MPUT when the file exists on the remote system.
* @param fileExistsModeExpression the String in SpEL syntax.
* @since 6.5
*/
public void setFileExistsModeExpressionString(String fileExistsModeExpression) {
Assert.hasText(fileExistsModeExpression, "'fileExistsModeExpression' must not be empty");
this.fileExistsModeExpression = EXPRESSION_PARSER.parseExpression(fileExistsModeExpression);
}
/**
* Determine the action to take when using GET and MGET operations when the file
* already exists locally, or PUT and MPUT when the file exists on the remote
@@ -495,9 +526,6 @@ public abstract class AbstractRemoteFileOutboundGateway<F> extends AbstractReply
*/
public void setFileExistsMode(FileExistsMode fileExistsMode) {
this.fileExistsMode = fileExistsMode;
if (FileExistsMode.APPEND.equals(fileExistsMode)) {
this.remoteFileTemplate.setUseTemporaryFileName(false);
}
}
/**
@@ -539,6 +567,7 @@ public abstract class AbstractRemoteFileOutboundGateway<F> extends AbstractReply
Assert.isNull(this.filter, "Filters are not supported with the rm and get commands");
}
this.standardEvaluationContext = ExpressionUtils.createStandardEvaluationContext(getBeanFactory());
if ((Command.GET.equals(this.command) && !this.options.contains(Option.STREAM))
|| Command.MGET.equals(this.command)) {
Assert.notNull(this.localDirectoryExpression, "localDirectory must not be null");
@@ -553,6 +582,11 @@ public abstract class AbstractRemoteFileOutboundGateway<F> extends AbstractReply
Option.RECURSIVE.toString() + " to obtain files in subdirectories");
}
if (FileExistsMode.APPEND.equals(this.fileExistsMode) && this.remoteFileTemplate.isUseTemporaryFileName()) {
logger.warn("FileExistsMode.APPEND is incompatible with useTemporaryFileName=true. " +
"Temporary filename will be ignored for APPEND mode.");
}
populateBeanFactoryIntoComponentsIfAny();
if (!this.remoteFileTemplateExplicitlySet) {
this.remoteFileTemplate.afterPropertiesSet();
@@ -573,7 +607,7 @@ public abstract class AbstractRemoteFileOutboundGateway<F> extends AbstractReply
private void setupLocalDirectory() {
File localDirectory =
ExpressionUtils.expressionToFile(this.localDirectoryExpression,
ExpressionUtils.createStandardEvaluationContext(getBeanFactory()), null,
this.standardEvaluationContext, null,
"localDirectoryExpression");
if (!localDirectory.exists()) {
try {
@@ -845,7 +879,8 @@ public abstract class AbstractRemoteFileOutboundGateway<F> extends AbstractReply
* @since 5.0
*/
protected String put(Message<?> message, Session<F> session, String subDirectory) {
String path = this.remoteFileTemplate.send(message, subDirectory, this.fileExistsMode);
FileExistsMode existsMode = resolveFileExistsMode(message);
String path = this.remoteFileTemplate.send(message, subDirectory, existsMode);
if (path == null) {
throw new MessagingException(message, "No local file found for " + message);
}
@@ -1130,7 +1165,7 @@ public abstract class AbstractRemoteFileOutboundGateway<F> extends AbstractReply
}
final File localFile =
new File(generateLocalDirectory(message, remoteDir), generateLocalFileName(message, remoteFilename));
FileExistsMode existsMode = this.fileExistsMode;
FileExistsMode existsMode = resolveFileExistsMode(message);
boolean appending = FileExistsMode.APPEND.equals(existsMode);
boolean exists = localFile.exists();
boolean replacing = exists && (FileExistsMode.REPLACE.equals(existsMode)
@@ -1351,6 +1386,31 @@ public abstract class AbstractRemoteFileOutboundGateway<F> extends AbstractReply
}
}
private FileExistsMode resolveFileExistsMode(Message<?> message) {
if (this.fileExistsModeExpression != null) {
Object evaluationResult = this.fileExistsModeExpression.getValue(this.standardEvaluationContext, message);
if (evaluationResult instanceof FileExistsMode resolvedMode) {
return resolvedMode;
}
else if (evaluationResult instanceof String modeAsString) {
try {
return FileExistsMode.valueOf(modeAsString.toUpperCase());
}
catch (IllegalArgumentException ex) {
throw new MessagingException(message,
"Invalid FileExistsMode string: '" + modeAsString + "'. Expected one of: " +
Arrays.toString(FileExistsMode.values()), ex);
}
}
else if (evaluationResult != null) {
throw new MessagingException(message,
"Expression returned invalid type for FileExistsMode: " +
evaluationResult.getClass().getName() + ". Expected FileExistsMode or String.");
}
}
return this.fileExistsMode;
}
private File generateLocalDirectory(Message<?> message, String remoteDirectory) {
EvaluationContext evaluationContext = ExpressionUtils.createStandardEvaluationContext(getBeanFactory());
if (remoteDirectory != null) {

127
spring-integration-file/src/test/java/org/springframework/integration/file/remote/gateway/RemoteFileOutboundGatewayTests.java
View File

@@ -1,5 +1,5 @@
/*
* Copyright 2002-2024 the original author or authors.
* Copyright 2002-2025 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@@ -29,6 +29,7 @@ import java.util.Calendar;
import java.util.Collection;
import java.util.Date;
import java.util.List;
import java.util.Map;
import java.util.concurrent.atomic.AtomicReference;
import org.junit.jupiter.api.Test;
@@ -70,6 +71,7 @@ import static org.mockito.Mockito.when;
* @author Gary Russell
* @author Liu Jiong
* @author Artem Bilan
* @author Jooyoung Pyoung
*
* @since 2.1
*/
@@ -640,6 +642,66 @@ public class RemoteFileOutboundGatewayTests {
outFile.delete();
}
@Test
@SuppressWarnings("unchecked")
public void testGetExistsExpression() throws Exception {
SessionFactory sessionFactory = mock(SessionFactory.class);
TestRemoteFileOutboundGateway gw = new TestRemoteFileOutboundGateway(sessionFactory, "get", "payload");
gw.setFileExistsModeExpressionString("headers[\"file.exists.mode\"]");
gw.setLocalDirectory(new File(this.tmpDir));
gw.afterPropertiesSet();
File outFile = new File(this.tmpDir + "/f1");
FileOutputStream fos = new FileOutputStream(outFile);
fos.write("foo".getBytes());
fos.close();
when(sessionFactory.getSession()).thenReturn(new TestSession() {
@Override
public TestLsEntry[] list(String path) {
return new TestLsEntry[] {
new TestLsEntry("f1", 1234, false, false, 12345, "-rw-r--r--")
};
}
@Override
public void read(String source, OutputStream outputStream)
throws IOException {
outputStream.write("testfile".getBytes());
}
});
// default (null)
MessageBuilder<File> out;
assertThatExceptionOfType(MessageHandlingException.class)
.isThrownBy(() -> gw.handleRequestMessage(new GenericMessage<>("f1")))
.withMessageContaining("already exists");
assertThatExceptionOfType(MessageHandlingException.class)
.isThrownBy(() -> gw.handleRequestMessage(
new GenericMessage<>("f1", Map.of("file.exists.mode", FileExistsMode.FAIL))))
.withMessageContaining("already exists");
out = (MessageBuilder<File>) gw.handleRequestMessage(
new GenericMessage<>("f1", Map.of("file.exists.mode", "IGNORE")));
assertThat(out.getPayload()).isEqualTo(outFile);
assertContents("foo", outFile);
out = (MessageBuilder<File>) gw.handleRequestMessage(
new GenericMessage<>("f1", Map.of("file.exists.mode", "append")));
assertThat(out.getPayload()).isEqualTo(outFile);
assertContents("footestfile", outFile);
out = (MessageBuilder<File>) gw.handleRequestMessage(
new GenericMessage<>("f1", Map.of("file.exists.mode", FileExistsMode.REPLACE)));
assertThat(out.getPayload()).isEqualTo(outFile);
assertContents("testfile", outFile);
outFile.delete();
}
private void assertContents(String expected, File outFile) throws Exception {
BufferedReader reader = new BufferedReader(new FileReader(outFile));
assertThat(reader.readLine()).isEqualTo(expected);
@@ -860,6 +922,69 @@ public class RemoteFileOutboundGatewayTests {
verify(session, times(1)).append(any(InputStream.class), anyString());
}
@Test
@SuppressWarnings("unchecked")
public void testPutExistsExpression() throws Exception {
SessionFactory<TestLsEntry> sessionFactory = mock(SessionFactory.class);
Session<TestLsEntry> session = mock(Session.class);
willReturn(Boolean.TRUE)
.given(session)
.exists(anyString());
RemoteFileTemplate<TestLsEntry> template = new RemoteFileTemplate<>(sessionFactory);
template.setRemoteDirectoryExpression(new LiteralExpression("foo/"));
template.setBeanFactory(mock(BeanFactory.class));
template.afterPropertiesSet();
TestRemoteFileOutboundGateway gw = new TestRemoteFileOutboundGateway(template, "put", "payload");
FileTransferringMessageHandler<TestLsEntry> handler = new FileTransferringMessageHandler<>(sessionFactory);
handler.setRemoteDirectoryExpression(new LiteralExpression("foo/"));
handler.setBeanFactory(mock(BeanFactory.class));
handler.afterPropertiesSet();
gw.afterPropertiesSet();
gw.setFileExistsModeExpressionString("headers[\"file.exists.mode\"]");
when(sessionFactory.getSession()).thenReturn(session);
MessageBuilder<String> requestMessageBuilder = MessageBuilder.withPayload("hello")
.setHeader(FileHeaders.FILENAME, "bar.txt");
Message<String> defaultMessage = requestMessageBuilder.build();
String path = (String) gw.handleRequestMessage(defaultMessage);
assertThat(path).isEqualTo("foo/bar.txt");
ArgumentCaptor<String> captor = ArgumentCaptor.forClass(String.class);
verify(session).write(any(InputStream.class), captor.capture());
assertThat(captor.getValue()).isEqualTo("foo/bar.txt.writing");
verify(session).rename("foo/bar.txt.writing", "foo/bar.txt");
Message<String> failMessage = requestMessageBuilder.setHeader("file.exists.mode", FileExistsMode.FAIL)
.build();
assertThatExceptionOfType(MessageDeliveryException.class)
.isThrownBy(() -> gw.handleRequestMessage(failMessage))
.withStackTraceContaining("The destination file already exists");
Message<String> replaceMessage = requestMessageBuilder.setHeader("file.exists.mode", "replace")
.build();
path = (String) gw.handleRequestMessage(replaceMessage);
assertThat(path).isEqualTo("foo/bar.txt");
captor = ArgumentCaptor.forClass(String.class);
verify(session, times(2)).write(any(InputStream.class), captor.capture());
assertThat(captor.getValue()).isEqualTo("foo/bar.txt.writing");
verify(session, times(2)).rename("foo/bar.txt.writing", "foo/bar.txt");
Message<String> appendMessage = requestMessageBuilder.setHeader("file.exists.mode", "APPEND")
.build();
path = (String) gw.handleRequestMessage(appendMessage);
assertThat(path).isEqualTo("foo/bar.txt");
captor = ArgumentCaptor.forClass(String.class);
verify(session).append(any(InputStream.class), captor.capture());
assertThat(captor.getValue()).isEqualTo("foo/bar.txt");
Message<String> ignoreMessage = requestMessageBuilder.setHeader("file.exists.mode", FileExistsMode.IGNORE)
.build();
path = (String) gw.handleRequestMessage(ignoreMessage);
assertThat(path).isEqualTo("foo/bar.txt");
// no more writes/appends
verify(session, times(2)).write(any(InputStream.class), anyString());
verify(session, times(1)).append(any(InputStream.class), anyString());
}
@Test
@SuppressWarnings("unchecked")
public void testMput() throws Exception {

18
src/reference/antora/modules/ROOT/pages/ftp/rft.adoc
View File

@@ -37,3 +37,21 @@ This is useful when you need to perform several high-level operations of the `Re
For example, `AbstractRemoteFileOutboundGateway` uses it with the `mput` command implementation, where we perform a `put` operation for each file in the provided directory and recursively for its sub-directories.
See the https://docs.spring.io/spring-integration/api/org/springframework/integration/file/remote/RemoteFileOperations.html#invoke[Javadoc] for more information.
Starting with version 6.5, the `AbstractRemoteFileOutboundGateway` supports dynamic resolution of `FileExistsMode` at runtime via SpEL expressions.
This allows you to determine the action to take when files already exist based on message content or other conditions.
To use this feature, configure the `fileExistsModeExpression` property on the gateway.
The expression can evaluate to:
* A `FileExistsMode` enum value (e.g., `FileExistsMode.REPLACE`)
* A string representation of a `FileExistsMode` (case-insensitive, e.g., "REPLACE", "append")
If the expression returns `null`, the default `fileExistsMode` configured on the gateway will be used.
See the https://docs.spring.io/spring-integration/api/org/springframework/integration/file/remote/gateway/AbstractRemoteFileOutboundGateway.html#setFileExistsModeExpression(org.springframework.expression.Expression)[Javadoc] for more information.
[IMPORTANT]
====
When using `FileExistsMode.APPEND`, temporary filename functionality is automatically disabled regardless of the `useTemporaryFileName` setting.
This is because appending to a temporary file and then renaming it would not achieve the intended append behavior.
====

6
src/reference/antora/modules/ROOT/pages/whats-new.adoc
View File

@@ -79,6 +79,12 @@ The `AbstractRecentFileListFilter` strategy has been introduced to accept only t
The respective implementations are provided: `RecentFileListFilter`, `FtpRecentFileListFilter`, `SftpRecentFileListFilter` and `SmbRecentFileListFilter`.
See xref:file/reading.adoc[Reading Files] for more information.
[[x6.5-file-exists-mode-expression]]
== FileExistsMode Expression Support
The remote file gateways (`AbstractRemoteFileOutboundGateway`) now support dynamic resolution of `FileExistsMode` at runtime via SpEL expressions.
See xref:ftp/rft.adoc[Remote File Gateways] for more information.
[[x6.5-hazelcast-changes]]
== Hazelcast Module Deprecations