diff --git a/spring-batch-excel/.editorconfig b/spring-batch-excel/.editorconfig new file mode 100644 index 0000000..3e1127a --- /dev/null +++ b/spring-batch-excel/.editorconfig @@ -0,0 +1,10 @@ +root = true + +[*.{adoc,bat,groovy,html,java,js,jsp,kt,kts,md,properties,py,rb,sh,sql,svg,txt,xml,xsd}] +charset = utf-8 + +[*.{groovy,java,kt,kts,xml,xsd}] +indent_style = tab +indent_size = 4 +continuation_indent_size = 8 +end_of_line = lf diff --git a/spring-batch-excel/.mvn/wrapper/maven-wrapper.jar b/spring-batch-excel/.mvn/wrapper/maven-wrapper.jar new file mode 100644 index 0000000..c6feb8b Binary files /dev/null and b/spring-batch-excel/.mvn/wrapper/maven-wrapper.jar differ diff --git a/spring-batch-excel/.mvn/wrapper/maven-wrapper.properties b/spring-batch-excel/.mvn/wrapper/maven-wrapper.properties new file mode 100644 index 0000000..be4fea7 --- /dev/null +++ b/spring-batch-excel/.mvn/wrapper/maven-wrapper.properties @@ -0,0 +1 @@ +distributionUrl=https://repo1.maven.org/maven2/org/apache/maven/apache-maven/3.6.3/apache-maven-3.6.3-bin.zip diff --git a/spring-batch-excel/README.adoc b/spring-batch-excel/README.adoc new file mode 100644 index 0000000..e5b5139 --- /dev/null +++ b/spring-batch-excel/README.adoc @@ -0,0 +1,115 @@ +# spring-batch-excel + +Spring Batch extension which contains an `ItemReader` implementation for Excel based on https://poi.apache.org[Apache POI]. It supports reading both XLS and XLSX files, for the latter there is also (experimental) streaming support. + +The `PoiItemReader` has the most features but is also the most memory intensive and might lead to memory issues with large XLS(X) sheets. + +To reduce the memory footprint the `StreamingXlsxItemReader` can be used, this will only keep the current row in memory and discard it afterwards. Not everything is supported while streaming the XLSX file, it can be that formulas don't get evaluated or lead to an error. + +## Configuration of `PoiItemReader` + +Next to the https://docs.spring.io/spring-batch/reference/html/configureJob.html[configuration of Spring Batch] one needs to configure the `PoiItemReader`. + +Configuration of can be done in XML or Java Config. + +### XML + +```xml + + + + + + +``` + +### Java Config + +```java +@Bean +@StepScope +public PoiItemReader excelReader() { + PoiItemReader reader = new PoiItemReader(); + reader.setResource(new FileSystemResource("/path/to/your/excel/file")); + reader.setRowMapper(rowMapper()); + return reader; +} + +@Bean +public RowMapper rowMapper() { + return new PassThroughRowMapper(); +} +``` + +## Configuration of `StreamingXlsxItemReader` + +Configuration can be done in XML or Java Config. + +### XML + +```xml + + + + + + +``` + +### Java Config + +```java +@Bean +@StepScope +public StreamingXlsxItemReader excelReader() { + StreamingXlsxItemReader reader = new StreamingXlsxItemReader(); + reader.setResource(new FileSystemResource("/path/to/your/excel/file")); + reader.setRowMapper(rowMapper()); + return reader; +} + +@Bean +public RowMapper rowMapper() { + return new PassThroughRowMapper(); +} +``` + + +## Configuration properties +[cols="1,1,1,4"] +.Properties for item readers +|=== +| Property | Required | Default | Description + +| `endAfterBlankLines` | no | 1 | The number of blank lines before stopping to read. +| `linesToSkip` | no | 0 | The number of lines to skip, this applies to each sheet in the Excel file, can be useful if the first couple of lines provide header information. +| `password` | no | `null` | The password used to protect an XLS file. Only works for XLS files not XLSX files (not supported with streaming). +| `resource` | yes | `null` | Location of the excel file to read, can be any resource supported by Spring. +| `rowMapper` | yes | `null` | transforms the rows read from the sheet(s) to an object which you can use in the rest of the process. +| `rowSetFactory` | no | `DefaultRowSetFactory` | For reading rows a `RowSet` abstraction is used. To construct a `RowSet` for the current `Sheet` a `RowSetFactory` is needed. The `DefaultRowSetFactory` constructs a `DefaultRowSet` and `DefaultRowSetMetaData`. For construction of the latter a `ColumnNameExtractor` is needed. At the moment there are 2 implementations +| `skippedRowsCallback` | no | `null` | When rows are skipped an optional `RowCallbackHandler` is called with the skipped row. This comes in handy when one needs to write the skipped rows to another file or create some logging. +| `strict` | no | `true` | This controls wether or not an exception is thrown if the file doesn't exists or isn't readable, by default an exception will be thrown. +|=== + + - `StaticColumnNameExtractor` uses a preset list of column names. + - `RowNumberColumnNameExtractor` (**the default**) reads a given row (default 0) to determine the column names of the current sheet + +## RowMappers +To map a read row a `RowMapper` is needed. Out-of-the-box there are 2 implementations. The `PassThroughRowMapper` and `BeanWrapperRowMapper`. + +### PassThroughRowMapper +Transforms the read row from excel into a `String[]`. + +### BeanWrapperRowMapper +Uses a `BeanWrapper` to convert a given row into an object. Uses the column names of the given `RowSet` to map column to properties of the `targetType` or prototype bean. + +```java + + + + + + + + +``` diff --git a/spring-batch-excel/README.md b/spring-batch-excel/README.md deleted file mode 100644 index 8195204..0000000 --- a/spring-batch-excel/README.md +++ /dev/null @@ -1,77 +0,0 @@ -# spring-batch-excel - -Spring Batch extension which contains `ItemReader` implementations for Excel. Support for both [JExcel][1] and [Apache POI][2] is available. Simple xls documents can be read with both implementations, however for reading the newer xlsx format [Apache POI][2] is required. - -## Configuration - -Next to the [configuration of Spring Batch](https://docs.spring.io/spring-batch/reference/html/configureJob.html) one needs to configure the `ItemReader` for the desired framework. -There are 2 `ItemReaders` one can configure: - -- `org.springframework.batch.item.excel.jxl.JxlItemReader` -- `org.springframework.batch.item.excel.poi.PoiItemReader` - -Configuration of both readers is the same. - -#### XML - - - - - - - - -#### Java Config - - @Bean - public PoiItemReader excelReader() { - PoiItemReader reader = new PoiItemReader(); - reader.setResource(new ClassPathResource("/path/to/your/excel/file")); - reader.setRowMapper(rowMapper()); - return reader; - } - - @Bean - public RowMapper rowMapper() { - return new PassThroughRowMapper(); - } - -Each reader takes a `resource` and a `rowMapper`. The `resource` is the location of the excel file to read and the `rowMapper` transforms the rows in excel to an object which you can use in the rest of the process. - -Optionally one can also set the `skippedRowsCallback`, `linesToSkip`, `strict` and `rowSetFactory` properties. - -##### skippedRowsCallback -When rows are skipped an optional `org.springframework.batch.item.excel.RowCallbackHandler` is called with the skipped row. This comes in handy when one needs to write the skipped rows to another file or create some logging. - -##### linesToSkip -The number of lines to skip, this applies to each sheet in the Excel file, can be useful if the first couple of lines provide header information. - -##### strict -By default `true`. This controls wether or not an exception is thrown if the file doesn't exists, by default an exception will be thrown. - -##### rowSetFactory -For reading rows a `RowSet` abstraction is used. To construct a `RowSet` for the current `Sheet` a `RowSetFactory` is needed. The `DefaultRowSetFactory` constructs a `DefaultRowSet` and `DefaultRowSetMetaData`. For construction of the latter a `ColumnNameExtractor` is needed. At the moment there are 2 implementations - - - `StaticColumnNameExtractor` uses a preset list of column names. - - `RowNumberColumnNameExtractor` (**the default**) reads a given row (default 0) to determine the column names of the current sheet - -## RowMappers -Next to the default `ItemReader` implementations there are also 2 `RowMapper` implementations. - -### PassThroughRowMapper -Transforms the read row from excel into a `String[]`. - -### BeanWrapperRowMapper -Uses a `BeanWrapper` to convert a given row into an object. Uses the column names of the given `RowSet` to map column to properties of the `targetType` or prototype bean. - - - - - - - - - - -[1]: http://jexcelapi.sourceforge.net -[2]: https://poi.apache.org \ No newline at end of file diff --git a/spring-batch-excel/mvnw b/spring-batch-excel/mvnw new file mode 100755 index 0000000..f13e138 --- /dev/null +++ b/spring-batch-excel/mvnw @@ -0,0 +1,236 @@ +#!/bin/sh +# ---------------------------------------------------------------------------- +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# ---------------------------------------------------------------------------- + +# ---------------------------------------------------------------------------- +# Maven2 Start Up Batch script +# +# Required ENV vars: +# ------------------ +# JAVA_HOME - location of a JDK home dir +# +# Optional ENV vars +# ----------------- +# M2_HOME - location of maven2's installed home dir +# MAVEN_OPTS - parameters passed to the Java VM when running Maven +# e.g. to debug Maven itself, use +# set MAVEN_OPTS=-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000 +# MAVEN_SKIP_RC - flag to disable loading of mavenrc files +# ---------------------------------------------------------------------------- + +if [ -z "$MAVEN_SKIP_RC" ] ; then + + if [ -f /etc/mavenrc ] ; then + . /etc/mavenrc + fi + + if [ -f "$HOME/.mavenrc" ] ; then + . "$HOME/.mavenrc" + fi + +fi + +# OS specific support. $var _must_ be set to either true or false. +cygwin=false; +darwin=false; +mingw=false +case "`uname`" in + CYGWIN*) cygwin=true ;; + MINGW*) mingw=true;; + Darwin*) darwin=true + # + # Look for the Apple JDKs first to preserve the existing behaviour, and then look + # for the new JDKs provided by Oracle. + # + if [ -z "$JAVA_HOME" ] && [ -L /System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK ] ; then + # + # Apple JDKs + # + export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Home + fi + + if [ -z "$JAVA_HOME" ] && [ -L /System/Library/Java/JavaVirtualMachines/CurrentJDK ] ; then + # + # Apple JDKs + # + export JAVA_HOME=/System/Library/Java/JavaVirtualMachines/CurrentJDK/Contents/Home + fi + + if [ -z "$JAVA_HOME" ] && [ -L "/Library/Java/JavaVirtualMachines/CurrentJDK" ] ; then + # + # Oracle JDKs + # + export JAVA_HOME=/Library/Java/JavaVirtualMachines/CurrentJDK/Contents/Home + fi + + if [ -z "$JAVA_HOME" ] && [ -x "/usr/libexec/java_home" ]; then + # + # Apple JDKs + # + export JAVA_HOME=`/usr/libexec/java_home` + fi + ;; +esac + +if [ -z "$JAVA_HOME" ] ; then + if [ -r /etc/gentoo-release ] ; then + JAVA_HOME=`java-config --jre-home` + fi +fi + +if [ -z "$M2_HOME" ] ; then + ## resolve links - $0 may be a link to maven's home + PRG="$0" + + # need this for relative symlinks + while [ -h "$PRG" ] ; do + ls=`ls -ld "$PRG"` + link=`expr "$ls" : '.*-> \(.*\)$'` + if expr "$link" : '/.*' > /dev/null; then + PRG="$link" + else + PRG="`dirname "$PRG"`/$link" + fi + done + + saveddir=`pwd` + + M2_HOME=`dirname "$PRG"`/.. + + # make it fully qualified + M2_HOME=`cd "$M2_HOME" && pwd` + + cd "$saveddir" + # echo Using m2 at $M2_HOME +fi + +# For Cygwin, ensure paths are in UNIX format before anything is touched +if $cygwin ; then + [ -n "$M2_HOME" ] && + M2_HOME=`cygpath --unix "$M2_HOME"` + [ -n "$JAVA_HOME" ] && + JAVA_HOME=`cygpath --unix "$JAVA_HOME"` + [ -n "$CLASSPATH" ] && + CLASSPATH=`cygpath --path --unix "$CLASSPATH"` +fi + +# For Migwn, ensure paths are in UNIX format before anything is touched +if $mingw ; then + [ -n "$M2_HOME" ] && + M2_HOME="`(cd "$M2_HOME"; pwd)`" + [ -n "$JAVA_HOME" ] && + JAVA_HOME="`(cd "$JAVA_HOME"; pwd)`" + # TODO classpath? +fi + +if [ -z "$JAVA_HOME" ]; then + javaExecutable="`which javac`" + if [ -n "$javaExecutable" ] && ! [ "`expr \"$javaExecutable\" : '\([^ ]*\)'`" = "no" ]; then + # readlink(1) is not available as standard on Solaris 10. + readLink=`which readlink` + if [ ! `expr "$readLink" : '\([^ ]*\)'` = "no" ]; then + if $darwin ; then + javaHome="`dirname \"$javaExecutable\"`" + javaExecutable="`cd \"$javaHome\" && pwd -P`/javac" + else + javaExecutable="`readlink -f \"$javaExecutable\"`" + fi + javaHome="`dirname \"$javaExecutable\"`" + javaHome=`expr "$javaHome" : '\(.*\)/bin'` + JAVA_HOME="$javaHome" + export JAVA_HOME + fi + fi +fi + +if [ -z "$JAVACMD" ] ; then + if [ -n "$JAVA_HOME" ] ; then + if [ -x "$JAVA_HOME/jre/sh/java" ] ; then + # IBM's JDK on AIX uses strange locations for the executables + JAVACMD="$JAVA_HOME/jre/sh/java" + else + JAVACMD="$JAVA_HOME/bin/java" + fi + else + JAVACMD="`which java`" + fi +fi + +if [ ! -x "$JAVACMD" ] ; then + echo "Error: JAVA_HOME is not defined correctly." >&2 + echo " We cannot execute $JAVACMD" >&2 + exit 1 +fi + +if [ -z "$JAVA_HOME" ] ; then + echo "Warning: JAVA_HOME environment variable is not set." +fi + +CLASSWORLDS_LAUNCHER=org.codehaus.plexus.classworlds.launcher.Launcher + +# traverses directory structure from process work directory to filesystem root +# first directory with .mvn subdirectory is considered project base directory +find_maven_basedir() { + local basedir=$(pwd) + local wdir=$(pwd) + while [ "$wdir" != '/' ] ; do + if [ -d "$wdir"/.mvn ] ; then + basedir=$wdir + break + fi + wdir=$(cd "$wdir/.."; pwd) + done + echo "${basedir}" +} + +# concatenates all lines of a file +concat_lines() { + if [ -f "$1" ]; then + echo "$(tr -s '\n' ' ' < "$1")" + fi +} + +export MAVEN_PROJECTBASEDIR=${MAVEN_BASEDIR:-$(find_maven_basedir)} +MAVEN_OPTS="$(concat_lines "$MAVEN_PROJECTBASEDIR/.mvn/jvm.config") $MAVEN_OPTS" + +# For Cygwin, switch paths to Windows format before running java +if $cygwin; then + [ -n "$M2_HOME" ] && + M2_HOME=`cygpath --path --windows "$M2_HOME"` + [ -n "$JAVA_HOME" ] && + JAVA_HOME=`cygpath --path --windows "$JAVA_HOME"` + [ -n "$CLASSPATH" ] && + CLASSPATH=`cygpath --path --windows "$CLASSPATH"` + [ -n "$MAVEN_PROJECTBASEDIR" ] && + MAVEN_PROJECTBASEDIR=`cygpath --path --windows "$MAVEN_PROJECTBASEDIR"` +fi + +# Provide a "standardized" way to retrieve the CLI args that will +# work with both Windows and non-Windows executions. +MAVEN_CMD_LINE_ARGS="$MAVEN_CONFIG $@" +export MAVEN_CMD_LINE_ARGS + +WRAPPER_LAUNCHER=org.apache.maven.wrapper.MavenWrapperMain + +# avoid using MAVEN_CMD_LINE_ARGS below since that would loose parameter escaping in $@ +exec "$JAVACMD" \ + $MAVEN_OPTS \ + -classpath "$MAVEN_PROJECTBASEDIR/.mvn/wrapper/maven-wrapper.jar" \ + "-Dmaven.home=${M2_HOME}" "-Dmaven.multiModuleProjectDirectory=${MAVEN_PROJECTBASEDIR}" \ + ${WRAPPER_LAUNCHER} $MAVEN_CONFIG "$@" diff --git a/spring-batch-excel/mvnw.cmd b/spring-batch-excel/mvnw.cmd new file mode 100644 index 0000000..bb9bb46 --- /dev/null +++ b/spring-batch-excel/mvnw.cmd @@ -0,0 +1,146 @@ +@REM ---------------------------------------------------------------------------- +@REM Licensed to the Apache Software Foundation (ASF) under one +@REM or more contributor license agreements. See the NOTICE file +@REM distributed with this work for additional information +@REM regarding copyright ownership. The ASF licenses this file +@REM to you under the Apache License, Version 2.0 (the +@REM "License"); you may not use this file except in compliance +@REM with the License. You may obtain a copy of the License at +@REM +@REM https://www.apache.org/licenses/LICENSE-2.0 +@REM +@REM Unless required by applicable law or agreed to in writing, +@REM software distributed under the License is distributed on an +@REM "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +@REM KIND, either express or implied. See the License for the +@REM specific language governing permissions and limitations +@REM under the License. +@REM ---------------------------------------------------------------------------- + +@REM ---------------------------------------------------------------------------- +@REM Maven2 Start Up Batch script +@REM +@REM Required ENV vars: +@REM JAVA_HOME - location of a JDK home dir +@REM +@REM Optional ENV vars +@REM M2_HOME - location of maven2's installed home dir +@REM MAVEN_BATCH_ECHO - set to 'on' to enable the echoing of the batch commands +@REM MAVEN_BATCH_PAUSE - set to 'on' to wait for a key stroke before ending +@REM MAVEN_OPTS - parameters passed to the Java VM when running Maven +@REM e.g. to debug Maven itself, use +@REM set MAVEN_OPTS=-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000 +@REM MAVEN_SKIP_RC - flag to disable loading of mavenrc files +@REM ---------------------------------------------------------------------------- + +@REM Begin all REM lines with '@' in case MAVEN_BATCH_ECHO is 'on' +@echo off +@REM enable echoing my setting MAVEN_BATCH_ECHO to 'on' +@if "%MAVEN_BATCH_ECHO%" == "on" echo %MAVEN_BATCH_ECHO% + +@REM set %HOME% to equivalent of $HOME +if "%HOME%" == "" (set "HOME=%HOMEDRIVE%%HOMEPATH%") + +@REM Execute a user defined script before this one +if not "%MAVEN_SKIP_RC%" == "" goto skipRcPre +@REM check for pre script, once with legacy .bat ending and once with .cmd ending +if exist "%HOME%\mavenrc_pre.bat" call "%HOME%\mavenrc_pre.bat" +if exist "%HOME%\mavenrc_pre.cmd" call "%HOME%\mavenrc_pre.cmd" +:skipRcPre + +@setlocal + +set ERROR_CODE=0 + +@REM To isolate internal variables from possible post scripts, we use another setlocal +@setlocal + +@REM ==== START VALIDATION ==== +if not "%JAVA_HOME%" == "" goto OkJHome + +echo. +echo Error: JAVA_HOME not found in your environment. >&2 +echo Please set the JAVA_HOME variable in your environment to match the >&2 +echo location of your Java installation. >&2 +echo. +goto error + +:OkJHome +if exist "%JAVA_HOME%\bin\java.exe" goto init + +echo. +echo Error: JAVA_HOME is set to an invalid directory. >&2 +echo JAVA_HOME = "%JAVA_HOME%" >&2 +echo Please set the JAVA_HOME variable in your environment to match the >&2 +echo location of your Java installation. >&2 +echo. +goto error + +@REM ==== END VALIDATION ==== + +:init + +set MAVEN_CMD_LINE_ARGS=%MAVEN_CONFIG% %* + +@REM Find the project base dir, i.e. the directory that contains the folder ".mvn". +@REM Fallback to current working directory if not found. + +set MAVEN_PROJECTBASEDIR=%MAVEN_BASEDIR% +IF NOT "%MAVEN_PROJECTBASEDIR%"=="" goto endDetectBaseDir + +set EXEC_DIR=%CD% +set WDIR=%EXEC_DIR% +:findBaseDir +IF EXIST "%WDIR%"\.mvn goto baseDirFound +cd .. +IF "%WDIR%"=="%CD%" goto baseDirNotFound +set WDIR=%CD% +goto findBaseDir + +:baseDirFound +set MAVEN_PROJECTBASEDIR=%WDIR% +cd "%EXEC_DIR%" +goto endDetectBaseDir + +:baseDirNotFound +set MAVEN_PROJECTBASEDIR=%EXEC_DIR% +cd "%EXEC_DIR%" + +:endDetectBaseDir + +IF NOT EXIST "%MAVEN_PROJECTBASEDIR%\.mvn\jvm.config" goto endReadAdditionalConfig + +@setlocal EnableExtensions EnableDelayedExpansion +for /F "usebackq delims=" %%a in ("%MAVEN_PROJECTBASEDIR%\.mvn\jvm.config") do set JVM_CONFIG_MAVEN_PROPS=!JVM_CONFIG_MAVEN_PROPS! %%a +@endlocal & set JVM_CONFIG_MAVEN_PROPS=%JVM_CONFIG_MAVEN_PROPS% + +:endReadAdditionalConfig + +SET MAVEN_JAVA_EXE="%JAVA_HOME%\bin\java.exe" + +set WRAPPER_JAR=""%MAVEN_PROJECTBASEDIR%\.mvn\wrapper\maven-wrapper.jar"" +set WRAPPER_LAUNCHER=org.apache.maven.wrapper.MavenWrapperMain + +# avoid using MAVEN_CMD_LINE_ARGS below since that would loose parameter escaping in %* +%MAVEN_JAVA_EXE% %JVM_CONFIG_MAVEN_PROPS% %MAVEN_OPTS% %MAVEN_DEBUG_OPTS% -classpath %WRAPPER_JAR% "-Dmaven.multiModuleProjectDirectory=%MAVEN_PROJECTBASEDIR%" %WRAPPER_LAUNCHER% %MAVEN_CONFIG% %* +if ERRORLEVEL 1 goto error +goto end + +:error +set ERROR_CODE=1 + +:end +@endlocal & set ERROR_CODE=%ERROR_CODE% + +if not "%MAVEN_SKIP_RC%" == "" goto skipRcPost +@REM check for post script, once with legacy .bat ending and once with .cmd ending +if exist "%HOME%\mavenrc_post.bat" call "%HOME%\mavenrc_post.bat" +if exist "%HOME%\mavenrc_post.cmd" call "%HOME%\mavenrc_post.cmd" +:skipRcPost + +@REM pause the script if MAVEN_BATCH_PAUSE is set to 'on' +if "%MAVEN_BATCH_PAUSE%" == "on" pause + +if "%MAVEN_TERMINATE_CMD%" == "on" exit %ERROR_CODE% + +exit /B %ERROR_CODE% diff --git a/spring-batch-excel/pom.xml b/spring-batch-excel/pom.xml index 84988f1..2c9f622 100644 --- a/spring-batch-excel/pom.xml +++ b/spring-batch-excel/pom.xml @@ -1,6 +1,6 @@ - + + 4.0.0 org.springframework.batch.extensions @@ -26,12 +25,28 @@ UTF-8 - 3.0.3.RELEASE - 2.6.12 - 3.11 + UTF-8 + 1.8 + + 4.3.2 + 4.1.2 + + + 3.18.1 + 5.7.1 + 2.13.3 + 3.6.0 + + + mdeinum + Marten Deinum + marten@deinum.biz + + + org.springframework.batch @@ -43,12 +58,6 @@ spring-batch-infrastructure ${spring.batch.version} - - net.sourceforge.jexcelapi - jxl - ${jxl.version} - true - org.apache.poi poi @@ -64,23 +73,28 @@ - junit - junit - 4.12 + org.junit.jupiter + junit-jupiter + ${junit.jupiter.version} test org.mockito mockito-core - 1.10.19 - test - - - org.springframework - spring-test - 4.0.5.RELEASE + ${mockito.version} test + + org.assertj + assertj-core + ${assertj.version} + + + org.apache.logging.log4j + log4j-core + ${log4j.version} + test + @@ -88,16 +102,19 @@ org.apache.maven.plugins maven-compiler-plugin - 3.2 + 3.8.1 ${java.version} ${java.version} + + -Xlint:all,deprecation + org.apache.maven.plugins maven-javadoc-plugin - 2.10.1 + 3.2.0 attach-javadocs @@ -110,7 +127,7 @@ org.apache.maven.plugins maven-source-plugin - 2.4 + 3.2.1 attach-sources @@ -120,6 +137,37 @@ + + org.apache.maven.plugins + maven-checkstyle-plugin + 3.1.1 + + + com.puppycrawl.tools + checkstyle + 8.32 + + + io.spring.javaformat + spring-javaformat-checkstyle + 0.0.27 + + + + + checkstyle-validation + validate + true + + io/spring/javaformat/checkstyle/checkstyle.xml + true + + + check + + + + diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/AbstractExcelItemReader.java b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/AbstractExcelItemReader.java new file mode 100644 index 0000000..f165d63 --- /dev/null +++ b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/AbstractExcelItemReader.java @@ -0,0 +1,298 @@ +/* + * Copyright 2006-2021 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.extensions.excel; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; + +import org.springframework.batch.extensions.excel.support.rowset.DefaultRowSetFactory; +import org.springframework.batch.extensions.excel.support.rowset.RowSet; +import org.springframework.batch.extensions.excel.support.rowset.RowSetFactory; +import org.springframework.batch.item.file.ResourceAwareItemReaderItemStream; +import org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader; +import org.springframework.beans.factory.InitializingBean; +import org.springframework.core.io.Resource; +import org.springframework.util.Assert; +import org.springframework.util.ClassUtils; + +/** + * {@link org.springframework.batch.item.ItemReader} implementation to read an Excel file. + * It will read the file sheet for sheet and row for row. It is loosy based on the + * {@link org.springframework.batch.item.file.FlatFileItemReader} + * + * @param the type + * @author Marten Deinum + * @since 0.1.0 + */ +public abstract class AbstractExcelItemReader extends AbstractItemCountingItemStreamItemReader + implements ResourceAwareItemReaderItemStream, InitializingBean { + + protected final Log logger = LogFactory.getLog(getClass()); + + private Resource resource; + + private int linesToSkip = 0; + + private int currentSheet = 0; + + private int endAfterBlankLines = 1; + + private RowMapper rowMapper; + + private RowCallbackHandler skippedRowsCallback; + + private boolean noInput = false; + + private boolean strict = true; + + private RowSetFactory rowSetFactory = new DefaultRowSetFactory(); + + private RowSet rs; + + private String password; + + public AbstractExcelItemReader() { + super(); + this.setName(ClassUtils.getShortName(this.getClass())); + } + + @Override + public T read() throws Exception { + T item = super.read(); + int blankLines = 0; + while (item == null) { + blankLines++; + if (blankLines >= this.endAfterBlankLines) { + return null; + } + item = super.read(); + if (item != null) { + return item; + } + } + return item; + } + + @Override + protected T doRead() { + if (this.noInput) { + return null; + } + + if (this.rs == null || !this.rs.next()) { + if (!nextSheet()) { + if (this.logger.isDebugEnabled()) { + this.logger.debug("No more sheets in '" + this.resource.getDescription() + "'."); + } + return null; + } + } + + // skip all the blank row from which content has been deleted but still a valid row + while (null != this.rs.getCurrentRow() && isInvalidValidRow(this.rs)) { + this.rs.next(); + } + try { + return (this.rs.getCurrentRow() != null) ? this.rowMapper.mapRow(this.rs) : null; + } + catch (Exception ex) { + throw new ExcelFileParseException("Exception parsing Excel file.", ex, this.resource.getDescription(), + this.rs.getMetaData().getSheetName(), this.rs.getCurrentRowIndex(), this.rs.getCurrentRow()); + } + } + + /** + * On restart this will increment rowSet to where job left off previously. + * Temporarily switch out the configured {@code RowMapper} so we can use the + * {@code #doRead} method and reuse the logic in there, but without actually map to + * instances (this to save memory and have better performance). + */ + @Override + protected void jumpToItem(final int itemIndex) { + RowMapper current = this.rowMapper; + this.rowMapper = (rs) -> null; + try { + for (int i = 0; i < itemIndex; i++) { + doRead(); + } + } + finally { + this.rowMapper = current; + } + } + + private boolean isInvalidValidRow(RowSet rs) { + for (String str : rs.getCurrentRow()) { + if (str.length() > 0) { + return false; + } + } + return true; + } + + @Override + protected void doOpen() throws Exception { + Assert.notNull(this.resource, "Input resource must be set"); + this.noInput = true; + if (!this.resource.exists()) { + if (this.strict) { + throw new IllegalStateException( + "Input resource must exist (reader is in 'strict' mode): " + this.resource); + } + this.logger.warn("Input resource does not exist '" + this.resource.getDescription() + "'."); + return; + } + + if (!this.resource.isReadable()) { + if (this.strict) { + throw new IllegalStateException( + "Input resource must be readable (reader is in 'strict' mode): " + this.resource); + } + this.logger.warn("Input resource is not readable '" + this.resource.getDescription() + "'."); + return; + } + + this.openExcelFile(this.resource, this.password); + this.noInput = false; + if (this.logger.isDebugEnabled()) { + this.logger.debug("Opened workbook [" + this.resource.getFilename() + "] with " + this.getNumberOfSheets() + + " sheets."); + } + } + + private boolean nextSheet() { + while (this.currentSheet < this.getNumberOfSheets()) { + final Sheet sheet = this.getSheet(this.currentSheet); + this.rs = this.rowSetFactory.create(sheet); + if (this.logger.isDebugEnabled()) { + this.logger.debug("Opening sheet " + sheet.getName() + "."); + } + + for (int i = 0; i < this.linesToSkip; i++) { + if (this.rs.next() && this.skippedRowsCallback != null) { + this.skippedRowsCallback.handleRow(this.rs); + } + } + if (this.logger.isDebugEnabled()) { + this.logger.debug("Openend sheet " + sheet.getName() + ", with " + sheet.getNumberOfRows() + " rows."); + } + this.currentSheet++; + if (this.rs.next()) { + return true; + } + } + return false; + } + + protected void doClose() throws Exception { + this.currentSheet = 0; + this.rs = null; + } + + /** + * Public setter for the input resource. + * @param resource the {@code Resource} pointing to the Excelfile + */ + public void setResource(final Resource resource) { + this.resource = resource; + } + + public void afterPropertiesSet() throws Exception { + Assert.notNull(this.rowMapper, "RowMapper must be set"); + } + + /** + * Set the number of lines to skip. This number is applied to all worksheet in the + * excel file! default to 0 + * @param linesToSkip number of lines to skip + */ + public void setLinesToSkip(final int linesToSkip) { + this.linesToSkip = linesToSkip; + } + + /** + * Get the sheet based on the given sheet index. + * @param sheet the sheet index + * @return the sheet or null when no sheet available. + */ + protected abstract Sheet getSheet(int sheet); + + /** + * The number of sheets in the underlying workbook. + * @return the number of sheets. + */ + protected abstract int getNumberOfSheets(); + + /** + * Opens the excel file and reads the file and sheet metadata. Uses a {@code Resource} to read the sheets, + * this file can optionally be password protected. + * @param resource {@code Resource} pointing to the Excel file to read + * @param password optional password + * @throws Exception when the Excel sheet cannot be accessed + */ + protected abstract void openExcelFile(Resource resource, String password) throws Exception; + + /** + * In strict mode the reader will throw an exception on + * {@link #open(org.springframework.batch.item.ExecutionContext)} if the input + * resource does not exist. + * @param strict true by default + */ + public void setStrict(final boolean strict) { + this.strict = strict; + } + + /** + * Public setter for the {@code rowMapper}. Used to map a row read from the underlying + * Excel workbook. + * @param rowMapper the {@code RowMapper} to use. + */ + public void setRowMapper(final RowMapper rowMapper) { + this.rowMapper = rowMapper; + } + + /** + * Public setter for the rowSetFactory. Used to create a {@code RowSet} + * implemenation. By default the {@code DefaultRowSetFactory} is used. + * @param rowSetFactory the {@code RowSetFactory} to use. + */ + public void setRowSetFactory(RowSetFactory rowSetFactory) { + this.rowSetFactory = rowSetFactory; + } + + /** + * Set the callback handler to call when a row is being skipped. + * @param skippedRowsCallback will be called for each one of the initial skipped lines + * before any items are read. + */ + public void setSkippedRowsCallback(final RowCallbackHandler skippedRowsCallback) { + this.skippedRowsCallback = skippedRowsCallback; + } + + public void setEndAfterBlankLines(final int endAfterBlankLines) { + this.endAfterBlankLines = endAfterBlankLines; + } + + /** + * The password used to protect the file to open. + * @param password the password + */ + public void setPassword(String password) { + this.password = password; + } + +} diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/ExcelFileParseException.java b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/ExcelFileParseException.java new file mode 100644 index 0000000..c419e79 --- /dev/null +++ b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/ExcelFileParseException.java @@ -0,0 +1,74 @@ +/* + * Copyright 2006-2021 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.extensions.excel; + +import org.springframework.batch.item.ParseException; + +/** + * Exception thrown when parsing excel files. The name of the sheet, the row number on + * that sheet and the name of the excel file can be passed in so that in exception + * handling we can reuse it. This class only has simply dependencies to make it is generic + * as possible. + * + * @author Marten Deinum + * @since 0.1.0 + */ +public class ExcelFileParseException extends ParseException { + + private final String filename; + + private final String sheet; + + private final String[] row; + + private final int rowNumber; + + /** + * Construct an {@link ExcelFileParseException}. + * @param message the message + * @param cause the root cause + * @param filename the name of the excel file + * @param sheet the name of the sheet + * @param rowNumber the row number in the current sheet + * @param row the row data as text + */ + public ExcelFileParseException(final String message, final Throwable cause, final String filename, + final String sheet, final int rowNumber, final String[] row) { + super(message, cause); + this.filename = filename; + this.sheet = sheet; + this.rowNumber = rowNumber; + this.row = row; + } + + public String getFilename() { + return this.filename; + } + + public String getSheet() { + return this.sheet; + } + + public int getRowNumber() { + return this.rowNumber; + } + + public String[] getRow() { + return this.row; + } + +} diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/RowCallbackHandler.java b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/RowCallbackHandler.java new file mode 100644 index 0000000..e7c9e72 --- /dev/null +++ b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/RowCallbackHandler.java @@ -0,0 +1,40 @@ +/* + * Copyright 2011 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.extensions.excel; + +import org.springframework.batch.extensions.excel.support.rowset.RowSet; + +/** + * Callback to handle skipped lines. Useful for header/footer processing. + * + * @author Marten Deinum + */ +public interface RowCallbackHandler { + + /** + * Implementations must implement this method to process each row of data in the + * {@code RowSet}. + *

This method should not call {@code next()} on the {@code RowSetSet}; it is only + * supposed to extract values of the current row. + *

Exactly what the implementation chooses to do is up to it: A trivial implementation + * might simply count rows, while another implementation might build a special header + * row. + * @param rs the {@code RowSet} to process (preset at the current row) + */ + void handleRow(RowSet rs); + +} diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/RowMapper.java b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/RowMapper.java similarity index 50% rename from spring-batch-excel/src/main/java/org/springframework/batch/item/excel/RowMapper.java rename to spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/RowMapper.java index d9f05d1..5265784 100644 --- a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/RowMapper.java +++ b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/RowMapper.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2015 the original author or authors. + * Copyright 2006-2021 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * https://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -13,28 +13,28 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.springframework.batch.item.excel; -import org.springframework.batch.item.excel.support.rowset.RowSet; +package org.springframework.batch.extensions.excel; + +import org.springframework.batch.extensions.excel.support.rowset.RowSet; /** * Map rows from an excel sheet to an object. * * @param the type * @author Marten Deinum - * @since 0.5.0 + * @since 0.1.0 */ public interface RowMapper { - /** - * Implementations must implement this method to map the provided row to - * the parameter type T. The row number represents the number of rows - * into a {@link Sheet} the current line resides. - * - * @param rs the RowSet used for mapping. - * @return mapped object of type T - * @throws Exception if error occured while parsing. - */ - T mapRow(RowSet rs) throws Exception; + /** + * Implementations must implement this method to map the provided row to the parameter + * type T. The row number represents the number of rows into a {@link Sheet} the + * current line resides. + * @param rs the RowSet used for mapping. + * @return mapped object of type T + * @throws Exception if error occured while parsing. + */ + T mapRow(RowSet rs) throws Exception; } diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/Sheet.java b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/Sheet.java new file mode 100644 index 0000000..70080a2 --- /dev/null +++ b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/Sheet.java @@ -0,0 +1,53 @@ +/* + * Copyright 2006-2021 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.extensions.excel; + +import org.springframework.lang.Nullable; + +/** + * Interface to wrap different Excel implementations like JExcel or Apache POI. + * + * @author Marten Deinum + * @since 0.1.0 + */ +public interface Sheet extends Iterable, AutoCloseable { + + /** + * Get the number of rows in this sheet. + * @return the number of rows. + */ + int getNumberOfRows(); + + /** + * Get the name of the sheet. + * @return the name of the sheet. + */ + String getName(); + + /** + * Get the row as a {@code String[]}. Returns {@code null} if the row doesn't exist. + * @param rowNumber the row number to read. + * @return a {@code String[]} or {@code null} + */ + @Nullable + String[] getRow(int rowNumber); + + @Override + default void close() throws Exception { + } + +} diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/mapping/BeanWrapperRowMapper.java b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/mapping/BeanWrapperRowMapper.java new file mode 100644 index 0000000..96a9297 --- /dev/null +++ b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/mapping/BeanWrapperRowMapper.java @@ -0,0 +1,382 @@ +/* + * Copyright 2006-2021 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.extensions.excel.mapping; + +import java.util.HashMap; +import java.util.HashSet; +import java.util.Map; +import java.util.Properties; +import java.util.Set; +import java.util.concurrent.ConcurrentHashMap; +import java.util.concurrent.ConcurrentMap; + +import org.springframework.batch.extensions.excel.RowMapper; +import org.springframework.batch.extensions.excel.support.rowset.RowSet; +import org.springframework.batch.support.DefaultPropertyEditorRegistrar; +import org.springframework.beans.BeanUtils; +import org.springframework.beans.BeanWrapperImpl; +import org.springframework.beans.MutablePropertyValues; +import org.springframework.beans.NotWritablePropertyException; +import org.springframework.beans.PropertyAccessor; +import org.springframework.beans.PropertyAccessorUtils; +import org.springframework.beans.factory.BeanFactory; +import org.springframework.beans.factory.BeanFactoryAware; +import org.springframework.beans.factory.InitializingBean; +import org.springframework.util.Assert; +import org.springframework.validation.BindException; +import org.springframework.validation.DataBinder; + +/** + * {@link RowMapper} implementation based on bean property paths. The {@link RowSet} to be + * mapped should have field name meta data corresponding to bean property paths in an + * instance of the desired type. The instance is created and initialized either by + * referring to to a prototype object by bean name in the enclosing BeanFactory, or by + * providing a class to instantiate reflectively.
+ *
+ * + * Nested property paths, including indexed properties in maps and collections, can be + * referenced by the {@link RowSet}names. They will be converted to nested bean properties + * inside the prototype. The {@link RowSet} and the prototype are thus tightly coupled by + * the fields that are available and those that can be initialized. If some of the nested + * properties are optional (e.g. collection members) they need to be removed by a post + * processor.
+ *
+ * + * To customize the way that {@link RowSet} values are converted to the desired type for + * injecting into the prototype there are several choices. You can inject + * {@link java.beans.PropertyEditor} instances directly through the + * {@link #setCustomEditors(Map) customEditors} property, or you can override the + * {@link #createBinder(Object)} and {@link #initBinder(DataBinder)} methods, or you can + * provide a custom {@link RowSet} implementation.
+ *
+ * + * Property name matching is "fuzzy" in the sense that it tolerates close matches, as long + * as the match is unique. For instance: + * + *

    + *
  • Quantity = quantity (field names can be capitalised)
  • + *
  • ISIN = isin (acronyms can be lower case bean property names, as per Java Beans + * recommendations)
  • + *
  • DuckPate = duckPate (capitalisation including camel casing)
  • + *
  • ITEM_ID = itemId (capitalisation and replacing word boundary with underscore)
  • + *
  • ORDER.CUSTOMER_ID = order.customerId (nested paths are recursively checked)
  • + *
+ * + * The algorithm used to match a property name is to start with an exact match and then + * search successively through more distant matches until precisely one match is found. If + * more than one match is found there will be an error. + * + * @param type + * @author Marten Deinum + * @since 0.1.0 + */ +public class BeanWrapperRowMapper extends DefaultPropertyEditorRegistrar + implements RowMapper, BeanFactoryAware, InitializingBean { + + private String name; + + private Class type; + + private BeanFactory beanFactory; + + private final ConcurrentMap> propertiesMatched = new ConcurrentHashMap<>(); + + private int distanceLimit = 5; + + private boolean strict = true; + + @Override + public void setBeanFactory(BeanFactory beanFactory) { + this.beanFactory = beanFactory; + } + + /** + * The maximum difference that can be tolerated in spelling between input key names + * and bean property names. Defaults to 5, but could be set lower if the field names + * match the bean names. + * @param distanceLimit the distance limit to set + */ + public void setDistanceLimit(int distanceLimit) { + this.distanceLimit = distanceLimit; + } + + /** + * The bean name (id) for an object that can be populated from the field set that will + * be passed into {@link #mapRow(RowSet)}. Typically a prototype scoped bean so that a + * new instance is returned for each field set mapped. + * + * Either this property or the type property must be specified, but not both. + * @param name the name of a prototype bean in the enclosing BeanFactory + */ + public void setPrototypeBeanName(String name) { + this.name = name; + } + + /** + * Public setter for the type of bean to create instead of using a prototype bean. An + * object of this type will be created from its default constructor for every call to + * {@link #mapRow(RowSet)}.
+ * + * Either this property or the prototype bean name must be specified, but not both. + * @param type the type to set + */ + public void setTargetType(Class type) { + this.type = type; + } + + /** + * Check that precisely one of type or prototype bean name is specified. + * @throws IllegalStateException if neither is set or both properties are set. + * @see org.springframework.beans.factory.InitializingBean#afterPropertiesSet() + */ + @Override + public void afterPropertiesSet() throws Exception { + Assert.state(this.name != null || this.type != null, "Either name or type must be provided."); + Assert.state(this.name == null || this.type == null, "Both name and type cannot be specified together."); + } + + /** + * Map the {@link org.springframework.batch.item.file.transform.FieldSet} to an object + * retrieved from the enclosing Spring context, or to a new instance of the required + * type if no prototype is available. + * @throws org.springframework.validation.BindException if there is a type conversion + * or other error (if the {@link org.springframework.validation.DataBinder} from + * {@link #createBinder(Object)} has errors after binding). + * @throws org.springframework.beans.NotWritablePropertyException if the + * {@link org.springframework.batch.item.file.transform.FieldSet} contains a field + * that cannot be mapped to a bean property. + * @see org.springframework.batch.item.file.mapping.FieldSetMapper#mapFieldSet(org.springframework.batch.item.file.transform.FieldSet) + */ + @Override + public T mapRow(RowSet rs) throws BindException { + T copy = getBean(); + DataBinder binder = createBinder(copy); + binder.bind(new MutablePropertyValues(getBeanProperties(copy, rs.getProperties()))); + if (binder.getBindingResult().hasErrors()) { + throw new BindException(binder.getBindingResult()); + } + return copy; + } + + /** + * Create a binder for the target object. The binder will then be used to bind the + * properties form a field set into the target object. This implementation creates a + * new {@link DataBinder} and calls out to {@link #initBinder(DataBinder)} and + * {@link #registerCustomEditors(org.springframework.beans.PropertyEditorRegistry)}. + * @param target the object to bind to. + * @return a {@link DataBinder} that can be used to bind properties to the target. + */ + protected DataBinder createBinder(Object target) { + DataBinder binder = new DataBinder(target); + binder.setIgnoreUnknownFields(!this.strict); + initBinder(binder); + registerCustomEditors(binder); + return binder; + } + + /** + * Initialize a new binder instance. This hook allows customization of binder settings + * such as the {@link DataBinder#initDirectFieldAccess() direct field access}. Called + * by {@link #createBinder(Object)}. + *

+ * Note that registration of custom property editors can be done in + * {@link #registerCustomEditors(org.springframework.beans.PropertyEditorRegistry)}. + *

+ * @param binder new binder instance + * @see #createBinder(Object) + */ + protected void initBinder(DataBinder binder) { + } + + @SuppressWarnings("unchecked") + private T getBean() { + if (this.name != null) { + return (T) this.beanFactory.getBean(this.name); + } + return BeanUtils.instantiateClass(this.type); + } + + private Properties getBeanProperties(Object bean, Properties properties) { + + Class cls = bean.getClass(); + + // Map from field names to property names + DistanceHolder distanceKey = new DistanceHolder(cls, this.distanceLimit); + if (!this.propertiesMatched.containsKey(distanceKey)) { + this.propertiesMatched.putIfAbsent(distanceKey, new ConcurrentHashMap<>()); + } + Map matches = new HashMap<>(this.propertiesMatched.get(distanceKey)); + + @SuppressWarnings({ "unchecked", "rawtypes" }) + Set keys = new HashSet(properties.keySet()); + for (String key : keys) { + + if (matches.containsKey(key)) { + switchPropertyNames(properties, key, matches.get(key)); + continue; + } + + String name = findPropertyName(bean, key); + + if (name != null) { + if (matches.containsValue(name)) { + throw new NotWritablePropertyException(cls, name, "Duplicate match with distance <= " + + this.distanceLimit + " found for this property in input keys: " + keys + + ". (Consider reducing the distance limit or changing the input key names to get a closer match.)"); + } + matches.put(key, name); + switchPropertyNames(properties, key, name); + } + } + + this.propertiesMatched.replace(distanceKey, new ConcurrentHashMap<>(matches)); + return properties; + } + + private String findPropertyName(Object bean, String key) { + + if (bean == null) { + return null; + } + + Class cls = bean.getClass(); + + int index = PropertyAccessorUtils.getFirstNestedPropertySeparatorIndex(key); + String prefix; + String suffix; + + // If the property name is nested recurse down through the properties + // looking for a match. + if (index > 0) { + prefix = key.substring(0, index); + suffix = key.substring(index + 1); + String nestedName = findPropertyName(bean, prefix); + if (nestedName == null) { + return null; + } + + Object nestedValue = getPropertyValue(bean, nestedName); + String nestedPropertyName = findPropertyName(nestedValue, suffix); + return (nestedPropertyName != null) ? nestedName + "." + nestedPropertyName : null; + } + + String name = null; + int distance = 0; + index = key.indexOf(PropertyAccessor.PROPERTY_KEY_PREFIX_CHAR); + + if (index > 0) { + prefix = key.substring(0, index); + suffix = key.substring(index); + } + else { + prefix = key; + suffix = ""; + } + + while (name == null && distance <= this.distanceLimit) { + String[] candidates = PropertyMatches.forProperty(prefix, cls, distance).getPossibleMatches(); + // If we find precisely one match, then use that one... + if (candidates.length == 1) { + String candidate = candidates[0]; + if (candidate.equals(prefix)) { // if it's the same don't + // replace it... + name = key; + } + else { + name = candidate + suffix; + } + } + distance++; + } + return name; + } + + private Object getPropertyValue(Object bean, String nestedName) { + BeanWrapperImpl wrapper = new BeanWrapperImpl(bean); + wrapper.setAutoGrowNestedPaths(true); + + Object nestedValue = wrapper.getPropertyValue(nestedName); + if (nestedValue == null) { + nestedValue = BeanUtils.instantiateClass(wrapper.getPropertyType(nestedName)); + wrapper.setPropertyValue(nestedName, nestedValue); + } + return nestedValue; + } + + private void switchPropertyNames(Properties properties, String oldName, String newName) { + String value = properties.getProperty(oldName); + properties.remove(oldName); + properties.setProperty(newName, value); + } + + /** + * Public setter for the 'strict' property. If true, then {@link #mapRow(RowSet)} will + * fail if the RowSet contains fields that cannot be mapped to the bean. + * @param strict fail if non-mappable properties are found + */ + public void setStrict(boolean strict) { + this.strict = strict; + } + + private static class DistanceHolder { + + private final Class cls; + + private final int distance; + + DistanceHolder(Class cls, int distance) { + this.cls = cls; + this.distance = distance; + + } + + @Override + public boolean equals(Object obj) { + if (this == obj) { + return true; + } + if (obj == null) { + return false; + } + if (getClass() != obj.getClass()) { + return false; + } + DistanceHolder other = (DistanceHolder) obj; + if (this.cls == null) { + if (other.cls != null) { + return false; + } + } + else if (!this.cls.equals(other.cls)) { + return false; + } + return this.distance == other.distance; + } + + @Override + public int hashCode() { + final int prime = 31; + int result = 1; + result = prime * result + ((this.cls == null) ? 0 : this.cls.hashCode()); + result = prime * result + this.distance; + return result; + } + + + } + +} diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/mapping/PassThroughRowMapper.java b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/mapping/PassThroughRowMapper.java similarity index 58% rename from spring-batch-excel/src/main/java/org/springframework/batch/item/excel/mapping/PassThroughRowMapper.java rename to spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/mapping/PassThroughRowMapper.java index aba282f..195c172 100644 --- a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/mapping/PassThroughRowMapper.java +++ b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/mapping/PassThroughRowMapper.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2014 the original author or authors. + * Copyright 2006-2021 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * https://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -13,23 +13,24 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.springframework.batch.item.excel.mapping; -import org.springframework.batch.item.excel.RowMapper; -import org.springframework.batch.item.excel.support.rowset.RowSet; +package org.springframework.batch.extensions.excel.mapping; + +import org.springframework.batch.extensions.excel.RowMapper; +import org.springframework.batch.extensions.excel.support.rowset.RowSet; /** - * Pass through {@link RowMapper} useful for passing the orginal String[] - * back directly rather than a mapped object. + * Pass through {@link RowMapper} useful for passing the orginal {@code String[]} back + * directly rather than a mapped object. * * @author Marten Deinum - * @since 0.5.0 + * @since 0.1.0 */ public class PassThroughRowMapper implements RowMapper { - @Override - public String[] mapRow(final RowSet rs) throws Exception { - return rs.getCurrentRow(); - } + @Override + public String[] mapRow(final RowSet rs) throws Exception { + return rs.getCurrentRow(); + } } diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/mapping/PropertyMatches.java b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/mapping/PropertyMatches.java new file mode 100644 index 0000000..9490c51 --- /dev/null +++ b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/mapping/PropertyMatches.java @@ -0,0 +1,136 @@ +/* + * Copyright 2006-2021 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.extensions.excel.mapping; + +import java.beans.PropertyDescriptor; +import java.util.ArrayList; +import java.util.Collections; +import java.util.List; + +import org.springframework.beans.BeanUtils; +import org.springframework.util.StringUtils; + +/** + * Helper class for calculating bean property matches, according to. Used by + * BeanWrapperImpl to suggest alternatives for an invalid property name.
+ * + * Copied and slightly modified from Spring core, + * + * @author Alef Arendsen + * @author Arjen Poutsma + * @author Juergen Hoeller + * @author Dave Syer + * @since 0.1.0 + * @see #forProperty(String, Class, int) + */ +final class PropertyMatches { + + /** + * Create PropertyMatches for the given bean property. + * @param propertyName the name of the property to find possible matches for + * @param beanClass the bean class to search for matches + * @param maxDistance the maximum property distance allowed for matches + * @return the prepared {@code PropertyMatches} + */ + static PropertyMatches forProperty(String propertyName, Class beanClass, int maxDistance) { + return new PropertyMatches(propertyName, beanClass, maxDistance); + } + + private final String propertyName; + + private final String[] possibleMatches; + + /** + * Create a new PropertyMatches instance for the given property. + * @param propertyName the name of the property to find possible matches for + * @param beanClass the bean class to search for matches + * @param maxDistance the maximum property distance allowed for matches + */ + private PropertyMatches(String propertyName, Class beanClass, int maxDistance) { + this.propertyName = propertyName; + this.possibleMatches = calculateMatches(BeanUtils.getPropertyDescriptors(beanClass), maxDistance); + } + + String[] getPossibleMatches() { + return this.possibleMatches; + } + + /** + * Generate possible property alternatives for the given property and class. + * Internally uses the getStringDistance method, which in turn uses the + * Levenshtein algorithm to determine the distance between two Strings. + * @param propertyDescriptors the JavaBeans property descriptors to search + * @param maxDistance the maximum distance to accept + * @return the calculated matches + */ + private String[] calculateMatches(PropertyDescriptor[] propertyDescriptors, int maxDistance) { + List candidates = new ArrayList<>(); + for (PropertyDescriptor propertyDescriptor : propertyDescriptors) { + if (propertyDescriptor.getWriteMethod() != null) { + String possibleAlternative = propertyDescriptor.getName(); + int distance = calculateStringDistance(this.propertyName, possibleAlternative); + if (distance <= maxDistance) { + candidates.add(possibleAlternative); + } + } + } + Collections.sort(candidates); + return StringUtils.toStringArray(candidates); + } + + /** + * Calculate the distance between the given two Strings according to the Levenshtein + * algorithm. + * @param s1 the first String + * @param s2 the second String + * @return the distance value + */ + private int calculateStringDistance(String s1, String s2) { + if (s1.length() == 0) { + return s2.length(); + } + if (s2.length() == 0) { + return s1.length(); + } + int[][] d = new int[s1.length() + 1][s2.length() + 1]; + + for (int i = 0; i <= s1.length(); i++) { + d[i][0] = i; + } + for (int j = 0; j <= s2.length(); j++) { + d[0][j] = j; + } + + for (int i = 1; i <= s1.length(); i++) { + char s_i = s1.charAt(i - 1); + for (int j = 1; j <= s2.length(); j++) { + int cost; + char t_j = s2.charAt(j - 1); + if (Character.toLowerCase(s_i) == Character.toLowerCase(t_j)) { + cost = 0; + } + else { + cost = 1; + } + d[i][j] = Math.min(Math.min(d[i - 1][j] + 1, d[i][j - 1] + 1), d[i - 1][j - 1] + cost); + } + } + + return d[s1.length()][s2.length()]; + } + +} diff --git a/spring-batch-excel/src/test/java/org/springframework/batch/item/excel/jxl/JxlItemReaderTest.java b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/mapping/package-info.java similarity index 50% rename from spring-batch-excel/src/test/java/org/springframework/batch/item/excel/jxl/JxlItemReaderTest.java rename to spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/mapping/package-info.java index 50845f9..517d694 100644 --- a/spring-batch-excel/src/test/java/org/springframework/batch/item/excel/jxl/JxlItemReaderTest.java +++ b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/mapping/package-info.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2014 the original author or authors. + * Copyright 2006-2021 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * https://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -14,19 +14,7 @@ * limitations under the License. */ -package org.springframework.batch.item.excel.jxl; - -import org.springframework.batch.item.excel.AbstractExcelItemReader; -import org.springframework.batch.item.excel.AbstractExcelItemReaderTests; - /** - * Test + * Default {@code RowMapper} implementations. */ -public class JxlItemReaderTest extends AbstractExcelItemReaderTests { - - @Override - protected AbstractExcelItemReader createExcelItemReader() { - return new JxlItemReader(); - } - -} +package org.springframework.batch.extensions.excel.mapping; diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/package-info.java b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/package-info.java new file mode 100644 index 0000000..2fba7ef --- /dev/null +++ b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/package-info.java @@ -0,0 +1,20 @@ +/* + * Copyright 2006-2021 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * Core interfaces for reading Excel files. + */ +package org.springframework.batch.extensions.excel; diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/poi/PoiItemReader.java b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/poi/PoiItemReader.java new file mode 100644 index 0000000..0582b79 --- /dev/null +++ b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/poi/PoiItemReader.java @@ -0,0 +1,95 @@ +/* + * Copyright 2006-2021 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.extensions.excel.poi; + +import java.io.File; +import java.io.FileNotFoundException; +import java.io.InputStream; + +import org.apache.poi.ss.usermodel.Row; +import org.apache.poi.ss.usermodel.Workbook; +import org.apache.poi.ss.usermodel.WorkbookFactory; + +import org.springframework.batch.extensions.excel.AbstractExcelItemReader; +import org.springframework.batch.extensions.excel.Sheet; +import org.springframework.core.io.Resource; + +/** + * {@link org.springframework.batch.item.ItemReader} implementation which uses apache POI + * to read an Excel file. It will read the file sheet for sheet and row for row. It is + * based on the {@link org.springframework.batch.item.file.FlatFileItemReader} + * + * This class is not thread-safe. + * + * @param the type + * @author Marten Deinum + * @since 0.1.0 + */ +public class PoiItemReader extends AbstractExcelItemReader { + + private Workbook workbook; + + private InputStream inputStream; + + @Override + protected Sheet getSheet(final int sheet) { + return new PoiSheet(this.workbook.getSheetAt(sheet)); + } + + @Override + protected int getNumberOfSheets() { + return this.workbook.getNumberOfSheets(); + } + + @Override + protected void doClose() throws Exception { + super.doClose(); + if (this.inputStream != null) { + this.inputStream.close(); + this.inputStream = null; + } + + if (this.workbook != null) { + this.workbook.close(); + this.workbook = null; + } + } + + /** + * Open the underlying file using the {@code WorkbookFactory}. Prefer {@code File} + * based access over an {@code InputStream}. Using a file will use fewer resources + * compared to an input stream. The latter will need to cache the whole sheet + * in-memory. + * @param resource the {@code Resource} pointing to the Excel file. + * @param password the password for opening the file + * @throws Exception is thrown for any errors. + */ + @Override + protected void openExcelFile(final Resource resource, String password) throws Exception { + + try { + File file = resource.getFile(); + this.workbook = WorkbookFactory.create(file, password, false); + } + catch (FileNotFoundException ex) { + this.inputStream = resource.getInputStream(); + this.workbook = WorkbookFactory.create(this.inputStream, password); + } + this.workbook.setMissingCellPolicy(Row.MissingCellPolicy.CREATE_NULL_AS_BLANK); + } + +} diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/poi/PoiSheet.java b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/poi/PoiSheet.java new file mode 100644 index 0000000..d39eb7e --- /dev/null +++ b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/poi/PoiSheet.java @@ -0,0 +1,137 @@ +/* + * Copyright 2006-2021 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.extensions.excel.poi; + +import java.util.Iterator; +import java.util.LinkedList; +import java.util.List; + +import org.apache.poi.ss.usermodel.Cell; +import org.apache.poi.ss.usermodel.CellType; +import org.apache.poi.ss.usermodel.DataFormatter; +import org.apache.poi.ss.usermodel.FormulaEvaluator; +import org.apache.poi.ss.usermodel.Row; + +import org.springframework.batch.extensions.excel.Sheet; +import org.springframework.lang.Nullable; + +/** + * Sheet implementation for Apache POI. + * + * @author Marten Deinum + * @since 0.1.0 + */ +class PoiSheet implements Sheet { + + private final DataFormatter dataFormatter = new DataFormatter(); + + private final org.apache.poi.ss.usermodel.Sheet delegate; + + private final int numberOfRows; + + private final String name; + + private FormulaEvaluator evaluator; + + /** + * Constructor which takes the delegate sheet. + * @param delegate the apache POI sheet + */ + PoiSheet(final org.apache.poi.ss.usermodel.Sheet delegate) { + super(); + this.delegate = delegate; + this.numberOfRows = this.delegate.getLastRowNum() + 1; + this.name = this.delegate.getSheetName(); + } + + /** + * {@inheritDoc} + */ + @Override + public int getNumberOfRows() { + return this.numberOfRows; + } + + /** + * {@inheritDoc} + */ + @Override + public String getName() { + return this.name; + } + + /** + * {@inheritDoc} + */ + @Override + @Nullable + public String[] getRow(final int rowNumber) { + final Row row = this.delegate.getRow(rowNumber); + return map(row); + } + + @Nullable + private String[] map(Row row) { + if (row == null) { + return null; + } + final List cells = new LinkedList<>(); + final int numberOfColumns = row.getLastCellNum(); + + for (int i = 0; i < numberOfColumns; i++) { + Cell cell = row.getCell(i); + CellType cellType = cell.getCellType(); + if (cellType == CellType.FORMULA) { + cells.add(this.dataFormatter.formatCellValue(cell, getFormulaEvaluator())); + } + else { + cells.add(this.dataFormatter.formatCellValue(cell)); + } + } + return cells.toArray(new String[0]); + } + + /** + * Lazy getter for the {@code FormulaEvaluator}. Takes some time to create an + * instance, so if not necessary don't create it. + * @return the {@code FormulaEvaluator} + */ + private FormulaEvaluator getFormulaEvaluator() { + if (this.evaluator == null) { + this.evaluator = this.delegate.getWorkbook().getCreationHelper().createFormulaEvaluator(); + } + return this.evaluator; + } + + @Override + public Iterator iterator() { + return new Iterator() { + private final Iterator delegateIter = PoiSheet.this.delegate.iterator(); + + @Override + public boolean hasNext() { + return this.delegateIter.hasNext(); + } + + @Override + public String[] next() { + return map(this.delegateIter.next()); + } + }; + } + +} diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/poi/package-info.java b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/poi/package-info.java new file mode 100644 index 0000000..b90bbe5 --- /dev/null +++ b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/poi/package-info.java @@ -0,0 +1,20 @@ +/* + * Copyright 2006-2021 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * Support classes for the Apache POI library. + */ +package org.springframework.batch.extensions.excel.poi; diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/streaming/StreamingSheet.java b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/streaming/StreamingSheet.java new file mode 100644 index 0000000..0be89f9 --- /dev/null +++ b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/streaming/StreamingSheet.java @@ -0,0 +1,303 @@ +/* + * Copyright 2006-2021 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.extensions.excel.streaming; + +import java.io.Closeable; +import java.io.IOException; +import java.io.InputStream; +import java.util.Arrays; +import java.util.HashMap; +import java.util.Iterator; +import java.util.Map; + +import javax.xml.stream.XMLStreamConstants; +import javax.xml.stream.XMLStreamException; +import javax.xml.stream.XMLStreamReader; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.apache.poi.ss.util.CellRangeAddress; +import org.apache.poi.ss.util.CellReference; +import org.apache.poi.xssf.eventusermodel.XSSFSheetXMLHandler; +import org.apache.poi.xssf.model.SharedStrings; +import org.apache.poi.xssf.model.Styles; +import org.apache.poi.xssf.usermodel.XSSFComment; +import org.xml.sax.Attributes; + +import org.springframework.batch.extensions.excel.Sheet; +import org.springframework.util.StringUtils; +import org.springframework.util.xml.StaxUtils; + +class StreamingSheet implements Sheet, Iterable, Closeable { + + private final Log logger = LogFactory.getLog(StreamingSheet.class); + + private final String name; + + private final InputStream is; + + private final XMLStreamReader reader; + + private final ValueRetrievingContentsHandler contentHandler; + + private final XSSFSheetXMLHandler sheetHandler; + + private int rowCount; + + private int colCount; + + StreamingSheet(String name, InputStream is, SharedStrings sharedStrings, Styles styles) { + this.name = name; + this.is = is; + this.contentHandler = new ValueRetrievingContentsHandler(); + this.sheetHandler = new XSSFSheetXMLHandler(styles, sharedStrings, this.contentHandler, false); + + try { + this.reader = StaxUtils.createDefensiveInputFactory().createXMLStreamReader(is); + } + catch (Exception ex) { + throw new IllegalStateException(ex); + } + } + + @Override + public int getNumberOfRows() { + return this.rowCount; + } + + @Override + public String getName() { + return this.name; + } + + @Override + public String[] getRow(int rowNumber) { + throw new UnsupportedOperationException("Getting row by index not supported when streaming."); + } + + private String[] nextRow() { + try { + while (this.reader.hasNext()) { + int type = this.reader.next(); + if (type == XMLStreamConstants.START_DOCUMENT) { + this.sheetHandler.startDocument(); + } + else if (type == XMLStreamConstants.END_DOCUMENT) { + this.sheetHandler.endDocument(); + return null; + } + else if (type == XMLStreamConstants.CHARACTERS) { + this.sheetHandler.characters(this.reader.getTextCharacters(), this.reader.getTextStart(), this.reader.getTextLength()); + } + else if (type == XMLStreamConstants.START_ELEMENT) { + String localName = this.reader.getLocalName(); + if ("dimension".equals(localName)) { + String v = this.reader.getAttributeValue(null, "ref"); + if (v != null && v.indexOf(':') > -1) { + CellRangeAddress range = CellRangeAddress.valueOf(v); + int rowEnd = range.getLastRow(); + int rowStart = range.getFirstRow(); + this.rowCount = rowEnd - rowStart + 1; + + int colStart = range.getFirstColumn(); + int colEnd = range.getLastColumn(); + this.colCount = colEnd - colStart + 1; + } + } + else { + Attributes delegating = new AttributesAdapter(this.reader); + this.sheetHandler.startElement(null, localName, null, delegating); + } + } + else if (type == XMLStreamConstants.END_ELEMENT) { + String tag = this.reader.getLocalName(); + this.sheetHandler.endElement(null, tag, null); + if ("row".equals(tag)) { + if (this.logger.isTraceEnabled()) { + this.logger.trace("Row ended, returning: " + + StringUtils.arrayToCommaDelimitedString(this.contentHandler.getValues())); + } + return this.contentHandler.getValues(); + } + } + } + } + catch (Exception ex) { + throw new IllegalStateException("Error reading file.", ex); + } + return null; + } + + @Override + public void close() throws IOException { + try { + this.reader.close(); + } + catch (XMLStreamException ex) { + // Ignore exception we cannot recover + } + + this.is.close(); + } + + @Override + public Iterator iterator() { + return new Iterator() { + + private String[] currentRow; + + @Override + public boolean hasNext() { + this.currentRow = nextRow(); + return this.currentRow != null; + } + + @Override + public String[] next() { + return this.currentRow; + } + }; + } + + private class ValueRetrievingContentsHandler implements XSSFSheetXMLHandler.SheetContentsHandler { + + private final Log logger = LogFactory.getLog(ValueRetrievingContentsHandler.class); + + private String[] values; + + @Override + public void startRow(int rowNum) { + if (this.logger.isTraceEnabled()) { + this.logger.trace("Start processing row: " + rowNum); + } + // Prepare for this row + if (this.values == null) { + this.values = new String[StreamingSheet.this.colCount]; + } + Arrays.fill(this.values, ""); + } + + @Override + public void endRow(int rowNum) { + if (this.logger.isTraceEnabled()) { + this.logger.trace("End processing row: " + rowNum); + } + } + + @Override + public void cell(String cellReference, String formattedValue, XSSFComment comment) { + int col = new CellReference(cellReference).getCol(); + if (this.logger.isTraceEnabled()) { + this.logger.trace("Setting value (" + cellReference + ") = " + formattedValue); + } + // This can happen if the dimensions cannot be read properly but there are + // still rows. + // Create a copy of the existing array and append to it. + if (this.values.length <= col) { + String[] newValues = Arrays.copyOf(this.values, col + 1); + Arrays.setAll(newValues, (idx) -> (newValues[idx] != null) ? newValues[idx] : ""); + this.values = newValues; + } + this.values[col] = formattedValue; + } + + String[] getValues() { + return Arrays.copyOf(this.values, this.values.length); + } + + } + + /** + * Minimal adapter for {@code Attributes} so that it works with the + * {@code XSSFSheetXMLHandler}. Adapts an {@code XMLStreamReader} so that it can be + * used as an {@code org.xml.sax.Attributes} implementation. + */ + private static final class AttributesAdapter implements Attributes { + + private final Map attributes = new HashMap<>(); + + private AttributesAdapter(XMLStreamReader delegate) { + for (int i = 0; i < delegate.getAttributeCount(); i++) { + String name = delegate.getAttributeLocalName(i); + String value = delegate.getAttributeValue(i); + this.attributes.put(name, value); + } + } + + @Override + public int getLength() { + return this.attributes.size(); + } + + @Override + public String getURI(int index) { + return null; + } + + @Override + public String getLocalName(int index) { + return null; + } + + @Override + public String getQName(int index) { + return null; + } + + @Override + public String getType(int index) { + return null; + } + + @Override + public String getValue(int index) { + return null; + } + + @Override + public int getIndex(String uri, String localName) { + return 0; + } + + @Override + public int getIndex(String qName) { + return 0; + } + + @Override + public String getType(String uri, String localName) { + return null; + } + + @Override + public String getType(String qName) { + return null; + } + + @Override + public String getValue(String uri, String localName) { + return this.attributes.get(localName); + } + + @Override + public String getValue(String qName) { + return this.attributes.get(qName); + } + + } + +} diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/streaming/StreamingXlsxItemReader.java b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/streaming/StreamingXlsxItemReader.java new file mode 100644 index 0000000..444984d --- /dev/null +++ b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/streaming/StreamingXlsxItemReader.java @@ -0,0 +1,115 @@ +/* + * Copyright 2006-2021 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.extensions.excel.streaming; + +import java.io.File; +import java.io.FileNotFoundException; +import java.io.IOException; +import java.io.InputStream; +import java.util.ArrayList; +import java.util.List; + +import org.apache.poi.openxml4j.exceptions.InvalidFormatException; +import org.apache.poi.openxml4j.opc.OPCPackage; +import org.apache.poi.openxml4j.opc.PackageAccess; +import org.apache.poi.xssf.eventusermodel.ReadOnlySharedStringsTable; +import org.apache.poi.xssf.eventusermodel.XSSFReader; +import org.apache.poi.xssf.model.SharedStrings; +import org.apache.poi.xssf.model.Styles; +import org.xml.sax.SAXException; + +import org.springframework.batch.extensions.excel.AbstractExcelItemReader; +import org.springframework.batch.extensions.excel.Sheet; +import org.springframework.core.io.Resource; + +/** + * Simple streaming reader without Apache POI. + * + * @param the type + * @author Marten Deinum + * @since 0.1.0 + **/ +public class StreamingXlsxItemReader extends AbstractExcelItemReader { + + private final List sheets = new ArrayList<>(); + + private OPCPackage pkg; + + private InputStream inputStream; + + @Override + protected Sheet getSheet(int sheet) { + return this.sheets.get(sheet); + } + + @Override + protected int getNumberOfSheets() { + return this.sheets.size(); + } + + @Override + protected void openExcelFile(Resource resource, String password) throws Exception { + try { + File file = resource.getFile(); + this.pkg = OPCPackage.open(file, PackageAccess.READ); + } + catch (FileNotFoundException ex) { + this.inputStream = resource.getInputStream(); + this.pkg = OPCPackage.open(this.inputStream); + } + XSSFReader reader = new XSSFReader(this.pkg); + initSheets(reader, this.pkg); + } + + private void initSheets(XSSFReader reader, OPCPackage pkg) throws IOException, InvalidFormatException { + XSSFReader.SheetIterator iter = (XSSFReader.SheetIterator) reader.getSheetsData(); + SharedStrings sharedStrings; + try { + sharedStrings = new ReadOnlySharedStringsTable(pkg); + } + catch (SAXException ex) { + throw new IllegalStateException("Cannot read shared-strings-table.", ex); + } + Styles styles = reader.getStylesTable(); + while (iter.hasNext()) { + InputStream is = iter.next(); + String name = iter.getSheetName(); + this.sheets.add(new StreamingSheet(name, is, sharedStrings, styles)); + } + + if (this.logger.isTraceEnabled()) { + this.logger.trace("Prepared " + this.sheets.size() + " sheets."); + } + } + + @Override + protected void doClose() throws Exception { + this.pkg.revert(); + + for (StreamingSheet sheet : this.sheets) { + sheet.close(); + } + this.sheets.clear(); + + if (this.inputStream != null) { + this.inputStream.close(); + this.inputStream = null; + } + super.doClose(); + } + +} diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/streaming/package-info.java b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/streaming/package-info.java new file mode 100644 index 0000000..fcbad2e --- /dev/null +++ b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/streaming/package-info.java @@ -0,0 +1,20 @@ +/* + * Copyright 2006-2021 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * Streaming implementation of ItemReaders. + */ +package org.springframework.batch.extensions.excel.streaming; diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/support/rowset/ColumnNameExtractor.java b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/support/rowset/ColumnNameExtractor.java similarity index 60% rename from spring-batch-excel/src/main/java/org/springframework/batch/item/excel/support/rowset/ColumnNameExtractor.java rename to spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/support/rowset/ColumnNameExtractor.java index dea1eed..59bdd19 100644 --- a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/support/rowset/ColumnNameExtractor.java +++ b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/support/rowset/ColumnNameExtractor.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2015 the original author or authors. + * Copyright 2006-2021 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * https://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -13,24 +13,24 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.springframework.batch.item.excel.support.rowset; -import org.springframework.batch.item.excel.Sheet; +package org.springframework.batch.extensions.excel.support.rowset; + +import org.springframework.batch.extensions.excel.Sheet; /** * Contract for extracting column names for a given {@code sheet}. * * @author Marten Deinum - * @since 0.5.0 + * @since 0.1.0 */ public interface ColumnNameExtractor { - /** - * Retrieves the names of the columns in the given Sheet. - * - * @param sheet the sheet - * @return the column names - */ - String[] getColumnNames(Sheet sheet); + /** + * Retrieves the names of the columns in the given {@code Sheet}. + * @param sheet the sheet + * @return the column names + */ + String[] getColumnNames(Sheet sheet); } diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/support/rowset/DefaultRowSet.java b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/support/rowset/DefaultRowSet.java new file mode 100644 index 0000000..6aa2bd4 --- /dev/null +++ b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/support/rowset/DefaultRowSet.java @@ -0,0 +1,89 @@ +/* + * Copyright 2006-2021 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.extensions.excel.support.rowset; + +import java.util.Iterator; +import java.util.Properties; + +import org.springframework.batch.extensions.excel.Sheet; + +/** + * Default implementation of the {@code RowSet} interface. + * + * @author Marten Deinum + * @since 0.1.0 + * @see DefaultRowSetFactory + */ +public class DefaultRowSet implements RowSet { + + private final Iterator sheetData; + + private final RowSetMetaData metaData; + + private int currentRowIndex = -1; + + private String[] currentRow; + + DefaultRowSet(Sheet sheet, RowSetMetaData metaData) { + this.sheetData = sheet.iterator(); + this.metaData = metaData; + } + + @Override + public RowSetMetaData getMetaData() { + return this.metaData; + } + + @Override + public boolean next() { + this.currentRow = null; + this.currentRowIndex++; + if (this.sheetData.hasNext()) { + this.currentRow = this.sheetData.next(); + return true; + } + return false; + } + + @Override + public int getCurrentRowIndex() { + return this.currentRowIndex; + } + + @Override + public String[] getCurrentRow() { + return this.currentRow; + } + + @Override + public Properties getProperties() { + final String[] names = this.metaData.getColumnNames(); + if (names == null) { + throw new IllegalStateException("Cannot create properties without meta data"); + } + + Properties props = new Properties(); + for (int i = 0; i < this.currentRow.length; i++) { + String value = this.currentRow[i]; + if (value != null) { + props.setProperty(names[i], value); + } + } + return props; + } + +} diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/support/rowset/DefaultRowSetFactory.java b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/support/rowset/DefaultRowSetFactory.java new file mode 100644 index 0000000..c1e4992 --- /dev/null +++ b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/support/rowset/DefaultRowSetFactory.java @@ -0,0 +1,44 @@ +/* + * Copyright 2006-2021 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.extensions.excel.support.rowset; + +import org.springframework.batch.extensions.excel.Sheet; + +/** + * {@code RowSetFactory} implementation which constructs a {@code DefaultRowSet} instance + * and {@code DefaultRowSetMetaData} instance. The latter will have the + * {@code ColumnNameExtractor} configured on this factory set (default + * {@code RowNumberColumnNameExtractor}). + * + * @author Marten Deinum + * @since 0.1.0 + */ +public class DefaultRowSetFactory implements RowSetFactory { + + private ColumnNameExtractor columnNameExtractor = new RowNumberColumnNameExtractor(); + + @Override + public RowSet create(Sheet sheet) { + DefaultRowSetMetaData metaData = new DefaultRowSetMetaData(sheet, this.columnNameExtractor); + return new DefaultRowSet(sheet, metaData); + } + + public void setColumnNameExtractor(ColumnNameExtractor columnNameExtractor) { + this.columnNameExtractor = columnNameExtractor; + } + +} diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/support/rowset/DefaultRowSetMetaData.java b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/support/rowset/DefaultRowSetMetaData.java new file mode 100644 index 0000000..a505efa --- /dev/null +++ b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/support/rowset/DefaultRowSetMetaData.java @@ -0,0 +1,56 @@ +/* + * Copyright 2006-2021 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.extensions.excel.support.rowset; + +import org.springframework.batch.extensions.excel.Sheet; + +/** + * Default implementation for the {@code RowSetMetaData} interface. + * + * Requires a {@code Sheet} and {@code ColumnNameExtractor} to operate correctly. + * Delegates the retrieval of the column names to the {@code ColumnNameExtractor}. + * + * @author Marten Deinum + * @since 0.1.0 + */ +public class DefaultRowSetMetaData implements RowSetMetaData { + + private final Sheet sheet; + + private final ColumnNameExtractor columnNameExtractor; + + private String[] columnNames; + + DefaultRowSetMetaData(Sheet sheet, ColumnNameExtractor columnNameExtractor) { + this.sheet = sheet; + this.columnNameExtractor = columnNameExtractor; + } + + @Override + public String[] getColumnNames() { + if (this.columnNames == null) { + this.columnNames = this.columnNameExtractor.getColumnNames(this.sheet); + } + return this.columnNames; + } + + @Override + public String getSheetName() { + return this.sheet.getName(); + } + +} diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/support/rowset/RowNumberColumnNameExtractor.java b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/support/rowset/RowNumberColumnNameExtractor.java similarity index 55% rename from spring-batch-excel/src/main/java/org/springframework/batch/item/excel/support/rowset/RowNumberColumnNameExtractor.java rename to spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/support/rowset/RowNumberColumnNameExtractor.java index 8a61df6..d91d23f 100644 --- a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/support/rowset/RowNumberColumnNameExtractor.java +++ b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/support/rowset/RowNumberColumnNameExtractor.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2014 the original author or authors. + * Copyright 2006-2021 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * https://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -13,27 +13,29 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.springframework.batch.item.excel.support.rowset; -import org.springframework.batch.item.excel.Sheet; +package org.springframework.batch.extensions.excel.support.rowset; + +import org.springframework.batch.extensions.excel.Sheet; /** - * {@code ColumnNameExtractor} which returns the values of a given row (default is 0) as the column - * names. + * {@code ColumnNameExtractor} which returns the values of a given row (default is 0) as + * the column names. * * @author Marten Deinum - * @since 0.5.0 + * @since 0.1.0 */ public class RowNumberColumnNameExtractor implements ColumnNameExtractor { - private int headerRowNumber; + private int headerRowNumber; - @Override - public String[] getColumnNames(final Sheet sheet) { - return sheet.getRow(headerRowNumber); - } + @Override + public String[] getColumnNames(final Sheet sheet) { + return sheet.getRow(this.headerRowNumber); + } + + public void setHeaderRowNumber(int headerRowNumber) { + this.headerRowNumber = headerRowNumber; + } - public void setHeaderRowNumber(int headerRowNumber) { - this.headerRowNumber = headerRowNumber; - } } diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/support/rowset/RowSet.java b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/support/rowset/RowSet.java new file mode 100644 index 0000000..7394f9d --- /dev/null +++ b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/support/rowset/RowSet.java @@ -0,0 +1,63 @@ +/* + * Copyright 2006-2021 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.extensions.excel.support.rowset; + +import java.util.Properties; + +/** + * Used by the {@code org.springframework.batch.item.excel.AbstractExcelItemReader} to + * abstract away the complexities of the underlying Excel API implementations. + * + * @author Marten Deinum + * @since 0.1.0 + */ +public interface RowSet { + + /** + * Retrieves the meta data (name of the sheet, number of columns, names) of this row + * set. + * @return a corresponding {@code RowSetMetaData} instance. + */ + RowSetMetaData getMetaData(); + + /** + * Move to the next row in the document. + * @return {@code true} if the row is valid, {@code false} if there are no more rows + */ + boolean next(); + + /** + * Returns the current row number. + * @return the current row number + */ + int getCurrentRowIndex(); + + /** + * Return the current row as a {@code String[]}. + * @return the row as a {@code String[]} + */ + String[] getCurrentRow(); + + /** + * Construct name-value pairs from the column names and string values. {@code null} + * values are omitted. + * @return some properties representing the row set. + * @throws IllegalStateException if the column name meta data is not available. + */ + Properties getProperties(); + +} diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/support/rowset/RowSetFactory.java b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/support/rowset/RowSetFactory.java similarity index 62% rename from spring-batch-excel/src/main/java/org/springframework/batch/item/excel/support/rowset/RowSetFactory.java rename to spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/support/rowset/RowSetFactory.java index 7ff204c..29be3db 100644 --- a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/support/rowset/RowSetFactory.java +++ b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/support/rowset/RowSetFactory.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2014 the original author or authors. + * Copyright 2006-2021 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * https://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -13,23 +13,24 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.springframework.batch.item.excel.support.rowset; -import org.springframework.batch.item.excel.Sheet; +package org.springframework.batch.extensions.excel.support.rowset; + +import org.springframework.batch.extensions.excel.Sheet; /** * Contract for factories which will construct a {@code RowSet} implementation. * * @author Marten Deinum - * @since 0.5.0 + * @since 0.1.0 */ public interface RowSetFactory { - /** - * Create a rowset instance. - * - * @param sheet an Excel sheet. - * @return a RowSet instance. - */ - RowSet create(Sheet sheet); + /** + * Create a rowset instance. + * @param sheet an Excel sheet. + * @return a {@code RowSet} instance. + */ + RowSet create(Sheet sheet); + } diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/support/rowset/RowSetMetaData.java b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/support/rowset/RowSetMetaData.java new file mode 100644 index 0000000..396524a --- /dev/null +++ b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/support/rowset/RowSetMetaData.java @@ -0,0 +1,39 @@ +/* + * Copyright 2006-2021 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.extensions.excel.support.rowset; + +/** + * Interface representing the the metadata associated with an Excel document. + * + * @author Marten Deinum + * @since 0.1.0 + */ +public interface RowSetMetaData { + + /** + * Retrieves the names of the columns for the current sheet. + * @return the column names. + */ + String[] getColumnNames(); + + /** + * Retrieves the name of the sheet the RowSet is based on. + * @return the name of the sheet + */ + String getSheetName(); + +} diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/support/rowset/StaticColumnNameExtractor.java b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/support/rowset/StaticColumnNameExtractor.java similarity index 51% rename from spring-batch-excel/src/main/java/org/springframework/batch/item/excel/support/rowset/StaticColumnNameExtractor.java rename to spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/support/rowset/StaticColumnNameExtractor.java index 5fe898d..98eff69 100644 --- a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/support/rowset/StaticColumnNameExtractor.java +++ b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/support/rowset/StaticColumnNameExtractor.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2014 the original author or authors. + * Copyright 2006-2021 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * https://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -13,29 +13,32 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package org.springframework.batch.item.excel.support.rowset; -import org.springframework.batch.item.excel.Sheet; +package org.springframework.batch.extensions.excel.support.rowset; + +import java.util.Arrays; + +import org.springframework.batch.extensions.excel.Sheet; /** * {@code ColumnNameExtractor} implementation which returns a preset String[] to use as - * the column names. Useful for those situations in which an Excel file without a header row - * is read + * the column names. Useful for those situations in which an Excel file without a header + * row is read * - * @author Marten Deinum - * @since 0.5.0 + * @author Marten Deinum + * @since 0.1.0 */ public class StaticColumnNameExtractor implements ColumnNameExtractor { - private final String[] columnNames; + private final String[] columnNames; - public StaticColumnNameExtractor(String[] columnNames) { - this.columnNames = columnNames; - } + public StaticColumnNameExtractor(String[] columnNames) { + this.columnNames = columnNames; + } - @Override - public String[] getColumnNames(Sheet sheet) { - return this.columnNames; - } + @Override + public String[] getColumnNames(Sheet sheet) { + return Arrays.copyOf(this.columnNames, this.columnNames.length); + } } diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/RowCallbackHandler.java b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/support/rowset/package-info.java similarity index 56% rename from spring-batch-excel/src/main/java/org/springframework/batch/item/excel/RowCallbackHandler.java rename to spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/support/rowset/package-info.java index 51f60a1..07dbb81 100644 --- a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/RowCallbackHandler.java +++ b/spring-batch-excel/src/main/java/org/springframework/batch/extensions/excel/support/rowset/package-info.java @@ -1,11 +1,11 @@ /* - * Copyright 2006-2014 the original author or authors. + * Copyright 2006-2021 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * https://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -14,18 +14,7 @@ * limitations under the License. */ -package org.springframework.batch.item.excel; - -import org.springframework.batch.item.excel.support.rowset.RowSet; - /** - * Callback to handle skipped lines. Useful for header/footer processing. - * - * @author Marten Deinum - * @since 0.5.0 + * RowSet abstraction for Excel documents. */ -public interface RowCallbackHandler { - - void handleRow(RowSet rs); - -} +package org.springframework.batch.extensions.excel.support.rowset; diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/AbstractExcelItemReader.java b/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/AbstractExcelItemReader.java deleted file mode 100644 index 0c43bc7..0000000 --- a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/AbstractExcelItemReader.java +++ /dev/null @@ -1,218 +0,0 @@ -/* - * Copyright 2006-2014 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.springframework.batch.item.excel; - -import org.apache.commons.logging.Log; -import org.apache.commons.logging.LogFactory; -import org.springframework.batch.item.excel.support.rowset.DefaultRowSetFactory; -import org.springframework.batch.item.excel.support.rowset.RowSet; -import org.springframework.batch.item.excel.support.rowset.RowSetFactory; -import org.springframework.batch.item.file.ResourceAwareItemReaderItemStream; -import org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader; -import org.springframework.beans.factory.InitializingBean; -import org.springframework.core.io.Resource; -import org.springframework.util.Assert; -import org.springframework.util.ClassUtils; - -/** - * {@link org.springframework.batch.item.ItemReader} implementation to read an Excel - * file. It will read the file sheet for sheet and row for row. It is loosy based on - * the {@link org.springframework.batch.item.file.FlatFileItemReader} - * - * @param the type - * @author Marten Deinum - * @since 0.5.0 - */ -public abstract class AbstractExcelItemReader extends AbstractItemCountingItemStreamItemReader implements - ResourceAwareItemReaderItemStream, InitializingBean { - - protected final Log logger = LogFactory.getLog(getClass()); - private Resource resource; - private int linesToSkip = 0; - private int currentSheet = 0; - private RowMapper rowMapper; - private RowCallbackHandler skippedRowsCallback; - private boolean noInput = false; - private boolean strict = true; - private RowSetFactory rowSetFactory = new DefaultRowSetFactory(); - private RowSet rs; - - public AbstractExcelItemReader() { - super(); - this.setName(ClassUtils.getShortName(this.getClass())); - } - - /** - * @return string corresponding to logical record according to - * {@link #setRowMapper(RowMapper)} (might span multiple rows in file). - */ - @Override - protected T doRead() throws Exception { - if (this.noInput || this.rs == null) { - return null; - } - - if (rs.next()) { - try { - return this.rowMapper.mapRow(rs); - } catch (final Exception e) { - throw new ExcelFileParseException("Exception parsing Excel file.", e, this.resource.getDescription(), - rs.getMetaData().getSheetName(), rs.getCurrentRowIndex(), rs.getCurrentRow()); - } - } else { - this.currentSheet++; - if (this.currentSheet >= this.getNumberOfSheets()) { - if (logger.isDebugEnabled()) { - logger.debug("No more sheets in '" + this.resource.getDescription() + "'."); - } - return null; - } else { - this.openSheet(); - return this.doRead(); - } - } - } - - @Override - protected void doOpen() throws Exception { - Assert.notNull(this.resource, "Input resource must be set"); - this.noInput = true; - if (!this.resource.exists()) { - if (this.strict) { - throw new IllegalStateException("Input resource must exist (reader is in 'strict' mode): " - + this.resource); - } - logger.warn("Input resource does not exist '" + this.resource.getDescription() + "'."); - return; - } - - if (!this.resource.isReadable()) { - if (this.strict) { - throw new IllegalStateException("Input resource must be readable (reader is in 'strict' mode): " - + this.resource); - } - logger.warn("Input resource is not readable '" + this.resource.getDescription() + "'."); - return; - } - - this.openExcelFile(this.resource); - this.openSheet(); - this.noInput = false; - if (logger.isDebugEnabled()) { - logger.debug("Opened workbook [" + this.resource.getFilename() + "] with " + this.getNumberOfSheets() + " sheets."); - } - } - - private void openSheet() { - final Sheet sheet = this.getSheet(this.currentSheet); - this.rs =rowSetFactory.create(sheet); - - - if (logger.isDebugEnabled()) { - logger.debug("Opening sheet " + sheet.getName() + "."); - } - - for (int i = 0; i < this.linesToSkip; i++) { - if (rs.next() && this.skippedRowsCallback != null) { - this.skippedRowsCallback.handleRow(rs); - } - } - if (logger.isDebugEnabled()) { - logger.debug("Openend sheet " + sheet.getName() + ", with " + sheet.getNumberOfRows() + " rows."); - } - - } - - /** - * Public setter for the input resource. - * - * @param resource the {@code Resource} pointing to the Excelfile - */ - public void setResource(final Resource resource) { - this.resource = resource; - } - - public void afterPropertiesSet() throws Exception { - Assert.notNull(this.rowMapper, "RowMapper must be set"); - } - - /** - * Set the number of lines to skip. This number is applied to all worksheet - * in the excel file! default to 0 - * - * @param linesToSkip number of lines to skip - */ - public void setLinesToSkip(final int linesToSkip) { - this.linesToSkip = linesToSkip; - } - - /** - * - * @param sheet the sheet index - * @return the sheet or null when no sheet available. - */ - protected abstract Sheet getSheet(int sheet); - - /** - * The number of sheets in the underlying workbook. - * - * @return the number of sheets. - */ - protected abstract int getNumberOfSheets(); - - /** - * - * @param resource {@code Resource} pointing to the Excel file to read - * @throws Exception when the Excel sheet cannot be accessed - */ - protected abstract void openExcelFile(Resource resource) throws Exception; - - /** - * In strict mode the reader will throw an exception on - * {@link #open(org.springframework.batch.item.ExecutionContext)} if the input resource does not exist. - * - * @param strict true by default - */ - public void setStrict(final boolean strict) { - this.strict = strict; - } - - /** - * Public setter for the {@code rowMapper}. Used to map a row read from the underlying Excel workbook. - * - * @param rowMapper the {@code RowMapper} to use. - */ - public void setRowMapper(final RowMapper rowMapper) { - this.rowMapper = rowMapper; - } - - /** - * Public setter for the rowSetFactory. Used to create a {@code RowSet} implemenation. By default the - * {@code DefaultRowSetFactory} is used. - * - * @param rowSetFactory the {@code RowSetFactory} to use. - */ - public void setRowSetFactory(RowSetFactory rowSetFactory) { - this.rowSetFactory = rowSetFactory; - } - - /** - * @param skippedRowsCallback will be called for each one of the initial skipped lines before any items are read. - */ - public void setSkippedRowsCallback(final RowCallbackHandler skippedRowsCallback) { - this.skippedRowsCallback = skippedRowsCallback; - } -} diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/ExcelFileParseException.java b/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/ExcelFileParseException.java deleted file mode 100644 index 8207910..0000000 --- a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/ExcelFileParseException.java +++ /dev/null @@ -1,70 +0,0 @@ -/* - * Copyright 2006-2014 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.springframework.batch.item.excel; - -import org.springframework.batch.item.ParseException; - -/** - * Exception thrown when parsing excel files. The name of the sheet, the row number on that sheet and the - * name of the excel file can be passed in so that in exception handling we can reuse it. This class only has - * simply dependencies to make it is generic as possible. - * - * @author Marten Deinum - * @since 0.5.0 - */ -public class ExcelFileParseException extends ParseException { - - private final String filename; - private final String sheet; - private final String[] row; - private final int rowNumber; - - /** - * Construct an {@link ExcelFileParseException}. - * - * @param message the message - * @param cause the root cause - * @param filename the name of the excel file - * @param sheet the name of the sheet - * @param rowNumber the row number in the current sheet - * @param row the row data as text - */ - public ExcelFileParseException(final String message, final Throwable cause, final String filename, - final String sheet, final int rowNumber, final String[] row) { - super(message, cause); - this.filename = filename; - this.sheet = sheet; - this.rowNumber = rowNumber; - this.row = row; - } - - public String getFilename() { - return this.filename; - } - - public String getSheet() { - return this.sheet; - } - - public int getRowNumber() { - return this.rowNumber; - } - - public String[] getRow() { - return this.row; - } - -} diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/Sheet.java b/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/Sheet.java deleted file mode 100644 index 503392f..0000000 --- a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/Sheet.java +++ /dev/null @@ -1,55 +0,0 @@ -/* - * Copyright 2006-2014 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.springframework.batch.item.excel; - -/** - * Interface to wrap different Excel implementations like JExcel, JXL or Apache POI. - * - * @author Marten Deinum - * @since 0.5.0 - */ -public interface Sheet { - - /** - * Get the number of rows in this sheet. - * - * @return the number of rows. - */ - int getNumberOfRows(); - - /** - * Get the name of the sheet. - * - * @return the name of the sheet. - */ - String getName(); - - /** - * Get the row as a String[]. Returns null if the row doesn't exist. - * - * @param rowNumber the row number to read. - * @return a String[] or null - */ - String[] getRow(int rowNumber); - - /** - * The number of columns in this sheet. - * - * @return number of columns - */ - int getNumberOfColumns(); -} diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/jxl/JxlItemReader.java b/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/jxl/JxlItemReader.java deleted file mode 100644 index 95e1e88..0000000 --- a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/jxl/JxlItemReader.java +++ /dev/null @@ -1,76 +0,0 @@ - -/* - * Copyright 2006-2014 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.springframework.batch.item.excel.jxl; - -import jxl.Workbook; -import jxl.read.biff.WorkbookParser; -import org.springframework.batch.item.excel.AbstractExcelItemReader; -import org.springframework.batch.item.excel.Sheet; -import org.springframework.core.io.Resource; -import org.springframework.util.ClassUtils; - -/** - * {@link org.springframework.batch.item.ItemReader} implementation which uses the JExcelApi to read an Excel - * file. It will read the file sheet for sheet and row for row. It is based on - * the {@link org.springframework.batch.item.file.FlatFileItemReader} - * - * @param the type - * @author Marten Deinum - * @since 0.5.0 - * - * @deprecated since JExcelAPI is an abandoned project (no release since 2009, with serious bugs remaining) - */ -@Deprecated -public class JxlItemReader extends AbstractExcelItemReader { - - private Workbook workbook; - - public JxlItemReader() { - super(); - this.setName(ClassUtils.getShortName(JxlItemReader.class)); - } - - @Override - protected void openExcelFile(final Resource resource) throws Exception { - this.workbook = WorkbookParser.getWorkbook(resource.getInputStream()); - } - - @Override - protected void doClose() throws Exception { - if (this.workbook != null) { - this.workbook.close(); - } - } - - @Override - protected Sheet getSheet(final int sheet) { - if (sheet < this.workbook.getNumberOfSheets()) { - return new JxlSheet(this.workbook.getSheet(sheet)); - } - return null; - } - - @Override - protected int getNumberOfSheets() { - if (this.workbook == null) { - throw new IllegalStateException("Workbook file not ready for reading!"); - } - return this.workbook.getNumberOfSheets(); - } - -} diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/jxl/JxlSheet.java b/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/jxl/JxlSheet.java deleted file mode 100644 index 994e477..0000000 --- a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/jxl/JxlSheet.java +++ /dev/null @@ -1,88 +0,0 @@ - -/* - * Copyright 2006-2014 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.springframework.batch.item.excel.jxl; - -import jxl.Cell; -import org.springframework.batch.item.excel.Sheet; - -/** - * {@link org.springframework.batch.item.excel.Sheet} implementation for JXL. - * - * @author Marten Deinum - * @since 0.5.0 - * @deprecated since JExcelAPI is an abandoned project (no release since 2009, with serious bugs remaining) - */ -@Deprecated -public class JxlSheet implements Sheet { - - private final jxl.Sheet delegate; - private final int numberOfRows; - private final int numberOfColumns; - private final String name; - - /** - * Constructor which takes the delegate sheet. - * - * @param delegate the JXL sheet - */ - JxlSheet(final jxl.Sheet delegate) { - super(); - this.delegate = delegate; - this.numberOfRows = this.delegate.getRows(); - this.numberOfColumns = this.delegate.getNumberOfImages(); - this.name=this.delegate.getName(); - } - - /** - * {@inheritDoc} - */ - @Override - public int getNumberOfRows() { - return this.numberOfRows; - } - - /** - * {@inheritDoc} - */ - @Override - public String[] getRow(final int rowNumber) { - if (rowNumber < getNumberOfRows()) { - final Cell[] row = this.delegate.getRow(rowNumber); - return JxlUtils.extractContents(row); - } else { - return null; - } - } - - /** - * {@inheritDoc} - */ - @Override - public String getName() { - return this.name; - } - - /** - * {@inheritDoc} - */ - @Override - public int getNumberOfColumns() { - return this.numberOfColumns; - } - -} diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/jxl/JxlUtils.java b/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/jxl/JxlUtils.java deleted file mode 100644 index 0b8d190..0000000 --- a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/jxl/JxlUtils.java +++ /dev/null @@ -1,102 +0,0 @@ -/* - * Copyright 2006-2014 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.springframework.batch.item.excel.jxl; - -import jxl.Cell; -import jxl.Workbook; -import org.springframework.util.ObjectUtils; -import org.springframework.util.StringUtils; - -import java.util.ArrayList; -import java.util.List; - -/** - * Class containing utility methods to work with JXL. - * - * @author Marten Deinum - * @since 0.5.0 - * @deprecated since JExcelAPI is an abandoned project (no release since 2009, with serious bugs remaining) - */ -@Deprecated -public final class JxlUtils { - - /** - * Private constructor to prevent easy instantiation. - */ - private JxlUtils() { - } - - /** - * Checks if the given cell is emtpy. The cell is empty if it contains no characters, it will trim spaces. - * - * @param cell to check - * @return true/false - * @see org.springframework.util.StringUtils#hasText(String) - */ - public static boolean isEmpty(final Cell cell) { - return cell == null || !StringUtils.hasText(cell.getContents()); - } - - /** - * Check if the given row (Cell[]) is empty. It is considered empty when the row is null, the array is empty or all - * the cells in the row are empty. - * - * @param row to check - * @return true/false - */ - public static boolean isEmpty(final Cell[] row) { - if (ObjectUtils.isEmpty(row)) { - return true; - } - for (final Cell cell : row) { - if (!isEmpty(cell)) { - return false; - } - } - return true; - } - - /** - * Check if the given workbook has any sheets. - * - * @param workbook to check - * @return true/false - */ - public static boolean hasSheets(final Workbook workbook) { - return workbook != null && workbook.getNumberOfSheets() > 0; - } - - /** - * Extract the content from the given row. - * - * @param row the row - * @return the content as String[] - */ - public static String[] extractContents(final Cell[] row) { - final List values = new ArrayList(); - if (!ObjectUtils.isEmpty(row)) { - for (final Cell cell : row) { - if (!isEmpty(cell)) { - values.add(cell.getColumn(), cell.getContents()); - } else { - values.add(cell.getColumn(), ""); - } - } - } - return values.toArray(new String[values.size()]); - } -} diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/jxl/package-info.java b/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/jxl/package-info.java deleted file mode 100644 index c4a8b23..0000000 --- a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/jxl/package-info.java +++ /dev/null @@ -1,4 +0,0 @@ -/** - * Support classes for the JExcel library. - */ -package org.springframework.batch.item.excel.jxl; \ No newline at end of file diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/mapping/BeanWrapperRowMapper.java b/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/mapping/BeanWrapperRowMapper.java deleted file mode 100644 index 1b76ac9..0000000 --- a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/mapping/BeanWrapperRowMapper.java +++ /dev/null @@ -1,396 +0,0 @@ -package org.springframework.batch.item.excel.mapping; - -import org.springframework.batch.item.excel.RowMapper; -import org.springframework.batch.item.excel.support.rowset.RowSet; -import org.springframework.batch.support.DefaultPropertyEditorRegistrar; -import org.springframework.beans.*; -import org.springframework.beans.factory.BeanFactory; -import org.springframework.beans.factory.BeanFactoryAware; -import org.springframework.beans.factory.InitializingBean; -import org.springframework.util.Assert; -import org.springframework.util.ReflectionUtils; -import org.springframework.validation.BindException; -import org.springframework.validation.DataBinder; - -import java.util.*; -import java.util.concurrent.ConcurrentHashMap; -import java.util.concurrent.ConcurrentMap; - -/** - * {@link RowMapper} implementation based on bean property paths. The - * {@link RowSet} to be mapped should have field name meta data corresponding - * to bean property paths in an instance of the desired type. The instance is - * created and initialized either by referring to to a prototype object by bean - * name in the enclosing BeanFactory, or by providing a class to instantiate - * reflectively.
- *
- * - * Nested property paths, including indexed properties in maps and collections, - * can be referenced by the {@link RowSet}names. They will be converted to - * nested bean properties inside the prototype. The {@link RowSet} and the - * prototype are thus tightly coupled by the fields that are available and those - * that can be initialized. If some of the nested properties are optional (e.g. - * collection members) they need to be removed by a post processor.
- *
- * - * To customize the way that {@link RowSet} values are converted to the - * desired type for injecting into the prototype there are several choices. You - * can inject {@link java.beans.PropertyEditor} instances directly through the - * {@link #setCustomEditors(Map) customEditors} property, or you can override - * the {@link #createBinder(Object)} and {@link #initBinder(DataBinder)} - * methods, or you can provide a custom {@link RowSet} implementation.
- *
- * - * Property name matching is "fuzzy" in the sense that it tolerates close - * matches, as long as the match is unique. For instance: - * - *
    - *
  • Quantity = quantity (field names can be capitalised)
  • - *
  • ISIN = isin (acronyms can be lower case bean property names, as per Java - * Beans recommendations)
  • - *
  • DuckPate = duckPate (capitalisation including camel casing)
  • - *
  • ITEM_ID = itemId (capitalisation and replacing word boundary with - * underscore)
  • - *
  • ORDER.CUSTOMER_ID = order.customerId (nested paths are recursively - * checked)
  • - *
- * - * The algorithm used to match a property name is to start with an exact match - * and then search successively through more distant matches until precisely one - * match is found. If more than one match is found there will be an error. - * - * - * @author Marten Deinum - * @since 0.5.0 - */ -public class BeanWrapperRowMapper extends DefaultPropertyEditorRegistrar implements RowMapper, BeanFactoryAware, InitializingBean { - - private String name; - - private Class type; - - private BeanFactory beanFactory; - - private ConcurrentMap> propertiesMatched = new ConcurrentHashMap>(); - - private int distanceLimit = 5; - - private boolean strict = true; - - /* - * (non-Javadoc) - * - * @see - * org.springframework.beans.factory.BeanFactoryAware#setBeanFactory(org - * .springframework.beans.factory.BeanFactory) - */ - @Override - public void setBeanFactory(BeanFactory beanFactory) { - this.beanFactory = beanFactory; - } - - /** - * The maximum difference that can be tolerated in spelling between input - * key names and bean property names. Defaults to 5, but could be set lower - * if the field names match the bean names. - * - * @param distanceLimit the distance limit to set - */ - public void setDistanceLimit(int distanceLimit) { - this.distanceLimit = distanceLimit; - } - - /** - * The bean name (id) for an object that can be populated from the field set - * that will be passed into {@link #mapRow(org.springframework.batch.item.excel.support.rowset.RowSet)}. Typically a - * prototype scoped bean so that a new instance is returned for each field - * set mapped. - * - * Either this property or the type property must be specified, but not - * both. - * - * @param name the name of a prototype bean in the enclosing BeanFactory - */ - public void setPrototypeBeanName(String name) { - this.name = name; - } - - /** - * Public setter for the type of bean to create instead of using a prototype - * bean. An object of this type will be created from its default constructor - * for every call to {@link #mapRow(org.springframework.batch.item.excel.support.rowset.RowSet)}.
- * - * Either this property or the prototype bean name must be specified, but - * not both. - * - * @param type the type to set - */ - public void setTargetType(Class type) { - this.type = type; - } - - /** - * Check that precisely one of type or prototype bean name is specified. - * - * @throws IllegalStateException if neither is set or both properties are - * set. - * @see org.springframework.beans.factory.InitializingBean#afterPropertiesSet() - */ - @Override - public void afterPropertiesSet() throws Exception { - Assert.state(name != null || type != null, "Either name or type must be provided."); - Assert.state(name == null || type == null, "Both name and type cannot be specified together."); - } - - /** - * Map the {@link org.springframework.batch.item.file.transform.FieldSet} to an object retrieved from the enclosing Spring - * context, or to a new instance of the required type if no prototype is - * available. - * - * @throws org.springframework.validation.BindException if there is a type conversion or other error (if - * the {@link org.springframework.validation.DataBinder} from {@link #createBinder(Object)} has errors - * after binding). - * @throws org.springframework.beans.NotWritablePropertyException if the {@link org.springframework.batch.item.file.transform.FieldSet} contains a - * field that cannot be mapped to a bean property. - * @see org.springframework.batch.item.file.mapping.FieldSetMapper#mapFieldSet(org.springframework.batch.item.file.transform.FieldSet) - */ - @Override - public T mapRow(RowSet rs) throws BindException { - T copy = getBean(); - DataBinder binder = createBinder(copy); - binder.bind(new MutablePropertyValues(getBeanProperties(copy, rs.getProperties()))); - if (binder.getBindingResult().hasErrors()) { - throw new BindException(binder.getBindingResult()); - } - return copy; - } - - /** - * Create a binder for the target object. The binder will then be used to - * bind the properties form a field set into the target object. This - * implementation creates a new {@link DataBinder} and calls out to - * {@link #initBinder(DataBinder)} and - * {@link #registerCustomEditors(org.springframework.beans.PropertyEditorRegistry)}. - * - * @param target the object to bind to. - * @return a {@link DataBinder} that can be used to bind properties to the - * target. - */ - protected DataBinder createBinder(Object target) { - DataBinder binder = new DataBinder(target); - binder.setIgnoreUnknownFields(!this.strict); - initBinder(binder); - registerCustomEditors(binder); - return binder; - } - - /** - * Initialize a new binder instance. This hook allows customization of - * binder settings such as the {@link DataBinder#initDirectFieldAccess() - * direct field access}. Called by {@link #createBinder(Object)}. - *

- * Note that registration of custom property editors can be done in - * {@link #registerCustomEditors(org.springframework.beans.PropertyEditorRegistry)}. - *

- * - * @param binder new binder instance - * @see #createBinder(Object) - */ - protected void initBinder(DataBinder binder) { - } - - @SuppressWarnings("unchecked") - private T getBean() { - if (name != null) { - return (T) beanFactory.getBean(name); - } - try { - return type.newInstance(); - } catch (InstantiationException e) { - ReflectionUtils.handleReflectionException(e); - } catch (IllegalAccessException e) { - ReflectionUtils.handleReflectionException(e); - } - // should not happen - throw new IllegalStateException("Internal error: could not create bean instance for mapping."); - } - - /** - * @param bean - * @param properties - * @return - */ - private Properties getBeanProperties(Object bean, Properties properties) { - - Class cls = bean.getClass(); - - // Map from field names to property names - DistanceHolder distanceKey = new DistanceHolder(cls, distanceLimit); - if (!propertiesMatched.containsKey(distanceKey)) { - propertiesMatched.putIfAbsent(distanceKey, new ConcurrentHashMap()); - } - Map matches = new HashMap(propertiesMatched.get(distanceKey)); - - @SuppressWarnings({"unchecked", "rawtypes"}) - Set keys = new HashSet(properties.keySet()); - for (String key : keys) { - - if (matches.containsKey(key)) { - switchPropertyNames(properties, key, matches.get(key)); - continue; - } - - String name = findPropertyName(bean, key); - - if (name != null) { - if (matches.containsValue(name)) { - throw new NotWritablePropertyException( - cls, - name, - "Duplicate match with distance <= " - + distanceLimit - + " found for this property in input keys: " - + keys - + ". (Consider reducing the distance limit or changing the input key names to get a closer match.)"); - } - matches.put(key, name); - switchPropertyNames(properties, key, name); - } - } - - propertiesMatched.replace(distanceKey, new ConcurrentHashMap(matches)); - return properties; - } - - private String findPropertyName(Object bean, String key) { - - if (bean == null) { - return null; - } - - Class cls = bean.getClass(); - - int index = PropertyAccessorUtils.getFirstNestedPropertySeparatorIndex(key); - String prefix; - String suffix; - - // If the property name is nested recurse down through the properties - // looking for a match. - if (index > 0) { - prefix = key.substring(0, index); - suffix = key.substring(index + 1, key.length()); - String nestedName = findPropertyName(bean, prefix); - if (nestedName == null) { - return null; - } - - Object nestedValue = getPropertyValue(bean, nestedName); - String nestedPropertyName = findPropertyName(nestedValue, suffix); - return nestedPropertyName == null ? null : nestedName + "." + nestedPropertyName; - } - - String name = null; - int distance = 0; - index = key.indexOf(PropertyAccessor.PROPERTY_KEY_PREFIX_CHAR); - - if (index > 0) { - prefix = key.substring(0, index); - suffix = key.substring(index); - } else { - prefix = key; - suffix = ""; - } - - while (name == null && distance <= distanceLimit) { - String[] candidates = PropertyMatches.forProperty(prefix, cls, distance).getPossibleMatches(); - // If we find precisely one match, then use that one... - if (candidates.length == 1) { - String candidate = candidates[0]; - if (candidate.equals(prefix)) { // if it's the same don't - // replace it... - name = key; - } else { - name = candidate + suffix; - } - } - distance++; - } - return name; - } - - private Object getPropertyValue(Object bean, String nestedName) { - BeanWrapperImpl wrapper = new BeanWrapperImpl(bean); - wrapper.setAutoGrowNestedPaths(true); - - Object nestedValue = wrapper.getPropertyValue(nestedName); - if (nestedValue == null) { - try { - nestedValue = wrapper.getPropertyType(nestedName).newInstance(); - wrapper.setPropertyValue(nestedName, nestedValue); - } catch (InstantiationException e) { - ReflectionUtils.handleReflectionException(e); - } catch (IllegalAccessException e) { - ReflectionUtils.handleReflectionException(e); - } - } - return nestedValue; - } - - private void switchPropertyNames(Properties properties, String oldName, String newName) { - String value = properties.getProperty(oldName); - properties.remove(oldName); - properties.setProperty(newName, value); - } - - /** - * Public setter for the 'strict' property. If true, then - * {@link #mapRow(RowSet)} will fail if the RowSet contains fields - * that cannot be mapped to the bean. - * - * @param strict fail if non-mappable properties are found - */ - public void setStrict(boolean strict) { - this.strict = strict; - } - - - private static class DistanceHolder { - private final Class cls; - - private final int distance; - - public DistanceHolder(Class cls, int distance) { - this.cls = cls; - this.distance = distance; - - } - - @Override - public int hashCode() { - final int prime = 31; - int result = 1; - result = prime * result + ((cls == null) ? 0 : cls.hashCode()); - result = prime * result + distance; - return result; - } - - @Override - public boolean equals(Object obj) { - if (this == obj) - return true; - if (obj == null) - return false; - if (getClass() != obj.getClass()) - return false; - DistanceHolder other = (DistanceHolder) obj; - if (cls == null) { - if (other.cls != null) - return false; - } else if (!cls.equals(other.cls)) - return false; - if (distance != other.distance) - return false; - return true; - } - } -} diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/mapping/PropertyMatches.java b/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/mapping/PropertyMatches.java deleted file mode 100644 index d1e947d..0000000 --- a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/mapping/PropertyMatches.java +++ /dev/null @@ -1,175 +0,0 @@ -package org.springframework.batch.item.excel.mapping; - -import org.springframework.beans.BeanUtils; -import org.springframework.util.ObjectUtils; -import org.springframework.util.StringUtils; - -import java.beans.PropertyDescriptor; -import java.util.ArrayList; -import java.util.Collections; -import java.util.List; - -/** - * Helper class for calculating bean property matches, according to. - * Used by BeanWrapperImpl to suggest alternatives for an invalid property name.
- * - * Copied and slightly modified from Spring core, - * - * @author Alef Arendsen - * @author Arjen Poutsma - * @author Juergen Hoeller - * @author Dave Syer - * - * @since 1.0 - * @see #forProperty(String, Class) - */ -final class PropertyMatches { - - //--------------------------------------------------------------------- - // Static section - //--------------------------------------------------------------------- - - /** Default maximum property distance: 2 */ - public static final int DEFAULT_MAX_DISTANCE = 2; - - - /** - * Create PropertyMatches for the given bean property. - * @param propertyName the name of the property to find possible matches for - * @param beanClass the bean class to search for matches - */ - public static PropertyMatches forProperty(String propertyName, Class beanClass) { - return forProperty(propertyName, beanClass, DEFAULT_MAX_DISTANCE); - } - - /** - * Create PropertyMatches for the given bean property. - * @param propertyName the name of the property to find possible matches for - * @param beanClass the bean class to search for matches - * @param maxDistance the maximum property distance allowed for matches - */ - public static PropertyMatches forProperty(String propertyName, Class beanClass, int maxDistance) { - return new PropertyMatches(propertyName, beanClass, maxDistance); - } - - - //--------------------------------------------------------------------- - // Instance section - //--------------------------------------------------------------------- - - private final String propertyName; - - private String[] possibleMatches; - - - /** - * Create a new PropertyMatches instance for the given property. - */ - private PropertyMatches(String propertyName, Class beanClass, int maxDistance) { - this.propertyName = propertyName; - this.possibleMatches = calculateMatches(BeanUtils.getPropertyDescriptors(beanClass), maxDistance); - } - - - /** - * Return the calculated possible matches. - */ - public String[] getPossibleMatches() { - return possibleMatches; - } - - /** - * Build an error message for the given invalid property name, - * indicating the possible property matches. - */ - public String buildErrorMessage() { - StringBuffer buf = new StringBuffer(); - buf.append("Bean property '"); - buf.append(this.propertyName); - buf.append("' is not writable or has an invalid setter method. "); - - if (ObjectUtils.isEmpty(this.possibleMatches)) { - buf.append("Does the parameter type of the setter match the return type of the getter?"); - } - else { - buf.append("Did you mean "); - for (int i = 0; i < this.possibleMatches.length; i++) { - buf.append('\''); - buf.append(this.possibleMatches[i]); - if (i < this.possibleMatches.length - 2) { - buf.append("', "); - } - else if (i == this.possibleMatches.length - 2){ - buf.append("', or "); - } - } - buf.append("'?"); - } - return buf.toString(); - } - - - /** - * Generate possible property alternatives for the given property and - * class. Internally uses the getStringDistance method, which - * in turn uses the Levenshtein algorithm to determine the distance between - * two Strings. - * @param propertyDescriptors the JavaBeans property descriptors to search - * @param maxDistance the maximum distance to accept - */ - private String[] calculateMatches(PropertyDescriptor[] propertyDescriptors, int maxDistance) { - List candidates = new ArrayList(); - for (int i = 0; i < propertyDescriptors.length; i++) { - if (propertyDescriptors[i].getWriteMethod() != null) { - String possibleAlternative = propertyDescriptors[i].getName(); - int distance = calculateStringDistance(this.propertyName, possibleAlternative); - if (distance <= maxDistance) { - candidates.add(possibleAlternative); - } - } - } - Collections.sort(candidates); - return StringUtils.toStringArray(candidates); - } - - /** - * Calculate the distance between the given two Strings - * according to the Levenshtein algorithm. - * @param s1 the first String - * @param s2 the second String - * @return the distance value - */ - private int calculateStringDistance(String s1, String s2) { - if (s1.length() == 0) { - return s2.length(); - } - if (s2.length() == 0) { - return s1.length(); - } - int d[][] = new int[s1.length() + 1][s2.length() + 1]; - - for (int i = 0; i <= s1.length(); i++) { - d[i][0] = i; - } - for (int j = 0; j <= s2.length(); j++) { - d[0][j] = j; - } - - for (int i = 1; i <= s1.length(); i++) { - char s_i = s1.charAt(i - 1); - for (int j = 1; j <= s2.length(); j++) { - int cost; - char t_j = s2.charAt(j - 1); - if (Character.toLowerCase(s_i) == Character.toLowerCase(t_j)) { - cost = 0; - } else { - cost = 1; - } - d[i][j] = Math.min(Math.min(d[i - 1][j] + 1, d[i][j - 1] + 1), - d[i - 1][j - 1] + cost); - } - } - - return d[s1.length()][s2.length()]; - } -} diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/mapping/package-info.java b/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/mapping/package-info.java deleted file mode 100644 index d75232c..0000000 --- a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/mapping/package-info.java +++ /dev/null @@ -1,4 +0,0 @@ -/** - * Default {@code RowMapper} implementations. - */ -package org.springframework.batch.item.excel.mapping; \ No newline at end of file diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/package-info.java b/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/package-info.java deleted file mode 100644 index c7993c4..0000000 --- a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/package-info.java +++ /dev/null @@ -1,4 +0,0 @@ -/** - * Core interfaces for reading Excel files - */ -package org.springframework.batch.item.excel; \ No newline at end of file diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/poi/PoiItemReader.java b/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/poi/PoiItemReader.java deleted file mode 100644 index 5934a98..0000000 --- a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/poi/PoiItemReader.java +++ /dev/null @@ -1,88 +0,0 @@ -/* - * Copyright 2006-2014 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.springframework.batch.item.excel.poi; - -import org.apache.poi.ss.usermodel.Row; -import org.apache.poi.ss.usermodel.Workbook; -import org.apache.poi.ss.usermodel.WorkbookFactory; -import org.springframework.batch.item.excel.AbstractExcelItemReader; -import org.springframework.batch.item.excel.Sheet; -import org.springframework.core.io.Resource; - -import java.io.Closeable; -import java.io.InputStream; -import java.io.PushbackInputStream; - -/** - * {@link org.springframework.batch.item.ItemReader} implementation which uses apache POI to read an Excel - * file. It will read the file sheet for sheet and row for row. It is based on - * the {@link org.springframework.batch.item.file.FlatFileItemReader} - * - * @param the type - * @author Marten Deinum - * @since 0.5.0 - */ -public class PoiItemReader extends AbstractExcelItemReader { - - private Workbook workbook; - - private InputStream workbookStream; - - @Override - protected Sheet getSheet(final int sheet) { - return new PoiSheet(this.workbook.getSheetAt(sheet)); - } - - @Override - protected int getNumberOfSheets() { - return this.workbook.getNumberOfSheets(); - } - - @Override - protected void doClose() throws Exception { - // As of Apache POI 3.11 there is a close method on the Workbook, prior version - // lack this method. - if (workbook instanceof Closeable) { - this.workbook.close(); - } - - if (workbookStream != null) { - workbookStream.close(); - } - this.workbook=null; - this.workbookStream=null; - } - - /** - * Open the underlying file using the {@code WorkbookFactory}. We keep track of the used {@code InputStream} so that - * it can be closed cleanly on the end of reading the file. This to be able to release the resources used by - * Apache POI. - * - * @param resource the {@code Resource} pointing to the Excel file. - * @throws Exception is thrown for any errors. - */ - @Override - protected void openExcelFile(final Resource resource) throws Exception { - workbookStream = resource.getInputStream(); - if (!workbookStream.markSupported() && !(workbookStream instanceof PushbackInputStream)) { - throw new IllegalStateException("InputStream MUST either support mark/reset, or be wrapped as a PushbackInputStream"); - } - this.workbook = WorkbookFactory.create(workbookStream); - this.workbook.setMissingCellPolicy(Row.CREATE_NULL_AS_BLANK); - } - -} diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/poi/PoiSheet.java b/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/poi/PoiSheet.java deleted file mode 100644 index f1ea678..0000000 --- a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/poi/PoiSheet.java +++ /dev/null @@ -1,128 +0,0 @@ -/* - * Copyright 2006-2014 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package org.springframework.batch.item.excel.poi; - -import org.apache.poi.ss.usermodel.Cell; -import org.apache.poi.ss.usermodel.DateUtil; -import org.apache.poi.ss.usermodel.FormulaEvaluator; -import org.apache.poi.ss.usermodel.Row; -import org.springframework.batch.item.excel.Sheet; - -import java.util.Date; -import java.util.LinkedList; -import java.util.List; - -/** - * Sheet implementation for Apache POI. - * - * @author Marten Deinum - * @since 0.5.0 - */ -public class PoiSheet implements Sheet { - - private final org.apache.poi.ss.usermodel.Sheet delegate; - private final int numberOfRows; - private final String name; - - private int numberOfColumns = -1; - private FormulaEvaluator evaluator; - - /** - * Constructor which takes the delegate sheet. - * - * @param delegate the apache POI sheet - */ - PoiSheet(final org.apache.poi.ss.usermodel.Sheet delegate) { - super(); - this.delegate = delegate; - this.numberOfRows = this.delegate.getLastRowNum() + 1; - this.name=this.delegate.getSheetName(); - } - - /** - * {@inheritDoc} - */ - @Override - public int getNumberOfRows() { - return this.numberOfRows; - } - - /** - * {@inheritDoc} - */ - @Override - public String getName() { - return this.name; - } - - /** - * {@inheritDoc} - */ - @Override - public String[] getRow(final int rowNumber) { - final Row row = this.delegate.getRow(rowNumber); - if (row == null) { - return null; - } - final List cells = new LinkedList(); - - for (int i = 0; i < getNumberOfColumns(); i++) { - Cell cell = row.getCell(i); - switch (cell.getCellType()) { - case Cell.CELL_TYPE_NUMERIC: - if (DateUtil.isCellDateFormatted(cell)) { - Date date = cell.getDateCellValue(); - cells.add(String.valueOf(date.getTime())); - } else { - cells.add(String.valueOf(cell.getNumericCellValue())); - } - break; - case Cell.CELL_TYPE_BOOLEAN: - cells.add(String.valueOf(cell.getBooleanCellValue())); - break; - case Cell.CELL_TYPE_STRING: - case Cell.CELL_TYPE_BLANK: - cells.add(cell.getStringCellValue()); - break; - case Cell.CELL_TYPE_FORMULA: - cells.add(getFormulaEvaluator().evaluate(cell).formatAsString()); - break; - default: - throw new IllegalArgumentException("Cannot handle cells of type " + cell.getCellType()); - } - } - return cells.toArray(new String[cells.size()]); - } - - private FormulaEvaluator getFormulaEvaluator() { - if (this.evaluator == null) { - this.evaluator = delegate.getWorkbook().getCreationHelper().createFormulaEvaluator(); - } - return this.evaluator; - } - - /** - * {@inheritDoc} - */ - @Override - public int getNumberOfColumns() { - if (numberOfColumns < 0) { - numberOfColumns = this.delegate.getRow(0).getLastCellNum(); - } - return numberOfColumns; - } -} diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/poi/package-info.java b/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/poi/package-info.java deleted file mode 100644 index 8d7eb30..0000000 --- a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/poi/package-info.java +++ /dev/null @@ -1,4 +0,0 @@ -/** - * Support classes for the Apache POI library. - */ -package org.springframework.batch.item.excel.poi; \ No newline at end of file diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/support/rowset/DefaultRowSet.java b/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/support/rowset/DefaultRowSet.java deleted file mode 100644 index 3fe99ca..0000000 --- a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/support/rowset/DefaultRowSet.java +++ /dev/null @@ -1,90 +0,0 @@ -/* - * Copyright 2006-2014 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.springframework.batch.item.excel.support.rowset; - -import org.springframework.batch.item.excel.Sheet; - -import java.util.Properties; - -/** - * Default implementation of the {@code RowSet} interface. - * - * @author Marten Deinum - * @since 0.5.0 - * - * @see org.springframework.batch.item.excel.support.rowset.DefaultRowSetFactory - */ -public class DefaultRowSet implements RowSet { - - private final Sheet sheet; - private final RowSetMetaData metaData; - - private int currentRowIndex = -1; - private String[] currentRow; - - DefaultRowSet(Sheet sheet, RowSetMetaData metaData) { - this.sheet = sheet; - this.metaData = metaData; - } - - @Override - public RowSetMetaData getMetaData() { - return metaData; - } - - @Override - public boolean next() { - currentRow = null; - currentRowIndex++; - if (currentRowIndex < sheet.getNumberOfRows()) { - currentRow = sheet.getRow(currentRowIndex); - return true; - } - return false; - } - - @Override - public int getCurrentRowIndex() { - return this.currentRowIndex; - } - - @Override - public String[] getCurrentRow() { - return this.currentRow; - } - - @Override - public String getColumnValue(int idx) { - return currentRow[idx]; - } - - @Override - public Properties getProperties() { - final String[] names = metaData.getColumnNames(); - if (names == null) { - throw new IllegalStateException("Cannot create properties without meta data"); - } - - Properties props = new Properties(); - for (int i = 0; i < currentRow.length; i++) { - String value = currentRow[i]; - if (value != null) { - props.setProperty(names[i], value); - } - } - return props; - } -} diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/support/rowset/DefaultRowSetFactory.java b/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/support/rowset/DefaultRowSetFactory.java deleted file mode 100644 index 196a49b..0000000 --- a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/support/rowset/DefaultRowSetFactory.java +++ /dev/null @@ -1,42 +0,0 @@ -/* - * Copyright 2006-2014 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.springframework.batch.item.excel.support.rowset; - -import org.springframework.batch.item.excel.Sheet; - -/** - * {@code RowSetFactory} implementation which constructs a {@code DefaultRowSet} instance and - * {@code DefaultRowSetMetaData} instance. The latter will have the {@code ColumnNameExtractor} configured - * on this factory set (default {@code RowNumberColumnNameExtractor}. - * - * @author Marten Deinum - * @since 0.5.0 - */ -public class DefaultRowSetFactory implements RowSetFactory { - - private ColumnNameExtractor columnNameExtractor = new RowNumberColumnNameExtractor(); - - @Override - public RowSet create(Sheet sheet) { - DefaultRowSetMetaData metaData = new DefaultRowSetMetaData(sheet); - metaData.setColumnNameExtractor(columnNameExtractor); - return new DefaultRowSet(sheet, metaData); - } - - public void setColumnNameExtractor(ColumnNameExtractor columnNameExtractor) { - this.columnNameExtractor = columnNameExtractor; - } -} diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/support/rowset/DefaultRowSetMetaData.java b/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/support/rowset/DefaultRowSetMetaData.java deleted file mode 100644 index 53b1cd2..0000000 --- a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/support/rowset/DefaultRowSetMetaData.java +++ /dev/null @@ -1,63 +0,0 @@ -/* - * Copyright 2006-2014 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.springframework.batch.item.excel.support.rowset; - -import org.springframework.batch.item.excel.Sheet; - -/** - * Default implementation for the {@code RowSetMetaData} interface. - * - * Requires a {@code Sheet} and {@code ColumnNameExtractor} to operate correctly. - * Delegates the retrieval of the column names to the {@code ColumnNameExtractor}. - * - * @author Marten Deinum - * @since 0.5.0 - */ -public class DefaultRowSetMetaData implements RowSetMetaData { - - private final Sheet sheet; - - private ColumnNameExtractor columnNameExtractor; - - DefaultRowSetMetaData(Sheet sheet) { - this.sheet = sheet; - } - - @Override - public String[] getColumnNames() { - return columnNameExtractor.getColumnNames(sheet); - } - - @Override - public String getColumnName(int idx) { - String[] names = getColumnNames(); - return names[idx]; - } - - @Override - public int getColumnCount() { - return sheet.getNumberOfColumns(); - } - - @Override - public String getSheetName() { - return sheet.getName(); - } - - public void setColumnNameExtractor(ColumnNameExtractor columnNameExtractor) { - this.columnNameExtractor = columnNameExtractor; - } -} diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/support/rowset/RowSet.java b/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/support/rowset/RowSet.java deleted file mode 100644 index 1db7752..0000000 --- a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/support/rowset/RowSet.java +++ /dev/null @@ -1,76 +0,0 @@ -/* - * Copyright 2006-2014 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.springframework.batch.item.excel.support.rowset; - -import java.util.Properties; - -/** - * Used by the {@code org.springframework.batch.item.excel.AbstractExcelItemReader} to abstract away - * the complexities of the underlying Excel API implementations. - * - * @author Marten Deinum - * @since 0.5.0 - */ -public interface RowSet { - - /** - * Retrieves the meta data (name of the sheet, number of columns, names) of this row set. - * - * @return a corresponding {@code RowSetMetaData} instance. - */ - RowSetMetaData getMetaData(); - - - /** - * Move to the next row in the document. - * - * @return true if the row is valid, false if there are no more rows - */ - boolean next(); - - /** - * Returns the current row number - * - * @return the current row number - */ - int getCurrentRowIndex(); - - /** - * Return the current row as a String[]. - * - * @return the row as a String[] - */ - String[] getCurrentRow(); - - /** - * Retrieves the value of the indicated column in the current row as a String object. - * - * @param idx the column index, 0 based - * @return a String objeect respresenting the column value. - */ - String getColumnValue(int idx); - - - /** - * Construct name-value pairs from the column names and string values. Null - * values are omitted. - * - * @return some properties representing the row set. - * @throws IllegalStateException if the column name meta data is not - * available. - */ - Properties getProperties(); -} diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/support/rowset/RowSetMetaData.java b/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/support/rowset/RowSetMetaData.java deleted file mode 100644 index aea4eab..0000000 --- a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/support/rowset/RowSetMetaData.java +++ /dev/null @@ -1,54 +0,0 @@ -/* - * Copyright 2006-2014 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.springframework.batch.item.excel.support.rowset; - -/** - * Interface representing the the metadata associated with an Excel document. - * - * @author Marten Deinum - * @since 0.5.0 - */ -public interface RowSetMetaData { - - /** - * Retrieves the names of the columns for the current sheet. - * - * @return the column names. - */ - String[] getColumnNames(); - - /** - * Retrieves the column name for the indicatd column. - * - * @param idx the index of the column, 0 based - * @return the column name - */ - String getColumnName(int idx); - - /** - * Retrieves the number of columns in the RowSet. - * - * @return the number of columns - */ - int getColumnCount(); - - /** - * Retrieves the name of the sheet the RowSet is based on. - * - * @return the name of the sheet - */ - String getSheetName(); -} diff --git a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/support/rowset/package-info.java b/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/support/rowset/package-info.java deleted file mode 100644 index 55a3f94..0000000 --- a/spring-batch-excel/src/main/java/org/springframework/batch/item/excel/support/rowset/package-info.java +++ /dev/null @@ -1,4 +0,0 @@ -/** - * RowSet abstraction for Excel documents - */ -package org.springframework.batch.item.excel.support.rowset; \ No newline at end of file diff --git a/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/AbstractExcelItemReaderTests.java b/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/AbstractExcelItemReaderTests.java new file mode 100644 index 0000000..49df02e --- /dev/null +++ b/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/AbstractExcelItemReaderTests.java @@ -0,0 +1,95 @@ +/* + * Copyright 2002-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.extensions.excel; + +import java.util.Arrays; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.junit.jupiter.api.AfterEach; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; + +import org.springframework.batch.extensions.excel.mapping.PassThroughRowMapper; +import org.springframework.batch.item.ExecutionContext; +import org.springframework.core.io.ClassPathResource; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.assertThatThrownBy; + +/** + * Base class for testing Excel based item readers. + * + * @author Marten Deinum + * @since 0.1.0 + */ +public abstract class AbstractExcelItemReaderTests { + + protected final Log logger = LogFactory.getLog(this.getClass()); + + protected AbstractExcelItemReader itemReader; + + @BeforeEach + public void setup() throws Exception { + this.itemReader = createExcelItemReader(); + this.itemReader.setLinesToSkip(1); // First line is column names + this.itemReader.setResource(new ClassPathResource("player.xls")); + this.itemReader.setRowMapper(new PassThroughRowMapper()); + this.itemReader.setSkippedRowsCallback((rs) -> this.logger.info("Skipping: " + Arrays.toString(rs.getCurrentRow()))); + configureItemReader(this.itemReader); + this.itemReader.afterPropertiesSet(); + ExecutionContext executionContext = new ExecutionContext(); + this.itemReader.open(executionContext); + } + + protected void configureItemReader(AbstractExcelItemReader itemReader) { + } + + @AfterEach + public void after() { + this.itemReader.close(); + } + + @Test + public void readExcelFile() throws Exception { + assertThat(this.itemReader.getNumberOfSheets()).isEqualTo(3); + String[] row; + do { + row = this.itemReader.read(); + if (this.logger.isTraceEnabled()) { + this.logger.trace("Read: " + Arrays.toString(row)); + } + if (row != null) { + assertThat(row).hasSize(6); + } + } + while (row != null); + Integer readCount = (Integer) ReflectionTestUtils.getField(this.itemReader, "currentItemCount"); + assertThat(readCount).isEqualTo(4321); + } + + @Test + public void testRequiredProperties() { + assertThatThrownBy(() -> { + final AbstractExcelItemReader reader = createExcelItemReader(); + reader.afterPropertiesSet(); + }).isInstanceOf(IllegalArgumentException.class); + } + + protected abstract AbstractExcelItemReader createExcelItemReader(); + +} diff --git a/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/BeanPropertyItemReaderTest.java b/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/BeanPropertyItemReaderTest.java new file mode 100644 index 0000000..3a5d0e8 --- /dev/null +++ b/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/BeanPropertyItemReaderTest.java @@ -0,0 +1,91 @@ +/* + * Copyright 2002-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.extensions.excel; + +import java.util.ArrayList; +import java.util.List; + +import org.assertj.core.api.SoftAssertions; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; + +import org.springframework.batch.extensions.excel.mapping.BeanWrapperRowMapper; +import org.springframework.batch.item.ExecutionContext; + +import static org.assertj.core.api.Assertions.assertThat; + +/** + * @author Marten Deinum + * @since 0.1.0 + */ +public class BeanPropertyItemReaderTest { + + private MockExcelItemReader reader; + + @BeforeEach + public void setup() throws Exception { + ExecutionContext executionContext = new ExecutionContext(); + + List rows = new ArrayList<>(); + rows.add(new String[] { "id", "lastName", "firstName", "position", "birthYear", "debutYear" }); + rows.add(new String[] { "AbduKa00", "Abdul-Jabbar", "Karim", "rb", "1974", "1996" }); + rows.add(new String[] { "AbduRa00", "Abdullah", "Rabih", "rb", "1975", "1999" }); + MockSheet sheet = new MockSheet("players", rows); + + this.reader = new MockExcelItemReader<>(sheet); + + BeanWrapperRowMapper rowMapper = new BeanWrapperRowMapper<>(); + rowMapper.setTargetType(Player.class); + rowMapper.afterPropertiesSet(); + + this.reader.setLinesToSkip(1); // Skip first row as that is the header + this.reader.setRowMapper(rowMapper); + + this.reader.afterPropertiesSet(); + this.reader.open(executionContext); + } + + @Test + public void readandMapPlayers() throws Exception { + Player p1 = this.reader.read(); + Player p2 = this.reader.read(); + Player p3 = this.reader.read(); + assertThat(p1).isNotNull(); + assertThat(p2).isNotNull(); + assertThat(p3).isNull(); + + SoftAssertions softly = new SoftAssertions(); + + // Check first player + softly.assertThat(p1.getId()).isEqualTo("AbduKa00"); + softly.assertThat("Abdul-Jabbar").isEqualTo(p1.getLastName()); + softly.assertThat("Karim").isEqualTo(p1.getFirstName()); + softly.assertThat("rb").isEqualTo(p1.getPosition()); + softly.assertThat(1974).isEqualTo(p1.getBirthYear()); + softly.assertThat(1996).isEqualTo(p1.getDebutYear()); + // Check second player + softly.assertThat("AbduRa00").isEqualTo(p2.getId()); + softly.assertThat("Abdullah").isEqualTo(p2.getLastName()); + softly.assertThat("Rabih").isEqualTo(p2.getFirstName()); + softly.assertThat("rb").isEqualTo(p2.getPosition()); + softly.assertThat(1975).isEqualTo(p2.getBirthYear()); + softly.assertThat(1999).isEqualTo(p2.getDebutYear()); + + softly.assertAll(); + } + +} diff --git a/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/BeanPropertyWithStaticHeaderItemReaderTest.java b/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/BeanPropertyWithStaticHeaderItemReaderTest.java new file mode 100644 index 0000000..71d71a0 --- /dev/null +++ b/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/BeanPropertyWithStaticHeaderItemReaderTest.java @@ -0,0 +1,96 @@ +/* + * Copyright 2006-2021 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.extensions.excel; + +import java.util.ArrayList; +import java.util.List; + +import org.assertj.core.api.SoftAssertions; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; + +import org.springframework.batch.extensions.excel.mapping.BeanWrapperRowMapper; +import org.springframework.batch.extensions.excel.support.rowset.DefaultRowSetFactory; +import org.springframework.batch.extensions.excel.support.rowset.StaticColumnNameExtractor; +import org.springframework.batch.item.ExecutionContext; + +import static org.assertj.core.api.Assertions.assertThat; + +/** + * @author Marten Deinum + * @since 0.1.0 + */ +public class BeanPropertyWithStaticHeaderItemReaderTest { + + private MockExcelItemReader reader; + + @BeforeEach + public void setup() throws Exception { + ExecutionContext executionContext = new ExecutionContext(); + + List rows = new ArrayList<>(); + rows.add(new String[] { "AbduKa00", "Abdul-Jabbar", "Karim", "rb", "1974", "1996" }); + rows.add(new String[] { "AbduRa00", "Abdullah", "Rabih", "rb", "1975", "1999" }); + MockSheet sheet = new MockSheet("players", rows); + + this.reader = new MockExcelItemReader<>(sheet); + + BeanWrapperRowMapper rowMapper = new BeanWrapperRowMapper<>(); + rowMapper.setTargetType(Player.class); + rowMapper.afterPropertiesSet(); + + this.reader.setRowMapper(rowMapper); + + DefaultRowSetFactory factory = new DefaultRowSetFactory(); + factory.setColumnNameExtractor(new StaticColumnNameExtractor( + new String[] { "id", "lastName", "firstName", "position", "birthYear", "debutYear" })); + this.reader.setRowSetFactory(factory); + this.reader.afterPropertiesSet(); + this.reader.open(executionContext); + } + + @Test + public void readandMapPlayers() throws Exception { + Player p1 = this.reader.read(); + Player p2 = this.reader.read(); + Player p3 = this.reader.read(); + assertThat(p1).isNotNull(); + assertThat(p2).isNotNull(); + assertThat(p3).isNull(); + + SoftAssertions softly = new SoftAssertions(); + + // Check first player + softly.assertThat(p1.getId()).isEqualTo("AbduKa00"); + softly.assertThat("Abdul-Jabbar").isEqualTo(p1.getLastName()); + softly.assertThat("Karim").isEqualTo(p1.getFirstName()); + softly.assertThat("rb").isEqualTo(p1.getPosition()); + softly.assertThat(1974).isEqualTo(p1.getBirthYear()); + softly.assertThat(1996).isEqualTo(p1.getDebutYear()); + // Check second player + softly.assertThat("AbduRa00").isEqualTo(p2.getId()); + softly.assertThat("Abdullah").isEqualTo(p2.getLastName()); + softly.assertThat("Rabih").isEqualTo(p2.getFirstName()); + softly.assertThat("rb").isEqualTo(p2.getPosition()); + softly.assertThat(1975).isEqualTo(p2.getBirthYear()); + softly.assertThat(1999).isEqualTo(p2.getDebutYear()); + + softly.assertAll(); + + } + +} diff --git a/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/MockExcelItemReader.java b/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/MockExcelItemReader.java new file mode 100644 index 0000000..52f7c6d --- /dev/null +++ b/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/MockExcelItemReader.java @@ -0,0 +1,62 @@ +/* + * Copyright 2006-2021 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.extensions.excel; + +import java.util.Collections; +import java.util.List; + +import org.springframework.core.io.ByteArrayResource; +import org.springframework.core.io.Resource; + +/** + * @author Marten Deinum + * @since 0.1.0 + */ +public class MockExcelItemReader extends AbstractExcelItemReader { + + private final List sheets; + + public MockExcelItemReader(MockSheet sheet) { + this(Collections.singletonList(sheet)); + } + + public MockExcelItemReader(List sheets) { + this.sheets = sheets; + super.setResource(new ByteArrayResource(new byte[0])); + } + + @Override + protected Sheet getSheet(int sheet) { + return this.sheets.get(sheet); + } + + @Override + protected int getNumberOfSheets() { + return this.sheets.size(); + } + + @Override + protected void openExcelFile(Resource resource, String password) throws Exception { + + } + + @Override + protected void doClose() throws Exception { + this.sheets.clear(); + } + +} diff --git a/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/MockSheet.java b/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/MockSheet.java new file mode 100644 index 0000000..54aa850 --- /dev/null +++ b/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/MockSheet.java @@ -0,0 +1,64 @@ +/* + * Copyright 2006-2021 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.extensions.excel; + +import java.util.Iterator; +import java.util.List; + +/** + * Sheet implementation usable for testing. Works in an {@code List} of {@xode String[]}. + * + * @author Marten Deinum + * @since 0.1.0 + */ +public class MockSheet implements Sheet { + + private final List rows; + + private final String name; + + public MockSheet(String name, List rows) { + this.name = name; + this.rows = rows; + } + + @Override + public int getNumberOfRows() { + return this.rows.size(); + } + + @Override + public String getName() { + return this.name; + } + + @Override + public String[] getRow(int rowNumber) { + if (rowNumber < getNumberOfRows()) { + return this.rows.get(rowNumber); + } + else { + return null; + } + } + + @Override + public Iterator iterator() { + return this.rows.iterator(); + } + +} diff --git a/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/Player.java b/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/Player.java new file mode 100644 index 0000000..5c488cf --- /dev/null +++ b/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/Player.java @@ -0,0 +1,95 @@ +/* + * Copyright 2002-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.extensions.excel; + +/** + * @author Marten Deinum + * @since 0.1.0 + */ +public class Player { + + private String id; + + private String position; + + private String lastName; + + private String firstName; + + private long birthYear; + + private int debutYear; + + private String comment; + + public String getId() { + return this.id; + } + + public void setId(String id) { + this.id = id; + } + + public String getPosition() { + return this.position; + } + + public void setPosition(String position) { + this.position = position; + } + + public String getLastName() { + return this.lastName; + } + + public void setLastName(String lastName) { + this.lastName = lastName; + } + + public String getFirstName() { + return this.firstName; + } + + public void setFirstName(String firstName) { + this.firstName = firstName; + } + + public long getBirthYear() { + return this.birthYear; + } + + public void setBirthYear(long birthYear) { + this.birthYear = birthYear; + } + + public int getDebutYear() { + return this.debutYear; + } + + public void setDebutYear(int debutYear) { + this.debutYear = debutYear; + } + + public String getComment() { + return this.comment; + } + + public void setComment(String comment) { + this.comment = comment; + } + +} diff --git a/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/ReflectionTestUtils.java b/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/ReflectionTestUtils.java new file mode 100644 index 0000000..66d4d36 --- /dev/null +++ b/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/ReflectionTestUtils.java @@ -0,0 +1,49 @@ +/* + * Copyright 2006-2021 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.extensions.excel; + +import java.lang.reflect.Field; + +import org.springframework.lang.Nullable; +import org.springframework.util.ReflectionUtils; + +/** + * Simplified version of {@code ReflectionTestUtils} from Spring. This to prevent a + * unneeded dependency on the Spring Test module. + * + * @author Marten Deinum + * @since 0.1.0 + */ +public final class ReflectionTestUtils { + + private ReflectionTestUtils() { } + + @Nullable + public static Object getField(Object targetObject, String name) { + Class targetClass = targetObject.getClass(); + + Field field = ReflectionUtils.findField(targetClass, name); + if (field == null) { + throw new IllegalArgumentException(String.format("Could not find field '%s' on %s or target class [%s]", + name, targetObject, targetClass)); + } + + ReflectionUtils.makeAccessible(field); + return ReflectionUtils.getField(field, targetObject); + } + +} diff --git a/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/mapping/BeanWrapperRowMapperTest.java b/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/mapping/BeanWrapperRowMapperTest.java new file mode 100644 index 0000000..a8cdf40 --- /dev/null +++ b/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/mapping/BeanWrapperRowMapperTest.java @@ -0,0 +1,129 @@ +/* + * Copyright 2002-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.extensions.excel.mapping; + +import java.util.ArrayList; +import java.util.List; + +import org.assertj.core.api.Assertions; +import org.assertj.core.api.SoftAssertions; +import org.junit.jupiter.api.Test; + +import org.springframework.batch.extensions.excel.MockSheet; +import org.springframework.batch.extensions.excel.Player; +import org.springframework.batch.extensions.excel.support.rowset.DefaultRowSetFactory; +import org.springframework.batch.extensions.excel.support.rowset.RowSet; +import org.springframework.context.ApplicationContext; +import org.springframework.context.annotation.AnnotationConfigApplicationContext; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.context.annotation.Scope; + +/** + * @author Marten Deinum + * @since 0.1.0 + */ +public class BeanWrapperRowMapperTest { + + @Test + public void givenNoNameWhenInitCompleteThenIllegalStateShouldOccur() { + Assertions.assertThatThrownBy(() -> { + BeanWrapperRowMapper mapper = new BeanWrapperRowMapper<>(); + mapper.afterPropertiesSet(); + }).isInstanceOf(IllegalStateException.class); + } + + @Test + public void givenAValidRowWhenMappingThenAValidPlayerShouldBeConstructed() throws Exception { + BeanWrapperRowMapper mapper = new BeanWrapperRowMapper<>(); + mapper.setTargetType(Player.class); + mapper.afterPropertiesSet(); + + List rows = new ArrayList<>(); + rows.add(new String[] { "id", "lastName", "firstName", "position", "birthYear", "debutYear" }); + rows.add(new String[] { "AbduKa00", "Abdul-Jabbar", "Karim", "rb", "1974", "1996" }); + MockSheet sheet = new MockSheet("players", rows); + + RowSet rs = new DefaultRowSetFactory().create(sheet); + rs.next(); + rs.next(); + + Player p = mapper.mapRow(rs); + + SoftAssertions softly = new SoftAssertions(); + softly.assertThat(p).isNotNull(); + softly.assertThat(p.getId()).isEqualTo("AbduKa00"); + softly.assertThat("Abdul-Jabbar").isEqualTo(p.getLastName()); + softly.assertThat("Karim").isEqualTo(p.getFirstName()); + softly.assertThat("rb").isEqualTo(p.getPosition()); + softly.assertThat(1974).isEqualTo(p.getBirthYear()); + softly.assertThat(1996).isEqualTo(p.getDebutYear()); + softly.assertThat(p.getComment()).isNull(); + softly.assertAll(); + + } + + @Test + public void givenAValidRowWhenMappingThenAValidPlayerShouldBeConstructedBasedOnPrototype() throws Exception { + + ApplicationContext ctx = new AnnotationConfigApplicationContext(TestConfig.class); + BeanWrapperRowMapper mapper = ctx.getBean("playerRowMapper", BeanWrapperRowMapper.class); + + List rows = new ArrayList<>(); + rows.add(new String[] { "id", "lastName", "firstName", "position", "birthYear", "debutYear" }); + rows.add(new String[] { "AbduKa00", "Abdul-Jabbar", "Karim", "rb", "1974", "1996" }); + MockSheet sheet = new MockSheet("players", rows); + + RowSet rs = new DefaultRowSetFactory().create(sheet); + rs.next(); + rs.next(); + Player p = mapper.mapRow(rs); + + SoftAssertions softly = new SoftAssertions(); + softly.assertThat(p).isNotNull(); + softly.assertThat(p.getId()).isEqualTo("AbduKa00"); + softly.assertThat("Abdul-Jabbar").isEqualTo(p.getLastName()); + softly.assertThat("Karim").isEqualTo(p.getFirstName()); + softly.assertThat("rb").isEqualTo(p.getPosition()); + softly.assertThat(1974).isEqualTo(p.getBirthYear()); + softly.assertThat(1996).isEqualTo(p.getDebutYear()); + softly.assertThat(p.getComment()).isEqualTo("comment from context"); + softly.assertAll(); + + } + + @Configuration + public static class TestConfig { + + @Bean + public BeanWrapperRowMapper playerRowMapper() { + BeanWrapperRowMapper mapper = new BeanWrapperRowMapper<>(); + mapper.setPrototypeBeanName("player"); + return mapper; + } + + @Bean + @Scope("prototype") + public Player player() { + Player p = new Player(); + p.setComment("comment from context"); + return p; + } + + } + +} diff --git a/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/poi/PoiItemReaderWithBlankRowSheetTest.java b/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/poi/PoiItemReaderWithBlankRowSheetTest.java new file mode 100644 index 0000000..549917c --- /dev/null +++ b/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/poi/PoiItemReaderWithBlankRowSheetTest.java @@ -0,0 +1,73 @@ +/* + * Copyright 2002-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.extensions.excel.poi; + +import java.util.Arrays; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; + +import org.springframework.batch.extensions.excel.ReflectionTestUtils; +import org.springframework.batch.extensions.excel.mapping.PassThroughRowMapper; +import org.springframework.batch.item.ExecutionContext; +import org.springframework.core.io.ClassPathResource; + +import static org.assertj.core.api.Assertions.assertThat; + +/** + * @author mishrk3 + * @author Marten Deinum + * @since 0.1.0 + */ +public class PoiItemReaderWithBlankRowSheetTest { + + private final Log logger = LogFactory.getLog(this.getClass()); + + private PoiItemReader itemReader; + + @BeforeEach + public void setup() throws Exception { + this.itemReader = new PoiItemReader<>(); + this.itemReader.setResource(new ClassPathResource("blankRow.xlsx")); + this.itemReader.setLinesToSkip(1); // First line is column names + this.itemReader.setRowMapper(new PassThroughRowMapper()); + this.itemReader.setSkippedRowsCallback((rs) -> this.logger.info("Skipping: " + Arrays.toString(rs.getCurrentRow()))); + this.itemReader.afterPropertiesSet(); + + ExecutionContext executionContext = new ExecutionContext(); + this.itemReader.open(executionContext); + } + + @Test + public void readExcelFileWithBlankRow() throws Exception { + assertThat(this.itemReader.getNumberOfSheets()).isEqualTo(1); + String[] row; + do { + row = this.itemReader.read(); + this.logger.debug("Read: " + Arrays.toString(row)); + if (row != null) { + assertThat(row).hasSize(4); + } + } + while (row != null); + Integer readCount = (Integer) ReflectionTestUtils.getField(this.itemReader, "currentItemCount"); + assertThat(readCount).isEqualTo(7); + } + +} diff --git a/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/poi/PoiItemReaderWithErrorsTest.java b/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/poi/PoiItemReaderWithErrorsTest.java new file mode 100644 index 0000000..48a8274 --- /dev/null +++ b/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/poi/PoiItemReaderWithErrorsTest.java @@ -0,0 +1,77 @@ +/* + * Copyright 2002-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.extensions.excel.poi; + +import java.util.Arrays; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.apache.poi.ss.usermodel.FormulaError; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; + +import org.springframework.batch.extensions.excel.ReflectionTestUtils; +import org.springframework.batch.extensions.excel.mapping.PassThroughRowMapper; +import org.springframework.batch.item.ExecutionContext; +import org.springframework.core.io.ClassPathResource; + +import static org.assertj.core.api.Assertions.assertThat; + +/** + * @author Marten Deinum + * @since 0.1.0 + */ +public class PoiItemReaderWithErrorsTest { + + private final Log logger = LogFactory.getLog(this.getClass()); + + private PoiItemReader itemReader; + + @BeforeEach + public void setup() throws Exception { + this.itemReader = new PoiItemReader<>(); + this.itemReader.setResource(new ClassPathResource("errors.xlsx")); + this.itemReader.setLinesToSkip(1); // First line is column names + this.itemReader.setRowMapper(new PassThroughRowMapper()); + this.itemReader.setSkippedRowsCallback((rs) -> this.logger.info("Skipping: " + Arrays.toString(rs.getCurrentRow()))); + this.itemReader.afterPropertiesSet(); + + ExecutionContext executionContext = new ExecutionContext(); + this.itemReader.open(executionContext); + } + + @Test + public void readExcelFileWithBlankRow() throws Exception { + assertThat(this.itemReader.getNumberOfSheets()).isEqualTo(1); + String[] row; + String[] lastRow = null; + do { + row = this.itemReader.read(); + this.logger.debug("Read: " + Arrays.toString(row)); + if (row != null) { + lastRow = row; + assertThat(row).hasSize(3); + } + } + while (row != null); + Integer readCount = (Integer) ReflectionTestUtils.getField(this.itemReader, "currentItemCount"); + assertThat(readCount).isEqualTo(3); + assertThat(lastRow[2]).isEqualTo(FormulaError.DIV0.getString()); + + } + +} diff --git a/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/poi/PoiItemReaderXlsPasswordTests.java b/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/poi/PoiItemReaderXlsPasswordTests.java new file mode 100644 index 0000000..f01071e --- /dev/null +++ b/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/poi/PoiItemReaderXlsPasswordTests.java @@ -0,0 +1,39 @@ +/* + * Copyright 2002-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.extensions.excel.poi; + +import org.springframework.batch.extensions.excel.AbstractExcelItemReader; +import org.springframework.batch.extensions.excel.AbstractExcelItemReaderTests; +import org.springframework.core.io.ClassPathResource; +/** + * @author Marten Deinum + * @since 0.1.0 + */ +public class PoiItemReaderXlsPasswordTests extends AbstractExcelItemReaderTests { + + @Override + protected void configureItemReader(AbstractExcelItemReader itemReader) { + itemReader.setResource(new ClassPathResource("player_with_password.xls")); + itemReader.setPassword("readme"); + } + + @Override + protected AbstractExcelItemReader createExcelItemReader() { + return new PoiItemReader<>(); + } + +} diff --git a/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/poi/PoiItemReaderXlsTests.java b/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/poi/PoiItemReaderXlsTests.java new file mode 100644 index 0000000..db67306 --- /dev/null +++ b/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/poi/PoiItemReaderXlsTests.java @@ -0,0 +1,33 @@ +/* + * Copyright 2002-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.extensions.excel.poi; + +import org.springframework.batch.extensions.excel.AbstractExcelItemReader; +import org.springframework.batch.extensions.excel.AbstractExcelItemReaderTests; + +/** + * @author Marten Deinum + * @since 0.1.0 + */ +public class PoiItemReaderXlsTests extends AbstractExcelItemReaderTests { + + @Override + protected AbstractExcelItemReader createExcelItemReader() { + return new PoiItemReader<>(); + } + +} diff --git a/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/poi/PoiItemReaderXlsxTests.java b/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/poi/PoiItemReaderXlsxTests.java new file mode 100644 index 0000000..0126b0f --- /dev/null +++ b/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/poi/PoiItemReaderXlsxTests.java @@ -0,0 +1,39 @@ +/* + * Copyright 2002-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.extensions.excel.poi; + +import org.springframework.batch.extensions.excel.AbstractExcelItemReader; +import org.springframework.batch.extensions.excel.AbstractExcelItemReaderTests; +import org.springframework.core.io.ClassPathResource; + +/** + * @author Marten Deinum + * @since 0.1.0 + */ +public class PoiItemReaderXlsxTests extends AbstractExcelItemReaderTests { + + @Override + protected void configureItemReader(AbstractExcelItemReader itemReader) { + itemReader.setResource(new ClassPathResource("player.xlsx")); + } + + @Override + protected AbstractExcelItemReader createExcelItemReader() { + return new PoiItemReader<>(); + } + +} diff --git a/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/streaming/StreamingXlsxItemReaderTest.java b/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/streaming/StreamingXlsxItemReaderTest.java new file mode 100644 index 0000000..e5639ef --- /dev/null +++ b/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/streaming/StreamingXlsxItemReaderTest.java @@ -0,0 +1,39 @@ +/* + * Copyright 2006-2021 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.extensions.excel.streaming; + +import org.springframework.batch.extensions.excel.AbstractExcelItemReader; +import org.springframework.batch.extensions.excel.AbstractExcelItemReaderTests; +import org.springframework.core.io.ClassPathResource; + +/** + * @author Marten Deinum + * @since 0.1.0 + */ +class StreamingXlsxItemReaderTest extends AbstractExcelItemReaderTests { + + @Override + protected void configureItemReader(AbstractExcelItemReader itemReader) { + itemReader.setResource(new ClassPathResource("player.xlsx")); + } + + @Override + protected AbstractExcelItemReader createExcelItemReader() { + return new StreamingXlsxItemReader<>(); + } + +} diff --git a/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/support/rowset/DefaultRowSetMetaDataTest.java b/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/support/rowset/DefaultRowSetMetaDataTest.java new file mode 100644 index 0000000..194a058 --- /dev/null +++ b/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/support/rowset/DefaultRowSetMetaDataTest.java @@ -0,0 +1,80 @@ +/* + * Copyright 2002-2014 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.extensions.excel.support.rowset; + +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.mockito.Mockito; + +import org.springframework.batch.extensions.excel.Sheet; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.mockito.BDDMockito.given; +import static org.mockito.Mockito.times; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.verifyNoMoreInteractions; + +/** + * Tests for {@link DefaultRowSetMetaData} + * + * @author Marten Deinum + * @since 0.1.0 + */ +public class DefaultRowSetMetaDataTest { + + private static final String[] COLUMNS = { "col1", "col2", "col3" }; + + private DefaultRowSetMetaData rowSetMetaData; + + private Sheet sheet; + + private ColumnNameExtractor columnNameExtractor; + + @BeforeEach + public void setup() { + this.sheet = Mockito.mock(Sheet.class); + this.columnNameExtractor = Mockito.mock(ColumnNameExtractor.class); + this.rowSetMetaData = new DefaultRowSetMetaData(this.sheet, this.columnNameExtractor); + } + + @Test + public void shouldReturnColumnsFromColumnNameExtractor() { + + given(this.columnNameExtractor.getColumnNames(this.sheet)).willReturn(COLUMNS); + + String[] names = this.rowSetMetaData.getColumnNames(); + + assertThat(names).isEqualTo(new String[] { "col1", "col2", "col3" }); + + verify(this.columnNameExtractor, times(1)).getColumnNames(this.sheet); + verifyNoMoreInteractions(this.sheet, this.columnNameExtractor); + } + + @Test + public void shouldGetAndReturnNameOfTheSheet() { + + given(this.sheet.getName()).willReturn("testing123"); + + String name = this.rowSetMetaData.getSheetName(); + + assertThat(name).isEqualTo("testing123"); + + verify(this.sheet, times(1)).getName(); + verifyNoMoreInteractions(this.sheet); + } + +} diff --git a/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/support/rowset/StaticColumnNameExtractorTest.java b/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/support/rowset/StaticColumnNameExtractorTest.java new file mode 100644 index 0000000..163dcdd --- /dev/null +++ b/spring-batch-excel/src/test/java/org/springframework/batch/extensions/excel/support/rowset/StaticColumnNameExtractorTest.java @@ -0,0 +1,52 @@ +/* + * Copyright 2006-2021 the original author or authors. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.springframework.batch.extensions.excel.support.rowset; + +import org.junit.jupiter.api.Test; + +import static org.assertj.core.api.Assertions.assertThat; + +/** + * @author Marten Deinum + * @since 0.1.0 + */ +public class StaticColumnNameExtractorTest { + + private static final String[] COLUMNS = { "col1", "col2", "col3", "foo", "bar" }; + + @Test + public void shouldReturnSameHeadersAsPassedIn() { + + StaticColumnNameExtractor columnNameExtractor = new StaticColumnNameExtractor(COLUMNS); + String[] names = columnNameExtractor.getColumnNames(null); + assertThat(names) + .isEqualTo(new String[] { "col1", "col2", "col3", "foo", "bar" }) + .isNotSameAs(COLUMNS); + } + + @Test + public void shouldReturnACopyOfTheHeaders() { + + StaticColumnNameExtractor columnNameExtractor = new StaticColumnNameExtractor(COLUMNS); + String[] names = columnNameExtractor.getColumnNames(null); + + assertThat(names) + .isEqualTo(new String[] { "col1", "col2", "col3", "foo", "bar" }) + .isNotSameAs(COLUMNS); + } + +} diff --git a/spring-batch-excel/src/test/java/org/springframework/batch/item/Player.java b/spring-batch-excel/src/test/java/org/springframework/batch/item/Player.java deleted file mode 100644 index 6b145b9..0000000 --- a/spring-batch-excel/src/test/java/org/springframework/batch/item/Player.java +++ /dev/null @@ -1,71 +0,0 @@ -package org.springframework.batch.item; - -/** - * Created by in329dei on 17-9-2014. - */ -public class Player { - - private String id; - private String position; - private String lastName; - private String firstName; - private long birthYear; - private int debutYear; - private String comment; - - public String getId() { - return id; - } - - public void setId(String id) { - this.id = id; - } - - public String getPosition() { - return position; - } - - public void setPosition(String position) { - this.position = position; - } - - public String getLastName() { - return lastName; - } - - public void setLastName(String lastName) { - this.lastName = lastName; - } - - public String getFirstName() { - return firstName; - } - - public void setFirstName(String firstName) { - this.firstName = firstName; - } - - public long getBirthYear() { - return birthYear; - } - - public void setBirthYear(long birthYear) { - this.birthYear = birthYear; - } - - public int getDebutYear() { - return debutYear; - } - - public void setDebutYear(int debutYear) { - this.debutYear = debutYear; - } - - public String getComment() { - return comment; - } - - public void setComment(String comment) { - this.comment = comment; - } -} diff --git a/spring-batch-excel/src/test/java/org/springframework/batch/item/excel/AbstractExcelItemReaderTests.java b/spring-batch-excel/src/test/java/org/springframework/batch/item/excel/AbstractExcelItemReaderTests.java deleted file mode 100644 index 30bff59..0000000 --- a/spring-batch-excel/src/test/java/org/springframework/batch/item/excel/AbstractExcelItemReaderTests.java +++ /dev/null @@ -1,94 +0,0 @@ -/* - * Copyright 2006-2014 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.springframework.batch.item.excel; - -import org.apache.commons.logging.Log; -import org.apache.commons.logging.LogFactory; -import org.junit.After; -import org.junit.Before; -import org.junit.Test; -import org.springframework.batch.item.ExecutionContext; -import org.springframework.batch.item.excel.mapping.PassThroughRowMapper; -import org.springframework.batch.item.excel.support.rowset.RowSet; -import org.springframework.core.io.ClassPathResource; -import org.springframework.test.util.ReflectionTestUtils; -import org.springframework.util.StringUtils; - -import static org.junit.Assert.assertEquals; - -/** - * Base class for testing Excel based item readers. - * - * @author Marten Deinum - */ -public abstract class AbstractExcelItemReaderTests { - - protected final Log logger = LogFactory.getLog(this.getClass()); - - protected AbstractExcelItemReader itemReader; - - private ExecutionContext executionContext; - - @Before - public void setup() throws Exception { - this.itemReader = createExcelItemReader(); - this.itemReader.setLinesToSkip(1); //First line is column names - this.itemReader.setResource(new ClassPathResource("org/springframework/batch/item/excel/player.xls")); - this.itemReader.setRowMapper(new PassThroughRowMapper()); - this.itemReader.setSkippedRowsCallback(new RowCallbackHandler() { - - public void handleRow(RowSet rs) { - logger.info("Skipping: " + StringUtils.arrayToCommaDelimitedString(rs.getCurrentRow())); - } - }); - configureItemReader(this.itemReader); - this.itemReader.afterPropertiesSet(); - executionContext = new ExecutionContext(); - this.itemReader.open(executionContext); - } - - protected void configureItemReader(AbstractExcelItemReader itemReader) { - } - - @After - public void after() throws Exception { - this.itemReader.close(); - } - - @Test - public void readExcelFile() throws Exception { - assertEquals(3, this.itemReader.getNumberOfSheets()); - String[] row; - do { - row = (String[]) this.itemReader.read(); - this.logger.debug("Read: " + StringUtils.arrayToCommaDelimitedString(row)); - if (row != null) { - assertEquals(6, row.length); - } - } while (row != null); - int readCount = (Integer) ReflectionTestUtils.getField(this.itemReader, "currentItemCount" ); - assertEquals(4321, readCount); - } - - @Test(expected = IllegalArgumentException.class) - public void testRequiredProperties() throws Exception { - final AbstractExcelItemReader reader = createExcelItemReader(); - reader.afterPropertiesSet(); - } - - protected abstract AbstractExcelItemReader createExcelItemReader(); - -} diff --git a/spring-batch-excel/src/test/java/org/springframework/batch/item/excel/BeanPropertyItemReaderTest.java b/spring-batch-excel/src/test/java/org/springframework/batch/item/excel/BeanPropertyItemReaderTest.java deleted file mode 100644 index 21b1ca9..0000000 --- a/spring-batch-excel/src/test/java/org/springframework/batch/item/excel/BeanPropertyItemReaderTest.java +++ /dev/null @@ -1,71 +0,0 @@ -package org.springframework.batch.item.excel; - -import org.junit.Before; -import org.junit.Test; -import org.springframework.batch.item.ExecutionContext; -import org.springframework.batch.item.Player; -import org.springframework.batch.item.excel.mapping.BeanWrapperRowMapper; - -import java.util.ArrayList; -import java.util.List; - -import static org.junit.Assert.*; - -/** - * Created by in329dei on 17-9-2014. - */ -public class BeanPropertyItemReaderTest { - - private MockExcelItemReader reader; - - private ExecutionContext executionContext; - - @Before - public void setup() throws Exception { - executionContext = new ExecutionContext(); - - List rows = new ArrayList(); - rows.add(new String[]{"id", "lastName", "firstName", "position", "birthYear", "debutYear"}); - rows.add( new String[]{"AbduKa00", "Abdul-Jabbar", "Karim", "rb", "1974", "1996"}); - rows.add( new String[]{"AbduRa00", "Abdullah", "Rabih", "rb", "1975", "1999"}); - MockSheet sheet = new MockSheet("players", rows); - - reader = new MockExcelItemReader(sheet); - - BeanWrapperRowMapper rowMapper = new BeanWrapperRowMapper(); - rowMapper.setTargetType(Player.class); - rowMapper.afterPropertiesSet(); - - reader.setLinesToSkip(1); // Skip first row as that is the header - reader.setRowMapper(rowMapper); - - reader.afterPropertiesSet(); - reader.open(executionContext); - } - - @Test - public void readandMapPlayers() throws Exception { - Player p1 = reader.read(); - Player p2 = reader.read(); - Player p3 = reader.read(); - assertNotNull(p1); - assertNotNull(p2); - assertNull(p3); - - // Check first player - assertEquals("AbduKa00", p1.getId()); - assertEquals("Abdul-Jabbar", p1.getLastName()); - assertEquals("Karim", p1.getFirstName()); - assertEquals("rb", p1.getPosition()); - assertEquals(1974, p1.getBirthYear()); - assertEquals(1996, p1.getDebutYear()); - // Check second player - assertEquals("AbduRa00", p2.getId()); - assertEquals("Abdullah", p2.getLastName()); - assertEquals("Rabih", p2.getFirstName()); - assertEquals("rb", p2.getPosition()); - assertEquals(1975, p2.getBirthYear()); - assertEquals(1999, p2.getDebutYear()); - } - -} diff --git a/spring-batch-excel/src/test/java/org/springframework/batch/item/excel/BeanPropertyWithStaticHeaderItemReaderTest.java b/spring-batch-excel/src/test/java/org/springframework/batch/item/excel/BeanPropertyWithStaticHeaderItemReaderTest.java deleted file mode 100644 index a5d88e8..0000000 --- a/spring-batch-excel/src/test/java/org/springframework/batch/item/excel/BeanPropertyWithStaticHeaderItemReaderTest.java +++ /dev/null @@ -1,74 +0,0 @@ -package org.springframework.batch.item.excel; - -import org.junit.Before; -import org.junit.Test; -import org.springframework.batch.item.ExecutionContext; -import org.springframework.batch.item.Player; -import org.springframework.batch.item.excel.mapping.BeanWrapperRowMapper; -import org.springframework.batch.item.excel.support.rowset.DefaultRowSetFactory; -import org.springframework.batch.item.excel.support.rowset.StaticColumnNameExtractor; - -import java.util.ArrayList; -import java.util.List; - -import static org.junit.Assert.*; - -/** - * Created by in329dei on 17-9-2014. - */ -public class BeanPropertyWithStaticHeaderItemReaderTest { - - private MockExcelItemReader reader; - - private ExecutionContext executionContext; - - @Before - public void setup() throws Exception { - executionContext = new ExecutionContext(); - - List rows = new ArrayList(); - rows.add( new String[]{"AbduKa00", "Abdul-Jabbar", "Karim", "rb", "1974", "1996"}); - rows.add( new String[]{"AbduRa00", "Abdullah", "Rabih", "rb", "1975", "1999"}); - MockSheet sheet = new MockSheet("players", rows); - - reader = new MockExcelItemReader(sheet); - - BeanWrapperRowMapper rowMapper = new BeanWrapperRowMapper(); - rowMapper.setTargetType(Player.class); - rowMapper.afterPropertiesSet(); - - reader.setRowMapper(rowMapper); - - DefaultRowSetFactory factory = new DefaultRowSetFactory(); - factory.setColumnNameExtractor(new StaticColumnNameExtractor(new String[]{"id", "lastName", "firstName", "position", "birthYear", "debutYear"})); - reader.setRowSetFactory(factory); - reader.afterPropertiesSet(); - reader.open(executionContext); - } - - @Test - public void readandMapPlayers() throws Exception { - Player p1 = reader.read(); - Player p2 = reader.read(); - Player p3 = reader.read(); - assertNotNull(p1); - assertNotNull(p2); - assertNull(p3); - - // Check first player - assertEquals("AbduKa00", p1.getId()); - assertEquals("Abdul-Jabbar", p1.getLastName()); - assertEquals("Karim", p1.getFirstName()); - assertEquals("rb", p1.getPosition()); - assertEquals(1974, p1.getBirthYear()); - assertEquals(1996, p1.getDebutYear()); - // Check second player - assertEquals("AbduRa00", p2.getId()); - assertEquals("Abdullah", p2.getLastName()); - assertEquals("Rabih", p2.getFirstName()); - assertEquals("rb", p2.getPosition()); - assertEquals(1975, p2.getBirthYear()); - assertEquals(1999, p2.getDebutYear()); - } - -} diff --git a/spring-batch-excel/src/test/java/org/springframework/batch/item/excel/MockExcelItemReader.java b/spring-batch-excel/src/test/java/org/springframework/batch/item/excel/MockExcelItemReader.java deleted file mode 100644 index 7a658f7..0000000 --- a/spring-batch-excel/src/test/java/org/springframework/batch/item/excel/MockExcelItemReader.java +++ /dev/null @@ -1,45 +0,0 @@ -package org.springframework.batch.item.excel; - -import org.springframework.core.io.ByteArrayResource; -import org.springframework.core.io.Resource; - -import java.util.Collections; -import java.util.List; - -/** - * Created by in329dei on 17-9-2014. - */ -public class MockExcelItemReader extends AbstractExcelItemReader { - - - private final List sheets; - - public MockExcelItemReader(MockSheet sheet) { - this(Collections.singletonList(sheet)); - } - - public MockExcelItemReader(List sheets) { - this.sheets=sheets; - super.setResource(new ByteArrayResource(new byte[0])); - } - - @Override - protected Sheet getSheet(int sheet) { - return sheets.get(sheet); - } - - @Override - protected int getNumberOfSheets() { - return sheets.size(); - } - - @Override - protected void openExcelFile(Resource resource) throws Exception { - - } - - @Override - protected void doClose() throws Exception { - sheets.clear(); - } -} diff --git a/spring-batch-excel/src/test/java/org/springframework/batch/item/excel/MockSheet.java b/spring-batch-excel/src/test/java/org/springframework/batch/item/excel/MockSheet.java deleted file mode 100644 index 9501bf3..0000000 --- a/spring-batch-excel/src/test/java/org/springframework/batch/item/excel/MockSheet.java +++ /dev/null @@ -1,50 +0,0 @@ -package org.springframework.batch.item.excel; - -import jxl.Cell; -import org.springframework.batch.item.excel.jxl.JxlUtils; - -import java.util.List; - -/** - * Sheet implementation usable for testing. Works in an {@code List} of {@xode String[]}. - * - * @author Marten Deinum - * @since 0.5.0 - */ -public class MockSheet implements Sheet { - - private final List rows; - private final String name; - - public MockSheet(String name, List rows) { - this.name = name; - this.rows = rows; - } - - @Override - public int getNumberOfRows() { - return rows.size(); - } - - @Override - public String getName() { - return this.name; - } - - @Override - public String[] getRow(int rowNumber) { - if (rowNumber < getNumberOfRows()) { - return this.rows.get(rowNumber); - } else { - return null; - } - } - - @Override - public int getNumberOfColumns() { - if (rows.isEmpty()) { - return 0; - } - return rows.get(0).length; - } -} diff --git a/spring-batch-excel/src/test/java/org/springframework/batch/item/excel/jxl/JxlUtilsTests.java b/spring-batch-excel/src/test/java/org/springframework/batch/item/excel/jxl/JxlUtilsTests.java deleted file mode 100644 index 3c13990..0000000 --- a/spring-batch-excel/src/test/java/org/springframework/batch/item/excel/jxl/JxlUtilsTests.java +++ /dev/null @@ -1,90 +0,0 @@ -/* - * Copyright 2006-2014 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.springframework.batch.item.excel.jxl; - -import jxl.Cell; -import jxl.Workbook; -import org.junit.Assert; -import org.junit.Before; -import org.junit.Test; -import org.mockito.Mockito; - -/** - * Tests for {@link org.springframework.batch.item.excel.jxl.JxlUtils}. - * - * @author Marten Deinum - * - */ -public class JxlUtilsTests { - - private final Cell cell1 = Mockito.mock(Cell.class); - private final Cell cell2 = Mockito.mock(Cell.class); - private final Cell cell3 = Mockito.mock(Cell.class); - private final Cell cell4 = Mockito.mock(Cell.class); - - private final Workbook workbook = Mockito.mock(Workbook.class); - - @Before - public void setup() { - Mockito.when(this.cell1.getContents()).thenReturn("foo"); - Mockito.when(this.cell2.getContents()).thenReturn(" "); - Mockito.when(this.cell3.getContents()).thenReturn(""); - Mockito.when(this.cell4.getContents()).thenReturn(null); - } - - /** - * Test the {@link org.springframework.batch.item.excel.jxl.JxlUtils#isEmpty( jxl.Cell)} method. - */ - @Test - public void checkIfCellsAreEmpty() { - Assert.assertFalse("Cell[1] should not be empty", JxlUtils.isEmpty(this.cell1)); - Assert.assertTrue("Cell[2] should be empty", JxlUtils.isEmpty(this.cell2)); - Assert.assertTrue("Cell[3] should be empty", JxlUtils.isEmpty(this.cell3)); - Assert.assertTrue("Cell[4] should be empty", JxlUtils.isEmpty(this.cell4)); - Assert.assertTrue("[null] should be empty", JxlUtils.isEmpty((Cell) null)); - } - - /** - * Test the {@link JxlUtils#isEmpty( jxl.Cell[])} method. - */ - @Test - public void checkIfRowIsEmpty() { - Assert.assertTrue("[null] should be empty", JxlUtils.isEmpty((Cell[]) null)); - Assert.assertTrue("[null] should be empty", JxlUtils.isEmpty(new Cell[0])); - Assert.assertFalse("Cell[1] should not be empty", - JxlUtils.isEmpty(new Cell[]{this.cell1, this.cell2, this.cell3})); - Assert.assertTrue("Cell[2] should be empty", JxlUtils.isEmpty(new Cell[]{this.cell2, this.cell3, null})); - } - - /** - * Test the {@link JxlUtils#hasSheets( jxl.Workbook)} method. - */ - @Test - public void checkIfWorkbookHasSheets() { - Assert.assertFalse("[null] doesn't have sheets.", JxlUtils.hasSheets(null)); - Mockito.when(this.workbook.getNumberOfSheets()).thenReturn(5); - Assert.assertTrue("Workbook should have sheets.", JxlUtils.hasSheets(this.workbook)); - Mockito.when(this.workbook.getNumberOfSheets()).thenReturn(0); - Assert.assertFalse("Workbook shouldn't have sheets.", JxlUtils.hasSheets(this.workbook)); - - } - - @Test - public void extractingContent() { - Assert.assertTrue("[null] should give empty array", JxlUtils.extractContents(null).length == 0); - } - -} diff --git a/spring-batch-excel/src/test/java/org/springframework/batch/item/excel/mapping/BeanWrapperRowMapperTest.java b/spring-batch-excel/src/test/java/org/springframework/batch/item/excel/mapping/BeanWrapperRowMapperTest.java deleted file mode 100644 index 2db9469..0000000 --- a/spring-batch-excel/src/test/java/org/springframework/batch/item/excel/mapping/BeanWrapperRowMapperTest.java +++ /dev/null @@ -1,100 +0,0 @@ -package org.springframework.batch.item.excel.mapping; - -import org.junit.Test; -import org.springframework.batch.item.Player; -import org.springframework.batch.item.excel.MockSheet; -import org.springframework.batch.item.excel.support.rowset.DefaultRowSetFactory; -import org.springframework.batch.item.excel.support.rowset.RowSet; -import org.springframework.context.ApplicationContext; -import org.springframework.context.annotation.AnnotationConfigApplicationContext; -import org.springframework.context.annotation.Bean; -import org.springframework.context.annotation.Configuration; -import org.springframework.context.annotation.Scope; - -import java.util.ArrayList; -import java.util.List; - -import static org.junit.Assert.*; - -/** - * @author Marten Deinum - * @since 0.5.0 - */ -public class BeanWrapperRowMapperTest { - - @Test(expected = IllegalStateException.class) - public void givenNoNameWhenInitCompleteThenIllegalStateShouldOccur() throws Exception { - BeanWrapperRowMapper mapper = new BeanWrapperRowMapper(); - mapper.afterPropertiesSet(); - } - - @Test - public void givenAValidRowWhenMappingThenAValidPlayerShouldBeConstructed() throws Exception { - BeanWrapperRowMapper mapper = new BeanWrapperRowMapper(); - mapper.setTargetType(Player.class); - mapper.afterPropertiesSet(); - - List rows = new ArrayList(); - rows.add(new String[]{"id", "lastName", "firstName", "position", "birthYear", "debutYear"}); - rows.add( new String[]{"AbduKa00", "Abdul-Jabbar", "Karim", "rb", "1974", "1996"}); - MockSheet sheet = new MockSheet("players", rows); - - - RowSet rs = new DefaultRowSetFactory().create(sheet); - rs.next(); - rs.next(); - - Player p = mapper.mapRow(rs); - assertNotNull(p); - assertEquals("AbduKa00", p.getId()); - assertEquals("Abdul-Jabbar", p.getLastName()); - assertEquals("Karim", p.getFirstName()); - assertEquals("rb", p.getPosition()); - assertEquals(1974, p.getBirthYear()); - assertEquals(1996, p.getDebutYear()); - assertNull(p.getComment()); - - } - - @Test - public void givenAValidRowWhenMappingThenAValidPlayerShouldBeConstructedBasedOnPrototype() throws Exception { - - ApplicationContext ctx = new AnnotationConfigApplicationContext(TestConfig.class); - BeanWrapperRowMapper mapper = new BeanWrapperRowMapper(); - mapper.setPrototypeBeanName("player"); - mapper.setBeanFactory(ctx); - mapper.afterPropertiesSet(); - - List rows = new ArrayList(); - rows.add(new String[]{"id", "lastName", "firstName", "position", "birthYear", "debutYear"}); - rows.add( new String[]{"AbduKa00", "Abdul-Jabbar", "Karim", "rb", "1974", "1996"}); - MockSheet sheet = new MockSheet("players", rows); - - RowSet rs = new DefaultRowSetFactory().create(sheet); - rs.next(); - rs.next(); - Player p = mapper.mapRow(rs); - - assertNotNull(p); - assertEquals("AbduKa00", p.getId()); - assertEquals("Abdul-Jabbar", p.getLastName()); - assertEquals("Karim", p.getFirstName()); - assertEquals("rb", p.getPosition()); - assertEquals(1974, p.getBirthYear()); - assertEquals(1996, p.getDebutYear()); - assertEquals("comment from context", p.getComment()); - } - - @Configuration - public static class TestConfig { - - @Bean - @Scope(value = "prototype") - public Player player() { - Player p = new Player(); - p.setComment("comment from context"); - return p; - } - - } -} diff --git a/spring-batch-excel/src/test/java/org/springframework/batch/item/excel/mapping/PassThroughRowMapperTest.java b/spring-batch-excel/src/test/java/org/springframework/batch/item/excel/mapping/PassThroughRowMapperTest.java deleted file mode 100644 index 7d35605..0000000 --- a/spring-batch-excel/src/test/java/org/springframework/batch/item/excel/mapping/PassThroughRowMapperTest.java +++ /dev/null @@ -1,47 +0,0 @@ -/* - * Copyright 2006-2014 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.springframework.batch.item.excel.mapping; - -import org.junit.Test; -import org.springframework.batch.item.excel.MockSheet; -import org.springframework.batch.item.excel.support.rowset.DefaultRowSetFactory; -import org.springframework.batch.item.excel.support.rowset.RowSet; - -import java.util.Collections; - -import static org.junit.Assert.assertArrayEquals; -import static org.junit.Assert.assertTrue; - -/** - * Tests for {@link PassThroughRowMapper}. - * - * @author Marten Deinum - */ -public class PassThroughRowMapperTest { - - private final PassThroughRowMapper rowMapper = new PassThroughRowMapper(); - - @Test - public void mapRowShouldReturnSameValues() throws Exception { - - final String[] row = new String[]{"foo", "bar", "baz"}; - MockSheet sheet = new MockSheet("mock", Collections.singletonList( row)); - RowSet rs = new DefaultRowSetFactory().create(sheet); - assertTrue(rs.next()); - assertArrayEquals(row, this.rowMapper.mapRow(rs)); - } - -} diff --git a/spring-batch-excel/src/test/java/org/springframework/batch/item/excel/poi/PoiItemReaderXlsTest.java b/spring-batch-excel/src/test/java/org/springframework/batch/item/excel/poi/PoiItemReaderXlsTest.java deleted file mode 100644 index 5636983..0000000 --- a/spring-batch-excel/src/test/java/org/springframework/batch/item/excel/poi/PoiItemReaderXlsTest.java +++ /dev/null @@ -1,30 +0,0 @@ -/* - * Copyright 2006-2014 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.springframework.batch.item.excel.poi; - -import org.apache.commons.logging.Log; -import org.apache.commons.logging.LogFactory; -import org.springframework.batch.item.excel.AbstractExcelItemReader; -import org.springframework.batch.item.excel.AbstractExcelItemReaderTests; - -public class PoiItemReaderXlsTest extends AbstractExcelItemReaderTests { - - @Override - protected AbstractExcelItemReader createExcelItemReader() { - return new PoiItemReader(); - } - -} diff --git a/spring-batch-excel/src/test/java/org/springframework/batch/item/excel/poi/PoiItemReaderXlsxTest.java b/spring-batch-excel/src/test/java/org/springframework/batch/item/excel/poi/PoiItemReaderXlsxTest.java deleted file mode 100644 index 0a658a9..0000000 --- a/spring-batch-excel/src/test/java/org/springframework/batch/item/excel/poi/PoiItemReaderXlsxTest.java +++ /dev/null @@ -1,33 +0,0 @@ -/* - * Copyright 2006-2014 the original author or authors. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.springframework.batch.item.excel.poi; - -import org.springframework.batch.item.excel.AbstractExcelItemReader; -import org.springframework.batch.item.excel.AbstractExcelItemReaderTests; -import org.springframework.core.io.ClassPathResource; - -public class PoiItemReaderXlsxTest extends AbstractExcelItemReaderTests { - - @Override - protected void configureItemReader(AbstractExcelItemReader itemReader) { - itemReader.setResource(new ClassPathResource("org/springframework/batch/item/excel/player.xlsx")); - } - - @Override - protected AbstractExcelItemReader createExcelItemReader() { - return new PoiItemReader(); - } -} diff --git a/spring-batch-excel/src/test/resources/blankRow.xlsx b/spring-batch-excel/src/test/resources/blankRow.xlsx new file mode 100644 index 0000000..9878e9f Binary files /dev/null and b/spring-batch-excel/src/test/resources/blankRow.xlsx differ diff --git a/spring-batch-excel/src/test/resources/errors.xlsx b/spring-batch-excel/src/test/resources/errors.xlsx new file mode 100644 index 0000000..5ef49fb Binary files /dev/null and b/spring-batch-excel/src/test/resources/errors.xlsx differ diff --git a/spring-batch-excel/src/test/resources/log4j.xml b/spring-batch-excel/src/test/resources/log4j.xml deleted file mode 100644 index eb0af73..0000000 --- a/spring-batch-excel/src/test/resources/log4j.xml +++ /dev/null @@ -1,37 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - \ No newline at end of file diff --git a/spring-batch-excel/src/test/resources/log4j2.xml b/spring-batch-excel/src/test/resources/log4j2.xml new file mode 100644 index 0000000..2bc9865 --- /dev/null +++ b/spring-batch-excel/src/test/resources/log4j2.xml @@ -0,0 +1,13 @@ + + + + + + + + + + + + + diff --git a/spring-batch-excel/src/test/resources/org/springframework/batch/item/excel/games.xls b/spring-batch-excel/src/test/resources/org/springframework/batch/item/excel/games.xls deleted file mode 100644 index a61f016..0000000 Binary files a/spring-batch-excel/src/test/resources/org/springframework/batch/item/excel/games.xls and /dev/null differ diff --git a/spring-batch-excel/src/test/resources/org/springframework/batch/item/excel/player.xlsx b/spring-batch-excel/src/test/resources/org/springframework/batch/item/excel/player.xlsx deleted file mode 100644 index 358688f..0000000 Binary files a/spring-batch-excel/src/test/resources/org/springframework/batch/item/excel/player.xlsx and /dev/null differ diff --git a/spring-batch-excel/src/test/resources/org/springframework/batch/item/excel/player.xls b/spring-batch-excel/src/test/resources/player.xls similarity index 99% rename from spring-batch-excel/src/test/resources/org/springframework/batch/item/excel/player.xls rename to spring-batch-excel/src/test/resources/player.xls index a68fc84..7c077d4 100644 Binary files a/spring-batch-excel/src/test/resources/org/springframework/batch/item/excel/player.xls and b/spring-batch-excel/src/test/resources/player.xls differ diff --git a/spring-batch-excel/src/test/resources/player.xlsx b/spring-batch-excel/src/test/resources/player.xlsx new file mode 100644 index 0000000..1521576 Binary files /dev/null and b/spring-batch-excel/src/test/resources/player.xlsx differ diff --git a/spring-batch-excel/src/test/resources/player_with_password.xls b/spring-batch-excel/src/test/resources/player_with_password.xls new file mode 100644 index 0000000..3dc0283 Binary files /dev/null and b/spring-batch-excel/src/test/resources/player_with_password.xls differ