1446 Commits

Author SHA1 Message Date
Henning Pöttker
fa7ba92662 Fix multi-threaded empty read for JdbcPagingItemReader
On empty input, the JdbcPagingItemReader cannot derive
a start value for the sort key to be used in further
queries. For multi-threaded steps, it is thus necessary
to prevent the reader from trying to read further pages
if the first page is empty.

Issue #3898
2021-09-24 10:38:55 +02:00
Rémi Kaeffer
a644ae31c2 Fix double "the" typo everywhere in source code 2021-09-03 19:09:41 +02:00
Henning Pöttker
38fe9a0262 Fix HippyMethodInvoker by not considering unsuitable candidates
Resolves #3794
2021-09-03 15:02:03 +02:00
Ilya Samartsev
91b393c270 Fix NPE in sql exception translation
Resolves #3968
2021-08-18 20:31:32 +02:00
abielewicz
9a52c6b55a Change the order in which datum writer is selected
Due to GenericRecord being checked before SpecificRecordBase,
GenericDatumWriter was used incorrectly for types that extend
SpecificRecordBase.

Resolves #3859
2021-05-18 17:53:46 +02:00
이동욱
8b57a464cf Replace ArrayList with LinkedList in ListItemReader
Issue #3782
2021-05-18 10:37:37 +02:00
Parikshit Dutta
67e3990949 Updated verifyCursorPosition default to true in JdbcCursorItemReaderBuilder
Issue #3893
2021-05-12 15:15:48 +02:00
Marten Deinum
f284e36a12 Reduce the toCharArray overhead
Both the PatternMatcher and JdbcParameterUtils use String.toCharArray for
processing. This will create a new char[] which needs to be collected. The
same code has been rewritten to use String.charAt to reduce the overhead.

Additionally unneeded object creation in the PatternMatcher has been removed
as well, to reduce further objects needed to be gc'ed.
2021-03-17 18:32:29 +01:00
Marten Deinum
8d645dbf33 Better sizes for StringBuilder
StringBuilder by default has a size of 16, however there
are some places that already create larger strings already.
For efficiency and garbage reduction it would be better to have
larger sizes to begin with.
2021-03-17 16:06:10 +01:00
Marten Deinum
ff3c7c1e58 Added isXXXEnabled for logging statements
Guarded the logging statements which do concatenation of strings
or calling toString on objects. When a log level isn't enabled
this still would produce garbage that would need to be collected.

Guarded all logging up to info, warn and error can be assumed to be
enabled on a production system.
2021-03-17 15:10:29 +01:00
Niels Ferguson
5982e20af8 Fix incorrect usage of StringBuilder#append in TransactionAwareBufferedWriter
Issue #3745
2021-03-10 10:39:21 +01:00
Mahmoud Ben Hassine
96beb18bf6 Add timeout when flushing items to Kafka in KafkaItemWriter
Issue #3773
2021-03-09 16:21:11 +01:00
Mahmoud Ben Hassine
4ffc0827ce Refine contribution #3827
* Update year in licence headers
* Update Javadoc
* Add `this` keyword where appropriate
2021-03-09 15:25:52 +01:00
Jacob Botuck
96606d92ad KafkaItemWriter.write should not return until items are confirmed to have been written
Issue #3773
2021-03-09 15:25:46 +01:00
Mahmoud Ben Hassine
85e71d1e07 Refine af14a2e471
* Make default constructors call other constructors with default values
* Fix formatting
* Update year in license headers
2021-02-23 16:52:35 +01:00
Marten Deinum
b3d5d31bfd Provide constructors for injection
Currently when reading/writing JSON and using the reader/marshaller provided
and you want a custom Gson or ObjectMapper instance it still creates the not
needed instance. Move the construction to a constructor and provide a constructor
to directly pass in the pre-configured Gson or ObjectMapper instance.

The same approach is used in Spring itself where Gson or ObjectMapper instances
can be passed in.
2021-02-23 16:52:24 +01:00
Mahmoud Ben Hassine
f9adf9466c Fix KafkaItemReaderTests
Tests in this class fail intermittently because
they send messages to Kafka in an asynchronous
way and assert on the results immediately without
waiting for the send operation to complete.

This commit updates the tests to wait for
send results before asserting on them
(similar to a140a9f5).

(cherry picked from commit 5826111b85)
2021-02-18 17:21:15 +01:00
Mahmoud Ben Hassine
a3cd01382a Polish contribution feb46fd3f7
* Update year in license header
* Use "this" keyword where appropriate
2020-11-27 23:31:16 +01:00
Aditya Sathe
ccd60e6e0e Make kafkaTemplate protected in KafkaItemWriter
Issue #3802
2020-11-27 23:30:55 +01:00
Antoine Kapps
305674f8ae Fix incorrect behaviour of RepositoryItemReader#jumpToItem() on restart
* Corrects unit tests with expected behavior.
* Creates integration tests highlighting Hibernate lazy-loading failure if the reader has requested the page on open().

Issue #1074
2020-11-20 15:00:59 +01:00
lucap
4724fdcf1a Use isQuoteCharacter instead of direct char comparison in DelimitedLineTokenizer
Issue #682
2020-11-20 11:24:51 +01:00
Parikshit Dutta
8e821fe1c5 Fix for setting name of StaxEventItemReader at build 2020-09-25 10:37:11 +02:00
Parikshit Dutta
67a0c7a372 Add support to build StaxEventItemReader when no Resource is provided
Resolves #3736
2020-09-22 10:07:25 +02:00
Mahmoud Ben Hassine
51a1ecddad Polish contribution 2020-09-21 13:25:57 +02:00
Santiago Molano
52ce019e60 Add support to build JsonItemReader when no Resource is provided
Resolves #3731
2020-09-21 13:19:24 +02:00
Ankur Trapasiya
6052852db6 Add connectionAutoCommit property in JdbcCursorItemReaderBuilder
Issue #3717
2020-06-03 11:12:57 +02:00
Philippe Marschall
75ce010127 Improve TransactionAwareBufferedWriter
TransactionAwareBufferedWriter offers a number of optimization
potentials. First it creates an unnecessary local, temporary char[]
in write(char[], int, int). Second it does not overwrite any of the
#write(String) methods leading to unnecessary intermediate copies.

 * avoid local, temporary char[] in #write(char[], int, int)
 * overwrite #write(String) methods to avoid copies

Together these two changes should help to reduce allocation rate.

Issue: #1166
2020-05-07 16:27:03 +02:00
Parikshit Dutta
c12d93d761 Removed redundant logical checks from MongoItemReaderBuilder and MongoItemReader
to prevent false alarm for not providing the limit for query or pagesize for the reader

Issue #3673
2020-04-23 12:25:37 +02:00
Ruslan Mustaev
88ffd06cce Add usePersist chain method to JpaItemWriterBuilder 2020-04-17 12:25:35 +02:00
Santiago Molano
70063d3996 Fixed FlatFileItemReaderBuilder LineTokenizer validation
Fixed validation for the FlatFileItemReaderBuilder where no
LineTokenizer had been provided.

Resolves: #3688
2020-03-31 16:18:18 -05:00
Sanghyuk Jung
0a0a2ec802 Fix constructor of JsonItemReader to call setExecutionContextName()
Resolves #3681
2020-03-24 18:39:17 +01:00
Mahmoud Ben Hassine
e398c3ea65 Add extra check on connection state in AbstractCursorItemReader#doClose
Issue #868
2020-03-18 21:48:46 +01:00
Mahmoud Ben Hassine
1a6ecfdb5d Fix line tokenizer validation in FlatFileItemReaderBuilder
Resolves #766
2020-03-17 23:28:13 +01:00
Chris Schaefer
40fd8bdc93 BATCH-2270: Allow ScriptEvaluator to be injectable in the ScriptItemProcessor 2020-03-13 09:40:30 +01:00
BenjaminHetzJelli
afb24ac66d Trim Keywords Followed By Whitespace Other Than The Character ' '
Modify `removeKeyWord(...)` such that the keyword is removed regardless
of what kind of whitespace follows. This is especially useful for those
who read in SQL from a file which has been formatted such that keywords
live on their own lines.

Added unit tests for trimming whitespace.

Resolves #765
2020-03-04 11:04:36 +01:00
Mahmoud Ben Hassine
170addd905 Remove deprecated usage of com.mongodb.util.JSON in MongoItemReader
(cherry picked from commit e0f78d1504)
2020-01-10 12:20:47 +01:00
Mahmoud Ben Hassine
d7ff73be3b Fix default value of comment prefix in FlatFileItemReaderBuilder
Before this commit, the default value of comment prefix in
FlatFileItemReaderBuilder was not consistent with the one in
FlatFileItemReader.

This commit changes the default value of comment prefix to # in
the builder to be consistent with the reader.

Resolves BATCH-2862
2019-12-02 16:25:49 +01:00
Drummond Dawson
a5e7f19301 Support empty comments in FlatFileItemReaderBuilder
Previously, if supplying empty comments or no comments to comments() of the FlatFileItemReaderBuilder, the comments would be ignored and instead use the default comments "#" provided by the FlatFileItemReader.

Resolves BATCH-2837
2019-12-02 11:14:56 +01:00
Mahmoud Ben Hassine
60dada567f Polish 62a2e65e20
Add null check for delimiter
2019-11-29 09:47:32 +01:00
Drummond Dawson
bfbaab5872 Support empty delimiter in DelimitedBuilder of FlatFileItemWriterBuilder
Previously, if supplying an empty delimiter, to the delimited().delimiter() of DelimitedBuilder from the parent FlatFileItemWriterBuilder, it would be ignored and instead use the default delimiter ",".

Resolves BATCH-2844
2019-11-29 09:45:58 +01:00
Mahmoud Ben Hassine
2b4a87b1b8 Remove the requirement for a Resource to be set
This commit removes the requirement for a Resource to be configured at
build time. This requirement prevented the use of this builder along
with the MultiResourceItemWriter.

Resolves BATCH-2720
2019-11-15 17:56:53 +01:00
Mahmoud Ben Hassine
89e43cd2c3 Make the list of streams final in CompositeItemStream
Resolves BATCH-1969
2019-10-21 11:44:42 +02:00
Mahmoud Ben Hassine
5b46c919ea Deprecate ListPreparedStatementSetter in favor of ArgumentPreparedStatementSetter
ListPreparedStatementSetter is almost a duplicate of
ArgumentPreparedStatementSetter except that it accepts a List of
arguments instead of an array of arguments.

Resolves BATCH-2796
2019-09-27 11:05:33 +02:00
Mahmoud Ben Hassine
2306141826 Polish a7092a21e4 2019-09-17 17:06:44 +02:00
mikeldpl
a7092a21e4 Added @Nullable annotation to overriding methods.
Resolves BATCH-2839
2019-09-17 16:50:44 +02:00
Gary Gregory
c4010fbffa Allow access from subclasses to JdbcBatchItemWriter instance variables 2019-09-06 14:38:23 +02:00
Michael Minella
a140a9f5a1 Attempting to fix the KafkaItemReaderTests#testReadFromMultiplePartitionsAfterRestart 2019-08-02 17:06:54 -05:00
Michael Minella
4b9554fcce Polish 2019-08-02 14:33:45 -05:00
Mahmoud Ben Hassine
f26165e764 Polish bd53168ddb 2019-07-30 18:02:34 +02:00
David Turanski
bd53168ddb Implement AvroItemReader and AvroItemWriter
Resolves BATCH-2833
2019-07-30 17:57:54 +02:00