Spring/spring-batch-extensions
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

[bq] refactoring & dependency upgrade

Browse Source
This commit is contained in:
Volodymyr
2024-08-18 18:36:21 +03:00
committed by GitHub
parent 50f93d3cab
commit 6f3277a2a3
29 changed files with 163 additions and 176 deletions
Download Patch File Download Diff File Expand all files Collapse all files

22
spring-batch-bigquery/README.adoc
View File

@@ -1,12 +1,14 @@
# spring-batch-bigquery
= spring-batch-bigquery
Spring Batch extension which contains an `ItemWriter` implementation for https://cloud.google.com/bigquery[BigQuery] based on https://github.com/googleapis/java-bigquery[Java BigQuery]. It supports writing https://en.wikipedia.org/wiki/Comma-separated_values[CSV], https://en.wikipedia.org/wiki/JSON[JSON] using https://cloud.google.com/bigquery/docs/batch-loading-data[load jobs].
Spring Batch extension which contains an `ItemWriter` implementation for https://cloud.google.com/bigquery[BigQuery] based on https://github.com/googleapis/java-bigquery[Java BigQuery].
It supports writing https://en.wikipedia.org/wiki/Comma-separated_values[CSV], https://en.wikipedia.org/wiki/JSON[JSON] using https://cloud.google.com/bigquery/docs/batch-loading-data[load jobs].
## Configuration of `BigQueryCsvItemWriter`
== Configuration of `BigQueryCsvItemWriter`
Next to the https://docs.spring.io/spring-batch/reference/html/configureJob.html[configuration of Spring Batch] one needs to configure the `BigQueryCsvItemWriter`.
```javaBigQueryCsv
[source,java]
----
@Bean
BigQueryCsvItemWriter<MyDto> bigQueryCsvWriter() {
WriteChannelConfiguration writeConfiguration = WriteChannelConfiguration
@@ -20,19 +22,19 @@ BigQueryCsvItemWriter<MyDto> bigQueryCsvWriter() {
.writeChannelConfig(writeConfiguration)
.build();
}
```
----
Additional examples could be found in https://github.com/spring-projects/spring-batch-extensions/blob/main/spring-batch-bigquery/src/test/java/org/springframework/batch/extensions/bigquery/writer/builder/[here].
## Configuration properties
== Configuration properties
[cols="1,1,4"]
.Properties for item writer
.Properties for an item writer
|===
| Property | Required | Description
| `bigQuery` | yes | BigQuery object that provided by BigQuery Java Library. Responsible for connection with BigQuery.
| `writeChannelConfig` | yes | BigQuery write channel config provided by BigQuery Java Library. Responsible for configuring data type, data channel, jobs that will be sent to BigQuery.
| `rowMapper` | no | Your own converter that specifies how to convert input CSV / JSON to byte array.
| `datasetInfo` | no | Your way to customize to how to create BigQuery dataset.
| `rowMapper` | no | Your own converter that specifies how to convert input CSV / JSON to a byte array.
| `datasetInfo` | no | Your way to customize how to create BigQuery dataset.
| `jobConsumer` | no | Your custom handler for BigQuery Job provided by BigQuery Java Library.
|===
|===