45 lines
1.5 KiB
Plaintext
45 lines
1.5 KiB
Plaintext
= spring-batch-bigquery
|
|
|
|
Spring Batch extension which contains an `ItemWriter` and `ItemReader` implementations for https://cloud.google.com/bigquery[BigQuery].
|
|
|
|
`ItemWriter` supports next formats (https://cloud.google.com/bigquery/docs/batch-loading-data[load jobs]):
|
|
|
|
* https://en.wikipedia.org/wiki/Comma-separated_values[CSV]
|
|
* https://en.wikipedia.org/wiki/JSON[JSON]
|
|
|
|
Based on https://github.com/googleapis/java-bigquery[Java BigQuery].
|
|
|
|
== Example of `BigQueryCsvItemWriter`
|
|
|
|
[source,java]
|
|
----
|
|
@Bean
|
|
BigQueryCsvItemWriter<MyDto> bigQueryCsvWriter() {
|
|
WriteChannelConfiguration writeConfiguration = WriteChannelConfiguration
|
|
.newBuilder(TableId.of("csv_dataset", "csv_table"))
|
|
.setAutodetect(true)
|
|
.setFormatOptions(FormatOptions.csv())
|
|
.build();
|
|
|
|
return new BigQueryCsvItemWriterBuilder<MyDto>()
|
|
.bigQuery(bigQueryService)
|
|
.writeChannelConfig(writeConfiguration)
|
|
.build();
|
|
}
|
|
----
|
|
|
|
== Example of `BigQueryItemReader`
|
|
|
|
[source,java]
|
|
----
|
|
@Bean
|
|
BigQueryItemReader<PersonDto> bigQueryReader() {
|
|
return new BigQueryQueryItemReaderBuilder<PersonDto>()
|
|
.bigQuery(bigQueryService)
|
|
.rowMapper(res -> new PersonDto(res.get("name").getStringValue()))
|
|
.query("SELECT p.name FROM persons p")
|
|
.build();
|
|
}
|
|
----
|
|
|
|
Additional examples could be found in the https://github.com/spring-projects/spring-batch-extensions/tree/main/spring-batch-bigquery/src/test/java/org/springframework/batch/extensions/bigquery[test folder]. |