diff --git a/spring-core/src/main/java/org/springframework/core/io/buffer/DataBuffer.java b/spring-core/src/main/java/org/springframework/core/io/buffer/DataBuffer.java index 61da1bfb4f..491fd0893a 100644 --- a/spring-core/src/main/java/org/springframework/core/io/buffer/DataBuffer.java +++ b/spring-core/src/main/java/org/springframework/core/io/buffer/DataBuffer.java @@ -40,9 +40,9 @@ import java.util.function.IntPredicate; *

The {@linkplain #capacity() capacity} of a {@code DataBuffer} is expanded on demand, * similar to {@code StringBuilder}. * - *

The main purpose of the {@code DataBuffer} abstraction is provide a convenient wrapper around - * {@link ByteBuffer}, similar to Netty's {@link io.netty.buffer.ByteBuf}, that can also be used on - * non-Netty platforms (i.e. Servlet). + *

The main purpose of the {@code DataBuffer} abstraction is to provide a convenient wrapper + * around {@link ByteBuffer} that is similar to Netty's {@link io.netty.buffer.ByteBuf}, but that + * can also be used on non-Netty platforms (i.e. Servlet). * * @author Arjen Poutsma * @since 5.0 @@ -237,14 +237,14 @@ public interface DataBuffer { ByteBuffer asByteBuffer(int index, int length); /** - * Expose this buffer's data as an {@link InputStream}. Both data and position are + * Expose this buffer's data as an {@link InputStream}. Both data and read position are * shared between the returned stream and this data buffer. * @return this data buffer as an input stream */ InputStream asInputStream(); /** - * Expose this buffer's data as an {@link OutputStream}. Both data and position are + * Expose this buffer's data as an {@link OutputStream}. Both data and write position are * shared between the returned stream and this data buffer. * @return this data buffer as an output stream */ diff --git a/spring-core/src/main/java/org/springframework/core/io/buffer/DataBufferFactory.java b/spring-core/src/main/java/org/springframework/core/io/buffer/DataBufferFactory.java index 789335a7b6..21e8f10f37 100644 --- a/spring-core/src/main/java/org/springframework/core/io/buffer/DataBufferFactory.java +++ b/spring-core/src/main/java/org/springframework/core/io/buffer/DataBufferFactory.java @@ -1,5 +1,5 @@ /* - * Copyright 2002-2016 the original author or authors. + * Copyright 2002-2017 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -19,7 +19,7 @@ package org.springframework.core.io.buffer; import java.nio.ByteBuffer; /** - * A factory for {@link DataBuffer}s,allowing for allocation and wrapping of + * A factory for {@link DataBuffer}s, allowing for allocation and wrapping of * data buffers. * * @author Arjen Poutsma @@ -46,14 +46,16 @@ public interface DataBufferFactory { DataBuffer allocateBuffer(int initialCapacity); /** - * Wrap the given {@link ByteBuffer} in a {@code DataBuffer}. + * Wrap the given {@link ByteBuffer} in a {@code DataBuffer}. Unlike + * {@linkplain #allocateBuffer(int) allocating}, wrapping does not use new memory. * @param byteBuffer the NIO byte buffer to wrap * @return the wrapped buffer */ DataBuffer wrap(ByteBuffer byteBuffer); /** - * Wrap the given {@code byte} array in a {@code DataBuffer}. + * Wrap the given {@code byte} array in a {@code DataBuffer}. Unlike + * {@linkplain #allocateBuffer(int) allocating}, wrapping does not use new memory. * @param bytes the byte array to wrap * @return the wrapped buffer */ diff --git a/src/docs/asciidoc/core.adoc b/src/docs/asciidoc/core.adoc index d0e81afacb..97cc569b52 100644 --- a/src/docs/asciidoc/core.adoc +++ b/src/docs/asciidoc/core.adoc @@ -34,4 +34,6 @@ include::core/core-aop-api.adoc[leveloffset=+1] include::core/core-null-safety.adoc[leveloffset=+1] +include::core/core-databuffer-codec.adoc[leveloffset=+1] + include::core/core-appendix.adoc[leveloffset=+1] diff --git a/src/docs/asciidoc/core/core-databuffer-codec.adoc b/src/docs/asciidoc/core/core-databuffer-codec.adoc new file mode 100644 index 0000000000..88ee6124fb --- /dev/null +++ b/src/docs/asciidoc/core/core-databuffer-codec.adoc @@ -0,0 +1,150 @@ +[[databuffers]] += Data Buffers and Codecs + +== Introduction + +The `DataBuffer` interface defines an abstraction over byte buffers. +The main reason for introducing it, and not use the standard `java.nio.ByteBuffer` instead, is Netty. +Netty does not use `ByteBuffer`, but instead offers `ByteBuf` as an alternative. +Spring's `DataBuffer` is a simple abstraction over `ByteBuf` that can also be used on non-Netty +platforms (i.e. Servlet 3.1+). + +== `DataBufferFactory` + +The `DataBufferFactory` offers functionality to allocate new data buffers, as well as to wrap +existing data. +The `allocate` methods allocate a new data buffer, with a default or given capacity. +Though `DataBuffer` implementation grow and shrink on demand, it is more efficient to give the +capacity upfront, if known. +The `wrap` methods decorate an existing `ByteBuffer` or byte array. +Wrapping does not involve allocation: it simply decorates the given data with a `DataBuffer` +implementation. + +There are two implementation of `DataBufferFactory`: the `NettyDataBufferFactory` which is meant +to be used on Netty platforms, such as Reactor Netty. +The other implementation, the `DefaultDataBufferFactory`, is used on other platforms, such as +Servlet 3.1+ servers. + +== The `DataBuffer` interface + +The `DataBuffer` interface is similar to `ByteBuffer`, but offers a number of advantages. +Similar to Netty's `ByteBuf`, the `DataBuffer` abstraction offers independent read and write +positions. +This is different from the JDK's `ByteBuffer`, which only exposes one position for both reading and +writing, and a separate `flip()` operation to switch between the two I/O operations. +In general, the following invariant holds for the read position, write position, and the capacity: + +-- +`0` <= _read position_ <= _write position_ <= _capacity_ +-- + +When reading bytes from the `DataBuffer`, the read position is automatically updated in accordance with +the amount of data read from the buffer. +Similarly, when writing bytes to the `DataBuffer`, the write position is updated with the amount of +data written to the buffer. +Also, when writing data, the capacity of a `DataBuffer` is automatically expanded, just like `StringBuilder`, +`ArrayList`, and similar types. + +Besides the reading and writing functionality mentioned above, the `DataBuffer` also has methods to +view a (slice of a) buffer as `ByteBuffer`, `InputStream`, or `OutputStream`. +Additionally, it offers methods to determine the index of a given byte. + +There are two implementation of `DataBuffer`: the `NettyDataBuffer` which is meant to be used on +Netty platforms, such as Reactor Netty. +The other implementation, the `DefaultDataBuffer`, is used on other platforms, such as Servlet 3.1+ +servers. + +=== `PooledDataBuffer` + +The `PooledDataBuffer` is an extension to `DataBuffer` that adds methods for reference counting. +The `retain` method increases the reference count by one. +The `release` method decreases the count by one, and releases the buffer's memory when the count +reaches 0. +Both of these methods are related to _reference counting_, a mechanism that is explained below. + +Note that `DataBufferUtils` offers useful utility methods for releasing and retaining pooled data +buffers. +These methods take a plain `DataBuffer` as parameter, but only call `retain` or `release` if the +passed data buffer is an instance of `PooledDataBuffer`. + +[[databuffer-reference-counting]] +==== Reference Counting + +Reference counting is not a common technique in Java; it is much more common in other programming +languages such as Object C and C++. +In and of itself, reference counting is not complex: it basically involves tracking the number of +references that apply to an object. +The reference count of a `PooledDataBuffer` starts at 1, is incremented by calling `retain`, +and decremented by calling `release`. +As long as the buffer's reference count is larger than 0 the buffer will not be released. +When the number decreases to 0, the instance will be released. +In practice, this means that the reserved memory captured by the buffer will be returned back to +the memory pool, ready to be used for future allocations. + +In general, _the last component to access a `DataBuffer` is responsible for releasing it_. +Withing Spring, there are two sorts of components that release buffers: decoders and transports. +Decoders are responsible for transforming a stream of buffers into other types (see <> below), + and transports are responsible for sending buffers across a network boundary, typically as an HTTP message. +This means that if you allocate data buffers for the purpose of putting them into an outbound HTTP +message (i.e. client-side request or server-side response), they do not have to be released. +The other consequence of this rule is that if you allocate data buffers that do not end up in the +body, for instance because of a thrown exception, you will have to release them yourself. +The following snippet shows a typical `DataBuffer` usage scenario when dealing with methods that +throw exceptions: + +[source,java,indent=0] +[subs="verbatim,quotes"] +---- +DataBufferFactory factory = ... +DataBuffer buffer = factory.allocateBuffer(); <1> +boolean release = true; <2> +try { + writeDataToBuffer(buffer); <3> + putBufferInHttpBody(buffer); + release = false; <4> +} finally { + if (release) { + DataBufferUtils.release(buffer); <5> + } +} + +private void writeDataToBuffer(DataBuffer buffer) throws IOException { <3> + ... +} +---- +<1> A new buffer is allocated. +<2> A boolean flag indicates whether the allocated buffer should be released. +<3> This example method loads data into the buffer. Note that the method can throw an `IOException`, +and therefore a `finally` block to release the buffer is required. +<4> If no exception occurred, we switch the `release` flag to `false` as the buffer will now be +released as part of sending the HTTP body across the wire. +<5> If an exception did occur, the flag is still set to `true`, and the buffer will be released +here. + +=== DataBufferUtils + +`DataBufferUtils` contains various utility methods that operate on data buffers. +It contains methods for reading a `Flux` of `DataBuffer` objects from an `InputStream` or NIO +`Channel`, and methods for writing a data buffer `Flux` to an `OutputStream` or `Channel`. +`DataBufferUtils` also exposes `retain` and `release` methods that operate on plain `DataBuffer` +instances (so that casting to a `PooledDataBuffer` is not required). + +[codecs] +== Codecs + +The `org.springframework.core.codec` package contains the two main abstractions for converting a +stream of bytes into a stream of objects, or vice-versa. +The `Encoder` is a strategy interface that encodes a stream of objects into an output stream of +data buffers. +The `Decoder` does the reverse: it turns a stream of data buffers into a stream of objects. +Note that a decoder instance needs to consider <>. + +Spring comes with a wide array of default codecs, capable of converting from/to `String`, +`ByteBuffer`, byte arrays, and also codecs that support marshalling libraries such as JAXB and +Jackson. +Withing the context of Spring WebFlux, codecs are used to convert the request body into a +`@RequestMapping` parameter, or to convert the return type into the response body that is sent back +to the client. +The default codecs are configured in the `WebFluxConfigurationSupport` class, and can easily be +changed by overriding the `configureHttpMessageCodecs` when inheriting from that class. +For more information about using codecs in WebFlux, see <>. diff --git a/src/docs/asciidoc/web/webflux.adoc b/src/docs/asciidoc/web/webflux.adoc index 37b3ad70b6..0959744a5d 100644 --- a/src/docs/asciidoc/web/webflux.adoc +++ b/src/docs/asciidoc/web/webflux.adoc @@ -434,7 +434,8 @@ for encoding and decoding the HTTP request and response body with Reactive Strea It builds on lower level contracts from `spring-core`: * {api-spring-framework}/core/io/buffer/DataBuffer.html[DataBuffer] -- abstraction for -byte buffers -- e.g. Netty `ByteBuf`, `java.nio.ByteBuffer` +byte buffers -- e.g. Netty `ByteBuf`, `java.nio.ByteBuffer`, see +<>. * {api-spring-framework}/core/codec/Encoder.html[Encoder] -- serialize a stream of Objects to a stream of data buffers * {api-spring-framework}/core/codec/Decoder.html[Decoder] -- deserialize a stream of data