spring-data-couchbase/src/main/asciidoc/repository.adoc

[[couchbase.repository]]
= Couchbase repositories

The goal of Spring Data repository abstraction is to significantly reduce the amount of boilerplate code required to implement data access layers for various persistence stores.

There are three backing mechanisms in Couchbase for repositories, described in the sections of this chapter:

 - <<couchbase.repository.n1ql>>
 - <<couchbase.repository.views.querying>>
 - <<couchbase.repository.spatial>>

Note that you can tune the consistency you want for your queries (see <<couchbase.repository.consistency>>).

[[couchbase.repository.configuration]]
== Configuration

While support for repositories is always present, you need to enable them in general or for a specific namespace. If you extend `AbstractCouchbaseConfiguration`, just use the `@EnableCouchbaseRepositories` annotation. It provides lots of possible options to narrow or customize the search path, one of the most common ones is `basePackages`.

.Annotation-Based Repository Setup
====
[source,java]
----
@Configuration
@EnableCouchbaseRepositories(basePackages = {"com.couchbase.example.repos"})
public class Config extends AbstractCouchbaseConfiguration {
    //...
}
----
====
An advanced usage is described in <<couchbase.repository.multibucket>>.

XML-based configuration is also available:

.XML-Based Repository Setup
====
[source,xml]
----
<couchbase:repositories base-package="com.couchbase.example.repos" />
----
====

[[couchbase.repository.usage]]
== Usage

In the simplest case, your repository will extend the `CrudRepository<T, String>`, where T is the entity that you want to expose. Let's look at a repository for a UserInfo:

.A UserInfo repository
====
[source,java]
----
import org.springframework.data.repository.CrudRepository;

public interface UserRepository extends CrudRepository<UserInfo, String> {
}
----
====

Please note that this is just an interface and not an actual class. In the background, when your context gets initialized, actual implementations for your repository descriptions get created and you can access them through regular beans. This means you will save lots of boilerplate code while still exposing full CRUD semantics to your service layer and application.

Now, let's imagine we `@Autowire` the `UserRepository` to a class that makes use of it. What methods do we have available?

[cols="2", options="header"]
.Exposed methods on the UserRepository
|===
| Method
| Description

| UserInfo save(UserInfo entity)
| Save the given entity.

| Iterable<UserInfo> save(Iterable<UserInfo> entity)
| Save the list of entities.

| UserInfo findOne(String id)
| Find a entity by its unique id.

| boolean exists(String id)
| Check if a given entity exists by its unique id.

| Iterable<UserInfo> findAll() (*)
| Find all entities by this type in the bucket.

| Iterable<UserInfo> findAll(Iterable<String> ids)
| Find all entities by this type and the given list of ids.

| long count() (*)
| Count the number of entities in the bucket.

| void delete(String id)
| Delete the entity by its id.

| void delete(UserInfo entity)
| Delete the entity.

| void delete(Iterable<UserInfo> entities)
| Delete all given entities.

| void deleteAll() (*)
| Delete all entities by type in the bucket.
|===

Now that's awesome! Just by defining an interface we get full CRUD functionality on top of our managed entity. All methods suffixed with (*) in the table are backed by Views, which is explained later.

While the exposed methods provide you with a great variety of access patterns, very often you need to define custom ones. You can do this by adding method declarations to your interface, which will be automatically resolved to requests in the background, as we'll see in the next sections.

[[couchbase.repository.n1ql]]
= N1QL based querying
As of version `4.0`, Couchbase Server ships with a new query language called `N1QL`. In `Spring-Data-Couchbase 2.0`, N1QL is the default way of doing queries and will allow you to fully derive queries from a method name.

Prerequisite is to have a N1QL-compatible cluster and to have created a PRIMARY INDEX on the bucket where the entities will be stored.

WARNING: If it is detected at configuration time that the cluster doesn't support N1QL while there are @N1QL annotated methods or non-annotated methods in your repository interface, a `UnsupportedCouchbaseFeatureException` will be thrown.

Here is an example:

.An extended UserInfo repository with N1QL queries
====
[source,java]
----
public interface UserRepository extends CrudRepository<UserInfo, String> {

    @Query("$SELECT_ENTITY$ WHERE role = 'admin' AND $FILTER_TYPE$")
    List<UserInfo> findAllAdmins();

    List<UserInfo> findByFirstname(String fname);
}
----
====

Here we see two N1QL-backed ways of querying.

The first one uses the `Query` annotation to provide a N1QL statement inline. Notice the special placeholders:

 - `$SELECT_ENTITY` allows to easily make sure the statement will select all the fields necessary to build the full entity (including document ID and CAS value).
 - `$FILTER_TYPE$` in the WHERE clause adds a criteria matching the entity type with the field that Spring Data uses to store type information.

The second one use Spring-Data's query derivation mechanism to build a N1QL query from the method name and parameters. This will produce a query looking like this: `SELECT ... FROM ... WHERE firstName = "valueOfFnameAtRuntime"`. You can combine these criteria, even do a count with a name like `countByFirstname` or a limit with a name like `findFirst3ByLastname`...

NOTE: Actually the generated N1QL query will also contain an additional N1QL criteria in order to only select documents that match the repository's entity class.

Most Spring-Data keywords are supported:
.Supported keywords inside @Query (N1QL) method names
[options = "header, autowidth"]
|===============
|Keyword|Sample|N1QL WHERE clause snippet
|`And`|`findByLastnameAndFirstname`|`lastName = a AND firstName = b`
|`Or`|`findByLastnameOrFirstname`|`lastName = a OR firstName = b`
|`Is,Equals`|`findByField`,`findByFieldEquals`|`field = a`
|`IsNot,Not`|`findByFieldIsNot`|`field != a`
|`Between`|`findByFieldBetween`|`field BETWEEN a AND b`
|`IsLessThan,LessThan,IsBefore,Before`|`findByFieldIsLessThan`,`findByFieldBefore`|`field < a`
|`IsLessThanEqual,LessThanEqual`|`findByFieldIsLessThanEqual`|`field <= a`
|`IsGreaterThan,GreaterThan,IsAfter,After`|`findByFieldIsGreaterThan`,`findByFieldAfter`|`field > a`
|`IsGreaterThanEqual,GreaterThanEqual`|`findByFieldGreaterThanEqual`|`field >= a`
|`IsNull`|`findByFieldIsNull`|`field IS NULL`
|`IsNotNull,NotNull`|`findByFieldIsNotNull`|`field IS NOT NULL`
|`IsLike,Like`|`findByFieldLike`|`field LIKE "a"` - a should be a String containing % and _ (matching n and 1 characters)
|`IsNotLike,NotLike`|`findByFieldNotLike`|`field NOT LIKE "a"` - a should be a String containing % and _ (matching n and 1 characters)
|`IsStartingWith,StartingWith,StartsWith`|`findByFieldStartingWith`|`field LIKE "a%"` - a should be a String prefix
|`IsEndingWith,EndingWith,EndsWith`|`findByFieldEndingWith`|`field LIKE "%a"` - a should be a String suffix
|`IsContaining,Containing,Contains`|`findByFieldContains`|`field LIKE "%a%"` - a should be a String
|`IsNotContaining,NotContaining,NotContains`|`findByFieldNotContaining`|`field NOT LIKE "%a%"` - a should be a String
|`IsIn,In`|`findByFieldIn`|`field IN array` - note that the next parameter value (or its children if a collection/array) should be compatible for storage in a `JsonArray`)
|`IsNotIn,NotIn`|`findByFieldNotIn`|`field NOT IN array` - note that the next parameter value (or its children if a collection/array) should be compatible for storage in a `JsonArray`)
|`IsTrue,True`|`findByFieldIsTrue`|`field = TRUE`
|`IsFalse,False`|`findByFieldFalse`|`field = FALSE`
|`MatchesRegex,Matches,Regex`|`findByFieldMatches`|`REGEXP_LIKE(field, "a")` - note that the ignoreCase is ignored here, a is a regular expression in String form
|`Exists`|`findByFieldExists`|`field IS NOT MISSING` - used to verify that the JSON contains this attribute
|`OrderBy`|`findByFieldOrderByLastnameDesc`|`field = a ORDER BY lastname DESC`
|`IgnoreCase`|`findByFieldIgnoreCase`|`LOWER(field) = LOWER("a")` - a must be a String
|===============

You can use both counting queries and <<repositories.limit-query-result>> features with this approach.

With N1QL, another possible interface for the repository is the `PagingAndSortingRepository` one (which extends CRUDRepository).
It adds two methods:
[cols="2", options="header"]
.Exposed methods on the PagingAndSortingRepository
|===
| Method
| Description

| Iterable<T> findAll(Sort sort);
| Allows to retrieve all relevant entities while sorting on one of their attributes.

| Page<T> findAll(Pageable pageable);
| Allows to retrieve your entities in pages. The returned `Page` allows to easily get the next page's `Pageable` as well as the list of items. For the first call, use `new PageRequest(0, pageSize)` as Pageable.
|===

TIP: You can also use `Page` and `Slice` as method return types as well with a N1QL backed repository.

The second way of querying, supported also in older versions of Couchbase Server, is the View-backed one that we'll see in the next section.

[[couchbase.repository.views]]
== Backing Views
This is the historical way of secondary indexing in Couchbase. Views are much more limited in terms of querying flexibility, and each custom method may very well need its own backing view, to be prepared in the cluster beforehand.

We'll only cover views to the extent to which they are needed, if you need in-depth information about them please refer to the official Couchbase Server manual and the Couchbase Java SDK manual.

As a rule of thumb, all repository CRUD access methods which are not "by a specific key" still require a single backing view, by default `all`, to find the one or more matching entities.

IMPORTANT: This is only true for the methods directly defined by the `CrudRepository` interface (the one marked with a `*` in `Table 1.` above), since your additional methods can now be backed by N1QL.

To cover the basic CRUD methods from the `CrudRepository`, one view needs to be implemented in Couchbase Server. It basically returns all documents for the specific entity and also adds the optional reduce function `_count`.

Since every view has a design document and view name, by convention we default to `all` as the view name and the uncapitalized (lowercase first letter) entity name as the design document name. So if your entity is named `UserInfo`, then the code expects the `all` view in the `userInfo` design document. It needs to look like this:

.The all view map function
====
[source,javascript]
----
// do not forget the _count reduce function!
function (doc, meta) {
  if (doc._class == "namespace.to.entity.UserInfo") {
    emit(null, null);
  }
}
----
====

Note that the important part in this map function is to only include the document IDs which correspond to our entity. Because the library always adds the `_class` property, this is a quick and easy way to do it. If you have another property in your JSON which does the same job (like a explicit `type` field), then you can use that as well - you don't have to stick to `_class` all the time.

Also make sure to publish your design documents into production so that they can be picked up by the library! Also, if you are curious why we use `emit(null, null)` in the view: the document id is always sent over to the client implicitly, so we can shave off a view bytes in our view by not duplicating the id. If you use `emit(meta.id, null)` it won't hurt much too.

[[couchbase.repository.views.querying]]
=== View based querying

In `2.0`, since N1QL has been introduced as a more powerful concept, view-backed queries have changed a bit outside of the CRUD methods:

 - the `@View` annotation is mandatory.
 - if you just want all the results from the view, you can let the framework guess the view name to use by just using the plain annotation `@View`. **You won't be able to customize** the `ViewQuery` (eg. adding limits and specifying a `startkey`) using this method anymore.
 - if you want your view query to have restrictions, those can be derived from the method name but in this case you **must** explicitly provide the `viewName` attribute in the annotation.
 - View based query derivation is limited to a few keywords and only works on simple keys (not compound keys like `[ age, fname ]`).
 - View based query derivation still needs you to include *one* valid property before keywords in the method name.

.An extended UserInfo repository with View queries
====
[source,java]
----
public interface UserRepository extends CrudRepository<UserInfo, String> {

    @View
    List<UserInfo> findAllAdmins();

    @View(viewName="firstNames")
    List<UserInfo> findByFirstnameStartingWith(String fnamePrefix);
}
----
====

Implementing your custom repository finder methods also needs backing views. The `findAllAdmins` guesses to use the `allAdmins` view in the `userInfo` design document, by convention. Imagine we have a field on our entity which looks like `boolean isAdmin`. We can write a view like this to expose them (we don't need a reduce function for this one, unless you plan to call one by prefixing your method with `count` instead of `find`!):

.The allAdmins map function
====
[source,javascript]
----
function (doc, meta) {
  if (doc._class == "namespace.to.entity.UserInfo" && doc.isAdmin) {
    emit(null, null);
  }
}
----
====

By now, we've never actually customized our view at query time. This is where the alternative, query derivation, comes along - like in our `findByFirstnameStartingWith(String fnamePrefix)` method.

.The firstNames view map function
====
[source,javascript]
----
function (doc, meta) {
  if (doc._class == "namespace.to.entity.UserInfo") {
    emit(doc.firstname, null);
  }
}
----
====

This view not only emits the document id, but also the firstname of every UserInfo as the key. We can now run a `ViewQuery` which returns us all users with a firstname of "Michael" or "Michele".

.Query a repository method with custom params.
====
[source,java]
----
// Load the bean, or @Autowire it
UserRepository repo = ctx.getBean(UserRepository.class);

// Find all users with first name starting with "Mich"
List<UserInfo> users = repo.findByFirstnameStartingWith("Mich");
----
====

On all these derived custom finder methods, you have to use the `@View` annotation with at least the view name specified (and you can also override the design document name, otherwise determined by convention).

IMPORTANT: For any other usage and customization of the `ViewQuery` that goes beyond that, recommended approach is to provide an implementation that uses the underlying template, like described in <<repositories.single-repository-behaviour>>.
For more details on behavior, please consult the Couchbase Server and Java SDK documentation directly.

For view-based query derivation, here are the supported keywords (A and B are method parameters in this table):

.Supported keywords inside @View method names
[options = "header, autowidth"]
|===============
|`Is,Equals`|`findAllByUsername`,`findByFieldEquals`|`key=A` - if only keyword, the method can have no parameter (return all items from the view)
|`Between`|`findByFieldBetween`|`startkey=A&endkey=B`
|`IsLessThan,LessThan,IsBefore,Before`|`findByFieldIsLessThan`,`findByFieldBefore`|`endkey=A`
|`IsLessThanEqual,LessThanEqual`|`findByFieldIsLessThanEqual`|`endkey=A&inclusive_end=true`
|`IsGreaterThanEqual,GreaterThanEqual`|`findByFieldGreaterThanEqual`|`startkey=A`
|`IsStartingWith,StartingWith,StartsWith`|`findByFieldStartingWith`|`startkey="A"&endkey="A\uefff"` - A should be a String prefix
|`IsIn,In`|`findByFieldIn`|`keys=[A]` - A should be a `Collection`/`Array` with elements compatible for storage in a `JsonArray` (or a single element to be stored in a `JsonArray`)
|===============

TIP: Note that the `reduce function` (not always a count) will be activated by prefixing with `count` and that <<repositories.limit-query-result>> is also supported.

WARNING: Compound keys are not supported, and neither are Or composition, Ignore Case and Order By. You have to include a valid entity property in the naming of your method.


Last method of querying in Couchbase (from Couchbase Server 4.0, like for N1QL) is querying for dimensional data through *Spatial Views*, as we'll see in the next section.

[[couchbase.repository.spatial]]
== Spatial View based querying
Couchbase can accommodate multi-dimensional data and query it with the use of special views, the Spatial Views. Such views allows to perform multi-dimensional queries, not only limited to geographical data.

Integration of these views in `Spring Data Couchbase` repositories is done through the `@Dimensional` annotation. Like `@View`, the annotation allows to indicate usage of a Spatial View as the backing mechanism for the annotated method. The annotation requires you to give the name of the `designDocument` and the `spatialViewName` to use. Additionally, you should specify the number of `dimensions` the view works with (unless it is the default classical 2).

Multi-dimensionality concept is interesting, it means you can craft views that allows you to answer questions like "find all shops that are within Manhattan and open between 14:00 and 23:00" (the third dimension of the view being the opening hours).

Couchbase's Spatial View support querying through ranges that represent "lowest" and "highest" values in each dimension, so for 2D it represents a bounding box, with the southwest-most point [x,y] as `startRange` and northeast-most point [x,y] as `endRange`.

WARNING: Some forms of Geographical queries can only be approximated, for instance querying with a `Polygon` or a `Circle` can only be approximated to the containing bounding `Box` (in DEBUG log level this will be made explicit with each such query).

The following query derivation keywords and parameters relative to geographical data in Spring Data are supported for Spatial Views:

.Supported keywords inside @Dimensional method names
[options = "header, autowidth"]
|===============
|Keyword|Sample|Remarks
|`Within,IsWithin`|`findByLocationWithin`|
|`Near,IsNear`|`findByLocationNear`|expects a `Point` and a `Distance`, will approximate to bounding box
|`Between`|`findByLocationWithinAndOpeningHoursBetween`|useful for dimensions beyond 2, adds two numerical values to the startRange and endRange respectively
|`GreaterThan,GreaterThanEqual,After`|`findByLocationWithinAndOpeningHoursAfter`|useful for dimensions beyond 2, adds a numerical value to the startRange
|`LessThan,LessThanEqual,Before`|`findByLocationWithinAndOpeningHoursBefore`|useful for dimensions beyond 2, adds a numerical value to the endRange
|===============

IMPORTANT: For "within" types of queries, the expected parameters map to geographical 2D data. Classes from the `org.springframework.data.geo` package are usually expected, but Polygon and Boxes can also be expressed as arrays of `Point`s.

TIP: Further dimensions are supported through keywords other than Within and Near and require numerical input.

=== Removing False Positives
`Circle` and `Polygon` will be approximated to their surrounding bounding `Box`, but we provide `PointInShapeEvaluator` (`AwtPointInShapeEvaluator`) to be used on the results if you want to remove false positives.

Here is an example of a repository using a spatial view with 3 dimensions (x, y and opening hours).
Second example demonstrates querying it within a `Polygon` then removing false positives in a second pass:

.Example of dimensional repository
====
[source,java]
----
public interface StoreRepository extends PagingAndSortingRepository<Store, String> {

  @Dimensional(designDocument = "store", spatialViewName = "byLocationOpening", dimensions = 3)
  public List<Store> findByLocationWithinAndOpenBefore(Polygon zone, Integer minOpeningHourMinute);

}
----
====

.Query a spatial view and eliminate false positives on the geo aspect
====
[source,java]
----
// Load the bean, or @Autowire it
StoreRepository repo = ctx.getBean(StoreRepository.class);

// Prepare an arbitrary polygon (a triangle)
Polygon zone = new Polygon(
  new Point(2.0, 0.0),
  new Point(4.0, 0.0),
  new Point(3.0, -5.5),
  new Point(2.0, 0.0)); //we explicitly closed the polygon

// Find all stores located inside the triangle.
// Use 3rd dimension to also query on opening hours, making sure one can go there at 8:00am
List<Store> stores = repo.findByLocationWithinAndOpenBefore(zone, 800);

// The view will return all matching stores that are actually within the enclosing rectangle around the polygon
// So we remove false positives
PointInShapeEvaluator evaluator = new AwtPointInShapeEvaluator();
List<Store> truePositives = new ArrayList<Store>(stores.size());
for (Store store : stores) {
  if (evaluator.pointInPolygon(store.getLocation(), zone)) {
    truePositives.add(store);
  } //else this is a false positive
}

// Alternatively, use "removeFalsePositives" to get the list right away.
truePositives = evaluator.removeFalsePositives(stores,
  //this method needs a Converter to know where to find the location
  new Converter<Store, Point>() {
    @Override
    public Point convert(Store source) {
      return source.getLocation();
    }
  },
  zone);
----
====

[[couchbase.repository.consistency]]
== Querying with consistency
One aspect that is often needed and doesn't have a direct equivalent in the Spring Data query derivation mechanism is
`query consistency`. In both view-based queries and N1QL, you have this concept that the secondary index can return stale
data, because the latest version hasn't been indexed yet. This gives the best performance at the expense of consistency.

If one wants to have stronger consistency, there are two possibilities described in the next sections.

=== Configure it on a global level
The global consistency used by generated queries (views and N1QL) is defined at the template level,
using `Consistency` enumeration (like `Consistency.READ_YOUR_OWN_WRITE`):

 - in xml, this is done via the `consistency` attribute on `<couchbase:template>`.
 - in javaConfig, this is done by overriding the `getDefaultConsistency()` method.

=== Provide an implementation
Provide the implementation and directly use `queryView` and `queryN1QL` methods on the template with a specific consistency
(see <<repositories.single-repository-behaviour>>).

 - one can specify the consistency on those via their respective query classes, according to the Couchbase Java SDK documentation.
 - for example for views `ViewQuery.stale(Stale.FALSE)`
 - for example for N1QL `Query.simple("SELECT * FROM default", QueryParams.build().consistency(ScanConsistency.REQUEST_PLUS));`


[[couchbase.repository.multibucket]]
== Working with multiple buckets
The Java Config version allows you to define multiple `Bucket` and `CouchbaseTemplate`, but in order to have different
repositories use different underlying buckets/templates, you need to follow these steps:

 * in your `AbstractCouchbaseConfiguration` implementation, override the `repositoryOperationsMapping` bean method.
 * have it return a new `RepositoryOperationsMapping` (or reuse the `super()` one)
 * configure the mapping by chaining calls to `map`, `mapEntity` and `setDefault`.
   ** `map` maps a specific repository interface to the `CouchbaseOperations` it should use
   ** `mapEntity` maps all unmapped repositories of a domain type / entity class to a common `CouchbaseOperations`
   ** finally the value passed to the constructor or `setDefault` maps all remaining unmapped repositories to a default
   `CouchaseOperations` (the default, using `couchbaseTemplate` bean unless modified).

The idea is that the framework will look for an entry corresponding to the repository's interface when instantiating it.
If none is found it will look at the mapping for the repository's domain type. Eventually it will fallback to the default setting.
Here is an example:

.Example of configuring multiple templates and repositories.
====
[source,java]
----
@Configuration
@EnableCouchbaseRepositories
public class ConcreteCouchbaseConfig extends AbstractCouchbaseConfig {

  //the default bucket and template must be created, implement abstract methods here to that end

  //we want all User objects to be stored in a second bucket
  //let's define the bucket reference...
  @Bean
  public Bucket userBucket() {
    return couchbaseCluster().openBucket("users", "");
  }

  //... then the template (inspired by couchbaseTemplate() method)...
  @Bean
  public CouchbaseTemplate userTemplate() {
    CouchbaseTemplate template = new CouchbaseTemplate(
        couchbaseClusterInfo(), //reuse the default bean
      userBucket(), //the bucket is non-default
        mappingCouchbaseConverter(), translationService() //default beans here as well
    );
    template.setDefaultConsistency(getDefaultConsistency());
    return template;
  }

  //... then finally make sure all repositories of Users will use it
  @Override
  public RepositoryOperationsMapping repositoryOperationsMapping() throws Exception {
    return super.repositoryOperationsMapping() //this is already using couchbaseTemplate as default
      .mapEntity(User.class, userTemplate()); //every repository dealing with User will be backed by userTemplate()
  }
}
----
====