[[bootstrap:indexing]]
= Configuring an Index

Apache Geode allows Indexes (or Indices) to be created on Region data to improve the performance of OQL queries.

In _Spring Data Geode_ (SDG), Indexes are declared with the `index` element:

[source,xml]
----
<gfe:index id="myIndex" expression="someField" from="/SomeRegion" type="HASH"/>
----

In _Spring Data Geode's_ XML schema (a.k.a. SDG namespace), `Index` bean declarations are not bound to a _Region_,
unlike Geode's native `cache.xml`.  Rather, they are top-level elements just like `&lt;gfe:cache&gt;`.  This allows
a developer to declare any number of Indexes on any _Region_ whether they were just created or already exist,
a significant improvement over Geode's native `cache.xml` format.

An `Index` must have a name.  A developer may give the `Index` an explicit name using the `name` attribute,
otherwise the _bean name_ (i.e. value of the `id` attribute) of the `Index` bean definition is used as
the `Index` name.

The `expression` and `from` clause form the main components of an `Index`, identifying the data to index
(i.e. the _Region_ identified in the `from` clause) along with what criteria (i.e. `expression`) is used
to index the data.  The `expression` should be based on what application domain object fields are used
in the predicate of application-defined OQL queries used to query and lookup the objects stored
in the _Region_.

For example, if I have a `Customer` that has a `lastName` property...

[source,java]
----
@Region("Customers")
class Customer {

  @Id
  Long id;

  String lastName;
  String firstName;

  ...
}
----

And, I also have an application defined SD[G] _Repository_ to query for `Customers`...

[source,java]
----
interface CustomerRepository extends GemfireRepository<Customer, Long> {

  Customer findByLastName(String lastName);

  ...
}
----

Then, the SD[G] _Repository_ finder/query method would result in the following OQL statement being executed...

[source,java]
----
SELECT * FROM /Customers c WHERE c.lastName = '$1'
----

Therefore, I might want to create an `Index` like so...

[source,xml]
----
<gfe:index id="myIndex" name="CustomersLastNameIndex" expression="lastName" from="/Customers" type="HASH"/>
----

The `from` clause must refer to a valid, existing _Region_ and is how an `Index` gets applied to a _Region_.
This is *not* _Sprig Data Geode_ specific; this is a feature of Apache Geode.

The `Index` `type` maybe 1 of 3 enumerated values defined by _Spring Data Geode's_
http://docs.spring.io/spring-data-gemfire/docs/current/api/org/springframework/data/gemfire/IndexType.html[IndexType]
enumeration: `FUNCTIONAL`, `HASH` and `PRIMARY_KEY`.

Each of the enumerated values correspond to one of the http://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/query/QueryService.html[QueryService]
`create[|Key|Hash]Index` methods invoked when the actual `Index` is to be created (or "defined"; more on "defining"
Indexes below).  For instance, if the `IndexType` is `PRIMARY_KEY`, then the
http://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/query/QueryService.html#createKeyIndex-java.lang.String-java.lang.String-java.lang.String-[QueryService.createKeyIndex(..)]
is invoked to create a `KEY` `Index`.

The default is `FUNCTIONAL` and results in one of the `QueryService.createIndex(..)` methods
being invoked.

See the _Spring Data Geode_ XML schema for a full set of options.

For more information on Indexing in Apache Geode, see http://geode.apache.org/docs/guide/11/developing/query_index/query_index.html[Working with Indexes]
in Apache Geode's User Guide.

== Defining Indexes

In addition to creating Indexes upfront as `Index` bean definitions are processed by _Spring Data Geode_
on _Spring_ container initialization, you may also *define* all of your application Indexes prior to creating
them by using the `define` attribute, like so...

[source,xml]
----
<gfe:index id="myDefinedIndex" expression="someField" from="/SomeRegion" define="true"/>
----

When `define` is set to `true` (defaults to `false`), this will not actually create the `Index` right then and there.
All "defined" Indexes are created all at once, when the _Spring_ `ApplicationContext` is "refreshed", or, that is,
when a `ContextRefreshedEvent` is published by the _Spring_ container.  _Spring Data Geode_ registers itself as
an `ApplicationListener` listening for the `ContextRefreshedEvent`.  When fired, _Spring Data Geode_ will call
http://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/query/QueryService.html#createDefinedIndexes--[QueryService.createDefinedIndexes()].

Defining Indexes and creating them all at once helps promote speed and efficiency when creating Indexes.

See http://geode.apache.org/docs/guide/11/developing/query_index/create_multiple_indexes.html[Creating Multiple Indexes at Once]
for more details.

== `IgnoreIfExists` and `Override`

Two _Spring Data Geode_ `Index` configuration options warrant special mention here: `ignoreIfExists` and `override`.

These options correspond to the `ignore-if-exists` and `override` attributes on the `&lt;gfe:index&gt;` element
in _Spring Data Geode's_ XML schema, respectively.

WARNING: Make sure you absolutely understand what you are doing before using either of these options.  These options can
affect the performance and/or resources (e.g. memory) consumed by your application at runtime.  As such, both of
these options are disabled (i.e. set to `false`) in SDG by default.

NOTE: These options are only available in _Spring Data Geode_ and exist to workaround known limitations
with Apache Geode; there are no equivalent options or functionality available in Geode itself.

Each option significantly differs in behavior and entirely depends on the type of Geode `Index` _Exception_ thrown.
This also means that neither option has any effect if a Geode Index-type _Exception_ is *not* thrown.  These options
are meant to specifically handle Geode `IndexExistsExceptions` and `IndexNameConflictExceptions`, which can occur
for various, sometimes obscure reasons.  But, in general...

* An http://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/query/IndexExistsException.html[IndexExistsException]
is thrown when there exists another `Index` with the same definition but different name when attempting to
create an `Index`.

* An http://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/query/IndexNameConflictException.html[IndexNameConflictException]
is thrown when there exists another `Index` with the same name but possibly different definition when attempting to
create an `Index`.

_Spring Data Geode's_ default behavior is to *_fail-fast_*, always!  So, neither `Index` _Exception_ will be "handled"
by default; these `Index` _Exceptions_ are simply wrapped in a SDG `GemfireIndexException` and rethrown.  If you wish
for _Spring Data Geode_ to handle them for you, then you can set either of these `Index` bean definition options.

`IgnoreIfExists` always takes *precedence* over `Override`, primarily because it uses less resources given it returns
the "existing" `Index` in both exceptional cases.

=== `IgnoreIfExists` Behavior

When an `IndexExistsException` is thrown and `ignoreIfExists` is set to `true` (or `&lt;gfe:index ignore-if-exists="true"&gt;`),
then the `Index` that would have been created by this `Index` bean definition / declaration will be "*ignored*",
and the "existing" `Index` will be returned.

There is very little consequence in returning the "existing" `Index` since the `Index` "definition" is the same,
as deemed by Geode itself, *not* SDG.

However, this also means that *no* `Index` with the "`name`" specified in your `Index` bean definition / declaration
will "actually" exist from Geode's perspective either (i.e. with
http://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/query/QueryService.html#getIndexes--[QueryService.getIndexes()]).
Therefore, you should be careful when writing OQL query statements that use _Query Hints_, especially _Hints_ that refer
to the application `Index` being "*ignored*".  Those _Query Hints_ will need to be changed.

Now, when an `IndexNameConflictException` is thrown and `ignoreIfExists` is set to `true` (or `&lt;gfe:index ignore-if-exists="true"&gt;`),
then the `Index` that would have been created by this `Index` bean definition / declaration will also be "*ignored*",
and the "existing" Index will be returned, just like when an `IndexExistsException` is thrown.

However, there is more risk in returning the "existing" `Index` and "*ignoring*" the application's definition
of the `Index` when an `IndexNameConflictException` is thrown since, for a `IndexNameConflictException`, while the "names"
of the conflicting Indexes are the same, the "definitions" could very well be different!  This obviously could have
implications for OQL queries specific to the application, where you would presume the Indexes were defined specifically
with the application data access patterns and queries in mind.  However, if like named Indexes differ in definition,
this might not be the case.  So, make sure you verify.

NOTE: SDG makes a best effort to inform the user when the `Index` being ignored is significantly different
in its definition from the "existing" `Index`.  However, in order for SDG to accomplish this, it must be able to "find"
the existing `Index`, which is looked up using the Geode API (the only means available).


=== `Override` Behavior

When an `IndexExistsException` is thrown and `override` is set to `true` (or `&lt;gfe:index override="true"&gt;`), then
the `Index` is effectively "_renamed_".  Remember, `IndexExistsExceptions` are thrown when multiple Indexes exist,
all having the same "definition" but different "names".

_Spring Data Geode_ can only accomplish this using Geode's API, by first "_removing_" the "existing" `Index`
and then "_recreating_" the `Index` with the *new* name.  It is possible that either the remove or subsequent
create invocation could fail.  There is no way to execute both actions atomically and rollback this joint operation
if either fails.

However, if it succeeds, then you have the same problem as before with the "_ignoreIfExists_" option.  Any existing OQL
query statement using "_Query Hints_" referring to the old `Index` by name must be changed.

Now, when an `IndexNameConflictException` is thrown and `override` is set to `true` (or `&lt;gfe:index override="true"&gt;`),
then potentially the "existing" `Index` will be "_re-defined_".  I say "potentially", because it is possible for the
"like-named", "existing" `Index` to have exactly the same definition and name when an `IndexNameConflictException`
is thrown.

If so, SDG is *smart* and will just return the "existing" Index as is, even on `override`.  There is no harm in this
since both the "name" and the "definition" are exactly the same.  Of course, SDG can only accomplish this when
SDG is able to "find" the "existing" `Index`, which is dependent on Geode's APIs.  If it cannot find it,
nothing happens and a SDG `GemfireIndexException` is thrown wrapping the `IndexNameConflictException`.

However, when the "definition" of the "existing" `Index` is different, then SDG will attempt to "_recreate_" the `Index`
using the `Index` definition specified in the `Index` bean definition /declaration.  Make sure this is what you want
and make sure the `Index` definition matches your expectations and application requirements.

=== How does `IndexNameConflictExceptions` actually happen?

It is probably not all that uncommon for `IndexExistsExceptions` to be thrown, especially when
multiple configuration sources are used to configure Geode (e.g. _Spring Data Geode_, Geode _Cluster Config_,
maybe Geode native `cache.xml`, the API, etc, etc).  You should definitely prefer 1 configuration method here
and stick with it.

_However, when does an `IndexNameConflictException` get thrown?_

One particular case is an `Index` defined on a `PARTITION` _Region_ (PR).  When an `Index` is defined on
a `PARTITION` _Region_ (e.g. "X"), Geode distributes the `Index` definition (and name) to other peer members
in the cluster that also host the same `PARTITION` _Region_ (i.e. "X").  The distribution of this `Index` definition
to and subsequent creation of this `Index` by peer members on a "need-to-know" basis (i.e. those hosting the same PR)
is performed asynchronously.

During this window of time, it is possible that these "pending" PR `Indexes` will not be identifiable by Geode,
such as with a call to http://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/query/QueryService.html#getIndexes--[QueryService.getIndexes()]
or with http://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/query/QueryService.html#getIndexes-org.apache.geode.cache.Region-[QueryService.getIndexes(:Region)].

As such, the only way for SDG or other Geode cache client applications (not involving _Spring_) to know for sure,
is to just attempt to create the `Index`.  If it fails with either an `IndexNameConflictException`,
or even an `IndexExistsException`, then you will know.  This is because the `QueryService` `Index` creation waits on
"pending" `Index` definitions, where as the other Geode API calls do not.

In any case, SDG makes a best effort and attempts to inform the user what has or is happening along with
the corrective action.  Given all Geode `QueryService.createIndex(..)` methods are synchronous, "blocking" operations,
then the state of Geode should be consistent and accessible after either of these Index-type _Exceptions_ are thrown,
in which case, SDG can inspect the state of the system and respond/act accordingly, based on the user's
desired configuration.

In all other cases, SDG will simply *_fail-fast_*!
