[[bootstrap:lucene]]
= Apache Lucene Integration

https://pivotal.io/pivotal-gemfire[Pivotal GemFire] integrates with http://lucene.apache.org/[Apache Lucene] to allow developers
to index and search on data stored in Pivotal GemFire using Lucene queries.  Search-based queries also includes
the capability to page through query results.

Additionally, _Spring Data for Pivotal GemFire_ adds support for query projections based on _Spring Data Commons_
Projection infrastructure.  This feature enables the query results to be projected into first-class,
application domain types as needed or required by the application use case.

However, a Lucene `Index` must be created before any Lucene search-based query can be ran.  A `LuceneIndex`
can be created in _Spring (Data for Pivotal GemFire)_ XML config like so...

[source,xml]
----
<gfe:lucene-index id="IndexOne" fields="fieldOne, fieldTwo" region-path="/Example"/>
----

Additionally, Apache Lucene allows the specification of
http://lucene.apache.org/core/6_5_0/core/org/apache/lucene/analysis/Analyzer.html[Analyzers] per field
and can be configured using...

[source,xml]
----
<gfe:lucene-index id="IndexTwo" lucene-service-ref="luceneService" region-path="/AnotherExample">
    <gfe:field-analyzers>
        <map>
            <entry key="fieldOne">
                <bean class="example.AnalyzerOne"/>
             </entry>
            <entry key="fieldTwo">
                <bean class="example.AnalyzerTwo"/>
             </entry>
        </map>
    </gfe:field-analyzers>
</gfe:lucene-index>
----

Of course, the `Map` can be specified as a top-level bean definition and referenced using the `ref` attribute
in the nested `<gfe:field-analyzers>` element like this, `<gfe-field-analyzers ref="refToTopLevelMapBeanDefinition"/>`.

Spring Data for Pivotal GemFire's `LuceneIndexFactoryBean` API and SDG's XML namespace also allows the addition of a
http://gemfire-95-javadocs.docs.pivotal.io/org/apache/geode/cache/lucene/LuceneSerializer.html[`org.apache.geode.cache.lucene.LuceneSerializer`]
to be specified when creating the `LuceneIndex`.  The `LuceneSerializer` is used to configure the way objects
are converted to Lucene documents for the index when the object is indexed.

To add an `LuceneSerializer` to the `LuceneIndex`, you only need to...

[source,xml]
----
<bean id="MyLuceneSerializer" class="example.CustomLuceneSerializer"/>

<gfe:lucene-index id="IndexThree" lucene-service-ref="luceneService" region-path="/YetAnotherExample">
    <gfe:lucene-serializer ref="MyLuceneSerializer">
</gfe:lucene-index>
----

Of course, you may specify the `LuceneSerializer` as a anonymous, nested bean definition as well, like so...

[source,xml]
----
<gfe:lucene-index id="IndexThree" lucene-service-ref="luceneService" region-path="/YetAnotherExample">
    <gfe:lucene-serializer>
        <bean class="example.CustomLuceneSerializer"/>
    </gfe:lucene-serializer>
</gfe:lucene-index>
----

Alternatively, a developer may declare or define a `LuceneIndex` in Spring Java config,
inside a `@Configuration` class with...

[source,java]
----
@Bean(name = "Books")
@DependsOn("bookTitleIndex")
PartitionedRegionFactoryBean<Long, Book> booksRegion(Pivotal GemFireCache gemfireCache) {

    PartitionedRegionFactoryBean<Long, Book> peopleRegion =
        new PartitionedRegionFactoryBean<>();

    peopleRegion.setCache(gemfireCache);
    peopleRegion.setClose(false);
    peopleRegion.setPersistent(false);

    return peopleRegion;
}

@Bean
LuceneIndexFactoryBean bookTitleIndex(Pivotal GemFireCache gemFireCache,
        LuceneSerializer luceneSerializer) {

    LuceneIndexFactoryBean luceneIndex = new LuceneIndexFactoryBean();

    luceneIndex.setCache(gemFireCache);
    luceneIndex.setFields("title");
    luceneIndex.setLuceneSerializer(luceneSerializer);
    luceneIndex.setRegionPath("/Books");

    return luceneIndex;
}

@Bean
CustomLuceneSerializer myLuceneSerialier() {
    return new CustomeLuceneSerializer();
}
----

There are a few limitations of Pivotal GemFire's, Apache Lucene integration and support.

First, a `LuceneIndex` can only be created on an Pivotal GemFire `PARTITION` Region.

Second, all `LuceneIndexes` must be created before the Region to which the `LuceneIndex` applies.

NOTE: To help ensure that all declared `LuceneIndexes` defined in a Spring context are created before the Regions
on which they apply, SDG includes the `org.springframework.data.gemfire.config.support.LuceneIndexRegionBeanFactoryPostProcessor`.
You may register this Spring https://docs.spring.io/spring/docs/current/javadoc-api/org/springframework/beans/factory/config/BeanFactoryPostProcessor.html[`BeanFactoryPostProcessor`]
in XML config using `<bean class="org.springframework.data.gemfire.config.support.LuceneIndexRegionBeanFactoryPostProcessor"/>`
The `o.s.d.g.config.support.LuceneIndexRegionBeanFactoryPostProcessor` may only be used when using SDG XML config.
More details about Spring's `BeanFactoryPostProcessors` can be found https://docs.spring.io/spring/docs/current/spring-framework-reference/core.html#beans-factory-extension-factory-postprocessors[here].

It is possible that these Pivotal GemFire restrictions will not apply in a future release which is why
the SDG `LuceneIndexFactoryBean` API takes a reference to the Region directly as well, rather than just the Region path.

This is more ideal if think about the case in which users may want to define a `LuceneIndex` on an existing Region
with data at a later point during the application's lifecycle and as requirements demand.  Where possible, SDG strives
to adhere to strongly-typed objects.  However, for the time being, you must use the `regionPath` property
to specify the Region to which the `LuceneIndex` will be applied.

NOTE: Additional, in the example above, you will notice the presence of Spring's `@DependsOn` annotation
on the "Books" Region bean definition.  This is used to create a dependency from the "Books" Region bean
to the "bookTitleIndex" LuceneIndex bean definition ensuring that the `LuceneIndex` will be created before
the Region on which it applies.

Now that we have a `LuceneIndex` we can perform Lucene based data access operations, such as queries.

== Lucene Template Data Accessors

_Spring Data for Pivotal GemFire_ provides 2 primary templates for Lucene data access operations, depending on
how low of a level your application is prepared to deal with.

The `LuceneOperations` interface defines query operations using Pivotal GemFire
http://gemfire-95-javadocs.docs.pivotal.io/org/apache/geode/cache/lucene/package-summary.html[Lucene types].

[source,java]
----
public interface LuceneOperations {

    <K, V> List<LuceneResultStruct<K, V>> query(String query, String defaultField [, int resultLimit]
        , String... projectionFields);

    <K, V> PageableLuceneQueryResults<K, V> query(String query, String defaultField,
        int resultLimit, int pageSize, String... projectionFields);

    <K, V> List<LuceneResultStruct<K, V>> query(LuceneQueryProvider queryProvider [, int resultLimit]
        , String... projectionFields);

    <K, V> PageableLuceneQueryResults<K, V> query(LuceneQueryProvider queryProvider,
        int resultLimit, int pageSize, String... projectionFields);

    <K> Collection<K> queryForKeys(String query, String defaultField [, int resultLimit]);

    <K> Collection<K> queryForKeys(LuceneQueryProvider queryProvider [, int resultLimit]);

    <V> Collection<V> queryForValues(String query, String defaultField [, int resultLimit]);

    <V> Collection<V> queryForValues(LuceneQueryProvider queryProvider [, int resultLimit]);
}
----

NOTE: The `[, int resultLimit]` indicates that the `resultLimit` parameter is optional.

The operations in the `LuceneOperations` interface match the operations provided by the Pivotal GemFire's
http://gemfire-95-javadocs.docs.pivotal.io/org/apache/geode/cache/lucene/LuceneQuery.html[LuceneQuery] interface.
However, SDG has the added value of translating proprietary Pivotal GemFire or Apache Lucene `Exceptions`
into _Spring's_ highly consistent and expressive DAO
http://docs.spring.io/spring/docs/current/spring-framework-reference/htmlsingle/#dao-exceptions[Exception Hierarchy],
particularly as many modern data access operations involve more than single store or repository.

Additionally, SDG's `LuceneOperations` interface can shield your application from interface breaking changes
introduced by the underlying Pivotal GemFire or Apache Lucene APIs when they do and will occur.

However, it would be remorse to only offer a Lucene Data Access Object (DAO) that only uses Pivotal GemFire
and Apache Lucene data types (e.g. Pivotal GemFire's `LuceneResultStruct`), therefore SDG gives you the
`ProjectingLuceneOperations` interface to remedy these important application concerns.

[source,java]
----
public interface ProjectingLuceneOperations {

    <T> List<T> query(String query, String defaultField [, int resultLimit], Class<T> projectionType);

    <T> Page<T> query(String query, String defaultField, int resultLimit, int pageSize, Class<T> projectionType);

    <T> List<T> query(LuceneQueryProvider queryProvider [, int resultLimit], Class<T> projectionType);

    <T> Page<T> query(LuceneQueryProvider queryProvider, int resultLimit, int pageSize, Class<T> projectionType);
}
----

The `ProjectingLuceneOperations` interface primarily uses application domain object types allowing you to work with
your application data.  The `query` method variants accept a projection type and the template applies the query results
to instances of the given projection type using the _Spring Data Commons_ Projection infrastructure.

Additionally, the template wraps the paged Lucene query results in an instance of the _Spring Data Commons_
`Page` abstraction.  The same projection logic can still be applied to the results in the page and are lazily projected
as each page in the collection is accessed.

By way of example, suppose I have a class representing a `Person` like so...

[source,java]
----
class Person {

    Gender gender;

    LocalDate birthDate;

    String firstName;
    String lastName;

    ...

    String getName() {
        return String.format("%1$s %2$s", getFirstName(), getLastName());
    }
}
----

Additionally, I might have a single interface to represent people as `Customers` depending on my application view...

[source,java]
----
interface Customer {

    String getName()

}
----

If I define the following `LuceneIndex`...

[source,java]
----
@Bean
LuceneIndexFactoryBean personLastNameIndex(Pivotal GemFireCache gemfireCache) {

    LuceneIndexFactoryBean personLastNameIndex =
        new LuceneIndexFactoryBean();

    personLastNameIndex.setCache(gemfireCache);
    personLastNameIndex.setFields("lastName");
    personLastNameIndex.setRegionPath("/People");

    return personLastNameIndex;
}
----

Then it is a simple matter to query for people as either `Person` objects...

[source,java]
----
List<Person> people = luceneTemplate.query("lastName: D*", "lastName", Person.class);
----

Or as a `Page` of type `Customer`...

[source,java]
----
Page<Customer> customers = luceneTemplate.query("lastName: D*", "lastName", 100, 20, Customer.class);
----

The `Page` can then be used to fetch individual pages of the results...

[source,java]
----
List<Customer> firstPage = customers.getContent();
----

Conveniently, the _Spring Data Commons_ `Page` interface implements `java.lang.Iterable<T>` too making it very easy
to iterate over the contents.

The only restriction to the _Spring Data Commons_ Projection infrastructure is that the projection type
must be an interface.  However, it is possible to extend the provided, out-of-the-box (OOTB)
SDC Projection infrastructure and provide a custom
http://docs.spring.io/spring-data/commons/docs/current/api/org/springframework/data/projection/ProjectionFactory.html[ProjectionFactory]
that uses https://github.com/cglib/cglib[CGLIB] to generate proxy classes as the projected entity.

A custom `ProjectionFactory` can be set on a Lucene template using `setProjectionFactory(:ProjectionFactory)`.

== Annotation configuration support

Finally, _Spring Data for Pivotal GemFire_ provides Annotation configuration support for `LuceneIndexes`.
Eventually, the SDG Lucene support will find its way into the _Repository_ infrastructure extension for Pivotal GemFire
so that Lucene queries can be expressed as methods on an application `Repository` interface, much like the
http://docs.spring.io/spring-data-gemfire/docs/current/reference/html/#gemfire-repositories.executing-queries[OQL support]
today.

However, in the meantime, if you want to conveniently express `LuceneIndexes`, you can do so directly on
your application domain objects like so...

[source,java]
----
@PartitionRegion("People")
class Person {

    Gender gender;

    @Index
    LocalDate birthDate;

    String firstName;

    @LuceneIndex;
    String lastName;

    ...
}
----

You must use SDG's Annotation configuration support along with the `@EnableEntityDefineRegions` and `@EnableIndexing`
Annotations to enable this feature...

[source,java]
----
@PeerCacheApplication
@EnableEntityDefinedRegions
@EnableIndexing
class ApplicationConfiguration {

  ...
}
----

NOTE: Keep in mind that `LuceneIndexes` can only be created on Apache Geode Servers since `LuceneIndexes` only apply
to `PARTTION` Regions.

Given our definition of the `Person` class above, the SDG Annotation configuration support
will find the `Person` entity class definition, determine that people will be stored in
a `PARTITION` Region called "People" and that the `Person` will have an OQL `Index` on `birthDate`
along with a `LuceneIndex` on `lastName`.

More will be described with this feature in subsequent releases.
