[[function-annotations]]
= Annotation Support for Function Execution

== Introduction

_Spring Data for Pivotal GemFire_ includes annotation support to simplify working with Pivotal GemFire
http://geode.apache.org/docs/guide/11/developing/function_exec/chapter_overview.html[Function Execution].
Under-the-hood, the Pivotal GemFire API provides classes to implement and register Pivotal GemFire
http://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/execute/Function.html[Functions]
that are deployed on Pivotal GemFire servers, which may then be invoked by other peer member applications
or remotely from cache clients.

Functions can execute in parallel, distributed among multiple Pivotal GemFire servers in the cluster, aggregating results
with the map-reduce pattern that are sent back to the caller.  Functions can also be targeted to run on a single server
or Region.  The Pivotal GemFire API supports remote execution of Functions targeted using various predefined scopes:
on Region, on members [in groups], on servers, etc.  The implementation and execution of remote Functions,
as with any RPC protocol, requires some boilerplate code.

_Spring Data for Pivotal GemFire_, true to _Spring's_ core value proposition, aims to hide the mechanics of remote Function execution
and allow developers to focus on core POJO programming and business logic.  To this end, _Spring Data for Pivotal GemFire_ introduces
annotations to declaratively register public methods of a POJO class as Pivotal GemFire Functions along with the ability to
invoke registered Functions [remotely] via annotated interfaces.

== Implementation vs Execution

There are two separate concerns to address implementation and execution.

First is Function implementation (server-side), which must interact with the
http://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/execute/FunctionContext.html[FunctionContext]
to access the invocation arguments,
http://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/execute/ResultSender.html[ResultsSender]
as well as other execution context information. The Function implementation typically accesses the Cache and/or Regions
and is registered with the
http://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/execute/FunctionService.html[FunctionService]
under a unique Id.

A cache client application invoking a Function does not depend on the implementation.  To invoke a Function,
the application instantiates an
http://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/execute/Execution.html[Execution]
providing the Function ID, invocation arguments and the Function target, which defines its scope:
Region, server, servers, member or members.  If the Function produces a result, the invoker uses a
http://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/execute/ResultCollector.html[ResultCollector]
to aggregate and acquire the execution results.  In certain cases, a custom `ResultCollector` implementation
is required and may be registered with the `Execution`.

NOTE: 'Client' and 'Server' are used here in the context of Function execution, which may have a different meaning
than client and server in Pivotal GemFire's client-server topology.  While it is common for an application using a `ClientCache`
to invoke a Function on one or more Pivotal GemFire servers in a cluster, it is also possible to execute Functions
in a peer-to-peer (P2P) configuration, where the application is a member of the cluster hosting a peer `Cache`.
Keep in mind that a peer member cache application is subject to all the same constraints of being a peer member
of the cluster.

[[function-implementation]]
== Implementing a Function

Using Pivotal GemFire APIs, the `FunctionContext` provides a runtime invocation context that includes the client's
calling arguments and a `ResultSender` implementation to send results back to the client.  Additionally,
if the Function is executed on a Region, the `FunctionContext` is actually an instance of `RegionFunctionContext`,
which provides additional information such as the target Region on which the Function was invoked
and any Filter (set of specific keys) associated with the `Execution`, etc.  If the Region is a PARTITION Region,
the Function should use the `PartitionRegionHelper` to extract only the local data.

Using _Spring_, a developer can write a simple POJO and use the _Spring_ container to bind one or more of it's
public methods to a Function.  The signature for a POJO method intended to be used as a Function must generally
conform to the client's execution arguments.  However, in the case of a Region execution, the Region data
may also be provided (presumably the data held in the local partition if the Region is a PARTITION Region).
Additionally, the Function may require the Filter that was applied, if any.  This suggests that the client and server
share a contract for the calling arguments but that the method signature may include additional parameters
to pass values provided by the `FunctionContext`.  One possibility is for the client and server to share
a common interface, but this is not strictly required.  The only constraint is that the method signature includes
the same sequence of calling arguments with which the Function was invoked after the additional parameters
are resolved.

For example, suppose the client provides a String and int as the calling arguments. These are provided
in the `FunctionContext` as an array:

`Object[] args = new Object[] { "test", 123 };`

Then, the _Spring_ container should be able to bind to any method signature similar to the following.
Let's ignore the return type for the moment:

[source,java]
----
public Object method1(String s1, int i2) {...}
public Object method2(Map<?, ?> data, String s1, int i2) {...}
public Object method3(String s1, Map<?, ?> data, int i2) {...}
public Object method4(String s1, Map<?, ?> data, Set<?> filter, int i2) {...}
public void method4(String s1, Set<?> filter, int i2, Region<?,?> data) {...}
public void method5(String s1, ResultSender rs, int i2);
public void method6(FunctionContest context);
----

The general rule is that once any additional arguments, i.e. Region data and Filter, are resolved,
the remaining arguments must correspond exactly, in order and type, to the expected Function method parameters.
The method's return type must be void or a type that may be serialized (either as a `java.io.Serializable`,
`DataSerializable` or `PdxSerializable`). The latter is also a requirement for the calling arguments.
The Region data should normally be defined as a Map, to facilitate unit testing, but may also be of type Region
if necessary. As shown in the example above, it is also valid to pass the `FunctionContext` itself,
or the `ResultSender`, if you need to control how the results are returned to the client.

=== Annotations for Function Implementation

The following example illustrates how SDG's Function annotations are used to expose POJO methods
as Pivotal GemFire Functions:

[source,java]
----
@Component
public class ApplicationFunctions {

   @GemfireFunction
   public String function1(String value, @RegionData Map<?, ?> data, int i2) { ... }

   @GemfireFunction("myFunction", batchSize=100, HA=true, optimizedForWrite=true)
   public List<String> function2(String value, @RegionData Map<?, ?> data, int i2, @Filter Set<?> keys) { ... }

   @GemfireFunction(hasResult=true)
   public void functionWithContext(FunctionContext functionContext) { ... }

}
----

Note, the class itself must be registered as a _Spring_ bean and each Pivotal GemFire Function is annotated
with `@GemfireFunction`.  In this example, _Spring's_ `@Component` annotation was used, but you may register the bean
by any method supported by _Spring_ (e.g. XML configuration or with a Java configuration class using _Spring Boot_).
This allows the _Spring_ container to create an instance of this class and wrap it in a
http://docs.spring.io/spring-data-gemfire/docs/current/api/org/springframework/data/gemfire/function/PojoFunctionWrapper.html[PojoFunctionWrapper].
_Spring_ creates a wrapper instance for each method annotated with `@GemfireFunction`.  Each wrapper instance shares
the same target object instance to invoke the corresponding method.

TIP: The fact that the POJO Function class is a _Spring_ bean may offer other benefits since it shares
the `ApplicationContext` with Pivotal GemFire components such as the Cache and Regions. These may be injected into the class
if necessary.

_Spring_ creates the wrapper class and registers the Function(s) with Pivotal GemFire's Function Service. The Function id used
to register the Functions must be unique.  Using convention it defaults to the simple (unqualified) method name.
The name can be explicitly defined using the `id` attribute of the `@GemfireFunction` annotation.
The `@GemfireFunction` annotation also provides other configuration attributes, `HA` and `optimizedForWrite`,
which correspond to properties defined by Pivotal GemFire's
http://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/execute/Function.html[Function] interface.
If the method's return type is void, then the `hasResult` property is automatically set to `false`;
otherwise, if the method returns a value the `hasResult` attributes is set to `true`.

Even for `void` return types, the annotation's `hasResult` attribute can be set to `true` to override this convention,
as shown in the `functionWithContext` method above.  Presumably, the intention is to use the `ResultSender` directly
to send results to the caller.

The `PojoFunctionWrapper` implements Pivotal GemFire's `Function` interface, binds method parameters and invokes the target method
in its `execute()` method.  It also sends the method's return value using the `ResultSender`.

=== Batching Results

If the return type is an array or Collection, then some consideration must be given to how the results are returned.
By default, the `PojoFunctionWrapper` returns the entire array or Collection at once.  If the number of elements
in the array or Collection quite is large, it may incur a performance penalty.  To divide the payload into smaller,
more maneable chunks, you can set the `batchSize` attribute, as illustrated in `function2`, above.

TIP: If you need more control of the `ResultSender`, especially if the method itself would use too much memory
to create the Collection, you can pass the `ResultSender`, or access it via the `FunctionContext` and use it directly
within the method to sends results back to the caller.

=== Enabling Annotation Processing

In accordance with _Spring_ standards, you must explicitly activate annotation processing for `@GemfireFunction`
annotations.

Using XML:

[source,xml]
----
<gfe:annotation-driven/>
----

Or by annotating a Java configuration class:

[source,java]
----
@Configuration
@EnableGemfireFunctions
class ApplicationConfiguration { .. }
----

[[function-execution]]
== Executing a Function

A process invoking a remote Function needs to provide the Function's ID, calling arguments, the execution target
(onRegion, onServers, onServer, onMember, onMembers) and optionally, a Filter set.  Using _Spring Data for Pivotal GemFire_,
all a developer need do is define an interface supported by annotations.  _Spring_ will create a dynamic proxy
for the interface, which will use the `FunctionService` to create an `Execution`, invoke the `Execution` and coerce
the results to the defined return type, if necessary.  This technique is very similar to the way
_Spring Data for Pivotal GemFire's Repository extension_ works, thus some of the configuration and concepts should be familiar.
Generally, a single interface definition maps to multiple Function executions, one corresponding to each method
defined in the interface.

=== Annotations for Function Execution

To support client-side Function execution, the following SDG Function annotations are provided: `@OnRegion`,
`@OnServer`, `@OnServers`, `@OnMember`, `@OnMembers`.  These annotations correspond to the `Execution` implementations
prodided by Pivotal GemFire's
http://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/execute/FunctionService.html[FunctionService].
Each annotation exposes the appropriate attributes. These annotations also provide an optional
`resultCollector` attribute whose value is the name of a _Spring_ bean implementing the
http://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/execute/ResultCollector.html[ResultCollector]
to use for the execution.

CAUTION: The proxy interface binds all declared methods to the same execution configuration.  Although, it is expected
that single method interfaces will be common, all methods in the interface are backed by the same proxy instance
and therefore all share the same configuration.

Here are a few examples:

[source,java]
----
@OnRegion(region="SomeRegion", resultCollector="myCollector")
public interface FunctionExecution {

    @FunctionId("function1")
    String doIt(String s1, int i2);

    String getString(Object arg1, @Filter Set<Object> keys);

}
----

By default, the Function ID is the simple (unqualified) method name.  The `@FunctionId` annotation can be used
to bind this invocation to a different Function ID.

=== Enabling Annotation Processing

The client-side uses _Spring's_ classpath component scanning capability to discover annotated interfaces. To enable
Function execution annotation processing in XML:

[source,xml]
----
<gfe-data:function-executions base-package="org.example.myapp.gemfire.functions"/>
----

The `function-executions` element is provided in the `gfe-data` namespace. The `base-package` attribute is required
to avoid scanning the entire classpath. Additional filters are provided as described in the _Spring_
http://docs.spring.io/spring/docs/current/spring-framework-reference/htmlsingle/#beans-scanning-filters[reference documentation].

Optionally, a developer can annotate her Java configuration class:

[source,java]
----
@EnableGemfireFunctionExecutions(basePackages = "org.example.myapp.gemfire.functions")
----

[[function-execution-programmatic]]
== Programmatic Function Execution

Using the Function execution annotated interface defined in the previous section, simply auto-wire your interface
into an application bean that will invoke the Function:

[source,java]
----
@Component
public class MyApplication {

    @Autowired
    FunctionExecution functionExecution;

    public void doSomething() {
         functionExecution.doIt("hello", 123);
    }
}
----

Alternately, you can use a Function execution template directly.  For example, `GemfireOnRegionFunctionTemplate`
creates an `onRegion` Function `Execution`.

.Using the `GemfireOnRegionFunctionTemplate`
====
[source,java]
----
Set<?, ?> myFilter = getFilter();
Region<?, ?> myRegion = getRegion();
GemfireOnRegionOperations template = new GemfireOnRegionFunctionTemplate(myRegion);
String result = template.executeAndExtract("someFunction", myFilter, "hello", "world", 1234);
----
====

Internally, Function `Executions` always return a `List`. `executeAndExtract` assumes a singleton `List`
containing the result and will attempt to coerce that value into the requested type.  There is also
an `execute` method that returns the `List` as is.  The first parameter is the Function ID.
The Filter argument is optional.  The following arguments are a variable argument `List`.

[[function-execution-pdx]]
== Function Execution with PDX

When using _Spring Data for Pivotal GemFire's_ Function annotation support combined with Pivotal GemFire's
http://geode.apache.org/docs/guide/11/developing/data_serialization/gemfire_pdx_serialization.html[PDX Serialization],
there are a few logistical things to keep in mind.

As explained above, and by way of example, typically developers will define Pivotal GemFire Functions using POJO classes
annotated with Spring Data for Pivotal GemFire
http://docs.spring.io/spring-data-gemfire/docs/current/api/org/springframework/data/gemfire/function/annotation/package-summary.html[Function annotations]
like so...

[source,java]
----
public class OrderFunctions {

  @GemfireFunction(...)
  Order process(@RegionData data, Order order, OrderSource orderSourceEnum, Integer count) { ... }

}
----

NOTE: The Integer type, count parameter is arbitrary as is the separation of the `Order` class and `OrderSource` Enum,
which might be logical to combine. However, the arguments were setup this way to demonstrate the problem with
Function executions in the context of PDX.

Your `Order` and `OrderSource` enum might be as follows...

[source,java]
----
public class Order ... {

  private Long orderNumber;
  private Calendar orderDateTime;
  private Customer customer;
  private List<Item> items

  ...
}


public enum OrderSource {
  ONLINE,
  PHONE,
  POINT_OF_SALE
  ...
}
----

Of course, a developer may define a Function `Execution` interface to call the 'process' Pivotal GemFire Server Function...

[source,java]
----
@OnServer
public interface OrderProcessingFunctions {
  Order process(Order order, OrderSource orderSourceEnum, Integer count);
}
----

Clearly, this `process(..)` `Order` Function is being called from a client-side with a `ClientCache`
(i.e. `<gfe:client-cache/>`) based application.  This implies that the Function arguments must also be serializable.
The same is true when invoking peer-to-peer member Functions (e.g. `@OnMember(s)) between peers in the cluster.
Any form of `distribution` requires the data transmitted between client and server, or peers, to be serialized.

Now, if the developer has configured Pivotal GemFire to use PDX for serialization (instead of Java serialization, for instance)
it is common for developers to also set the `pdx-read-serialized` attribute to *true* in their configuration
of the Pivotal GemFire server(s)...

[source,xml]
----
<gfe:cache ... pdx-read-serialized="true"/>
----

Or from a Pivotal GemFire cache client application...

[source,xml]
----
<gfe:client-cache ... pdx-read-serialized="true"/>
----

This causes all values read from the cache (i.e. Regions) as well as information passed between client and servers,
or peers, to remain in serialized form, including, but not limited to, Function arguments.

Pivotal GemFire will only serialize application domain object types that you have specifically configured (registered),
with either Pivotal GemFire's
http://gemfire-90-javadocs.docs.pivotal.io/org/apache/geode/pdx/ReflectionBasedAutoSerializer.html[ReflectionBasedAutoSerializer],
or specifically (and recommended) using a "custom" Pivotal GemFire
http://gemfire-90-javadocs.docs.pivotal.io/org/apache/geode/pdx/PdxSerializer.html[PdxSerializer].  If you are using
_Spring Data for Pivotal GemFire's_ Repository extension to _Spring Data Common's_ Repository abstraction and infrastructure,
you might even want to consider using _Spring Data for Pivotal GemFire's_
http://docs.spring.io/spring-data-gemfire/docs/current/api/org/springframework/data/gemfire/mapping/MappingPdxSerializer.html[MappingPdxSerializer],
which uses a entity's mapping meta-data to determine data from the application domain object that will be serialized
to the PDX instance.

What is less than apparent, though, is that Pivotal GemFire automatically handles Java Enum types regardless of whether they are
explicitly configured or not (i.e. registered with a `ReflectionBasedAutoSerializer` using a regex pattern
and the `classes` parameter, or are handled by a "custom" Pivotal GemFire `PdxSerializer`), despite the fact that Java Enums
implement `java.io.Serializable`.

So, when a developer sets `pdx-read-serialized` to *true* on Pivotal GemFire Servers where the Pivotal GemFire Functions
(including Spring Data for Pivotal GemFire Function annotated POJO classes) are registered, then the developer
may encounter surprising behavior when invoking the Function `Execution`.

What the developer may pass as arguments when invoking the Function is...

[source,java]
----
orderProcessingFunctions.process(new Order(123, customer, Calendar.getInstance(), items), OrderSource.ONLINE, 400);
----

But, what the Pivotal GemFire Function on the Server gets is...

[source,java]
----
process(regionData, order:PdxInstance, :PdxInstanceEnum, 400);
----

The `Order` and `OrderSource` have been passed to the Function as
http://gemfire-90-javadocs.docs.pivotal.io/org/apache/geode/pdx/PdxInstance.html[PDX instances].
Again, this is all because `pdx-read-serialized` is set to *true*, which may be necessary in cases where
the Pivotal GemFire Servers are interacting with multiple different clients (e.g. Java, native clients, such as C++/C#, etc).

This flies in the face of _Spring Data for Pivotal GemFire's_ "strongly-typed", Function annotated POJO class method signatures,
as the developer is expecting application domain object types, not PDX serialized instances.

So, _Spring Data for Pivotal GemFire_ includes enhanced Function support to automatically convert method arguments passed to
the Function that are of type PDX to the desired application domain object types defined by the Function method's
parameter types.

However, this also requires the developer to explicitly register a Pivotal GemFire `PdxSerializer` on the Pivotal GemFire Servers
where _Spring Data for Pivotal GemFire_ Function annotated POJOs are registered and used, e.g. ...

[source,java]
----
<bean id="customPdxSerializer" class="x.y.z.gemfire.serialization.pdx.MyCustomPdxSerializer"/>

<gfe:cache ... pdx-serializer-ref="customPdxSerializeer" pdx-read-serialized="true"/>
----

Alternatively, a developer my use Pivotal GemFire's
http://gemfire-90-javadocs.docs.pivotal.io/org/apache/geode/pdx/ReflectionBasedAutoSerializer.html[ReflectionBasedAutoSerializer]
for convenience.  Of course, it is recommended that you use a "custom" `PdxSerializer` where possible to maintain
finer grained control over your serialization strategy.

Finally, _Spring Data for Pivotal GemFire_ is careful not to convert your Function arguments if you treat your Function arguments
generically, or as one of Pivotal GemFire's PDX types...

[source,java]
----
@GemfireFunction
public Object genericFunction(String value, Object domainObject, PdxInstanceEnum enum) {
 ...
}
----

_Spring Data for Pivotal GemFire_ only converts PDX type data to the corresponding application domain types if and only if
the corresponding application domain types are on the classpath the the Function annotated POJO method expects it.

For a good example of "custom", "composed" application-specific Pivotal GemFire `PdxSerializers` as well as appropriate
POJO Function parameter type handling based on the method signatures, see Spring Data for Pivotal GemFire's
https://github.com/spring-projects/spring-data-gemfire/blob/2.0.0.M2/src/test/java/org/springframework/data/gemfire/function/ClientCacheFunctionExecutionWithPdxIntegrationTest.java[ClientCacheFunctionExecutionWithPdxIntegrationTest] class.
