Spring/spring-data-mongodb
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Polished map-reduce tests and formatted documentation with XMLEditor.

Browse Source
This commit is contained in:
Oliver Gierke
2011-09-04 14:00:09 +02:00
parent 745e1f313d
commit 091246a9aa
2 changed files with 67 additions and 66 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
spring-data-mongodb/src/test/java/org/springframework/data/mongodb/core/mapreduce/MapReduceOptionsTests.java
View File

@@ -15,20 +15,13 @@
*/
package org.springframework.data.mongodb.core.mapreduce;
import static org.junit.Assert.*;
import static org.hamcrest.CoreMatchers.*;
import org.junit.Test;
import org.springframework.data.mongodb.core.query.Criteria;
import com.mongodb.BasicDBObject;
import com.mongodb.DBObject;
public class MapReduceOptionsTests {
@Test
public void testFinalize() {
MapReduceOptions o = new MapReduceOptions().finalizeFunction("code");
new MapReduceOptions().finalizeFunction("code");
}
}

124
src/docbkx/reference/mongodb.xml
View File

@@ -242,7 +242,7 @@ public class MongoApp {
</itemizedlist>
<section id="mongodb-required-jars">
<title>Required Jars</title>
@@ -1385,8 +1385,8 @@ import static org.springframework.data.document.mongodb.query.Update;
<para>GeoSpatial queries are also supported and are described more in the
section <link linkend="mongo.geospatial">GeoSpatial Queries</link>.</para>
<para>Map-Reduce operations are also supported and are described more in the
section <link linkend="mongo.mapreduce">Map-Reduce</link>.</para>
<para>Map-Reduce operations are also supported and are described more in
the section <link linkend="mongo.mapreduce">Map-Reduce</link>.</para>
<section id="mongodb-template-query">
<title>Querying documents in a collection</title>
@@ -1841,65 +1841,71 @@ GeoResults&lt;Restaurant&gt; = operations.geoNear(query, Restaurant.class);</pro
<section id="mongo.mapreduce">
<title>Map-Reduce</title>
<para>You can query MongoDB using Map-Reduce which is useful for batch processing, data aggregation, and
for when the query language doesn't fulfill your needs. Spring provides integration with MongoDB's map reduce
by providing methods on MongoOperations to simplify the creation and execution of Map-Reduce operations.
It also integrates
with Spring's <ulink url="http://static.springsource.org/spring/docs/3.0.x/spring-framework-reference/html/resources.html">Resource abstraction</ulink>
abstraction. This will let you place your JavaScript files on the file system, classpath, http server or any other Spring Resource implementation and
then reference the JavaScript resources via an easy URI style syntax, e.g. 'classpath:reduce.js;.
Externalizing JavaScript code in files is preferable to embedding them as Java strings in your code. You can still pass JavaScript code
as Java strings if you prefer.
</para>
<para>You can query MongoDB using Map-Reduce which is useful for batch
processing, data aggregation, and for when the query language doesn't
fulfill your needs. Spring provides integration with MongoDB's map reduce
by providing methods on MongoOperations to simplify the creation and
execution of Map-Reduce operations. It also integrates with Spring's
<ulink
url="http://static.springsource.org/spring/docs/3.0.x/spring-framework-reference/html/resources.html">Resource
abstraction</ulink> abstraction. This will let you place your JavaScript
files on the file system, classpath, http server or any other Spring
Resource implementation and then reference the JavaScript resources via an
easy URI style syntax, e.g. 'classpath:reduce.js;. Externalizing
JavaScript code in files is preferable to embedding them as Java strings
in your code. You can still pass JavaScript code as Java strings if you
prefer.</para>
<section id="mongo.mapreduce.example" lang="">
<title>Example Usage</title>
<para>To understand how to perform Map-Reduce operations an example from the book 'MongoDB - The definitive guide' is used. In this example
we will create three documents that have the values [a,b], [b,c], and [c,d] respectfully. The values in each document are associated with the key 'x' as shown below.
For this example assume these documents are in the collection named "jmr1".
<programlisting>{ "_id" : ObjectId("4e5ff893c0277826074ec533"), "x" : [ "a", "b" ] }
<para>To understand how to perform Map-Reduce operations an example from
the book 'MongoDB - The definitive guide' is used. In this example we
will create three documents that have the values [a,b], [b,c], and [c,d]
respectfully. The values in each document are associated with the key
'x' as shown below. For this example assume these documents are in the
collection named "jmr1". <programlisting>{ "_id" : ObjectId("4e5ff893c0277826074ec533"), "x" : [ "a", "b" ] }
{ "_id" : ObjectId("4e5ff893c0277826074ec534"), "x" : [ "b", "c" ] }
{ "_id" : ObjectId("4e5ff893c0277826074ec535"), "x" : [ "c", "d" ] }
</programlisting>
A map function that will count the occurance of each letter in the array for each document is shown below
<programlisting language="java">function () {
</programlisting> A map function that will count the occurance of each letter
in the array for each document is shown below <programlisting
language="java">function () {
for (var i = 0; i &lt; this.x.length; i++) {
emit(this.x[i], 1);
}
}
</programlisting>
The reduce function that will sum up the occurance of each letter across all the documents is shown below
<programlisting language="java">function (key, values) {
</programlisting> The reduce function that will sum up the occurance of each
letter across all the documents is shown below <programlisting
language="java">function (key, values) {
var sum = 0;
for (var i = 0; i &lt; values.length; i++)
sum += values[i];
return sum;
}
</programlisting>
Executing this will result in a collection as shown below.
<programlisting>
</programlisting> Executing this will result in a collection as shown below.
<programlisting>
{ "_id" : "a", "value" : 1 }
{ "_id" : "b", "value" : 2 }
{ "_id" : "c", "value" : 2 }
{ "_id" : "d", "value" : 1 }
</programlisting>
Assuming that the map and reduce functions are located in map.js and reduce.js and bundled in your jar so they are available on the classpath, you
can execute a map-reduce operation and obtain the results as shown below
<programlisting language="java">
MapReduceResults&lt;ValueObject> results = mongoOperations.mapReduce("jmr1", "classpath:map.js", "classpath:reduce.js", ValueObject.class);
</programlisting> Assuming that the map and reduce functions are located in
map.js and reduce.js and bundled in your jar so they are available on
the classpath, you can execute a map-reduce operation and obtain the
results as shown below <programlisting language="java">
MapReduceResults&lt;ValueObject&gt; results = mongoOperations.mapReduce("jmr1", "classpath:map.js", "classpath:reduce.js", ValueObject.class);
for (ValueObject valueObject : results) {
System.out.println(valueObject);
}
</programlisting>
The output of the above code is
<programlisting>
</programlisting> The output of the above code is <programlisting>
ValueObject [id=a, value=1.0]
ValueObject [id=b, value=2.0]
ValueObject [id=c, value=2.0]
ValueObject [id=d, value=1.0]
</programlisting>
The MapReduceResults class implements <classname>Iterable</classname> and provides access to the raw output, as well as timing and count statisticas. The <classname>ValueObject</classname> class is simply
<programlisting language="java">
</programlisting> The MapReduceResults class implements
<classname>Iterable</classname> and provides access to the raw output,
as well as timing and count statisticas. The
<classname>ValueObject</classname> class is simply <programlisting
language="java">
public class ValueObject {
private String id;
@@ -1924,30 +1930,32 @@ public class ValueObject {
}
}
</programlisting>
By default the output type of INLINE is used so you don't have to specify an output collection. To specify additional map-reduce options use an overloaded method
that takes an additional <classname>MapReduceOptions</classname> argument. The class <classname>MapReduceOptions</classname> has a fluent API so adding additional options can be done
in a very compact syntax. Here an example that sets the output collection to "jmr1_out". Note that setting only the output collection assumes a
default output type of REPLACE.
<programlisting language="java">
MapReduceResults&lt;ValueObject> results = mongoOperations.mapReduce("jmr1", "classpath:map.js", "classpath:reduce.js",
</programlisting> By default the output type of INLINE is used so you don't
have to specify an output collection. To specify additional map-reduce
options use an overloaded method that takes an additional
<classname>MapReduceOptions</classname> argument. The class
<classname>MapReduceOptions</classname> has a fluent API so adding
additional options can be done in a very compact syntax. Here an example
that sets the output collection to "jmr1_out". Note that setting only
the output collection assumes a default output type of REPLACE.
<programlisting language="java">
MapReduceResults&lt;ValueObject&gt; results = mongoOperations.mapReduce("jmr1", "classpath:map.js", "classpath:reduce.js",
new MapReduceOptions().outputCollection("jmr1_out"), ValueObject.class);
</programlisting>
There is also a static import <literal>import static org.springframework.data.mongodb.core.mapreduce.MapReduceOptions.options;</literal> that can be used to make the syntax slightly more compact
<programlisting language="java">
MapReduceResults&lt;ValueObject> results = mongoOperations.mapReduce("jmr1", "classpath:map.js", "classpath:reduce.js",
</programlisting> There is also a static import <literal>import static
org.springframework.data.mongodb.core.mapreduce.MapReduceOptions.options;</literal>
that can be used to make the syntax slightly more compact
<programlisting language="java">
MapReduceResults&lt;ValueObject&gt; results = mongoOperations.mapReduce("jmr1", "classpath:map.js", "classpath:reduce.js",
options().outputCollection("jmr1_out"), ValueObject.class);
</programlisting>
You can also specify a query to reduce the set of data that will be used to feed into the map-reduce operation. This will remove the document that contains [a,b] from consideration for map-reduce operations.
<programlisting language="java">
</programlisting> You can also specify a query to reduce the set of data that
will be used to feed into the map-reduce operation. This will remove the
document that contains [a,b] from consideration for map-reduce
operations. <programlisting language="java">
Query query = new Query(where("x").ne(new String[] { "a", "b" }));
MapReduceResults&lt;ValueObject> results = mongoOperations.mapReduce(query, "jmr1", "classpath:map.js", "classpath:reduce.js",
MapReduceResults&lt;ValueObject&gt; results = mongoOperations.mapReduce(query, "jmr1", "classpath:map.js", "classpath:reduce.js",
options().outputCollection("jmr1_out"), ValueObject.class);
</programlisting>
Note that you can specify additional limit and sort values as well on the query but not skip values.
</para>
</programlisting> Note that you can specify additional limit and sort values
as well on the query but not skip values.</para>
</section>
</section>