From 6dcf760129f523f6b079229341e29d606e6ddf83 Mon Sep 17 00:00:00 2001 From: "J. Brisbin" Date: Thu, 16 Dec 2010 14:13:43 -0600 Subject: [PATCH] Lots more documentation of Map/Reduce, Link Walking, and bucket schema updating. --- src/docbkx/reference/riak.xml | 164 +++++++++++++++++++++++++++++++++- 1 file changed, 163 insertions(+), 1 deletion(-) diff --git a/src/docbkx/reference/riak.xml b/src/docbkx/reference/riak.xml index 4d1d6dfe9..9a50349c1 100644 --- a/src/docbkx/reference/riak.xml +++ b/src/docbkx/reference/riak.xml @@ -161,7 +161,7 @@ public class Example { // If your entry is Content-Type: application/octet-stream, // you can access the raw bytes. - byte[] b = riak.getAsBytes(bucket, key); // No conversion at all + byte[] b = riak.getAsBytes(bucket, key); } } @@ -170,6 +170,168 @@ public class Example { + + +
+ Map/Reduce + + Riak supports Map/Reduce functionality in a couple different ways. You can specify the Javascript source to execute (termed "anonymous" Javascript), you can reference some Javascript already stored in Riak at a specfic bucket and key, or you can reference an Erlang module and function. The Map/Reduce support in SDKV covers all these bases by giving you meaningful abstractions over the Map/Reduce job that represent the various aspects of the Map/Reduce process. + + At the highest level, every Map/Reduce request is represented by a MapReduceJob. The MapReduceJob represents the inputs, the phases, and the optional arg to send to Riak to execute the Map/Reduce job. The toJson method is responsible for serializing the entire job into the appropriate JSON data to send to Riak. + +
+ Specifying Inputs + + Riak will accept either a string denoting the bucket in which to get the list of keys to operate on, or a List of Lists denoting the bucket/key pairs to operate on while executing this Map/Reduce job. If you call the addInputs method on the job passing a List with a single string entry, the job will assume you want to operate on an entire bucket. Otherwise, you'll need to pass a multi-dimensional List of bucket/key pairs. + + To operate on an entire bucket: + bucket = new ArrayList() {{ + add("mybucket"); +}}; +job.addInputs(bucket); // Will M/R entire bucket + ]]> + + + To operate on a set of keys: + pair = new ArrayList() {{ + add("mybucket"); + add("mykey"); +}}; +List keys = new ArrayList() {{ + add(pair); +}}; +job.addInputs(keys); // Will M/R only specified keys + ]]> + +
+ +
+ Defining Phases + + Map/Reduce operations in Riak are broken up into phases. Phases contain a MapReduceOperation. There are currently two implementations to handle Javascript or Erlang M/R operations: JavascriptMapReduceOperation and ErlangMapReduceOperation. + + An example Map/Reduce job defining a single "map" phase defined in anonymous Javascript might look like this: + bucket = new ArrayList() {{ + add("mybucket"); +}}; + +job.addInputs(bucket); // M/R the entire bucket + +MapReduceOperation mapOper = new JavascriptMapReduceOperation("function(v){ ...M/R function body... }"); +MapReducePhase mapPhase = new RiakMapReducePhase("map", "javascript", mapOper); + +job.addPhase(mapPhase); + ]]> + +
+ +
+ Executing and Working with the Result + + To execute a configured job on your Riak server, use either the synchronous execute or asynchronous submit methods of your configured RiakTemplate: + + + +...or... + +List o = riak.execute(job, MyPojo.class); // Coerce to given type + +...or... + +Future> f = riak.submit(job); // Job runs in a separate thread + ]]> + +
+
+ +
+ Managing Bucket Properties + + It's sometimes useful to manage settings like the Quality-of-Service parameters w and dw (write and durable write thresholds) and the n_val setting at the bucket level. It's also possible to list the keys in a particular bucket by calling the getBucketSchema method, passing true as the second parameter, which tells the RiakTemplate to list the keys. + + To list the keys in a bucket, you would do something like this: + + schema = riak.getBucketSchema("mybucket"); +List keys = schema.get("keys") +for(String key : keys) { + ...do something with each key... +} + ]]> + + + To update the bucket settings, pass a Map of properties: + + props = new HashMap(); +props.put("n_val", 6); +props.put("dw", 3); + +riak.updateBucketSchema("mybucket", props); + ]]> + + Only the properties specified in the passed-in Map will be updated. Properties that have already been set in previous operations and not specified in this operation will be unaffected. + + +
+
Working with streams