开发者

MongoDB MapReduce: Global variables within map function instance?

开发者 https://www.devze.com 2023-01-03 04:32 出处:网络
I\'ve written a MapReduce in MongoDB and would like to use a global variable as a cache to write to/read from. I know it is not possible to have global variables across map function instances - I just

I've written a MapReduce in MongoDB and would like to use a global variable as a cache to write to/read from. I know it is not possible to have global variables across map function instances - I just want a gl开发者_JS百科obal variable within each function instance. This type of functionality exists in Hadoop's MapReduce so I was expecting it to be there in MongoDB. But the following does not seem to work:

var cache = {}; // Does not seem to work!
function () {
  var hashValue = this.varValue1 + this.varValue2;
  if(typeof(cache[hashValue])!= 'undefined') {
    // Do nothing, we've processed at least one input record with this hash
  } else {
    // Process the input record
    // Cache the record
    cache[hashValue] = '1';
  }
}

Is this not allowed in MongoDB's MapReduce implementation, or am I doing something wrong in JavaScript (not experienced in JS)?


Looking at the docs, I'm finding the following:

db.runCommand(
 { mapreduce : <collection>,
   map : <mapfunction>,
   reduce : <reducefunction>
   [, scope : <object where fields go into javascript global scope >]
 }
);

I think that "scope" variable is what you need.

There's a test / example on Github that uses the "scope" variable.

I'm still new to this stuff, but hopefully that's enough to get you started.


As Gates VP said, you need to add cache into global scope. So, to provide complete answer, considering your script, this is what you'll need to do:

db.runCommand(
 { mapreduce : <your collection>,
   map : <your map function, or reference to it>,
   reduce : <your reduce function, or reference to it>,
   scope : { cache : {} }
 }
);

The command will inject contents of the 'scope' object parameter into your global context. The caching then will work per how you are using it in your map function. I've tested this.

0

精彩评论

暂无评论...
验证码 换一张
取 消