Meteor 1.0 - Mongo asks for variables as key, including $ inc
I am working with a large dataset that needs to be efficient with its Mongo queries. The application uses the Ford-Fulkerson algorithm to calculate recommendations and runs in polynomial time, so efficiency is extremely important. The syntax is ES6, but everything is basically the same.
This is an approximation of the data I am working with. An array of elements and one element are mapped to other elements:
let items = ["pen", "marker", "crayon", "pencil"];
let match = "sharpie";
In the end, we'll iterate over more match
and increase the pairing weight by 1. So, after passing the function, my ideal data looks like this:
{
sharpie: {
pen: 1,
marker: 1,
crayon: 1,
pencil: 1
}
}
To further clarify, the value next to each key is weight
that relationship, that is, the number of times these elements have been paired together. I would like this to happen:
// For each in the items array, check to see if the pairing already
// exists. If it does, increment. If it does not, create it.
_.each(items, function(item, i) {
Database.upsert({ match: { $exist: true }}, { match: { $inc: { item: 1 } } });
})
The problem, of course, is that Mongo does not allow parenthesis notation, nor does it allow variable names as keys ( match
). Another problem, as I found out, is that Mongo also has problems with deeply nested $inc
( 'The dollar ($) prefixed field \'$inc\' in \'3LhmpJMe9Es6r5HLs.$inc\' is not valid for storage.' }
) statements .
Is there anything I can do to make this as few requests as possible? I am open to suggestions.
EDIT
I tried to create objects to pass to Mongo query:
_.each(items, function(item, i) {
let selector = {};
selector[match] = {};
selector[match][item] = {};
let modifier = {};
modifier[match] = {};
modifier[match]["$inc"] = {};
modifier[match]["$inc"][item] = 1
Database.upsert(selector, modifier);
Unfortunately it still doesn't work. $inc
splits the request and it won't let me go more than one level to change something.
Decision
This is the function I ended up implementing. Works great! Thanks Matt.
_.each(items, function(item, i) {
let incMod = {$inc:{}};
let matchMod = {$inc:{}};
matchMod.$inc[match] = 1;
incMod.$inc[item] = 1;
Database.upsert({node: item}, matchMod);
Database.upsert({node: match}, incMod);
});
source to share
I think the problem is coming from your ER model. a is sharpie
not an autonomous entity, thorium is an element. The relationship between 1 item and other items is such that 1 item has many items (1: M is recursive) and each conjugate item has a weight.
Fully normalized, you will have a table of elements and a table of weights. The element table will have elements. Weight table would have something like item1
, item2
, weight
(in this case you can have an asymmetrical weight, for example sharpie:pencil = 1
, pencil:sharpie = .5
which is useful in calculating the pushback in the FFA, but I do not think that this is applicable in your case.
Great, now let's mongotize.
When we say 1 item has many items, "many" probably won't exceed a few thousand (think 16MB document size). This means that it is actually 1-to-few, which means that we can nest data using either subdocs or fields.
So let's take a look at this circuit!
doc =
{
_id: "sharpie",
crayon: 1,
pencil: 1
}
What do we see? sharpie
is not a key, it is a value. It makes everything easy. We leave objects as fields. The reason we don't use an array of objects is because it is faster and cleaner (no need to iterate over the array to find the appropriate one _id
).
var match = "sharpie";
var items = ["pen", "marker", "crayon", "pencil"];
var incMod = {$inc:{}};
var matchMod = {$inc:{}};
matchMod.$inc[match] = 1;
for (var i = 0; i < items.length; i++) {
Collection.upsert({_id: items[i]}, matchMod);
incMod.$inc[items[i]] = 1;
}
Collection.upsert({_id: match}, incMod);
This is the easy part. The hard part is figuring out why you want to use FFA for the suggestion engine: -P.
source to share