Using buffered data from the overflow sort phase
We have the mongoDB 2.6.4 replica set installed and is trying to diagnose this behavior. We get Runner error: Overflow sort stage buffered data usage of 33598393 bytes exceeds internal limit of 33554432 bytes
it when we expect we won't. The collection contains millions of records and contains a composite index that includes the key that is sorted. As an example
looks like that:
{ from: 1, time : -1, otherA : 1, otherB : 1}
our search
find.collection({ from : { $in : ["a", "b"] }, time : { $gte : timestamp },
otherA : {$in:[...]}, otherB : {$in:[...]}})
.sort( time : -1 )
mongoDB parallels (clauses) this request looks like this:
{ from : a }, { time : { $gte : timestamp }, ... }
{ from : b }, { time : { $gte : timestamp }, ... }
The explanation for each step says scanAndOrder: false, which implies that the index was used to return the results. This all seems fine, however mongoDB client gets Runner error: buffer overflow buffer error. This apparently means that the sort was done in memory. Is it because it does in-memory type merging? Or is there another reason this error might occur?
source to share
I also had a memory overflow problem. I am using PHP with MongoDB for document management. When I access a collection that probably has a large set of documents, it throws an error. From the following link, it can only sort up to 32MB of data at a time. http://docs.mongodb.org/manual/reference/limits/#Sorted-Documents .
So, as described in MongoDocs about sorting, I sorted the array converted from the MongoCursor object using the PHP method instead of the Mongo sort () method.
Hope this helps you. Thank.
source to share