Improve the performance of mango queries
I am using a CMS node called Keystone, which uses MongoDB for data storage, allowing for fairly liberal control over data and access. I have a very complex model called Family , which contains about 250 fields, a bunch of links, and several dozen methods. I have a form on my site that allows the user to enter the required information to create a new Family recordhowever the processing time is long (12 seconds on localhost and over 30 seconds on my Heroku instance) The problem I am running into is Heroku emits an application error for any processes that run in 30 seconds, which means that I need to optimize my query. All processing is very fast except for one function. Below is the offending function:
const Family = keystone.list( 'Family' );
exports.getNextRegistrationNumber = ( req, res, done ) => {
console.time('get registration number');
const locals = res.locals;
Family.model.find()
.select( 'registrationNumber' )
.exec()
.then( families => {
// get an array of registration numbers
const registrationNumbers = families.map( family => family.get( 'registrationNumber' ) );
// get the largest registration number
locals.newRegistrationNumber = Math.max( ...registrationNumbers ) + 1;
console.timeEnd('get registration number');
done();
}, err => {
console.timeEnd('get registration number');
console.log( 'error setting registration number' );
console.log( err );
done();
});
};
processing in mine .then()
happens in milliseconds, however it Family.model.find()
takes too long to complete. Any advice on how to speed up the process would be very grateful. There are about 40,000 family records that are trying to complete the query, and the field registrationNumber
already has an index.
source to share
Makes sense to then()
execute quickly, but find()
takes some time; Finding the largest value in a recordset is a relatively fast database operation, while retrieving a set can potentially be very time consuming depending on a number of factors.
If you are just reading the data and presenting it to the user via REST or some kind of visual interface, you can use lean()
that will return plain javascript objects. By default, you return mongoose.Document
, which in your case is not needed, since no data manipulation occurs after your read request; you just get the data.
More importantly , it seems like you only need one entry: the entry with the largest registrationNumber
. You should always use findOne()
when looking for a single record in any set of records to maximize performance.
See the previous answer using findOne
node.js in implementation, or see mongoDB for general information on this collection method.
source to share