Insert all documents from one collection to another collection in MongoDB database

I have a python script that collects data every day and inserts it into a MongoDB collection (~ 10M docs). Sometimes the job fails and I am left with partial data that is not useful to me. I would like to insert data into the intermediate collection first and then copy or move all documents from the intermediate collection to the final collection only when the job is over and the data is complete. I can't seem to find a direct solution to this, like a "bulk" type operation, but it looks like it should be.

In SQL it would be something like this:

INSERT INTO final_table
SELECT *
FROM staging_table

      

I thought db.collection.copyTo () would work for this, but it seems that it makes the destination collection a clone of the original collection.

Also, I know from this: mongodb move documents from one collection to another collection that I can do something like the following:

var documentsToMove = db.collectionA.find({});
documentsToMove.forEach(function(doc) {
    db.collectionB.insert(doc);
}

      

But it looks like there should be a more efficient way.

So how can I take all documents from one collection and insert them into another collection in the most efficient way?

NOTE: there is already data in the final collection. New documents that I want to move will be appended to this data, for example if there are 2 documents in my intermediate collection and there are 10 documents in my last collection, I would have 12 documents in my final collection after I moved the data intermediate level.

+3


source to share


2 answers


You can use db.cloneCollection (); see mondb cloneCollection



0


source


if you don't need the intermediate collection anymore, you can simply use the rename option.



 switch to admin db
db.runCommand({renameCollection:"staging.CollectionA",to:"targetdb.CollectionB"})

      

0


source







All Articles