Clever Geek Handbook

How to handle DuplicateKeyError in MongoDB (pyMongo)?

Can someone please tell me how to handle DuplicateKeyError in MongoDB?

I am writing a python script where I move multiple documents from two different collections to a third. There is little overlap between the two collections due to the presence of multiple identical documents (with identical ObjectId). This leads to the following:

DuplicateKeyError: collection of duplicate keys errors E11000: index admin.collection_test: key id dup: {: ObjectId ('593a920b529e170d4b8fbf72')}

To get rid of the error, I use:

try:
    do something
except pymongo.errors.DuplicateKeyError:
    pass

I expect that using "try-except" will move all non-overlapping documents to the third collection, but instead the script will just stop working as soon as the first overlap occurs (a pre-existing document in the collection), Any help would be appreciated!

+6


source to share


3 answers 


for doc in documents:
    client.update_one({'_id': doc['_id']}, doc, upsert=True)


You can use update_one with upsert = True. This updates the doc with the new document if the doc exists, otherwise it creates a new document.

+3


source


If you are iterating over documents, try using continue

instead pass

.



for doc in documents:
    try:
        # insert into new collection
    except pymongo.errors.DuplicateKeyError:
        # skip document because it already exists in new collection
        continue
+5


source


DATABASE CONNECTION

connection = pymongo.MongoClient("192.168.2.202", 27017)

CREATE DATABASE

database = connection['my_database']

CREATE A COLLECTION

collection = database['my_collection']

INSERT DOCUMENTS INTO COLLECTION

url="http://some/api/url/path/?format=json"
data = {
    '_id': url,
    'timestamp': datetime.datetime.now(),
    'data': {
        'XX': 1,
        'YY': 2,
        'ZZ':  3
    }
}

AVOID DUPLICATES - THIS WILL CREATE A NEW DOCUMENT IF HAS AN IDENTIFIER NOT EXISTING

collection.update_one({'_id': url},  {"$set": data}, upsert=True)
0


source






More articles:


All Articles