Removing duplicate objects in master data (iphone)
In my iPhone app, I provide users with an insight into industry news. I am getting this list as xml file from my server. Parsing and inserting xml data into my main data repository is not a problem, but there are a few cases where I might get duplicate posts in the news.
I thought it would be a good solution to insert all updates while processing the XML feed and then remove any duplicates. But I cannot figure out how to do this. How do I remove duplicate objects in Apple's database engine?
To make it specific, let's say I have news:
News - uniqueId (set by external system) - Title - newsText
Is there any concise way to tell Core Data to just remove duplicate objects where the duplicate object is defined as an object with the same "uniqueInd"? Ie, not making an explicit fetch in my code and not inserting an object if an object with its same "uniqueInd" already exists?
source to share
Core Data does not support removing "duplicates" because the only concept of an object identifier is NSManagedObjectID
that assigned to each object. Since you cannot set this ID manually, you cannot use it to uniquely insert. You have (at least) two options:
-
Do insert sampling as you suggest. Testing will show if this is going too far: until you test it, don't assume this solution won't work. You could improve performance by doing the entire insert in
NSInMemoryPersistentStore
and then moving that persistent store to a persistent (on-disk) store to save. If you can keep all inserted objects in memory, it will be very fast. -
You can insert all objects, save, then select and delete all but one of the objects. Again, this may be more effective than you suspect.
In both cases, the amount of the code is minimal. The Core Data Tools in Instru.app will be your best tool for assessing the effectiveness of both approaches.
source to share