How do I remove duplicate lists containing items in a different order?
Suppose I have a list [[1,2,3], [4,5,6], [3,2,1]], the first and last elements (which are also lists) of the list are actually the same - they contain the same set of numbers, only with a different order. If I want to create a function that takes a list of a list and remove those duplicate list items (in this case, remove either [1,2,3] or [3,2,1] and keep the rest), how do I do it?
Thank you
source to share
Use set
and frozenset
.
Sets only unique items and they are unordered.
>>> {3, 1, 2} == {2, 1, 3}
True
But they can only store immutable elements, so we'll need it as well frozenset
.
>>> {set([1, 2, 3])}
Traceback (most recent call last):
File "<ipython-input-16-ddc0311520e7>", line 1, in <module>
{set([1, 2, 3])}
TypeError: unhashable type: 'set'
>>> {frozenset([1, 2, 3])}
set([frozenset([1, 2, 3])])
Demo:
>>> lst = [[1,2,3],[4,5,6],[3,2,1]]
>>> seen = set()
>>> [x for x in lst if frozenset(x) not in seen and not seen.add(frozenset(x))]
[[1, 2, 3], [4, 5, 6]]
source to share
Convert each element of the list to a set and then to a tuple and form a new set to get the expected result.
data = [[1,2,3],[4,5,6],[3,2,1]]
print(set(tuple(set(item)) for item in data))
# {(4, 5, 6), (1, 2, 3)}
Here we can use frozenset
like this
print(set(frozenset(item) for item in data))
# {frozenset({1, 2, 3}), frozenset({4, 5, 6})}
source to share
Another way to do it is to use a map:
map(list, set(map(frozenset, lst)))
This will give you [[1, 2, 3], [4, 5, 6]]
What it basically does is frozenset to lst. We use frozenset because the collection stores immutable items. Executing the map (frozenset, lst) gives us: [frozenset ([1, 2, 3]), frozenset ([4, 5, 6]), frozenset ([1, 2, 3])] From this we apply set and get : set ([frozenset ([1, 2, 3]), frozenset ([4, 5, 6])])
and the mapping list to this gives us [[1, 2, 3], [4, 5, 6]]
source to share