Python list - Remove consecutive duplicates (3 Ways)

In this article, we will discuss 3 different ways of removing consecutive duplicate elements from a python list. This is what we want to achieve.

input - [1, 2, 4, 7, 3, 7, 8, 4, 4, 9]
output - [1, 2, 4, 7, 3, 7, 8, 4, 9]

So we are not removing all duplicated elements. We are only eliminating consecutive duplicate elements.

Let’s check out the 3 different ways.

Method 1 - For loops will never go out of fashion

x = [1,2,4,7,3,7,8,4,4,9]

previous_value = None
new_lst = []

for elem in x:
   if elem != previous_value:
       new_lst.append(elem)
       previous_value = elem

print(new_lst)
>>> [1, 2, 4, 7, 3, 7, 8, 4, 9]

Method 2 - Let’s be more Pythonic! Use Enumerate

Here is a one-liner solution to the problem.

x = [1,2,4,7,3,7,8,4,4,9]

print([v for i, v in enumerate(x) if i == 0 or v != x[i-1]])
>>> [1, 2, 4, 7, 3, 7, 8, 4, 9]

Tell me that wasn’t pretty !!

Method 3 - It’s Python! There must be a library

Here is the 3rd way using the itertools library.

from itertools import groupby

x = [1,2,4,7,3,7,8,4,4,9]
print([i[0] for i in groupby(x)])
>>> [1, 2, 4, 7, 3, 7, 8, 4, 9]

Now the question is, which one of the above methods is the fastest?

Using the time and matplotlib library, I plotted the time taken for each of those methods.

You can see that the 3rd approach (using the itertools library) is the fastest of all the 3 ways discussed in this article.

Hope you enjoyed this article. Let me know if you have a better way of doing this, through the comments.