Get unique values from a list in python

Getting unique values from a list in Python is a very common task, especially when working with datasets where duplicates can appear. The good news is that Python provides multiple ways to handle this, from simple built-in functions to more advanced library-based solutions.

The quickest and most common method is to use a set(), since sets automatically store only unique values. For example:

my_list = [1, 2, 2, 3, 4, 4, 5]

unique_values = list(set(my_list))

print(unique_values)  # [1, 2, 3, 4, 5]

However, one drawback of set() is that it does not preserve the original order of elements. If order matters, you can use:

Dictionary fromkeys (Python 3.7+):

   unique_values = list(dict.fromkeys(my_list))

List comprehension with condition:

 unique_values = []

[unique_values.append(x) for x in my_list if x not in unique_values]

For larger datasets, especially when working with tabular data, pandas provides a very efficient method:

import pandas as pd

unique_values = pd.Series(my_list).unique()

Key Points:

• Use set() for quick deduplication (fastest, but unordered).
  
• Use dict.fromkeys() to preserve the original order.
  
• Use pandas when dealing with large data collections or dataframes.
  

 In short, if you just need unique values, go with set(). But if you care about keeping the original order, dict.fromkeys() or list comprehension works best.