How to choose sample from a large dataset such that each unique row from the dataset is selected at least once in the sample? Is there a way of doing this in python?
Let's say you have a dataframe with 10,000 rows, and you have only 1,000 unique ones.
You can do:
df_unique = df.drop_duplicates() df_sample = df.sample(n) df_final = pd.concat([df_unique, df_sample], axis=0)
In the above code, n is the amount of sample you want. In this way you can assure that every unique row is in your dataset and you have more samples on it.