1
$\begingroup$

I am looking for a very specific sampling technique which pertains to a very large dataset with mixed data type i.e, I have categorical as well as continous variables and want to have a sample that represents the population of such kind of data as closely as possible. It would be appreciable if anyone could help me out of with this. Thanks!

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

This would need some data preprocessing:

  1. Get the different main categories (ex: bikes and car)

  2. If there are several mix of categories, get the quantities of each configuration in order to get the right proportions of the samples (see 4).

  3. Get random sample within each category (10% bikes and 10% cars)

  4. Ensure that those samples have the right quantity in proportions regarding to the whole population (if there are 600 bikes and 100 cars, you should have 60 bikes and 10 cars)

  5. Ensure that each sample distribution shape is similar their related category distribution (using all data). This step is crucial, because some categories'samples may not have enough data to represent the whole data set correctly. If you don't have enough data, increase the overall sampling ratio or redo a random sample. Example with python Seaborn:

    sns.displot( data=df, x="Price", col="Type", kind="hist", aspect=1.4, log_scale=10, bins=20 ) enter image description here Source: https://towardsdatascience.com/10-examples-to-master-distribution-plots-with-python-seaborn-4ea2ceea906a

  6. Merge all samples into a general sample set.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you so much for your helpful suggestion! $\endgroup$
    – ml_learner15
    Commented Sep 6, 2021 at 7:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.