1
$\begingroup$

Something that has always bothered me is summarizing distributions when feature engineering for a machine learning process. Does anybody have best practices for this?

Example: Imagine a dataset showing orders of products by customers. You want to summarize customer behaviour.

In that example, if we're to only focus on customers' order value, the order value for each customer would have some distribution D(x).

Now, can create variables describing features of each customer's distributions (min, mean, median, max, quartiles, IQR, etc.), but are there best practices around which features tend to provide the most information upon extraction? Moreover, is there some way to contain the information of a distribution in a single variable?

$\endgroup$
  • $\begingroup$ I've created several features based on different aspects of the distribution, but the choice has been highly problem-specific. What kind of customer behavior do you want to model? Is it with ordering patterns, or something else? $\endgroup$ – Paul Jul 27 '17 at 19:16
  • $\begingroup$ My question isn't specific to a situation, but is more general. To answer your question, I would be looking to describe some sort of numerical behaviour (ordering patterns of quantities and prices over time durations and product groups, website behaviours, etc.) Here is a good structure for how I try to exhaust feature engineering alternatives: elitedatascience.com/feature-engineering-best-practices. But my musings are more theoretical: can you describe a distribution of a behaviour in a singe variable, rather than an "engineered" collection? $\endgroup$ – Michael Dyatchenko Jul 31 '17 at 22:55
  • $\begingroup$ If the distribution is parametric, record the parameters, otherwise consider using the quantiles, which can frequently estimated using a database (e.g., in BigQuery or PostgreSQL), as the representation. As you probably know, the quantile function uniquely and completely describes a distribution. $\endgroup$ – Emre Sep 25 '17 at 18:34
1
$\begingroup$

At the end each distribution can be described by a function with parameters. Can be a Gaussian, polynomial etc. In principle you can choose functions that only have one free parameter and fit this one. Depending on your data you might be able to guess a function class that seems to fit the underlying distributions. You can then use the fit parameters as input for your data set. Example: Fit a Gaussian and use mean, normalization and sigma = 3 parameters for your model.

$\endgroup$
0
$\begingroup$

I personally go with Association Rules. Since orders are aggregated by transactions (customers), does buying Coke displace buying Pepsi.. Buying IOS displace buying Android? will end with a nice heat map ;)

Along with ideas around and practices, try your own art around benefiting the the business (the beauty here).

$\endgroup$
0
$\begingroup$

Each distribution could be estimated with kernel density estimation (KDE). KDE is a non-parametric way to estimate the probability density function of a random variable. Kernel density estimation is smooths the data / summarizing the data. It has the advantage of not having to pick a specific distribution apriori. The disadvantage of a KDE is the data is not summarized in a single variable, the data is summarized as a function.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.