Distribution of the target to be learned

I want to regress on this target, have tried multiple transformations to bring it to normal but its not helping, read some stuffs online but none of the suggestions have worked till now.

I am attaching the residual histogram as well, somehow the residuals are normally distributed.

Distribution of Residuals

Thanks in Advance.


1 Answer 1


To my understanding you should be looking for something like a Gaussian Mixture Model - GMM or a Kernel Density Estimation - KDE model to fit to your data.

There are many implementations of these models and once you've fitted the GMM or KDE, you can generate new samples stemming from the same distribution or get a probability of whether a new sample comes from the same distribution.

In python an example would be like this:(directly taken from here)

import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import norm
from sklearn.neighbors import KernelDensity

# Plot a 1D density example
N = 100
X = np.concatenate((np.random.normal(0, 1, int(0.3 * N)),
                    np.random.normal(5, 1, int(0.7 * N))))[:, np.newaxis]

X_plot = np.linspace(-5, 10, 1000)[:, np.newaxis]

true_dens = (0.3 * norm(0, 1).pdf(X_plot[:, 0])
             + 0.7 * norm(5, 1).pdf(X_plot[:, 0]))

fig, ax = plt.subplots()
ax.fill(X_plot[:, 0], true_dens, fc='black', alpha=0.2,
        label='input distribution')

for kernel in ['epanechnikov', 'tophat', 'gaussian']:
    kde = KernelDensity(kernel=kernel, bandwidth=0.5).fit(X)
    log_dens = kde.score_samples(X_plot)
    ax.plot(X_plot[:, 0], np.exp(log_dens), '-',
            label="kernel = '{0}'".format(kernel))

ax.text(6, 0.38, "N={0} points".format(N))

ax.legend(loc='upper left')
ax.plot(X[:, 0], -0.005 - 0.01 * np.random.random(X.shape[0]), '+k')

ax.set_xlim(-4, 9)
ax.set_ylim(-0.02, 0.4)

enter image description here

In the end the kde model, could be used for sampling new data points or predicting the probability of a new sample to have been generated from this distribution.

You should play around with different kernels in KDE models or number of base distributions in GMMs, along with other parameters to get optimal results for your data.

  • $\begingroup$ Thanks for the response. I haven't tried yet sampling data points using KDE or GMM. I have one more concern regarding , whether this can be target be successfully regressed. Will keep you posted about my results. $\endgroup$ Jul 13, 2017 at 13:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.