How to model a Bimodal distribution of target variable

Question

I want to regress on this target, have tried multiple transformations to bring it to normal but its not helping, read some stuffs online but none of the suggestions have worked till now.

I am attaching the residual histogram as well, somehow the residuals are normally distributed.

Thanks in Advance.

Answer 1

To my understanding you should be looking for something like a Gaussian Mixture Model - GMM or a Kernel Density Estimation - KDE model to fit to your data.

There are many implementations of these models and once you've fitted the GMM or KDE, you can generate new samples stemming from the same distribution or get a probability of whether a new sample comes from the same distribution.

In python an example would be like this:(directly taken from here)

import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import norm
from sklearn.neighbors import KernelDensity

# Plot a 1D density example
N = 100
np.random.seed(1)
X = np.concatenate((np.random.normal(0, 1, int(0.3 * N)),
                    np.random.normal(5, 1, int(0.7 * N))))[:, np.newaxis]

X_plot = np.linspace(-5, 10, 1000)[:, np.newaxis]

true_dens = (0.3 * norm(0, 1).pdf(X_plot[:, 0])
             + 0.7 * norm(5, 1).pdf(X_plot[:, 0]))

fig, ax = plt.subplots()
ax.fill(X_plot[:, 0], true_dens, fc='black', alpha=0.2,
        label='input distribution')

for kernel in ['epanechnikov', 'tophat', 'gaussian']:
    kde = KernelDensity(kernel=kernel, bandwidth=0.5).fit(X)
    log_dens = kde.score_samples(X_plot)
    ax.plot(X_plot[:, 0], np.exp(log_dens), '-',
            label="kernel = '{0}'".format(kernel))

ax.text(6, 0.38, "N={0} points".format(N))

ax.legend(loc='upper left')
ax.plot(X[:, 0], -0.005 - 0.01 * np.random.random(X.shape[0]), '+k')

ax.set_xlim(-4, 9)
ax.set_ylim(-0.02, 0.4)
plt.show()

In the end the kde model, could be used for sampling new data points or predicting the probability of a new sample to have been generated from this distribution.

You should play around with different kernels in KDE models or number of base distributions in GMMs, along with other parameters to get optimal results for your data.