5
$\begingroup$

As per my understanding through books & Google Search,

GOSS (Gradient-Based One Side Sampling) is a novel sampling method that downsamples the instances on the basis of gradients. As we know instances with small gradients are well trained (small training error) and those with large gradients are undertrained. A naive approach to downsample is to discard instances with small gradients by solely focussing on instances with large gradients but this would alter the data distribution. In a nutshell, GOSS retains instances with large gradients while performing random sampling on instances with small gradients. Source

LightGBM uses a novel technique of Gradient-based One-Side Sampling (GOSS) to filter out the data instances for finding a split value while XGBoost uses pre-sorted algorithm & Histogram-based algorithm for computing the best split.

Can someone please explain the math behind GOSS?

$\endgroup$
0
$\begingroup$

Wang et al., (2019) have provided a nice and clear explanation. Please, check out their paper to find the answer you are looking for:

Part II. BAYESIAN OPTIMIZED LIGHTGBM

Section: A. The Principle of the LightGBM

Wang, R., Liu, Y., Ye, X., Tang, Q., Gou, J., Huang, M., & Wen, Y. (2019). Power System Transient Stability Assessment Based on Bayesian Optimized LightGBM. 2019 IEEE 3Rd Conference On Energy Internet And Energy System Integration (EI2). doi: 10.1109/ei247390.2019.9062027

$\endgroup$
2
  • $\begingroup$ link seems to be dead $\endgroup$ – Nikos M. Jan 5 at 8:38
  • $\begingroup$ Fixed the link. You can also find the paper by googling the doi: 10.1109/ei247390.2019.9062027 $\endgroup$ – Bruno Ambrozio Jan 19 at 12:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.