The original dataset is of ~17K compound structures almost equally divided with labels indicating yes or no, after heavy use of mol2vec and rdkit I have created ~300 datapoints

Using the boosted trees method on the same shuffled train and test dataset gives 98% train accuracy and 89% test accuracy, but a simple neural network gives 100% train and test accuracy

I have checked the code again to ensure target leakage is not occurring, I have also coded it from the scratch twice to ensure I haven't made any mistake, yet I do not believe I should be getting 100% accuracy on both train and test

Does this mean that the model is actually accurate due to so many data points?

Training log

  • $\begingroup$ Can you clarify one more thing: was feature selection done using only a training subset of the data? $\endgroup$
    – C8H10N4O2
    Sep 7, 2020 at 17:04
  • $\begingroup$ @C8H10N4O2 No, it was done on the entire dataset. After this it was split into train and test $\endgroup$ Sep 7, 2020 at 17:14
  • 1
    $\begingroup$ That was a mistake. Until it’s time to test, treat the test data as if they don’t exist. $\endgroup$
    – Dave
    Sep 7, 2020 at 17:19

2 Answers 2


Yes - Getting 100% accuracy is possible for neural networks compared to tree-based models. Neural networks can learn non-linear relationships through the activation function. Tree-based models are restricted to piece-wise linear relationships.


Doing feature selection on the full dataset can lead to just this scenario, especially when so many features are available. This tread from the Stats stack exchange has some more information on how this happens, but the upshot is to make sure to do feature selection on some subset ("training set").


Please give this a try and speak up if you're still running into issues--just update the question with the relevant additional detail. It's certainly possible to hit 100%, but you would want to make sure there was no information leakage.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.