general Infos about my dataset: I have 40k data points and 5 features. I'm doing regression and trying to build a model that can predict the error of a GPS. for example imagine that your vehicle GPS is making an error of 10 meters and you want to correct it. So I brought another super GPS which is very accurate and measured 40k data while driving so in my dataset I have some vehicle informations which are speed, Acceleration, yaw rate, timestamp and wheel angle and I have position Informations which are the ground truth longitudes latitudes and the false longitudes and latitudes from my normal GPS. I'm transofrming those latitudes and longitudes to an x and y just to know how much should I shift my false longitudes and latitudes so that my position can be more accurate and similar to the Ground truth values. Can my data be bad in this case? I'm trying to predict the error in longs and lats that the GPS make so that I can later correct it so it's a regression problem and I'm using those features above to do that which I think they are informative since speed, acceleration, yaw rate and wheel angle are related somehow to a position(am I wrong?)
I'm asking this generally, I read some articles in the internet, that say that data is sometimes bad or the quality of the data is bad but I don't know what the mysterious sentence really mean.
I also had the problem when training neural networks that my loss start to decrease for the first 10-20 epochs and then it stuck on some high value and the network stops learning like if it were struggling to go out of that loss value but it can't. I tried to use only 100 data points instead of all the 40k and I noticed that it worked good, the NN achieved to fit those but as I increase the number of data points the performance become worse(do you have any ideas on this?)
some people suggest that I don't have many data and many features and in this case it would be better to use some machine learning approach since it outperforms NNs in case of small datasets or if I have few features like in my case so I also tried using random forest and I noticed that it gives better results than neural networks but it also doesn't generalize well, even if it gave me good results on train and validation sets, when I try it on test data(data that the random forest have never seen), it perform really bad.
so I was reading in the internet what can cause those problems and I noticed that I sometimes saw people or articles that claim that maybe the quality of the data is bad! but what does this really mean? I thought neural networks can map any kind of data, if I have one feature and one target then neural networks can map those two together, at least it can overfit the data right?
so can someone please tell me what is bad data or better how do I know if my data are bad? well if there is a way to know that then I would probably save time and not start working on a project that will take me a month to complete and then figure out my data is bad. Also can you tell me whether my case make sense ? I mean I find it weird that NNs gives very bad performance way worse than random forest. at least my NN should overfit the data or am I wrong?