I'm using a set of features, says $X_1, X_2, ..., X_m $, to predict a target value $Y$, which is a continuous value from zero to one.
At first, I try to use a linear regression model to do the prediction, but it does not perform well. The root-mean-squared error is about 0.35, which is quite high for prediction of a value from 0 to 1.
Then, I have tried different models, e.g., decision-tree-based regression, random-forest-based regression, gradient boosting tree regression and etc. However, all of these models also do not perform well. (RMSE $\approx $0.35, there is not significant difference with linear regression)
I understand there are many possible reasons for this problem, such as: feature selection or choice of model, but maybe more fundamentally, the quality of data set is not good.
My question is: how can I examine whether it is caused by bad data quality?
BTW, for the size of data set, there are more than 10K data points, each of which associated with 105 features.
I have also tried to investigate importance of each feature by using decision-tree-based regression, it turns out that, only one feature (which should not be the most outstanding feature in my knowledge to this problem) have an importance of 0.2, while the rest of them only have an importance less than 0.1.