New answers tagged regression
2
- Use RandomForest as XGBoost is more prone to overfitting and comparatively difficult to tune hyperparameters
Tune at least these parm -
param_grid = { 'n_estimators': [ ], 'max_features': [ ], 'max_depth' : [ ], 'criterion' :['gini', 'entropy']}
- Try imputation based on your domain knowledge and using other Features e.g. Correleation
- Scaling is not ...
0
Unevenly spaced / irregular times series is totally different from (regular) time series analysis. In fact, in my knowledge there is no perfect statistical model for it. There are some packages in R like ust and few more.
1
When evaluating xgboost (or any overfitting prone model), I would plot a validation curve. Validation curve shows the evaluation metric, in your case R2 for training and set and validation set for each new estimator you add. You would usually see both training and validation R2 increase early on, and if R2 for training is still increasing, while R2 for ...
2
Overfitting can be identified by checking validation metrics such as accuracy and loss. The validation metrics usually increase until a point where they stagnate or start declining when the model is affected by overfitting.
If our model does much better on the training set than on the test set, then we’re likely overfitting.
You can use Occam's razor test: ...
0
You should be using an evaluation metric like area under the ROC curve not R^2. R^2 is good for continuous unbounded variables not classification. This is the most important thing you should do. If your outcome variable is highly imbalanced you might want to use precision recall.
More about Precision-Recall and ROC.
You need to do parameter tuning with Grid ...
2
I've tangled with modeling pricing systems over the last two years and one of my key learnings applies here:
Available sales data is often a bad basis for straight-forward prediction tasks and the reason for this is fairly simple:
If you classify all prices (or transactions of a given product at a price x) into "Accepted" and "Not accepted&...
3
The direct way to check your model for overfitting is to compare its performance on a training set with its performance on a testing set; overfitting is when your train score is significantly above your cv score. According to your comments, your r2 score is 0.97 on the training set, and 0.86 on your testing set (or similarly, 0.88 cv score, mean across 10 ...
0
One way is to transform the daily data into dynamic quarterly data, eg by averaging over a moving window (that represents a quarter) and similarly for other statistics.
These windows can be overlapping, eg by a factor $o$%, no need to be non-overlapping.
This way the data are transformed to quarter period statistics while not actually reducing amount of data ...
2
These are several things you can try:
Use quartic error, $(y - \hat{y})^4$, instead of quadratic error. This is going to penalize a lot big errors, way more than MSE. The issue is that this is not implemented in xgboost, and you would need to develop a custom loss.
If your target is always positive, you can use the target as training weights. This will give ...
1
Your reasoning sounds similar to curriculum learning: the idea is to pass training data to the network not randomly, but based on some scoring function $f$. It was proven that a network learns faster and better, even though the process itself is not well understood. I think it is a better approach than the one you suggested: I am afraid that, if you pass ...
4
Yes, gradient boosted trees can make predictions outside the training labels' range. Here's a quick example:
from sklearn.datasets import make_classification
from sklearn.ensemble import GradientBoostingRegressor
X, y = make_classification(random_state=42)
gbm = GradientBoostingRegressor(max_depth=1,
n_estimators=10,
...
2
Prediction of a Decision tree will lie within the limits of the target because at the end either the record will fall to a specific target leaf if the depth is not controlled Or it will be average on multiple targets.
With the second approach too, it can't cross the limit of the target.
Coming to Ensembling -
Bagging -
Bagging simply averages multiple trees. ...
2
In Catboost(gradient boosting) I dont know, but in decision trees and random forest the answer is no.
The final prediction is done based in the "mean" of the instances that fell in the leave. I say "mean" but its not necessary the mean. For random forest is the mean of that mean.
Now your question, can I have a predicted value bigger than ...
0
To add to this.
The formal definition of linearity of a function f is:
$f(a*z1+b*z2)=a*f(z1)+b*f(z2)$
0
tw = tf.Variable(initial_value=np.random.randn(), dtype=tf.float32, trainable=True)
tb = tf.Variable(initial_value=0, dtype=tf.float32, trainable=True)
cost = lambda :tf.reduce_mean(tf.square(tx*tw+tb-ty))
optimizer = tf.keras.optimizers.SGD(0.01)
train = optimizer.minimize(cost, var_list = [tw,tb])
Reference to tw and tb directly in cost function rather ...
0
Yes you can use neural network for any non linear function. Maybe your variables don't have a strong linear correlation but you can train it using neural network to find the best pattern possible . Although it is tough to say whether it will generalize or not
0
Simply calculate the derivative of the price level with respect to your time unit (in other words between two subsequent measurements).
If the derivative is 0 at some interval, it means that the price has been stable.
If the derivative is positive, it means that the price has increased in that range.
If the derivative is negative, then the price has ...
0
What stops you looking at lags?
df = data.frame(c(1,2,3,4,4,3,4,5,6,7))
colnames(df)<-c("price")
df$delta=lag(df$price, 1)
df$inc = df$price-df$delta
df
Would yield:
price delta inc
1 1 NA NA
2 2 1 1
3 3 2 1
4 4 3 1
5 4 4 0
6 3 4 -1
7 4 3 1
8 5 4 1
9 6 ...
0
The suggestion in the comment is appropriate.
Still, if you want to try NN, you may try these suggestions -
- None of the models seems best as per general guidlines
- Keep ReLu as all the hidden layer, linear for last layer(Regression)
- Standardization/Normalization must be done before training
- Add Batch normalization layer
Can also try
Since your ...
0
In the simplest case, doing regression with Transformers is just a matter of changing the loss function.
BERT-like models that use the representation of the first technical token as an input to the classifier. You can replace the classifier with a regressor and pretty much nothing will change. The error from the regressor will get propagated to the rest of ...
0
It seems to me that this question is better off on stackoverflow.
Nevertheless, X_cal gets generated from X_train and X_train from valid. But this is an atleast 2-dimensional dataframe with new_host and sequence. Like the error says you should only input data that is 1-dimensional.
0
Spark is designed to be distributed across a cluster and use stochastic gradient descent (SGD) to optimize linear regression.
There is overhead for cluster infrastructure (even when the "cluster" is a single local node). Also, SGD is an iterative method that uses many batches to find a solution.
Given that your problem is 70k rows, it would be ...
0
The fmla is the model formula: fmla <- formula(blood_pressure ~ age + weight)
So the correct solution should be
# bloodpressure is in the workspace
summary(bloodpressure)
# Create the formula and print it
fmla <- formula(blood_pressure ~ age + weight)
fmla
# Fit the model: bloodpressure_model
bloodpressure_model <- lm(fmla, data = bloodpressure)
...
0
The general idea is that there is a true joint probability $p(x,y)$ which can be factorized like
$$p(x,y) = p(y|x)p(x)$$
$p(x)$ basically measures the probability of observing the input $x$, while $p(y|x)$ represents the true relationship between $x$ and $y$.
We would like to learn a model distribution $p_\theta(x, y)$ which is factorized accordingly
$$p_\...
1
I think it's complete alright. In fact, the second model mathematical expression is given by
y=x3f(x1, x2, x3),
which is just like the first model but with some specific feature engineering. I don't see any possibility for data leakage.
1
Writing a custom loss function could be handy, but it may be simpler to just try to treat this as a class balance problem for your regression model. For starters, just try undersampling all of the higher and medium grades until they're close to balanced with your failing students. Given your number of data points and features you can probably still just ...
1
Try writing a custom loss function for a regression model!
Keras' neural networks support this, for example. See https://stackoverflow.com/q/43818584/745868
(But many other libraries give support for this as well)
The only special thing about your custom loss function is that it doesn't add up the error of a datapoint if min(pred_y, actual_y) >= THRESHOLD
0
There is no point in looking at the prediction at this point. Your model has not learnt the data yet.
Suggestions -
When copy from a source. Try copying everything first. Run it and then start tweaking for learning Or to map with your own need. You have changed a few things as compared to what Jason has in his post
Use the wisdom of researchers. So, you ...
0
I could be because of your loss function. I would suggest using "mean_squared_error" as the loss function. Also, try to normalize your data before feeding it into the neural networks.
1
The first question about missing data is always why is it missing?
Have you checked or know why the data is missing and whether it is MAR, MCAR or not missing at random?
If your data is MCAR imputation is generally fine and your lower test metric might simply indicate a suboptimal imputation strategy. In this case you could try MICE or similar more advanced ...
1
First you should define a metric that suits the problem $R^2$ in your case.
Do a correct cross-validation and train test splits.
And then choose in the cross validation which option has the best results for your model (imputing missing or xgboost no imputing). This way you are doing an empirical experiment and selecting the best result.
Probably you want to ...
0
AUC is for binary classification tasks only, it's not possible to calculate it on a regression model. However it's plenty of other metrics one can use to check a regressor performance, such as MAE and MSE.
0
It should be a bug in their server. The fmla variable should have the same contents in your code and in theirs. This is because the last assignment on both scripts is
fmla <- lm(blood_pressure ~ age + weight, data=bloodpressure)
3
Think carefully before you do this. You have no idea what the underlying height distribution is. Here are four possibilities.
If you were building a regression model, each of these sets of height data would be interpreted differently. However, if replaced by your ordinal variable, they all look numerically equivalent. If you use this variable as a ...
Top 50 recent answers are included
