Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [regression]

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

0
votes
0answers
9 views

Metric (rather than RMSE, MSE, etc.) to choose the best model in terms of the ability to detect peaks better

I have created multiple regression models and wanted to choose the best one. One common metric would be RMSE, as you know. When I looked at the results, second model (RMSE = 0.15) was better able to ...
0
votes
0answers
17 views
0
votes
0answers
8 views

Finding optimal parameter values under higher-order effects

I have a series of computer experiments. In each experiment, I run one of two programs, each with about 6 parameters (4 of which are common to both programs; some parameters are continuous, others ...
1
vote
1answer
31 views

Random forest vs. XGBoost vs. MLP Regressor for estimating claims costs

Context I'm building a (toy) machine learning model estimate the cost of an insurance claim (injury related). Aim is to teach myself machine learning by doing. I have settled on three algorithms to ...
0
votes
0answers
13 views

Help interpreting the result of linear regression and confidence interval (beginner level)

I have 200 regressors and one response. I want to predict the response and have used Ordinary Least Squares based regression. The plot is given below where the red dots represent the predicted ...
0
votes
0answers
8 views

Multidimensional regression: How to make sure the error variances are the same along different dimensions? Should I be doing this?

I have data that has equal variance along each of the target dimensions, but if i analyze the results of training i notice that that my trained model does not have the same error variances along each ...
1
vote
0answers
15 views

predictive clustering trees in Python?

I am faced with a time series forecasting cold-start problem, specifically I am forecasting energy consumption of businesses where historic consumption data is available only for training but not new ...
1
vote
0answers
26 views

How to visualize multivariate regression results by plotting plane of best fit?

I am implementing multivariate linear regression using numpy, pandas and matplotlib. I am ...
0
votes
0answers
4 views

Will the combination of Chaid/CART tree modelling improve the accuracy of the Decision Tree Regression Model?

Will the performance of the decision tree regression model significanlty improve if we consider CHAID modelling first by identifying the key continous/categorical dependent variables and then builidng ...
1
vote
0answers
17 views

How do I implement stochastic gradient descent correctly?

I'm trying to implement stochastic gradient descent in MATLAB however I am not seeing any convergence. Mini-batch gradient descent worked as expected so I think that the cost function and gradient ...
0
votes
0answers
18 views

Predicting stock market index values using individual stocks

I'm trying to predict the market trend (i.e. predict the value of a stock market index, e.g. S&P 500) using the stocks in the index. My data-set is as follows: ...
0
votes
1answer
24 views

Adding a custom constraint to weighted least squares regression model

I am trying to run a weighted least squares model that looks something like this (but could be different): $y = \beta_0 + \beta_1 x + \beta_2 log(x) + \epsilon$ with weights $w_1, w_2, ..$ However,...
0
votes
1answer
22 views

How high rmse value can be?

While solving the questions for machine learning, I got two values for R square from 2 different regressors, i.e, 0.9999 and 0.9769. So, which should go for as both could lead to overfitting? Thanks ...
0
votes
0answers
21 views

How does on test regression for a subspace or matrix factorization?

I've recently been reading a lot of papers and watching a lot of videos on both subspace learning, and matrix factorization. One thing is particularly eluding me though - how does any of this get ...
0
votes
0answers
9 views

Bouding box regression in R-CNN (and similar methods)

Here is the bounding-box regression technique used in R-CNN What I don't understand is why $t_w$ needs to be $\log (G_w / P_w)$ instead of just $G_w / P_w$ (i.e. using normalized coordinates just ...
0
votes
0answers
23 views

Developing a time to default regression model to predict the time to default

Background: I used XGBoost to develop a probability model to get a probability measure of a particular loan defaulting. The results are very satisfactory, now my task is to develop a time-to default ...
2
votes
0answers
29 views

Test RMSE of polynomial regression drops when using more variables?

I am testing polynomial regression for a data set of 50 variables and a sample size of 5000. I ordered the coefficients of the linear model from high to low and then made different models using the p ...
0
votes
0answers
8 views

Regression methods for multi-dimensional categorical input and multi-dimensional real-valued output?

I wonder if there are useful regression methods for multi-dimensional categorical input and multi-dimensional real-valued output. Could random forest be one of those?
2
votes
1answer
15 views

Predicting descrete value problem in regression or classification

In machine learning, regression algorithms attempt to estimate the mapping function (f) from the input variables (x) to numerical or continuous output variables (y). I have one usecase where I ...
2
votes
1answer
41 views

multi-output regression problem with tensorflow

number of features: 12 , -15 < each feature < 15 number of targets: 6 , 0 < each target < 360 number of examples: 262144 my normalization: I normalized the features so that they are ...
1
vote
1answer
32 views

Residual plot is tight to zero, but low R2 score

Hi I am a beginner at data science, and currently trying to use Gradient Boost Regressor to predict car price based on several attribute such as machine capacity, car brand, car type etc. I am ...
0
votes
0answers
21 views

Decomposing R^2 into independent variables

Consider a linear regression model: $$y = β_0 + β_1X_1 + β_2X_2 + ... + β_kX_k + ε$$ where $R^2 = 1 - (SSR/SST)$. I would like to determine the contribution of a factor $i$ (call it $R^2_i$) into ...
0
votes
0answers
15 views

Correlation / regression / association between one categorical variable and two non-independent others

Let's say I want to measure association / correlation between one categorical variables, and two others which are not independent. As an example (not the one I'm using), I have a data set with three ...
0
votes
0answers
16 views

looping program for MLP Keras prediction [migrated]

I am (sort of a beginner starting out) experimenting with Keras on a time series data application where I created a regression model and then saved it to run on a different Python script. The time ...
2
votes
0answers
51 views

Custom Lambda layer Keras outputs predictions. I get 'An operation has `None` for gradient' error

I have a Lambda layer that takes input from previous layer, makes some preprocessing. Output of the Lambda layer is a prediction, and keras.losses.mean_squared_error is used. ...
0
votes
0answers
15 views

Distribution function of observations in the errors-in-variables model

Consider the linear relation $\mathbf{y} = \mathbf{H}\mathbf{x} + \mathbf{z}$. The vector $\mathbf{x}$ is unknown, and a noisy version of the system matrix $\mathbf{H}$, i.e., $\tilde{\mathbf{H}} = \...
0
votes
0answers
13 views

Detecting overfitting in deep learning in a regression task

I have regression task at hand, but I am not sure how to consistently detect/prevent overfitting. As my code's currently written, it will save the model on each epoch as long as the validation loss ...
1
vote
0answers
30 views

Ensemble technique for combining predictions from classification and regression algorithms

Given an anomaly detection problem - A, I have divided the problem into two independent subtasks -A1 and ...
1
vote
2answers
25 views

Gradient Descent Convergence

I'm a double major in Math and CS interested in Machine Learning. I'm currently taking the popular Coursera course by Prof. Andrew. He's talking and explaining Gradient Descent but I can't avoid ...
0
votes
1answer
25 views

How to choose between classification Vs regression approach?

I need to predict the profitability of the products of a retailer. I can either predict the absolute value of the profit the products will make (continuous outcome) or predict whether the products ...
0
votes
0answers
19 views

Any model can reach to train incrementally but doesn't lose accuracy/mse?

I have huge dataset in my single computer so that memory is insufficient. I want to train incrementally with xgboost, but I found incremental training isn't like what I thought. I thought: ...
1
vote
0answers
8 views

Ensemble learning for multiple hypothesis classes

Just to confirm if the following description falls in the category of ensemble learning. Suppose given a training set $D=\{(X,Y)\}$ we are asked to train a regressor. But now the way we do it is to ...
0
votes
1answer
16 views

What is the possible range of SVR parameters range?

I'm working on a regression problem. While tunning the Parameters of SVR I got the following values c=100, gamma= 10 and epsilon =100. For which I got 95 percent r-square. My question is what is the ...
2
votes
2answers
51 views

What activation function should I use for a specific regression problem?

Which is better for regression problems create a neural net with tanh/sigmoid and exp(like) activations or ReLU and linear? Standard is to use ReLU but it's brute force solution that requires certain ...
1
vote
0answers
53 views

How to model non-linear demand function?

I am trying to build a dynamic pricing algorithm on intermittent data (a lot of zeros between non-zero values). I have on average 100 non-zero data points for each product. However, it seems to be ...
0
votes
0answers
40 views

CNN saliency maps for regression problems

I've been using CNNs for regression tasks. While I know there are techniques such as integrated gradients and guided backpropogation to generate saliency maps, most examples generate saliency maps ...
0
votes
0answers
32 views

Python Library for Neural Networks (no Tensors)

I’m having an absolute nightmare with Keras and TF and I think by now it’s time to attempt a different approach with a different library (plan C is to build the network from scratch) My neural ...
1
vote
1answer
14 views

does index of my data which is of type “Date time index” plays a part in reggression?

I'm new to data science and I'm working on a regression problem. My question is the index of my data which is of type "Date time index" plays a part in regression? I mean is it Okay if i drop the ...
2
votes
1answer
31 views

Can I use Linear Regression to model a nonlinear function?

I have recently started studying the basics about regression, and as a beginner I started by Linear Regression. I read this article that says that for this particular type of regression the ...
0
votes
2answers
23 views

How do I correctly build model on given data to predict target parameter?

I have some dataset which contains different paramteres and data.head() looks like this Applied some preprocessing and performed Feature ranking - ...
1
vote
0answers
12 views

Do batch norm makes sense for regression problems?

When my network is performing regression (like DQN) it makes sense to use batchnorm in network when output of my network should vary from [0, 100000]? one way to tackle it is to normalize output but ...
1
vote
1answer
34 views

find closest numbers approximately

I have a table like this: TableTest Col1 Col2 Col3 Col4 5 6 7 8 12 6 5 6 2 3.5 6 1 And I want to find the closest row ...
0
votes
1answer
37 views

Accuracy and Loss in MLP

I am trying to explore models for predicting whether the a team will win or lose based on features about the team and their opponent. My training data is 15k samples with 760 numerical features. Each ...
0
votes
0answers
10 views

The Type of response variable in linear mixed model

Iam working on mixed linear model and I want to predict the sucess or the failure of the campaign . Is it possible for response variable in mixed linaer mode to be as 0 or 1
0
votes
0answers
11 views

Regression variable colums (not all measures available)?

I have a dataset it looks this way X = [location, measure_type, value] y = [target_value] The goal is to predict the target_value. It is the same for each location. So for example for location "A" ...
2
votes
1answer
42 views

How to do vocabulary estimation based on observed writings?

Below is a scatter plot of the data set I am dealing with. The X axis is the total number of words per essay for a particular individual, and they Y axis is the number of unique words. In principle, ...
0
votes
0answers
11 views

How to add control variable in regression using sklearn

I am trying to perform controlled regression using sklearn, I have been using sklearn for fitting dependent variable and independent variable, however, if there is a variable that I want to control ...
0
votes
2answers
23 views

How to handle continuous values and a binary target?

This is going to be a very beginner's question. I have a datset of continues features like LoanAmount, LoanDuration(multiclass?), ... ClientIncome, ClientFreeSources, etc. and a binary target whether ...
0
votes
1answer
20 views

How do you search a high dimensional for the global maxima using as few samples as possible?

Suppose the value at any point in the space is defined by Y = f(x1, x2 .. xk). For simplicity, we can assume that x takes only binary values. Which means that we have a total of 2^k possible values. I ...
-1
votes
1answer
189 views

k-fold cross validation in keras for regression using sklearn [closed]

I am using a wrapper to use sklearn k-fold cross-validation with keras for a regression problem with ANN. but the accuracies i get look very weird. It has worked fine for a classification problem. I ...