Questions tagged [regression]

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

Filter by
Sorted by
Tagged with
23
votes
2answers
23k views

Why do we need to discard one dummy variable?

I have learned that, for creating a regression model, we have to take care of categorical variables by converting them into dummy variables. As an example, if, in our data set, there is a variable ...
6
votes
2answers
355 views

Confidence interval interpretation in linear regression when errors are not normally distributed

I've read that "If the error distribution is significantly non-normal, confidence intervals may be too wide or too narrow" (source). So, can anyone elaborate on this? When are the confidence intervals ...
1
vote
1answer
482 views

Removing constant from the regression model

I am trying to calibrate two variables $(X,Y)$ of different measuring techniques from two instruments, the result of the linear regression analysis appears as shown in the image. The result shows the ...
11
votes
2answers
3k views

Solving a system of equations with sparse data

I am attempting to solve a set of equations which has 40 independent variables (x1, ..., x40) and one dependent variable (y). The total number of equations (number of rows) is ~300, and I want to ...
9
votes
4answers
2k views

Can Boosted Trees predict below the minimum value of the training label?

I am using gradient Gradient Boosted Trees (with Catboost) for a Regression task. Can GBtrees predict a label that is below the minimum (or above the max) that was seen in the training ? For instance ...
6
votes
3answers
2k views

xgboost: Is there a way to perform regression on rates/percentages data?

I have a dependent variable, $Y$, that is made up of rates/percentages data, so each value is between $0$ and $1$. I was attracted to the xgboost library because it allows focusing in on specific ...
3
votes
4answers
561 views

What is the difference between classification and regression?

I understand classification....a discrete response or category, like animal is dog or cat. The author says..."Regression techniques predict continuous changes such as the change in temperature, power ...
24
votes
5answers
71k views

Validation loss is not decreasing

I am trying to train a LSTM model. Is this model suffering from overfitting? Here is train and validation loss graph:
11
votes
2answers
10k views

MAD vs RMSE vs MAE vs MSLE vs R²: When to use which?

In regression problems, you can use various different metrics to check how well your model is doing: Mean Absolute Deviation (MAD): In $[0, \infty)$, the smaller the better Root Mean Squared Error (...
5
votes
2answers
19k views

Get multiple output from Keras

I have a regression problem which I have to predict 3 numerical values from a provided data. For example let's say I have a data set containing ...
5
votes
5answers
7k views

GridSearch without CV

I create a Random Forest and Gradient Boosting Regressor by using GridSearchCV. For the Gradient Boosting Regressor, it takes too long for me. But I need to know which are the best parameters for the ...
6
votes
3answers
9k views

Neural network for Multiple integer output

I have a data set that contains 135 input features and 132 output values to be predicted. The input features are all numeric floating point values and each output value would be an integer between [0,...
1
vote
2answers
111 views

Regression Algorithms in Production

I am interested in predicting if a doctor would prescribe a specific drug and have chosen Logistic Regression as a starting point. I have a few questions: Is feature selection the first step to take ...
1
vote
1answer
28 views

How to analyze repeated measure data for prediction?

In my work, we collect sales data of our products. We have a set of 1st level customers (lets call that group as jacks) with whom we do we business. These jacks then sell our products to end customers ...
0
votes
1answer
111 views

ANN regression accuracy and loss stuck

I have a data set on predicting solar power generation, the dataset has 20 independent var and 1 dependent. The accuracy of my model is stuck at 60%. I have tried several models but this accuracy is ...
28
votes
3answers
48k views

Neural Network for Multiple Output Regression

I have a dataset containing 34 input columns and 8 output columns. One way to solve the problem is to take the 34 inputs and build individual regression model for each output column. I am wondering ...
16
votes
4answers
40k views

What does "linear in parameters" mean?

The model of linear regression is linear in parameters. What does this actually mean?
27
votes
3answers
18k views

Why do we convert skewed data into a normal distribution

I was going through a solution of the Housing prices competition on Kaggle (Human Analog's Kernel on House Prices: Advance Regression Techniques) and came across this part: ...
10
votes
3answers
5k views

Is there a library that would perform segmented linear regression in python?

There is a package named segmented in R. Is there a similar package in python?
10
votes
3answers
33k views

Xgboost - How to use feature_importances_ with XGBRegressor()?

How could we get feature_importances when we are performing regression with XGBRegressor()? There is something like ...
9
votes
2answers
14k views

Tensorflow regression model giving same prediction every time

...
9
votes
1answer
9k views

Neural network with flexible number of inputs?

Is it possible to create a neural network which provides a consistent output given that the input can be in different length vectors? I am currently in a situation where I have sampled a lot of audio ...
5
votes
3answers
899 views

Quasi-categorical variables - any ideas?

Let's say I'm trying to predict a person's electricity consumption, using the time of day as a predictor (hours 00-23), and further assume I have a hefty but finite amount of historical measurements. ...
14
votes
3answers
5k views

Predict the best time of call

I have a dataset including a set of customers in different cities of California, time of calling for each customer, and the status of call (True if customer answers the call and False if customer does ...
9
votes
2answers
7k views

Proper way of fighting negative outputs of a regression algorithms where output must be positive all the way

Maybe it is a bit general question. I am trying to solve various regression tasks and I try various algorithms for them. For example, multivariate linear regression or an SVR. I know that the output ...
5
votes
3answers
4k views

Best regression model to use for sales prediction

I have the following variables along with sales data going back a few years: date # simple date, can be split in year, month etc shipping_time (0-6 weeks) # 0 weeks means in stock, more weeks means ...
5
votes
2answers
12k views

Prediction Intervals Using XGBoost

I want to obtain the prediction intervals of my xgboost model which I am using to solve a regression problem. I am using the python code shared on this blog, and not really understanding how the ...
15
votes
3answers
5k views

Modelling Unevenly Spaced Time Series

I have a continuous variable, sampled over a period of a year at irregular intervals. Some days have more than one observation per hour, while other periods have nothing for days. This makes it ...
8
votes
2answers
2k views

Fitting lines through large point clouds

I have a large set of points (order of 10k points) formed by particle tracks (movement in the xy plane in time filmed by a camera, so 3d - 256x256px and ca 3k frames in my example set) and noise. ...
4
votes
2answers
1k views

Interpreting the evaluation result of multiple linear regression

I am learning the multiple linear regression model. I've built a model and using R command: summary(model) I got this result: ...
3
votes
1answer
779 views

How is the 'feature_importance_' value calculated in sklearn random forest regressor?

I have 9000 sample, with five features, and one output variable (all are numerical, continuous values). I used random forest regression method using scikit modules. ...
11
votes
3answers
2k views

Can regression trees predict continuously?

Suppose I have a smooth function like $f(x, y) = x^2+y^2$. I have a training set $D \subsetneq \{((x, y), f(x,y)) | (x,y) \in \mathbb{R}^2\}$ and, of course, I don't know $f$ although I can evaluate $...
7
votes
2answers
6k views

Regression model to predict probability of rare event

I have a dataset with around 900.000 records, around 1000 of which are marked as positive (the studied event occurred). The probability of the event occurring is always low (i.e. < 0.1), and I ...
4
votes
1answer
126 views

Common Techniques to Generate from a Regression Neural Network Model

I am used to train neural networks that are designed for generation, such as GANs or VAEs. I am wondering what are the common techniques to generate data that would minimize the target/energy learned ...
1
vote
1answer
3k views

Predicting a Continuous output in a dataset with categories

Let's say this: I'm working on a machine learning project and I'm working on a dataset with a 4250,13 as shape and it is already grouped in 7 categories! Note that those categories can't be ...
6
votes
3answers
707 views

Regression: How to deal with positive skewness in continuous target variable

I'm working on a regression problem. My aim is to "learn" the distribution of a continuous target $y$ as good as possible to make predictions. My model looks like: $$y_i=\beta X_i + u_i.$$ $y$ is ...
4
votes
2answers
259 views

Prediction in Machine Learning

When we use a regression algorithm in out dataset it's because we assume that there is a relation between our input data and some quantitative value. This is expressed as : $y = f(x)+\varepsilon $, ...
3
votes
2answers
1k views

Are RNN or LSTM appropriate Neural Networks approaches for multivariate time-series regression?

Dear Data Science community, For a small project, I've started working on Neural networks as a regression tool, but I am still confused about possibilities of some variants. Here's what I am aiming ...
2
votes
3answers
3k views

How to convert regression into classification?

So I have a regression problem with bunch of features X, and labels in the amount (price $). How can I convert it to classification problem? I have read about convert label from continuous to ...
1
vote
1answer
37 views

What quantile is used for the initial DummyRegressor for Gradient Boosting Regressor in scikit-learn?

According to the documentation of Scikit-Learn Gradient Boosting Regressor: init: estimator or ‘zero’, default=None: An estimator object that is used to compute the initial predictions. init has to ...
1
vote
2answers
158 views

Chose the right regression analysis

In R I have data where head(data) gives ...
0
votes
2answers
88 views

how Lasso regression helps to shrinks the coefficient to zero and why ridge regression dose not shrink the coefficient to zero?

How Lasso regression helps feature selection of model by making the coefficient to zero? , I could see few below with below diagram ,can any please explain in simple terms how to corelate below ...
5
votes
3answers
7k views

Difference between Ridge and Linear Regression

From what I have understood, the Ridge Regression is just having the loss function for an optimization problem with the addition of the regularization term (L2 Norm in the case of Ridge). However I am ...
4
votes
2answers
3k views

Multi-output regression problem with Keras

number of features: 12 , -15 < each feature < 15 number of targets: 6 , 0 < each target < 360 number of examples: 262144 my normalization: I normalized the features so that they are ...
3
votes
1answer
2k views

Should the output of regression models, like SVR, be normalized?

I have a regression problem which I solved using SVR. Accidentally, I normalized my output along with the inputs by removing the mean and dividing by standard ...
3
votes
2answers
7k views

Large mean squared error in sklearn regressors

I'm a beginner in machine learning and I want to build a model to predict the price of houses. I prepared a dataset by crawling a local housing website and it consists 1000 samples and only 4 features ...
2
votes
1answer
55 views

What method/algorithm for constrained multi-target regression

I am working with three dimensional measurement data and want to model them using a multivariate linear regression. I have already implemented a simple gradient descent algorithm to solve the classic ...
2
votes
2answers
41 views

How much can bias decrease performance of the network at the beginnng of the training?

I am writing a custom framework and in it I'm trying to train a simple network to predict the addition function. The network: 1 hidden layer of 3 Neurons 1 output layer cost function used is ...
2
votes
2answers
42 views

Using a neural network to learn regression in image processing

I have a camera system with some special optics that warp the field of view of the camera, dependent on two variables, $\theta_1$ and $\theta_2$. Given a specific configuration of these two variables, ...
2
votes
2answers
60 views

Boosted tree regression loss function when data has occasionally very large values to predict?

I have a regression problem where most of my target variables are down in the range 5-30, but occasionally the target variable will spike up to 100, 500, or even 5000. These values are not spurious ...