Share Your Experience: Take the 2024 Developer Survey

# Questions tagged [linear-regression]

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

773 questions
Filter by
Sorted by
Tagged with
1 vote
58 views

### Mean Absolute Error from Scratch in NumPy

I recently tried implementing MAE from scratch in NumPy. The loss value and the slope seem to be equivalent to what Scikit-learn outputs, but for some reason the intercept value seems to converge to ...
• 11
1 vote
34 views

### Are there any general theoretical results about the behavior of data in the neighborhood of a single data point?

I know from calculus that any relatively well-behaved function $y=f(x)$ can be approximated by a linear function $y=ax+b$ within a sufficiently small neighborhood around each point of an independent ...
• 1,146
11 views

### Should you seasonally decompose TS data before linear regression?

I want to apply the U-MIDAS method which is basically Least Square regression to a cross sectioned time series. Do I need to seasonally decompose my X and Y and should I test for unit root? Some of ...
23 views

### Using a very very small learning rate to not diverge?

i just started with machine learning and today i tried implementing the gradient descent algorithm for linear regression. If i use a bigger value for alpha(the learning rate) the absolute value of w ...
1 vote
29 views

### Linear regression with confidence interval

I am running a multivariate linear regression on noisy data, where the amount of error for each measurement is known (or at least estimated). It works reasonably well with weighted linear regression ...
• 111
14 views

### Data splitting for OLS regression

This is what I have done :: divided my dataset into training and testing sets--> got significant features via. feature selection using sequential feature selector ( MLxtend) on the training set--&...
123 views

### What type of technique can be used to solve this question?

Apology for the ambiguous title, I do not know the term. I have data of some products which a few variables: origin, weight, brand. For example: Product A = "China, 100g, Brand X" Product ...
• 1,094
37 views

### Correlation between predictions vs correlation between targets

In a multi-target model framework - where a separate model is estimated for each target - how can one take into account for correlations between targets during the training process ? For example say I ...
• 121
7 views

### The best order for analysis steps in building econometric model with time series linear regression

I am working on a project whose goal is to build a linear regression model for a time series dataset. I was provided with a blueprint of all required analysis steps. This led me to wonder what is the ...
• 103
22 views

### Machine learning model that takes multiple records as input to help predict the last

I want to create a ML model that is able to forecast the yield from a farm. My data source gives me data about the inspections from the field, but that is too much info to fit in 1 record, so there ...
22 views

### ML Methods For Modelling Latent Variables

I have some time series predictor variables, $\{\mathbf{X}_t\} = \{\mathbf{X}_0, \ldots, \mathbf{X}_n\}$, and some other time series data $\{\mathbf{Z}_t\} = \{\mathbf{Z}_0, \ldots, \mathbf{Z}_n\}$. ...
• 101
17 views

### 5-fold cross validation in R: getting error Age variable different lengths

I am tasked to do a 5-fold cross validation for my R grad course with pga golf data. I continually get an error for a certain variable, Age, saying different lengths. Here is the error code: ...
1 vote
51 views

14 views

### Autoregressive forecasting with distinct models problem

I got $n$ features - $f$ (used as an input as well as a target). Since I'm using linear regression and want to avoid situation in which weights of a model fit not only for $fi$ but for all of $f$ (...
112 views

### Why linear kernel regression is equivalent to plain linear regression?

I am trying to understand either intuitively/geometricaly and/or mathematicaly why the followings are equivalent: Classic Ordinay Least Squares linear regression Linear-kernelized Ordinary Least ...
• 101
16 views

### Can reducing information improve regression prediction?

Variable A is either 0 or 1. It is 0 if the sum of variables a + b + c + d … is less than some constant threshold, and is 1 if the sum of variables a + b + c + d … is greater than some constant ...
• 111
1 vote
32 views

### With infinite observations, would the weights resulting from ridge regression be the same as simple linear regression?

As the number of observations approaches infinity, do the weights of a linear regression approach the weights of a linear regression with L2 penalty?
• 111
107 views

### Workflow when making a machine learning model

I'm new to data science, and kinda confused with the workflow and steps to make a model. Before learning the math and concepts behind the algorithms like SVM, linear regressions, etc, I would just ...
1 vote
75 views

### Linear Regression and Logistic Regression

I'm a beginner, and I'm wondering whether a logistic regression in a nut-shell is just normalizing a linear regression? Correct me if I'm wrong, but I came to this conclusion because the predicted ...
12 views

### Using nearest neighbor in RANSAC

I found many resources online talking about nearest neighbor concept in RANSAC. For example, figure 2 of this paper, this article and this repo talk about nearest neighbor in the context of RANSAC. ...
• 103
453 views

### Why is it difficult to use a linear regression model for the classification problems?

Why is it difficult to use a linear regression model for the classification problems?
18 views

### Should I use an intercept even if my regression model's r-squared value reduces by a lot?

I'm using Python to create a good linear regression model and am having trouble getting good results for my r-squared value. A quick rundown of what the data is: – Sales: This dependent variable ...
42 views

### How to properly linearize data (if possible)

I was assigned the task of linearizing some of my data, which exhibits a non-linear appearance. When using the distfit library, it indicated that my data's distribution is closest to a gamma function. ...
24 views

### Calculating the solution of OLS efficiently when adding one feature at a time

We know that the analytical solution for an OLS problem is $𝛽̂ =(𝐗^T𝐗)^{-1}𝐗^𝑇𝐲$. I am looking for an efficient algorithm to solve for $𝛽̂$ when I add one feature at a time. More specifically, ...
• 101
1 vote
45 views

### One-Hot encoded variables dominates importance among other variables

I am currently training some machine learning models to predict the 28-day compressive strength of cement, a continuous real-valued variable. The available dataset comprises samples from three ...
• 21
52 views

### Data Cleanup for Regression

I have a simple dataset of 1 output and 1 input and want to fit a linear regression to the dataset. The data has a certain level of noise to it (potentially driven by another input, which I will ...
20 views

### What are some Models/Methods to reduce noise using environmental data?

I have a set of pressure datasets from a mechanical device that frequently moves around the country. I also have several sets of environmental data (Altitude, ambient temperature etc.) from those ...
674 views

### Parameter estimation in linear regression

Another test Q I couldn't answer - We have marks of students belonging to 3 sections - A,B,C and two genders - M & F. Which regression model will not be able to estimate all the parameters? 1 ) ...
• 197
35 views

### What Model to Choose for a NN with a Very Wide Output Layer?

The input of my neural network consists of 20 features, whereas the output consists of 20,000 of them (predicting a "quantum classical shadow" based on a few parameters: the rotation angle ...
• 21
22 views

### ValueError: operands could not be broadcast together with shapes (13159,3) (13159,)

I am trying to predict the target variable and finding the difference from actual variable using polynomial regression. However predicted variable is an array of 3 dimension with the shape as (13159,3)...
63 views

### A hypercube with side length 1 in d dimensions is defined to be the set of points

The Question: A hypercube with side length 1 in d dimensions is defined to be the set of points (x1, x2, ..., xd) such that for all j = 1, 2, ..., d. The boundary of the hypercube is defined to be ...
25 views

### Mean Absolute Error vs Mean Squared Error

why MAE is not used widely unlike MSE? In what scenarios you would prefer to use one over the other. Explain mathematically too. I was asked in an interview. I referred MSE vs MAE in linear regression ...
• 161
81 views

### Linear Model With Highly Correlated Attributes Producing Inconsistent Weights

I know that having correlated attributes violates the linear model assumption of independent attributes, and I'm not interested in creating a more sophisticated model to tease apart the dependent ...
91 views

### Can I decompose SHAP interaction values like a linear regression?

I had a question regarding the shap interaction matrix. Suppose I have 500 samples with 2 features. Then my interaction matrix becomes (500,2,2). I want to calculate the SHAP values of each feature ...
1 vote
76 views

### Why Cost function is differentiable?

I've a very basic question about cost functions. I'm studying gradient descent and there we're using partial differentiation of features "Theta". But isn't the cost function an absolute ...
• 11
70 views

### Does LinearRegression uses Gradient Descent for finding slope and y-intercept of the best fit line?

I know that Gradient Descent is an optimization algorithm used for optimizing the cost of the loss function. Does Linear Regression model of the sklearn package use ...