Questions tagged [linear-regression]

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

Filter by
Sorted by
Tagged with
0
votes
0answers
4 views

Loss function for normal distribution regression problem

My project involves training an input of random uniformly distributed data using regression (this is my approach) to output random normally distributed data. The issue with formulating the problem is ...
1
vote
1answer
33 views

What's the correct cost function for Linear Regression

As we all know the cost function for linear regression is: Where as when we use Ridge Regression we simply add lambda*slope**2 but there I always seee the below as cost function of linear Regression ...
0
votes
0answers
13 views

data analysis leads to linear regression model: how to proceed with prognosis?

Data analysis of a large dataset of project management data together with working hours led me to a surprisingly simple linear model over the key milestones of all projects. Now I am a bit at loss on ...
1
vote
0answers
19 views

AI algorithm model that outputs a list of unknown length [closed]

I have a dataset with the following x columns: date time is_weekend is_holiday start_intersection end_intersection The output is a list of intersections, that connect start_intersection with ...
1
vote
0answers
17 views

Does the appliance of R-squared to non-linear models depends on how we calculate it?

Does the appliance of R-squared to non-linear models depends on how we calculate it? $R^2 = \frac{SS_{exp}}{SS_{tot}}$ is going to be an inadequate measure for non-linear models since an increase of $...
0
votes
0answers
20 views

average of two models with training set N/2 vs one model with training set N

I'm new to ML and I got a question about training model. Imagine linear regression $Y=\beta^TX+ \epsilon$ and we have training set D (size=N). I have two options: Train model use whole D and we get $\...
2
votes
1answer
37 views

Does gradient descent always find global minimum for specific regression type?

From my understanding, linear regression is used for predicting an output based on an input using a linear equation that is optimally fitted to some input data. We choose the best fitted linear ...
1
vote
2answers
26 views

Multiple Linear Regression for House Price Prediction score is 0.28 [closed]

I am trying to make predictions using this dataset What I have done so far: Dropped the Administrative column Encoded the categorical data using ...
1
vote
0answers
19 views

Implementation of a perceptron

I want to implement a single perceptron for linear regression using the following formulas: the input data for the first case is one column (x(392, 1); y(392, 1)) and for the second case is (x(392, 7)...
1
vote
1answer
20 views

One predictor variable and 3 response variable (categorical and continuous) [closed]

If I have predictor variables which are a mixture of continuous and categorical, and a response variable that is continuous. What approach should I apply? Linear regression, logistic regression or k ...
1
vote
1answer
22 views

The effect of the λ in the Ridge regression

Why by increasing value of λ in Ridge estimator the slope of the line is decreasing? How exactly λ affects to the y = kx + b?
1
vote
0answers
8 views

Error while calculating accuracy and matrix multiplication in tensor flow code for regression [closed]

I was writing a code for linear regression using tensor flow but I was getting errors while calculating matrix multiplication using tensor flow and while calculating accuracy. ...
0
votes
0answers
14 views

Can I retrain my best model on all available data? [duplicate]

I split data on Zillow single-unit properties into train-validation-test 70-15-15 and trained a few different sklearn linear models to predict selling price. I chose the best one based on validation ...
1
vote
0answers
14 views

Group points to reduce data set such that the linear regression stays the same

I have a very long dataset and I'm trying to reduce it by grouping the data in periods of 24 hours. In this way, there will be a single data point that represents that day, but they must yield the ...
0
votes
0answers
7 views

cost function diverging in batch gradient descent

I am trying to implement the gradient descent method in python. I would like the calculation to stop when abs(J-J_new) reaches a certain tolerance level (i.e. it ...
1
vote
1answer
16 views

Approximating weight of individual items from sum of their weight

Problem I have a list of orders, approximation of their total weight and list of items they contain. I need to determine approximate weight of individual items. In other words, I have a few thousand ...
1
vote
0answers
19 views

Linear regression of times series data with heteroskedasticity

I am trying to find out if stock market movements, on average and in extreme conditions, affect gold prices. I am following the regression model proposed by Baur and McDermott (2010) which is given as:...
3
votes
2answers
112 views

Constraining linear regressor parameters in scikit-learn?

I'm using sklearn.linear_model.Ridge to use ridge regression to extract the coefficients of a polynomial. However, some of the coefficients have physical ...
0
votes
0answers
16 views

Ordinary Least Squares: How to find the two features that jointly minimize mean squared error given individual coefficients?

Say you are given $d$ features with $N$ data points and trying to predict target $y$. You first try to fit an ordinary least squares model to each feature (+ bias term), that is finding $(\beta_{0i}, \...
1
vote
0answers
17 views

Sudden jumps in accuracy with logistic regression and bag of words : "glm.fit: algorithm did not converge"

I work on a bag of words, on the Toxic Comments Classifications challenge. The challenge is closed but the dataset is very nice to learn. I use R, tf-idf, tm, and logistic regression. I have a strange ...
0
votes
0answers
31 views

Gradient descent in linear regression converges but the trend line is incorrect

For the dataset https://physics.info/linear-regression/dash-world.txt, I have been trying to implement linear regression for predicting the men record times as a function of year. I have used gradient ...
1
vote
0answers
17 views

Detecting a Piecewise, Noisy, Linear Signal, with Constant Slope and Changing Y-Intercepts

I am trying to algorithmically detect a 2D linear signal under some noisy data. It is almost a textbook candidate for Robust Linear Regression, except for the fact that, while the slope remains ...
0
votes
0answers
62 views

How to conclude from RMSE and R-suared value that our model is good or bad?

I used two different columns from dataset as targets and using logistic regression. Output for target 1 ...
0
votes
1answer
22 views

Linear regression and gradient descend equations

I'm pretty new to ML and was starting out with linear regression combined with gradient descend. This is the equation I was trying to achieve using javascript- And this is what I came up with in js- <...
0
votes
1answer
24 views

Run linear regression fit on 2 1D array [closed]

On doing this- ...
0
votes
1answer
11 views

Gradient descent different implementation cause error

We know that we can get closer to the local minimum of the function by descending our argument according to that rule $$w1 = w0 − γ∇f$$ For example I have a linear regression model that depends on $b,...
1
vote
2answers
52 views

How do I decide on my X and Y variables for the prediction of a coin toss?

So I'm new to data science and was trying to solve a few problems that my mentor gave to me. I came across this question where there are multiple coin tosses and ten of them are recorded. I am ...
0
votes
0answers
15 views

prove E[(TSS - RSS)/p] > $\sigma^2$ in multiple linear regression

In Intro to statistical learning, Chapter-3 for Linear Regression, in the subsection 3.2.2 , Unit "One: Is There a Relationship Between the Response and Predictors?" , it is mentioned that: ...
0
votes
0answers
25 views

Linear regression to find differences between model performances

For one of my projects I needed to create classification models for each of many products. In order to see which classifier performs best, I created one SVM, RandomForest and Naive Bayes model for ...
0
votes
0answers
12 views

predicting average time with regression

I have a trip duration dataset that looks like this: I want to use other parameters to predict the waiting time (wait_sec). The waiting time refers to the time the vehicle is stuck in traffic or so. ...
0
votes
1answer
25 views

Why is linear regression not doing worse with a low weighted attribute?

I've been able to build a few linear regression models that can predict a material strength quite well: minimum RMSE of 17.95 using 11 attributes that I have selected from 159 original attributes. The ...
0
votes
0answers
13 views

Linear models: Imputing missing not at random

This question is a continuation of a similar question for linear models instead of Tree-based model. Given that linear models (e.g. lasso, ridge, Linear regression, elastic net, etc.) can't handle ...
1
vote
2answers
65 views

Is it usual for Scikit learn's standard scaler to cause non-invertibility?

For example, I am trying to perform linear regression on the following set of data Data examples: $X = [[1, 20], [3, 40], [5, 60]]$ (each row is an example, there are three examples, each with a ...
0
votes
2answers
47 views

Definition of linear model

I am new to machine learning and am a bit confused about the definition of a linear model. I've searched many sources and the most common definition is: The term linear model implies that the model ...
1
vote
0answers
21 views

Statistical significance of SVD least squares

I was not able to find any info on how least squares using singular value decomposition should be statistically evaluated. I have a dataset for which I did both multivariate regression and regression ...
0
votes
0answers
20 views

Best approach for univariate time series predictions?

I have a univariate time series. where I'm trying to predict a current value of a variable based on the previous 10 values of the same variable. I tried three approaches: 1- linear regression where I ...
1
vote
0answers
9 views

Does anyone know of literature regarding a Neural Net boosted GBM?

For obvious reasons, most GBMs created in the private sector are tree boosted. Occasionally, one might want a linear boosted GBM so that the residual models collapse into a simple linear combination. ...
1
vote
1answer
43 views

Extremely negative r^2

I use a linear regression to predict house prices (https://www.kaggle.com/c/house-prices-advanced-regression-techniques/overview). My linear regression sometimes works great with R^2 of 0.8 and ...
1
vote
1answer
23 views

Creating radial basis for linear regression Python

I'm trying to do time series forecasting with linear regression like it's done in this video: Radial basis forecasting starting from 5:50. I understand the basic idea of basis, but I don't think I ...
2
votes
1answer
75 views

is SST=SSE+SSR only in the context of linear regression?

the problem of regression is to minimize the sum of squared errors, i.e. $\sum\limits_{i=1}^n (y_i - \hat{y}_i)^2 = 0$ . But only in linear regression could you use the expression $\hat{y}_i = \beta_0 ...
1
vote
0answers
32 views

I have a data set of optimal values after simulations, How can I find if this dataset follows a specific pattern or any relation exists?

In the simulation I am conducting, I have a set of triangles and I select the optimal triangle based on my metric. After every simulation, I obtain an optimal triangle and I note down the lengths of ...
1
vote
0answers
22 views

Linear regression on sparse matrix?

I have a matrix with sparse data. A small extract from it is seen below. The columns represent years and the rows represent different race tracks. The feature values are velocities on that specific ...
1
vote
2answers
50 views

ideal algorithms to demonstrate overfitting or underfitting

When one tries to look up concepts such as overfitting and underfitting, the most common thing that pops up is polynomial regression. Why is polynomial regression often used to demonstrate these ...
0
votes
0answers
9 views

Downward trend in the residuals vs fit plot with constant variance interpretation?

I have build a regression model where the residual vs fit plot indicates a downward trend with a constant variance. I am having trouble interpreting and understand the problem at hand. Below is the ...
0
votes
0answers
24 views

How can I use transfer learning to predict height given age in Japan, using a model developed with USA data?

Suppose I have a (training) set of $n$ observation $\{(Y_i^{(U)},X_i^{(U)})\}_{i=1}^n$ of age $X_i^{(U)}$ and height $Y_i^{(U)}$ from people in the USA. Now suppose I also have a (test) set of $m$ ...
1
vote
0answers
168 views

Linear Regression error - InvalidArgumentError: assertion failed: [Labels must be <= n_classes - 1] [Condition x <= y did not hold element-wise:] [closed]

I am doing a linear regression model in TensorFlow. I have applied the code I saw on a course with my own dataset but I am getting an error I don't understand: This is a link to the Google ...
0
votes
0answers
18 views

Why is my training score below my testing score?

I'm learning data science, and currently practicing with the Titanic Dataset. I'm doing a simple logistic regression using scikit-learn, and plotting the learning curves of that model with Matplotlib: ...
2
votes
1answer
35 views

How to fit a KNN and then a linear regression with those neighbors?

How do I fit a KNN to get the $k$ nearest neighbors and then aggregate the those neighbors into a fit using a linear regression (instead of a weighted average) in Scikit-Learn? I've tried creating a ...
4
votes
1answer
123 views

Linear Regression bad results after log transformation

I have a dataset that has the following columns: The variable I'm trying to predict is "rent". My dataset looks a lot similar to what happens in this notebook. I tried to normalize the rent ...
0
votes
0answers
8 views

Gaussian Mixture Classification Implementation with multidimensional trainning data

I'm trying to implement the gaussian mixture classification (GMC) implementation from scratch using python. The training dataset consists of 10 folds each of size $\left[100x64\right]$. In addition, ...

1
2 3 4 5
13