Questions tagged [linear-regression]

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

Filter by
Sorted by
Tagged with
3
votes
0answers
31 views

Linear Regression bad results after log transformation

I have a dataset that has the following columns: The variable I'm trying to predict is "rent". My dataset looks a lot similar to what happens in this notebook. I tried to normalize the rent ...
0
votes
0answers
7 views

Gaussian Mixture Classification Implementation with multidimensional trainning data

I'm trying to implement the gaussian mixture classification (GMC) implementation from scratch using python. The training dataset consists of 10 folds each of size $\left[100x64\right]$. In addition, ...
2
votes
2answers
25 views

Good approach to increase accuracy for a continuous value that is highly variable/sensitive to the inputs?

I am trying to predict a continuous 'Y' variable using a variety of algorithms and feature engineering techniques. My issue is that Y is extremely variable and I reached a asymptote in accuracy. This ...
0
votes
2answers
24 views

What's the intuition behind nonlinear predictors in multivariate regression?

I'm learning statistical learning with the well known ISLR (Introduction to Statistical Learning with Applications in R) and doing the exercises, right now in the linear chapter regression. Despite ...
1
vote
2answers
31 views

Does Homoscedasticity applies only for linear regression models?

In statistics, a sequence of random variables is homoscedastic if all its random variables have the same finite variance. This is also known as homogeneity of variance. (Wikipedia) Is this assumption ...
0
votes
0answers
19 views
1
vote
2answers
26 views

Linear Regression with Category variables

I'm currently learning and exploring machine learning and understand the basics of linear regression based on two numerical variables, but now I wish to go a little further and need some guidance ...
0
votes
1answer
25 views

Using Linear Regression to Learn Polynomial Regression

Let's start by considering one-dimensional data, i.e., $d=1$. In OLS regression, we would learn the function $$ f(x)=w_{0}+w_{1} x, $$ where $x$ is the data point and $\mathbf{w}=\left(w_{0}, w_{1}\...
0
votes
1answer
21 views

Why not using linear regression for finetuning the last layer of a neural network?

In transfer learning, often only the last layer of the network is retrained using gradient descent. However, the last layer of a common neural network performs only a linear transformation, so why do ...
0
votes
1answer
19 views

Regression and Classification, which is better in financial market price prediction?

I want to use a model to trade in finanical market. which i have several features, like macd, rsi, or other common features. and my target is to make a tradeable predict in every time point. so my ...
0
votes
0answers
17 views

Is it a good idea to use the mean and standard deviation of coefficients from other models as my prior in Bayesian Regression?

I have a dataset that I’ve been playing around with for school I have gotten very good results with a bunch of methods (Ridge, Lasso, ElasticNet, SVM, Bagging, Stacking and NN even) Now I’m having a ...
0
votes
1answer
20 views

Why is shuffling timeseries a bad thing?

I'm trying to understand precisely why it is a bad idea to shuffle time-series when splitting train and test data. Like, what is false about shuffling time-series? How does it tamper with the model?
1
vote
0answers
34 views

How to maximize a log linear regression equation satisfying a constraint?

I have a log linear equation of the form $y = w_1(\log{X1}) + w_2(\log{X2}) + ... + w_n(\log{Xn})$. How can I find the value of X's that maximize the value of y subject to a constraint $(X_1+X_2+...+...
1
vote
1answer
25 views

Mathematical bias and weight vs machine learning bias and weight

I am a little confused about the term Bias and Weight with respect to machine learning. Say we want to predict the heights of people whose weights are given. So plot weights to x-axis and height to ...
0
votes
1answer
23 views

unable to predict by LinearRegression

Should I add csv as text in SO question? There's lot more data. ...
1
vote
0answers
16 views

How to generate data for future price predictions with Linear regression and Python

I have written the following Python code that makes predictions using linear regression model. However, I'll appreciate you pointing out what went wrong with how I generated data['X_forecast']. I'm ...
0
votes
0answers
14 views

Theano error when performing Linear Regression

I'm trying to perform Linear Regression using Theano, but there is something I might be missing or doing wrong because I receive an error message, here you have a reproducible example: ...
1
vote
1answer
22 views

Has anyone heard of a model similar to a random forest which fits a linear regression model in its leaf nodes?

That is, each leaf node in each decision tree learns a linear model. Anyone heard of this kind of model? Even better, anyone know of implementations?
0
votes
0answers
48 views

sklearn gridsearch lasso regression: find specific number of coefficients

I am using GridSearchCV and Lasso regression in order to fit a dataset composed out of Gaussians. I keep this example similar to this tutorial. My goal is to find the best solution with a restricted ...
0
votes
1answer
24 views

How does the equation “dW = - (2 * (X^T ).dot(Y - Y_hat)) / m” comes in Linear Regression (using Matrix + Gradient Descent)?

I was trying to code the Linear Regression in Python using Matrix Multiplication method using Gradient Descent and followed a code where there was no mention what is the loss but just a code as Per ...
0
votes
1answer
25 views

LinearRegression with fixed slope parameter

I have some data $(x_{1},y_{1}), (x_{2},y_{2}), ..., (x_{n},y_{n})$, where both $x$ and $y$ represent real numbers (float). I want use Scikit-learns LinearRegression model to fit a model of the form: $...
1
vote
0answers
19 views

How to process categorical variable having lots of unique values in linear regression?

I have House Price dataset and I am using linear regression to predict the house price. while data preprocessing I found a variable called "Location" and it have around 342 unique value. For ...
0
votes
2answers
44 views

Selecting most important features for multilinear regression

I have a set of 25 features. I would like to choose the best features for my model. Originally, I was looking at the correlation of features with respect to response, and only taking those which are ...
0
votes
0answers
16 views

Regression for prediction: is there any benefit of regularization?

What is the benefit of having flatter regression line in linear regression? Is there a proven benefit for prediction? Are there experiments that show regression with regularization performs better on ...
0
votes
1answer
92 views

Dot product and linear regression

I'm studying PCA and my professor said something about finding the linear regression by doing the dot product of both axis. Could someone explain to me why? The dot product returns a number. What's ...
2
votes
2answers
50 views

How to interpret linear trends in residuals?

I am trying to compare companies in the same industry and see how Profit and Number of employees correlated. My linear regression looks something like this: Given the nature of the dataset, the model ...
0
votes
0answers
13 views

How to do feature reduction for a log-linear regression model

I'm building a log-linear regression model and I have 18 different variables in my model. 13 out of 18 variables I'm using are hot-encoded variables for holiday, e.g. showing which holiday it is. I ...
1
vote
0answers
57 views

Can elastic net l1 ratio be greater than 1?

I have multiple datasets that I trained with ElasticNetCV (sklearn), and I noticed that many of them selected l1_ratio = 1 as ...
0
votes
0answers
6 views

Encoding Data and huge loss during ANN training

I just started to learn on ANN and tried to experiment on my own on a Linear Regression. I got a dataset which had housing prices for a city. Tried going through this but my model gives me a huge loss....
3
votes
0answers
43 views

How to properly do feature selection when comparing different models? [closed]

Context: I'm currently crafting and comparing machine learning models to predict housing data. I have around 32000 data points, 42 features, and I'm predicting housing price. I'm comparing Random ...
0
votes
3answers
64 views

Whether Interaction terms should be included in Linear Regression analysis?

I am working on a linear model with 6 independent variables and when thinking about including an interaction I got lost. An interaction exists if the level of one independent variable is affected by ...
3
votes
1answer
35 views

How best to use the resale transaction year in predicting housing prices?

I'm looking into the classic problem of predicting apartment prices (resale market) based on the their type, size, location, etc. Pretty straightforward and Linear Regression or Regression Trees give ...
3
votes
1answer
79 views

Why do machine learning engineers insist on training with more data than validation set?

Among my colleagues I have noticed a curious insistence on training with, say, 70% or 80% of data and validating on the remainder. The reason it is curious to me is the lack of any theoretical ...
1
vote
2answers
25 views

Python function returning a 4x4 matrix instead of a floating number like in an equivalent Octave function in a Linear Regression problem

I am trying to translate code from Octave to Python, and I am stuck. I am aware they are libraries out there such as scikit-learn etc., but for my own learnin,g I would like to be able to implement ...
1
vote
1answer
25 views

Is knn similar to this version of k-means?

If we use k-means in a dataset where k is equal to the number of points in the dataset, and each cluster is made out of only a point. Considering that we have given a distance method, we can classify ...
3
votes
1answer
74 views

Which definition of Likelihood function is correct?

In the online version of the Deep Learning book on chapter 5 the estimator for likelihood function is defined as: That is the product of individual probabilities. After taking the log it arrives at ...
1
vote
0answers
6 views

Error term in probabilistic interpretation of least squares update rule

I have read in Stanford's CS229 course notes that to justify the least-squares update rule with probability, the following is assumed: $$y^{(i)} = \theta^Tx^{(i)}+\epsilon^{(i)}$$ , where $\epsilon^{(...
0
votes
0answers
9 views

Enforce Floor limit when predicting values using Multioutput Regression with Gradient Booster

I have a very simple program below that builds a model using multi-output regression. Even though all the training data consists of positive float values I'm discovering that predictions made often ...
1
vote
0answers
8 views

How do you calculate how many coefficients are necessary in polynomial regression?

So I can't seem to find much on this by searching so I came here. Let's say I had 3 variables $x_1,x_2,x_3$ and the let's say the degree of the polynomial was $d=2$, I can define the length of a ...
0
votes
0answers
105 views

How to retrieve results summary from statsmodels GLM with regularization?

I'm trying to fit a GLM to predict continuous variables between 0 and 1 with statsmodels. Because I have more features than data, I need to regularize. ...
0
votes
0answers
11 views

Linear Regression Model Validation with Transformed Data

I worked on a model that I applied a log10 transformation to the dependent variable. I am having trouble with manually calculating the R2 for both train and test dataset. The model looks like this. <...
0
votes
1answer
20 views

For very simple linear regression can we quantify the prediction accuracy hit between using one hot encoding and simple numerical mapping?

Suppose I had a simple linear regression model that had the following input or X variable: ...
0
votes
0answers
10 views

Are units sold,order quantity,Profit,loss continuous or discreet?

Continuous :Cant be counted and has no limit Discreet : Can be counted Please help to understand if my assumption is right or wrong linear regression is used to predict only continuous variables so ...
1
vote
1answer
43 views

Confused about polynomial regression with multiple variables

I'm trying to create a multivariable polynomial regression model from scratch but I'm getting kind of confused by how to structure it. So, I have an array of feature vectors such that each vector can ...
0
votes
1answer
29 views

Train error vs. Test error in linear regression by samples analysis

I have run a multivariate linear regression model on a small set of about 3500 samples. While the model's error is as large as expected, I also ran a bias vs. variance analysis by comparing the train ...
0
votes
0answers
31 views

Multivariable linear gradient descent resulting in inf

I am trying to implement a multivariable gradient descent algorithm, it seems to start working fine, and works on smaller datasets, but applying it to larger datasets the variables overflow and cause ...
0
votes
2answers
65 views

Predicting the likelihood that a prediction from a linear regression model is accurate

So to set up the problem: I have a data set that had labeled data like colour, brand and quality as independent variables and the dependent is RRP (price). I have made a linear regression model using ...
2
votes
0answers
23 views

Interpretation of the output from qqPlot (using car library)

Basically, I have created a linear model and am testing to verify the normality of my errors. As a result, I have used the qqPlot function from the car library and have gotten the graph that can be ...
2
votes
0answers
22 views

Reproduce Figure 3.2 in Introduction to Statistical Learning

Has anyone reproduced Figure 3.2 in Introduction to Statistical Learning (James et al)? https://trevorhastie.github.io/ISLR/ISLR%20Seventh%20Printing.pdf They have a contour plot with circles. Here is ...
1
vote
0answers
20 views

Performing Regression on Text and Image together in the most efficient way

I have a dataset with texts and images. The texts are present in a CSV file, which I am able to read using Pandas. The CSVs contain the image names, and I have the corresponding pngs which are ...

1
2 3 4 5
12