Questions tagged [pearsons-correlation-coefficient]
The pearsons-correlation-coefficient tag has no usage guidance.
35
questions
0
votes
0
answers
39
views
Similiar reconstruction for Pytorch VAE
This is my first question here, so if I don't offer enough information for my question to be answered, please let me know.
I am currently working on my Bachelor Thesis, in which I aim to integrate ...
0
votes
1
answer
36
views
Pearson coefficients different for same data?
Is it possible to have different Pearson coefficients over the same input data?
- 17
0
votes
1
answer
28
views
Difference between pearson correlation and shapley
When we say pearson correlation we are talking about the correlation between input with other inputs or an output. Ok. Does that necessarily mean if we have + value of Pearson between input A and ...
- 113
2
votes
1
answer
341
views
If my target variable is binary, is it better to use Pearson's or Spearman's for my correlation vector?
I'm using a corr vector, combined with RFE, to perform feature selection.
I keep reading conflicting things online as to whether I should use Pearson's or Spearman's...
- 23
0
votes
0
answers
22
views
Partial correlation coefficient - where is it used?
Recently I've learnt about something called partial correlation coefficient (denoted as $\rho_{i,j|1...i,j...n}$ or in short, say $\rho_{i,j}$), which is like Pearson correlation between variables $...
- 103
0
votes
0
answers
12
views
Two terms - observed correlation and population correlation are used frequently? Clarify the definitions
population correlation or true correlation is different from observed correlation? what is the purpose of finding population correlation?
- 605
0
votes
0
answers
17
views
Averaging subregion correlation coefficients into a single measure
I've got a 257x257 correlation matrix of functional connectivity (fMRI) data. It is a symmetric matrix where each value is the Pearsons correlation of the brain area in the row with the brain area in ...
- 101
0
votes
2
answers
410
views
How do I combine two different measures of correlation coefficients?
In the dataset, we have a numerical feature and a numerical target. We are calculating the Pearson coefficient and Spearman rank correlation.
Pearson to track the linear relationship
and Spearman to ...
- 391
0
votes
0
answers
14
views
Find the most impactfuls parameters multivariate output unsupervised ML
I am currently on a proect where my df has more than 600 parameters of analog sensors (A parameters) and about 50 other parameters (F parameters). I want to find for each of these 50 parameters (F ...
- 1
2
votes
2
answers
57
views
Which method of correlation is appropriate for two paired lists of numbers?
I have a program which produces an image, and I use a metric to understand how accurate that image is. I choose five cases (A, B, C, D, E), and make a list of the accuracy metric for each case:
...
- 21
0
votes
1
answer
117
views
Scaling and handling highly correlated features in tabular data for regression
I am working on a regression problem trying to predict a target variable with seven predictor variables. I have a tabular dataset of 1400 rows. Before delving into the machine learning to build a ...
- 101
0
votes
1
answer
1k
views
Python: calculate the weighted average correlation coefficient
I am calculating the volatility (standard deviation) of returns of a portfolio of assets using the variance-covariance approach. Correlation coefficients and asset volatilities have been estimated ...
- 101
1
vote
1
answer
133
views
Correlation analysis yields conflicting results. Positive Pearson and Negative Spearman
I have four features x1,x2,x3,x4. All of their correlation with y are similar in Pearson correlation and in Spearman rank correlation separately. However, all these are +0.15 in Pearson and -0.6 in ...
- 113
1
vote
0
answers
478
views
Is pearson correlation matrix a good indicator for label encoded categorical and numeric independent data?
I have a dataset having 22 independent variables out of which 15 are categorical data that has already been label encoded i.e the dtype is int64 and the contents are in a range of 0 to n (n is the ...
- 113
0
votes
1
answer
76
views
Correlation with target variable for regression problem
Given the following dataframe
age job salary
0 1 Doctor 100
1 2 Engineer 200
2 3 Lawyer 300
...
with ...
- 755
1
vote
0
answers
119
views
Optimizing rolling Pearson correlation
I have Pandas DataFrame with multiple columns (3000 or more) with timeseries in them (I have dates as indecies).
...
- 113
0
votes
0
answers
125
views
Agglomerative Clustering (average linkage) and Pearson Correlation
Does having a positive or negative correlation between features being clustered affect the agglomerative clustering result?
I have three columns in my dataset, and I'm trying to figure out if I should ...
- 101
1
vote
0
answers
648
views
Pearson correlation on two categorical variables
I am using the fourth-corner method in one of my papers (for those who need the name). The method was developed to test associations between variables in two datasets. In my case, the datasets ...
- 11
2
votes
0
answers
25
views
Which method to use to remove correlation between independent variables comprising of both categorical and numerical variables? [duplicate]
The independent variables in the dataset contains categorical variables such as
Gender ( 2 levels)
Mode of Shipment ( 3 levels)
Product Importance ( 4 levels)
and Numerical Variables such as
...
1
vote
1
answer
3k
views
Calculating correlation for categorical variables
I am struggling to find out a suitable way to calculate correlation coefficient for categorical variables. Pearson's coefficient is not supported for categorical ...
- 189
0
votes
2
answers
468
views
How many features do I select when doing feature selection for regression algorithms? Is R2 and RMSE good measures of success for overfitting?
Context: I'm currently crafting and comparing machine learning models to predict housing data. I have around 32000 data points, 42 features, and I'm predicting housing price. I'm comparing Random ...
- 31
3
votes
1
answer
446
views
Pearson correlation with data sets that have values on different scales [closed]
I have two datasets with which I want to do a Pearson correlation analysis. I have carried out the analysis which makes sense, however I want to be sure it is valid given that both datasets have ...
- 39
2
votes
1
answer
111
views
How to interpret Low Pearson correlation coefficient between stable signals and high Pearson correlation coefficient between unstable signals?
I calculated the Pearson correlation coefficient between two signals, that described the state of the unit. During normal operation of the unit, both signals were fairly stable and fluctuated very ...
- 87
0
votes
2
answers
194
views
What statistical method should i use to find Correlation between number of days and AmountEarned
I am new to Data Science and I have a python data frame with Number of days, CountofJobs, and AmountEarned what statistical method should I use to find a correlation between Days and AmountEarned.
<...
0
votes
1
answer
40
views
Determine relationship between users and age?
I would like to understand how to find an association between users, spam and email's age.
My dataset looks like as follows:
...
- 151
4
votes
2
answers
328
views
what is the difference in terms namely Correlation, correlated and collinearity?
A website says Correlation refers to an increase/decrease in a dependent variable with an increase/decrease in an independent variable. Collinearity refers to two or more independent variables acting ...
- 605
9
votes
2
answers
4k
views
What is the meaning of a quadratic relation when r = 0?
A website (on page 4) says:
The correlation coefficient is a measure of linear relationship and thus a value
of r = 0 does not imply there is no relationship between the variables. For
...
- 605
1
vote
3
answers
295
views
Does Karl Pearson correlation indicate linear relationship between two variables?
Wikipedia and literature do not seem to convey correct interpretation of Karl Pearson correlation. Also, some of the authors interpret it as a linear correlation or association. To me it simply tells ...
- 605
1
vote
0
answers
42
views
Professionals appear to interpret sample correlation (e.g. Karl Pearson) as if it represents linear correlation. Is it the correct interpretation? [closed]
I am stressed following the wrong interpretation. What is the correct way of understanding a correlation coefficient.
- 605
0
votes
1
answer
145
views
Correlation coefficient and non linear association [closed]
Correlation and Linear Regression
Author:
Lisa Sullivan, PhD
Professor of Biostatistics
Boston University School of Public Health
Says there may be non-linear association which the correlation ...
- 605
2
votes
2
answers
270
views
Transforming negative correlated non linear variable to linear positive correlated variable
At my office, I am stuck in a weird situation. I am asked to perform a regression algorithm on the data, in which the target variable is continuous having values range between 0.6 to 0.9 with 8 digits ...
- 178
0
votes
1
answer
660
views
Calibrating Correlation
I am facing a weird problem in my on going project and thought if someone here could help me out with this. Actually I have large data set. I have to perform a regression task on top of that. While ...
- 178
40
votes
1
answer
45k
views
Pearson vs Spearman vs Kendall
What are the characteristics of the three correlation coefficients and what are the comparisons of each of them/assumptions?
Can somebody kindly take me through the concepts?
user86099
6
votes
3
answers
8k
views
Should features be correlated or uncorrelated for features-selection with the help of multiple regression analysis?
I have seen researchers using Pearson correlation coefficient to find out the relevant features - to keep the features that have a high correlation value with the target. The implication is that the ...
- 469
1
vote
1
answer
384
views
Association between Categorical Variables and regression
We perform data analysis and build models. Say, for example, I built a regression model that has more than one predictor (multiple regression). We then check many things: normality, multicollinearity, ...
