Questions tagged [pearsons-correlation-coefficient]

Filter by
Sorted by
Tagged with
0 votes
0 answers
9 views

Metric for correlation of two paired lists of numbers

I have a program which produces an image, and I use a metric to understand how accurate that image is. I choose five cases (A, B, C, D, E), and make a list of the accuracy metric for each case: ...
user avatar
0 votes
0 answers
13 views

Scaling and handling highly correlated features in tabular data for regression

I am working on a regression problem trying to predict a target variable with seven predictor variables. I have a tabular dataset of 1400 rows. Before delving into the machine learning to build a ...
user avatar
  • 101
0 votes
1 answer
173 views

Python: calculate the weighted average correlation coefficient

I am calculating the volatility (standard deviation) of returns of a portfolio of assets using the variance-covariance approach. Correlation coefficients and asset volatilities have been estimated ...
user avatar
  • 101
0 votes
0 answers
18 views

Correlation analysis yields conflicting results. Positive Pearson and Negative Spearman

I have four features x1,x2,x3,x4 all of their correlation with y are similar in Pearson and in Spearman separately. However, all these are +0.15 in Pearson and -0.6 in Spearman, more or less. Does ...
user avatar
0 votes
0 answers
77 views

Is pearson correlation matrix a good indicator for label encoded categorical and numeric independent data?

I have a dataset having 22 independent variables out of which 15 are categorical data that has already been label encoded i.e the dtype is int64 and the contents are in a range of 0 to n (n is the ...
user avatar
  • 103
0 votes
1 answer
35 views

Correlation with target variable for regression problem

Given the following dataframe age job salary 0 1 Doctor 100 1 2 Engineer 200 2 3 Lawyer 300 ... with ...
user avatar
1 vote
0 answers
48 views

Optimizing rolling Pearson correlation

I have Pandas DataFrame with multiple columns (3000 or more) with timeseries in them (I have dates as indecies). ...
user avatar
0 votes
0 answers
9 views

How to calculate relation function of State and Action in delayed action effect Environment?

I'm trying to calculate corr-coef(or other good relation function) of State and Action in 'delayed action effect' Envrironment. In this environment, agent observes states, then it returns action and ...
user avatar
  • 1
0 votes
0 answers
13 views

Analysing process data with sub groupings and checking for correlation

I have a dataset of process data for different equipment with many sensors. I would like to check the correlation of the different sensors to see if there is any strong correlation between some ...
user avatar
  • 378
0 votes
0 answers
71 views

Agglomerative Clustering (average linkage) and Pearson Correlation

Does having a positive or negative correlation between features being clustered affect the agglomerative clustering result? I have three columns in my dataset, and I'm trying to figure out if I should ...
user avatar
  • 101
1 vote
0 answers
301 views

Pearson correlation on two categorical variables

I am using the fourth-corner method in one of my papers (for those who need the name). The method was developed to test associations between variables in two datasets. In my case, the datasets ...
user avatar
2 votes
0 answers
22 views

Which method to use to remove correlation between independent variables comprising of both categorical and numerical variables? [duplicate]

The independent variables in the dataset contains categorical variables such as Gender ( 2 levels) Mode of Shipment ( 3 levels) Product Importance ( 4 levels) and Numerical Variables such as ...
user avatar
1 vote
1 answer
1k views

Calculating correlation for categorical variables

I am struggling to find out a suitable way to calculate correlation coefficient for categorical variables. Pearson's coefficient is not supported for categorical ...
user avatar
  • 189
0 votes
1 answer
317 views

How many features do I select when doing feature selection for regression algorithms? Is R2 and RMSE good measures of success for overfitting?

Context: I'm currently crafting and comparing machine learning models to predict housing data. I have around 32000 data points, 42 features, and I'm predicting housing price. I'm comparing Random ...
user avatar
3 votes
1 answer
208 views

Pearson correlation with data sets that have values on different scales [closed]

I have two datasets with which I want to do a Pearson correlation analysis. I have carried out the analysis which makes sense, however I want to be sure it is valid given that both datasets have ...
user avatar
2 votes
1 answer
61 views

How to interpret Low Pearson correlation coefficient between stable signals and high Pearson correlation coefficient between unstable signals?

I calculated the Pearson correlation coefficient between two signals, that described the state of the unit. During normal operation of the unit, both signals were fairly stable and fluctuated very ...
user avatar
  • 87
0 votes
1 answer
76 views

What statistical method should i use to find Correlation between number of days and AmountEarned

I am new to Data Science and I have a python data frame with Number of days, CountofJobs, and AmountEarned what statistical method should I use to find a correlation between Days and AmountEarned. <...
user avatar
0 votes
1 answer
34 views

Determine relationship between users and age?

I would like to understand how to find an association between users, spam and email's age. My dataset looks like as follows: ...
user avatar
  • 151
4 votes
2 answers
145 views

what is the difference in terms namely Correlation, correlated and collinearity?

A website says Correlation refers to an increase/decrease in a dependent variable with an increase/decrease in an independent variable. Collinearity refers to two or more independent variables acting ...
user avatar
9 votes
2 answers
2k views

What is the meaning of a quadratic relation when r = 0?

A website (on page 4) says: The correlation coefficient is a measure of linear relationship and thus a value of r = 0 does not imply there is no relationship between the variables. For ...
user avatar
0 votes
1 answer
134 views

Does Karl Pearson correlation indicate linear relationship between two variables ? Or it indicates nonlinear relationship or simply the correlation?

Wikipedia and literature do not seem to convey correct interpretation of Karl Pearson correlation. A few authors interpret it as a linear correlation or association. To me it simply tells direction ...
user avatar
1 vote
0 answers
39 views

Professionals appear to interpret sample correlation (e.g. Karl Pearson) as if it represents linear correlation. Is it the correct interpretation? [closed]

I am stressed following the wrong interpretation. What is the correct way of understanding a correlation coefficient.
user avatar
0 votes
1 answer
106 views

Correlation coefficient and non linear association [closed]

Correlation and Linear Regression Author: Lisa Sullivan, PhD Professor of Biostatistics Boston University School of Public Health Says there may be non-linear association which the correlation ...
user avatar
2 votes
2 answers
154 views

Transforming negative correlated non linear variable to linear positive correlated variable

At my office, I am stuck in a weird situation. I am asked to perform a regression algorithm on the data, in which the target variable is continuous having values range between 0.6 to 0.9 with 8 digits ...
user avatar
0 votes
1 answer
545 views

Calibrating Correlation

I am facing a weird problem in my on going project and thought if someone here could help me out with this. Actually I have large data set. I have to perform a regression task on top of that. While ...
user avatar
33 votes
1 answer
32k views

Pearson vs Spearman vs Kendall

What are the characteristics of the three correlation coefficients and what are the comparisons of each of them/assumptions? Can somebody kindly take me through the concepts?
user avatar
1 vote
1 answer
231 views

Association between Categorical Variables and regression

We perform data analysis and build models. Say, for example, I built a regression model that has more than one predictor (multiple regression). We then check many things: normality, multicollinearity, ...
user avatar