Questions tagged [correlation]
A measure of the degree of linear association among a pair of variables.
171
questions
0
votes
1answer
14 views
Similarity Measure of Simulated Time Series vs Observed time Series
In my work I have an observed Time Series and Simulated ones. I want to compare the Light Curves and check for similarityto find out which simulated curve fits best respectivley which parameters ...
1
vote
1answer
16 views
Causation vs correlation problem in case of fixing other parameters and changing only one
I have a game that players play. D7 retention is defined as percentage of players who installed my game and after exactly 7 days later they played my game.
Now I am doing analytics and it shows that ...
0
votes
1answer
22 views
What should be the criteria to select features using correlation factors between features? [duplicate]
For the titanic dataset, I have done some feature engineering (one-hot encoded the features) and now I have developed a heatmap to view the correlation between different features.
I'm not able to ...
1
vote
1answer
17 views
SelectKBest and Correlation returns me excatly same feature selection. How?
Im working on selecting most effective features from a dataset with over that 2000 features. Im using different algorithms for that (selectKBest with chi-square, Extra Trees, Correlation etc.) But ...
1
vote
0answers
32 views
Similarity Measure Time Series
In my work I have an observed Time Series and Simulated ones. I want to compare the Light Curves and check for similarityto find out which simulated curve fits best respectivley which parameters ...
0
votes
1answer
7 views
Changes in the standard Heatmap plot - symmetric bar colors, show only diagonal values, and column names at x,y axis ticks
I have a heatmap image (correlation between all matrix columns) and I'm straggling to preform all the changes below within the same image:
bar colors should be symmetric around zero (e.g., ...
1
vote
2answers
50 views
Correlation vs Multicollinearity
I have been taught to check correlation matrix before going for any algorithm.
I have a few questions around the same:
Pearson Correlation is for numerical variables only.
What if we have to check ...
0
votes
0answers
18 views
Interrater-reliability for several items at interval or ordinal scale with 2-3 different raters
I would like to compute the interrater-reliability in R for a sample (n=150) in which each individual was rated by 2 to 3 raters (in ca. 40% of the sample 2 raters, 60% 3 raters). Raters were ...
0
votes
0answers
25 views
Correlation between imagery and target
I'm using a cnn model to predict target values which were received by special measurements, not from images. That is why, there's some doubt about whether input imagery data is sufficient for the ...
0
votes
0answers
22 views
How to compare 2 elements of a column with each other?
I am using R. I have a column with 4 elements {car, bus, walk, train}. I would like to compare the car and bus with each other with the help of the 3 other columns. ...
0
votes
0answers
17 views
Dendrogram/cluster analysis of correlation matrix
I want to take a universe of potential, trade-able instruments and allocate them to portfolio managers. Traditionally, this is done using a sector classification, ...
0
votes
0answers
23 views
records with perfect correlation to the answer. Drop or Keep?
I have about 1000 records (5 numeric, 5 categorical vars) and about 25 of them have something in 5-level categorical variable that just gives away answer. It's just too obvious and I'm not worried ...
2
votes
1answer
40 views
Incorrect correlation results
I am working on a predictive system of a finite outcome with output -1 or 1, depending on the values detected by 10 sensors.
...
1
vote
1answer
26 views
Feature selection method explanation
In the context of feature relevance, I am trying to understand the meaning of the correlation method for feature selection. Can somebody please explain if the following results of the correlation ...
0
votes
0answers
20 views
Determining users that have been sent too many emails
The experiment is a set of users in a marketing program. We are sending campaigns to the user and want to figure out whether the user is being sent too many emails or not.
Suppose you have a dataset ...
3
votes
1answer
61 views
How to handle associated features in machine learning
I am working on a classification project in which some features are linked and I'm not sure how to handle them.
I will simplify my project like that :
There are different jobs, and multiple ...
0
votes
1answer
26 views
How to measure the correlation between categorical variables and a continuous variable
I have the following list of the names of the categorical variables in my dataset:
...
2
votes
2answers
57 views
How can I find correlation between features?
The problem
I want to figure out how routers correlate between each other. Like, if a specific error occurred in router A, and almost at the same time the error occurs in router B, they probably have ...
0
votes
0answers
6 views
Is it worth graphing a correlation plot between your features and your target?
Is it worth graphing the correlation between the most important features and the target variable after you have done either PCA or L1 Regularization to identify the most important features? I guess ...
1
vote
1answer
27 views
Are there any rules regarding correlation between the variables used in a CNN?
I was wondering if there are any rules regarding correlation between the predictor variables or between the predictor and outcomes variables of a CNN. What should be the value of correlation? Does it ...
0
votes
0answers
15 views
Do I discard all my dependent variables as proved by chi-squared test of independence?
I have 134 categorical columns in my data. 7 of which are categorical variables [ one variable is highly unbalanced and has 34 classes while all other variables just has 3-5 classes in each variable ...
0
votes
2answers
52 views
Correlation feature selection followed by regression
I have quarterly results data for a company with around 100 variables. Total 60 quarters results are available (total records 60).
sample data: (only few columns & 10 rows)
I would like know ...
1
vote
1answer
23 views
Group-by correlation vs All observations correlation
Let's suppose that I have a dataset with datapoints about footballers.
The data are about footballers' performance and information (e.g goals, assists, injuries, age, weight etc) on a monthly basis ...
0
votes
1answer
49 views
Categorical Variable and Target Variable
Though a similar question is answered here , but I wanted to take a different approach. Assuming that I have a binary target variable 1/0 and a categorical variable Gender M/F. From this, I can have a ...
2
votes
2answers
108 views
Can I add features that are parts of another feature?
I am building a model (implementing both logistic regression and Xgboost) to understand the importance/significance of each feature in whether a customer is going to repurchase to understand what ...
1
vote
1answer
50 views
Automatically Remove highly correlated features using Variance Inflation Factor?
I want to be able to automatically remove highly correlated features. I am performing a classification problem using a set of 20-30 features and some may be correlated.
Multiple features can be ...
0
votes
0answers
15 views
Best technique for testing if an event causes a trend?
Example - A major holiday is coming up and the frequency of a product being purchased starts increasing significantly as it gets closer to the day of the holiday. Then after the holiday, the frequency ...
0
votes
1answer
28 views
how to deal with two high correlations feature which both has a low correlation with target
I am doing a prediction of house trade money.
Here is the correlation matrix of features whose correlations are larger than 0.3 as follows:
...
2
votes
1answer
386 views
What measures can I use to find correlation between categorical features and binary label?
For analyzing numerical features, we have correlation. What measures do we have to analyse the relevance of a categorical feature to the target value? If there isn't a direct measure, how can we ...
0
votes
4answers
199 views
How to find the relationship between two categorical variables?
I'm building a model and trying to get the relationship between two multi-level categorical variables.
For example, I want to know the relationship between race and likelihood of graduate from ...
0
votes
1answer
26 views
Using a feature in prediction even if it gets zero as p-value?
I created two binary classification based logistic regression models and I got these results:
Model 1:
Accuracy: 67.51%
AUC: 65.21%
Model 2:
Accuracy: 67,99%
AUC: 65,70%
The second ...
0
votes
1answer
23 views
What does coefficient mean for a binary independent variable in multiple linear regression?
Let's say I have a multiple linear regression model where my dependent variable, Y, is an integer. And, one of my independent variables --x1-- is binary --let's say either 0 or 1.
We know that sign ...
0
votes
0answers
10 views
create joint prob distribution or empirical relation for two variables
There are two variables, X1 and X2. The experimental study shows that they are highly correlated. Are there any reliable ways to create an empirical mapping(or equation) between X1 and X2. Assuming ...
7
votes
1answer
154 views
Generate predictions that are orthogonal (uncorrelated) to a given variable
I have an X matrix, a y variable, and another variable ORTHO_VAR. I need to predict the <...
1
vote
0answers
11 views
What kind of test should I use to determine correlation between the value of a commodity and consumer sentiment?
I’m currently studying data science and am trying to apply my skills to a small project.
Basically, I’ve collected data about a commodity’s value over two time periods (30 data points each in 4 week ...
1
vote
2answers
51 views
Establish relationship between two sets of data
I have two data sets - Product to Features and Products to Parts
...
1
vote
0answers
127 views
Pearson's Coefficient for classification problem
Can we use Pearson's Correlation Coefficient to calculate the correlation between features and target in the case of a classification problem?
0
votes
1answer
44 views
Correlation between Time Series Indicators ( Stock Prices )
I am new to time series analysis and I am currently tackling a stock market prediction problem.
I have a set of market indicators (such as Bollinger Bands, ADX etc) which are derived from the time ...
3
votes
2answers
101 views
Manual feature engineering based on the output
So, I'm working on a ML model that would have as potential predictors : age , a code for his city , his social status ( married / single and so on ) , number of his children and the output signed ...
3
votes
0answers
43 views
Fitting model to differenced time series
I have a time series on daily stock price of company(2013 data points).I took a first order difference and the following acf and pacf plots of the differenced series were obtained. However, I am ...
0
votes
1answer
82 views
Clustering by Distributions of Groups of observations
I have data as shown below. Like the groups shown A,B,C.. there are roughly 5400 groups of data and 15 observations for every group (I sampled 6-7 for each group and pasted them here).
...
0
votes
2answers
23 views
Causality analysis
I'm trying to find causes for my lack of sleep through data. I'm hoping to be able to input information throughout the day like "drank coffee", "Exercised", "meditation","what I ate for dinner"... and ...
1
vote
1answer
100 views
Dealing with correlated features when calculating permutation importance
I have implemented the permutation importance calculation as found here in my attempt to identify features that contribute little to my model's (Gradient Boosted Tree model) predictive power.
The ...
0
votes
1answer
190 views
How to measure correlation between several categorical features and a numerical label in Python?
I have for a few weeks measured the time it takes for a product to be released through a automated release pipeline. I have several different categorical features such as "Product Category", "Product ...
1
vote
1answer
46 views
How to find correlation between time-series of different units?
I have 3 time-series data.
NDVI(normalized difference vegetation index) mean
Precipitation
Temperature
All of these have their own unit.
Now I want to find similarity/correlation between NDVI and ...
0
votes
1answer
171 views
How to select variables based on the mean correlation in a correlation matrix?
I have a set of independent variables and I am calculating the correlation matrix between them using the Pearson Correlation Coefficient in Python. A part of the matrix looks like this:
From this ...
0
votes
0answers
7 views
Identifying causation between content <> ad kpis
I have a bunch of content tagged with various types of content (e.g. political news, sports, opinion etc.) and a bunch of kpis related to ads placed next to this content (e.g click-throughs, purchases ...
1
vote
1answer
66 views
Machine learning algorithm that uses the Pearson or Spearman correlation?
I've come across linear and multiple regression, SVM, random forests. Does any know of a machine learning algorithm that uses the Pearson correlation or Spearman correlation?
Best,
Dave
0
votes
2answers
74 views
Trying to find the correlation between inputs and output
I have tried the pandas code for trying to find out the correlation between the output and the inputs I am feeding. Here is the code:
...
0
votes
2answers
1k views
Correlation between nominal categorical variables
I have two arrays, whose values are nominal categorical variables. Each element represents a zone of a city: in the first vector we have the class each zone belongs to (so these might also be seen as ...
