Questions tagged [correlation]
A measure of the degree of linear association among a pair of variables.
207
questions
2
votes
0answers
18 views
Is there any solution for modified t-test in python for spatial data?
I am searching for a solution to conduct a modified t-test or any other possibility in order to compute significant or non-significant relations in a spatial dataset (which can be a stack of rasters ...
0
votes
0answers
15 views
Correlation between numeric variables of different meaning
I have a data set containing information about buildings, such as the area code, the neighborhood code, the number of floors, the year in which the house was built and the year in which restorations ...
2
votes
0answers
9 views
Correlation matrix on multi-time scales
I am trying to see how diversified a portfolio is, using the portfolio holdings weights and their return streams.
For example, let's say that the portfolio has 3 stocks, so I will have the 3x1 W (...
0
votes
0answers
7 views
Dealing with built hierchical linear dependencies in my features
In a supervised learning problem, I am working on a dataset with a lot of hierachical linear dependencies between features. Let me be more clear about what I mean : In my dataset, there is a lot of ...
1
vote
0answers
35 views
3
votes
2answers
39 views
Chose among highly correlated variables
I am working on a Kaggle dataset and I am trying to build a predictive model for the "Chance of Admit" (dependent variable) of students to the university of their interest.
Below you can find the ...
2
votes
1answer
16 views
Clustering Weekday Weekend Data and Multicollinearity
Hi I have data of weekday and weekend step counts in which I extracted metrics from them such as the wd steps, we steps, standard deviation of wd steps, standard deviation of we steps and so on...
<...
1
vote
2answers
35 views
0
votes
0answers
4 views
What is the best approach (and why) to identify a conic section given the the points along its cross-section and its vector magnitudes over time?
I have an N-body simulation that calculates the position, velocity, and acceleration of each body at every frame over the course of some duration. As an example, I tested the algorithm using our solar ...
1
vote
1answer
12 views
Upper bound on 'relatedness'?
We have ~100 answers to a questionnaire with five questions (Q5). Independently from that, we have about 50, somewhat overlapping, features describing the people who answers the questions (F50). After ...
0
votes
1answer
110 views
Calibrating Correlation
I am facing a weird problem in my on going project and thought if someone here could help me out with this. Actually I have large data set. I have to perform a regression task on top of that. While ...
0
votes
0answers
15 views
How can I compare continuous and non-binary categorical data in Excel?
I'm trying to find the correlation between vehicle days on lot vs different categorical variables (color, make, model, etc). I know it's possible with binary data, but I haven't found a simple way to ...
3
votes
1answer
537 views
Pearson vs Spearman vs Kendall
What are the characteristics of three correlation coefficients and what are the comparisons of each correlation coefficients, Assumptions.
Can somebody kindly take me through the concepts
1
vote
1answer
21 views
Relationship between two sets of dates
My wife has seizures a few times a month and takes medication for the seizures once a month. So I have two sets of dates - the dates of the seizures and the dates she got her medication.
I'm trying ...
1
vote
3answers
63 views
Correlation between categorical variables based on the target distribution
Let $X$ be a category with very high cardinality and $Y$ be my target. when I look at $X$ distribution to $Y$ I see that some of the levels are very similar to each other . I would like to find a way ...
0
votes
1answer
29 views
Question mark on Correlation Matrix with RapidMiner
I'm using RapidMiner to evaluate correlation between attribute in my dataset. The problem is that a lot of values appear with '?'.
Someone can help me?
This is a sample of data
-1
votes
1answer
146 views
How to find and calculate correlation in a data set which has category and continuous variables? [duplicate]
I am working on an Insurance domain use case to predict if an existing customer will buy a second insurance policy or not. I have a few personal details saved under different categories like Marital ...
2
votes
1answer
109 views
Why is pandas corr() deleting columns?
I'm doing a basic correlation analysis but for some reason pandas corr() is deleting columns, not sure why.
...
2
votes
1answer
30 views
How do I assess whether two time series change together?
In this example, at timepoint 5, both signals move up together. I would like to quantify these similar movements, and ideally disregard the parts where the signals are almost constant.
What ...
2
votes
2answers
47 views
What expresses if/how two variables are dependent on each other?
The accepted answer to question Why is a correlation matrix symmetric? includes this:
The correlation matrix is a measure of linearity. It does not express how two variables are dependent on each ...
1
vote
1answer
31 views
List of Correlations to Correlation Matrix [closed]
I am using python to do some data analysis and I need to represent the following table as a correlation matrix. The correlation value is a value between -1 and 1.
...
0
votes
0answers
12 views
Generating correlated time series with different probability distributions
I have the following problem to solve: I need to generate two time series, each modeling a variable (lets call them VarA and VarB). For each of these variables, I have a PDF (PdfA, PdfB) which belongs ...
0
votes
0answers
28 views
How to analyzes seaborn's regplot of features from kaggle's houses prices and decide to remove a specific feature?
As a machine learning beginner, I am trying my best to, to solve kaggle's houses prices competition.
I am doing pretty well. Some of the fields are confusing.
The data set contains about 80 features,...
0
votes
2answers
31 views
Drawing validation set from test set
I am building a 3 neural network models on dataset that is already separated to train and test sets. From my analysis, I found that this dataset has values on test set which don't exist in the train ...
0
votes
1answer
27 views
Best way for correlation analysis
I am looking for an alternative way of analysing my data which will be used for ML. I am using Matlab to implement my code. Using my code, I obtain arrays which are used to describe signals. Each ...
0
votes
2answers
34 views
Monthly data versus yearly prediction
How would you see the relation of a monthly variable categorical with a yearly target value float for differents years in order to predict.
In the farming industry the relation is not so simple for ...
0
votes
1answer
33 views
correlation between categorical(ordinal) and discrete(continuous) value [duplicate]
I am doing my bi variate analysis but right now looking to see the correlation between my atributes
some are categorical 5 levels and others amount of money.
how can I see the correlation between ...
1
vote
1answer
31 views
Checking if ML model is possible
How can I check if a machine learning model is feasible on a given dataset? What techniques like EDA, correlation etc. can be used to judge if a model is possible i.e. data and predictor variables ...
0
votes
2answers
15 views
How can I do the correlation between two estimators?
I'm working with several estimators of all kind. Then, I want to stack these estimators, and the best is if they have low correlation between them.
I suppose that the correlation method depends on ...
1
vote
2answers
38 views
Finding correlation between MNIST digits
What way would be correct to calculate the correlation between say digit '1' and digit '7' images from MNIST? Taking average values of all digit '1' pixels and digit '7' pixels to compute correlation ...
0
votes
1answer
21 views
Pearson correlation make disapear the target column
I have a dataframe with some numerical and categorical values. I want to do some feature selection to visualise a low-dimensional split in the dataset when the target variable is the grade. Yet, when ...
0
votes
1answer
45 views
Correlation Matrix Issues: Can't Convert Variables to Numerics
QUESTIONS:
Can you tell me how I can convert int and factor variables to numeric variables in R?
Can you tell me why ...
8
votes
1answer
77 views
Features reduction for the not correlated data set
I am working with classification problem on a training data set, which have 100 features. All the features in pairs haven't visible correlation. One can see it in the example pair plot for the some of ...
5
votes
0answers
34 views
Influence of trend on (supposedly) correlated time series
TL;DR: What is the impact of a linear trend on the correlation between time series that are (most likely) not spuriously correlated?
I'm currently trying to reconstruct/cross-validate an analysis ...
2
votes
1answer
62 views
How to wisely choose machine algorithm for the given data set?
I have a data sample of RSSI value from three different devices and based on the RSSI sample, it should tell from which location the data sample is arrived. Below are the sample data set,
...
0
votes
1answer
20 views
Use correlation matrix scores as starting weights and bias inputs for neural network?
I have a neural net that's generating an average 15% error across the three outputs it gives. My problem is two of the three really have about a 2% error while the third has around 40%.
I was ...
1
vote
2answers
41 views
Similarity Measure of Simulated Time Series vs Observed time Series
In my work I have an observed Time Series and Simulated ones. I want to compare the Light Curves and check for similarityto find out which simulated curve fits best respectivley which parameters ...
1
vote
1answer
18 views
Causation vs correlation problem in case of fixing other parameters and changing only one
I have a game that players play. D7 retention is defined as percentage of players who installed my game and after exactly 7 days later they played my game.
Now I am doing analytics and it shows that ...
0
votes
1answer
27 views
What should be the criteria to select features using correlation factors between features? [duplicate]
For the titanic dataset, I have done some feature engineering (one-hot encoded the features) and now I have developed a heatmap to view the correlation between different features.
I'm not able to ...
2
votes
1answer
156 views
SelectKBest and Correlation returns me excatly same feature selection. How?
Im working on selecting most effective features from a dataset with over that 2000 features. Im using different algorithms for that (selectKBest with chi-square, Extra Trees, Correlation etc.) But ...
2
votes
0answers
77 views
Similarity Measure Time Series
In my work I have an observed Time Series and Simulated ones. I want to compare the Light Curves and check for similarityto find out which simulated curve fits best respectivley which parameters ...
0
votes
1answer
96 views
Changes in the standard Heatmap plot - symmetric bar colors, show only diagonal values, and column names at x,y axis ticks
I have a heatmap image (correlation between all matrix columns) and I'm straggling to preform all the changes below within the same image:
bar colors should be symmetric around zero (e.g., ...
3
votes
2answers
154 views
Correlation vs Multicollinearity
I have been taught to check correlation matrix before going for any algorithm.
I have a few questions around the same:
Pearson Correlation is for numerical variables only.
What if we have to check ...
0
votes
0answers
20 views
Interrater-reliability for several items at interval or ordinal scale with 2-3 different raters
I would like to compute the interrater-reliability in R for a sample (n=150) in which each individual was rated by 2 to 3 raters (in ca. 40% of the sample 2 raters, 60% 3 raters). Raters were ...
0
votes
0answers
28 views
Correlation between imagery and target
I'm using a cnn model to predict target values which were received by special measurements, not from images. That is why, there's some doubt about whether input imagery data is sufficient for the ...
0
votes
0answers
33 views
How to compare 2 elements of a column with each other?
I am using R. I have a column with 4 elements {car, bus, walk, train}. I would like to compare the car and bus with each other with the help of the 3 other columns. ...
0
votes
0answers
20 views
Dendrogram/cluster analysis of correlation matrix
I want to take a universe of potential, trade-able instruments and allocate them to portfolio managers. Traditionally, this is done using a sector classification, ...
0
votes
0answers
24 views
records with perfect correlation to the answer. Drop or Keep?
I have about 1000 records (5 numeric, 5 categorical vars) and about 25 of them have something in 5-level categorical variable that just gives away answer. It's just too obvious and I'm not worried ...
2
votes
1answer
54 views
Incorrect correlation results
I am working on a predictive system of a finite outcome with output -1 or 1, depending on the values detected by 10 sensors.
...
1
vote
1answer
35 views
Feature selection method explanation
In the context of feature relevance, I am trying to understand the meaning of the correlation method for feature selection. Can somebody please explain if the following results of the correlation ...
