Questions tagged [data-analysis]
Data analysis is a process of inspecting, cleansing, transforming, and modeling data with the goal of discovering useful information, informing conclusions, and supporting decision-making.
281 questions
4
votes
1
answer
63
views
Spatially constrained geospatial similarity
What's the current methodology for clustering geospatial data by features?
Example: I have some demographic dataset. Let's say this contains average home price and population density.
So, an example ...
CommunityBot
- 1
0
votes
2
answers
59
views
Data Analysis Process
How do you feed data from a data warehouse to Python for ad-hoc analysis?
My day-to-day work is to answer ad-hoc questions, and 95% of the data I need is in our data warehouse. I often query data from ...
CommunityBot
- 1
0
votes
1
answer
39
views
How to measure Covid impact by analysing credit card transaction of customer
I Want to know how can I identify that is the customer is in financial distress due to the COVID situation using its credit card transactions.
I have a daily transaction of customers till current ...
CommunityBot
- 1
3
votes
1
answer
266
views
Pandas: Calculating the best seller to incorporate number of houses sold
I have been analysing seller data and trying to get insights. I have written a groupby statement to get the average price of selling for every seller
...
CommunityBot
- 1
0
votes
1
answer
47
views
Best way to analyse change in amplitude of cyclic data?
I have some cyclic data that I'm trying to analyse which I've posted below:
I would like to analyse some overall trends of the data. The peak-to-peak amplitude appears to be decreasing, however, I'm ...
CommunityBot
- 1
1
vote
2
answers
276
views
Predicting Customer Activity Absence
Could you please assist me with the following question?
I have a customer activity data frame that looks like this:
It contains at least 500.000 customers and a "time series" of 42 months. ...
CommunityBot
- 1
0
votes
0
answers
6
views
Intelligently Calculating Return on Ad Spend
Am looking for a way to project return on advertising spend (ROAS). Does not need to be a complex model (in fact would much prefer it if it isn't!).
We have all the data coming in through our SQL ...
3
votes
1
answer
1k
views
Comparing rows in Data Studio
I am using Data Studio for a project and I am connecting to my BigQuery table.
My table contains following columns:
Date
Store_name
Footfall
I'd love to compare the footfall of two stores using Data ...
CommunityBot
- 1
1
vote
2
answers
74
views
Looking for a couple of ideas please
I’ve got some data by postal zone that includes:
Postal zone code
Average rent value per square foot
Brand affinity 1
Brand affinity 2
Brand affinity 3
Brand affinity 4
…and so on
The brand affinity ...
CommunityBot
- 1
0
votes
2
answers
383
views
How to get periodicity from timeseries data?
I would like to create a recommendation system for a smart home application. I gather the data in a time-series database. The app monitors the on/off state of a smart lamp and can create daily ...
CommunityBot
- 1
0
votes
1
answer
40
views
Advice on how to approach a recommendation model for marketing
I am working on a project to provide recommendations to the marketing team to launch effective campaigns. The dataset I have has data on existing customers, their demographic and billing details as ...
CommunityBot
- 1
0
votes
1
answer
58
views
I am doing a forecasting for corona virus cases in NY, I have two models and not sure which one i should choose
I am using exponential smoothing and using tableau for forecasting. The first model I included trend and removed seasonality and it predicted the number of cases going up but the quality I got ...
CommunityBot
- 1
0
votes
1
answer
156
views
Is there a standardized way to do data analysis?
Is there a standard way to do data analysis?
So for example, something like this:
1. Data mining
2. Data cleaning
3. xx
4. Data and result interpretations
I ...
CommunityBot
- 1
0
votes
1
answer
76
views
comparing data from first quarter in two years
How can I compare data from two quarters? What software can I use to do it parallelly side by side so I can compare them. What kind of factors can I take into consideration, I'm posting a sample data ...
CommunityBot
- 1
0
votes
4
answers
74
views
Matching new set of data with pre-defined sets
I have sets of data describing sets of levels of requirements needed for certain sets of tasks. The following is a tabulated example:
Note that the data values are on a scale from 0 to 10.
My ...
CommunityBot
- 1
2
votes
1
answer
68
views
Predicting if a search keyword is lower volume based only on high volume keywords
My friend was asked this question in an interview for analytics and I cannot figure out the answer so I would like to see how could this data science problem be solved. Here's the problem:
Let's say ...
CommunityBot
- 1
1
vote
1
answer
58
views
How to test likelihood hypothesis on dataset?
How to test the following hypothesis?
The larger the fare the more likely the customer is to be travailing alone.
Using the data below, how would one be able to test the hypothesis?
...
CommunityBot
- 1
0
votes
1
answer
769
views
What's the best approach to dealing with missing data in a dataset?
I have a dataset that contains missing values in some columns. I would like to know what is the best approach to deal with this missing data. Should I remove rows with missing data or fill in missing ...
CommunityBot
- 1
0
votes
1
answer
43
views
Trying to write a code that detects a maximum value before a drop off in slope
I'm trying to write code for a force test that will output the maximum force before structural failure occurs. I'm a bit of a novice to python, so the issue here might be something simple that I'm ...
CommunityBot
- 1
0
votes
1
answer
599
views
Online Generating Dendrograms with imported CSV file
Is there any website out there that performs hierarchical clustering on an imported file like CSV to generate dendrograms or similarity matrix? Free or paid does not matter.
So far I found heatmapper....
CommunityBot
- 1
0
votes
1
answer
138
views
Marketing Spend Optimization Techniques
I need some help with market spend optimization. I’m working with a client who’s running an offline operation that’s primarily driven by online marketing (fb, google, twitter etc). They had asked me ...
CommunityBot
- 1
0
votes
0
answers
16
views
getting ideas to start my learning
i have plan to start my career on data analytics and i need a guildline how to start and where to start ,if you are ready to give some hints through that I'll get some clarity and i'll start my ...
0
votes
1
answer
780
views
ImportError: cannot import name 'Settings' from 'pandas_profiling.config' (/usr/local/lib/python3.7/dist-packages/pandas_profiling/config.py)
I'm trying to import pandas profiling on goggle colab.
!pip install https://github.com/ydataai/pandas-profiling/archive/master.zip
after succesfully installing ...
CommunityBot
- 1
0
votes
1
answer
51
views
Feature engineering for interest-based age classification
I have a dataset which has users (rows) with the list of their interests (IABs), which looks like this
...
CommunityBot
- 1
5
votes
3
answers
150
views
How can data science teams inside businesses measure costs and efficiency of their technical work?
How can data science teams measure and improve costs of their technical work, when they often don't know the monetary value of the datasets and insights they are producing? Are they using industry ...
CommunityBot
- 1
0
votes
3
answers
330
views
Difference between Data analytics and Data Analysis
What we are referring to when we say Data analytics and Data analysis?
CommunityBot
- 1
0
votes
1
answer
30
views
How to Classify Blueberries as "Crunchy", "Juicy" or "Soft" using Acoustic Signal Processing and Machine Learning?
I'm working on a project to classify blueberries based on their texture—specifically, whether they are soft, juicy, or crunchy—using the sounds they produce when crushed. I have about 1100 audio ...
- 173
0
votes
2
answers
3k
views
Data Conversion to Time Series in R
I have sales data for 2018 and 2019. I would like to convert it to time series. The data does not have daily sales:
...
CommunityBot
- 1
0
votes
1
answer
127
views
Analysis for basic weight training analysis?
TL;DR: I'm doing a fairly basic project which involves exercise. It seems that descriptive statistics and basic data vis (ex: line graph) would be most appropriate for this project, but I wonder if ...
CommunityBot
- 1
0
votes
1
answer
113
views
Insights betwwen two columns/variables in Dataframe
I have data in two columns one is range of old credit score (Input score range) and new credit score (cvsc100). How do i find insights from both of them ? where the old is range of values and other ...
CommunityBot
- 1
0
votes
1
answer
69
views
Understanding Box plots
I am new to box plots and I am trying to understand it. Consider the following figures :
According to my understanding, outliers will exist after the max. line and before the min. line. So according ...
CommunityBot
- 1
-1
votes
1
answer
27
views
machine learning - data science - data analysis
I have a research project in the machine learning area. In this study, dataset contains more than 4000 numbers categorized in four columns. I am going to find or predict a possible relation between ...
- 68
0
votes
3
answers
10k
views
Discarding non-english words in column
I have some non-english words/sentences in my data.
I tokenized my text and tried using nltk.corpus.words.words() but its not really helpful as it also removes the ...
- 1
0
votes
1
answer
174
views
Scaling and handling highly correlated features in tabular data for regression
I am working on a regression problem trying to predict a target variable with seven predictor variables. I have a tabular dataset of 1400 rows. Before delving into the machine learning to build a ...
- 6,543
0
votes
0
answers
7
views
Within-subject analysis of the impact of energetic music on video game performance: Wilcoxon or Friedman?
I am conducting an amateur study of the impact of music on video game performance across multiple rounds of gameplay.
The gameplay loop is simple: Arrows are falling down, and you have to hit them at ...
- 1
0
votes
1
answer
347
views
From what percentage should I remove a column that has empty values?
I have a problem. I am currently cleaning and preprocessing my data.
Suppose I have the following data set
...
CommunityBot
- 1
0
votes
0
answers
29
views
Is it okay to ignore missing data despite a very small percentage?
My dataset has seven cols and 63 mil rows. There are two columns with missing data. The first one [Terminal Number] has 400k rows of missing data but thankfully, I can extract values from another ...
0
votes
0
answers
18
views
Angular Distribution Function for Clustered Data Points
I was tasked recently with analyzing a cluster of individual points (each point representing a particulate), and I was advised that it would be good to apply an angular distribution function (ADF) on ...
5
votes
3
answers
411
views
ETL and Data Engineering - is it purely the knowledge of tools or is there theory behind it?
I would like to better understand what a good Data Englineer must know or what he does. Job descriptions mostly list tools that are required, such as Python.
If it is possible to separate Data ...
- 335
0
votes
0
answers
13
views
Which parameters of the Sholl analysis summary do I need to compare to get descriptive graphs about the complexity of single neurons?
For my master thesis I am working with Purkinje cells from Zebrafish with a condition that makes these neurons less complex. I am treating them with a construct that is hypothesized to improve this ...
- 1
1
vote
1
answer
160
views
0
votes
0
answers
16
views
What to include in fact and dimension table from election database
I am working with election dataset of India of year 2014
and data for 2019
I also have table for party names and descriptions
and finally state name with code.
I am not getting how do i create a ...
1
vote
1
answer
280
views
Quantitatively evaluate similarity between two corpus of texts
I want to assess how similar, or different two corpora are, and if the similarity is statistically significant. Something close to a Kolmogorov–Smirnov test in statistics, but for text data.
For ...
CommunityBot
- 1
0
votes
0
answers
12
views
Plotting related multiple plots sharing an axis but each disjoint from each other
I want to plot related multiple plots sharing an axis but each disjoint from each other, i.e., same x-axis feature and scale but each of the other features has their own separate y-axis region with it'...
- 1
0
votes
0
answers
7
views
How to Identify Equipment Churn from Laboratory Service Records Without Direct Churn Labels?
I'm analyzing a dataset encompassing 20 years of laboratory equipment service records, which includes the equipment ID, service dates, types of equipment (HOOD_TYPE), and descriptions of performed ...
- 1
0
votes
0
answers
9
views
The best order for analysis steps in building econometric model with time series linear regression
I am working on a project whose goal is to build a linear regression model for a time series dataset. I was provided with a blueprint of all required analysis steps. This led me to wonder what is the ...
- 103
1
vote
2
answers
78
views
Affine 2D mapping in python
I have two sets of 2D data $A$ and $B$ (representing 2D positions on a 2D plane $x,y$) which are related (the first pair of $x,y$ of $A$ is related to the first pair of $x,y$ of $B$ for instance). I ...
- 1
0
votes
1
answer
23
views
Using boxplots of previous day’s percentage change in S&P index to predict today's return based on yesterday's movement
In the book "An Introduction to Statistical Learning with Applications in Python, Trevor Hastie et al., Springer", there's the following paragraph:
The left-hand panel of Figure 1.2 ...
- 8,197
0
votes
0
answers
74
views
Interpreting Perplexity, U_mass coherence and Cv score trends for a Latent Dirichlet Allocation Model
I'm running an LDA model through gensim. To my understanding, closer the u_mass coherence score is to zero, higher is the interpretability of the topics that come up. I'm getting the u_mass coherence ...
- 1
-2
votes
1
answer
63
views
How to decide your first self-project for Data Science/Analysis [closed]
Suppose a complete beginner(in the field of data analysis/data science), which is my case indeed, decided to make a project for his/her resume.
These are the following skills he have learnt:
Python(...
- 24