Questions tagged [data-analysis]
Data analysis is a process of inspecting, cleansing, transforming, and modeling data with the goal of discovering useful information, informing conclusions, and supporting decision-making.
281 questions
0
votes
0
answers
6
views
Intelligently Calculating Return on Ad Spend
Am looking for a way to project return on advertising spend (ROAS). Does not need to be a complex model (in fact would much prefer it if it isn't!).
We have all the data coming in through our SQL ...
0
votes
0
answers
16
views
getting ideas to start my learning
i have plan to start my career on data analytics and i need a guildline how to start and where to start ,if you are ready to give some hints through that I'll get some clarity and i'll start my ...
0
votes
1
answer
30
views
How to Classify Blueberries as "Crunchy", "Juicy" or "Soft" using Acoustic Signal Processing and Machine Learning?
I'm working on a project to classify blueberries based on their texture—specifically, whether they are soft, juicy, or crunchy—using the sounds they produce when crushed. I have about 1100 audio ...
-1
votes
1
answer
27
views
machine learning - data science - data analysis
I have a research project in the machine learning area. In this study, dataset contains more than 4000 numbers categorized in four columns. I am going to find or predict a possible relation between ...
- 1
0
votes
0
answers
7
views
Within-subject analysis of the impact of energetic music on video game performance: Wilcoxon or Friedman?
I am conducting an amateur study of the impact of music on video game performance across multiple rounds of gameplay.
The gameplay loop is simple: Arrows are falling down, and you have to hit them at ...
- 1
0
votes
0
answers
29
views
Is it okay to ignore missing data despite a very small percentage?
My dataset has seven cols and 63 mil rows. There are two columns with missing data. The first one [Terminal Number] has 400k rows of missing data but thankfully, I can extract values from another ...
0
votes
0
answers
18
views
Angular Distribution Function for Clustered Data Points
I was tasked recently with analyzing a cluster of individual points (each point representing a particulate), and I was advised that it would be good to apply an angular distribution function (ADF) on ...
0
votes
0
answers
13
views
Which parameters of the Sholl analysis summary do I need to compare to get descriptive graphs about the complexity of single neurons?
For my master thesis I am working with Purkinje cells from Zebrafish with a condition that makes these neurons less complex. I am treating them with a construct that is hypothesized to improve this ...
- 1
0
votes
0
answers
16
views
What to include in fact and dimension table from election database
I am working with election dataset of India of year 2014
and data for 2019
I also have table for party names and descriptions
and finally state name with code.
I am not getting how do i create a ...
0
votes
0
answers
12
views
Plotting related multiple plots sharing an axis but each disjoint from each other
I want to plot related multiple plots sharing an axis but each disjoint from each other, i.e., same x-axis feature and scale but each of the other features has their own separate y-axis region with it'...
- 1
0
votes
0
answers
7
views
How to Identify Equipment Churn from Laboratory Service Records Without Direct Churn Labels?
I'm analyzing a dataset encompassing 20 years of laboratory equipment service records, which includes the equipment ID, service dates, types of equipment (HOOD_TYPE), and descriptions of performed ...
- 1
0
votes
0
answers
9
views
The best order for analysis steps in building econometric model with time series linear regression
I am working on a project whose goal is to build a linear regression model for a time series dataset. I was provided with a blueprint of all required analysis steps. This led me to wonder what is the ...
- 103
0
votes
1
answer
23
views
Using boxplots of previous day’s percentage change in S&P index to predict today's return based on yesterday's movement
In the book "An Introduction to Statistical Learning with Applications in Python, Trevor Hastie et al., Springer", there's the following paragraph:
The left-hand panel of Figure 1.2 ...
- 155
1
vote
2
answers
78
views
Affine 2D mapping in python
I have two sets of 2D data $A$ and $B$ (representing 2D positions on a 2D plane $x,y$) which are related (the first pair of $x,y$ of $A$ is related to the first pair of $x,y$ of $B$ for instance). I ...
- 11
0
votes
0
answers
74
views
Interpreting Perplexity, U_mass coherence and Cv score trends for a Latent Dirichlet Allocation Model
I'm running an LDA model through gensim. To my understanding, closer the u_mass coherence score is to zero, higher is the interpretability of the topics that come up. I'm getting the u_mass coherence ...
- 1
-2
votes
1
answer
63
views
How to decide your first self-project for Data Science/Analysis [closed]
Suppose a complete beginner(in the field of data analysis/data science), which is my case indeed, decided to make a project for his/her resume.
These are the following skills he have learnt:
Python(...
0
votes
0
answers
2k
views
AI tools/LLM models to analyze excel data
I have a bunch of excel files containing information of employee attendance. I want to use NLP-based search to ask questions like: Which employee has taken most leaves?; What dates witness a high ...
- 327
0
votes
0
answers
107
views
Heavy right skewed target
I scrapped Data science job postings of almost 1000 jobs for salary prediction. I wanted to bin the salary column and build a classification model from the job descriptions. But the (salary)target is ...
- 21
0
votes
0
answers
11
views
Exploring Data Science for Enhanced Reporting
I'm a self thought data analyst who has been primarily focused on generating standard reports. However, this year, my role is evolving. My boss has challenged me to not just report data but to extract ...
- 1
0
votes
1
answer
47
views
Best way to analyse change in amplitude of cyclic data?
I have some cyclic data that I'm trying to analyse which I've posted below:
I would like to analyse some overall trends of the data. The peak-to-peak amplitude appears to be decreasing, however, I'm ...
- 1
3
votes
1
answer
1k
views
Pandas Profiling Not Working
I have been working on EDA lately and i got to know about Pandas Profiling ,
but i am unable to import the module Pandas_Profiling , I could import the module on Google Collab but couldnt import the ...
0
votes
1
answer
26
views
Working with functions [closed]
What are some good sources for learning how to deal with functions in Python and R. I have used them a lot of times and they can be a bit challenging. I like to write out a plan of what I want done ...
0
votes
1
answer
27
views
How should I plot 2D data as 3D plot?
I got data (1503, 38697) (cell(datapoint), genes(features)), each elements has integer values (gene expression).
so something like
...
0
votes
0
answers
25
views
To Become a Data Analyst?
currently I am in year4 and also working as a product owner who would want to shift the occupation path to data analyst in banking, but the problem is I don't know where to start. May I have some ...
- 1
0
votes
1
answer
1k
views
ValueError: X has 54 features, but DecisionTreeClassifier is expecting 53 features as input
I am analysing and prediction 2023 Cricket World Cup based on previous given dataset.
This is Exploratary analysis:
Feature selection and Training model:
Applying Random forest classifier algorithm:
...
0
votes
1
answer
43
views
Trying to write a code that detects a maximum value before a drop off in slope
I'm trying to write code for a force test that will output the maximum force before structural failure occurs. I'm a bit of a novice to python, so the issue here might be something simple that I'm ...
1
vote
0
answers
153
views
Is there an elegant way to quantify the mix shift effect?
Imagine there is an online auction business selling 1000+ varieties of fruits. The demand for these fruits change over time, and the prices of these fruits is set by the seller.
Supposed I am ...
- 111
4
votes
1
answer
512
views
Solve tough clustering problem with overlapping clusters
I'm having some trouble to solve a hard clustering problem.
I have a 2D dataset characterized by non spherical and partially overlaping clusters with different densities.
I've read a lot about ...
0
votes
1
answer
16
views
Bibliographic references on statistical analysis of classification results in machine learning
I am looking for a good must-have reference about statistical analysis of classification results in machine learning.
What books do you recommend about it?
0
votes
1
answer
699
views
Is it essential to remove NULL values when doing EDA?
I'm doing a Data Analytics project on Youtube Statistics.
Assume that I want to get into the entertainment industry and decided to make a youtube channel. Before that, I need to identify what content ...
- 1
0
votes
1
answer
40
views
Advice on how to approach a recommendation model for marketing
I am working on a project to provide recommendations to the marketing team to launch effective campaigns. The dataset I have has data on existing customers, their demographic and billing details as ...
- 1
0
votes
1
answer
79
views
Analysing 1GB of 200K of data rows having 50 columns
Total Newbie question here :
5 years ago, I wrote a web application in LAMP stack not using a framework, targeting end users, which now has over 200,000 rows where each row has 50+ columns and the ...
- 103
1
vote
0
answers
22
views
Most popular location
This feels really simple, but looking for some guidance to find the most popular location for orders.
I have three locations: Location A, Location B, and Location C.
Location A had 43 orders out of a ...
- 111
0
votes
2
answers
59
views
Data Analysis Process
How do you feed data from a data warehouse to Python for ad-hoc analysis?
My day-to-day work is to answer ad-hoc questions, and 95% of the data I need is in our data warehouse. I often query data from ...
1
vote
2
answers
74
views
Looking for a couple of ideas please
I’ve got some data by postal zone that includes:
Postal zone code
Average rent value per square foot
Brand affinity 1
Brand affinity 2
Brand affinity 3
Brand affinity 4
…and so on
The brand affinity ...
- 11
2
votes
2
answers
787
views
what qualifies as a data leakage?
I am currently working on a binary classification problem using imbalanced data. The algorithm that I am using is random forest. The problem is about predicting whether each sales project will meet ...
- 2,655
0
votes
1
answer
769
views
What's the best approach to dealing with missing data in a dataset?
I have a dataset that contains missing values in some columns. I would like to know what is the best approach to deal with this missing data. Should I remove rows with missing data or fill in missing ...
- 1
0
votes
3
answers
330
views
Difference between Data analytics and Data Analysis
What we are referring to when we say Data analytics and Data analysis?
0
votes
1
answer
48
views
Study data science at university [closed]
Is it a waste to study data science in university, despite the advancement of artificial intelligence and automation of analysis?
regards
0
votes
0
answers
29
views
What is the steps to generate data from a plot and use them to have a predication using python?
I have a plot that represent a BH curve for magnatic material. The material have a behavior for each H value for two different temperature 25 C and 100 C. Figure 1. I need to extract the data for each ...
- 1
0
votes
1
answer
463
views
Regression with time series data
I want to predict temperature when time (datetime type, hourly data for five months) and humidity is given. Before starting in python, I created a regression model in excel. But instead of predicting ...
- 1
0
votes
1
answer
54
views
Customer Analysis - How to handle unbalanced data?
I'm taking a data analysis course and decided to work on a customer analysis project. In the data, I have three countries:
USA (539 unique users)
BRA(385 unique users)
TUR (129 unique users)
I'm ...
- 103
0
votes
1
answer
51
views
Feature engineering for interest-based age classification
I have a dataset which has users (rows) with the list of their interests (IABs), which looks like this
...
- 1
1
vote
1
answer
109
views
Is Data Analysis/Science good for online freelancing (Upwork, Freelancer...)? [closed]
My interest in data analysis/science began about a year ago because it combines statistics, analysis, and programming. Unfortunately, I was surprised that there was almost no demand for it in the job ...
- 11
0
votes
1
answer
13
views
What's the word for aggregating by two attributes together, rather than separately?
Let's imagine we have this underlying data:
...
- 111
0
votes
1
answer
18
views
Dropping duplicated when visualising different granularity in the same df
I have a dataset about a bunch of users and their comment on Instagram, each row is a comment. Some of the columns of this table are related to the users and some of them have multiple comments.
I'm ...
- 222
0
votes
1
answer
69
views
Understanding Box plots
I am new to box plots and I am trying to understand it. Consider the following figures :
According to my understanding, outliers will exist after the max. line and before the min. line. So according ...
- 145
0
votes
1
answer
23
views
How do I handle "Greater than X" in a field of integers?
I've been tasked with cleaning a dataset with a "Drive Time" column that lists times taken to drive to a specific location in whole minutes. The values range from 3 to 180 minutes but there ...
- 1
0
votes
1
answer
24
views
Total Retention Rate Calculated from Categories
I am calculating retention for 3 categories and then total, and I am trying to double check my total, but my check formula isn't working.
I am comparing the last 14 days (let's call it Period 1) to ...
1
vote
1
answer
280
views
Quantitatively evaluate similarity between two corpus of texts
I want to assess how similar, or different two corpora are, and if the similarity is statistically significant. Something close to a Kolmogorov–Smirnov test in statistics, but for text data.
For ...
- 53