Questions tagged [data-analysis]
Data analysis is a process of inspecting, cleansing, transforming, and modeling data with the goal of discovering useful information, informing conclusions, and supporting decision-making.
284
questions
0
votes
0
answers
5
views
How should I plot 2D data as 3D plot?
I got data (1503, 38697) (cell(datapoint), genes(features)), each elements has integer values (gene expression).
so something like
...
0
votes
0
answers
18
views
To Become a Data Analyst?
currently I am in year4 and also working as a product owner who would want to shift the occupation path to data analyst in banking, but the problem is I don't know where to start. May I have some ...
- 1
0
votes
0
answers
23
views
How to infer the following graphs for dimensionality reduction?
I'm dealing with a high-dimensional(1600 features, 9500 columns) binary classification problem. My current accuracy, and other metrics are not upto the mark. I am trying different feature selection ...
0
votes
1
answer
71
views
ValueError: X has 54 features, but DecisionTreeClassifier is expecting 53 features as input
I am analysing and prediction 2023 Cricket World Cup based on previous given dataset.
This is Exploratary analysis:
Feature selection and Training model:
Applying Random forest classifier algorithm:
...
0
votes
0
answers
30
views
Calculate the curve of a list and apply that to calculate the values of a larger list
I have a list of ~125 sorted ascending values and I am converting those to percentages of total so that they add up to 100% (shortened example would be: 0%, 0%, 15%, 15%, 30%, 40% with the total of ...
- 1
0
votes
1
answer
22
views
Trying to write a code that detects a maximum value before a drop off in slope
I'm trying to write code for a force test that will output the maximum force before structural failure occurs. I'm a bit of a novice to python, so the issue here might be something simple that I'm ...
0
votes
0
answers
24
views
Growth (sales) forecast based on season
How to make a forecast based on last year (season)?
I have no experience in the field
Make a forecast of the sales in september 2023 based on last years sales
Example A
Sales august 2022 (last year): ...
- 101
1
vote
0
answers
47
views
Is there an elegant way to quantify the mix shift effect?
Imagine there is an online auction business selling 1000+ varieties of fruits. The demand for these fruits change over time, and the prices of these fruits is set by the seller.
Supposed I am ...
- 111
3
votes
1
answer
62
views
Solve tough clustering problem with overlapping clusters
I'm having some trouble to solve a hard clustering problem.
I have a 2D dataset characterized by non spherical and partially overlaping clusters with different densities.
I've read a lot about ...
0
votes
1
answer
13
views
Bibliographic references on statistical analysis of classification results in machine learning
I am looking for a good must-have reference about statistical analysis of classification results in machine learning.
What books do you recommend about it?
0
votes
0
answers
8
views
Displaying PowerBI and Microsoft Excel Files for Data Analytics Portfolio
I tried uploading a PowerBI file and an Excel file to GitHub, but it doesn't render them as GitHub would a Jupyter Notebook, for example.
What service can I upload the aforementioned files to for a ...
- 101
0
votes
1
answer
73
views
Is it essential to remove NULL values when doing EDA?
I'm doing a Data Analytics project on Youtube Statistics.
Assume that I want to get into the entertainment industry and decided to make a youtube channel. Before that, I need to identify what content ...
- 1
0
votes
0
answers
14
views
Find the time and magnitude of abrupt change in time series
I have several time series where there sometimes are abrupt continuous shifts in the data (see example below).
Specifically these are measurements so the reference level (RL) must have changed. E.g., ...
- 1
0
votes
1
answer
35
views
Advice on how to approach a recommendation model for marketing
I am working on a project to provide recommendations to the marketing team to launch effective campaigns. The dataset I have has data on existing customers, their demographic and billing details as ...
- 1
0
votes
0
answers
55
views
Implicit feedback based recommendation - User - User recommendation
My dataset involves customer_id, item_id and Count of purchases. My datset size is small...we have 15 users and 317 items..
Currently, am trying to build a recommendation system based on user based ...
- 2,555
0
votes
0
answers
20
views
Statistical approach for comparing ranks
I'm looking for a statistical approach to compare ranks produced by 2 versions of an ranking algorithm (A & B) against the actual ranks in the system. This is about ranking hospitals and the ...
- 101
0
votes
1
answer
43
views
Analysing 1GB of 200K of data rows having 50 columns
Total Newbie question here :
5 years ago, I wrote a web application in LAMP stack not using a framework, targeting end users, which now has over 200,000 rows where each row has 50+ columns and the ...
- 103
1
vote
0
answers
22
views
Most popular location
This feels really simple, but looking for some guidance to find the most popular location for orders.
I have three locations: Location A, Location B, and Location C.
Location A had 43 orders out of a ...
- 111
0
votes
0
answers
18
views
I'm looking for a dataset on enviroment monitoring
I'm looking for a dataset related to environmental monitoring, made up of values obtained from various types of sensors (such as temperature, pressure, CO2...etc) for the purpose of a classification ...
- 1
0
votes
2
answers
39
views
Data Analysis Process
How do you feed data from a data warehouse to Python for ad-hoc analysis?
My day-to-day work is to answer ad-hoc questions, and 95% of the data I need is in our data warehouse. I often query data from ...
1
vote
2
answers
61
views
Looking for a couple of ideas please
I’ve got some data by postal zone that includes:
Postal zone code
Average rent value per square foot
Brand affinity 1
Brand affinity 2
Brand affinity 3
Brand affinity 4
…and so on
The brand affinity ...
- 11
1
vote
2
answers
163
views
what qualifies as a data leakage?
I am currently working on a binary classification problem using imbalanced data. The algorithm that I am using is random forest. The problem is about predicting whether each sales project will meet ...
- 2,555
0
votes
1
answer
148
views
What's the best approach to dealing with missing data in a dataset?
I have a dataset that contains missing values in some columns. I would like to know what is the best approach to deal with this missing data. Should I remove rows with missing data or fill in missing ...
- 1
0
votes
0
answers
336
views
TypeError: Could not convert group to numeric
I am going through Krish Naik's "End To End ML Project With Deployment" course on YouTube. Right now, I am in Tutorial 3. Here is the link to the video.
I am facing an issue with BIVARIATE ...
- 1
0
votes
3
answers
92
views
Difference between Data analytics and Data Analysis
What we are referring to when we say Data analytics and Data analysis?
0
votes
1
answer
42
views
Study data science at university [closed]
Is it a waste to study data science in university, despite the advancement of artificial intelligence and automation of analysis?
regards
0
votes
0
answers
16
views
Having trouble manipulating data to make better dashboard
I have a test dataset which has has 39 columns. Index is given below:
Sample data is as follows:
Some columns are omitted due to privacy concerns. The colored cells in the index belong to one type. ...
0
votes
0
answers
27
views
What is the steps to generate data from a plot and use them to have a predication using python?
I have a plot that represent a BH curve for magnatic material. The material have a behavior for each H value for two different temperature 25 C and 100 C. Figure 1. I need to extract the data for each ...
- 1
0
votes
1
answer
186
views
Regression with time series data
I want to predict temperature when time (datetime type, hourly data for five months) and humidity is given. Before starting in python, I created a regression model in excel. But instead of predicting ...
- 1
0
votes
1
answer
48
views
Customer Analysis - How to handle unbalanced data?
I'm taking a data analysis course and decided to work on a customer analysis project. In the data, I have three countries:
USA (539 unique users)
BRA(385 unique users)
TUR (129 unique users)
I'm ...
- 103
0
votes
1
answer
33
views
Feature engineering for interest-based age classification
I have a dataset which has users (rows) with the list of their interests (IABs), which looks like this
...
- 1
1
vote
1
answer
84
views
Is Data Analysis/Science good for online freelancing (Upwork, Freelancer...)? [closed]
My interest in data analysis/science began about a year ago because it combines statistics, analysis, and programming. Unfortunately, I was surprised that there was almost no demand for it in the job ...
- 11
0
votes
1
answer
13
views
What's the word for aggregating by two attributes together, rather than separately?
Let's imagine we have this underlying data:
...
- 111
0
votes
0
answers
15
views
Why do you need more subjects than levels in RM ANOVA?
Honestly, I'm hoping I even asked this question appropriately. It's one of those things I never quite grasped, but just accepted to be true, that when running a within-subjects ANOVA, you have to have ...
- 1
0
votes
1
answer
17
views
Dropping duplicated when visualising different granularity in the same df
I have a dataset about a bunch of users and their comment on Instagram, each row is a comment. Some of the columns of this table are related to the users and some of them have multiple comments.
I'm ...
- 222
0
votes
1
answer
30
views
Understanding Box plots
I am new to box plots and I am trying to understand it. Consider the following figures :
According to my understanding, outliers will exist after the max. line and before the min. line. So according ...
- 145
0
votes
1
answer
20
views
How do I handle "Greater than X" in a field of integers?
I've been tasked with cleaning a dataset with a "Drive Time" column that lists times taken to drive to a specific location in whole minutes. The values range from 3 to 180 minutes but there ...
- 1
0
votes
1
answer
17
views
Total Retention Rate Calculated from Categories
I am calculating retention for 3 categories and then total, and I am trying to double check my total, but my check formula isn't working.
I am comparing the last 14 days (let's call it Period 1) to ...
0
votes
0
answers
31
views
Want to create 2 column metrics in Pandas based on same dataset
Snapshot of the data given below:
I want to create two new columns from this data set.
I need to create a scatter plot to display how each firm performed in the year 2017 based on their average score ...
- 11
1
vote
1
answer
51
views
Quantitatively evaluate similarity between two corpus of texts
I want to assess how similar, or different two corpora are, and if the similarity is statistically significant. Something close to a Kolmogorov–Smirnov test in statistics, but for text data.
For ...
- 53
0
votes
0
answers
12
views
Marvel Phase 4 Review Comparison Data Analysis
I have some skepticism about the validity of the charts below comparing the critic and audience reviews for Phase 4 of the MCU to the previous 3 phases. There are over 18 movies and tv shows in Phase ...
0
votes
0
answers
21
views
Any ideas on deciding a hotel competitive group?
Suppose for each hotel, I need to find a group of hotels which are competitive to this hotel. Competitive hotels mean they probably share the same group of customers, and the customers will make a ...
1
vote
1
answer
16
views
Complex graphical representation based on monthly groups
I'm still fairly new to R, and am having some real issues turning a data frame into a graphical representation.
I downloaded some data for house prices in the UK and stripped it down to just the ...
- 11
0
votes
1
answer
22
views
How do I get a big-picture feel for my database?
Background: I was just granted access to my company's SQL Server (MS SQL 2019 via Azure Data Studio). As I have not yet been assigned any specific projects I want start digging.
I've asked my DBA to ...
1
vote
1
answer
25
views
Data Analytics Documantaions
I am working as a data analyst in a company. Me and my colleagues use different tools and software to analyze the data and make the reports (e.g., Excel, Python, R, Alteryx, SQL, Tableau).
Each one ...
- 1,985
0
votes
0
answers
29
views
Methods for measuring/identifying themes in complaints data that are "interesting"?
all.
I am posting this as more of a discussion post, rather than a question, as I am trying to come up with ideas for a project I have been asked to consider.
For context, I work for a customer ...
0
votes
0
answers
26
views
Substantive differences between using the term "more likely" vs. "higher share" when reporting survey data?
I have a question on proper language to describe differences in survey data. I'm doing some simple descriptive statistics using nationally representative survey data.
I am using svyttest in r to test ...
- 1
0
votes
1
answer
35
views
Success metric of database migration using row counts
Description
I have a problem where I'm tasked to successfully transform and repurpose data from one SQL server to another. Call the source $\text{src}$ and the target database $\text{tgt}$.
In order ...
- 101
0
votes
0
answers
124
views
How to test and discover dependency between time series?
$x_1(t), x_2(t), x_3(t), ...$ are independent variables, and $y(t)$ is another variable I suspect to be dependent. I have dataset of the evolution of $x_1(t), x_2(t), x_3(t), ..., y(t)$ over time in ...
2
votes
1
answer
751
views
Is it Possible to plot Scatter Plot + Histogram + Correlation Values in a single plot (in python)?
I recently came across corrmorant package in R.
It allows to plot all three basic EDA plots together: Scatter Plot + Histogram + Correlation Values.
Is it possible ...
- 123