Questions tagged [data-analysis]

Data analysis is a process of inspecting, cleansing, transforming, and modeling data with the goal of discovering useful information, informing conclusions, and supporting decision-making.

Filter by
Sorted by
Tagged with
0 votes
0 answers
17 views

How can "profiling" be done in a dataset? What are the different techniques?

I am currently doing an analysis in which I need to "profile" each record. For example, let's say I have a dataset of accounts with customer information (name, id, address, money spent, ...
user avatar
0 votes
1 answer
15 views

How do I not display rows that have an empty value when trying to output a dataframe with pandas [closed]

I have this table where there are missing values under the value2 column. Value1 Value2 1000 1000 1000 500 1000 560 1000 560 What I would like to do is to display the above table but without ...
user avatar
  • 17
0 votes
0 answers
13 views

Data Sets for _Pricing Analytics_ by Paczkowski?

I am working through Pricing Analytics by Paczkowski https://www.amazon.com/Pricing-Analytics-Walter-R-Paczkowski/dp/1138623938/, and it would help me greatly if the data sets for the various examples ...
user avatar
1 vote
1 answer
25 views

Having weird accuracy graph on deep learning binary classification model

I am doing deep learning binary classification on some data and got very weird results with the accuracy metric. In the first few epochs, it doesn't change at all but then it goes on this weird linear ...
user avatar
  • 125
0 votes
0 answers
8 views

Approach to check multicollinearity for data with categorical and continuous features?

I would like to know what should be the best approach to check for multicollinearity if my data has categorical and continuous variables like:- Age, Income, Department(more than 2 category), Gender(2 ...
user avatar
0 votes
1 answer
15 views

Marketing Spend Optimization Techniques

I need some help with market spend optimization. I’m working with a client who’s running an offline operation that’s primarily driven by online marketing (fb, google, twitter etc). They had asked me ...
user avatar
0 votes
0 answers
12 views

How do I get the divided values of two columns that are a result from a groupby method [migrated]

I currently have a dataframe that was made by the following example code df.groupby(['col1', 'col2', 'Count'])[['Sum']].agg('sum') which looks like this col1 col2 ...
user avatar
  • 17
1 vote
1 answer
27 views

How to return the number of values that has a specific count

I would like to find how many occurrences of a specific value count a column contains. For example, based on the data frame below, I want to find how many values in the ID column are repeated twice <...
user avatar
  • 17
-1 votes
0 answers
19 views

Optimal representative participation in the age of AI/ML? [closed]

Just answered the great questions of the 2022 Stack Overflow Developer Survey and earned the Census - Badge, here I see the number of filled surveys and as an enthusiastic beginner in data science I ...
user avatar
  • 99
0 votes
1 answer
16 views

Determine the effect on margins of a price increase

I hope you can help guide me in the right direction! Any advice is appreciated! Situation I'm currently analyzing the effect of a price increase from a retailer on a few 100 products. I'm interested ...
user avatar
0 votes
0 answers
17 views

Are there any open datasets for commercial use?

I am creating a bootcamp for data analyst and it's been 2 days I am looking for some good dataset fit for commercial use that I can use to create Tableau and Power BI tutorials. Even on kaggle some ...
user avatar
  • 141
0 votes
0 answers
11 views

Unable to generate useful insights on a highly cardinal data

I'm working on CRM data, did some cleaning, encoding and ran a decision tree classifier from which i plotted a feature_importance graph From that I found that Sales person column is one of the ...
user avatar
0 votes
0 answers
32 views

Find how the properties of an entity affect a certain property of its surrounding

We have a set of things (physical entities): ($A_1$, $A_2$, $A_3$, $A_4$,...). Each of those has certain attributes that we can measure at time $t = 0$. Let $B_i$ represent those attributes, so each ...
user avatar
0 votes
0 answers
17 views

Approaches on grouping/clustering network device data

So I come from more of a computer science background, and recently have been trying to find a solution to a data-centered problem. I would like to try experimenting different data-science methods on ...
user avatar
  • 1
1 vote
0 answers
22 views

How to re-generate dataset df at time t2 while having df at time t1 and cross-sectional dataset df' at time t1 and t2?

I have a travel survey dataset df' collected in 2017 and also 2019. Note that individuals (households here) are not necessarily identical in 2017 and 2019 but their features are. Dataset df' in 2017: ...
user avatar
1 vote
1 answer
29 views

How to test likelihood hypothesis on dataset?

How to test the following hypothesis? The larger the fare the more likely the customer is to be travailing alone. Using the data below, how would one be able to test the hypothesis? ...
user avatar
0 votes
0 answers
13 views

Looking for data analysis techniques and approach

I'm new into ML and I need to do a data analysis on a dataset which I created myself but I don't know what techniques should I use exactly. Namely, I have a dataset with the following attributes: <...
user avatar
  • 101
1 vote
1 answer
15 views

How do I estimate the benefit of one business method over another when all my data is confounded?

I'm trying to estimate the benefit of using one method over another, but the existing data is confounded and I'm struggling to figure it out. So my company submits debit orders to customers to collect ...
user avatar
0 votes
1 answer
13 views

Calculating relative risk in R

I am working with the nhanes data from 2017-2018. I want to determine the relative risk of being overweight (Adipös) when being "Poor". And this grouped by Gender and Ethnicity. So far I ...
user avatar
  • 25
1 vote
1 answer
27 views

Replacing count by percentage in R

I have the following code : ...
user avatar
  • 25
0 votes
1 answer
25 views

Clustering a set of matrices with kmeans/other approaches

I am posting here as overflow community told me it was better to ask it here. I was trying to cluster a set of matrices in order to obtain $n$ clusters based on matrix similarity. In practice, my ...
user avatar
  • 111
0 votes
0 answers
11 views

How can I do continuum regression in R?

I am looking for a R package that does continuum regression. More concrete I need a function that does continuum regression s.t. I can evaluate the values afterwards. At least extracting MSE or RMSE ...
user avatar
  • 5
0 votes
0 answers
7 views

I want to do analysis on trends moving across categories . What I want to know is what are the suitable ways to identify a moving trend

For en example for a trend movement is USA Beverage moves to USA Alcoholic Beverage after some time. So what are the statistical tests available to identify about this kind of movement in two time ...
user avatar
0 votes
0 answers
16 views

Correlation analysis yields conflicting results. Positive Pearson and Negative Spearman

I have four features x1,x2,x3,x4 all of their correlation with y are similar in Pearson and in Spearman separately. However, all these are +0.15 in Pearson and -0.6 in Spearman, more or less. Does ...
user avatar
0 votes
0 answers
16 views

Using ML Interpretability Techniques for Data Analysis Instead of Strictly Model Analysis

Hope you lot are doing alright. I have been looking into Explainable AI and model interpretability lately, and I had an idea but am wondering whether it would constitute a valid use case. There is a ...
user avatar
0 votes
0 answers
15 views

Why we need a statistical hypothesis testing for correlations between variables when we can check using scatter plots?

Need Guidance on correlation test. When we need to perform correlation between 2 variables, we generally start with scatter plot. Sometime it is suggested to perform hypothesis testing as well. I'm ...
user avatar
0 votes
0 answers
47 views

Using Markov Chains in digital customer journey

I have data of each page visited by a customer in a session, my objective is to find out the most optimal path where we see the maximum conversion rate. My idea Is to use Markov Chain to identify that ...
user avatar
4 votes
1 answer
190 views

How to interpret standard deviation calculated by excel

I have some percentage values in my excel spreadsheet like: When i use the STDEV function to calculate the standard deviation with these values, i get ...
user avatar
1 vote
1 answer
56 views

SKLearn PCA explained_variance_ration cumsum gives array of 1

I have a problem with PCA. I read that PCA needs clean numeric values. I started my analysis with a dataset called trainDf with shape ...
user avatar
  • 113
0 votes
1 answer
128 views

Study names not showing in forest plot (R)

Context: I followed an existing R script to create a forest plot in R using the forest function from the metafor package. Problem: The plot is showing, however, the study names are not showing on it ...
user avatar
  • 39
0 votes
2 answers
72 views

Evaluating if metric of one group is higher than the metric of another group when group sizes differ significantly

I am working with a dataset that contains data of applicant income, gender, and loan status (whether or not the person was approved for a loan). I've created the following plots from the data. The ...
user avatar
  • 73
0 votes
0 answers
13 views

use cases of geocoding maps

Tableau has interactive maps that can be used for custom geocoding. I know companies often divide regions for sales and marketing purposes but could anyone think of any specific use cases or examples ...
user avatar
  • 181
0 votes
1 answer
16 views

How to assess a trend during time?

I wanted to know which analysis method would you suggest for assessing a trend during years. Let us say we have data on antibiotic resistance in a particular bacteria from 2010 until 2021. How can we ...
user avatar
0 votes
0 answers
17 views

Finding negative drop in time series data

I'm trying to detect the negative peaks in this time-series data. There are two series of data (catch: they fall at the same time). The problem I'm facing is that the peak is relative to the ...
user avatar
0 votes
0 answers
17 views

How to match people in preference ranked survey results?

I'm sending out a survey that I want to use to create pairs. For example, each person indicates whether they want to be a mentor, or a mentee. They then stack rank 10 topics that they're interested in ...
user avatar
0 votes
0 answers
11 views

Tableau - Filter Buttons not filtering correctly

I created filter buttons to use as a filter for 6 category's (measures) in Tableau Public (latest version) so when you click on them the values for each display on my "Military Enrollment By ...
user avatar
  • 1
0 votes
2 answers
110 views

Which analytics metrics are useful?

I am working on a home project and the question is : A session is defined as a period of activity in the app. If there is a period longer than 5 minutes when the user does not do anything actively ...
user avatar
1 vote
1 answer
35 views

Knit with R markdown [closed]

I have tried running some code on RStudio Desktop, within the Chunk the codes runs smoothly, but when knitting to view as html, I get the message that something is wrong with a line of code. What can ...
user avatar
0 votes
0 answers
41 views

Methods to extract / merge signal from three noisy time series (of same event)

I have three time series of same length, all containing magnitude measurements of the same event "A". But each time series is using a different method of measurement. My goal is to merge the ...
user avatar
  • 101
1 vote
2 answers
46 views

For a student who is a beginner in quantitative research and statistics, which is the better statistical tool to start: R or IBM SPSS? Why? [closed]

Currently, I am writing my research design. However, I am still indecisive on what statistical tool should I use for the data analysis. I tried looking up on the internet and there are disparate ...
user avatar
1 vote
0 answers
48 views

How to represent/aggregate data belonging to same group, and feed to model for training?

I have a problem in which there are couple of records that belong to the same group, like multiple observations. For example this data set has three groups, For each group, I have three observations....
user avatar
  • 113
0 votes
1 answer
30 views

Cost Efficient Machine Learning Development Using Cloud GPUs

Let's say I have a Juypter Notebook I am working on where I am analyzing, visualizing, testing, etc. various Machine Learning Models with different hyperparameters on some arbitrary data set or I am ...
user avatar
  • 11
0 votes
1 answer
42 views

Online Generating Dendrograms with imported CSV file

Is there any website out there that performs hierarchical clustering on an imported file like CSV to generate dendrograms or similarity matrix? Free or paid does not matter. So far I found heatmapper....
user avatar
  • 1
1 vote
0 answers
33 views

How to integrate Zoho analytics with Jupyter notebook? [closed]

I am trying to connect Zoho Analytics and Python for importing data from Zoho Analytics. I have already installed !pip install zoho-analytics-connector. What should ...
user avatar
0 votes
1 answer
74 views

How to find median/average values between data frames with slightly different columns?

I am trying to combat run-to-run variance of the data I collect by combining the data from different runs and finding the mean/average. The problem is that in each run there is a chance that some of ...
user avatar
0 votes
0 answers
23 views

Weighted Voting: Accuracy and Coverage in Class Weight

I have a data set in which the data is coming from various sources. Approx 3k records were verified manually and respective source is tagged if the data comes from that source and is valid/correct. I ...
user avatar
0 votes
0 answers
19 views

What is the best statistical test to compare the competitiveness of 10 football/soccer leagues within a time range?

I was wondering what stat test is the best to find out which football/soccer league is the most competitive. The leagues that I would choose for this analysis are either based on the countries that ...
user avatar
0 votes
0 answers
19 views

generate synthetic data for multivariate data points

I have some data like.. ...
user avatar
0 votes
0 answers
31 views

ML techniques for subscription data

What are the ML methods one should use for subscription data comprising of visits, monthly active user, user interaction, product usage, location, source. predicting cost for our services.
user avatar
0 votes
0 answers
12 views

Video games with built-in logging

I'd like to get some practice with reinforcement learning and data analysis in the domain of video games. I'm looking for a game that logs user behaviours (e.g. UI click events, player position, item ...
user avatar
  • 1,051

1
2 3 4 5 6