Questions tagged [data-analysis]
Data analysis is a process of inspecting, cleansing, transforming, and modeling data with the goal of discovering useful information, informing conclusions, and supporting decision-making.
281 questions
1
vote
1
answer
112
views
How is DS used in the case of Payment Gateways?
I know it's a general question but what type of analytics can be done in this case?
How can we apply machine learning models here?
- 321
1
vote
1
answer
22
views
How to order the data with respect to data type
I am having large data set (82 variables)
Is there any way to arrange data such a way that I have to get all numerical variables firstly then categorical variables so that I can run hypothesis ...
- 25
0
votes
1
answer
129
views
how to know what my data are uninformative and that machine learning will not work with it?
following this question, I'm making a data analysis again because I tried to use machine learning algorithms like Random forest to predict a value from certain features but it didn't work for me. I ...
- 243
1
vote
1
answer
2k
views
How to create a score for a SWOT analysis (strengths, weaknesses, opportunities, and threats)?
I'm developing a participatory social environmental diagnostic. To do this, I'm using primary (qualitative data from interviews with stakeholders) and secondary data (local socioeconomic data).
From ...
- 113
3
votes
2
answers
78
views
is it a good idea to take the derivative or integral of some features and add them as new features in machine learning?
I'm learning how to do feature Engineering and come across some ideas in my head that's why I want to ask if I had some dataset with some features let's say 2 features and I have a timestamp column ...
- 243
1
vote
0
answers
47
views
Forecasting future close returns
I have end-of-day time-series dataset with following data
Ticker Date Open High Low Close Volume
I have calculated end-of-day closing returns ...
- 121
2
votes
1
answer
66
views
Analysis of sales performance
This is my first question on this stackexchange - i hope i'm in the right place and i'm asking the right question.
I work for a business, we sell certain products.
We are running a trial currently on ...
- 121
0
votes
1
answer
27
views
How to asses employees performance using their data? [closed]
I have some employees' data consisting of three variables - the name of the employee, a value denoting their positive behavior and a value denoting their negative behavior.
How to score employees ...
0
votes
3
answers
55
views
How to treat column with potentially meaningful NaNs
My data set has a column that indicates the time taken (in days) for members on a site - each with an ID - to sign up for an event. This can range between 1 to 300 days, with about half of the rows ...
- 1
3
votes
1
answer
126
views
Detecting specific points in (noisy) dataset
In my recent work I've came across a problem where I need to find certain points in quickly oscillating data. Let's work with syntethic data instead of real measurements and ignore the noise for the ...
- 131
0
votes
1
answer
127
views
Analysis for basic weight training analysis?
TL;DR: I'm doing a fairly basic project which involves exercise. It seems that descriptive statistics and basic data vis (ex: line graph) would be most appropriate for this project, but I wonder if ...
- 9
1
vote
0
answers
22
views
Best Method for Data Analysis on a 100 numerical IVs and 200 numerical DVs
I think I might need the help of this valuable community for a task. I have been given a dataset for 100 numerical independent variables (IVs) that predict output for 200 numerical values (from monte ...
- 11
1
vote
0
answers
190
views
Filling 2 different values for missing/NaN columns
I am doing a binary classification problem (TARGET = 0 or 1). My dataset contains some NaN ...
- 23
0
votes
1
answer
49
views
Best way for correlation analysis
I am looking for an alternative way of analysing my data which will be used for ML. I am using Matlab to implement my code. Using my code, I obtain arrays which are used to describe signals. Each ...
- 1
10
votes
4
answers
2k
views
Math PhD (Nonlinear Programming) switching to Data Science?
I am a math Ph.D. student who is interested in going to the industry as a Data Scientist after graduation. I will briefly give some background on my education before posing my question, so that it is ...
- 123
1
vote
2
answers
6k
views
1
vote
4
answers
2k
views
When to normalize or regularize features in Data Science
How can I know when I need to use normalization and when regularization of features.
I know that when I have big difference between min and max values in some feature / column that i need to scale ...
- 199
1
vote
1
answer
28
views
Causation vs correlation problem in case of fixing other parameters and changing only one
I have a game that players play. D7 retention is defined as percentage of players who installed my game and after exactly 7 days later they played my game.
Now I am doing analytics and it shows that ...
- 121
0
votes
1
answer
272
views
How Retail Analytics Project Can Be Done?
I have a project and I couldn't understand what I have to do because I am new to retail analytics.
They said "Our goal is to measure the effects of promotion on sales" and "Your goal is ...
- 11
1
vote
2
answers
73
views
I have a 20.2 GB dataset which is entirely consisting of 1 file. Need help on how to open the file [closed]
Whenever I am dealing with datasets that have weird extensions that I have not encountered before, I typically open them with notepad to see how the data looks like before using pandas to analyze them....
- 181
0
votes
1
answer
1k
views
Handling unwanted negative numbers
With me is a dataset collected from IoT sensors with one column labeled “Soil Humidity” measured in percentage. It stands to reason then that all the values be positive percentages, however there’s a ...
- 3
0
votes
1
answer
673
views
counter vector fit transform cosine similarity memory error
count_matrix = count.fit_transform(off_data3['bag_of_words'])
I have count_matrix shape with
count_matrix.shape
(476147, 482824)
...
- 513
1
vote
0
answers
231
views
Is it possible retrive hourly data from daily weather data?
I'm a beginner, so sorry if my question could be basic. Reading on internet I've found example written in python that makes the reverse of my questions (covert from daily to monthly).
My problem is ...
- 111
0
votes
1
answer
76
views
Simple EDA: what to visualize?
I want to visualize a data set for EDA having 8 parameters and 2 class labels.
I am confused which parameters to select and on what basis? Also is I want to know whether it would be better to apply ...
- 23
1
vote
2
answers
19k
views
ValueError: pos_label=1 is not a valid label: array(['N', 'Y'], dtype='<U1')
X = train_encoded_df.iloc[:, 1: ]
y = train_encoded_df["Loan_Status"]
print("Precision:",metrics.precision_score(y_test, y_pred))
My training data contains the ...
- 13
0
votes
1
answer
96
views
Plot misclassfied values in matplotlib
I have two lists. One is target and another is predicted values in a binary classification. The target is always zero. And I have to plot the results of the prediction against target in such a way, ...
1
vote
1
answer
43
views
Data analytics in road transportation
What kind of insights can we give from point A to point B transactions. A and B are two places, All I can think of is about descriptive statistics like how many transactions from A and what is count ...
- 21
0
votes
1
answer
88
views
In which CRAN mirror is the quadprog package available?
I was trying to find the accuracy of some predicted data and I came to know that I can use the forecast package to do this. When I installed it and gave library(forecast), it said it needs quadprog ...
3
votes
2
answers
441
views
why One-Hot Encoder can avoid the situation that the model will misunderstand the data to be in some kind of order if the data has been Label Encoding
We know that we prefer to using One-Hot Encoding not Label Encoding when processing with non-ordinal data.
And I real a blog which give the difference between Label Encoding and One-Hot Encoding.
So ...
- 287
0
votes
1
answer
76
views
comparing data from first quarter in two years
How can I compare data from two quarters? What software can I use to do it parallelly side by side so I can compare them. What kind of factors can I take into consideration, I'm posting a sample data ...
- 21
0
votes
1
answer
521
views
No target variable in my data
I have a list of transactions of bus route from place to place, I don't have any target variable here. I was asked to give meaningful insights from the data, what can I do here?
I cleaned the data, ...
- 21
0
votes
2
answers
343
views
White noise seasonality
Is the above graph a white noise?
I'm confused by the spikes at certain places.
The above plot has been obtained after doing a first order differencing on a time series. How can I justify whether ...
- 231
1
vote
0
answers
89
views
Model for time series analysis
I'm new to data analysis and ML in general. I'm working with some friends on this problem: We're trying to predict when a component of a machine will stop working properly so the client can change it ...
- 11
3
votes
1
answer
190
views
How to figure out if the problem is time series Forecasting or not?
Recently I have encountered a time-series based, where I have a dataset which contains data for call-centers performance. The dataset contains information about the number of calls made by customers ...
3
votes
2
answers
389
views
Possible Challenges for a Data Science Escape Room
Dear Data Science Community,
as my project for my bachelorthesis I am working on a concept for a physical Data Science Escape Room. The goal of the escape room is to create awareness for data science ...
8
votes
2
answers
9k
views
Covariance as inner product
Why is covariance considered as inner product if there is no projection of one vector onto another?
Right now I perceive this as just a multiplication of $x$ segment of vector($x_i - \bar{x}$) and $y$...
- 143
0
votes
2
answers
5k
views
How to create word2vec for phrases and then calculate cosine similarity
I just started using word2vec and have no idea how to create vectors (using word2vec) of two lists, each containing set of words and phrases and then how to calculate the cosine similarity between ...
- 129
0
votes
4
answers
10k
views
How to get date columns in pandas?
I need a separate date column(in this format2018-10-08) in this table. please help.
...
6
votes
0
answers
95
views
Fitting model to differenced time series
I have a time series on daily stock price of company(2013 data points).I took a first order difference and the following acf and pacf plots of the differenced series were obtained. However, I am ...
- 231
1
vote
1
answer
245
views
How could I go about finding the weights or importance of inputs based on outputs?
I have a table who's inputs (sfm, fr, and doc) all affect the outputs (...
- 155
3
votes
1
answer
1k
views
Comparing rows in Data Studio
I am using Data Studio for a project and I am connecting to my BigQuery table.
My table contains following columns:
Date
Store_name
Footfall
I'd love to compare the footfall of two stores using Data ...
- 171
0
votes
1
answer
52
views
Which different visualizations to make for time series analysis?
I hope you're doing great. I have recently started working on a time series project and I have just finished preparing my data and calculating trends (daily, weekly, yearly) and I wonder what are the ...
- 171
2
votes
1
answer
447
views
EDA for analysis of nominal variable with high cardinality
I have a nominal variable (car model) with very high cardinality (~8500 labels) and I would like to analyse its relation with a binary target variable. While I can create logical groups and compare ...
- 321
5
votes
5
answers
56k
views
Additive vs Multiplicative model in Time Series Data
The above time series plot is a daily closing stock index of a company. I want to know which model between additive and multiplicative best suits the above data. I know what the two models are, but i ...
- 231
2
votes
2
answers
916
views
Analysis of Time Series data
The below graph is a scatterplot of daily stock price. My aim is to predict future stock price of the company.
From the scatterplot it seems that it is a multiplicative model, so I tried to "...
- 231
0
votes
1
answer
565
views
clustering with heterogeneous (quantitative and qualitative)data?
I'm a Phd student and I have the results of some approaches (algorithms) that I would like to analyze. Data (results) are stored in csv files as follows: - the lines describe each algorithm with its ...
4
votes
1
answer
6k
views
Monthly trend with fb prophet-Interpreting the graph
I have monthly data with month/year in one column and price on another. I would like to get a yearly trend with fb prophet library in python (how to use monthly data with the library is explained at ...
- 167
1
vote
1
answer
70
views
Comparative Analysis of two sets of data
I have 2 sets of data which consist of marks of 60 students in a particular subject in year 1 and year 2 respectively.
Year 1 : 86, 76, 87, 67, 89, 95.... so on (60 students)
Year 2 : 82, 67, 99, 77,...
2
votes
2
answers
154
views
Predicting missing data. Looking for good data predicting technique
I am analysing data for Countries Trade GDP. Some of the countries have missing GDP value for given a year. However, I have Grand Total for the entire region for that year. Is there a good data ...
0
votes
1
answer
146
views
Will I be able to fetch a Data Analyst job if I learn everything in this image?
India
Asking such a question because I am interested learn everything myself and I am confused about what to read.
And I am interested by reading books, will that suffice? or should I compulsorily ...
