Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [data-analysis]

The tag has no usage guidance.

0
votes
1answer
17 views

Comparative Analysis of two sets of data

I have 2 sets of data which consist of marks of 60 students in a particular subject in year 1 and year 2 respectively. Year 1 : 86, 76, 87, 67, 89, 95.... so on (60 students) Year 2 : 82, 67, 99, 77,...
0
votes
0answers
18 views

Rate quality of sources

Sorry if the title is misleading, I don't know how to describe the issue correctly, which also makes it hard to google. Say we have historical weather predictions from three different sources. All ...
0
votes
0answers
9 views

Common data from multiple file [closed]

My question is pretty simple, I have multiple files that contains sevral numbers (int), In order to speed up some processing (Sub string search) I would like to somehow summarize the common character ...
0
votes
0answers
3 views

Game Data Analysis (Stats)

Analyzing some game data and wanted some help to know how a specific column affects the game economy dynamics. Game Scenario: • This game data comes from a video game in which players earn items ...
0
votes
2answers
24 views

Predicting missing data. Looking for good data predicting technique

I am analysing data for Countries Trade GDP. Some of the countries have missing GDP value for given a year. However, I have Grand Total for the entire region for that year. Is there a good data ...
0
votes
1answer
42 views

Will I be able to fetch a Data Analyst job if I learn everything in this image?

India Asking such a question because I am interested learn everything myself and I am confused about what to read. And I am interested by reading books, will that suffice? or should I compulsorily ...
0
votes
0answers
34 views

Finding FAQs from dataset with its other attributes from other dataset

I'm new to Data Science and want to know the procedure that I should follow to tackle the following problem : I've been given 2 datasets. The first dataset has 2 columns : ...
1
vote
0answers
8 views

Finding change maximum change in the value using Redshift

Following is the problem I want to solve. But I don't know how to implement it. I am using Redshift to store data. Following is the format of the data stored in Redshift. It is sales history for ...
0
votes
2answers
39 views

Building predictive model with low correlated data [closed]

I have been working on a project with low features and only few entry fields ( 4 to be exact ). All the data in the dataset is barely correlated to each other. Is there some organized way or ...
1
vote
1answer
20 views

How to make a Power Spectral Density Plot in R

I have a time series point process representing neuron spikes. I have computed and plotted autocovariance using acf but now I need to plot the Power Spectral ...
2
votes
0answers
22 views

Dimension of the manifold on which my data sits

Suppose that I have data points, in the form of vectors with binary entries. We create a metric space, or Vietoris-Rips complex, using the Hamming distance between the data points. I would like to ...
1
vote
0answers
12 views

How to get the survival duration prediction for each individual in the data by using the Kaplan-Meier method?

I am trying to learn how to use the Kaplan-Meier survival estimator model in the lifelines package. The documentation says that the KaplanMeierFitter.fit function ...
0
votes
1answer
47 views

Sparse Representation vs Dense Representation

The one of the benefits of ReLUs is sparsity. Sparsity arises when a≤0 (a = wX+b). The more such units that exist in a layer the more sparse the resulting representation. Sigmoids on the other hand ...
1
vote
1answer
73 views

SVM model classifying into one class only, after standardization

I'm trying to use SVM in R (e1071 package) to classify samples as normal or tumor. I have two separate data sets - Training (~50 samples, 100 features) and Test (~60 samples). These data sets are ...
1
vote
1answer
19 views

Understanding a dataset (prior to applying ML models) with no metadata given

How do you understand a dataset when there is no metadata given (no details about the attributes given in the dataset)? It is difficult to comprehend the attribute names as only the short forms are ...
1
vote
2answers
34 views

What kind of machine learning can predict a probability rather than classification?

For example, I want to predict a probability of whether raining tomorrow, and I can't get the prob. of cloudy or sunny...etc day. I only get events(attributes of weather) like temperature, humidity, ...
0
votes
0answers
17 views

EDA : Model decision making

I am working on this dataset. Completely new to the machine learning. It seems to be a time series data. 1) Can I go for regression or classification modelling for this dataset? The target id in ...
1
vote
1answer
14 views

Different types of analysis which can be done with Webdata of users

I have rich customer data for site usage, particularly the web analytics data. Some of the data fields I have are: Product Usage details Geographic details Session data: Capture what all they did ...
0
votes
1answer
62 views

Exploratory Data Analysis

I am working on this dataset. Dataset has missing values. What would be the best method to impute the missing values. Also values are missing from target feature as well. So far I have dropped those ...
0
votes
1answer
23 views

which forecasting models could be chosen?

I'm new for data analysis. I got some data from the regional environmental center. Measurements: Datetime, PointID, SubstanceID, Value (substances concentrations in air), MeteoID ,NextValue ...
1
vote
0answers
21 views

How to measure retention statistics?

I have a dataset with the ID, name, joining date, leaving date as features. I was asked to measure employee retention and health of it. What can I derive from these? What are some latest trends and ...
0
votes
1answer
24 views

data analysis EDA issues, indent or type

I am a newbie in DS world, right now i am working on some EDA practice, and run into an issue here. Here is my code: Convert some columns to numeric values ...
7
votes
1answer
131 views

How far can one go with excel? [closed]

in my business we handle all analytics through Excel. This includes mostly scheduling, production planning and accounting operations. We currently are looking into adding a bit of predictive modelling ...
1
vote
1answer
14 views

Metrics/Methods for deciding duration of video retention for on-demand websites

This might be a general question but I thought this might be the best place to brainstorm. If I had a video website that only wants to retain videos on-demand for a certain number of days before ...
1
vote
0answers
12 views

What is the difference between 'if the data if of good quality' and 'if the data is tidy'?

I'm doing Data Analyst nanodegree from udacity. I'm confused between the difference even after going through the lecture a few times.
0
votes
0answers
13 views

How to calculate time between 2 actions on different tables, also considering some of start of stop actions are missing?

I'm collecting data from a production line and there are sensors at the beginning and the end. Very simple, start sensor and finish sensor send timestamps whenever they detect a product on them. ...
1
vote
0answers
26 views

Checking My Data Analysis Work [closed]

I am self studying Data Analysis, and I need expert opinions to correct my work. Having a mentoring help is the best option for my case, but I cannot afford it at this stage.. I finished a small ...
1
vote
0answers
112 views

Extract title from xml file using python [closed]

Extract title from xml file using python
0
votes
0answers
33 views

fill missing values

I have the following imdb dataset https://github.com/RUG-IDS/Course-2017/blob/master/Assignments/Assignment%201/movievalue.csv I want to clean the dataset and fill the missing values, but I noticed ...
0
votes
0answers
57 views

What really does a Data Scientist? [duplicate]

I'm asking that, because after going to a presentation I understood, that Data Scientist is practically someone who only makes calculations and statistics, neither program or very little. Can someone ...
0
votes
1answer
116 views

Anomaly Detection for Large Time Series Data [closed]

I am working on detecting anomalies within a large time series data set. It is updated on a regular basis and consists of more than 30 parameters. I am using R as a reference language. It is a first ...
1
vote
1answer
71 views

Predicting Customer Activity Absence

Could you please assist me with to following question? I have a customer activity dataframe that looks like this: It contains at least 500.000 customers and a "timeseries" of 42 months. The ones and ...
0
votes
3answers
45 views

Matching new set of data with pre-defined sets

I have sets of data describing sets of levels of requirements needed for certain sets of tasks. The following is a tabulated example: Note that the data values are on a scale from 0 to 10. My ...
1
vote
1answer
36 views

Correlate an array of categorical features to binary outcome

I have a data set that looks like this: target,items 1,[i1,i3] 1,[i4,i5,i9] 0,[i1] ... The variable target is 0-1 outcome. The feature "items" is a set of items (...
0
votes
1answer
30 views

Devise a single metric to measure retention of listeners using the service on a daily basis for the last month

I have dummy data for a music streaming service, the data is as follows:- Monthly uniques = the number of unique listeners on each platform for the entirety of a given month All-time uniques = the ...
0
votes
4answers
69 views

Tools to explore various datasets

TL:DR - Do you know of good automated tools to explore a dataset? Long version: I have a few different datasets to work with from most various areas of business. I wonder if there are good software/...
3
votes
0answers
41 views

Pandas: Calculating the best seller to incorporate number of houses sold

I have been analysing seller data and trying to get insights. I have written a groupby statement to get the average price of selling for every seller ...
1
vote
0answers
9 views

Numbers of participants in crowdsourcing tasks

Within the recent years, leveraging crowdsourcing for performing, mainly intelligent-intensive, tasks has become prevalent. An important question in this regard, that I could not find any specific ...
-2
votes
1answer
38 views

How to improve when starting out in data science [closed]

A little bit of background info,I recently started a new job as a data analyst where I have to use data to provide insights on the IT operations and devops process of my company. I’m fresh out of ...
0
votes
2answers
221 views

Data cleaning and data transformation before EDA?

What is the most common order of data cleaning, data transformation and exploratory data analysis? For me it seems most logical to do data cleaning, then EDA and finally data transformation (encoding ...
0
votes
2answers
93 views

Are Hadoop and Python SciPy used for the same? [closed]

Right now I've been doing a bit of research because I'm quite new to Big Data world. Among several other tools or frameworks, I've read about Apache Hadoop and Python for data analysis. Specifically,...
0
votes
0answers
148 views

Audio Analysis: How to identify whether a speaker have finished speaking ?

Just like Siri on Apple phone, I am trying to detect when the speaker has completed speaking. Once I identify that I will convert that to speech to text to understand the question and reply to guy ...
1
vote
1answer
23 views

Visualizing software metrics

I have the below sets of data per application, you can call them as software metrics. These metrics vary depending on the size of an application. ...
0
votes
0answers
33 views

EDA Learning Resources

Are there any good resources to familiarize one with fundamentals of EDA and best practices? I understand that the topic is very vast and there might not be one reference fits all type of answer. ...
4
votes
3answers
2k views

What is the difference between data analysis and machine learning?

It seems to me that machine learning (especially deep learning) can work with thousands (even millions) of different inputs. After training an ML model on the inputs, the computer has hopefully "...