Questions tagged [data]

Questions mostly concerned with managing data, without focus on pre-processing or modelling.

Filter by
Sorted by
Tagged with
2
votes
0answers
52 views

Free/open interactive softwares/plugins for end-users' high-dimensional data visualization

Aside from questions about How to visualize data of a multidimensional dataset (TIMIT), the Purpose of visualizing high dimensional data? and High-dimensional data: What are useful techniques to know?,...
4
votes
3answers
2k views

What are the methods to ensure that the population split for A/B test is random?

Before launching an A/B test, what are the methods to ensure that the population split in control and target group is random for a particular label say, purchase rate.
5
votes
4answers
329 views

General way to reduce features

Let's say I have a giant dataset (600+ columns) and I have no idea what insights I might get from it or what model I want to run. What are some of the best ways to find the most influential columns/...
2
votes
0answers
63 views

Tag categorizer

I'm using Google prediction API and I need to perform tag categorization for free text. So when I receive a text like "Cant update application" is tagged with #Apps #General #CantUpdate. My ...
2
votes
0answers
69 views

Data Scientists | Sectors , Departments and Profiles [closed]

I'm searching a lot about what people call Data Scientist. From my brief search, I've came with the conclusion that Data Scientists' are probably what you may call the future evolution of a Business ...
2
votes
1answer
587 views

PCA on acceleration time series data

Has anyone attempted Principal component analysis on time series and acceleration data(ie. data from accelerometer and sensors) and tried compressing it as well as regenerating back the data with ...
2
votes
1answer
87 views

Spatial clustering based on response to inputs and building a reduced model

Stats version: I have a few measurements of a function that takes three inputs and produces a few 2D fields of outputs: $f_i(a,b,c;x,y)$, with $f$ being a vector of several quantities. I would like to ...
3
votes
4answers
4k views

Mathematics major for data science

So I'm a recent transfer 2nd year student from Computer Science major to Mathematics major. Though I do have a bit of an issue here. I can choose between the applied mathematics, pure mathematics and ...
-2
votes
1answer
1k views

create a company to make money by winning Kaggle competition [closed]

I am thinking of creating a data science company (team) that aims to make money by winning Kaggle competitions. It will be actually a onsite data science learning service but I want to create the ...
4
votes
1answer
46 views

How to deal with analyzing optional survey data

Consider this scenario. There is a website that ranks every type of sandwich. For each type of sandwich, users fill out a survey about the following: A) how tasty it is (on a scale from 1-5); B) how ...
3
votes
2answers
177 views

Domain-specific data science programs

I am looking for domain-specific data science programs (as a major, not a minor or specialization). I found programs in bio-statistics (and other disciplines in public health), MS in marketing ...
3
votes
0answers
315 views

Central Probability Interval

So, I'm having problems learning for a data analysis lecture. The topic is the construction of the "central interval" as our lecturer calls it. The concept shall path the way to understand confidence &...
6
votes
1answer
1k views

Do I have to standardize my new polynomial features?

I have a vector X with n features previously standardized. If I want to generate new polynomial features (let say adding square features), do I need to do another standardization on these new ...
1
vote
0answers
49 views

How to select a bunch of optimized data from a larger data set?

How to select a bunch of optimized data from a larger data set? I have a data set containing the products to be sold to the customers. And the products were sold before, it has some information, like ...
0
votes
1answer
60 views

how often should one sample a dataset?

I have some archive data that show the variation of a certain quantity X measured over 1-year intervals. I'm now facing the problem of understanding how X correlates with another variable Y, that I ...
2
votes
3answers
424 views

How to store complex tables and structures?

I'm wondering about general approaches to storing complex tables and structures. For example, imagine I have a table like this: ...
3
votes
1answer
63 views

Establishing data science programs as an independent discipline

What are the benefits of establishing data science programs as an independent discipline rather than creating data scientists in different fields? (for students, universities and the society as a ...
1
vote
2answers
193 views

Data Science education curriculum design and guidelines in Computer Science and other Disciplines

I am studying data science education curriculum design and guidelines. All resources I have up to now is about data science in statistics education and library and information science education. Is ...
1
vote
1answer
7k views

How to generate bootstrapping samples in R?

Here is my original dataset from a voting activity. Each participant voted for one option (A,B,C,D,E,F) listed below. The numerical value is the total number of participants who voted for each option. ...
1
vote
0answers
47 views

Generate a set of abstract search terms

I am developing an application for research purposes that uses randomly generated tweets to detect emerging conceptual topics. I need to collect a large amount of them (let's say, beyond 5 million) ...
1
vote
1answer
565 views

Data frame mutation in R

I have a data frame of the following format: Symbol Date Time Profit $BANKNIFTY 4/1/2010 9:55:00 -1.18% <br>$BANKNIFTY 4/1/2010 12:30:00 -2.84% $BANKNIFTY 4/1/2010 12:45:00 7.17% &...
4
votes
2answers
78 views

Finding aggregated information of data

I am new to data science. I have a dataset of around 200,000 records, having 5 columns. There is a field called, class. For each ...
2
votes
0answers
679 views

Weighted k nearest neighbor search

I've searched quite a bit and haven't landed on any useful results. The problem statement is: Given a set of vectors, I wish to find its approximate k-nearest neighbors. The caveat here is that each ...
3
votes
1answer
99 views

Advise on making predictions given collection of dimensions and corresponding probabilities

I am a CS graduate but am very new to data science. I could use some expert advise/insight on a problem I am trying to solve. I've been through the titanic tutorial on gaggle.com which I think was ...
5
votes
1answer
9k views

How to preprocess different kinds of data (continuous, discrete, categorical) before Decision Tree learning

I want to use some Decision Tree learning, such as the Random Forest classifier. I have data of different types: continuous, discrete and categorical. How do I have to preprocess data in order to ...
2
votes
0answers
50 views

What proxies could be used to assess economic value of Stackoverflow for its users?

I'm intrigued by the open data provided by StackExchange, and have been running some really interesting queries on the data.stackexchange.com page (using the Stackoverflow dataset). In particular, I ...
3
votes
1answer
273 views

python - Will this data mining approach work? Is it a good idea?

I need to extract fields like the document number, date, and invoice amount from a bunch of .csv files, which I believe are referred to as "unstructured text." I have some labeled input files and will ...
3
votes
2answers
136 views

What kind of research can be done with genomic data?

It's well known that science has given us large amount of free accessible data, such as http://www.1000genomes.org and http://www.ncbi.nlm.nih.gov/genbank. How can we play around with the data and ...