Questions tagged [dataset]

A dataset is a collection of data, often in tabular or matrix form. This tag is NOT intended for data requests ("where can I find a dataset about ...") --> see OpenData

Filter by
Sorted by
Tagged with
8
votes
3answers
948 views

Interactive Graphing while logging data

I'm looking to graph and interactively explore live/continuously measured data. There are quite a few options out there, with plot.ly being the most user-friendly. Plot.ly has a fantastic and easy to ...
1
vote
0answers
82 views

Cross-sell models and additional holders

I would like to pose a question about how to treat additional holders in the propensity-to-buy models of banking products. Up to now I was only taking into considerations the clients as first holders....
4
votes
2answers
553 views

API for Company Data Enrichment Suggestions

I'm looking for API suggestions for enriching data on companies. Currently I use the Crunchbase API to look up a company's name or domain and I am trying to gather the domain/name (if I don't already ...
6
votes
1answer
190 views

Evaluating Recommendation engines

What is the standard way for evaluating and comparing different algorithms while developing recommendation system? Whether we need to have a predetermined annotated ranked dataset and then compare ...
2
votes
1answer
253 views

Confused about description of YearPrediction Dataset

https://archive.ics.uci.edu/ml/datasets/YearPredictionMSD According to the description given in the above link, the Attribute information specifies "average and covariance over all 'segments', each ...
1
vote
2answers
232 views

DBPedia as Table not having all the properties

I browsed a sample for available data at http://dbpedia.org/page/Sachin_Tendulkar. I wanted these properties as columns, so I downloaded the CSV files from http://wiki.dbpedia.org/DBpediaAsTables. ...
2
votes
0answers
301 views

Creating Data model for mahout recommendation engine

I am trying to build an item-item similarity matching recommendation engine with mahout. The data set is as in the following format ( attributes are in text not in numerals format ) ...
3
votes
2answers
892 views

API for historical housing prices

I'm looking for an (ideally free) API that would have time series avg/median housing prices by zip code or city/state. Quandl almost fits the bill, but it returns inconsistent results across ...
4
votes
2answers
130 views

Reduction of multiple answers to single variable

The questionnaire for the data is here The first question takes multiple entry for the same question, I want to reduce this to a single variable. How do I do it? The clean data is available here. NB:...
2
votes
3answers
8k views

Large categorical dataset for regression

I need to collect several large datasets (thousands of samples, dozens of features) for regression with only categorical inputs. I already look for such datasets in the UCI repository, but I did not ...
6
votes
2answers
801 views

Matrix properties and machine learning/data mining

I'm doing some data analysis in a Statistical Pattern Recognition course using PRML. We analyzed a lot of matrix properties, like eigenvalues, column independence, positive semi-definite matrix, etc. ...
3
votes
1answer
773 views

MovieLens data set

I want to analyze MovieLens data set and load on my machine the M1 file. I combine actually two data files (ratings.dat and movies.dat) and sort the table according ...
4
votes
3answers
2k views

Single Layer Perceptron with three classes

I need some help with a single layered perceptron with multiple classes. What I need to do is classify a dataset with three different classes, by now I just learnt how to do it with two classes, so I ...
5
votes
2answers
121 views

Amalgamating multiple datasets with different variables coding

I have several datasets with thousands of variables. This different datasets have different variables for the same thing. Is there a way to automatically/semi-automatically check compatible variables ...
3
votes
3answers
10k views

How to classify and cluster this time series data

I have post already the question few months ago about my project that I'm starting to work on. This post can be see here: Human activity recognition using smartphone data set problem Now, I know ...
1
vote
1answer
61 views

Versatile data structure for combined statistics

Not sure if this is Math, Stats or Data Science, but I figured I would post it here to get the site used. As a programmer, when you have a system/component implemented, you might want to allow some ...
3
votes
1answer
102 views

Trouble representing a problem

I have a problem and I'm having trouble representing it - first I thought I should use graph theory (nodes and edges) and now I'm not sure. My data is some tanks names and it's volumes, those tanks ...
29
votes
4answers
12k views

Quick guide into training highly imbalanced data sets

I have a classification problem with approximately 1000 positive and 10000 negative samples in training set. So this data set is quite unbalanced. Plain random forest is just trying to mark all test ...
1
vote
0answers
30 views

Freebase Related Models

It may be unlikely that anyone knows this but I have a specific question about Freebase. Here is the Freebase page from the Ford Taurus automotive model . It has a property called "Related Models". ...
3
votes
1answer
3k views

Data sets for evaluating text retrieval quality [closed]

I'm searching for data sets for evaluating text retrieval quality. TF-IDF is a popular similarity measure, but is it the best choice? And which variant is the best choice? Lucenes Scoring for example ...
1
vote
0answers
25 views

Apps to manage/host data sets

I'm looking for the best solution to manage and host datasets for journalistic pursuits. I am assessing https://www.documentcloud.org and http://datahub.io/. Can anyone explain the differences ...
3
votes
1answer
212 views

Query similarity: how much data is used in practice?

I recently read Similarity Measures for Short Segments of Text (Metzler et al.). It describes basic methods for measuring query similarity, and in the paper, the data consists of queries and their ...
7
votes
5answers
4k views

Where can I find free spatio-temporal dataset for download?

Where can I find free spatio-temporal dataset for download so that I can play with it in R ?
1
vote
0answers
379 views

Can anyone provide the 24 hour challenge dataset from Climate Corporation?

I have no knowledge about the climate or soil. And I just want to find out more about these kind of dataset. I heard that Climate Corporation asked its candidates to perform statistical analysis on ...
1
vote
0answers
41 views

Subgraph isomorphism and Anti-monotone property

While finding frequent subgraphs in single large graph, subgraph isomorphism (test) is not considered because its not anti-monotone. How and why subgraph isomorphism is not anti-monotone ?
4
votes
4answers
287 views

Handling huge dataset imbalance (2 class values) and appropriate ML algorithm

I have train and test sets of chronological data consisting of 305000 instances and 70000,appropriately. There are 15 features in each instance and only 2 possible class values ( NEW,OLD). The problem ...
13
votes
2answers
2k views

Analyzing A/B test results which are not normally distributed, using independent t-test

I have a set of results from an A/B test (one control group, one feature group) which do not fit a Normal Distribution. In fact the distribution resembles more closely the Landau Distribution. I ...
3
votes
1answer
4k views

R aggregate() with dates

I am working on a data set that has multiple traffic speed measurements per day. My data is from the city of chicago, and it is taken every minute for about six months. I wanted to consolidate this ...
24
votes
3answers
42k views

Data Science Project Ideas [closed]

I don't know if this is a right place to ask this question, but a community dedicated to Data Science should be the most appropriate place in my opinion. I have just started with Data Science and ...
49
votes
5answers
11k views

Should I go for a 'balanced' dataset or a 'representative' dataset?

My 'machine learning' task is of separating benign Internet traffic from malicious traffic. In the real world scenario, most (say 90% or more) of Internet traffic is benign. Thus I felt that I should ...
8
votes
4answers
1k views

Job title similarity

I'm trying to define a metric between job titles in IT field. For this I need some metric between words of job titles that are not appearing together in the same job title, e.g. metric between the ...
9
votes
2answers
1k views

NASDAQ Trade Data

I am trying to find stock data to practice with, is there a good resource for this? I found this: ftp://emi.nasdaq.com/ITCH/ but it only has the current year. I already have a way of parsing the ...
2
votes
1answer
131 views

Getting GitHub repository information by different criteria

New to the Data Science forum, and first poster here! This may be kind of a specific question (hopefully not too much so), but one I'd imagine others might be interested in. I'm looking for a way to ...
18
votes
3answers
8k views

Dataset for Named Entity Recognition on Informal Text

I'm currently searching for labeled datasets to train a model to extract named entities from informal text (something similar to tweets). Because capitalization and grammar are often lacking in the ...
3
votes
1answer
942 views

Data preparation and machine learning algorithm for click prediction

I am new to machine learning. I have a task at hand of predicting click probability given user information like city, state, OS version, OS family, device, browser family, browser version, etc. I have ...
9
votes
3answers
647 views

Network analysis classic datasets

There are several classic datasets for machine learning classification/regression tasks. The most popular are: Iris Flower Data Set; Titanic Data Set; Motor Trend Cars; etc. But does anyone know ...
10
votes
3answers
14k views

Where can I download historical market capitalization and daily turnover data for stocks?

There are plenty of sources which provide the historical stock data but they only provide the OHLC fields along with volume and adjusted close. Also a couple of sources I found provide market cap data ...
13
votes
6answers
458 views

Datasets understanding best practices

I am a CS master student in data mining. My supervisor once told me that before I run any classifier or do anything with a dataset I must fully understand the data and make sure that the data is clean ...
1
vote
4answers
891 views

First steps on a new cleaned dataset

What is the very first thing you do when you get your hands on a new data set (assuming it is cleaned and well structured)? Please share sample code snippets as I am sure this would be extremely ...
5
votes
2answers
189 views

Techniques for trend extraction from unbalanced panel data

My data set is formatted like this: User-id | Threat_score aaa 45 bbb 32 ccc 20 The list contains the top 100 users with the highest threat ...
9
votes
4answers
3k views

Suggest text classifier training datasets

Which freely available datasets can I use to train a text classifier? We are trying to enhance our users engagement by recommending the most related content for him, so we thought If we classified ...
26
votes
7answers
25k views

Publicly available social network datasets/APIs

As an extension to our great list of publicly available datasets, I'd like to know if there is any list of publicly available social network datasets/crawling APIs. It would be very nice if alongside ...
173
votes
35answers
28k views

Publicly Available Datasets

One of the common problems in data science is gathering data from various sources in a somehow cleaned (semi-structured) format and combining metrics from various sources for making a higher level ...