Questions tagged [dataset]

A dataset is a collection of data, often in tabular or matrix form. This tag is NOT intended for data requests ("where can I find a dataset about ...") --> see OpenData

Filter by
Sorted by
Tagged with
3
votes
1answer
1k views

What are examples for XOR, parity and multiplexer problems in decision tree learning?

In scikit-learn documentation and in decision tree learning Wikipedia article there is mention of "There are concepts that are hard to learn because decision trees do not express them easily, such as ...
0
votes
1answer
26 views

Looking for advice: data transfer

I'm requesting data from a government body, and they asked me what format I want to receive the data in. This will be a table of about 400,000 rows and about 10 columns. My options are: "comma or tab ...
1
vote
1answer
298 views

Find intersection of data between rows and columns

Below is an Excel sheet with crops and countries. "1" means that the crop is grown in that country whiles "0" means it is not grown in that country. I want to find where there is an intersection (as ...
1
vote
1answer
75 views

Yelp dataset needed [closed]

I am writing a paper on aspect based sentiment analysis and I need datasets appropriate having only positive and negative classes to compare my result to other papers. Two of them are below which have ...
0
votes
1answer
1k views

Find-out abnormal behavior over the time

I want to detect abnormal behaviour in a oil pipe where the oil is flowing with some constant pressure. I have a sensor which monitors and pressure over the time and push it to my cloud server. I got ...
14
votes
3answers
8k views

Is there a person class in ImageNet? Are there any classes related to humans?

If I look at one of the many sources for the Imagenet classes on the Internet I cannot find a single class related to human beings (and no, harvestman is not someone who harvests, but it's what I knew ...
3
votes
1answer
727 views

Loss function and data distribution for positive / negative examples

I have an AlexNet architecture that predicts in a supervised-way, whether a given image belongs into classes 1-10. But it should also recognize if it doesn't fit into any of these classes. My thought ...
3
votes
1answer
460 views

What is the allowable limit of oversampling?

Suppose I have 2 classes. One class has 16 samples and the other class has 435 samples. Is it justified to oversample the class with 16 sample to have a 435 number of samples? Or is it better to ...
0
votes
1answer
31 views

How can I download data from online table quickly? [closed]

I am trying to do download all the marathon results from 1996 to 2016 from the following link. There are more than 400000 entries, but I can only show 1000 results per page. Is there any way that ...
1
vote
1answer
139 views

Building a data frame and matrix for citation network

I have a question about how to create a matrix that will take the lists defined below and make them the entries for the columns and rows. My data involves two lists: one lists originals sources (or ...
1
vote
1answer
61 views

I only have microsoft office, how can i write code to manipulate relational data sets?

At work, I am in a role that does not have a lot of computer permissions. I do have the Microsoft office suite. I have skills with programming and would like to be able to use my experience to do more ...
3
votes
1answer
2k views

Reinforcement Learning on data only (NO emulators)

My team and I started digging into RL for the purpose of a specific application. We have plenty of data of an agent carrying out suboptimal policies (states and rewards...). It is too costly for us ...
3
votes
1answer
89 views

What is the appropriate name for this dataset?

I'll assume most of us know about cross-validation and are accustomed to using Training and Test sets. However, I've also seen references to a third set that you use to test a completed model. It's ...
1
vote
3answers
2k views

Creating data model out of .csv file using Python

I want to create a data model out of a .csv file using Python. I mean to create dependencies, for example the primary key and stuff such that I can check if the new ...
1
vote
0answers
65 views

Initialize a recommender system with no dataset

Consider a platform for content recommendation based on the user history. The contents are books and articles and by history I mean what the user has read, what he has shared and so on. I know that ...
1
vote
1answer
109 views

Dataset of extremely low-dimensional images for PCA

I am looking for a public data-set of images that differ from each other only slightly, so that after applying PCA they can be reconstructed with a small error from ...
5
votes
1answer
326 views

Should I prevent augmented data to leak to the test/cross validation sets

I have been working with the cats vs dogs dataset from kaggle which consist on 25000 images of cats and dogs labelled accordingly (btw, great dataset, totally recommended!) One of the things I did ...
4
votes
1answer
255 views

What does BNG stand for

When i look at the available datasets in https://www.openml.org i often see a BNG dataset with no further information about it. Can someone explane what BNG means in this context? I am especially ...
1
vote
1answer
39 views

Can't interpret the text information and ratings matrix imported to NN

I have a Recommender system which uses a Collaborative bayesian approach using pSDAE for recommending scientific articles from the Citeulike Dataset The text information (as input to pSDAE) is in the ...
2
votes
1answer
270 views

multi channel feature classification using deep feed forward neural network on tensorflow [closed]

Can any one suggest a dataset that contains multi channel/multi dimensional feature values that can be classified using a deep feed forward neural network? Please ask me if the question is not clear ...
2
votes
1answer
580 views

Dataset management: What are some strategies/solutions for efficiently storing datasets with their versions?

The problem: I've N classification models (independent), for each of these N models, I've different versions (eg: V0, V1, ..., Vfinal_production,Vexperimental). I'm looking for a way to store my ...
1
vote
0answers
21 views

Data set for Q & A system training [closed]

What is the standard (or just good) open data set to train a model for question answering? Ideally, not only in English. Thanks!
3
votes
4answers
2k views

rows to columns in data.table R (or Python)

This is something I can't achieve with the reshape2 library for R. I have the following data: ...
0
votes
1answer
2k views

Compound Annual Growth Rate Function in R Statistics

I need to calculate the compound annual growth rate for two periods of time: 2014 & 2009. I'm currently working with Excel, I have two columns with data for each year (One column with only data of ...
2
votes
1answer
113 views

Datasets in NLP research papers

I am looking for a dataset containing a large number of NLP research papers and abstracts. Are there any open access datasets like that? If so, can you please share the details?
1
vote
2answers
705 views
2
votes
1answer
58 views

Unknown return value in CNTK 103A

I came across this statement from CNTK 103A: gzfname, h = urlretrieve(src, './delete.me') I understand all the rest but that ...
1
vote
2answers
697 views

Cluster titles or ingredients of food into n-categories

I have a dataset which has information about food recipes (in german), that looks like this: Here is a link to a small .csv file (first 1000 rows of my data) https://drive.google.com/file/d/...
3
votes
1answer
2k views

Is it legal to scrape YouTube videos for training data? [closed]

Is it legal, at least as long as I am not "selling" the video under my name, to scrape YouTube videos to train a neural network? If it is not, is there a procedure to get permission for the above? I ...
4
votes
2answers
690 views

What does it mean for the training data to be generated by a probability distribution over datasets

I was reading the Deep Learning book and came across the following para (page 109, second para): The training and test data are generated by a probability distribution over datasets called the data-...
3
votes
1answer
16k views

Error 'Expected 2D array, got 1D array instead:'

While performing a simple fitting operation on the Titanic dataset. The following is my code: ...
1
vote
1answer
29 views

Should I modify dataset?

I am working on a dataset which contains more than 80 features, and thousands of instances. Among those features there are some nominal ones, such as IP SOURCE, IP DESTINATION, Flow ID, which don't ...
-1
votes
1answer
501 views

Age range prediction according to browsing history [closed]

I need to predict age or age range of a user according to his/her browsing history from a news website. The only data I am able to get is USER-ID, URLs-visited, TAGs from those URLs, "which category ...
-1
votes
1answer
122 views
1
vote
0answers
36 views

How to create a neural network with many different values in Matlab? [closed]

I want to create a neural network in Matlab. I have 16 import matrices in each iteration, my import data changes in every iteration and there are more than 100000 iterations. The target is a matrix ...
0
votes
1answer
305 views

How should classification be done for a very small data set?

I am looking at data from the London Data Store based on social characteristics between London boroughs. Since there are only about 30 London boroughs, the data sets I am looking at are naturally ...
0
votes
1answer
827 views

How to find the datasets for skill test (like java, python, c c++, etc.) [closed]

I want to extract the skills from a resume using Natural languge processing. To train my model I don't any dataset. where can I get a sample dataset for extracting skills from resume?
5
votes
1answer
3k views

How can l get 50 % examples in training set and 50% in test set for each class when splitting data?

l have a dataset of 200 examples with 10 classes. l would like to split the dataset into training set 50% and test set 50%. for each class, l have 20 examples. Hence, l would like to get for each ...
2
votes
0answers
255 views

Keras or TensorFlow Examples for Working with Large Text Datasets (~10M Sentences)

I'm looking for any tutorials or examples for using either Keras or TensorFlow on large text datasets (at least a few million sentences). I've looked at some of the examples using the ...
1
vote
1answer
119 views

Do you know a dataset for regression where deep learning outperforms svm and random forests? [closed]

I am currently trying to learn about deep learning. I asked myself where deep learning outperforms SVM and Random Forest on regression? Do you have any dataset for regression where deep learning ...
3
votes
2answers
2k views

How to deal with large data sets

So i'm very new to this, forgive my silly questions. I've got some data I need to analyse, I would usually just use excel it doesnt seem to be able to do the job. I have .csv files containing data ...
-1
votes
1answer
6k views

Error in f1(x) : argument “b” is missing, with no default

f1 <- function(a,b,c,d,e,f) { -111.605*a-208.39+(14.882-b)^2+35.29813*c-.001251205/d-1.050695*e+11.63420*f } ...
2
votes
1answer
70 views

Recombining rare values

I have a column called item_colour which describes the colour of products in my dataset. There are 85 colours and some of the colours are unique or they represent a small part of the data ( I have ...
2
votes
1answer
7k views

How Can I Compute Information-Gain for Continuous- Valued Attributes

Decision trees handle only discrete values, but the continuous values we need to transform to discrete. My question is HOW? I know the steps which are: Sort the value A in increasing order. Find ...
13
votes
4answers
15k views

One hot encoding alternatives for large categorical values?

Hi have dataframe with large categorical values over 1600 categories is there any way I can find alternatives so that I don't have over 1600 columns. I found this below interesting link http://...
1
vote
1answer
130 views

Detecting and confirming sentiments in chat messages

Say I have a spreadsheet of 1000 chat messages that are labeled as happy, sad or funny. What is the best way of detecting if the labels are indeed correct or if they need improvement? How would I go ...
12
votes
2answers
3k views

why we need to handle data imbalance?

I need to know why we need to deal with data imbalance. I know how to deal with it and different methods to solve the issue which is by up sampling or down sampling or by using Smote. For example, if ...
0
votes
1answer
638 views

Creating the optimal set of utterances to train a natural language processing engine

Context I am using natural language processing engines such as IBM's Watson Conversation and Microsoft's LUIS, which take a sentence and classify its intent. For example, "i want to buy food" => "...
2
votes
0answers
49 views

Are there deduplication algorithms that do not work on a metric space?

Recently I got interested in the process of data cleansing and specifically in record linkage. Thus far I read about deterministic and probabilistic approaches to deduplicate data sets and to some ...
1
vote
3answers
482 views

What are key dataset requirements for topic models and word embeddings?

I have a dataset of 2000 documents where avg doc size is 300 words. The vocab is dominated by domain-specific words. My goal is to find similar documents. For this, I tried LDA, LSI, Doc2Vec (topics=...

1
12 13
14
15 16
21