Questions tagged [dataset]

A dataset is a collection of data, often in tabular or matrix form. This tag is NOT intended for data requests ("where can I find a dataset about ...") --> see OpenData

Filter by
Sorted by
Tagged with
0
votes
0answers
13 views

Standard for aligning data to week based on Mon to Sun for forecasting

I need to produce a forecast where i have predicted volume to a standard definition of a week. This definition is simply the week starts on Monday and Ends on Sunday. Is there an ISO standard that ...
0
votes
0answers
9 views

Shuffle dataset of sequences for Restricted Boltzmann Machines

Suppose I want to train an RBM (or even a DBN architecture) and then fine-tune the parameter training a Feedforward NN. In my case, the dataset is composed of time series, so in principle there is a ...
1
vote
3answers
74 views

Test data significantly different from training data

What are the risks if the test data is significantly different from the training data? Is the most significant problem related to both?
1
vote
2answers
30 views

How can I measure if a population has the same distribution as other?

Is there a population similarity index of some kind which could help me determine if two populations in two different datasets are the same or at least similar? The datasets have the exact same ...
0
votes
0answers
636 views

How to resolve IndexError: too many indices for array

I am attempting to load my dataset of images using the following code. ...
2
votes
2answers
99 views

Manually creating plants images dataset for machine learning plant type classification and leaf segmentation

A group I work with wants to create its own plants data set that will be used for multiple projects like plant type classification and leaf segmentation for starters. They are willing to provide all ...
0
votes
2answers
30 views

What are some good machine learning techniques for developing rules that govern changes in data?

Say I read a set of data in and it looks like so: [ 1, 2, 3, 4, 5 ] Now, there's an unknown rule set by my data's source that says if the 3rd element in my list ...
1
vote
2answers
46 views

What exactly is A dataset? is a database contain more than 2 tables counted as 1 or 2 datasets?

A data set (or dataset) is a collection of data. Most commonly a data set corresponds to the contents of a single database table, or a single statistical data matrix, where every column of the table ...
1
vote
3answers
56 views

What is the correct procedure when “joining” data takes ~6 hours?

I am dealing with bike-share data. I have 2 DataFrames: trips_df (subset shown), total entries = ...
1
vote
1answer
112 views

How to download BAIR action free robot pushing dataset?

I'm trying to download BAIR action free robot pushing dataset. I tried downloading from here. In browser, it shows its size is 30GB, but downloads some data and ...
1
vote
0answers
12 views

How to download BibTex raw dataset for deep learning (LSTM )?

I am trying to test my model on the benchmark text classification dataset "Bibtex". The Bibtex dataset is available in tf-idf vectors but LSTM works on sequences. Bag of Words is a vectorized ...
0
votes
0answers
28 views

ML with imbalanced binary dataset

I have a problem I am trying to solve: - imbalanced dataset with 2 classes - one class dwarfs the other one (923 vs 38) - f1_macro score when the dataset is used as-is to train RandomForestClassifier ...
3
votes
0answers
110 views

How do I implement masking in TensorFlow eager execution?

I am training a stateful RNN on variable length sequences (optional: see my previous question for more details). I padded the sequences to a fixed length with the value -1. The when batches are ...
1
vote
0answers
155 views

Using TF Dataset API to process sequences for stateful RNN

I am trying to use the TensorFlow (v1.13) Dataset API to save and load long sequences for a stateful RNN. Basically, lets say I have n_seq sequences, each fixed ...
0
votes
1answer
675 views

How to label a dataset for Machine Learning?

I have a collection of educational dataset. The dataset consists of a username and their review for the course. I want to analyze the data for sentiment analysis. How can I label the data to train ...
0
votes
0answers
14 views

Using fastai library to get data from Google

So I was doing the fastai online course and I have a doubt in lecture 2 (link for the code given below). First of all, when we are using Google to generate the dataset, where have we ensured that the ...
0
votes
1answer
51 views

Data set of vectors of SVG paths for digits

I have used the MNIST data set many times to train models for digit recognition based on object character recognition (OCR). I am now trying to do the same but with a data set of svg paths.. I am ...
0
votes
0answers
20 views

Car Insurance Policy tips

I'm pretty new to Data Science and I'm studying how car insurances work for my next task at work. I'd like to know if it's possible to calculate the best price given a dataset with quote prices and ...
5
votes
1answer
41 views

Data enrichment of geographical records

I have a user_data table with various fields, some of them are based on geography. I'd like to enrich the data with additional columns, like ...
3
votes
1answer
932 views

Splitting train/test sets by an identifier?

I know sklearn has train_test_split() to split a train and test set. But I read that, even with setting a random seed, if your actual dataset is updated regularly, ...
0
votes
0answers
7 views

What data formats/pipelining are best to store and wrangle data which contains both text and float vectors?

Often in NLP project the data points contain both text and float embeddings, and it's very tricky to deal with. CSVs take up a ton of memory and are slow to load. But most the other data formats seem ...
0
votes
0answers
13 views

Determine learned topics in text

I have a large number of texts (each about 1000 words). Every text contains a various number of topics, usually expressed as a sentence. I've extracted and categorize 22 topics and found 5000 of them ...
2
votes
0answers
21 views

Fuzzily join two large sets of postal addresses

I have two tables of postal address information - the one is about 2 million records, the other roughly 40 million. They have quite bad quality, and also are not quite compatible with each other (...
0
votes
0answers
14 views

how to classify whole sets data instead of single points

I have many datasets containing these features (characterization transistor): ['VG', 'VD', 'VS', 'ID', 'IS', 'IG', 'absID', 'sqrtID', 'gm2', 'absIG'] I want to divide Good working transistors from ...
0
votes
2answers
613 views

Smaller test data set than training data set in machine learning

I would like to train different machine learning algorithms (SVM, Random Forest, CNN etc.) for the same data set (e.g. MNIST) und then compare their accuracies. The goal would be to find out from ...
0
votes
1answer
69 views

how to draw a negative exponential plot of this situation with python

I have a dataset with binary output ($Y$) and I have a column (Duration) contains the duration of each task that is stored by "days" and varied from 1day to 350 days. when I think logically in our ...
1
vote
2answers
161 views

Time series normalization using min max technique

I have a time series dataset and I would like to normalize the data (diff which is of type list) as below using Min Max ...
3
votes
1answer
39 views

What happens to the left over unpicked data in Random Forest

I believe in Random forest we pick random samples of training data with replacement. My question is there still is a possibility that we might leave some data out. What happens to that. Does it not ...
1
vote
2answers
70 views

Selection of Features and Data in Random Forest

First, I am confused whether at each node in all the trees, do we randomly pick features from the lot to be pitted for best split or does each tree get a random subset of feature and then all the ...
1
vote
0answers
34 views

Keras Binary Classification val_acc won't go past ~67; Full data and code included

I'm working on a binary classification in Keras with a Tensorflow backend. No matter how much I tweak, I can't seem to get my model past a val_acc of 67%. Is there something I'm missing, or is this ...
0
votes
1answer
24 views

Balanced vs total dataset rows, which one is better?

I work on a dataset concerning games playing results. i.e every child play an indefinite number of games and it has as output (y) two possible values "success" or "Failure". It's about 800 000 ...
0
votes
1answer
74 views

Convert a repetitive list into Pandas Dataframe [closed]

How do you convert something like this: A: 1 B: 2 C: 3 ### A: 5 B: 5 C: 6 ### A: 2 B: 5 C: 7 into a dataset where the first row would be the first section with ...
3
votes
3answers
235 views

What's the point of the test set?

I get the point of a validation and training set, but the importance of a test set doesn't click for me. Let's say you train a model and you try your best to avoid overfitting by testing your model ...
7
votes
2answers
410 views

how to check the distribution of the training set and testing set are similar

I have been playing the Kaggle Competition and I find there is a situation that the distribution of the training set and testing set are different, so I am wondering how to check the distribution of ...
1
vote
1answer
72 views

Is this a data issue, or a model issue? A Keras binary classification model

I've been trying to create a binary classification model that predicts wether there will be a train delay based on the train and time. Here is a link to the data The issue I'm having is that my ...
1
vote
2answers
37 views

Extracting data from a data frame with conditions

Given the first data frame consisting of 4 columns ...
4
votes
1answer
276 views

Training a classifier when some of the features are unknown

I am training a classifier in Matlab with a dataset that I created. Unfortunately some of the features in the dataset were not recorded. I currently have the unknown features set as -99999. So, for ...
1
vote
1answer
52 views

How to make this plot with python

I want to draw plot by python based on this data ...
1
vote
1answer
257 views

Anyone have fruit disease dataset? [closed]

I am doing a project on fruit disease recognition and classification. Anyone have an existing dataset of fruit diseases? Can you help me to find one?
0
votes
0answers
13 views

Convert binary outcome to multiple categorical outcome

I have a dataset containing binary labels Y with values "correct" or "wrong". I want to study the reason of wrong state, based on several conditions made from multiple features. For example I want ...
1
vote
1answer
10 views

Text Dataset Help: Need to figure out how to get the publication date of a text

I am a highschool student working on a science fair project in which me and a friend plan to use a neural network for a classification problem. In this case the thing classified will be text and the ...
1
vote
1answer
16 views

Machine Learning Dataset: Easy enough for fully connected, but not easy enough for logistic regression

I was wondering if someone could direct me to a dataset for a classification task with the following conditions: Multinomial logistic regression alone does not learn a good classifier A series of ...
0
votes
2answers
31 views

What fits in a Data Description Report/ Data Exploration Report?

So I am trying to get familiar with Crisp-DM and found the terms "Data Description Report" and "Data Exploration Report", which seem oddly vague in their definition. So far I only found this right ...
0
votes
0answers
12 views

Estimate the effects of product components on the price

I am studying a specific problem but I think to question is of more general form: I have a dataset of different models of a product. All the models have the same components but every component of a ...
0
votes
0answers
60 views

(Almost) linearely separable dataset, where can I find one?

I'm implementing the perceptron algorithm and the voted perceptron algorithm for an assignment for university. For that I need to find some decent datasets.. I've tried the UCI repos and I've come up ...
0
votes
2answers
85 views

Keras Classifier returns similar output for all Predictions

I completed training the model with an accuracy of 1.000 and a validation accuracy of 0.9565. Unfortunately whenever i input a image into my model i get the same output regardless. Am i doing ...
0
votes
1answer
330 views

ValueError: Error when checking input: expected conv2d_13_input to have shape (3, 150, 150) but got array with shape (150, 150, 3)

I am trying to train the model, I keep ending up with this ValueError: ValueError: Error when checking input: expected conv2d_13_input to have shape (3, 150, 150) but got array with shape (150, 150,...
0
votes
1answer
88 views

How can I detect patterns and/or keywords or phrases?

I am collecting data in a database via php from apache. I am interested in detecting patterns in each column for now. For example manual examination of the data shows the pattern phpmyadmin is ...
0
votes
0answers
18 views

Data set descriptions for frequent item-set mining data set

Recently I started working on Frequent Itemset Mining and came across this link (http://fimi.uantwerpen.be/data/) hosting popular data sets that many research articles used to evaluate their itemset ...