Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [dataset]

A dataset is a collection of data, generally represented in tabular form, with columns signifying different variables and rows signify different members of the set.

0
votes
1answer
7 views

what is step by step if I want to search prediction from 4 datasets?

the fourth data is (train_data, test_data, previous_data, and information_history_data). The goals is searching user's rate on the loan to bank. I confused first step to doing because there many ...
0
votes
0answers
8 views

How to combine Keras fit_generator and tfrecord shards?

I am reading lots of article about benefits tfrecord, and how it overcome io bottleneck, but problem is i have large dataset and i cant load it all on once on RAM, so i must split my tfrecords into ...
0
votes
0answers
34 views

What's wrong with the following PYTHON code? [on hold]

I do not understand why this question is marked as irrelevant to data science? My data is stored in pandas data-frame in CSV format. Folium is used to visualized geographical data based on cleaned ...
0
votes
0answers
4 views

What is the difference between BAIR action free and action conditioned dataset?

The papers SV2P and SVAP talk about BAIR robot pushing dataset in 2 settings: action-free and action-conditioned. From the papers, I only understood that in action-conditioned setting, another action ...
0
votes
1answer
22 views

K-Fold and Random Subsampling (RSS) Dataset generation?

Let say if I have a large dataset of 300k instances with 200 features, I want to reduce its size. Can I apply K-Fold technique to the 200 features then the trimmed dataset are applied with RSS to trim ...
0
votes
1answer
13 views

How can I train a many-to-one RNN with an array of 2D matrices?

I have eye tracking data for every word of a novel. Features for every word is given separately. I want to take groups of 100 words to make a sample and then use each of these samples as a single ...
2
votes
1answer
22 views

How to Setup Multivariate Time Series Dataset for Classification

First post on StackExchange. I’m fairly new to ML, with about 1 year of experience so please pardon any ignorance or misuse of terms. I have a multivariate time series dataset where I would like to ...
0
votes
0answers
9 views

Standard for aligning data to week based on Mon to Sun for forecasting

I need to produce a forecast where i have predicted volume to a standard definition of a week. This definition is simply the week starts on Monday and Ends on Sunday. Is there an ISO standard that ...
0
votes
0answers
8 views

Shuffle dataset of sequences for Restricted Boltzmann Machines

Suppose I want to train an RBM (or even a DBN architecture) and then fine-tune the parameter training a Feedforward NN. In my case, the dataset is composed of time series, so in principle there is a ...
1
vote
3answers
50 views

Test data significantly different from training data

What are the risks if the test data is significantly different from the training data? Is the most significant problem related to both?
0
votes
0answers
15 views

very low val_acc in LSTM for predicting numbers in sequence

I have problem with my LSTM network or my data. I woudl like to create LSTM network to predict number based on previous N numbers. This numbers will be encoded piano notes. About my dataset: 1. I ...
1
vote
2answers
28 views

How can I measure if a population has the same distribution as other?

Is there a population similarity index of some kind which could help me determine if two populations in two different datasets are the same or at least similar? The datasets have the exact same ...
0
votes
0answers
18 views

How to resolve IndexError: too many indices for array

I am attempting to load my dataset of images using the following code. ...
0
votes
1answer
10 views

Manually creating plants images dataset for machine learning plant type classification and leaf segmentation

A group I work with wants to create its own plants data set that will be used for multiple projects like plant type classification and leaf segmentation for starters. They are willing to provide all ...
0
votes
2answers
24 views

What are some good machine learning techniques for developing rules that govern changes in data?

Say I read a set of data in and it looks like so: [ 1, 2, 3, 4, 5 ] Now, there's an unknown rule set by my data's source that says if the 3rd element in my list ...
0
votes
0answers
10 views

which license Lenna image is under? is it free to cite? if yes, how to cite this image? [closed]

per wiki, Lenna is the name given to a standard test image widely used in the field of image processing since 1973. which license Lenna image is under? is it free to cite? if yes, how to cite this ...
1
vote
2answers
40 views

What exactly is A dataset? is a database contain more than 2 tables counted as 1 or 2 datasets?

A data set (or dataset) is a collection of data. Most commonly a data set corresponds to the contents of a single database table, or a single statistical data matrix, where every column of the table ...
1
vote
3answers
41 views

What is the correct procedure when “joining” data takes ~6 hours?

I am dealing with bike-share data. I have 2 DataFrames: trips_df (subset shown), total entries = ...
0
votes
0answers
9 views

Where can I find image dataset for crop yield prediction? [migrated]

There's a research paper that I read and I want to reproduce their work for which I need the right dataset. link to the paper: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3821323/
1
vote
0answers
9 views

How to download BAIR action free robot pushing dataset?

I'm trying to download BAIR action free robot pushing dataset. I tried downloading from here. In browser, it shows its size is 30GB, but downloads some data and ...
1
vote
0answers
10 views

How to download BibTex raw dataset for deep learning (LSTM )?

I am trying to test my model on the benchmark text classification dataset "Bibtex". The Bibtex dataset is available in tf-idf vectors but LSTM works on sequences. Bag of Words is a vectorized ...
0
votes
0answers
16 views

ML with imbalanced binary dataset

I have a problem I am trying to solve: - imbalanced dataset with 2 classes - one class dwarfs the other one (923 vs 38) - f1_macro score when the dataset is used as-is to train RandomForestClassifier ...
3
votes
0answers
16 views

How do I implement masking in TensorFlow eager execution?

I am training a stateful RNN on variable length sequences (optional: see my previous question for more details). I padded the sequences to a fixed length with the value -1. The when batches are ...
1
vote
0answers
12 views

Using TF Dataset API to process sequences for stateful RNN

I am trying to use the TensorFlow (v1.13) Dataset API to save and load long sequences for a stateful RNN. Basically, lets say I have n_seq sequences, each fixed ...
0
votes
0answers
8 views

Movie plots and reviews dataset [migrated]

Is there any dataset with movie plots and reviews (not only rating)? I have had a look at the existing dataset, but they contain only one of the two (i.e. only plots or only reviews). I would be ...
0
votes
1answer
37 views

How to label a dataset for Machine Learning?

I have a collection of educational dataset. The dataset consists of a username and their review for the course. I want to analyze the data for sentiment analysis. How can I label the data to train ...
0
votes
0answers
11 views

Plant Disease Detection - Machine Learning [migrated]

Does anyone knows a good dataset used to detect disease in plants? I'm using PlantVillage but it seems that the pictures are laboratorial and because of that can't be used in a real situation. Thanks ...
0
votes
0answers
10 views

Using fastai library to get data from Google

So I was doing the fastai online course and I have a doubt in lecture 2 (link for the code given below). First of all, when we are using Google to generate the dataset, where have we ensured that the ...
0
votes
1answer
12 views

Data set of vectors of SVG paths for digits

I have used the MNIST data set many times to train models for digit recognition based on object character recognition (OCR). I am now trying to do the same but with a data set of svg paths.. I am ...
0
votes
0answers
16 views

Car Insurance Policy tips

I'm pretty new to Data Science and I'm studying how car insurances work for my next task at work. I'd like to know if it's possible to calculate the best price given a dataset with quote prices and ...
5
votes
1answer
35 views

Data enrichment of geographical records

I have a user_data table with various fields, some of them are based on geography. I'd like to enrich the data with additional columns, like ...
3
votes
1answer
59 views

Splitting train/test sets by an identifier?

I know sklearn has train_test_split() to split a train and test set. But I read that, even with setting a random seed, if your actual dataset is updated regularly, ...
0
votes
1answer
22 views

Dataset about flash floods

Where I can find spatial dataset about Flash floods with a set of parameters highly contributing to flash floods occurrence such as precipitation and DEM ?
0
votes
0answers
5 views

What data formats/pipelining are best to store and wrangle data which contains both text and float vectors?

Often in NLP project the data points contain both text and float embeddings, and it's very tricky to deal with. CSVs take up a ton of memory and are slow to load. But most the other data formats seem ...
0
votes
0answers
10 views

Determine learned topics in text

I have a large number of texts (each about 1000 words). Every text contains a various number of topics, usually expressed as a sentence. I've extracted and categorize 22 topics and found 5000 of them ...
0
votes
0answers
10 views

how to classify whole sets data instead of single points

I have many datasets containing these features (characterization transistor): ['VG', 'VD', 'VS', 'ID', 'IS', 'IG', 'absID', 'sqrtID', 'gm2', 'absIG'] I want to divide Good working transistors from ...
0
votes
2answers
47 views

Smaller test data set than training data set in machine learning

I would like to train different machine learning algorithms (SVM, Random Forest, CNN etc.) for the same data set (e.g. MNIST) und then compare their accuracies. The goal would be to find out from ...
0
votes
1answer
39 views

how to draw a negative exponential plot of this situation with python

I have a dataset with binary output ($Y$) and I have a column (Duration) contains the duration of each task that is stored by "days" and varied from 1day to 350 days. when I think logically in our ...
3
votes
1answer
37 views

What happens to the left over unpicked data in Random Forest

I believe in Random forest we pick random samples of training data with replacement. My question is there still is a possibility that we might leave some data out. What happens to that. Does it not ...
1
vote
2answers
42 views

Selection of Features and Data in Random Forest

First, I am confused whether at each node in all the trees, do we randomly pick features from the lot to be pitted for best split or does each tree get a random subset of feature and then all the ...
1
vote
0answers
23 views

Keras Binary Classification val_acc won't go past ~67; Full data and code included

I'm working on a binary classification in Keras with a Tensorflow backend. No matter how much I tweak, I can't seem to get my model past a val_acc of 67%. Is there something I'm missing, or is this ...
0
votes
1answer
23 views

Balanced vs total dataset rows, which one is better?

I work on a dataset concerning games playing results. i.e every child play an indefinite number of games and it has as output (y) two possible values "success" or "Failure". It's about 800 000 ...
0
votes
1answer
18 views

Convert a repetitive list into Pandas Dataframe [closed]

How do you convert something like this: A: 1 B: 2 C: 3 ### A: 5 B: 5 C: 6 ### A: 2 B: 5 C: 7 into a dataset where the first row would be the first section with ...
3
votes
3answers
196 views

What's the point of the test set?

I get the point of a validation and training set, but the importance of a test set doesn't click for me. Let's say you train a model and you try your best to avoid overfitting by testing your model ...
5
votes
2answers
70 views

how to check the distribution of the training set and testing set are similar

I have been playing the Kaggle Competition and I find there is a situation that the distribution of the training set and testing set are different, so I am wondering how to check the distribution of ...
1
vote
1answer
44 views

Is this a data issue, or a model issue? A Keras binary classification model

I've been trying to create a binary classification model that predicts wether there will be a train delay based on the train and time. Here is a link to the data The issue I'm having is that my ...
1
vote
2answers
36 views

Extracting data from a data frame with conditions

Given the first data frame consisting of 4 columns ...
4
votes
1answer
211 views

Training a classifier when some of the features are unknown

I am training a classifier in Matlab with a dataset that I created. Unfortunately some of the features in the dataset were not recorded. I currently have the unknown features set as -99999. So, for ...
1
vote
1answer
42 views

How to make this plot with python

I want to draw plot by python based on this data ...