Questions tagged [dataset]

A dataset is a collection of data, often in tabular or matrix form. This tag is NOT intended for data requests ("where can I find a dataset about ...") --> see OpenData

Filter by
Sorted by
Tagged with
2
votes
0answers
5 views

What are some (non image) datasets where deep networks are required for good performance?

It is well known that deep networks tend to outperform shallow networks and other classical machine learning techniques such as boosting on learning tasks involving images. I believe this is because ...
0
votes
1answer
241 views

Equivalence of Tidy Data and Third Normal Form

In Hadley Wickham's "Tidy Data" paper, he states that In tidy data: Each variable forms a column. Each observation forms a row. Each type of observational unit forms a table. ...
0
votes
1answer
23 views

Sampling labeled data for anomaly detection

I'm currently working on a project that requires the use of unsupervised anomaly detection, but I'm unable to find a relavent data set, so I'm considering the following option: Assuming I have a data ...
3
votes
1answer
163 views

Shifted feature distribution across different datasets

I am trying to validate a classifier using two different training and testing datasets. The feature I am considering is a feature constructed doing the fold-change between two original features, i.e. ...
0
votes
2answers
16 views

How to apply data binning on reviews data? [closed]

I need to apply data binning on a set of reviews, I have searched for some data binning methods for reviews and long-texts and couldn't find anything other than classification. Is NLP or ...
1
vote
0answers
8 views

Why crossvalind and cvpartition results are different?

I did a Test on my data set with "cvpartition" and "crossvalind" functions (in MATLAB) with the same parameters and data set, but why the error results are different and which one is correct? Codes ...
0
votes
0answers
13 views

Best way to build a huge courses catalog [closed]

I am trying to build a dataset with courses from several sources: Udemy, Coursera, Udacity, Pluralsight, etc The Dataset should contain this fields, among others: ...
2
votes
2answers
129 views

World's Wine's Dataset

I'm conducting a machine learning project and therefore looking for a world's wine's dataset to create a classification model based on user's ratings. Can someone recommend me any datasets? Haven't ...
1
vote
1answer
319 views

tensorflow categorical data with vocabulary list - Expected binary or Unicode string, got [0,1,2,…]

I'm brand new to machine learning (having just completed the google machine learning crash course) and thought it would be good to try my hand at a Kaggle competition as a good starter to some real ...
0
votes
0answers
13 views

How to do normalize distribution technique on a simple matrix?

This is just a practice about normal distribution in MATLAB and not a real project. I created a simple matrix and want to test normal distribution on it, it's ...
0
votes
0answers
10 views

Population Data

I'm trying to creating a heatmap using Esri Arcgis but need to find a data set of the population within a 1-mile and 3-mile radius. Does anyone know if there is a free data set that I could use? ...
0
votes
2answers
27 views

How do i find the most frequently occuring category for each unique customer?

I have a dataset with 30k unique users with each customer having multiple transaction. I aggregated the dataset with one record per user and containing aggregated information such as average ...
0
votes
1answer
17 views

How to set and check normal distribution on a data set?

Sorry if my question is simple, I have a data set with two class and want to check and set normal distribution on it(if it was necessary, in MATLAB). But the question is that I should use it on every ...
2
votes
1answer
88 views

Dataset where svm performance is significantly different from random forest

Is there a specific dataset where svm performs significantly better or worse than random forest? I know that the performance could depend on the dataset but is there a specific dataset?
9
votes
4answers
12k views

One hot encoding alternatives for large categorical values?

Hi have dataframe with large categorical values over 1600 categories is there any way I can find alternatives so that I don't have over 1600 columns. I found this below interesting link http://...
1
vote
0answers
14 views

Bayesian network in Python: both construction and sampling

For a project, I need to create synthetic categorical data containing specific dependencies between the attributes. This can be done by sampling from a pre-defined Bayesian Network. After some ...
0
votes
1answer
18 views

Is there any data set containing list of windows software and the category each one of them belongs to?

I wanted a dataset which lists many windows softwares according to their categories for a project. eg: anaconda: software development, roblox: gaming, VLC: video player etc. I searched but could not ...
1
vote
4answers
220 views

Which type of model to use and what's my target variable

I am a beginner in Data Science field, so sorry if my question is too basic. The task is to build an ad bidding model for online marketing which allows you to deliver targeted ads to the right ...
0
votes
1answer
7 views

Face dataset organized by folder [closed]

I'm looking for a quite little/medium dataset (from 50MB to 500MB) that contains photos of famous people organized by folder. The tree structure have to bee something like this: ...
0
votes
1answer
26 views

How much is the Class Imbalance Problem rates?

I'm working on a data set and wanted to know is there a standard rate about Class Imbalance problem or not? I have 47 samples in Class A and 150 Sample in class B , should I use Class Imbalance ...
3
votes
1answer
50 views

features importance

Suppose I have dataset labeled with two classes such as healthy and unhealthy and I applied feature selection (features importance)on dataset. How can I know that features are important to which ...
0
votes
0answers
10 views

How to best use Large images in training set for deep learning

I would like to ask you about how I should deal with the images I have. They are really large. They have this shape: (3000, 4000, 3). I'm working on a multilabel classification model. And I want to ...
0
votes
0answers
5 views

Load in many files from google drive into colab

I am trying to load in 30k images (600mb) from Google drive into Google Colaboratory to further process them with Keras/PyTorch. Therefore I have first mounted my Google drive using: ...
1
vote
1answer
26 views

Datasets for Topic Modeling [closed]

I'm looking to try and use deep learning methods for topic modeling as opposed to the more traditional methods of lda and word embedding methods. However, I'm having trouble finding good labeled ...
2
votes
1answer
93 views

Reinforcement Learning on real time data over a web server

Question: is it possible to implement a reinforcement learning model over a NodeJS server? This server would be receiving binary forms of data (open /close; yes/no) in real time. The objective for ...
0
votes
1answer
35 views

How to scale a variable when not knowing the maximum

I have a dataset with different features where some of them are not categorical, so they need to be scaled or normalized (especially the target). However, normalizing between 0-1 for instance means ...
0
votes
0answers
6 views

Is there a dataset that contains images of talking people with information about their mouth state ( open, closed, teeth showing, etc.)?

Is there a dataset that contains images of talking people with information about their mouth state ( open, closed, teeth showing, etc.)? I would like to classify images based on whether or not a ...
-1
votes
2answers
56 views

What is the approx minimum size of dataset required to build 90% correct model?

I am working with a financial dataset size which is around 3000. I have attempted the supervised-learning regression techniques and not able to go beyond 70% accuracy. Features: 10 Data size:3700 ...
0
votes
2answers
458 views

Prediction after one hot encoding

I have a regression model that I want to make prediction based on values that I will get from an end user. In my dataset, I have one categorical variable region ...
6
votes
4answers
759 views

Working with inaccurate (incorrect) dataset

This is my problem description: "According to the Survey on Household Income and Wealth, we need to find out the top 10% households with the most income and expenditures. However, we know that these ...
1
vote
2answers
160 views

Time series normalization using min max technique

I have a time series dataset and I would like to normalize the data (diff which is of type list) as below using Min Max ...
0
votes
1answer
50 views

How to approach the PlantVillage dataset?

I'm working on the PlantVillage dataset and i want to predict the type of the disease from the image of a leaf. The dataset is labeled in pairs (Type of the plant,Healthy/name of the disease). I'm ...
1
vote
1answer
62 views

How to split a dataset into train and test sets for time series (multiple step-multiple output forecasting)?

I am trying to use an LSTM neural net to do multiple step / multiple output forecasting (I predict multiple values in one time knowing some values in the past). But, I have realized that I must be ...
0
votes
1answer
37 views

Which Machine Learning methods would be good for a beginner-level Data Science project for described dataset? [closed]

I want to do my first beginner-level project regarding Data Science. I picked my data set which is described below. I want to make a Python script which will make some predictions on our data set ...
1
vote
0answers
35 views

How to deal with large datasets?

I have some experience with data science but I wanted some insight on how to deal with a very large dataset. I understand simply downloading it to your computer is not plausible so where do you even ...
0
votes
0answers
10 views

Wavelet transform to identify 'meaningful' data changes

Can I use the wavelet transform to find the 'meaningful' changes in a series of 1d data? For example I have a series of integers representing angles of line segments. The majority of these values are ...
0
votes
2answers
56 views

How to handle addresses of the restaurants to feed the data-set in the ML model?

I have data from different restaurants which have also address of the restaurants now I want to predict the food delivery timing based on the given data, now the restaurant address is one of the ...
2
votes
1answer
40 views

Could I add a one hot encoding to each feature representing “has data” versus “has no data”

I have a data set that has some holes in it. I was wondering if I could add two columns for each feature representing this feature has data and ...
3
votes
3answers
710 views

Where can I find datasets with large number of features (100000) but less samples(100-1000)?

I am working on a dimensionality reduction algorithm for datasets with high feature to sample ratio. I need datasets with around 100000 features or more and 100-1000 samples with class labels to ...
2
votes
0answers
21 views

Fuzzily join two large sets of postal addresses

I have two tables of postal address information - the one is about 2 million records, the other roughly 40 million. They have quite bad quality, and also are not quite compatible with each other (...
0
votes
0answers
17 views

Audio dataset preprocessing to perform cry detection

I am building a neural network to perform cry detection (i.e., binary classification of cry/non-cry situations) when capturing sound in a house environment. To do so, I performed the following steps: ...
1
vote
1answer
54 views

If the entire production dataset is known is advance, should I use it when fitting normalization parameters?

I found in multiple sources the recommendation not to fit the normalization parameters on the combined train/test dataset when evaluating the model, to prevent data leakage. I presume this ...
0
votes
1answer
25 views

The distribution of dataset train and test are the differents, how to fix this?

I am new in data science and like some help to understand my problem. For instance, I have two signals non-stationary for the same condition (figure 1). I acquisition them at different times(in the ...
0
votes
0answers
11 views

Anonymizing the CSV data using Python Faker

To protect the sensitive customer data, I came across a useful python project faker. However, this project only generates the fake values after random iterations. Is it possible to anonymize the ...
1
vote
1answer
71 views

Converting VOC2012 annotation .xml to segmentation .png

Does anyone know a good way to convert VOC2012 annotations files in xml to SegmentationClass ...
0
votes
2answers
91 views

Classification Dataset which is linearly non separable

I am trying to find a dataset which is linearly non-separable. I checked the Iris dataset and the UCI website says: The data set contains 3 classes of 50 instances each, where each class refers to ...
0
votes
1answer
25 views

Stata: Trying to merge two data sets with common year observations

I am currently trying to merge two datasets with a common "year" variable using Stata. One dataset is purely for incorporating CPI with an associated year, and so the only variables are "year" and "...
0
votes
1answer
15 views

How much of a disadvantage is a small sample size?

I am examining a petition involving all UK constituencies. In this dataset 2 of the 632 constituencies have not participated in the petition - in terms of data quality how does this affect my ...
1
vote
2answers
97 views

Pre-processing on MRI images

I have MRI images of brain tumors collected from a hospital (not a benchmark dataset). And I am planning to use them to predict/classify tumour types using a typical machine learning approach: texture ...
0
votes
0answers
12 views

Space and time fourier transform of data

I have discrete data for function $H(x,y,t)$ in the interval $0<x<x_0$, $0<y<y_0$ and $0<t<t_0$. I'm wondering if there is a way to find the function in Fourier space $H_f(q_x,q_y,w)$...