Questions tagged [dataset]

A dataset is a collection of data, often in tabular or matrix form. This tag is NOT intended for data requests ("where can I find a dataset about ...") --> see OpenData

Filter by
Sorted by
Tagged with
0
votes
0answers
4 views

How to extract data from SEER Software? [closed]

I'm using SEER software and want to extract data about breast cancer. There is so many data sets that I don't know about them, Is there some on can help me on this. This software is so confusing and ...
1
vote
0answers
9 views

How to speed up propensity matching (matchit) in R?

My dataset has 7 columns(patient level data columns being age,gender etc and a target flag). My set has around 12 Million rows around 16K targets out of the same. I'm trying to match similar patients ...
0
votes
1answer
11 views

How to reshape ndarray to calculate the mean squared error?

y_train.shape = (7654,1) y_test.shape = (1914,1) mae = mean_absolute_error(y_train, y_test) gives: "ValueError: Found input variables with inconsistent numbers ...
1
vote
0answers
6 views

What are the standard parameters values for SMOTE technique?

I'm working on an imbalanced class data set (200 samples) with 2 classes, first class has 50 sample and second has 150 sample. My questions: When I use SMOTE technique on my data set my total ...
0
votes
1answer
12 views

Is it possible to change test and train data size when using crossvalind function with Kfold param?

I was looking at MATLAB Help and want to work with "crossvalind" function. It would two parameters that you can use it. If you use "HoldOut" you can define partition size of test and train data set ...
0
votes
0answers
14 views

Learning on data with different periodicity

Say I have two tabular data sources, one with yearly data, one with monthly data. Each data set has instances with unique identifier, and associated variables for each periods. I also have a monthly ...
4
votes
1answer
31 views

What are some (non image) datasets where deep networks are required for good performance?

It is well known that deep networks tend to outperform shallow networks and other classical machine learning techniques such as boosting on learning tasks involving images. I believe this is because ...
0
votes
2answers
20 views

How to apply data binning on reviews data? [closed]

I need to apply data binning on a set of reviews, I have searched for some data binning methods for reviews and long-texts and couldn't find anything other than classification. Is NLP or ...
1
vote
0answers
8 views

Why crossvalind and cvpartition results are different?

I did a Test on my data set with "cvpartition" and "crossvalind" functions (in MATLAB) with the same parameters and data set, but why the error results are different and which one is correct? Codes ...
0
votes
0answers
14 views

Best way to build a huge courses catalog [closed]

I am trying to build a dataset with courses from several sources: Udemy, Coursera, Udacity, Pluralsight, etc The Dataset should contain this fields, among others: ...
0
votes
0answers
17 views

How to do normalize distribution technique on a simple matrix?

This is just a practice about normal distribution in MATLAB and not a real project. I created a simple matrix and want to test normal distribution on it, it's ...
0
votes
0answers
12 views

Population Data

I'm trying to creating a heatmap using Esri Arcgis but need to find a data set of the population within a 1-mile and 3-mile radius. Does anyone know if there is a free data set that I could use? ...
0
votes
1answer
18 views

How to set and check normal distribution on a data set?

Sorry if my question is simple, I have a data set with two class and want to check and set normal distribution on it(if it was necessary, in MATLAB). But the question is that I should use it on every ...
1
vote
0answers
14 views

Bayesian network in Python: both construction and sampling

For a project, I need to create synthetic categorical data containing specific dependencies between the attributes. This can be done by sampling from a pre-defined Bayesian Network. After some ...
0
votes
1answer
26 views

How much is the Class Imbalance Problem rates?

I'm working on a data set and wanted to know is there a standard rate about Class Imbalance problem or not? I have 47 samples in Class A and 150 Sample in class B , should I use Class Imbalance ...
0
votes
0answers
10 views

How to best use Large images in training set for deep learning

I would like to ask you about how I should deal with the images I have. They are really large. They have this shape: (3000, 4000, 3). I'm working on a multilabel classification model. And I want to ...
0
votes
0answers
8 views

Load in many files from google drive into colab

I am trying to load in 30k images (600mb) from Google drive into Google Colaboratory to further process them with Keras/PyTorch. Therefore I have first mounted my Google drive using: ...
1
vote
1answer
26 views

Datasets for Topic Modeling [closed]

I'm looking to try and use deep learning methods for topic modeling as opposed to the more traditional methods of lda and word embedding methods. However, I'm having trouble finding good labeled ...
0
votes
0answers
6 views

Is there a dataset that contains images of talking people with information about their mouth state ( open, closed, teeth showing, etc.)?

Is there a dataset that contains images of talking people with information about their mouth state ( open, closed, teeth showing, etc.)? I would like to classify images based on whether or not a ...
0
votes
1answer
35 views

How to scale a variable when not knowing the maximum

I have a dataset with different features where some of them are not categorical, so they need to be scaled or normalized (especially the target). However, normalizing between 0-1 for instance means ...
-1
votes
2answers
60 views

What is the approx minimum size of dataset required to build 90% correct model?

I am working with a financial dataset size which is around 3000. I have attempted the supervised-learning regression techniques and not able to go beyond 70% accuracy. Features: 10 Data size:3700 ...
0
votes
1answer
7 views

Face dataset organized by folder [closed]

I'm looking for a quite little/medium dataset (from 50MB to 500MB) that contains photos of famous people organized by folder. The tree structure have to bee something like this: ...
1
vote
0answers
35 views

How to deal with large datasets?

I have some experience with data science but I wanted some insight on how to deal with a very large dataset. I understand simply downloading it to your computer is not plausible so where do you even ...
0
votes
0answers
10 views

Wavelet transform to identify 'meaningful' data changes

Can I use the wavelet transform to find the 'meaningful' changes in a series of 1d data? For example I have a series of integers representing angles of line segments. The majority of these values are ...
0
votes
1answer
37 views

Which Machine Learning methods would be good for a beginner-level Data Science project for described dataset? [closed]

I want to do my first beginner-level project regarding Data Science. I picked my data set which is described below. I want to make a Python script which will make some predictions on our data set ...
0
votes
2answers
58 views

How to handle addresses of the restaurants to feed the data-set in the ML model?

I have data from different restaurants which have also address of the restaurants now I want to predict the food delivery timing based on the given data, now the restaurant address is one of the ...
0
votes
0answers
18 views

Audio dataset preprocessing to perform cry detection

I am building a neural network to perform cry detection (i.e., binary classification of cry/non-cry situations) when capturing sound in a house environment. To do so, I performed the following steps: ...
2
votes
1answer
41 views

Could I add a one hot encoding to each feature representing “has data” versus “has no data”

I have a data set that has some holes in it. I was wondering if I could add two columns for each feature representing this feature has data and ...
0
votes
1answer
26 views

The distribution of dataset train and test are the differents, how to fix this?

I am new in data science and like some help to understand my problem. For instance, I have two signals non-stationary for the same condition (figure 1). I acquisition them at different times(in the ...
0
votes
0answers
13 views

Anonymizing the CSV data using Python Faker

To protect the sensitive customer data, I came across a useful python project faker. However, this project only generates the fake values after random iterations. Is it possible to anonymize the ...
0
votes
1answer
25 views

Stata: Trying to merge two data sets with common year observations

I am currently trying to merge two datasets with a common "year" variable using Stata. One dataset is purely for incorporating CPI with an associated year, and so the only variables are "year" and "...
0
votes
1answer
15 views

How much of a disadvantage is a small sample size?

I am examining a petition involving all UK constituencies. In this dataset 2 of the 632 constituencies have not participated in the petition - in terms of data quality how does this affect my ...
0
votes
0answers
12 views

How to make a heatmap using keypoints from MPII data set

So I am tiring to figure this out but keep running into a wall. So what I want to do is create a heatmap from a 2D image based on the key points generated from the MPII dataset. I have found a CSV ...
0
votes
0answers
12 views

Space and time fourier transform of data

I have discrete data for function $H(x,y,t)$ in the interval $0<x<x_0$, $0<y<y_0$ and $0<t<t_0$. I'm wondering if there is a way to find the function in Fourier space $H_f(q_x,q_y,w)$...
1
vote
0answers
13 views

Finding frequency as a function of wavevector ( Fourier Space)

I have data for a function $G(x,y,t)$ where $x,y$ show space position (2D) and $t$ is time. I know from Fourier transformation definition that: $G(x,y,t)=\int q \:dq \: d\omega \: d\theta\: \hat{G}(q,...
0
votes
0answers
9 views

What units are depth prediction dataset such as kitti or nyu in? km? meters?

What units are depth prediction dataset such as kitti or nyu in? km? meters? In general, when running a depth estimation network, do we pass in ground truth as km or m or some other more normalized ...
0
votes
0answers
4 views

Common text data sets in form of panel data [migrated]

I want to test machine learning tasks on time-divided textual data set. For this purpose, I want to use a common text data set which is already validated and "good" for use. I already found a Web of ...
1
vote
0answers
20 views

Column of Lists into Regex vs. Tuple

I'm working on preparing/analysing this dataset of Russian propaganda ads, in which there are columns which contain lists. 'interests_categories', for example, has a row which contains ['Unknown', '...
0
votes
0answers
11 views

Free historical news data base?

I spent some time looking for news databases and APIs. I can find a lot of paid ones. The data google provides when searching for news is perfect for what I am after (something with a deep history, ...
1
vote
3answers
28 views

How to extract insights from the given data?

Ok, I have this data with 3 columns, unique id, raw text, and review text. My task is play with the dataset and find meaningful insights from it. Raw text is in plain English but review text is in ...
0
votes
0answers
18 views

Is there a technique to calculate a monthly median from a known weekly median?

I currently have a dataset that conveys the financial details for each constituency in the UK. I wish to create monthly reports for income however the only associated value is "WeeklyMedian" - is ...
3
votes
2answers
214 views

Need machine learning algorithm to fill in time-series data

I am currently dealing with a time-series data set with cyclical gaps every 30 minutes (30 minutes of data, 30 minutes of no data). Is there a relatively simple way of using scikit-learn (or some ...
0
votes
1answer
37 views

Sampling labeled data for anomaly detection

I'm currently working on a project that requires the use of unsupervised anomaly detection, but I'm unable to find a relavent data set, so I'm considering the following option: Assuming I have a data ...
2
votes
1answer
34 views

Dataset from sequence of messages

I have a sorted dataset by datestamp which looks like this: ...
1
vote
1answer
22 views

How to represent a dataset as a linked list-like graph?

I would like to visualize this data set using Python and probably a visualization tool like Matplotlib. The data set contains three columns: a user id with a question, a user id with an answer, and ...
0
votes
2answers
20 views

What's the meaning of precomputed features?

When i learn about deep learning, I found dataset with precomputed features form. Link (http://cs.stanford.edu/people/karpathy/deepimagesent/coco.zip). What's the different with usual dataset?
1
vote
3answers
109 views

How to anonymize (de-identify) data in Python?

I have tried a simple algorithm to anonymize the data using the de-identification technique. But the code doesn't work for me. I want to anonymize the data by slightly changing the values of strings ...
0
votes
0answers
11 views

How can I use MIT CBCL dataset in python

I have downloaded the MIT CBCL face dataset, but I can't find any code online on how to use it in python. the MIT data set is in two formats .greynorm and .pgm format which I don't know how to use any ...
0
votes
0answers
7 views

How to write custom de-identification algorithm in Python?

I have tried a simple algorithm to anonymize my data using the de-identification technique. But the code doesn't work for me. I want to anonymize the data by slightly changing the values. The data ...