We’re rewarding the question askers & reputations are being recalculated! Read more.

Questions tagged [dataset]

A dataset is a collection of data, often in tabular or matrix form. This tag is NOT intended for data requests ("where can I find a dataset about ...") --> see OpenData

Filter by
Sorted by
Tagged with
0
votes
0answers
4 views

Space and time fourier transform of data

I have discrete data for function $H(x,y,t)$ in the interval $<0x<x0$,$0<y<y0$ and $0<t<t0$. I'm wondering if there is a way to find the function in Fourier space $H_f(q_x,q_y,w)$?
1
vote
0answers
10 views

Finding frequency as a function of wavevector ( Fourier Space)

I have data for a function $G(x,y,t)$ where $x,y$ show space position (2D) and $t$ is time. I know from Fourier transformation definition that: $G(x,y,t)=\int q \:dq \: d\omega \: d\theta\: \hat{G}(q,...
0
votes
0answers
6 views

What units are depth prediction dataset such as kitti or nyu in? km? meters?

What units are depth prediction dataset such as kitti or nyu in? km? meters? In general, when running a depth estimation network, do we pass in ground truth as km or m or some other more normalized ...
0
votes
0answers
4 views

Common text data sets in form of panel data [migrated]

I want to test machine learning tasks on time-divided textual data set. For this purpose, I want to use a common text data set which is already validated and "good" for use. I already found a Web of ...
1
vote
0answers
19 views

Column of Lists into Regex vs. Tuple

I'm working on preparing/analysing this dataset of Russian propaganda ads, in which there are columns which contain lists. 'interests_categories', for example, has a row which contains ['Unknown', '...
0
votes
0answers
11 views

Free historical news data base?

I spent some time looking for news databases and APIs. I can find a lot of paid ones. The data google provides when searching for news is perfect for what I am after (something with a deep history, ...
1
vote
3answers
28 views

How to extract insights from the given data?

Ok, I have this data with 3 columns, unique id, raw text, and review text. My task is play with the dataset and find meaningful insights from it. Raw text is in plain English but review text is in ...
0
votes
0answers
17 views

Is there a technique to calculate a monthly median from a known weekly median?

I currently have a dataset that conveys the financial details for each constituency in the UK. I wish to create monthly reports for income however the only associated value is "WeeklyMedian" - is ...
3
votes
2answers
194 views

Need machine learning algorithm to fill in time-series data

I am currently dealing with a time-series data set with cyclical gaps every 30 minutes (30 minutes of data, 30 minutes of no data). Is there a relatively simple way of using scikit-learn (or some ...
0
votes
1answer
20 views

Sampling labeled data for anomaly detection

I'm currently working on a project that requires the use of unsupervised anomaly detection, but I'm unable to find a relavent data set, so I'm considering the following option: Assuming I have a data ...
2
votes
1answer
30 views

Dataset from sequence of messages

I have a sorted dataset by datestamp which looks like this: ...
-1
votes
0answers
33 views

Approximating standard error of future data using training data [closed]

If we are calculating the standard error of a subset (assume 4000 validation labels out of 40000 total). Why don't we get reliable information on the position of the tumor in future data? The ...
1
vote
1answer
20 views

How to represent a dataset as a linked list-like graph?

I would like to visualize this data set using Python and probably a visualization tool like Matplotlib. The data set contains three columns: a user id with a question, a user id with an answer, and ...
0
votes
2answers
15 views

What's the meaning of precomputed features?

When i learn about deep learning, I found dataset with precomputed features form. Link (http://cs.stanford.edu/people/karpathy/deepimagesent/coco.zip). What's the different with usual dataset?
1
vote
3answers
61 views

How to anonymize (de-identify) data in Python?

I have tried a simple algorithm to anonymize the data using the de-identification technique. But the code doesn't work for me. I want to anonymize the data by slightly changing the values of strings ...
0
votes
0answers
10 views

How can I use MIT CBCL dataset in python

I have downloaded the MIT CBCL face dataset, but I can't find any code online on how to use it in python. the MIT data set is in two formats .greynorm and .pgm format which I don't know how to use any ...
0
votes
0answers
7 views

How to write custom de-identification algorithm in Python?

I have tried a simple algorithm to anonymize my data using the de-identification technique. But the code doesn't work for me. I want to anonymize the data by slightly changing the values. The data ...
0
votes
1answer
46 views

How to save API data into CSV format?

Is there an easy way to convert API data with python commands and output the results in CSV format? Here's what I'm doing now: ...
1
vote
1answer
24 views

Why do train, test, validation datasets need to have the same distribution?

I've found a lot of martial on how to deal with differently distributed train/test/validation data sets, however I'm struggling to find out why they they need to have the same distribution. Can ...
1
vote
1answer
31 views

MemoryError when concatenating a large data-frame

I am creating a tool that reads CSV fields and allows the user to specify columns they want to categorize and then categorize those columns. My issue is that these CSV files are quite large and when ...
0
votes
0answers
18 views

How to avoid dataset shift or covariate shift? [duplicate]

I wonder If there can occur a dataset shift or covariate shift through the following train_test_split-method? ...
1
vote
1answer
23 views

What dataset was Stanford NER trained on?

I would like to re-train the Stanford NER library from scratch as a 1 class model. Only 3,4 and 7 class models are available out of the box. Is it possible to obtain the data that the model was ...
0
votes
0answers
15 views

What's the size of Google's complete Conceptual Captions image captioning dataset with all the images downloaded from the listed URLs?

The original dataset provided by Google, here, consists of 'Image URL - Caption' pairs in both the provided training and validation sets. I have to work on an image captioning project and wanted to ...
0
votes
0answers
11 views

How to get a sample/summarised data from a table for data analysis?

I am currently working on EHR data analysis. I have a measurement table which has information lab test information of the patients. However the record count is huge. These are table count in my EHR ...
0
votes
0answers
6 views

Dataset for Title Detection and Hierarchization

I am working on a project which deals with detecting Headings in a PDF and developing a hierarchy out of documents(similar to Table Of Contents Generation).I have read some related papers(1,2).But ...
1
vote
0answers
16 views

How can I measure data anonymity when present summary statistics

I am looking to present users with aggregate statistics of a highly sensitive underlying dataset. See example data below: ...
0
votes
1answer
18 views

Recommendations to generate a training set of images

We are about to start a series of experiments in physics which will generate 8.5 thousands of images per month. This is a study on microscopic material reactions so the images are not 'normal' objects ...
1
vote
1answer
137 views

can I use z-score normalization even if it doesn't make sense for my data to be negative?

I'm planning to use z-score as a Normalization Method for a Project but I noticed if I do that then I ll have a data in the range [-1, 1] which is wierd because I have data that doesn't make sense for ...
0
votes
1answer
27 views

Is it possible to identify what data a neural net was trained on? [closed]

For example, if there are 5 major data sets for a certain topic. Would it be possible to see if a deployed network was trained off of one them? How would one approach this?
0
votes
1answer
16 views

Dataset : Handling rows where some column value is not applicable

I am working on some LTE Data say of shape (100000, 10) . Now suppose I have a column Jitter Avg. Now Jitter occurs only for any connected call and for call which could not be setup due to some ...
1
vote
1answer
30 views

K-fold-cross-validation if training dataset is much smaller than test dataset?

I'm a beginner in machine learning and I have a special case in which I have only a small training dataset of about 500 images and a test dataset of 10,000 images. Does it still make sense to do a 10-...
0
votes
0answers
20 views

How can I build a dataset with a few numerical attributes and an image attribute

I'm gathering data to make a dataset but I got a problem in this case I want to build a dataset containing height, age, sex and a picture from bones but I don't know in which format and how I can do ...
0
votes
1answer
27 views

Best way for correlation analysis

I am looking for an alternative way of analysing my data which will be used for ML. I am using Matlab to implement my code. Using my code, I obtain arrays which are used to describe signals. Each ...
0
votes
0answers
62 views

Building a speech commands dataset for audio recognition applications

I'm working on a DL project to recognize (10 - 15) Arabic speech commands from a continuous stream of audio, and I want to create a dataset similar to Google's Speech Commands dataset. Fortunately, I ...
-1
votes
1answer
20 views

How to plot datasets 1 factors for K mean clustering python

I'm unable to plot the data for K mean clusering algo usingsklearn as it throws this error : TypeError: scatter() missing 1 required positional argument: 'y' Here is the function I have written to ...
0
votes
1answer
12 views

Should I have “normal” sampled data in my dataset?

I am busy working on a project to find the reasons why kids in normal households are doing badly in school. I have a dataset of which consists of kids that live in environments where the family is ...
2
votes
2answers
247 views

Advice and Ideas appreciated - Machine Learning one man project

I have a project where I am supposed to start from scratch and learn how machine Learning works. So far everything is working out better than expected but I feel as I am offered to many ways to choose ...
0
votes
2answers
26 views

How do i find the most frequently occuring category for each unique customer?

I have a dataset with 30k unique users with each customer having multiple transaction. I aggregated the dataset with one record per user and containing aggregated information such as average ...
1
vote
2answers
39 views

Enormous dataset, how to proceed?

So I have started my master thesis and I have been handed a time series dataset with 2000 rows and nearly 600 columns of data. I have dealt with time series before, but nowhere similar to this level ...
0
votes
0answers
5 views

Training data for Tesseract

I have to train my tesseract to detect different variations of a letter for example, u,û,ü,ù should all mean u. Is this possible and if yes how should I train it and how should the dataset be made ...
0
votes
1answer
31 views

Split tuples with labeled samples in training, validation and test sets

I was reading through all the internet and i can't find nothing similar what i am looking for, i only saw this topic for pd.DataFrame, np.ndarray and list datasets but i didn't find nothing explaining ...
0
votes
1answer
18 views

appending file text into list with file name in python

Here 20_newsgroups is a folder which contain 20 folder and each folder contain some file , i just want to make datasets containind words and file name its gives Error ...
0
votes
0answers
18 views

How to avoid different accuracies when training with subsets?

when trying to train a CNN with randomly selected small subsets (each same size) of the training data set, I get different results in accuracy (the accuracy varies from 0.75 to 0.85). I determine the ...
1
vote
3answers
63 views

Using Google Translate API to create a Translation Dataset

Is it a good idea? ;-) Is it legal to do so? Is it legal to release such a dataset to public? Say I have a language X for which I want to create a dataset for translation to/from English, for which I ...
0
votes
0answers
8 views

pattern recognition tool

From what i've seen while searching google and the site this is most likely way simpler than what normally get's asked but i've not been able to find anything that could help me with it. What i'm ...
0
votes
0answers
26 views

Dataset Image creation suggestions

I am trying to create a dataset where the Text is mixed, e.g. " I love football" can be written as " I l()v3 F00tba11". The idea is that by using Tesseract I can find the pattern to match these two ...
1
vote
1answer
126 views

Machine learning method to predict event date

Let's say I have a big dataset consisting of variables including but not limited to the start/end date of loans, their notional amount, a loan prepayment indicator etc. My goal is to create a model ...
0
votes
0answers
26 views

dataset shift, covariate shift or sample selection bias in small subsets?

I try to show from which training data size on which machine learning method (CNN, SVM etc.) achieves better performance. For this I would like to use subsets of different sizes from the datasets of ...
0
votes
1answer
20 views

How to Proceed with Tokenized text content which is given in number?

I have one data set of customer review, but the text data is given is tokenized text number. I am unable to proceed thinking about how to proceed? As I am encountering such data set the first time, ...
0
votes
1answer
30 views

How to Avoid rarely used discrete feature values in a dataset

On Google's ML crash course it states: Good feature values should appear more than 5 or so times in a data set. Doing so enables a model to learn how this feature value relates to the label. ...