Questions tagged [dataset]

A dataset is a collection of data, often in tabular or matrix form. This tag is NOT intended for data requests ("where can I find a dataset about ...") --> see OpenData

Filter by
Sorted by
Tagged with
0
votes
0answers
8 views

Free news APIs? [migrated]

Over the past few years, I have managed to find all sorts of excellent data sources across the web, almost always for free, including static data (e.g. .csv), but ...
0
votes
0answers
8 views

Where can I get an untokenized version of GLUE's SST-2 dataset?

On the GLUE faq, they say: Similarly, for SST, the data provided is already tokenized. We're working on obtaining a version that is not tokenized. Feel free to train on other distributions of ...
1
vote
3answers
33 views

The effect of imbalanced distribution of data

I read on Google's ML website if I have classification dataset with a ratio of 90% for one classification and 10% of the data for another classification. In that case, should I use the exact same ...
0
votes
1answer
24 views

Gradient Descent or Normal Equation?

Hi guys I am really struggling with this question. I need to pick the correct choice: Suppose you have a dataset with m = 50 examples and n = 15 features for each example. You want to use ...
0
votes
1answer
23 views

Intuition behind the PCA algorithm

I am trying to understand PCA intuitively. Here it goes: After finding the eigenvectors and eigenvalues of the covariance matrix of the dataset, the eigenvalues will represent how spread out the ...
0
votes
0answers
18 views

How to handle a data set with large number (about 75%) of binary variables?

I am doing a research right now and want to classify (predict) churns of costumers using machine learning. My data set consists of about 500,000 observations with 20 variables: 15 are binary, 2 ...
-2
votes
0answers
14 views

time series data modeling for deep learning [closed]

what is the best format to feed the input data, which are time series with varying density over time, to a deep learning network, while at any iteration we want to feed a batch of data including a ...
0
votes
0answers
11 views

Comparing two different datasets and updating the first dataset using python [closed]

I have two dataframes. Sample data is attached above. I have all the record in second dataframe , but in first dataframe i have pieces of records. How can i compare the first dataframe with second ...
1
vote
2answers
33 views

Wanting to overfit model to say if data has been seen before

I would like to train a scalable model that has as an input row of a database and has an output of either 1 or 0 depending upon whether it has seen this entry of the database before or not. The ...
-2
votes
0answers
17 views

Kaggle Lung Cancer Dataset MISSING (Data Science Bowl 2017)

For a couple of days now, I have been looking for the Kaggle Lung Cancer Dataset (Data Science Bowl 2017) containing 2D, 3D, and labels of CT scans for lung cancer, but I did not succeed yet. It is ...
0
votes
0answers
8 views

Alarms prediction. Data preparation stage. Am I thinking in the right way?

Intro I have the dataset that contains the next information: Node Name (contains over 300 unique names) Event Date and Time (spotty one-year data) Alarm (3 alarm types) Clearing Time (the date ...
0
votes
0answers
9 views

How to generate synthetic datasets using randi and rand?

I have real datasets of 3 years. Now, I want to generate more datasets based on the behavior of these datasets to test my algorithm. Is there any way to generate synthetic dataset using ...
0
votes
0answers
14 views

Finding features that influence net revenue

Using machine learning I would like to identify features that influence net revenue and make conclusions from data based on that. The data set is a car sharing ...
1
vote
0answers
16 views

Publicly available news APIs/datasets?

In addition to our list of publicly available datasets, I'd like to know if there is any list of publicly available news datasets/crawling APIs. It would be very nice if alongside with a link to the ...
1
vote
1answer
17 views

Where to find height dataset, or datasets in General

Hi there smart people, I am new to data Science and wanted to take my first few steps. Unfortunately I struggle to find datasets or any data at all regarding my topics of interest. For example, I ...
2
votes
1answer
40 views

Given a machine learning algorithm, what is the minimum size of the training set for it?

I understand that the more data we have, the more reliable is our model trained on that data. I also understand that the more parameters a machine learning model has, the more training data it ...
0
votes
0answers
14 views

How to convert a non gaussian distribution into a gaussian destribution?

Suppose I have a dataset inwhich there are few dimensions that distribution over them is non gaussian and this means, skewness is nonzero (possitive or negative). This is caused by some outliers in my ...
0
votes
0answers
13 views

How to compare two samples taken from the same dataset?

I have data from almost 10 different sensors and was trying to downsample the data. I tried the reservoir algorithm and the uniform sampling and the random sampling techniques. I was wondering how I ...
-2
votes
1answer
37 views

To build a machine learning model of large data sets of 20,000 rows using python [closed]

What are the essential steps to build a Machine Learning model to predict the following 4 classes: whether the user was a male, a female, a brand (non-individual) or unknown. The dataset contains 20,...
0
votes
1answer
23 views

CNN for subsets of a dataset - how to tune hyperparameters

I have a dataset and would like to train CNNs on subsets of different size of the dataset. I already have a CNN, which classifies very well if I use the entire dataset. Now the question arises if I ...
0
votes
1answer
13 views

How to split the data as per the label name count using sklearn

Before training a model, I would like to split the data in a 80/20 ratio. For example, in my dataset of 3000 rows, I have different labels. Out of which A is one label name, has 100 records in the ...
0
votes
0answers
10 views

Trouble working with a corpus

I'm quite new to the world of machine learning and I want to start experimenting on my own. So I wanna do some work related to comics and manga and I found an interesting article here where they say ...
0
votes
0answers
9 views

Context Dataset in Time Series Forecasting

In time series forecasting, what is context dataset? How different is it from the actual training dataset and what role does it play either in training or validation?
0
votes
0answers
4 views

Dataset Format - firstratedata [migrated]

I was looking at the data sets published in firstratedata. I noticed some deficiencies in formats. Does anybody has any guess about it ? BRENTCMDUSD There is one more column in the end. What does ...
0
votes
0answers
4 views
0
votes
0answers
18 views

How to prepare training data for deep learning models

I am working on a project which involves the application of deep learning models. I have collected training data. In collected images, I have more than one object in interest. I am not very clear how ...
0
votes
0answers
10 views

Study resources for data manipulation interview questions

A colleague of mine is studying for tech roles, and they're asked to solve a consistent type of problem during the phone screenings: practicing manipulating data (sets, hash tables/dictionaries, ...
0
votes
0answers
5 views

Is there an ISO or other standard or best practice for structuring records related to organisations?

I'm working on a project which includes redesigning the way an application represents data about organisations. The current design is quite "flat" - meaning for example that if I want to add a new ...
0
votes
1answer
30 views

CIFAR-100: What is the difference between vehicles 1 and vehicles 2?

The superclasses in the CIFAR-100 dataset are mutually exclusive and all but the vehicle ones are quite well defined by its label. Example: It is very clear why bees belong to the superclass ...
0
votes
2answers
28 views

Curation of a dataset of audio files in different formats

I have a large dataset of audio files (around 80k files of around 45 minutes long each) that are related to my work. Some of them are in format ".m4a" and others in format ".mp3". I am not a computer ...
0
votes
1answer
17 views

Is there any kaggle competition for finding the feature for affecting revenue? [closed]

I have a question about finding the most significant feature of the data that affects the company revenue. I have data set that contains car share company data (car model, completed trips, lead_time, ...
0
votes
1answer
16 views

Should inputs be shuffled for training a model with expected time dependency?

Consider a prediction model with numerical inputs and outputs. Suppose data is inserted tick by tick, i.e., when new data is available it is inserted asynchronously. Current output depends on current ...
0
votes
4answers
39 views

Analyzing country data. Does duplicating observations by population for regression make sense?

Question: While analyzing country happiness data via OLS regression, should I duplicate observations based on country population? Example: If duplicating per million, the U.S. would have 327 ...
0
votes
1answer
17 views

Explain forward filling and backward filling (data filling)

Can I understand in this way? Let me know if any statement is wrong or not accurate. Reason of data filling: Assume I have a consecutive data (e.g., daily log data), and partial data are missing. In ...
1
vote
0answers
29 views

Mobile App Recommendation: How to get the rate of a specific user submit for a specific application

I have a mobile app recommendation project, so I need data set which has user-app matrix-rate. Actually, I want to know what rate does a specific user submit for a specific application. in other words,...
1
vote
1answer
9 views

SMOTE for multi-instance learning i.e num_rows(x_train) > num_rows(y_train)

I have an imbalanced dataset and I wish to predict classes(0 or 1). Sample x_train: ...
1
vote
2answers
33 views

Where can I find dataset for word analogy task?

In the paper of Word2Vec by Thomas Mikolov and others, there is a accuracy report on the full Semantic-Syntactic data set. Where I can find this dataset or a related dataset for word analogy task? ...
1
vote
1answer
13 views

Product classification based on Description of a product in eCommerce website

I am searching for an eCommerce dataset which should have a product category and description for my academic research in Machine learning. Please suggest any public dataset available. ...
0
votes
1answer
40 views

How to extract global features of images in Python?

I am working on a problem involving signature verification, but am not understanding how to create a dataset for it. I would like to take some genuine and forged signatures and make them into a ...
3
votes
1answer
54 views

Relation between an underlying function and the underlying probability distribition function of data

I heard and read a lot of times the following statements and got a lot of confusion over time. Statement 1: The goal of machine learning is to get a function from the given data Statement 2: ...
0
votes
0answers
7 views

Smoothing a random noise causes undesirable results

I have some sample data with 7 features out of which only one feature is different. All other 6 features are same. But I add white noise to the data and multiply it with a factor to test some ML ...
0
votes
0answers
12 views

How should i prepare my dataset for efficient object detection and multi class classification?

I am currently using faster RCNN inception v2 model for detection and classification of object of 2 different type which are very closely identical (eg. rip apple and partially stall apple). how ...
0
votes
1answer
18 views

Very simple real-valued time-series dataset for RNN prototyping

Is there a simple real-valued time-series dataset on which a vanilla RNN model can be trained. With "very simple" I mean only two to four real-valued inputs per time step and a single real-valued ...
-1
votes
1answer
26 views

Inputting Data for Machine Learning in PyTorch

I have scoured the internet and documentation, but have not yet been able to find a simple explanation of how to load data and/or simply type data into pytorch. I am just learning this library, and I ...
6
votes
2answers
29 views

When to use missing data imputation in the data analysis problem?

I want to run statistical analysis of a dataset and build a logistic regression model and multinominal linear model by R according to the research question. But I was wondering which step should I use ...
0
votes
1answer
29 views

What machine learning algorithm should I use for specific user configuration?

I have a data-set that contains thousands of employee data, including their role, department (Applications Developer, IT Support, Network Management etc.), and using one-hot encoding all of the ...
0
votes
1answer
26 views

How can I filter and balance a Windowed Tensorflow dataset with a binary classification label, based on the label?

I have an unbalanced tensorflow windowed dataset with labels (over 90% negative examples) which I am trying to balance by filtering. I am having trouble filtering as my windowed dataset with labels ...
0
votes
1answer
19 views

Algorithm for user profiling without distinct profiles

I am trying to design an algorithm that takes in a new user with the variables department, location, job_role etc. and I want a machine-learning algorithm to decide ...
1
vote
1answer
19 views

Application of Machine learning in modelling experimental system as a PhD topic

I am working as a control Engineer in a high energy physics lab. I am looking for a PhD thesis topic. Our experimental system is highly non linear and can't be modeled using standard white box ...