Questions tagged [dataset]

A dataset is a collection of data, often in tabular or matrix form. This tag is NOT intended for data requests ("where can I find a dataset about ...") --> see OpenData

Filter by
Sorted by
Tagged with
1
vote
1answer
21 views

Create new data set

I am going to classify the hotel reviews as positive, negative and neutral. For train the model I couldn't find any data set with labels positive, negative and neutral. I could find this Hotel Reviews ...
0
votes
1answer
11 views

How to check the randomness or the distribution of an attribute in a dataset?

I have a dataset. I want to make its randomness nature match with another dataset (2nd one). Basically I want to modify the 2nd dataset values and make them similiar in distribution or nature in ...
0
votes
0answers
14 views

Predicting future of power consumption in repeating manufacturing process

I have this situation. We are tracking the power consumption of an industrial machine and by looking at the power consumption (in watt) we're trying to predict whenever something will break resulting ...
0
votes
1answer
26 views

Call routing using AI

I'm trying to find out how AI can help with efficient customer service, in fact call routing to the right agent. My usecase is given context of a query from a customer and agents' expertise, how can ...
0
votes
1answer
53 views

Pyspark Dataframes to Pandas and ML Ops - Parallel Execution Hold?

If I convert a spark dataframe into a pandas dataframe and subsequently apply pandas operations and sklearn models to the dataset in databricks, will the operations from pandas and sklearn be ...
0
votes
0answers
9 views

What are and how can I unserstand/process geo levels id from this dataset?

I am using the dataset from this driven data competition: https://www.drivendata.org/competitions/57/nepal-earthquake/data/ There are three features (geo_level_id_1, geo_level_id_2, geo_level_id_3) ...
5
votes
2answers
246 views

Discrimination vs Calibration - Machine Learning Models

I came across a new term called Calibration while reading about prediction models. Can you please help me understand how different it is from ...
0
votes
0answers
17 views

Review Helpfulness Datasets

My team is looking for a user review dataset which we can compare to one of these Amazon datasets. The dataset needs to include text reviews and a review helpfulness rating. I know this is a long-...
0
votes
0answers
35 views

How to build wake-word detection dataset from keyword pronunciations

I have quite a few audio files which are just pronunciation of one particular word, of course there are different tones, accents etc. I want to build model for wake-word detection, so I need to ...
0
votes
0answers
8 views

Is test data needed on model with no hyper-parameter

I am doing classification using Linear Discriminant Analysis (LDA), which has no hyper-parameters. I am aware the difference between validation and test set, i.e. validation is used for hyper-...
0
votes
0answers
28 views

Short text data set for classification & active learning project

I am looking for datasets that contains sentences/ very short documents for a project in the field of active learning in text classification. I've looked on yelp / imdb reviews dataset but the ...
0
votes
0answers
10 views

How can I use the twitter API to extract terms or words in Arabic language without coding?

I have received approval from twitter developers to access and collect their data set in Arabic but I don't know how I can utilize their API without using writing code. I would like to extract data ...
1
vote
2answers
31 views

I am working on a pose dataset and want to convert this data into CSV format. How to convert this data into a single row of data

This is the data for one pose. I have a lot of data like this which i want to convert to CSV to apply machine learning algorithm. I am assuming I have to convert this data into a single row of comma ...
1
vote
0answers
13 views

open source time series sales data for forecasting

I'm looking for open source time series sales data (past 2-3 yrs or more) that contains at least the following variables. ...
0
votes
0answers
25 views

Dealing with low variation data

So my current project involves using a neural network to try and predict the probability of a player getting a kill in a first-person shooter. I've recorded a number of features that should be ...
0
votes
1answer
508 views

Custom keras dataset generator not accepted by fit_generator

I've got a problem trying to use generators in Keras. I've got TensorFlow 2.1 with Python 3.7. I'm using the Keras version bundled with TensorFlow I have defined a class for my generator derived from ...
6
votes
3answers
790 views

Is it possible to use a generative model to “share” private data?

Let's say we have some data set, with lots of instances $X$ and a target $y$. If it is of some importance, you may assume that it is a "real life" data set : medium sized, with important correlations, ...
0
votes
0answers
71 views

PyTorch Dataset/DataLoader workflow with multiple .h5 files

I have multiple .h5 files, totaling 50GB, all of the same two-column format (first column: value, second column: label). I only have enough RAM to load one .h5 file at a time. I am unsure on how to ...
0
votes
1answer
53 views

What to do if a specific label of a category appears only a few times?

Let's say I am trying to predict whether a car will be auctioned or not (not what I'm actually trying to do, but it represents it pretty well) using tabular data. I have the year the car was made, its ...
0
votes
1answer
15 views

Help with breaking up a column in PowerBI to make a bar chart

My data set is a list of documents. I have a column in my data set - 'Independent Variable'. This column can contain nothing, i.e. 'NA', or one variable, i.e. 'var1', or multiple, i.e. 'var1;var2;var3'...
2
votes
1answer
25 views

Quantify the amount of differences of one data set to another

Basically I'm seeking the best method for my case scenario here. For example with the image shown below, the left one is poincare maps dataset #1, and the right is dataset #2. What's the best method ...
3
votes
1answer
33 views

How to determine the best way to replace empty records in the dataset?

I am doing a project related to predicting the next glucose value based on his/her past records. But in some patients, some of the recordings are missing. There are 2 scenarios of how the blood ...
2
votes
2answers
55 views

Creating a custom dataset for object detection

I'm currently trying to build a model to recognize approximately 10 labels (food items) in a fairly controlled environment (refrigerator). I was unable to find datasets that worked well enough for my ...
1
vote
1answer
14 views

Domain scoring based on ranking

I am a computer science student working on a small information retrieval project. I have a dictionary with a domain as a key and it's ranking as value. Based on that ranking, I need to score every ...
-1
votes
1answer
47 views

searching for clouds (in the sky) images big dataset

I'm looking for a big dataset of clouds (in the sky) ground based images. i need tens of thousands of images. It is important that the images will be ground based and not from satellite/ flights. I'...
-1
votes
1answer
54 views

Custom Pandas function for formatting

My data consists of a lot of Dataframes having the format as below: ...
0
votes
0answers
19 views

How to prepare data for multi-class tasks

I wish to train a multiclass audio classification NN. I am following this paper and this tutorial. The thing is, for audio segments containing more than a single class, ...
0
votes
1answer
20 views

How do I build a model to predict which service a customer will use on an app?

There's an app with over 50 services. I have the data on the type of service a specific customer (they have a unique customer number) does on the app, the date, location, time, duration on a service ...
2
votes
0answers
14 views

Combining Aggregated Dependent Data with Granular Independent Data

As data science newbie, I thought I was on the right tract until something doesn't seem right after plotting. A simplified structure of my dataset is as follows: FIPS (Census Tract ID) Drug-related ...
0
votes
0answers
31 views

creating a depth image dataset in Unity for object detection

I'm trying to train a model to detect and classify poses of silhouettes in depth images. The goal is to detect old the falls of old people at home. I'm using SSD and YOLO. As you may noticed, having ...
3
votes
1answer
47 views

Combining time-series data from different devices

I am using four different sensor models (four different devices) to gather some biometric data(eg. heart rate) for the same user, for the same time period. In an ideal scenario, all the devices are ...
0
votes
0answers
7 views

Place365 Mean Pixel Subtraction Values

In Place205, input images are zero-centered by mean pixel (rather than mean image) subtraction with the following values explained in readme.txt file when ...
0
votes
0answers
44 views

Finding patterns in dataset

I've got a big dataset which looks like: ...
0
votes
0answers
70 views

Using Images uploaded in google drive in Colab

Does anyone know how to use already uploaded images in Google Drive to colaboratory for creating a trainloader? I'm creating the trainloader and model in Pytorch
0
votes
1answer
37 views

How should I read the following heatmap?

I've been playing around with the linear regression, and I was thought that before commencing it's always good to plot an heatmap to see whether there are features that somehow is worth testing for ...
4
votes
2answers
79 views

Is there a way to recognize which of these scatter plots is “better”?

I doubt better is the correct adjective, apologies for that. What I mean is this: I have a set of files (1200~) each paired with a scatterplot image. I need to find a way to classify which data files ...
2
votes
2answers
44 views

Creating pronunciation dictionary for ASR

I am working on ASR(automatic speech recoginition) on Somali data as master thesis and now I am stuck with how to create a phonetics or pronunciation dictionary for it. I searched over net and could ...
0
votes
1answer
94 views

Logistic Regression outperforms MLPClassifier

I am new in ML and I am trying to train classifier. I have a tiny dataset, just 90 examples, I divided it 70/30 train/test set and started to train. As I know MLP must outperform Logistic Regression, ...
0
votes
0answers
7 views

Virtual Generation of Synthetic Ancient and Dirty English Documents

I have a collection of dirty background image, below is the sample: I have also a collection of an actual image of a dirty document with text on it, just like below: My problem with my actual image ...
2
votes
1answer
73 views

When should we use the principles of tidy data by Hadley Whickham, and when should we avoid using it?

I have been studying and data science for the past 6 months, however I just came across the principles of tidy data by Hadley Wickham in an article by Jean-Nicholas Hould. This completely changed my ...
0
votes
1answer
42 views

On which data evaluate an object detection model ? (similar or real life data ?)

I'm training an object detection model (SSD300) to detect and classify body poses in thermal images. Even I have more than 2k different poses, the background does not change much (I have only 5 ...
0
votes
1answer
50 views

What should I do with a test and train set not having same classes?

Assume we have a train dataset with 5 classes a,b,c,d,e but the test dataset have only d and e with extra class f not present in train set. If i want to do machine learning with this datset in weka or ...
0
votes
1answer
1k views

How do I read a dat file, for which I don't know its structure?

Is there any way to atleast read the text from the dat file. I have its corresponding mdf file hence I know what all data and columns are there in it. How do I figure out the contents in my dat file. ...
0
votes
1answer
52 views

single v/s multiple user machine learning model

What are the merits and demerits of training machine learning models(One-class svm, isolation forest, DBSCAN) for anomaly detection on single-user data set and multiple user data set.? keep in mind ...
0
votes
0answers
8 views

How can partial dependence help us see whether a dip in the data is caused by external factors or not?

I'm watching this FastAI course, and we're trying to predict the price of bulldozers. At one point, we figured the year the bulldozers were made is very important, so we plotted a graph with x = year ...
0
votes
0answers
44 views

Train a neural network through comparisons between images using Keras

I have a Dataset composed of 2 arrays of images and an array of desired outputs. Arrays have the same length. I have to train a neural network using keras able to recognize who is younger/older/same ...
1
vote
0answers
11 views

Non-parametric regression on set of time series: One model for each or one for all series?

Let's say I have a set of 1D time series which values have been samples in equip-distant time steps with timestamps $1,2,3,...$, they have all the same lengths and are somewhat similar in shape. I ...
0
votes
1answer
20 views

How to group a unique column and find the mean value in a separate column in R

My table of data looks like this image (but with many more lines): I want to be able to find the mean age of each region, so the mean age for region SSC20790 and so on. I believe I can use the <...
1
vote
0answers
16 views

Tool/Technology/Framework to process file with 1GB

I'm new to datascience, but I have the following problem: 1) Read a file of 1GB, which each line is a json object 2) There are two more files, much smaller, which I need to JOIN some data In this ...
1
vote
1answer
27 views

How to compute modulo of a hash?

Let's say that I have a set of users in my database, that have GUIDs as their IDs. I use xxhash to generate fixed-length hashes for each value, so that I can then ...

1
3 4
5
6 7
22