Questions tagged [dataset]

A dataset is a collection of data, often in tabular or matrix form. This tag is NOT intended for data requests ("where can I find a dataset about ...") --> see OpenData

Filter by
Sorted by
Tagged with
1
vote
1answer
75 views

Split Dataset in two Explainable Subsets

is there a general approach for splitting a dataset into two or more subsets so that a classification problem can be solved with higher accuracy for the resulting separate subsets? I have a very ...
0
votes
1answer
54 views

What to do if a specific label of a category appears only a few times?

Let's say I am trying to predict whether a car will be auctioned or not (not what I'm actually trying to do, but it represents it pretty well) using tabular data. I have the year the car was made, its ...
9
votes
6answers
729 views

What are some of the best practices for sharing data and models with colleagues?

As a data scientist who recently joined a new team, I wanted to ask the community how they share data and models among their colleagues. Currently I have to resort to storing data in some central ...
1
vote
0answers
15 views

Making a tagged Part-of-Speech corpus with the help of a lexicon

I have a part-of-speech lexicon that has two columns, words and part-of-speech tags, inside a Pandas's Dataframe. Also, I have a list of tokens (words) in another Dataframe. I want to take each ...
1
vote
1answer
71 views

Public dataset for news articles with their associated categories for multilabel data classification

I am wondering if there are any public datasets of news, like New York Times (NYT) or similar to various news categories such as politics, entertainment, lifestyle, general news, sports etc. I want to ...
2
votes
1answer
149 views

Dataset where svm performance is significantly different from random forest

Is there a specific dataset where svm performs significantly better or worse than random forest? I know that the performance could depend on the dataset but is there a specific dataset?
0
votes
0answers
44 views

How to predict variables based on multiple samples?

The problem I tried to do some ML models but everytime I had some weird plots and I couldn't understand these scores. I'm relatively new to ML and maybe someone can help me with this data with some ...
1
vote
1answer
14 views

Domain scoring based on ranking

I am a computer science student working on a small information retrieval project. I have a dictionary with a domain as a key and it's ranking as value. Based on that ranking, I need to score every ...
0
votes
0answers
9 views

Creating the Yolo Training datasets from Scrapped video as training model

I have gathered the 200 car type video for one car type and model Auto detection with YOLO (Accessible by this link), and converted the video of folders to photo by this colab page shown here: So i ...
1
vote
4answers
535 views

Which type of model to use and what's my target variable

I am a beginner in Data Science field, so sorry if my question is too basic. The task is to build an ad bidding model for online marketing which allows you to deliver targeted ads to the right ...
1
vote
1answer
53 views

How do I prepare data in which each output row depends on multiple input rows?

My goal is to predict the value of Y based on multiple values of X1 and X2 for each observation of Y. In my example, I want to predict whether a customer will file for bankruptcy (table 1) based on ...
0
votes
1answer
23 views

How do I build a model to predict which service a customer will use on an app?

There's an app with over 50 services. I have the data on the type of service a specific customer (they have a unique customer number) does on the app, the date, location, time, duration on a service ...
2
votes
1answer
130 views

Reinforcement Learning on real time data over a web server

Question: is it possible to implement a reinforcement learning model over a NodeJS server? This server would be receiving binary forms of data (open /close; yes/no) in real time. The objective for ...
-1
votes
1answer
33 views

Can we identify that an academic dataset was used for commercial purpose [closed]

There are many datasets released on the internet. Authors of many of these datasets state that the datasets are strictly for academic usage and not for commercial purposes. Although some datasets are ...
0
votes
1answer
23 views

How to merge all the data to have a final dataset [closed]

I am working with a problem that has different tables, end goal to predict if a customer will end up subscribing based on purchases. Mother table containing user_id,register_reason ...
0
votes
0answers
40 views

How to set thresholds to automate the management of model drift?

I have a trained ML Model and new data coming in every week. The new data sometimes vary too much (in a statistical sense) and, therefore, the performance of the ML model degrades. The data-set schema ...
2
votes
2answers
95 views

features importance

Suppose I have dataset labeled with two classes such as healthy and unhealthy and I applied feature selection (features importance)on dataset. How can I know that features are important to which ...
0
votes
1answer
26 views

Encoding of high cardinality multi-label categorical feature?

This is the problem of binary classification: "1" - the subscriber is a driver (belongs to the segment of drivers), "0" - the subscriber is not a driver (does not belong to the ...
1
vote
2answers
566 views

Time series normalization using min max technique

I have a time series dataset and I would like to normalize the data (diff which is of type list) as below using Min Max ...
2
votes
2answers
70 views

RMSE is higher for bigger values of target variable - how to decrease

I am solving a problem with machine learning and I have some data with two integer type independent variables and a continuous dependent variable. I am optimising to RMSE. I had fairly large RMSE ...
0
votes
3answers
63 views

How do i get dummies for this dataset

I am using an udemy course for MachineLearning and I am trying to form a dummy for my variable the column is Country I want to change to France Germany Spain France ...
8
votes
3answers
5k views

Where can I find freely available multi-label datasets online? [closed]

I'm trying to find multi-label classfication datasets, which are available for free online. By "multi-label" I mean that each instance can be labeled with anywhere from a single to $k$ labels, where ...
0
votes
1answer
159 views

How to approach the PlantVillage dataset?

I'm working on the PlantVillage dataset and i want to predict the type of the disease from the image of a leaf. The dataset is labeled in pairs (Type of the plant,Healthy/name of the disease). I'm ...
1
vote
1answer
38 views

Ingredients, Recipes and recipe ratings. I would like to predict the rating based on combination of ingredients

I would really appreciate some help on the first steps to my problem, suggestions of modeling techniques i could use or relevant research (i could not find any). I have a list of ingredients (150 in ...
0
votes
0answers
28 views

NLP - Found input variables with inconsistent numbers samples

I'm trying to train a model to read the greetings from the sample dataset collected from Tripadvisor and I've been getting the following error when I am trying to ...
1
vote
1answer
63 views

If the entire production dataset is known is advance, should I use it when fitting normalization parameters?

I found in multiple sources the recommendation not to fit the normalization parameters on the combined train/test dataset when evaluating the model, to prevent data leakage. I presume this ...
2
votes
2answers
1k views

Train-Test split for a recommender system

In all implementations of recommender systems I've seen so far, the train-test split is performed in this manner: ...
1
vote
1answer
159 views

Converting VOC2012 annotation .xml to segmentation .png

Does anyone know a good way to convert VOC2012 annotations files in xml to SegmentationClass ...
0
votes
1answer
35 views

Augmentation on test dataset and validation dataset

I'm training a segmentation model (computer-vision). Thus, my dataset contains images and masks (binary segmentation of objects). I'm augmenting the training dataset (applying random crop, rotation or ...
2
votes
2answers
394 views

Pre-processing on MRI images

I have MRI images of brain tumors collected from a hospital (not a benchmark dataset). And I am planning to use them to predict/classify tumour types using a typical machine learning approach: texture ...
0
votes
2answers
528 views

Classification Dataset which is linearly non separable

I am trying to find a dataset which is linearly non-separable. I checked the Iris dataset and the UCI website says: The data set contains 3 classes of 50 instances each, where each class refers to ...
1
vote
1answer
53 views

On which data evaluate an object detection model ? (similar or real life data ?)

I'm training an object detection model (SSD300) to detect and classify body poses in thermal images. Even I have more than 2k different poses, the background does not change much (I have only 5 ...
4
votes
2answers
156 views

Was this dataset analysed correctly? [closed]

There's a Twitter thread going around that claims there are signs of voter fraud due to anomalies in the election vote count data set. You can download the dataset here and find the script to generate ...
0
votes
1answer
15 views

Negative examples and false positives in object detection using SSD model

I am training a model to detect some objects. I have 8 classes and there are other similar objects that I don't want the model to detect. I have tried to make an unknown class for them but they are so ...
0
votes
1answer
25 views

Is there any data set containing list of windows software and the category each one of them belongs to?

I wanted a dataset which lists many windows softwares according to their categories for a project. eg: anaconda: software development, roblox: gaming, VLC: video player etc. I searched but could not ...
2
votes
1answer
89 views

How does skewed data affect deep neural networks?

I'm playing around with deep neural networks for a regression problem. The dataset I have is skewed right and for a linear regression model, I would typically perform a log transform. Should I be ...
0
votes
0answers
15 views

Data storage for Intrusion Detection System

I write an IDS(in C) for a KNX network for my thesis. For this, I just store all the telegrams within an SQLite database. But I'm not sure if a SQL database is the best here. I want to "ask ...
1
vote
1answer
330 views

How to split a dataset into train and test sets for time series (multiple step-multiple output forecasting)?

I am trying to use an LSTM neural net to do multiple step / multiple output forecasting (I predict multiple values in one time knowing some values in the past). But, I have realized that I must be ...
1
vote
2answers
26 views

How to work with Sensitive (personal) Data, without actually seeing it?

I'm currently working on a project that has sensitive data. I'm allowed to see the data on a single platform (company PC), but outside of that platform I cannot process it. I'd like to train a ML ...
-1
votes
1answer
27 views

Problem importing imblearn [closed]

I am trying to do: import imblearn in my jupyter notebook, but when I run the cell I get the following error message: ...
0
votes
0answers
17 views

How to correctly label images with multiple objects

I have 3 types of images: A: Images of apples B: Images of bananas, however some of these banana images also contain apples in the observable background Is it enough to just label the bounding boxes ...
0
votes
1answer
28 views

Training Accuracy is getting higher, but Valid Loss and Accuracy is same every epoch

I'm doing a transfer learning with ResNet50. My dataset is clothes(224x224x3), and 49 category(classes) -> training data 1000 per 1 category, total 49000. And valid data 200 per 1 category, total ...
5
votes
2answers
423 views

Can all data be represented tabularly?

... assuming that tables allowed for nested non-primitive values. I'm trying to think of any instance where data cannot be represented as tables (or a series of tables)... and I can't think of any. ...
3
votes
2answers
170 views

Training, Testing and Validation Dataset [closed]

I'm training a Unet model for tumor segmentation. I have a dataset of 400 patients for that. The used images are CT scans (3D images) that I divide into 2D images (for a total of 30k 2D images). I am ...
0
votes
0answers
13 views

Inferring composition of a subset of classes of multiclass dataset

I've come across a problem in assessing the design of a Machine Learning solution I am involved in, which I will describe using a commonly known dataset, the Iris Dataset, and some drawings. Suppose ...
3
votes
1answer
201 views

German Chatbot or conversational AI

I want to build a chatbot mostly BERT(Transformer) based in the German Language. But I do not find any German chatbot data set! So does it make sense to use google translator API to translate the ...
2
votes
1answer
33 views

Playing cards object detection problem?

I'm trying to train an object detection model to detect some of playing cards for example it will detect 1,2,3, king, queen and jack and i'm making a class of not and i have put examples of other ...
0
votes
1answer
29 views

Reuters Dataset Labels description

the list of Reuters dataset labels are provided with the dataset and also available in varous online resources. Here is an example. But I couldn't find what each of these labels actually means. I was ...
0
votes
1answer
34 views

How to normalize the data correctly in spam dataset

I'm working on the spam dataset to classify the inputs into binary classes. my problem is that: the observations in the dataset are floats small numbers in the first 53 column, and the 54 is float ...
-1
votes
2answers
1k views

How to deal with nan values after merging / joining two dataframes? [closed]

A lot of time after merging two pandas dataframes, I end up with NaNs in the new dataframe, that's just how the way it is, because one csv does not have all the ID's that the other has (Two dataframes ...

1
3 4
5
6 7
23