As of May 31, 2023, we have updated our Code of Conduct.

Questions tagged [dataset]

A dataset is a collection of data, often in tabular or matrix form. This tag is NOT intended for data requests ("where can I find a dataset about ...") --> see OpenData

Filter by
Sorted by
Tagged with
0 votes
1 answer
12 views

CWRU Bearing fault

I am new to ML. I have been asked to use a pre-trained GRU model for detecting a bearing fault in CWRU. is pre-trained model another name for transfer learned model?
Asma Shah's user avatar
0 votes
0 answers
20 views

Should I use Boruta feature selection if I'm comparing two datasets?

I'm a beginner to machine learning and AI so I really could you some guidance on this. I am currently comparing two different datasets with different classification algorithms. One of the datasets has ...
StrWrs_Nerd's user avatar
0 votes
0 answers
21 views

Finding a relation between three variables

I am new to data mining and have learnt about association rules mining, classification analysis, cluster analysis and outlier analysis. So, to find relationship between three variables, regression ...
Scholar7's user avatar
3 votes
0 answers
51 views

Public data set with exciting data

I'm going to give a talk about a tool for viewing tabular datasets. I'll first give this talk at a meetup, then hopefully at a series of conferences over the next few years. I want to demonstrate to ...
Ram Rachum's user avatar
0 votes
0 answers
65 views

What is the difference between SMOTE before PCA and after PCA

We all know that PCA (Principal Component Analysis) is a popular statistical tool to reduce the dimensionality in a dataset. SMOTE (Synthetic Minority Over-sampling Technique) allows you to generate ...
Sivadithiyan official's user avatar
1 vote
2 answers
42 views

how to build an images dataset efficiently

I have several image datasets and I am going to combine all of them into 1 giant dataset I decided to load the images which are in different formats (SVG, PIL images, etc ) and save them as png but ...
Mohamed Amine's user avatar
0 votes
0 answers
17 views

How to load time series data set, from .csv into keras 3D tensor

I would like to load, from CSV, where data are listed row by row, to load into Keras 3D tensor for LTMS. I've got error, that it requires 3D data, but I've only got 2D from CSV. Thank you in advance.
Fuzzy System's user avatar
0 votes
2 answers
86 views

When is the right moment to split the dataset?

I would like to ask a question about the dataset splitting. If I have a dataset can I perform preprocessing (imputation, scaling, ecc.) on the entire dataset and then splitting it into train and ...
Flavio Brienza's user avatar
0 votes
1 answer
36 views

I'm Looking for dataset with Name, Father's Name, Mother's name, Age, Gender, Ethnicity, ~Birthplace [closed]

I am new to Data Science and I don't know where to find datasets, I Google searched, Asked ChatGPT and friends, but couldn't find what I was looking for. My project is to generate new names based on ...
JeX's user avatar
  • 1
0 votes
0 answers
19 views

What is semi-ground truth?

What is semi-ground truth and how it is different from ground truth? Can you explain with an example?
heyula's user avatar
  • 1
0 votes
0 answers
14 views

What approach can be take to design a food menu intelligently?

I am given following problem, for which I am seeking for advice. Problem Statement Suppose you are given a catalogue of dishes with their cost and calories. Can you design an algorithm/model which can ...
Gaurav Gupta's user avatar
0 votes
0 answers
35 views

Is this overfitting? (generative model)

I am working with a generative method, and the network seems to perform well on training data and slightly less well on test data, but the generated data is somehow significantly worse than either of ...
quail's user avatar
  • 31
0 votes
1 answer
48 views

Alternatives to Toronto Book Corpus

As the toronto book corpus is no longer available (or rather, only in lowercase), I am looking for an alternative dataset of comparable language variety, quality, and size. Any suggestions? The ...
postnubilaphoebus's user avatar
0 votes
2 answers
338 views

How to do train-test split for multi-class classification

I am performing multi-class classification problem of different concentrations of Acetaminophen in a specified dataset. My data is in the form of images and I am using CNN. I have compiled all the ...
Zelreedy's user avatar
0 votes
0 answers
12 views

Speaker Recognition using Deep Learning

I am trying to develop a speaker recognition model using deep learning. I want to use the VoxCeleb2 audio dataset but I cannot find it. The one on their website has YouTube links, but I would rather ...
Omar's user avatar
  • 1
0 votes
0 answers
32 views

Why are all PyTorch dataloader proccesses in S state (interruptible sleep) except one?

Short version: I'm using PyTorch dataloading library to load data in parallel for training a deep learning model. When I look at the CPU usage with htop, I see a ...
Pablo Messina's user avatar
1 vote
0 answers
21 views

How much data and computation power do I need to train a machine translation model using Transformer architecture?

I am working right now on creating a dataset to use in creating a machine translation model to translate between two dialects. I have two questions that I am trying to find an answer for: How much ...
user13288833's user avatar
1 vote
1 answer
24 views

Should I drop duplicates over features but no target

I'm in a debate with someone about a problem where there are duplicates over features (i.e. $ X_1 = X_2 $ but $ Y_1 != Y_2 $). My point of view is that we should keep those datas, as they can be ...
EzrielS's user avatar
  • 311
0 votes
1 answer
20 views

How do I handle "Greater than X" in a field of integers?

I've been tasked with cleaning a dataset with a "Drive Time" column that lists times taken to drive to a specific location in whole minutes. The values range from 3 to 180 minutes but there ...
Leon's user avatar
  • 1
0 votes
0 answers
9 views

tensorflow dataset 'plant_leaves' cannot be imported to google Colab : DownloadRetryLimitReachedError

I'm trying to upload a tensorflow dataset called plant_leaves but it won't work and it shows a runtime error :
Imen Bakir's user avatar
0 votes
0 answers
14 views

How to combine multiple similar columns in a dataset derived of multiple sources?

I am trying to combine data from multiple sources. One issue I face is that the same columns are named differently. For example, the following refer to the same columns: SampleID, sample id, sampleID,...
risingStar's user avatar
0 votes
0 answers
14 views

How to deal with degeneracies in data

I am working with a dataset where I want to predict some features (let's call them G) from some other features (we'll call them W). The problem is that there is no one-to-one mapping from W to G. In ...
quail's user avatar
  • 31
0 votes
0 answers
20 views

How to scale a vector into a smaller vector?

I have a vector A of 6 integers [1000,900,800,700,600,500] and each number corresponds to another vector B [0,1,2,3,4,5]. How can I get a vector A_new of 5 integers if I scale the vector B to [0,0.25,...
sayuri's user avatar
  • 11
0 votes
0 answers
13 views

Using subgroup discovery for a very small subset of a huge dataset?

I want to use subgroup discovery to find some interesting patterns in a small subset of my dataset. The dataset contains about 100.000 records, the subset I want to look at only 100 records. Does it ...
user543's user avatar
4 votes
2 answers
55 views

Merging data approach in Data Science projects

This is more of an infrastructural question about data science. How would you manage data merging in your GitHub repository? As an example, as a data scientist I might be working on my branch and ...
Mattia Surricchio's user avatar
0 votes
0 answers
14 views

How to create a synthetic dataset for object detection with GTA-V

I am planning to do a project of object detection in the off-road environment. There is not much labeled data available for this. I was thinking about creating my own dataset with GTA-V but I don't ...
programmer_04_03's user avatar
0 votes
0 answers
9 views

Set labels in R summary table

I'm creating a function in R for plotting and calculate an ANOVA model (because I have to repeat this operation a lot of time). I wrote this function: ...
rik99's user avatar
  • 1
0 votes
0 answers
10 views

Is there data on the number of residential units in each building?

I am totally new to conducting data science, so I don't know if this is the place to ask, but I am trying to find data on the number of residential units in each building in the USA for a project I am ...
graphtheory123's user avatar
0 votes
1 answer
14 views

Total Retention Rate Calculated from Categories

I am calculating retention for 3 categories and then total, and I am trying to double check my total, but my check formula isn't working. I am comparing the last 14 days (let's call it Period 1) to ...
user485656's user avatar
0 votes
0 answers
24 views

Validation accuracy decreases when I use ImageDataGenerator instead of manually creating labels

I was getting 85 percent validation accuracy before using ImageDataGenerator, now validation accuracy for binary classification decreases even to 14 percent. and never passes 60 percent. I couldn't ...
mmusag's user avatar
  • 1
1 vote
0 answers
28 views

How can I detect errors/irregularity in a dataset

I've been working on a pathfinding project using topographic data for my software development course, even though I've never really had calculus lessons. The algorithm works fine by itself, however, ...
Maximus's user avatar
  • 11
1 vote
1 answer
30 views

what labeling format has negative Bbox values in labels?

I have a labeled dataset for object detection few thousands of images with annotation on csv file the csv contains these columns image_path, class, xmax, xmin, ymax, ymin looks like Pascal voc format ...
Mustafa Azzurri's user avatar
0 votes
0 answers
119 views

Where can I get REDD dataset if http://redd.csail.mit.edu/ is not working?

Reference Energy Disaggregation Data Set (REDD) is used for research in Non-intrusive Load Monitoring (NILM). Every paper and blogs points towards http://redd.csail.mit.edu/ as data downling link. ...
santobedi's user avatar
  • 103
1 vote
2 answers
83 views

How to decide the most suitable technique to handle Class Imbalance

For handling an imbalanced dataset, we have a variety of techniques like adjusting class weights, oversampling, undersampling, SMOTE and its different variations (RCSMOTE, GSMOTE, DBSMOTE). My ...
tanmay's user avatar
  • 173
1 vote
1 answer
42 views

If I'm comparing performance between two different datasets should sample and class size be uniform?

If I'm comparing performance between classification models on two different datasets should the number of samples per class, the number of classes, and features per sample be the relatively the same ...
StrWrs_Nerd's user avatar
0 votes
0 answers
15 views

How to reference an intermediary object in scala?

I am using spark/scala to do some transformations on a spark dataframe. I have the following code: ...
ptushev's user avatar
  • 11
0 votes
0 answers
29 views

How to tokenize custom dataset?

I have generated a dataset of quadratic equations and I want to train T5 model to solve them. Dataset looks like this: Could somebody please give a hint on how to tokenize this correctly? What to do ...
Alisa Petrova's user avatar
0 votes
0 answers
16 views

Is there any datset which has human judgement scores between 2 english senctences only?

I am trying to propose an evaluation metric in NLP and I need to compare it with other baseline metric such as Bleu scores. I need a dataset which has 2 english sentences (say same image being ...
Rajat Gaur's user avatar
0 votes
0 answers
29 views

How to create synthetic data with custom feature values?

Currently I create a synthetic dataset using the python sdv package and an existing dataset as below: ...
Nht_e0's user avatar
  • 21
2 votes
0 answers
509 views

How many samples in dataset are required to fine-tune BERT for binary classification?

I'm trying to fine-tune a BERT-based model for a binary classification task (data is in English). The dataset I'm working with is quite small (~500 samples, out of which 80% are currently used for ...
Occasus's user avatar
  • 21
1 vote
0 answers
116 views

Grouped stratified train-val-test split for a multilabel dataset

So this is indeed nontrivial. I was wondering if there is a fast heuristic algorithm for performing grouped stratified dataset split on a multilabel dataset. Stratification is usually performed to ...
jasperhyp's user avatar
1 vote
1 answer
31 views

How to select target words for Lexical Simplification dataset

I am trying to compile a Lexical Simplification dataset, which contains sentences, target words, and their simpler substitutes. I have already found some similar datasets in various languages, here ...
Mar's user avatar
  • 21
1 vote
2 answers
47 views

How to evaluate Natural Question-Answer Generation pairs?

I am trying to generate Natural Question-Answer for a specific domain. I am using a Large Language Model (LLM). I have only context to generate question-answers but don't have any ground truth. How to ...
Aaditya ura's user avatar
0 votes
0 answers
114 views

Is it possible to turn the gmail inbox into a dataset for AI fine tuning?

My idea is to have a dataset of my gmail emails and replies. The purpose is to create a bot that can reply new emails based on all past correspondence in my inbox. How do i prepare such dataset from ...
Mrfab's user avatar
  • 1
0 votes
1 answer
23 views

Why does PowerBI take longer to load data than PowerQuery takes to load preview?

I have a stored procedure in SQL Server and when I open PowerQuery, I can get a preview of the whole table in a minute or two. But once I click 'Apply and Close', it will take AGES to load and finally ...
PowerBIUser's user avatar
1 vote
2 answers
28 views

Tool for visualizing the evolution of a dataset

I wonder if anyone knows of a tool that allows to visualize how a dataset has evolved over time. There are tools for tracking experiments (neptune, mlflow, etc.) but they are based on keeping track of ...
Jorgemar's user avatar
  • 241
1 vote
1 answer
54 views

Input shape - How to feed metadata to a ML model?

I have data such as metadata: hospital layout, number of rooms, number of patients in a day etc. and then I have data regarding the doctor’s check-ins. Which is more granular. How do I feed this data ...
StephM's user avatar
  • 11
1 vote
1 answer
29 views

Where/how share annotated datasets (curated by humans)?

Once we have invested resources into annotating and reviewing some datasets (manually, by humans) how should we proceed to share them? How can we make them findable and accessible? Is it possible to ...
Quora Feans's user avatar
0 votes
0 answers
12 views

Should I standardise my domain specific text prior to annotation/creation of a Q&A dataset?

I'm going to create my own question and answer dataset to fine-tune a BERT model, however before starting I am trying to understand if any standardisation of the text needs to be performed. The data I ...
Jon's user avatar
  • 1
2 votes
1 answer
171 views

Incremental clustering algorithm

I am looking for an incremental clustering algorithm. By incremental I mean an algorithm that builds clusters starting from an initial dataset and that is able to progressively ingest new items/...
Sirion's user avatar
  • 121

1
2
3 4 5
30