Questions tagged [dataset]
A dataset is a collection of data, often in tabular or matrix form. This tag is NOT intended for data requests ("where can I find a dataset about ...") --> see OpenData
1,495
questions
1
vote
1
answer
251
views
Converting VOC2012 annotation .xml to segmentation .png
Does anyone know a good way to convert VOC2012 annotations files in xml to SegmentationClass ...
CommunityBot
- 1
1
vote
1
answer
9k
views
Tensorflow Model fitting ValueError: Layer sequential expects 1 inputs, but it received 520 input tensors
I am trying to train a model using Tensorflow. I am reading a huge csv file using tf.data.experimental.make_csv_dataset
Here is my code:
Imports:
...
CommunityBot
- 1
1
vote
1
answer
89
views
Comparing two data distributions, not the probability distribution
Let's assume that I have the 200 MNIST data. These 200 data are divided into two training dataset. one training dataset has 100 and another training dataset has the remaining 100.
I trained my two CNN ...
CommunityBot
- 1
0
votes
1
answer
15
views
How should I plot 2D data as 3D plot?
I got data (1503, 38697) (cell(datapoint), genes(features)), each elements has integer values (gene expression).
so something like
...
- 453
0
votes
2
answers
295
views
Test data missing values
I've been given a test and train data set, and I have null values in my test data set. What do I do with it?
CommunityBot
- 1
0
votes
1
answer
257
views
I need sources of interrogative, exclamatory, and imperative sentences
I am working on accumulating a large database of labeled sentences for several projects/experiments. At present I am only using Wikipedia and Project Gutenberg as sources of data. Between these two ...
CommunityBot
- 1
0
votes
2
answers
209
views
Open retail/ecommerce dataset with customers, transactions, products, loyalty, discounts, promotion information?
Does anyone know of an open retail/e-commerce dataset with information about customers, transactions, products, loyalty, discounts, and promotion information?
I am planning to create 360-degree ...
0
votes
0
answers
21
views
How can I read hdf5 files stored as 1-D array. and view them as images?
I have a large image classification dataset stored in the format .hdf5. The dataset has the labels and the images stored in the ...
- 111
0
votes
1
answer
119
views
What is the main concept of using lexical,linguistic, semantic or syntactic approach in NLP for cyberbullying
I am really in need of some explanation, I am working on an NLP cyber-bullying detection tool which I will deploy to the web using Django framework, however, am stuck on some idea, can someone explain ...
CommunityBot
- 1
0
votes
0
answers
14
views
Looking for datasets on automobile parts information for machine learning
I'm embarking on a machine learning project that requires a comprehensive dataset of automobile parts information. The goal is to train a model that can identify and categorize various auto parts, ...
- 1
1
vote
1
answer
1k
views
Training loop stops after the first epoch in PyTorch
I'm trying to train a seq2seq model using PyTorch using the Multi30K dataset from Dutch to English language.
Here is my snippet of code:
CustomMulti30k class
...
CommunityBot
- 1
4
votes
1
answer
4k
views
Weighted loss functions vs weighted sampling?
For image classification tasks, is there a practical difference between using weighted loss functions vs. using weighted sampling? (I would appreciate theoretical arguments, experience or published ...
CommunityBot
- 1
0
votes
0
answers
6
views
How to get / calculate statistics about voxceleb2?
VoxCeleb2 contains ~1M utterances from ~6000 speakers (multilingual).
I want to get statistics about voxceleb2:
number of utterances & speakers for each language
I tried to find the metadata ...
- 363
0
votes
0
answers
15
views
Seeking datasets for training a Language Model on U.S. mortgage loan processes
I'm in the process of training a Language Model (LLM) and require datasets that encompass various aspects of the U.S. mortgage loan process. The model's aim is to understand and simulate decision-...
- 1
0
votes
1
answer
46
views
Proper way to build time series dataset
I have daily time series but need to predict weekly total.
Here is example:
date
col0
col1
col2
col3
col4
target
0
2023-01-01 00:00:00
-1.021
0.708
-0.564
-0.502
0.244
-106.538
1
2023-01-02 00:00:...
- 198
0
votes
0
answers
24
views
propper feature encoding
I working with the following data set
also here is it's detailed description of "packet_dat" column
I can't understand how I can encode packet_dat column into proper feature so my ...
- 101
0
votes
0
answers
11
views
Packet data needed fot the model
I want to use this data set https://huggingface.co/datasets/rdpahalavan/network-packet-flow-header-payload for network attack classification
Here is packet structure from the README file.
Assuming I ...
- 101
2
votes
2
answers
229
views
Log analysis dataset with labeled cybersecurity issues
I am seeking to find a dataset with log files that have labeled cybersecurity issues. As I am trying to build a cybersecurity log analysis model there is no preference on the type of the log, but ...
CommunityBot
- 1
0
votes
1
answer
63
views
Image parameters for SRGAN
In some implementations of SRGAN I've noticed, that datasets consist of the high-resolution images and the low-resolution images are created later by, e.g. resizing (decreasing the size) hr-images. ...
CommunityBot
- 1
1
vote
1
answer
102
views
On which data evaluate an object detection model ? (similar or real life data ?)
I'm training an object detection model (SSD300) to detect and classify body poses in thermal images.
Even I have more than 2k different poses, the background does not change much (I have only 5 ...
CommunityBot
- 1
0
votes
0
answers
14
views
Feature selection on datasets with both categorical and numerical features
I'm proposing a novel methodology for feature selection in the context of tabular datasets that contain both numerical and categorical features. In order to prove the efficacy of my methodology, I ...
0
votes
0
answers
121
views
ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type int) in Python - from_gerator fails
I know this question has answers, but they are not working.
why this fails ?
(and why tensorflow has troubles in reading numpy!!)
...
- 1
0
votes
0
answers
19
views
statistical analysis of two data sets have different number of sampling
I have two datasets from my statistical fault analysis: one with 400 iterations and another with 500 iterations. Both datasets have the same maximum and minimum values, and some data points are ...
- 1
0
votes
1
answer
170
views
How to prepare training data for deep learning models
I am working on a project which involves the application of deep learning models. I have collected training data. In collected images, I have more than one object in interest. I am not very clear how ...
CommunityBot
- 1
0
votes
1
answer
88
views
Multilabel classification: Choosing threshold
I'm creating a multilabel classification approach based on sentence embeddings applied to text taken from a chatbot.
We have the following:
a training dataset of 2,500 lines, where each line is a ...
CommunityBot
- 1
0
votes
1
answer
24
views
A guide to learn data analysis
I'm new to data analysis and I need to do a data analysis project using clustering methods for a course in R. I have no idea how to start and choose my data set. I'm looking for some resources. Is ...
0
votes
0
answers
17
views
How to make a dataset for Audio Speech Recognition of Arabic Language
I am trying to map Quranic Verses for the identification of errors. Different audios for the same verse have different durations as it depends on the reciter. I used Audacity to normalize but the ...
- 1
1
vote
1
answer
86
views
How to Proceed with Tokenized text content which is given in number?
I have one data set of customer review, but the text data is given is tokenized text number. I am unable to proceed thinking about how to proceed?
As I am encountering such data set the first time, ...
CommunityBot
- 1
2
votes
1
answer
168
views
Size of datasets over years
I am looking for statistics, to understand the evolution of the size of the (public) dataset over the years.
I just found the following statistics:
The poll of KDnuggets that actually shows that over ...
CommunityBot
- 1
0
votes
0
answers
14
views
Open Source Model or dataset for attention map visualization
Helloo (:
I'm trying to develop a python application where users can upload an image e.g. of their website and receive a heatmap or attention map in return. This map should indicate where users are ...
- 1
0
votes
2
answers
344
views
Data set of vectors of SVG paths for digits
I have used the MNIST data set many times to train models for digit recognition based on object character recognition (OCR).
I am now trying to do the same but with a data set of svg paths.. I am ...
0
votes
0
answers
14
views
What do I need to take into account if I want to test one model on different test datasets?
I'm just new to machine learning, I want to test a generative model on two different test datasets to compare its performance on different datasets. What do I need to take into account about the ...
- 1
1
vote
1
answer
591
views
How to split a dataset into train and test sets for time series (multiple step-multiple output forecasting)?
I am trying to use an LSTM neural net to do multiple step / multiple output forecasting (I predict multiple values in one time knowing some values in the past). But, I have realized that I must be ...
CommunityBot
- 1
0
votes
2
answers
64
views
Information Visualization: which parameters should a data scientist plot to best explain the work behind a Neural Network?
I'm a beginner in the field and I wrote and trained my first Convolutional Neural Network from scratch. I would like to know what types of graphics and parameters a data scientist uses to plot to ...
CommunityBot
- 1
0
votes
1
answer
144
views
How to drop the previous rows of a database based on a matching value in a column?
So I am currently trying to sort through a data frame containing attribute classes and values of teams. However, my data has multiple rows of different classes and values of the same Team ID/Attribute ...
CommunityBot
- 1
1
vote
1
answer
5k
views
Extracting data and saving data from hdf5
I have a hdf5 file, and I want to extract a part of the data and save it as the same format.
The data type is
DATATYPE H5T_IEEE_F32BE
So far I have ...
CommunityBot
- 1
0
votes
0
answers
22
views
Model for small sample time series
Im a total novice and i need to estimate some kind of relation-proof model(Granger test results, correl matrix are already provide some evidence) with following dataset:
20 observations (2001-2021), 4 ...
2
votes
2
answers
829
views
How can I use my own dataset for Image segmentation using Tensorflow
I have a huge problem using my own created dataset for image segmentation using Tensorflow. The dataset that I've build contain images like the one shown below:
The problem that I have is: How do I ...
CommunityBot
- 1
2
votes
2
answers
560
views
How to quantitatively compare two or more complex data sets
I have a series of datasets that are composed of 100 or so variables and a corresponding response variable. I am often faced with the question of trying to attribute differences in the response ...
CommunityBot
- 1
0
votes
1
answer
71
views
difference btw data distribution and frquency distribution
I have a dataset with 'n' features and corresponding labels(binary in nature). How can I calculate the data distribution and frequency distribution of the same? What is the difference btw the two?
CommunityBot
- 1
0
votes
0
answers
30
views
Optimal ML classification approach
Background: I have an app data (impressions, user activities) that I can use as features for a multiclass classifier (5 classes).
I just want to discuss about some things that our team is having a ...
1
vote
3
answers
417
views
Keras Classifier returns similar output for all Predictions
I completed training the model with an accuracy of 1.000 and a validation accuracy of 0.9565. Unfortunately whenever i input a image into my model i get the same output regardless. Am i doing ...
CommunityBot
- 1
0
votes
1
answer
65
views
Batch normalization for multiple datasets?
I am working on a task of generating synthetic data to help the training of my model. This means that the training is performed on synthetic + real data, and tested on real data.
I was told that batch ...
CommunityBot
- 1
0
votes
1
answer
24
views
how to build model using two input dataset in which there is no common column to merge or combine
I want to create model for truck company in which trucks delivers the car for customers.i have two data sets. one is customer details like how many cars they want from particular area or terminal and ...
- 326
0
votes
1
answer
175
views
Plot multiple catplots from a dataframe
I have a dataframe and I want to plot multiple catplots. I want to use a loop but i do not know how to code one for this. Any examples?
...
CommunityBot
- 1
1
vote
1
answer
85
views
How to draw a sample from data set with respect to a given categorical or numerical variable based on given freely chosen distribution? (Python)
Say I have a data set for some past period. Now new data appears and for a given variable in the data and we find that the distributions have shifted (for example with "age" it would be that suddenly ...
CommunityBot
- 1
4
votes
1
answer
160
views
Relation between an underlying function and the underlying probability distribition function of data
I heard and read a lot of times the following statements and got a lot of confusion over time.
Statement 1: The goal of machine learning is to get a function from
the given data
Statement 2: The goal ...
CommunityBot
- 1
0
votes
1
answer
44
views
Very basic but how to understand data statistically for machine learning?
I’m trying to solidify my statistics so I really know how to use them in my analysis/models. However my concept of statistical testing gets completely messed up by context. I’m unsure defining exactly ...
- 36
0
votes
0
answers
225
views
Where can I get REDD dataset if http://redd.csail.mit.edu/ is not working?
Reference Energy Disaggregation Data Set (REDD) is used for research in Non-intrusive Load Monitoring (NILM). Every paper and blogs points towards http://redd.csail.mit.edu/ as data downling link. ...
0
votes
1
answer
26
views
Time Series Forecasting for Multiple Store Sales with Simultaneous Timestamps
I have a sales dataset with each store having a unique identifier. The dataset contains daily sales data for each store over a period of around two-years. I'm looking to build a time series ...
- 198