Questions tagged [dataset]
A dataset is a collection of data, often in tabular or matrix form. This tag is NOT intended for data requests ("where can I find a dataset about ...") --> see OpenData
1,527
questions
0
votes
1
answer
251
views
What is the difference between SMOTE before PCA and after PCA
We all know that PCA (Principal Component Analysis) is a popular statistical tool to reduce the dimensionality in a dataset. SMOTE (Synthetic Minority Over-sampling Technique) allows you to generate ...
- 101
1
vote
2
answers
295
views
question about splitting data set by attributes and labels
Why the dataset need to use X as attributes and y as label?
WHy not use a X to have it all?
CommunityBot
- 1
0
votes
1
answer
14
views
How to create a conversation dataset from a website without API?
I am currently doing my thesis on Natural Language Processing and it involves studying how people text online in a community so that it can be used to simulate conversational agents that can mimic ...
2
votes
2
answers
1k
views
Generating synthetic data based off existing real data (in Python)
I am looking for an approach to generate synthetic data for anomaly detection. We have real data, but want to inject anomalies to battle-test the model (the real data is too limited for likely future ...
CommunityBot
- 1
0
votes
1
answer
74
views
Keras MLP not working
For my degree's final project I am working with Keras and trying to build different AI models. I'm having trouble with an MLP. First I preprocess the UNSW-NB15 dataset and later use it as input in a ...
CommunityBot
- 1
1
vote
1
answer
45
views
Does f1 score evaluate only the model or does it also enable us to observe and evaluate the data?
I have a dataset. This dataset consists of the data that the actual picture that needs to be drawn, that is, the 100-point graded paper, and the similarity between 100 and 0 points graded pictures ...
CommunityBot
- 1
0
votes
0
answers
17
views
Predicting contract renewals with a linear regression model
Good day all
I have an idea where I want to predict contract renewals over time for a service based company. Essentially, taking the total amount of times a customer decides to renew their contracts ...
- 1
2
votes
1
answer
49
views
Return the gradient and y intercept (m, b) to create two lines to best fit the data
I have been working on this task for a few hours now and have been unsuccessful with getting the target result. I have tried using multiple methods of trying to split the dataset using different ...
CommunityBot
- 1
1
vote
1
answer
247
views
Segment 5-7 min audio into sentence wise audio clips for creating speech recognition dataset
I am trying to create a speech recognition dataset, especially for Indian Accents.
I am taking from colleagues to build this.
Daily I send an article link and ask them to record and upload it to ...
CommunityBot
- 1
0
votes
1
answer
766
views
Using Python/Pandas to compare columns of unequal length
https://ibb.co/cyCzVgy
I would like to be able to compare the NEW_PRICE to the LINE_TOTAL_ALLOWED amounts based on the ...
CommunityBot
- 1
0
votes
2
answers
803
views
How to read the labeled enron dataset categories?
I am trying to use the labeled Enron dataset (link) but I am really confused about the labeling system they use. I understand the Cat_[1-12]_level_weight is some ...
- 1
1
vote
1
answer
1k
views
How to resize image along with their mask?
I have original images of the size 1935x1481. I am using labelme to annotate the images. I am creating polygons on the original image. Is there a way to resize the image along with their mask? I am ...
CommunityBot
- 1
0
votes
0
answers
6
views
why images in dataset for face recognition are put into separate folders?
All the datasets for the face recognition I have come across so far, use the structure that is described in this question: Face dataset organized by folder
(The images of the same object are stored ...
- 1
2
votes
1
answer
168
views
Reinforcement Learning on real time data over a web server
Question: is it possible to implement a reinforcement learning model over a NodeJS server?
This server would be receiving binary forms of data (open /close; yes/no) in real time.
The objective for ...
CommunityBot
- 1
0
votes
1
answer
97
views
Data transformation pipeline error
When I'm making data transformation pipeline on a dataset I keep getting error as " all the input array dimensions except for concatenation axis must match exactly, but along dimension 0, the ...
CommunityBot
- 1
0
votes
1
answer
79
views
Better understand the scatter plots
I'm trying to understand how to read a scatter plot based on the results of my confusion matrix. This is the CM:
I saw an high correlation between column 5 and column 6, a low correlation with column ...
CommunityBot
- 1
0
votes
1
answer
75
views
Why does PowerBI take longer to load data than PowerQuery takes to load preview?
I have a stored procedure in SQL Server and when I open PowerQuery, I can get a preview of the whole table in a minute or two. But once I click 'Apply and Close', it will take AGES to load and finally ...
CommunityBot
- 1
0
votes
0
answers
10
views
A dataset for same animal recognition
Is there any dataset for animal recognition (dogs and cats for starters) that would be similar to 'Labelled Faces in the Wild ' for human faces? I've searched the web in general and Kaggle datasets, ...
- 1
1
vote
2
answers
546
views
How to group a unique column and find the mean value in a separate column in R
My table of data looks like this image (but with many more lines):
I want to be able to find the mean age of each region, so the mean age for region SSC20790 and so on. I believe I can use the <...
CommunityBot
- 1
2
votes
1
answer
5k
views
How to resize MNIST images to fit AlexNet model
I am using the keras API to load in the MNIST dataset. My problem is I need to use AlexNet as my algorithm. Understanding the AlexNet model, I require to start with 277x277 images but the MINST ...
CommunityBot
- 1
1
vote
1
answer
130
views
Initialize a recommender system with no dataset
Consider a platform for content recommendation based on the user history. The contents are books and articles and by history I mean what the user has read, what he has shared and so on. I know that ...
CommunityBot
- 1
0
votes
2
answers
29
views
Pytorch `DataSet.__getitem__()` called with `index` bigger than `__len__()`
I have following torch dataset (I have replaced actual code to read data from files with random number generation to make it minimal reproducible):
...
- 6,614
1
vote
2
answers
155
views
Clustering multiple csv files that cannot fit in RAM
I have multiple csv files each of which has at least 200MB of data across 12 columns. Each csv file possibly can fall into 4 categories or labels. I am trying to see which clusters each of these files ...
- 68
0
votes
2
answers
390
views
Data set of vectors of SVG paths for digits
I have used the MNIST data set many times to train models for digit recognition based on object character recognition (OCR).
I am now trying to do the same but with a data set of svg paths.. I am ...
CommunityBot
- 1
1
vote
1
answer
775
views
LFR as synthesized data for dynamic community detection methods
In recent methods for community detection in dynamic networks, LFR benchmark is used as dynamic dataset generator, but I thought it is for static community based data generation. For example in paper ...
CommunityBot
- 1
1
vote
1
answer
47
views
Generating artificial data to extend learning set
I have dataset containing 42 instances(X) and one final Y on which i want to perform LASSO regression.All are continuous and numerical. As the sample size small, I wish to extend it. I am kind of ...
CommunityBot
- 1
0
votes
1
answer
114
views
Is there any data set containing list of windows software and the category each one of them belongs to?
I wanted a dataset which lists many windows softwares according to their categories for a project. eg: anaconda: software development, roblox: gaming, VLC: video player etc. I searched but could not ...
CommunityBot
- 1
1
vote
1
answer
58
views
k nearest neighbors method, temporal trend in error
I have this set of data that looks like this
I was asked o build a $k$-nearest neighbors algorithm for it which I just finished building. I have this question in regards to the data that I do not ...
CommunityBot
- 1
1
vote
1
answer
102
views
How do I prepare data in which each output row depends on multiple input rows?
My goal is to predict the value of Y based on multiple values of X1 and X2 for each observation of Y.
In my example, I want to predict whether a customer will file for bankruptcy (table 1) based on ...
CommunityBot
- 1
3
votes
0
answers
70
views
How to find the tec data? [closed]
I want to lstm model with inputs are sequence vectors consisting of TEC, magnetic activity index
(ap) and solar radio flux at 10.7 cm (F107).
TEC index compose the input vector with dimension 33.
TEC ...
- 31
1
vote
1
answer
119
views
Predicting invoice data of 12 month using only 1 month data
I have only 1 month of historical data of invoices can I predict next 12 month of data with good accuracy if it is possible then which model should I used for prediction?
Thanks
CommunityBot
- 1
2
votes
1
answer
119
views
Where can I find free multi-instance single-label datasets for object detection?
I'm trying to find free multi-instance single-label datasets for object detection online.
By "multi-instance and single-label" I mean that each image contains only object belonging to one ...
CommunityBot
- 1
1
vote
1
answer
137
views
what labeling format has negative Bbox values in labels?
I have a labeled dataset for object detection
few thousands of images with annotation on csv file
the csv contains these columns
image_path, class, xmax, xmin, ymax, ymin
looks like Pascal voc format ...
CommunityBot
- 1
2
votes
1
answer
43
views
splitting small dataset into train and test set/
I have a very small dataset with only 70 samples an 18 features. The issue is that in this type of data set, splitting the data set into two parts, training and testing, causes a series of important ...
- 1,717
0
votes
1
answer
46
views
Why are ordered lists considered one dimensional?
At Uni I am currently taking data visualization as a course.
While reading thru the script I stumbled upon two examples for one dimensional data.
Example one:
Bus stops with their name and order (...
CommunityBot
- 1
1
vote
1
answer
10k
views
Tensorflow Model fitting ValueError: Layer sequential expects 1 inputs, but it received 520 input tensors
I am trying to train a model using Tensorflow. I am reading a huge csv file using tf.data.experimental.make_csv_dataset
Here is my code:
Imports:
...
CommunityBot
- 1
0
votes
1
answer
453
views
How to handle ID variables when splitting data for machine learning?
I'm new to machine learning, and I'm working with some international head-to-head sports competition data. I used relational data creation techniques in tidyverse to join several data sources to ...
CommunityBot
- 1
2
votes
1
answer
86
views
What to do if a specific label of a category appears only a few times?
Let's say I am trying to predict whether a car will be auctioned or not (not what I'm actually trying to do, but it represents it pretty well) using tabular data. I have the year the car was made, its ...
CommunityBot
- 1
1
vote
1
answer
20
views
Correlation of groups together or separately?
Apologies. I am trying to format the table but I am unable to get it in the right format.
I have a tabular time-series data that I am working with and have questions on how to calculate correlation.
I ...
1
vote
1
answer
260
views
Converting VOC2012 annotation .xml to segmentation .png
Does anyone know a good way to convert VOC2012 annotations files in xml to SegmentationClass ...
CommunityBot
- 1
0
votes
0
answers
16
views
Importing csv file into sql workbench
I just started a small data cleaning project. I am facing some problems importing a table as a csv file. When I use "table data import wizard", only about 500 rows were imported, but the ...
- 1
1
vote
4
answers
809
views
Which type of model to use and what's my target variable
I am a beginner in Data Science field, so sorry if my question is too basic.
The task is to build an ad bidding model for online marketing which allows you to deliver targeted ads to the right ...
CommunityBot
- 1
0
votes
1
answer
8
views
Faster preprocessing for Arabic texts
Background
I'm analyzing a relatively large text-based Arabic dataset using Python (50,000 - 70,000 text files; total size ~5GB).
I want to segment, stem, and POS tag the dataset. I am aware of two ...
- 1
0
votes
1
answer
282
views
I need sources of interrogative, exclamatory, and imperative sentences
I am working on accumulating a large database of labeled sentences for several projects/experiments. At present I am only using Wikipedia and Project Gutenberg as sources of data. Between these two ...
CommunityBot
- 1
3
votes
1
answer
63
views
Am I supposed to be using Mixed Effects?
I ran a GLS random effects regression on some NBA data in Stata, and I was told that it was wrong because I didn't use mixed effects model. This may every well be the case, but I was quite confused by ...
CommunityBot
- 1
0
votes
1
answer
81
views
Image parameters for SRGAN
In some implementations of SRGAN I've noticed, that datasets consist of the high-resolution images and the low-resolution images are created later by, e.g. resizing (decreasing the size) hr-images. ...
CommunityBot
- 1
-1
votes
1
answer
26
views
machine learning - data science - data analysis
I have a research project in the machine learning area. In this study, dataset contains more than 4000 numbers categorized in four columns. I am going to find or predict a possible relation between ...
- 68
0
votes
1
answer
128
views
What is the main concept of using lexical,linguistic, semantic or syntactic approach in NLP for cyberbullying
I am really in need of some explanation, I am working on an NLP cyber-bullying detection tool which I will deploy to the web using Django framework, however, am stuck on some idea, can someone explain ...
CommunityBot
- 1
0
votes
0
answers
8
views
Creating an Index that measures sum AND count/frequency
I work at an R1 university and am trying to synthesize a dataset that has information about grants that faculty here have won. I want a quick-and-dirty index that balances two things: how much a given ...
0
votes
1
answer
132
views
Multiprocessing data loading in colab
I want to convert mozilla common voice dataset from mp3 to wav. But this dataset is large and convertion takes many time. How can I make this convertion in colab with multiprocessing to decrease time ...
CommunityBot
- 1