Questions tagged [dataset]

A dataset is a collection of data, often in tabular or matrix form. This tag is NOT intended for data requests ("where can I find a dataset about ...") --> see OpenData

Filter by
Sorted by
Tagged with
0
votes
0answers
3 views

Label data set for sentiment analysis

I am a beginner in this field. I have a scrapped review data set. It contains review socre (1 - 10) and review content. I am going to label the reviews according to the review score like below : 0-2 -&...
0
votes
0answers
12 views

how to get length of files loaded with keras to dataset

I am trying to concatenate two datasets that were created using the keras.preprocessing.image_dataset_from_directory() with the paths as follows: ...
2
votes
1answer
21 views

Difference between grayscaled and binary mnist dataset

When making a cnn you could use the classic mnist dataset containing grayscale images. I am considering transforming them to simple binary images instead, the questions is should i? It will be much ...
0
votes
0answers
39 views

How deep learning extracts the features from images in image classification?

I'm working on an image classification project. I just want to know some basic level clarification. how does a neural network learn only the relevant features based on the label? Let's say I'm ...
0
votes
0answers
11 views

Should I apply a transformation to columns with INTEGERS, in case I want to reduce the skewness of that column?

I am performing EDA on a dataset of Hotel Reservations. Target is Categorical stating if a given customer will cancel the reservation or not. Dataset has 25 features, 30244 entries. I have two ...
0
votes
0answers
7 views

How to introduce a parameter for measuring change in data over time

In my project, I need to introduce a measure for 'movement' using a 3axis accelerometer (ADXL345). As sketched below: I thought about introducing some micro-changes, i.e. absolute change in ...
1
vote
1answer
36 views
+50

Where can I find free multi-instance single-label datasets for object detection?

I'm trying to find free multi-instance single-label datasets for object detection online. By "multi-instance and single-label" I mean that each image contains only object belonging to one ...
0
votes
0answers
16 views

Can the same data set (dynamic) be described as Chaotic & Pareto?

I'm trying to abstract the mathematical part of the problem as much as possible before the details follow, There's this dynamic data set that's $O(2^{32})$, a recent result described it as a power-law ...
0
votes
0answers
24 views

How to run PCA when data contains some categorial features?

Assume that we have a dataset with various features, and some of the features are categorial. And PCA dosn't work good on categorical features. How should I handle such datasets using PCA, what is ...
1
vote
1answer
41 views

Can this dataset be separated linearly?

Is this dataset linearly separable? If not, can it be converted into one by applying some function as it seems to follow the same pattern? Also, which classification algorithms could be used to fit ...
1
vote
1answer
20 views

Should I train from scratch or use pre-trained weights?

With yolov4, I am training an 80k images dataset that is used to classify different species of fish. Currently, I am using the following pre-trained weights: ...
0
votes
0answers
15 views

combine multiple datasets for generating a model

I am working on anomaly detection and have a dataset with multiple graphs. For instance, one graph has 100K edges and it belongs to video game activities (API calls). Another graph belongs to attack ...
0
votes
0answers
7 views

Is there a speeches dataset for summarization? [migrated]

I'm currently learning about text summarization and I'd like to fine-tune a model for summarizing speeches (i.e. texts of speeches by political leaders or parliament debates, NOT voice recordings). ...
3
votes
2answers
90 views

Which statistical test should I use to show significance in favour of a system?

I have a dataset of the following form: System A Rating System B Rating 4.5 5 3 4 5 3 etc. etc. I have 155 such data points gathered using a survey. Which statistical test should I use to show ...
0
votes
0answers
25 views

Automate downloading datasets via Colab

My desktop computer recently broke, and I'm currently working on a small laptop with barely 500mb of space left. I need to download about 100gb of files from the DFAUST dataset. I was wondering if ...
0
votes
2answers
20 views

Is it ok that my computer vision dataset just is loads of frames and images instead of video?

I am new computer vision and I am having a dataset with consecutive frames of videos. I want to ask that is it okay? How will you deal with it to output a video with some detections?
0
votes
0answers
20 views

How can i augment a tabular dataset? [closed]

I'm trying to solve a classification problem, however to improve my model i would like to try to augment my dataset. I don't have experience on augmenting a tabular dataset that looks like this I ...
1
vote
0answers
21 views

When doing feature selection, are features like “year”, “month” considered as ordinal features or should I convert them to strings?

I am working on a hotel reservation dataset that has both categorical and numerical (continuous and discrete) features (26 columns, 30244 rows). Target is also categorical and it says if the user &...
0
votes
0answers
19 views

Label description for scikit-learn Diabetes dataset

I am studying how to build a regression model using the Diabetes dataset from scikit-learn and I cannot find a good interpretation for the target value, meaning that I do not know what value would be ...
0
votes
1answer
23 views

Implementing a model for a language to another

I have a dataset of sentences of language X and Y (2 columns, for example, "abc def lang" ==> "xyz pqrt mno uages"). I want to have a output as a table that translates word by ...
1
vote
0answers
13 views

What algorithm i use for a Wikipedia bot? [closed]

i want to create a bot for Wikipedia which fight vandalism. It's like spam detection but people can remove text or add text. I want to use machine learning to do this. I don't know which machine ...
1
vote
1answer
11 views

XGboost with group-specific and individual-specific data

My dataset consists of a combination of two databases. One database consists of individual-level data on the characteristics and compensation of the top five executive officers of big American ...
1
vote
0answers
16 views

Iteratively improving ML model on a small dataset

I have a spam classification model which I created using a very small dataset.I have exported it as shown ...
1
vote
2answers
13 views

question about splitting data set by attributes and labels

Why the dataset need to use X as attributes and y as label? WHy not use a X to have it all?
1
vote
0answers
10 views

Train and validation sets splits using load_data

I'm using the package "datasets". The code I have: ...
1
vote
2answers
27 views

How to train and evaluate machine learning models with growing/changing datasets over time

Assume that you have a classification machine learning model, and you start with an initial dataset that contains 3 classes. You split the initial dataset into training/testing spits, you train the ...
3
votes
1answer
34 views

How to double audio dataset?

I am trying to develop a mispronunciation detection model for English speech. I use TIMIT dataset, this is phoneme labeled audio dataset. A phoneme is any of the perceptually distinct units of sound. ...
1
vote
0answers
12 views

creating dataset/storing vectors from c++ to python

I am new to the data science field. I need to create dataset which consists of n inputs (vectors of size 5) and n outputs (vectors of size > 10,000). These vectors are generated using c++ code. ...
1
vote
0answers
11 views

Incompatible shape error when trying to fit my model

I am using keras to make a CNN model. I have a dataset of images which consists of training and validation tfrecord files, the images in the tfrecords are grayscale. I load in the my 2 datasets by ...
0
votes
1answer
23 views

KNN Variance using a high value of K and cross-validation

it has come to my understanding, that a value of K=1, gives a high variance because we are only using only one data point, hence we are very likely to model the noise in that training example. Bias: ...
1
vote
1answer
16 views

How to resize image along with their mask?

I have original images of the size 1935x1481. I am using labelme to annotate the images. I am creating polygons on the original image. Is there a way to resize the image along with their mask? I am ...
0
votes
0answers
13 views

Grape dataset for smart viticulture

I'm involved in a smart viticulture project. I need some datasets for machine learning models. Datasets about grape disease or ripeness related to climate variables or grape.
2
votes
0answers
16 views

Best Method for Using Experts to Score Qualitative Data

TL;DR: Looking for recommendations or resources on the best method to have subject matter experts score qualitative data to train ML model. Problem: I am working on a problem in the biochemistry ...
0
votes
0answers
7 views

High resolution satellite data for climate analysis

i'm into a smart agriculture project. I need to get past info about climate in a specific region. Could you recommend satellite climate data with highest resolution?
0
votes
1answer
23 views

Currency Normalization for Salary Prediction

I have a dataset (350k data points) with data of employees across different regions over the last 10 years. The dataset consists of their skills, the region they are in, the industry, their current ...
1
vote
1answer
35 views

How to generate training instances from multivariate normal distribution? [closed]

I would like to generate a training and test set for the binary classification problem. I want the instances to be generated from a multivariate normal distribution and have 5 features. Can someone ...
1
vote
1answer
23 views

What if irrelevant features impact the outputs?

We know that weather conditions in China is not very useful to predict price of a house in Spain (as life experience). However, when we drop that China's weather condition, the accuracy is reduced ...
0
votes
0answers
26 views

Examples of Open source huge business intelligence datasets

I've observed that many of the datasets available for traditional ML and data science algorithms seem to be in the order of MB. I assumed these may be because earlier computers were not that ...
0
votes
1answer
19 views

Accuracy on Validation and Test set, Overfit?

Just a quick question, I am building a ML model right now however I am receiving very similar (72.2 and 72.4 for example)% for both Accuracy and F1-Score on my Validation Dataset and my unseen Test ...
0
votes
1answer
23 views

Which dataset I should use when I am retraining my model?

I trained my deep learning model using x dataset and now I got new dataset and I give it name as y. I want to retrain my model on this new dataset which is y. Do I need to use x+y dataset or just y? ...
1
vote
0answers
28 views

Is scraping job posts and gathering skills tags illegal? [closed]

I want to scrape a freelancer site (freelancer.com) and gather their public projects skill tags. Is this illegal to do? And is publishing it illegal?
0
votes
0answers
51 views

dataframe merge on string not working

having two dataframes each having common column which is country_code. while merging it by 'inner' join, it's returns empty dataframe. but i can see there is many values are common like IND, USA.... <...
0
votes
0answers
12 views

Prediction ability by small sample size datasets

All the datasets i have for modelling are relativ small - ca. 80 rows and there is no way how to increase the sample size. If there is no significant or dominant parameter / input (usually there is ...
0
votes
0answers
11 views

Computing class_weights using native tf.Dataset

There are already some answers on DSE about computing class-weights here but they require you to already know the number of samples in each class and assume it is ...
0
votes
2answers
103 views

Public dataset for account to account payments

I'm looking for a dataset which contains account to account payments (bank transfers). Ideally, this dataset would contain labeled data for transactions or accounts known to be victims of phishing ...
0
votes
0answers
6 views

How to perform cropping on large file dataset using ImageGenerators?

I have a large dataset of RGB images with 220x220x3. I want to crop the region of interest from each of them. For that, I have defined my own function. Im trying to use this with DataImageGenerators ...
1
vote
1answer
19 views

Pandas: Group by Single Column Entries [closed]

So have this table above. I'm trying to aggregate the occupations such that the table results in: I've tried using df.groupby(['Occupation']) but I get an error. ...
1
vote
1answer
33 views

Training a neural network with TWO possible correct outputs for one input

I have a system as a black box that has two correct outputs for a single input sample. now I want to train a neural network to generate at least one of the correct outputs for that input sample. what ...
1
vote
0answers
18 views

Matching documents from different sets with tfidf and cosine distance

I have two different set of documents S1, S2, with 30 text documents each. Using some text representation method, such as tfidf ...
3
votes
1answer
41 views

Storing Large dataset for processing and analysis of data

I am new to data engineering and wanted to know , what is the best way to store more than 3000 GB of data for further processing and analysis ? I am specifically looking for open source resources . I ...

1
2 3 4 5
24