As of May 31, 2023, we have updated our Code of Conduct.

Questions tagged [dataset]

A dataset is a collection of data, often in tabular or matrix form. This tag is NOT intended for data requests ("where can I find a dataset about ...") --> see OpenData

Filter by
Sorted by
Tagged with
0 votes
0 answers
74 views

Song playlist recommendation system

I want to build a recommender system to suggest similar songs to continue a playlist (similar to what Spotify does by recommending similar songs at the end of a playlist). I want to build two models: ...
Pybubb's user avatar
  • 1
1 vote
1 answer
279 views

How to use confidence labels?

I have 2 sets of training data in csv files. The training data have class labels, 1 for memorable, and 0 for not memorable. In addition, there is also a confidence label for each sample. The class ...
user avatar
0 votes
0 answers
88 views

Do I need to encode numerical variables like "year"?

I have a simple time-series dataset. it has a date-time feature column. ...
smarks70's user avatar
0 votes
1 answer
34 views

When to split Test and Training data from the full Dataset

I'm about to put my implementation into a pipeline and I'm now faced with the dilemma on when to actually split the test and training set? I have the following steps that I currently do (the names are ...
joesan's user avatar
  • 137
0 votes
2 answers
1k views

Create new rows based on a value in a column

My dateset is generated like the example df = {'event':['A','B','C','D'], 'budget':['123','433','1000','1299'], 'duration_days':['6','3','4','2']} I need ...
conradoov's user avatar
2 votes
1 answer
314 views

The single CSV created by combining a large number of CSV files is too large to process. What options do I have?

The dataset I am currently working on has more than 100 csv files, with each of size more than 250MB. These are files containing time series data captured from different locations and all the files ...
odd_wolf's user avatar
0 votes
1 answer
197 views

How can I generate anomalies in a dataset?

I'm building a tensorflow model to detect anomalies in an electricity smart meter data and I'm using UK-DALE Dataset. How can I introduce anomalies in the data so I can test the model?
Mohamed Osama's user avatar
0 votes
0 answers
64 views

Is there any way to get data sets from mldata.org?

I need data sets for my project."Record of heart sound " which has Recordings of normal and abnormal heartbeats used to recognize heart murmurs etc. link: http://mldata.org/repository/data/...
bighero666's user avatar
1 vote
0 answers
23 views

How to re-generate dataset df at time t2 while having df at time t1 and cross-sectional dataset df' at time t1 and t2?

I have a travel survey dataset df' collected in 2017 and also 2019. Note that individuals (households here) are not necessarily identical in 2017 and 2019 but their features are. Dataset df' in 2017: ...
Armin Mir's user avatar
1 vote
0 answers
13 views

Question Categorization Dataset(s)

I'm looking for a dataset or group of datasets I could combine that would contain numerous examples of the following types of question categories. free response (ex: What is the capital of Portugal?) ...
Sam Crochet's user avatar
1 vote
0 answers
23 views

How do I evaluate if my data represent the target variable before training a machine learning algorithm?

I have a dataset of points cloud where each point in the point cloud has a variable. I am trying to relate the local geometry features to that point variable by using FPFH, This means I am generating ...
Hamzalihi's user avatar
0 votes
1 answer
20 views

Optimization of a simple M x N dataset

I have a dataset consisting of M questionnaires and N students. Each students replied to some questionnaires. I would like to make the dataset better, by removing some questionnaires and/or some ...
Samuel Faure's user avatar
0 votes
0 answers
9 views

How to check the validity of data? if There are repeated Y values against X data

I need to find the best-fitted curve for this data, I am not sure which model should I use? Can someone suggest me a model? I also have doubts that this dataset is not valid. As there are multiple Y ...
Adnan Ali's user avatar
  • 101
1 vote
0 answers
15 views

Are there any known methods to generalize multiple trajectories into one "optimal" path based on energy consumption?

Say that I have a database with timeseries coordinate data from a vehicle going from A to B multiple times but with slightly varying trajectories each time, leading to different amounts of energy ...
Max Månsson's user avatar
0 votes
1 answer
61 views

Difference between scaling just x or x and y in PCA / principle component regresseion

Before doing principle component regression it is important to scale the data. But which data exactly? Is it enough if I just scale X or do I have to scale the whole data set, containing X and Y (=...
Sally's user avatar
  • 5
-1 votes
1 answer
72 views

step-by-step creation of models with accumulation of predictors vs GridSearch

Can you please tell me if step linear models of the independent variable "ols_step_both_p()" (R) are possible with the accumulation of predictors in the amount of 58, 220 and 299, naturally ...
Егор Козлов's user avatar
0 votes
1 answer
18 views

Correlation Wikipedia translated pages vs number of in links is weird (scatterplot)?

I'm trying to find a correlation measure for the number of Wikipedia pages an entity (an article) has been translated to vs number of links that point to that page (both measures that can point to the ...
Idkwhatnomeis's user avatar
0 votes
1 answer
702 views

Is there a way to combine multiple ML models where each use datasets with different features?

I have a dataset where some features (c,d) apply to only when a feature (a) is a specific value. For example ...
Bruce's user avatar
  • 1
1 vote
0 answers
24 views

Suggestions for labeling regression data to improve model accuracy

I'm working on a convolutional neural network that should predict up to 3 (x,y) coordinate pairs representing the waypoints of a concrete path, given an input image. This network will be used to help ...
pmitch's user avatar
  • 11
0 votes
1 answer
22 views

What correlation measure for Wikipedia translated pages vs number of in links?

I'm trying to find a correlation measure for the number of Wikipedia pages an entity (an article) has been translated to vs number of links that point to that page (both measures that can point to the ...
Idkwhatnomeis's user avatar
1 vote
1 answer
50 views

how to calculate evolution change in percentage in a dataset compared to itself

scenario: lets say I have a table for number of customers visiting a shop per day. now I want to calculate that how much change has happened in terms of number of visitors throughout the past 30 days ...
sepisoad's user avatar
  • 113
1 vote
1 answer
30 views

Looking for in depth knowledge in evalution metric

I am dealing with an unbalanced dataset. The total instances in my dataset is 1273 and the Yes class is 174 and No class is 1099. So the unbalance ratio is like 1:6. Now I know ...
Encipher's user avatar
  • 359
2 votes
1 answer
36 views

Evaluate two Recommender models trained with different data

Suppose you are given two Recommender Systems to evaluate, A and B. Model A is trained with ...
kevin_was_here's user avatar
1 vote
1 answer
85 views

Clustering multiple csv files that cannot fit in RAM

I have multiple csv files each of which has at least 200MB of data across 12 columns. Each csv file possibly can fall into 4 categories or labels. I am trying to see which clusters each of these files ...
python_beginner's user avatar
0 votes
1 answer
175 views

WMT: What are the differences of WMT14, WMT15 and WMT16 datasets?

Each year, the Workshop on Statistical Machine Translation (WMT) holds a conference that focuses on new tasks, papers, and findings in the field of machine translation. Let's say we are talking about ...
Ramón Wilhelm's user avatar
3 votes
1 answer
61 views

How to collect a computer vision dataset?

Context: my team is researching some plant-growth patterns using object detection models. We have collected a small set of images (a few thousand) and ran some simple experiments. The results were ...
Trauer's user avatar
  • 131
0 votes
0 answers
15 views

Joining two datasets without knowledge of keys

Which technique should I use to attempt to find common simple keys between two datasets in order to join them, without any knowledge of neither the column-names, nor which columns are key-columns. Let'...
Mads Boyd-Madsen's user avatar
1 vote
1 answer
24 views

Merging 2 datasets

In the given task, they've provides us with 2 datasets namely the test & train dataset. So, I was wondering if we could merge those 2 datasets into 1 data-frame and delete the duplicates. Would ...
nithin krishna's user avatar
0 votes
0 answers
224 views

Can SMOTE be used for non-binary classification?

There is a class imbalance present in my dataset and I would like to balance the dataset. The dependent variable's features are (0,1,2,3,4). How do I make use of SMOTE, SMOTE-N, SMOTE-NC when if they'...
nithin krishna's user avatar
0 votes
1 answer
40 views

Algorithms for Vertex or Node Correspondence

Given a graph G, and another graph with the same number of vertices G’, one can define a vertex correspondence function f, from the vertex set of G to the vertex set of G’. The correspondence function ...
Matthew Crawford's user avatar
0 votes
2 answers
1k views

Comparing distributions Python

I have a larger dataset (random variable) 'x' containing values approximating a Gaussian distribution. From 'x', a much smaller random variable 'y' is sampled without replacement. I want to compare ...
Arun's user avatar
  • 121
0 votes
0 answers
38 views

How to unify weights in my dataset

I have a symptom-disease network that consists of four attributes: symptom, disease, co-occurrence and TF-IDF. I'm considering the TF-IDF attribute as the weight of my network edges and symptom and ...
Alireza Azhdari's user avatar
4 votes
3 answers
4k views

MNIST Digit dataset requires login

I want to download the MNIST handwritten digit data from the offical site: https://yann.lecun.com/exdb/mnist/ But it wants me to enter a username and a password. How can I download the data? Does ...
Jakob's user avatar
  • 141
1 vote
1 answer
16 views

Verify data provenance used for ML

Is it possible to make a verifiable data provenance log for datasets used for machine learning? Sort of a hash that would confirm that a collection of datasets were actually used to train a given ...
stanislav-iablokov's user avatar
0 votes
1 answer
84 views

Visualize duplicate ID (unique column) from dataset

I have a dataset contains ID and year of their occurrence. This is the sample of my dataset: ID Year 1234 2018 1234 2019 5678 2017 5678 2020 .... .... I would like to visualize the ID's and the ...
Kevin M's user avatar
0 votes
1 answer
191 views

Dataset with Multiple Choice Questions for fine tuning

I hope it's allowed to ask here, but I am looking for a dataset (the format is not that important) that is similar to SQuAD, but it also contains false answers to the questions. I wanna use it to fine ...
futuredataengineer's user avatar
1 vote
1 answer
27 views

Are the random variables in the figure independent?

I have this plot of 2 random variables (Assume X and Y) . A point represents a value of Y for a given value of X. I think these variables cannot be considered independent. Here is my reasoning. ...
Abhisek Dash's user avatar
0 votes
0 answers
38 views

Regarding pos tagging

I working on a dataset, I did the pos_tagging using nltk. Now I want to know which sequence of grammar is most common in my rows, then I want to define a chunk grammar based on a common grammar ...
aadil gani's user avatar
0 votes
0 answers
28 views

Handling multiple object types in a large dataset to build a better ML model

I have a dataset with multiple object data_types. From the dataset, most of them were made into categorical features (Using One-hot encoding as well). But few of them have object data types with 6M ...
Pikaschu's user avatar
0 votes
1 answer
729 views

Trying to convert a .txt file into .npy file to split and use for train/validation/test

I have a large data set in .txt form, and was trying to split it into three for testing, validation, and training by transforming the data into a ...
besa's user avatar
  • 23
0 votes
1 answer
202 views

Applicability of ARIMA model on non stationary data

I have a time series dataset that does not have the stationary property. The dataset is monotonically increasing or sometimes showing no change over periods of time. Can I apply the ARIMA model to ...
Jd_mahmud's user avatar
1 vote
1 answer
226 views

Why we call Mix-up method is a data augmentation technique?

I am bit confused in the Mixup data augmentation technique, let me explain the problem briefly: What is Mixup For further detail you may refer to original paper . We double or quadruple the data ...
Ahmad's user avatar
  • 138
0 votes
1 answer
29 views

Creating Dataset for Classification, How much balanced a good dataset should be?

I am creating a dataset with 4 classes, and there are 50K rows, I am already getting 86% accuracy, 0.85 Precison, 0.86 Recall and 0.71 F1-Score on SVM with 80,20 split. I have to publish this dataset ...
Adnan Ali's user avatar
  • 101
1 vote
0 answers
19 views

Dataset for IoT activities [closed]

First of all, I am not sure whether this is the correct Stackexchange site for this question. If not, please let me know :) My question: I am looking for a dataset pertaining to IoT activities. ...
Nht_e0's user avatar
  • 21
1 vote
0 answers
78 views

How to represent/aggregate data belonging to same group, and feed to model for training?

I have a problem in which there are couple of records that belong to the same group, like multiple observations. For example this data set has three groups, For each group, I have three observations....
abdul's user avatar
  • 113
0 votes
1 answer
1k views

Training loop stops after the first epoch in PyTorch

I'm trying to train a seq2seq model using PyTorch using the Multi30K dataset from Dutch to English language. Here is my snippet of code: CustomMulti30k class ...
mhannani's user avatar
  • 101
1 vote
0 answers
15 views

What is an efficient implementation of a custom map-style dataset for a hdf5 file with irregular structure?

I have a hdf5 file that contains picture of a certain number of people, from a certain number of source cameras, for many seconds. So it is like this: ...
Manveru's user avatar
  • 111
0 votes
1 answer
23 views

Calculating statistical ranks between datasets with unpaired observations

The problem is the following: I have multiple datasets for which I want to calculate a ranking for each. All observations contained in the datasets can be arbitrarily permuted, so they are unpaired, ...
Steve's user avatar
  • 21
0 votes
1 answer
932 views

Why do I get different results at inference time even with fixed seed?

I am a very beginner in deep learning and am playing with voice cloning project. I trained my dataset and used the trained model to synthesize some sentences and was surprised to get a very different ...
Cara Duf's user avatar
1 vote
0 answers
88 views

How to integrate Zoho analytics with Jupyter notebook? [closed]

I am trying to connect Zoho Analytics and Python for importing data from Zoho Analytics. I have already installed !pip install zoho-analytics-connector. What should ...
Pikaschu's user avatar

1
3 4
5
6 7
30