Questions tagged [dataset]

A dataset is a collection of data, often in tabular or matrix form. This tag is NOT intended for data requests ("where can I find a dataset about ...") --> see OpenData

Filter by
Sorted by
Tagged with
0
votes
0answers
5 views

How to read multiple JSON files to a dataframe in R? can I get the code with the comments on it to understand it better?

I have multiple json files in a folder called "app users". These json files has NAN values as well in it.
0
votes
0answers
13 views

Handling repeating data from different individuals

I have a dataset that has some unique values but also includes information from multiple individuals that are repeating, meaning they are describing the same attributes and can have the same or ...
1
vote
0answers
15 views

Tokenizer returning incorrect values and losing a lot of data

(cross posted from main stackoverflow) This is a weird situation so I hope I can explain it correctly. My partner and I are working on a ML project where we create a model that predicts whether a ...
2
votes
0answers
19 views

Return the gradient and y intercept (m, b) to create two lines to best fit the data

I have been working on this task for a few hours now and have been unsuccessful with getting the target result. I have tried using multiple methods of trying to split the dataset using different ...
1
vote
0answers
9 views

Looking for multi output image datasets

I'm looking for image datasets that have multiple labels. So far I could only find one dataset of age, sex and ethnicity prediction but I'm looking for something a little less known than that one. ...
-1
votes
0answers
18 views

Write the function regex_pattern that takes a string as input and returns first word and year in the string after pattern matching [closed]

import re def regex_pattern(string): return if name == 'main': string = input() print(regex_pattern(string))
0
votes
0answers
26 views

A data in which an employee id is given in multiple months and many categorical features are there. To predict future retention. Recommend what to do?

I have this dataset in which we have to predict the retention of employees,i.e. how much will an employee stay in a company? This seems easy but the main obstruction here is that the same employee_id ...
0
votes
0answers
9 views

What is a good API or dataset for the “main” airport by country (not city)

As above. LON is an example of being the UKs main airport although technically that isn’t a single airport code it covers all London airports. The same is for STO which covers all of stockholms main ...
0
votes
0answers
18 views

How do you do data management?

I have about a few million records (small CSV / JSON file) from different sources, with about 50k added everyday. All on my local host. Until now, I have been using simple file structure to manage ...
0
votes
0answers
12 views

Question and answers datasets

I am looking for a dataset similar to XQuAD. I would need it in German, but it is not tragic if it is in another language since it can be translated. It would also be okay if the format is not the ...
0
votes
0answers
6 views

I am trying to sum up the "travel time in minute" for lane "GP" and grouping it under a single "Toll ID"

I am trying to sum up the "travel_time_minutes" for lane_type "GP" and grouping it under a single "Toll ID". I mean 0.11 + 0.50 + 1.21 for GP and retaining the Toll ID of ...
0
votes
1answer
20 views

How to make model not too dependent on one variable?

Let's suppose I have a generic model: Variable A | Variable B | Variable C | Variable D Variable Dis a categorical variable. ( ...
3
votes
0answers
32 views

Is it Scientifically Rigorous to create a multimodal ML Model with data from multiple sources

I'm attempting to create a multimodal machine learning model for disease diagnosis. However, I'm having quite a bit of difficulty finding public data sets with all the data I need. For example, I need ...
0
votes
0answers
22 views

Constructing heterogeneous datasets

Based on the "Dynamic Ensemble Selection Methods for Heterogeneous Data Mining" paper published by Chris Ballard and Wenjia Wang, I would be grateful if you could guide me on how they ...
1
vote
1answer
16 views

Create Period column based on a date column where the first month is 1, second 2, etc

I have a dataset with many project's monthly expendituries (cost curve), like this one: Project Date Expenditure(USD) Project A 12-2020 500 Project A 01-2021 1257 Project A 02-2021 125889 Project ...
0
votes
0answers
9 views

How do I create series of windows for loading into Keras ConvLSTM?

I am trying to replicate the result of Pham et al. (2020) using Keras. The model architecture is built and training converges on a small subset of data, but now I need to feed it the full dataset for ...
2
votes
1answer
25 views

Best way to preprocess data

I need to create a machine learning model to predict if a structure is an hotel or an apartment. I have a dataset structured as well: ...
0
votes
2answers
20 views

How to find criterion that best separates two populations in a dataset?

I have a dataset of two identified populations that contains various parameters for each data point. I would like to find the best criterion, i.e. the relation between e.g. three of those parameters, ...
0
votes
1answer
31 views

Encoding with OrdinalEncoder: TypeError: unhashable type: 'numpy.ndarray'

I'm trying to do a Random Forest in a data-set with numerical and categorical variables in order to obtain a categorical result (two possible classes, column name "predicción"). I'm using ...
-1
votes
1answer
71 views

ValueError:Input 0 of layer sequential is incompatible with the layer

i represented each row in my dateset as a 15552 cell which is spectrogram colored image(72723), that is represent the audio features, 72*72 is the size of spectrogram image and 3 referred to the 3 ...
0
votes
0answers
16 views

What is the best statistical test to compare the competitiveness of 10 football/soccer leagues within a time range?

I was wondering what stat test is the best to find out which football/soccer league is the most competitive. The leagues that I would choose for this analysis are either based on the countries that ...
0
votes
1answer
24 views

Sampling a data based on average and variance of another data

I have a set of textual datasets that have the following average and variance tokens lengths: ...
1
vote
1answer
85 views

How to detect anomalies in each feature - time series

I have a dataset with 5 features corresponding to 5 sensors that measure each three seconds the state of an accelerator. It is structured as well: ...
1
vote
0answers
72 views

How to train and evaluate model based on inner ordering of data subsets?

I've got a problem which I don't know how to frame properly (what techniques to use, what data structure, etc). Here's a rough definition of it: ...
1
vote
1answer
15 views

data.iterrows() + plt.imread() alternative, really exhausting

Currently, I'm loading my images for the training model with TensorFlow, like this: ...
1
vote
0answers
14 views

subtracting from multiple rows in a cvs file in python [closed]

I am trying to figure out how to subtract a set amount from a row if it goes over a value. For example, if I have a list of library accounts and I need to write a function whereby if an individual ...
0
votes
1answer
21 views

ValueError: Layer model_4 expects 1 input(s), but it received 10 input tensors

I've directory structure like this, for my dataset: |--train |--test |--valid In the train folder, these are pairs of images like xyz_sat and xyz_mask. So I've ...
0
votes
1answer
25 views

Query regarding the 'Data type' of features in Machine Learning

Should all the features in a dataset be converted to the same data type? For instance, if all the features have numerical values, some int & some float, should they all be converted to float? What ...
1
vote
0answers
21 views

Does time-series dependence penalize prediction with CNNs?

I'm trying to use CNNs on time series data (EEG), measured on different people. Each person has 10-20 recorded signals of different lengths and every subject has one global class assigned. Example: ...
1
vote
0answers
17 views

Comparing imputation strategies on a data set with no missing value

I am trying to evaluate a few data imputation methods and I am trying to see which one has a more accurate result. I have a dataset with no missing values(called X)....
0
votes
0answers
10 views

Compare multiple confusion matrix with unbalanced data?

I have a dataset with about 240k points, where the first 30k are "normal" (first class) and all the other 210k are considered anomalies (second class). I have applied on it four different ...
2
votes
0answers
14 views

Adding data in PyTorch that will not be in features or labels

I have a neural network that I'm training on a set of 7 features, but would like to also add an extra data array that is not going to be a feature or a label. Specifically, I'm training a network to ...
1
vote
1answer
23 views

Removing outliers from a multi-dimensional dataset & Data augmentation

Removing the outliers of a single-dimensional data can be easily done by removing the points that are outside of the IQR range. But how should the process of detecting and removing outliers be done if ...
1
vote
1answer
22 views

How to load a dataset with a specific structure in tfds library?

I have a dataset that it's classes arranged in the following way: /dataset/train/images/class1/ /dataset/train/images/class2/ . . . /dataset/train/images/classN/ ...
0
votes
1answer
25 views

Test data missing values

I've been given a test and train data set, and I have null values in my test data set. What do I do with it?
0
votes
0answers
12 views

Anyone have daily household energy demand data?

I am working on a model and simulation project that requires daily energy demand/power consumption data. kWh/hour or min or even seconds. We are taking in data/hour and using it for our calculations. ...
1
vote
1answer
18 views

Looking for a language dataset for the creation of a translator [closed]

I need a English-Nigerian language dataset (Hausa).
1
vote
0answers
22 views

How to handle insufficient information about the collected data in the dataset?

I am trying to create a ML model to predict foodborne diseases. I found online a few datasets with the informations I need. The problem is that none of them explains how they got the medical measures. ...
0
votes
0answers
86 views

Huggingface - TypeError: 'TensorSliceDataset' object is not subscriptable

I'm trying to make my own model for translate a language to another with T5ForConditionalGeneration and Huggingface using no pretrained model (I need to use my own dataset and tokenizer because no ...
0
votes
1answer
22 views

How to deal with data having 0 values in many columns?

I am trying to implement logistic regression but the dataset that I have have many columns with skewed data and most of them have 0 as values. I also the skewness of data for many columns its going ...
1
vote
2answers
59 views

data preprocessing: missing labels

I have a classification task to predict {0,1} labels, but in my dataset I only have data with 1 labels. Class indicates that the buyer has made a purchase of some product. Features: user id, product ...
0
votes
0answers
37 views

how to scrape images from facebook group

I want to scrape images from several Facebook groups some of them are public and some or not I am new to web scraping but I tried to look for solutions with selenium or BeautifulSoup or scrappy but I ...
1
vote
0answers
15 views

Methods for comparing datasets for abnormalities

My team at work receives datasets from various companies each quarter, and we're wanting to build some sort of machine learning model that compares the new dataset to our existing data for any ...
1
vote
0answers
19 views

Methods to combine datasets from different time periods

Consider a multivariate time series forecasting task where I have two datasets A and B. A goes from 1960 to 2020 and B goes from 2010 to 2020. There is a feature f ...
0
votes
2answers
38 views

If I have two variables with strong correlation, should I delete one and leave the other in my data

I have a large dataset, where I should make a binary prediction. The fact is that, after analyzing the data, I found that some variables are positively correlated to each other. So, I was wondering ...
0
votes
0answers
6 views

Does transfer learning works with more precise data?

I have a problem when I have 2 types of datasets for a classification task: a large one with which I intend to teach general rules of the field a smaller one, where quite similar instances occur but ...
0
votes
0answers
8 views

How to trim the train, test and validate datasets for LSTM to avoid shape issues?

I am trying to get an LSTM to optimize for regression of stock returns, but I keep encountering shape errors during the fit() phase. Specified a list with shape [30,2] from a tensor with shape [1,2] ...
1
vote
0answers
13 views

Machine Learning Model Input and Output Flow

I am working on a backend for structuring and submitting data into a ML model. I have 3 questions regarding this process. What is the best method to feed the model continuous data (updated every 30m ...
1
vote
1answer
32 views

Trained model performs worse on the whole dataset

I used pytorch as the training framework and the official pytorch imagenet example to train the image classification model with my custom dataset. My custom dataset has 2 different label (good and bad)...
1
vote
1answer
38 views

convert a column into rows [closed]

I used a webscraper and i got data only in one column like ABC123456 I need the data in this form A B C1 2 34 5 6 Is there any way?

1
2 3 4 5
26