Questions tagged [dataset]

A dataset is a collection of data, often in tabular or matrix form. This tag is NOT intended for data requests ("where can I find a dataset about ...") --> see OpenData

Filter by
Sorted by
Tagged with
0 votes
0 answers
9 views

A good set of datasets/models for testing an NLP technique

I am a machine learning researcher who up until this point has primarily worked on Computer Vision problems. However, I have an idea for an NLP technique involving a novel Transformer architecture, ...
Anonymous's user avatar
0 votes
0 answers
15 views

How can I perform an analysis of the NHIS imputed income variables?

I have downloaded five family income variables from https://nhis.ipums.org/nhis-action/variables/group?id=economic_income (INCPPOINT1, INCPPOINT2, INCPPOINT3, INCPPOINT4, INCPPOINT5) for they years ...
tryingtogetsmth's user avatar
0 votes
0 answers
17 views
+50

Cluster/Similarity problem with two datasets of different cardinality

I want to cluster financial products according to their similarity. I have two dataset of different cardinality: One-to-One dataset: One ID has One attribute/feature per column - Describes a ...
Maeaex1's user avatar
  • 528
0 votes
0 answers
15 views

Expanding torch dataLoader by repeating elements (or something equivlent)

I have a pytorch classifier with stochastic elements. For each MNIST minibatch, I would like to feed it in K times (from the same starting state), and then average the resulting weight updates in ...
Ari Herman's user avatar
0 votes
0 answers
7 views

Terminology: What is it called when the filter criteria is added to the data table?

When we want to focus on a group within our dataset, it is common to filter it. Suppose we have a data table showing how many points 4 players earned in a game. Table 1: Game Scores for 4 Players Age ...
madprogramer's user avatar
0 votes
0 answers
9 views

Create Dataset (ADS) to build predictve model for servey data

I have collected labeled survey data spanning 8 months in 2023, capturing satisfaction levels ranging from "very satisfied" to "very dissatisfied." My objective is to develop a ...
Mohamed SELAMA's user avatar
0 votes
0 answers
8 views

Datasets for demonstrating data/model/concept drift

Are there any well-known datasets that are useful for illustrating the concept of data and/or model/concept drift? I can create my own out of, say, MNIST (by e.g. removing digits and adding them in ...
Jennie Lan Chien's user avatar
0 votes
1 answer
33 views

How to use API Documentation of dataset

Some websites provide a link to a dataset (in Excel sheets format) which allows the dataset to be downloaded. But some others additionally provide API documentation, like this site. Can you please ...
IwillLearn's user avatar
0 votes
1 answer
17 views

Object Classification Dataset Creation

There is a problem I've faced recently which I'm not sure my approach is proper or not. There is bunch of field videos which I run a semi-supervised detection model to extract crops to train my ...
spawnfile's user avatar
0 votes
2 answers
26 views

Optimal Data Split

I have a multiclass problem (3 classes) that looks to predict if someone will buy a product, neutral or not. I have initial features of in-app activity data such as likes, share, bookmark, share, ...
Marc Atanante's user avatar
0 votes
2 answers
29 views

Lack of imagination when creating projects

Whenever I find a set of data to carry out a project, I end up running out of ideas and don't know how to proceed with the project. I don't know how to proceed or what information to look for. How do ...
Davi Komura's user avatar
0 votes
1 answer
22 views

different range of target values in neural network

I am working on a neural network regression code. The dataset includes 14 features in the range value between -1 and 1. while the target variable is changing among (0.000759) to (1100). The target ...
Mali's user avatar
  • 1
0 votes
0 answers
9 views

How should I save data in deeplearning with nosql (mongodb)?

I usually use file system to manage data for my deep learning model, but one of my boss told me to make nosql database to manage data. Datasets I use have m rows, and n columns of count matrix and ...
containletters's user avatar
1 vote
1 answer
31 views

How to know the confidence of a classification on unlabeled data generated after model training?

I have created (in python) the code for a Random Forest classification model for a labeled dataset using sklearn. The model works very well. ...
Daniel Vieira's user avatar
0 votes
1 answer
30 views

Best strategy for handling missing groups of features

I am currently working on a ML problem where the features used for modelling are sourced from different places/providers. It is very unlikely to find the features from all the different sources to be ...
recentadvances's user avatar
0 votes
0 answers
13 views

What distribution-shift datasets exist for ML?

What datasets there are which contain a (labelled) distribution shift or which distribution-shifted datasets share classes? Currently I have found some interesting results using CIFAR10 and the ...
Somniare's user avatar
0 votes
0 answers
18 views

Open source dataset (manufacturing, machine operations)

I am looking for an open source dataset from the manufacturing domain (sensor data, time series) with specific traits. It should stem from a process consisting of a sequence of distinct machine ...
sinpalabras's user avatar
0 votes
0 answers
10 views

Acquiring Ground Truth for ISIC 2019 Challenge Test Set for Publication Purposes

I have been working with the dataset from the ISIC 2019 challenge, which is not live anymore. I was hoping to find the ground truth data for the test set published on the official website for further ...
Marzi Heidari's user avatar
0 votes
1 answer
46 views

How would normalizing be affected by outliers? And how to avoid it?

I have a data set that boils down to Three clomuns: 1.Supplier name 2. Number of transactions with supplier 3. Total value of those transaction. I'm trying to find the best way to rank all suppliers ...
Rakuzan's user avatar
0 votes
1 answer
30 views

Better understand the scatter plots

I'm trying to understand how to read a scatter plot based on the results of my confusion matrix. This is the CM: I saw an high correlation between column 5 and column 6, a low correlation with column ...
JimBelushi2's user avatar
0 votes
1 answer
24 views

Request for Assistance with Acquiring a Diverse Cry Dataset for Research Project

I am working on a research project aimed at classifying babies' cries based on their needs. However, I have encountered difficulties in obtaining a suitable cry dataset. The only dataset I was able to ...
Maryam Awari-Alimi's user avatar
0 votes
3 answers
29 views

Data preprocessing

I just want to know how to determine whether to remove the missing values or to impute them with mean , median or mode. I usually remove the missing values but it decreases the size of the dataset by ...
Yousaf Chaudhary's user avatar
0 votes
1 answer
30 views

Multilabel classification: Choosing threshold

I'm creating a multilabel classification approach based on sentence embeddings applied to text taken from a chatbot. We have the following: a training dataset of 2,500 lines, where each line is a ...
LasiusMind's user avatar
1 vote
0 answers
64 views

Annotating and Structuring a Dataset for Duplicate Detection

I'm currently working on a project that requires the detection of duplicate bands in Western blot images. The task involves two types of duplicates: ...
Emmanuel's user avatar
  • 111
0 votes
0 answers
11 views

How to subset and approximate a private dataset's characteristics?

Assume an Object Detection task. The private dataset has its bounding-box (bbox) characteristics, and assume only width-height measures are relevant. Assume we know what is that width-height ...
Natan ZB's user avatar
0 votes
1 answer
12 views

Classification Problematics : Feature Number Variance & Feature Repetition

I have a harsh case study (in my mind). The problem is I need make binary classification on Quality of Service (good or bad). I have a feedback on quality on groups of devices belonging to company. I ...
secuf's user avatar
  • 1
0 votes
0 answers
7 views

I want to import my states legislation into a database. I need advice in parsing large HTML, TXT into an importable form. It is Large PDF initialy

My data consists of large pdf files containing state legislation. I can convert the pdf to xml, txt or html, and I can manually go through, identify the tables and write a script to extract only the ...
RBB's user avatar
  • 1
0 votes
0 answers
66 views

Best practice for fine tuning LLM

I have a dataset that i have collect for specific topic the dataset is in these format : raw text (similar to shake spare dataset) where it has no label or input, just text Question and answer ...
Mustafa Alahmid's user avatar
0 votes
1 answer
64 views

Can bert uncased predict text classification on foreign data?

I am trying to do the fake news/real news classification and used a pre-trained bert uncased model as transfer learning and it gave a solid 81% accuracy. But the problem is while doing sanity checks, ...
Jasmin Wilson's user avatar
0 votes
0 answers
6 views

Dataframe shape for multi-point prediction using LSTM & time-serise data

Can I ask a very simple question about LSTM time-series data prediction? I have 1258 data points which are Amazon's historical stock price by day unit. And I build the below data structure in order to ...
orde.r's user avatar
  • 113
0 votes
0 answers
40 views

ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type int) in Python - from_gerator fails

I know this question has answers, but they are not working. why this fails ? (and why tensorflow has troubles in reading numpy!!) ...
user305883's user avatar
0 votes
0 answers
25 views

How do I test with custom images for a model that was trained on quick draw npy dataset

I have been trying to test my CNN model on a custom doodle of an apple that I drew. But even when I preprocess the image to have the same shape with the training data, the model gives wrong prediction ...
Marcuss's user avatar
2 votes
1 answer
38 views

Smart Selection of Training Data for Fine-tuning Language Models in Small Domains

Background I am working to make language models (for example, Stanford's Alpaca model) perform well on a new small domain through fine-tuning on domain-specific dataset $D$. If the size of $D$ is $N$, ...
Mr.Robot's user avatar
  • 173
0 votes
0 answers
12 views

Do Hospital Episode Statistics (HES) data (NHS UK) contain labs?

Can anyone tell me if HES data contain lab results such as blood tests commissioned and their result? That can be in any one of the HES datasets. Thank you.
user2822693's user avatar
0 votes
1 answer
26 views

Spam tweets dataset needed [closed]

Where can I find labelled spam tweets dataset that has at least 50000 rows. I've searched everywhere; but there are very few datasets available, and those datasets are way too small. I am in urgent ...
Kingster20's user avatar
0 votes
1 answer
62 views

Open retail/ecommerce dataset with customers, transactions, products, loyalty, discounts, promotion information?

Does anyone know of an open retail/e-commerce dataset with information about customers, transactions, products, loyalty, discounts, and promotion information? I am planning to create 360-degree ...
exan's user avatar
  • 101
2 votes
2 answers
150 views

Training GPT Model with Swagger Documents, Need Help with Model Fit

We are currently developing an application that performs actions based on user input using 'gpt-3.5-turbo-0613'. For example, if a user enters the prompt "create a user with the name Potter and ...
mostwelcome's user avatar
1 vote
2 answers
37 views

Tips for scraping crypto data in the right way

I am scraping data from crypto site and want to use neural network algorithm for predicting data. the way i save data is like these: and there is bunch of other features like open/high/low/close for ...
mohammad ariyan rad's user avatar
0 votes
0 answers
15 views

Multiple coords to one data_vars in xr.dataset

I am trying to create a xarray dataset for a list of events (labeled 1 or 0, so it's an array of integers). To each event three numbers need to be attached - a date, longitude, and latitude. I provide ...
Shaz's user avatar
  • 125
1 vote
0 answers
24 views

Is there a statistical test for comparing my dataset percentages with published percentages?

I have a clinical dataset which is showing the percentage of A is 57% and B is 35% for where an tumours is initially found. However the literature is showing it is A is 73% and B is 23%. Is there a ...
Matt Smith's user avatar
-1 votes
2 answers
92 views

How to generate synthetic dataset with dependent and independent features?

I want to generate a synthetic dataset, in python, composed of 10 features. 5 are dependent among each others and 5 are independent. Generate dependent features. ...
I Sui's user avatar
  • 57
0 votes
2 answers
68 views

Is there any rationale for performing SMOTE-ENN before train-test-split?

I have created a classification model for predicting data, and the problem is that the two classes are highly imbalanced I have a problem. I have created a classification model for predicting data, ...
Hello is me's user avatar
0 votes
1 answer
28 views

How to handle ID variables when splitting data for machine learning?

I'm new to machine learning, and I'm working with some international head-to-head sports competition data. I used relational data creation techniques in tidyverse to join several data sources to ...
Emr8980's user avatar
1 vote
0 answers
36 views

Accounting for differences in average teacher salaries, adjusted for inflation and the cost of living in each state

My aim is to determine whether teacher salaries in all 50 of the United States over between 2013 and 2023—adjusted for inflation and the cost of living—differ significantly. I would like to ask what ...
Nazarene's user avatar
0 votes
0 answers
19 views

Problem with the size of test data?

I am working on my thesis, which has two different research questions: Evaluate transformer models while incorporating non textuel features Evaluate the importance of data quality in tranformer ...
Aymen Ba's user avatar
0 votes
0 answers
12 views

I'm looking for a dataset on enviroment monitoring

I'm looking for a dataset related to environmental monitoring, made up of values obtained from various types of sensors (such as temperature, pressure, CO2...etc) for the purpose of a classification ...
PiEmmeC's user avatar
0 votes
2 answers
74 views

How to create a plot of specific row with every column using python and which package to choose matplotlib or seaborn?

I'm trying to create a plot for the ranking of each country from 2002 to 2023. I created this dataset by loading each csv file from the respective years, which contains the Countries and Ranking ...
SAUBHIK KUMAR's user avatar
0 votes
1 answer
43 views

Data transformation pipeline error

When I'm making data transformation pipeline on a dataset I keep getting error as " all the input array dimensions except for concatenation axis must match exactly, but along dimension 0, the ...
Amy's user avatar
  • 1
0 votes
1 answer
22 views

Domain Constraints

Consider a dataset with 5 numerical features: A, B, C, D and E. Can we train a deep learning/machine learning model which can learn constraints between (A,B) and (C,D,E) i.e I want to learn a function ...
Vivek Kumar's user avatar
0 votes
1 answer
56 views

Keras MLP not working

For my degree's final project I am working with Keras and trying to build different AI models. I'm having trouble with an MLP. First I preprocess the UNSW-NB15 dataset and later use it as input in a ...
alex martinez's user avatar

1
2 3 4 5
30